Optimization Algorithm

An Optimization Algorithm is a search algorithm that can be applied by a optimization system (to solve an optimization task).

• Context:
  • It can range from being a Combinatorial Optimization Algorithm to being a Continuous Optimization Algorithm, based on the type of variables and solution space.
  • It can range from being a Single-Variable Optimization Algorithm to being a Multi-Variable Optimization Algorithm (MVO), based on the number of variables.
  • It can range from being a Total-Space Optimization Algorithm to being a Local Optimization Algorithm, based on how much of the search space is covered.
  • It can range from being a Sequential Model-based Optimization Algorithm to being a Parallel Model-based Optimization Algorithm, based on how the algorithm queries points.
  • It can range from being an Offline Optimization Algorithm to being an Online Optimization Algorithm, based on when it gets information.
  • It can range from being an Exact Optimization Algorithm to being an Approximate Optimization Algorithm, based on the optimality guarantees.
  • It can range from being a Maximization Algorithm to being a Minimization Algorithm, based on the objective.
  • It can be implemented by an Optimization System to solve an Optimization Task.
  • ...
• Example(s):
  • a Convex Optimization Algorithm,
  • a Stochastic Gradient Descent Algorithm.
  • a Bayesian Optimization Algorithm.
  • …
• Counter-Example(s):
  • a Bayesian Parameter Estimation Algorithm,
  • a Maximum Likelihood Estimation Algorithm,
  • a Method of Moments,
  • a Minimum Mean Squared Error Algorithm,
  • a Minimum Chi-Square Estimation Algorithm.
• See: Greedy Algorithm, Statistical Inference, Parameter Estimation, Local Maximum, Absolute Maximum.

---

---