Optimization Algorithm

An Optimization Algorithm is a search algorithm that can be applied by a optimization system (to solve an optimization task).

• Context:
  • It can range from being a Combinatorial Optimization Algorithm to being a Continuous Optimization Algorithm.
  • It can range from being a Global Optimization Algorithm to being a Local Optimization Algorithm.
  • It can range from being an Offline Optimization Algorithm to being an Online Optimization Algorithm.
  • It can range from being an Exact Optimization Algorithm to being an Approximate Optimization Algorithm, depending on the task's optimality guarantees.
  • It can range from being a Single-Variable Optimization Algorithm to being a Multi-Variable Optimization Algorithm (MVO).
  • It can range from being a Maximization Algorithm to being a Minimization Algorithm.
  • It can range from being A Sequential Model-based Optimization Algorithm to being a A Parallel Model-based Optimization Algorithm.
  • ...
• Example(s):
  • a Convex Optimization Algorithm,
  • a Stochastic Gradient Descent Algorithm.
  • …
• Counter-Example(s):
  • a Bayesian Parameter Estimation Algorithm,
  • a Maximum Likelihood Estimation Algorithm,
  • a Method of Moments,
  • a Minimum Mean Squared Error Algorithm,
  • a Minimum Chi-Square Estimation Algorithm.
• See: Greedy Algorithm, Statistical Inference, Parameter Estimation, Local Maximum, Absolute Maximum.

---

---