OPRO (Optimization by Prompting)

A OPRO (Optimization by Prompting) is a meta-prompting framework that leverages large language models to optimize prompts based on performance trajectorys.

• AKA: OPRO, Optimization by Prompting, Google OPRO, LLM-as-Optimizer Method.
• Context:
  • It can use LLMs as optimizers to generate improved prompts from performance history.
  • It can maintain optimization trajectorys showing prompt evolution and score progression.
  • It can generate prompt candidates by analyzing successful patterns and failure modes.
  • It can evaluate prompt performance through task-specific metrics and success rates.
  • It can implement meta-optimization loops where LLMs reason about optimization strategy.
  • It can leverage in-context learning to understand optimization patterns from examples.
  • It can apply exploration strategys to avoid local optima in prompt space.
  • It can utilize natural language reasoning to explain improvement rationale.
  • It can incorporate constraints to maintain prompt validity and task relevance.
  • It can scale to complex optimization problems through hierarchical decomposition.
  • ...
  • It can range from being a Simple OPRO to being a Complex OPRO, depending on its optimization sophistication.
  • It can range from being a Single-Task OPRO to being a Multi-Task OPRO, depending on its task coverage.
  • It can range from being a Zero-Shot OPRO to being a Few-Shot OPRO, depending on its example requirement.
  • It can range from being a Greedy OPRO to being an Exploratory OPRO, depending on its search strategy.
  • ...
• Example(s):
  • OPRO for Mathematical Optimization, which solves linear programming problems through prompt optimization.
  • OPRO for Prompt Engineering, which automatically discovers effective prompts for downstream tasks.
  • OPRO for Hyperparameter Tuning, which optimizes model configurations via natural language.
  • OPRO for Algorithm Design, which generates algorithm variants through LLM reasoning.
  • ...
• Counter-Example(s):
  • Gradient-Based Optimization, which uses mathematical gradients rather than LLM reasoning.
  • Grid Search, which uses exhaustive enumeration rather than intelligent search.
  • Random Search, which lacks optimization guidance and performance tracking.
  • Manual Optimization, which relies on human expertise rather than automated reasoning.
• See: Meta-Prompting Framework, LLM-Based Optimization, Performance Trajectory, Prompt Generation Task, In-Context Learning, Natural Language Reasoning, Optimization Loop, Google Research, Prompt Optimization Method.

---