Accuracy Score Measure

An Accuracy Score Measure is a forecasting performance measure that standardizes prediction accuracy relative to median participant performance.

• AKA: Standardized Accuracy Score, Relative Accuracy Measure, Normalized Performance Score, Tournament Accuracy Metric.
• Context:
  • It can typically calculate accuracy score measure values using logarithmic scoring rules.
  • It can typically normalize accuracy score measure results across different question types.
  • It can typically enable accuracy score measure comparisons between participant groups.
  • It can often benchmark against accuracy score measure baselines like algorithm performance.
  • It can often incorporate accuracy score measure adjustments for question difficulty.
  • It can often support accuracy score measure aggregation across time periods.
  • It can range from being a Simple Accuracy Score Measure to being a Complex Accuracy Score Measure, depending on its calculation method.
  • It can range from being a Binary Accuracy Score Measure to being a Continuous Accuracy Score Measure, depending on its outcome type.
  • It can range from being a Individual Accuracy Score Measure to being a Group Accuracy Score Measure, depending on its aggregation level.
  • It can range from being a Short-Term Accuracy Score Measure to being a Long-Term Accuracy Score Measure, depending on its evaluation period.
  • ...
• Examples:
  • Tournament-Specific Accuracy Score Measures, such as:
    • XPT Accuracy Score, using log score normalization.
    • Good Judgment Open Score, with percentile ranking.
  • Platform-Specific Accuracy Score Measures, such as:
    • Metaculus Score, incorporating peer comparison.
    • PredictIt Score, based on market performance.
  • Research Accuracy Score Measures, such as:
    • IARPA ACE Score, for intelligence analysis.
  • ...
• Counter-Examples:
  • Raw Brier Score, which lacks normalization.
  • Binary Accuracy, which lacks probabilistic assessment.
  • Speed Measure, which measures response time.
• See: Forecasting Performance Measure, Brier Score Measure, Logarithmic Scoring Rule, Standardized Absolute Forecast Error Measure, Calibration Measure, Relative Performance Metric, Competitive Prediction System, Probabilistic Forecasting Task, Wisdom of Crowds Effect.

---