Agresti-Coull F1 Confidence Interval Method

An Agresti-Coull F1 Confidence Interval Method is a plus-four confidence interval method that adds fixed pseudo-counts to confusion matrix elements before computing F1 confidence intervals to achieve stable 95-96% coverage probability.

• AKA: Plus-Four F1 Interval Method, Adjusted Wald F1 CI Method, Fixed Inflation F1 Interval, Agresti-Coull Adjusted F1 Method.
• Context:
  • It can typically add 2 pseudo-successes and 2 pseudo-failures before interval calculation.
  • It can typically achieve 95-96% average coverage (slightly conservative) across parameter space.
  • It can typically provide simple fixed inflation independent of observed data values.
  • It can often stabilize intervals by pulling extreme F1 estimates toward 0.5.
  • It can often be recommended for sample sizes n ≥ 40 according to Brown et al. (2001).
  • It can often outperform Wald F1 Confidence Interval Method by ~10% in coverage probability.
  • It can range from being a Standard Agresti-Coull F1 Confidence Interval Method to being a Modified Agresti-Coull F1 Confidence Interval Method, depending on its adjustment constant.
  • It can range from being a Symmetric Agresti-Coull F1 Confidence Interval Method to being a Asymmetric Agresti-Coull F1 Confidence Interval Method, depending on its interval construction.
  • It can range from being a Single-Level Agresti-Coull F1 Confidence Interval Method to being a Multi-Level Agresti-Coull F1 Confidence Interval Method, depending on its confidence level.
  • It can range from being a Component-Wise Agresti-Coull F1 Confidence Interval Method to being a Direct Agresti-Coull F1 Confidence Interval Method, depending on its application strategy.
  • ...
• Example(s):
  • Standard Plus-Four Applications, such as:
    • Original: TP=8, FP=2, FN=3; Adjusted: TP'=10, FP'=4, FN'=5.
    • F1 from adjusted: 2*10/(2*10+4+5) = 20/29 = 0.690 vs raw 0.762.
    • 95% CI: [0.55, 0.82] with 95.5% actual coverage.
  • Coverage Performance Studys, such as:
    • n=50: Achieves 95.8% average coverage across all F1 values.
    • n=100: Achieves 95.3% coverage, very close to nominal.
    • Minimum coverage 94.2% even at extreme F1 values.
  • Comparison with Other Methods, such as:
    • Wald CI: 85% coverage, Agresti-Coull: 95.5% coverage.
    • Wilson: 95.3% coverage but more complex computation.
    • Plus-four simpler than Wilson, nearly equivalent performance.
  • ...
• Counter-Example(s):
  • Wald F1 Confidence Interval Method, which uses no adjustment and undercovers.
  • Wilson Score F1 Confidence Interval Method, which uses data-dependent adjustment.
  • Jeffreys Prior F1 Interval Method, which adds 0.5 instead of 2.
• See: Plus-Four Adjustment Method, Confidence Interval Method, F1 Score, Coverage Probability, Wilson Score F1 Confidence Interval Method, Fixed Inflation Method, Pseudo-Count Method, Brown-Cai-DasGupta Recommendation, Small Sample Inference, Conservative Estimation, Coverage Probability Validation Method.

---