Coverage Empirical Studies Catalog Method

A Coverage Empirical Studies Catalog Method is a research synthesis method that documents empirical coverage results from published studies comparing confidence interval methods for F1 scores and related performance metrics.

• AKA: F1 CI Coverage Research Catalog, Empirical Coverage Database Method, Coverage Study Repository Method, CI Performance Evidence Base.
• Context:
  • It can typically document Lam (2024) findings: Wilson outperforms Wald and exact methods.
  • It can typically include Brown et al. (2001): Wilson achieves 95.3% vs Wald 85.1% coverage.
  • It can typically reference Koyama et al. (2022): Multi-class F1 coverage as low as 80% with delta method.
  • It can often synthesize consistent findings across multiple independent studies.
  • It can often highlight parameter regions where methods fail (boundaries, small n).
  • It can often provide quantitative benchmarks for method comparison.
  • It can range from being a Single-Study Coverage Empirical Studies Catalog Method to being a Meta-Analysis Coverage Empirical Studies Catalog Method, depending on its synthesis scope.
  • It can range from being a Binary Coverage Empirical Studies Catalog Method to being a Multi-Class Coverage Empirical Studies Catalog Method, depending on its classification type.
  • It can range from being a Historical Coverage Empirical Studies Catalog Method to being a Current Coverage Empirical Studies Catalog Method, depending on its temporal focus.
  • It can range from being a Qualitative Coverage Empirical Studies Catalog Method to being a Quantitative Coverage Empirical Studies Catalog Method, depending on its analysis type.
  • ...
• Example(s):
  • Key Coverage Findings, such as:
    • Wald: 85% mean coverage, 0% at boundaries (multiple studies).
    • Wilson: 95.3% mean, never below 90% minimum (Brown et al. 2001).
    • BCa: 94.8% mean, handles skewness well (Efron & Tibshirani 1993).
  • Small Sample Study Results, such as:
    • n=20: Wald 82%, Wilson 94%, BCa 93% (Lam 2024).
    • n=12: Bootstrap can give [0,100%] correctly reflecting uncertainty.
    • n<10: Most methods struggle, Wilson+CC most reliable.
  • Boundary Behavior Studys, such as:
    • F1→0: Wald coverage 5%, Wilson 93%, BCa 94%.
    • F1→1: Wald coverage 45%, Wilson 95%, BCa 94%.
    • Extreme skewness handled only by asymmetric methods.
  • ...
• Counter-Example(s):
  • Theoretical Coverage Analysis, using mathematical proofs only.
  • Single Simulation Study, not synthesizing multiple sources.
  • Anecdotal Evidence Method, based on individual experiences.
• See: Research Synthesis Method, Coverage Probability Validation Method, Empirical Study, Lam 2024 Study, Brown-Cai-DasGupta 2001, Koyama 2022 Study, Meta-Analysis, Evidence-Based Practice, F1 Interval Selection Guide Method, Systematic Review.

---