Jeffreys Prior F1 Interval Method

A Jeffreys Prior F1 Interval Method is a Bayesian confidence interval method that uses Beta distribution with Jeffreys prior (adding 0.5 to parameters) to construct F1 score intervals similar to Wilson score intervals.

• AKA: Beta(0.5,0.5) F1 Interval Method, Bayesian F1 CI Method, Jeffreys F1 Posterior Interval, Half-Count F1 Interval Method.
• Context:
  • It can typically model F1 using Beta(F1·d + 0.5, (1-F1)·d + 0.5) where d = TP+FP+FN.
  • It can typically add 0.5 pseudo-counts (Jeffreys prior) rather than 2 (Agresti-Coull).
  • It can typically produce intervals very similar to Wilson Score F1 Confidence Interval Method.
  • It can often provide Bayesian interpretation as posterior credible interval.
  • It can often achieve good coverage properties through variance stabilization.
  • It can often handle boundary cases better than uninformative priors.
  • It can range from being a Standard Jeffreys Prior F1 Interval Method to being a Modified Jeffreys Prior F1 Interval Method, depending on its prior weight.
  • It can range from being a Symmetric Jeffreys Prior F1 Interval Method to being a Asymmetric Jeffreys Prior F1 Interval Method, depending on its posterior shape.
  • It can range from being a Exact Jeffreys Prior F1 Interval Method to being a Approximate Jeffreys Prior F1 Interval Method, depending on its computation method.
  • It can range from being a Conservative Jeffreys Prior F1 Interval Method to being a Liberal Jeffreys Prior F1 Interval Method, depending on its coverage tendency.
  • ...
• Example(s):
  • Basic Jeffreys Applications, such as:
    • TP=10, FP=2, FN=3, d=15, F1=0.769.
    • Beta parameters: α = 0.769*15 + 0.5 = 12.04, β = 3.96.
    • 95% credible interval from Beta(12.04, 3.96): [0.58, 0.89].
  • Boundary Behaviors, such as:
    • F1=0 in sample: Beta(0.5, d+0.5) gives positive upper bound.
    • F1=1 in sample: Beta(d+0.5, 0.5) gives lower bound < 1.
    • Never produces degenerate [0,0] or [1,1] intervals.
  • Coverage Comparisons, such as:
    • Jeffreys: 94.8% average coverage (close to Wilson's 95.3%).
    • More stable than uniform prior Beta(F1·d, (1-F1)·d).
    • Slightly less conservative than Agresti-Coull (95.5%).
  • ...
• Counter-Example(s):
  • Uniform Prior F1 Interval, which uses Beta(1,1) prior.
  • Agresti-Coull F1 Confidence Interval Method, which adds 2 instead of 0.5.
  • Frequentist F1 Interval Method, which has no prior.
• See: Bayesian Inference Method, Beta Distribution, Jeffreys Prior, Credible Interval, Wilson Score F1 Confidence Interval Method, Pseudo-Count Method, Variance Stabilization, Posterior Distribution, Conjugate Prior, F1 Score, Coverage Probability.

---