McNemar's Test Algorithm

A McNemar's Test Algorithm is a paired comparison algorithm that tests marginal homogeneity in 2x2 contingency tables for paired binary data.

• AKA: McNemar Chi-Square Test, McNemar's Change Test, Paired Binary Test Algorithm.
• Context:
  • It can (typically) analyze Before-After Studys with binary outcomes.
  • It can (typically) compare Paired Classifiers on same test sets.
  • It can (typically) use Chi-Square Distributions with one degree of freedom.
  • It can (typically) detect Asymmetric Error Patterns between paired models.
  • ...
  • It can (often) require Continuity Corrections for small samples.
  • It can (often) provide More Power than independent tests for paired designs.
  • ...
  • It can range from being an Exact McNemar's Test Algorithm to being an Asymptotic McNemar's Test Algorithm, depending on its computation method.
  • It can range from being a Standard McNemar's Test Algorithm to being a Corrected McNemar's Test Algorithm, depending on its continuity adjustment.
  • It can range from being a Two-Sided McNemar's Test Algorithm to being a One-Sided McNemar's Test Algorithm, depending on its hypothesis direction.
  • It can range from being a Simple McNemar's Test Algorithm to being an Extended McNemar's Test Algorithm, depending on its table dimension.
  • ...
• Example(s):
  • Classifier Comparison McNemar's Tests for ML model comparisons.
  • Medical Treatment McNemar's Tests for treatment effects.
  • Diagnostic Test McNemar's Tests for test agreement.
  • A/B Testing McNemar's Tests for conversion rate comparisons.
  • ...
• Counter-Example(s):
  • Chi-Square Independence Tests, which assume independent samples.
  • Fisher's Exact Tests, which handle independent 2x2 tables.
  • Cochran's Q Tests, which extend to multiple treatments.
• See: Paired Comparison Algorithm, Contingency Table, Binary Classification Performance Measure, 5x2 Cross-Validation Algorithm, Hypothesis Test, Chi-Square Test, Model Comparison Task.

---