Multiple Hypothesis Testing Framework

A Multiple Hypothesis Testing Framework is a statistical hypothesis testing framework that addresses the challenges of conducting multiple statistical hypothesis tests simultaneously while controlling overall error rates.

• AKA: Multiple Comparisons Framework, Multiple Testing Framework, Simultaneous Hypothesis Testing Framework.
• Context:
  • It can typically control Family-Wise Error Rate (FWER) to limit the probability of any Type I Error.
  • It can typically control False Discovery Rate (FDR) to limit the expected proportion of false positives.
  • It can often employ Bonferroni Correction for conservative error control.
  • It can often utilize Benjamini-Hochberg Procedure for less conservative FDR control.
  • It can range from being a Conservative Multiple Hypothesis Testing Framework to being a Liberal Multiple Hypothesis Testing Framework, depending on its error control stringency.
  • It can range from being a Single-Step Multiple Hypothesis Testing Framework to being a Sequential Multiple Hypothesis Testing Framework, depending on its adjustment procedure.
  • It can range from being a Independent Multiple Hypothesis Testing Framework to being a Dependent Multiple Hypothesis Testing Framework, depending on its test correlation structure.
  • It can range from being a Parametric Multiple Hypothesis Testing Framework to being a Non-Parametric Multiple Hypothesis Testing Framework, depending on its distributional assumptions.
  • ...
• Example(s):
  • Genomic Association Testing Frameworks, such as:
    • Genome-Wide Association Study Framework testing millions of SNPs.
    • Gene Expression Analysis Framework comparing thousands of genes.
  • Clinical Trial Multiplicity Frameworks, such as:
    • Multiple Endpoint Testing Framework for primary and secondary outcomes.
    • Subgroup Analysis Framework for treatment effect heterogeneity.
  • Neuroimaging Analysis Frameworks, such as:
    • Voxel-Wise Testing Framework for brain imaging data.
    • Network Analysis Framework for connectivity studies.
  • ...
• Counter-Example(s):
  • A Single Hypothesis Testing Framework, which tests only one hypothesis.
  • A Exploratory Data Analysis Framework, which doesn't require error control.
  • A Descriptive Statistics Framework, which doesn't involve hypothesis testing.
• See: Statistical Hypothesis Testing Task, Multiple Testing Problem, Family-Wise Error Rate, False Discovery Rate, Bonferroni Correction, Holm-Bonferroni Method, Benjamini-Hochberg Procedure, Statistical Decision Framework, Type I Error, Statistical Power.

---