Permutation Test Algorithm

A Permutation Test Algorithm is a statistical resampling algorithm that generates null distributions by randomly permuting data labels to perform hypothesis tests.

• AKA: Randomization Test Algorithm, Exact Test Algorithm, Fisher's Permutation Test.
• Context:
  • It can (typically) provide Exact P-Values without distributional assumptions.
  • It can (typically) test Null Hypothesises through label shuffling.
  • It can (typically) handle Non-Normal Data and small sample sizes.
  • It can (typically) maintain Type I Error Rates at nominal levels.
  • ...
  • It can (often) require Exhaustive Enumeration for small datasets.
  • It can (often) use Monte Carlo Sampling for large permutation spaces.
  • ...
  • It can range from being an Exact Permutation Test Algorithm to being a Monte Carlo Permutation Test Algorithm, depending on its enumeration strategy.
  • It can range from being a One-Sample Permutation Test Algorithm to being a Multi-Sample Permutation Test Algorithm, depending on its group count.
  • It can range from being a Simple Permutation Test Algorithm to being a Stratified Permutation Test Algorithm, depending on its permutation constraints.
  • It can range from being a Univariate Permutation Test Algorithm to being a Multivariate Permutation Test Algorithm, depending on its variable dimension.
  • ...
• Example(s):
  • Two-Sample Permutation Test Algorithms for group comparisons.
  • Correlation Permutation Test Algorithms for association testing.
  • ANOVA Permutation Test Algorithms for multi-group comparisons.
  • Regression Permutation Test Algorithms for coefficient significance.
  • ...
• Counter-Example(s):
  • Bootstrap Resampling Algorithms, which resample with replacement rather than permute labels.
  • Parametric Test Algorithms, which assume specific distributions.
  • Jackknife Algorithms, which delete observations rather than shuffle labels.
• See: Statistical Resampling Algorithm, Hypothesis Test, P-Value, Null Distribution, Bootstrap Resampling Algorithm, Non-Parametric Test, Model Comparison Task.

---