Repeated Cross-Validation Algorithm

A Repeated Cross-Validation Algorithm is a cross-validation algorithm that performs multiple iterations with different random splits to reduce variance in performance estimates.

• AKA: Repeated K-Fold CV Algorithm, Multiple Random Subsampling Algorithm, Iterated Cross-Validation Algorithm.
• Context:
  • It can (typically) reduce Estimation Variance through multiple randomizations.
  • It can (typically) provide Confidence Intervals for performance metrics.
  • It can (typically) detect Split-Dependent Variation in model performance.
  • It can (typically) smooth Performance Estimates across random partitions.
  • ...
  • It can (often) require Linear Scaling of computational cost with repetition count.
  • It can (often) improve Statistical Power for model comparisons.
  • ...
  • It can range from being a Low-Repetition Cross-Validation Algorithm to being a High-Repetition Cross-Validation Algorithm, depending on its iteration count.
  • It can range from being a Fixed-K Repeated Cross-Validation Algorithm to being a Variable-K Repeated Cross-Validation Algorithm, depending on its fold variation.
  • It can range from being a Independent Repeated Cross-Validation Algorithm to being a Stratified Repeated Cross-Validation Algorithm, depending on its sampling constraint.
  • It can range from being a Sequential Repeated Cross-Validation Algorithm to being a Parallel Repeated Cross-Validation Algorithm, depending on its execution mode.
  • ...
• Example(s):
  • 10x10 Cross-Validation Algorithms with ten repetitions of 10-fold CV.
  • 5x2 Cross-Validation Algorithms for paired comparisons.
  • Repeated Stratified K-Fold Algorithms for imbalanced data.
  • Repeated Random Subsampling Algorithms for holdout variation.
  • ...
• Counter-Example(s):
  • Single K-Fold Cross-Validation Algorithms, which use one random split.
  • Leave-One-Out Cross-Validation Algorithms, which are deterministic.
  • Bootstrap Resampling Algorithms, which use sampling with replacement.
• See: Cross-Validation Algorithm, K-Fold Cross-Validation Algorithm, 5x2 Cross-Validation Algorithm, Performance Estimation Algorithm, Variance Reduction Method, Model Validation Task, Statistical Power.

---