Hierarchical Cross-Validation Algorithm

A Hierarchical Cross-Validation Algorithm is a cross-validation algorithm that employs multiple validation levels for different evaluation purposes.

• AKA: Multi-Level Cross-Validation Algorithm, Layered Cross-Validation Algorithm.
• Context:
  • It can (typically) separate Parameter Selection Levels from performance estimation levels.
  • It can (typically) prevent Selection Bias through independent validation layers.
  • It can (typically) support Complex Model Pipelines with multiple tuning stages.
  • It can (typically) maintain Statistical Validity across hierarchical decisions.
  • ...
  • It can (often) require Exponential Computation Time with hierarchy depth.
  • It can (often) produce Conservative Estimates compared to flat cross-validation algorithms.
  • ...
  • It can range from being a Two-Level Hierarchical Cross-Validation Algorithm to being a Multi-Level Hierarchical Cross-Validation Algorithm, depending on its hierarchy depth.
  • It can range from being a Sequential Hierarchical Cross-Validation Algorithm to being a Parallel Hierarchical Cross-Validation Algorithm, depending on its execution strategy.
  • It can range from being a Fixed-Structure Hierarchical Cross-Validation Algorithm to being a Adaptive Hierarchical Cross-Validation Algorithm, depending on its structure flexibility.
  • It can range from being a Homogeneous Hierarchical Cross-Validation Algorithm to being a Heterogeneous Hierarchical Cross-Validation Algorithm, depending on its level uniformity.
  • ...
• Example(s):
  • Nested Cross-Validation Algorithms for hyperparameter optimization.
  • Triple Cross-Validation Algorithms for feature selection and model selection.
  • Cascaded Cross-Validation Algorithms for ensemble construction.
  • Recursive Cross-Validation Algorithms for architecture search.
  • 5x2 Cross-Validation Algorithms with hierarchical comparison structures.
  • ...
• Counter-Example(s):
  • Flat Cross-Validation Algorithms, which use single validation levels.
  • Simple Holdout Algorithms, which lack hierarchical structures.
  • Bootstrap Algorithms, which use resampling rather than hierarchical partitions.
• See: Cross-Validation Algorithm, Nested Cross-Validation Algorithm, Model Selection Task, Hyperparameter Optimization Task, Validation Task, Performance Estimation Algorithm, Statistical Validation Method.

---