Multivariate Delta Method for Macro-F1 Variance Method

A Multivariate Delta Method for Macro-F1 Variance Method is a multivariate variance estimation method that applies vector calculus to the joint distribution of all class-specific counts to derive macro-F1 variance with between-class correlations.

• AKA: Joint Distribution Macro-F1 Variance Method, Multi-Class Delta Method F1 SE, Correlated Classes Macro-F1 Variance, Vector Delta Method for Macro-F1.
• Context:
  • It can typically use Jacobian matrix J of macro-F1 with respect to all class counts (TPᵢ, FPᵢ, FNᵢ).
  • It can typically compute Var(Macro-F1) = J · Cov(all counts) · Jᵀ for full covariance propagation.
  • It can typically model between-class correlations arising from shared features or label correlations.
  • It can often handle K classes with 3K count variables in a 3K×3K covariance matrix.
  • It can often reduce to Independent Groups Assumption in Variance Estimation Method as special case.
  • It can often incorporate hierarchical class structures through block covariance patterns.
  • It can range from being a Full Multivariate Delta Method for Macro-F1 Variance Method to being a Block-Diagonal Multivariate Delta Method for Macro-F1 Variance Method, depending on its correlation structure.
  • It can range from being a Parametric Multivariate Delta Method for Macro-F1 Variance Method to being a Empirical Multivariate Delta Method for Macro-F1 Variance Method, depending on its covariance estimation.
  • It can range from being a Fixed-K Multivariate Delta Method for Macro-F1 Variance Method to being a Variable-K Multivariate Delta Method for Macro-F1 Variance Method, depending on its class count handling.
  • It can range from being a Exact Multivariate Delta Method for Macro-F1 Variance Method to being a Approximate Multivariate Delta Method for Macro-F1 Variance Method, depending on its computational precision.
  • ...
• Example(s):
  • Three-Class Correlated Systems, such as:
    • Classes A,B,C with correlation ρ(A,B)=0.3, ρ(A,C)=0.1, ρ(B,C)=0.4.
    • 9×9 covariance matrix for (TP₁,FP₁,FN₁,TP₂,FP₂,FN₂,TP₃,FP₃,FN₃).
    • Macro-F1 SE=0.028 with correlations vs 0.032 assuming independence.
  • Hierarchical Class Structures, such as:
    • Animal taxonomy: strong correlation within mammals, birds, reptiles.
    • Block covariance: high within-group, low between-group correlation.
    • Variance reduction from positive within-block correlations.
  • Large-Scale Applications, such as:
    • 100-class problem: 300×300 covariance matrix.
    • Sparse covariance approximation for computational efficiency.
    • Threshold ρ<0.05 correlations to zero for sparsity.
  • ...
• Counter-Example(s):
  • Independent Groups Assumption in Variance Estimation Method, which ignores correlations.
  • Pairwise Covariance Method, which only considers adjacent classes.
  • Bootstrap Macro-F1 Variance, which estimates empirically.
• See: Variance Estimation Method, Multivariate Statistics, Delta Method, Macro-F1 Measure from Group Counts Method, Jacobian Matrix, Vector Calculus, Covariance Matrix, Class Correlation, Hierarchical Classification, Block Matrix, Macro-F1 P-Value Calculation Method.

---