Omnibus Statistical Test

An Omnibus Statistical Test is a statistical hypothesis testing task that examines overall statistical significance across multiple population parameters or group comparisons simultaneously.

• AKA: Omnibus Test, Global Statistical Test, Overall Significance Test, Joint Hypothesis Test, Overall F-Test.
• Context:
  • It can (typically) test whether explained variance significantly exceeds unexplained variance across all factors.
  • It can (typically) evaluate null hypotheses about multiple parameters jointly rather than individually.
  • It can (typically) control Family-Wise Error Rate for the overall hypothesis before post-hoc tests.
  • It can (typically) serve as a gatekeeper test determining whether detailed pairwise comparisons are warranted.
  • It can (often) detect overall effects even when individual parameter effects vary in significance.
  • It can (often) yield non-significant results when some parameters show no effect despite other significant effects.
  • It can (often) have lower statistical power for specific effects than targeted tests.
  • It can (often) be followed by Multiple Comparison Tests or Statistical Contrasts to identify specific differences.
  • It can (often) require Adjustment for Multiple Comparisons like Bonferroni Correction for subsequent tests.
  • It can range from being a Parametric Omnibus Test to being a Non-Parametric Omnibus Test, depending on its distributional assumptions.
  • It can range from being a Fixed-Effects Omnibus Test to being a Random-Effects Omnibus Test, depending on its effect model.
  • It can range from being a One-Way Omnibus Test to being a Multi-Way Omnibus Test, depending on its factor count.
  • It can integrate with Statistical Software Packages for automated analysis.
  • It can integrate with Experimental Design Systems for power calculations.
  • ...
• Example(s):
  • ANOVA Omnibus Tests, such as:
    • One-Way ANOVA F-Test testing equality of means across multiple groups.
    • Two-Way ANOVA F-Test testing main effects and interaction effects.
    • Repeated Measures ANOVA F-Test for within-subjects designs.
    • Mixed ANOVA F-Test for combined between and within factors.
    • MANOVA Omnibus Test for multivariate group comparisons.
    • ANCOVA F-Test testing while controlling covariates.
  • Regression Omnibus Tests, such as:
    • Overall Regression F-Test testing all regression coefficients jointly.
    • Partial F-Test testing subsets of predictors.
    • Wald Test testing multiple parameter restrictions.
    • Score Test testing model constraints.
    • Likelihood Ratio Test comparing nested models.
  • Chi-Square Omnibus Tests, such as:
    • Pearson Chi-Square Test for contingency table independence.
    • Likelihood Ratio Chi-Square Test for model comparison.
    • Chi-Square Goodness-of-Fit Test for distribution testing.
    • Omnibus Chi-Square Test in Logistic Regression for blocks of variables.
  • Non-Parametric Omnibus Tests, such as:
    • Kruskal-Wallis Test as non-parametric alternative to one-way ANOVA.
    • Friedman Test for non-parametric repeated measures.
    • Quade Test for ranked ANCOVA alternative.
    • Jonckheere-Terpstra Test for ordered alternatives.
  • Multivariate Omnibus Tests, such as:
    • Hotelling's T-Squared Test for multivariate mean vectors.
    • Wilks' Lambda Test for multivariate group differences.
    • Pillai's Trace Test for robust multivariate testing.
    • Roy's Largest Root Test for maximum eigenvalue.
    • Lawley-Hotelling Trace Test for overall multivariate effects.
  • Time Series Omnibus Tests, such as:
    • Box-Pierce Test for autocorrelation.
    • Ljung-Box Test for time series independence.
    • Portmanteau Test for residual correlation.
    • Breusch-Godfrey Test for serial correlation.
  • Structural Equation Model Omnibus Tests, such as:
    • Chi-Square Model Fit Test for overall model fit.
    • CFI Test for comparative fit.
    • RMSEA Test for approximate fit.
  • ...
• Counter-Example(s):
  • Pairwise Comparison Test, which compares specific pairs rather than overall significance.
  • Post-Hoc Test, which follows omnibus tests for specific comparisons.
  • Planned Contrast Test, which tests specific hypotheses rather than global ones.
  • Stationarity Test, which tests time series properties rather than group differences.
  • Normality Test, which tests distributional assumptions rather than parameter differences.
  • Homogeneity Test, which tests variance equality rather than mean differences.
  • Simple Effects Test, which examines effects at specific levels.
• See: Analysis of Variance, F-Test, Chi-Square Test, Likelihood-Ratio Test, Multiple Comparison Problem, Family-Wise Error Rate, Post-Hoc Analysis, Statistical Contrast, Fisher's LSD Test, Tukey's HSD Test, Scheffé's Method, Type I Error Control, Bonferroni Correction, Statistical Power, Effect Size.

---

---

References

2016

• (Wikipedia, 2016) ⇒ http://en.wikipedia.org/wiki/Omnibus_test Retrieved 2016-08-28
  • Omnibus tests are a kind of statistical test. They test whether the explained variance in a set of data is significantly greater than the unexplained variance, overall. One example is the F-test in the analysis of variance. There can be legitimate significant effects within a model even if the omnibus test is not significant. For instance, in a model with two independent variables, if only one variable exerts a significant effect on the dependent variable and the other does not, then the omnibus test may be non-significant. This fact does not affect the conclusions that may be drawn from the one significant variable. In order to test effects within an omnibus test, researchers often use contrasts.

In addition, Omnibus test as a general name refers to an overall or a global test. Other names include F-test or Chi-squared test.

Omnibus test as a statistical test is implemented on an overall hypothesis that tends to find general significance between parameters' variance, while examining parameters of the same type [...]

Omnibus tests commonly refers to either one of those statistical tests:

• ANOVA F test to test significance between all factor means and/or between their variances equality in Analysis of Variance procedure ;
• The omnibus multivariate F Test in ANOVA with repeated measures ;
• F test for equality/inequality of the regression coefficients in Multiple Regression;
• Chi-Square test for exploring significance differences between blocks of independent explanatory variables or their coefficients in a logistic regression.

---