Permutation Test

References

http://www2.stat.duke.edu/~ar182/rr/examples-gallery/PermutationTest.html
- QUOTE: Permutation tests are a group of nonparametric statistics. Here we use a permutation test to test the null hypothesis that two different groups come from the same distribution. The notation and examples shown here are borrowed from Efron and Tibshirani’s An Introduction to the Bootstrap [1]. Our specific problem is that we observe two groups of data [math]\displaystyle{ \begin{eqnarray*} F \rightarrow \mathbf{z} &= \{ z_{1}, \ldots, z_{n} \}\\ G \rightarrow \mathbf{y} &= \{ y_{1}, \ldots, y_{m} \} \end{eqnarray*} }[/math] and we are interested in testing the null hypothesis that [math]\displaystyle{ H_{0}: F = G }[/math].

(Wikipedia, 2016) ⇒ http://en.wikipedia.org/wiki/Exact_test 2016-08-13
- So when the result of a statistical analysis is said to be an “exact test” or an “exact p-value”, it ought to imply that the test is defined without parametric assumptions and evaluated without using approximate algorithms. In principle however it could also mean that a parametric test has been employed in a situation where all parametric assumptions are fully met, but it is in most cases impossible to prove this completely in a real world situation. Exceptions when it is certain that parametric tests are exact include tests based on the binomial or Poisson distributions. Sometimes permutation test is used as a synonym for exact test, but although all permutation tests are exact tests, not all exact tests are permutation tests.

(Upton & Cook, 2008) ⇒ Graham Upton, and Ian Cook. (2008). “A Dictionary of Statistics, 2nd edition revised." Oxford... University Press. ISBN:0199541450
- QUOTE: Permutation Test: A simple type of hypothesis test. Denote the value of some test statistic by T1. The observed *data values are randomly redistributed amongst the experimental units. The test statistic is calculated for each such redistribution. Depending on the number of data values, either all possible permutations are made, or a random selection (of say 1000 permutations) is made. For each permutation the value of the test statistic is considered. The significance of the value T is determined by the proportion of permutations that lead to values greater than, or equal to, I1