See: Statistically Significant Result, Statistical Significance Measure.
References
- http://en.wikipedia.org/wiki/Statistical_significance
- In Statistics, a result is called statistically significant if it is unlikely to have occurred by Chance. "A statistically significant difference" simply means there is statistical evidence that there is a difference; it does not mean the difference is necessarily large, important, or significant in the common meaning of the word.
- The significance level of a test is a traditional Frequentist Statistical Hypothesis Testing concept. In simple cases, it is defined as the probability of making a decision to reject the null hypothesis when the Null Hypothesis is actually true (a decision known as a Type I Error, or "false positive determination"). The decision is often made using the p-Value: if the p-value is less than the significance level, then the null hypothesis is rejected. The smaller the p-value, the more significant the result is said to be.
- In more complicated, but practically important cases, the significance level of a test is a probability such that the probability of making a decision to reject the null hypothesis when the Null Hypothesis is actually true is no more than the stated probability. This allows for those applications where the probability of deciding to reject may be much smaller than the significance level for some sets of assumptions encompassed within the null hypothesis.