What does statistical significance mean?

• A. The result proves the hypothesis beyond reasonable doubt.
• B. The result is always large in magnitude.
• C. The result is independent of sample size.
• D. The observed result is unlikely to have occurred by chance, given a predefined threshold.

Answer: (D)

Statistical significance is about how likely the observed result would occur if there were no real effect. It uses a predefined threshold (the alpha level) to decide whether the data are unlikely enough under the null hypothesis. When the p-value falls below this threshold, we say the result is statistically significant, meaning it’s unlikely to have happened by chance given the threshold we set.

This is why the best description is that the observed result is unlikely to have occurred by chance, given a predefined threshold. It does not guarantee that the hypothesis is true beyond doubt, and it doesn’t say anything about how large the effect is. Magnitude can be small or large and still be significant depending on sample size and variability. Also, significance depends on how big the sample is: with a large sample, tiny effects can become statistically significant, while a small sample might miss large effects. Finally, significance doesn’t measure practical importance; it only concerns probability under the null hypothesis.