Daniel & Friends / Glossary / Statistical significance
Research and dataStatistical significance
Statistical significance indicates how likely it is that a difference or relationship in the data arose by chance. A result is statistically significant when that probability falls below a threshold set in advance, commonly 5 percent.
Daniel Votruba ·
In practice
A significant difference can still be too small to matter. Look at effect size next to significance, and fix the test duration upfront so an early green result does not mislead you.
Related terms
- P-value: The p-value is the probability of seeing a result as extreme as the one measured, or more extreme, if the null hypothesis (no real difference) were true.
- Confidence interval: A confidence interval is a range that, with a chosen level of confidence (most often 95 percent), contains the true value of a figure for the whole population.
- A/B test: An A/B test compares two versions, for example two headlines or two forms, by showing each to a different half of the visitors.
- Multivariate test (MVT): A multivariate test compares several combinations of multiple elements on a page or in an ad at once, for example headline, image and button.
More terms in the group Research and data
Zero-party data · Sample and representativeness · Anonymization and pseudonymization · Respondent · BI and dashboard · Research panel · Big data · Questionnaire (survey) · CAWI, CATI and CAPI · Quantitative research · CDP (customer data platform) · Qualitative research · Conjoint analysis · OKR (objectives and key results) · Consent to data processing · North Star Metric · Correlation and causation · Neuromarketing · Data clean room · Mystery shopping · Data governance · Data warehouse and data lake · MaxDiff · Desk research
Terms in practice
Need help putting them to work?
Terms are one thing, putting them to work is another. We go through which of them pay off for you and say so plainly. You get a first opinion for free and we reply within 24 hours.
Book a call