How this instrument works
When you run a single hypothesis test at significance level alpha = 0.05, you accept a 5% chance of a false positive on that one test. Run 10 independent tests at the same threshold and the chance that at least one comes back 'significant' purely by chance climbs well above 5% — the more comparisons you run, the more your overall false-positive rate inflates, even though every individual test still uses the same 0.05 cutoff.
The Bonferroni correction is the simplest fix: divide your chosen significance level by the number of comparisons, m, and use that smaller number as the new per-test threshold. With alpha = 0.05 and 10 comparisons, each individual test now has to clear the far stricter bar of 0.005 before you call it significant, which keeps the overall (family-wise) chance of any false positive across all 10 tests close to the original 5%.
It is deliberately conservative — as m grows, the adjusted alpha shrinks fast, which protects hard against false positives but makes it correspondingly harder to detect a real effect (a higher false-negative risk). That trade-off is why it's the standard first tool taught for multiple-comparisons problems, even though researchers sometimes reach for less conservative alternatives (like the Holm-Bonferroni or Benjamini-Hochberg procedures) when many comparisons make Bonferroni's strictness too costly.
- Enter your chosen significance level into Significance level (alpha) — commonly 0.05 or 0.01, and it must be between 0 and 1.
- Enter the total number of hypothesis tests you're running into Number of comparisons (m) — a whole number, at least 1.
- Read Bonferroni-adjusted alpha — the new, stricter per-test significance threshold to use in place of your original alpha.
- Compare each individual test's p-value against the adjusted alpha, not the original one, when deciding significance.
- With m = 1 (a single test), the adjustment leaves alpha unchanged — the correction only bites once you're running more than one comparison.
Worked example — alpha = 0.05 across 10 comparisons
Enter 0.05 into Significance level (alpha) and 10 into Number of comparisons (m) — you're running 10 hypothesis tests, at a standard 5% threshold, and want to correct for running that many at once. Bonferroni-adjusted alpha reads 0.005, computed simply as 0.05 / 10.
In practice: instead of calling any test with p < 0.05 'significant', you now require p < 0.005 before treating a result as significant. This keeps the overall chance of at least one false positive across all 10 tests close to the original 5%, rather than the inflated rate you'd get by checking all 10 against the unadjusted 0.05 threshold.
Questions
Why does running more comparisons inflate the false-positive rate?
Because each individual test carries its own independent chance of a false positive, and those chances stack up across tests. With 10 independent tests each run at alpha = 0.05, the probability that at least one turns up significant by chance alone is roughly 1 − (0.95)^10 ≈ 40% — far higher than the 5% you might assume from looking at a single test's threshold.
What is the difference between alpha and the Bonferroni-adjusted alpha?
Alpha is your original per-test significance threshold, typically 0.05, chosen before you knew how many tests you'd run. The adjusted alpha (alpha / m) is the stricter threshold you actually compare each test's p-value against once you know m, the total number of comparisons — it exists specifically to correct for running more than one test.
Is the Bonferroni correction too strict?
It can be, especially with a large number of comparisons — dividing by m = 100 shrinks alpha = 0.05 down to 0.0005, a threshold that can be very hard for a genuinely real effect to clear. Bonferroni is intentionally conservative to control the family-wise error rate tightly; less strict alternatives like Holm-Bonferroni or the Benjamini-Hochberg false discovery rate procedure are often used instead when that strictness costs too much statistical power.
Do the comparisons need to be independent for Bonferroni to work?
No — that's actually one of its strengths. The Bonferroni correction remains mathematically valid (conservative, if anything) even when the m tests are correlated with each other, unlike some more powerful alternatives that assume independence. That robustness is part of why it remains the standard default despite being stricter than necessary in many cases.
What counts as 'a comparison' when setting m?
Every hypothesis test you plan to interpret as part of the same family of decisions — every pairwise group comparison, every subgroup analysis, every outcome variable tested. If you ran 5 tests but only report and interpret 3, m should still reflect all 5 planned comparisons; deciding m after seeing which tests looked interesting defeats the purpose of the correction.
Can the adjusted alpha come out to exactly the original alpha?
Yes, whenever m = 1 — dividing by 1 leaves alpha unchanged, which makes sense because the correction only exists to account for running more than one test. As soon as m rises to 2 or more, the adjusted alpha drops below the original, and it keeps shrinking as m grows further.