13 Research methods, statistics, critical review and evidence-based practice13.9 Differential and analytic statistics

13.9.5 Type I error, Type II error, power and sample size

Creator: Resident Doctor Committee
Published: 2026-10-14T21:05:44.632Z

3 min read2 figures

Written by Resident Doctor Committee · Published 14 October 2026

Try it

Power, alpha and beta

Interactive

If there is no real differenceIf the real difference is this size

Effect size to detect

A standardised difference: 0.2 small, 0.5 moderate, 0.8 large.

Participants per group

Power: 80.7%chance of finding a real effect
Type II error rate: 19.3%missing a real effect
Type I error rate: 5%calling noise a finding
Needed for 80% power: 63 per group
Needed for 90% power: 85 per group

Demanding a smaller alpha moves the cut-off to the right, which shrinks the false positive tail and grows the false negative one. Only more participants, or a bigger real effect, shrinks both.

An effect size (d) of 0.5, a 4-point HAM-D difference over an SD of 8, with 64 patients per group gives about 80% power. Drop the effect size to 0.25 and find the number per group that restores 80% power. Then move alpha to 0.01 and watch the Type II error rate grow.

Type I and type II errors

Test decision	Null hypothesis true	Null hypothesis false
Rejects the null	Type I error, probability alpha	Correct rejection
Fails to reject the null	Correct non-rejection	Type II error, probability beta

A type I error is a false positive: rejecting a true null hypothesis, which declares an effect that does not exist.
A type II error is a false negative: failing to reject a false null hypothesis, which misses a real effect.
The significance level (alpha) is the risk of a type I error, fixed before the study as the decision threshold.
The p-value is calculated from the data afterwards and compared with alpha.

Two overlapping bell curves: a solid black curve where the null hypothesis is true and a dashed violet curve where a real effect exists, split by a vertical critical value line. The null tail beyond the line is shaded red as alpha, the alternative area before the line amber as beta, and the rest of the alternative curve green as power. — The red tail of the null curve beyond the critical value is alpha, the type I error (false positive), and the amber part of the real effect curve short of it is beta, the type II error (false negative), leaving power, 1 minus beta, in green. Drawn for MRCPsychiatrist

Precision

The standard error is the standard deviation divided by the square root of n, so quadrupling the sample halves it.
The standard error describes the precision of an estimate, not the spread of individual scores.
The confidence level is 1 minus alpha.
A confidence interval is a range that captures the true population value in that share of repeated samples, and its width shows precision.

Multiple testing

Many comparisons inflate the type I error rate, so a lone significant result among many unplanned tests is likely chance, hypothesis-generating and in need of replication.
Bonferroni correction divides alpha by the number of tests to hold the overall type I error rate near the nominal level.

Power

Power is the probability of detecting an effect that truly exists, equal to 1 minus beta. It is a design property, so it cannot be inferred from a response rate or other reported percentages.

Change	Effect on power	Why
Larger sample	Rises	Standard error shrinks
Larger true or expected effect	Rises	More signal relative to noise
Less outcome variability	Rises, with fewer type II errors	Standard error shrinks
Stricter alpha	Falls	More extreme evidence needed

Tightening alpha is the one change that buys type I protection at the cost of more type II error.
A placebo comparator gives a larger expected effect than an active comparator, so the trial has more power.
Recruiting fewer participants than the power calculation requires lowers power and raises type II error risk, but the type I rate stays fixed by alpha.
Small studies give unstable estimates, and those that reach significance tend to overstate the true effect.

Two stacked panels, each with a null curve centred on zero and a dashed real effect curve. Moving from significance level 0.05 (top) to 0.01 (bottom) shifts the critical value to the right, so the red alpha tail shrinks, the amber beta area grows from 0.20 to 0.44 and the green power area falls from 0.80 to 0.56. — Tightening alpha from 0.05 to 0.01 with sample size, variance and effect unchanged pushes the critical value outwards, so fewer false positives cost a larger beta and power falls from 0.80 to 0.56. Drawn for MRCPsychiatrist

Reading a non-significant result

A wide confidence interval crossing the null may include important effects, so the result may be a type II error, not proof of no effect.
In a case-control study, an exposure that few participants have gives low power, so an interval crossing 1 may be a missed real association.
A narrow interval can exclude clinically important effects, so interpret the interval, because post hoc power is uninformative.

Power calculation and sample size

A power calculation is done a priori, before recruitment, to fix the minimum sample size that detects a clinically worthwhile difference at a chosen alpha and target power.

For two equal groups, n per group = 2 x (z alpha/2 + z beta)^2 x sigma^2 / delta^2, where sigma is the standard deviation and delta the target difference.
For a 3-point difference on the Montgomery-Åsberg Depression Rating Scale, with sigma 6, two-sided alpha 0.05 and 80% power, n = 2 x (1.96 + 0.84)^2 x 36 / 9 = 62.7, so 63 per arm.
Halving the difference roughly quadruples the sample: 1.5 points needs 251 per arm, and the 63-per-arm design has only about 30% power for it.
With 15% expected dropout, recruit 63 / 0.85, rounded up to 75.

Cluster randomisation

Cluster randomisation has less power than randomising the same number of individuals, because correlated outcomes within clusters shrink the effective sample size.
The intracluster correlation coefficient (ICC) is the share of total outcome variance lying between clusters, as opposed to within them.
The design effect is 1 + (m minus 1) x ICC, where m is cluster size; m = 10 and ICC = 0.05 give 1.45, so about 45% more participants are needed.

Equivalence and non-inferiority

Equivalence trials need the largest samples, because a narrow two-sided margin demands a very precise estimate.
Non-inferiority trials with a wide one-sided margin need far fewer participants.

Revision material for the MRCPsych examinations. Not clinical guidance, and not for use in patient care.