13 Research methods, statistics, critical review and evidence-based practice13.8 Descriptive statistics

13.8.7 Confidence intervals

2 min read2 figures

Written by Resident Doctor Committee · Published

Try it

What a confidence interval tells you

Interactive

An estimate of 2.50 with a 95% confidence interval from 0.02 to 4.98, which excludes the null value of 0.No difference2.50 (0.02 to 4.98)-10-50510Difference between the groups

The only one of these you can usually do anything about.

Standard error
1.265
Margin of error
2.4791.96 standard errors
Interval
0.02 to 4.98
Crosses the null?
Nono difference is not compatible

To halve this interval you would need about 161 patients, four times as many. A wide interval that crosses the null has not shown that the treatment does nothing; it has shown that the study was too small to say.

Drop n from 40 to 30 and watch the 95% CI for a 2.5-point fall in PHQ-9 score cross 0, then raise n to 160 to halve its width. Back at n = 40, switch confidence to 90% and 99% to see the interval narrow and widen.

What a 95% confidence interval claims

  • A 95% confidence interval (CI) for a population mean expresses confidence about where the true population mean lies, not about individual values, the sample or future estimates.

  • The 95% describes the method: intervals built this way from repeated samples contain the true value 95% of the time.

  • Saying there is a 95% probability that the true mean lies in one particular interval is not correct for a standard CI; that statement belongs to a Bayesian credible interval.

  • A CI cannot be applied to a single patient and does not give the proportion of patients who will respond.

Each bar is a 95% confidence interval from a separate sample, and most cross the fixed true mean while a few in blue miss it, which is why the 95% describes the method and not any single interval. Randy.l.goodrich, CC BY-SA 4.0, via Wikimedia Commons (cropped)

What an interval shows

  • A CI gives the range of population effect sizes compatible with the data, the precision of the estimate, the direction of effect and statistical significance.

  • It reflects sampling error only, so a narrow interval shows precision, not absence of bias.

Calculating an interval

  • 95% CI for a mean = mean ± 1.96 × SE, where the standard error (SE) = standard deviation / √n and n is the sample size.

  • A mean depression score of 22.4 (standard deviation 6.0, n = 100) gives SE = 0.6, so the 95% CI is 21.2 to 23.6.

Confidence level

Multiplier

90%

1.645

95%

1.96

99%

2.576

  • Lowering the level from 95% to 90% narrows the interval symmetrically about the same point estimate, because the multiplier falls from 1.96 to 1.645, so the interval misses the true value more often.

  • Intervals narrow as the sample grows, because SE falls with √n, and widen with greater variability, few events or a higher confidence level.

The same point estimate gives a 90% interval (z = 1.645) narrower than the 95% (z = 1.96) and 99% (z = 2.576) intervals, and all three stay centred on 4.0 because only the multiplier changes. Drawn for MRCPsychiatrist

Null values and significance

Measure

Examples

No-effect value

Ratio

Relative risk, odds ratio, hazard ratio

1

Difference

Difference in means, relative risk reduction (1 minus the relative risk)

0

  • A two-sided test at the 5% level rejects the null exactly when the 95% CI excludes the null value.

  • At p = 0.05, one limit of the 95% CI sits on the null value.

  • A hazard ratio whose 95% CI includes 1 shows no statistically significant change in risk, even when the point estimate lies away from 1.

  • Significance of an association can change between regression models; in each model, read it from whether the odds ratio's 95% CI spans 1.

Reading the whole interval

  • A trial that misses significance is inconclusive, not negative, when the favourable end of its CI still reaches a clinically important benefit.

  • A relative risk of 0.78 (95% CI 0.58 to 1.05) includes 1 but is compatible with anything from a 42% reduction to a 5% increase in risk.

Revision material for the MRCPsych examinations. Not clinical guidance, and not for use in patient care.