13 Research methods, statistics, critical review and evidence-based practice13.4 Systematic reviews

13.4.5 Fixed effect and random effects meta analysis

2 min read1 figure

Written by Resident Doctor Committee · Published

Try it

Pooling studies in a forest plot

Interactive

6 studies pooled by fixed effects. The pooled risk ratio is 1.20 from 1.11 to 1.31, with I squared of 53%.Trial A1.20 (1.07 to 1.35), 49%Trial B1.10 (0.960 to 1.26), 36%Trial C1.60 (1.19 to 2.15), 8%Trial D1.45 (0.980 to 2.15), 4%Trial E2.10 (1.17 to 3.78), 2%Trial F0.900 (0.520 to 1.56), 2%Pooled1.20 (1.11 to 1.31)0.5001.002.004.00Risk ratio, log scale
Pooled risk ratio
1.201.11 to 1.31
I squared
53%variation beyond chance
Cochran's Q
10.65on 5 degrees of freedom
Tau squared
0.015between-study variance

The trials here are invented to make the arithmetic visible. With I squared of 53%, a random-effects pool is the honest one: it admits that the trials are not all estimating the same number.

The fixed effect diamond is about RR 1.20 (1.11 to 1.31). Switch to random effects: the boxes become more alike and the diamond widens to about 1.27 (1.09 to 1.47). I squared stays the same, because heterogeneity belongs to the data, not the model.

What each model assumes

Fixed-effect model

Random-effects model

Assumed truth

All studies estimate one common true effect

Each study has its own true effect, drawn from a distribution of effects

Why study results differ

Chance sampling error alone

Sampling error plus real variation between studies

What is estimated

The common effect

The mean of the distribution of effects

Weight given to each study

1 / variance

1 / (variance + between-study variance)

  • The fixed-effect model is also called the common-effect model.

  • The between-study variance (tau squared) is estimated from the data, not known.

How the two answers differ

  • A random-effects model differs from a fixed-effect one only by adding the estimated between-study variance.

  • When heterogeneity is present, this added variance widens the pooled confidence interval.

  • The added variance also evens out the weights, so smaller studies gain influence and the pooled estimate can move.

  • When the study intervals overlap and I squared is 0%, the two models give the same answer.

Compare each pair of bars: once the between-study variance is added, the two most precise trials, A and B, lose share of the pooled estimate and the four smaller trials gain it, because random effects evens out the weights. Illustrative data. Drawn for the MRCPsychiatrist statistics course

Choosing a model

  • A random-effects model is chosen when statistical heterogeneity is present, as shown by the heterogeneity test and I squared.

  • Clinical or methodological diversity between studies does not by itself call for a random-effects model.

  • A random-effects model describes heterogeneity but does not explain it, so look for causes with justified subgroup analyses.

  • A confidence interval describes uncertainty about the mean effect, whereas a prediction interval gives the plausible effect in a new comparable setting, including the spread between studies.

Reading a forest plot

  • Each study is plotted as an estimate with its 95% confidence interval against the line of no effect.

  • A study is non-significant when its 95% confidence interval includes the line of no effect.

  • The number of study intervals that cross the null value is therefore the number of non-significant studies.

Revision material for the MRCPsych examinations. Not clinical guidance, and not for use in patient care.