13 Research methods, statistics, critical review and evidence-based practice13.1 Introductory concepts
13.1.1 Critical appraisal across study designs
2 min read
Written by Resident Doctor Committee · Published
New in the 2027 syllabus
The College added this topic for exams from January 2027.
Start with the question
Critical appraisal asks three things. Do the study's methods support its answer? How uncertain is that answer? Does it apply to the clinical decision?
First, check that the design fits the question. A trial estimates the effect of an intervention. An interview study can explain how patients experience treatment.
Keep two ideas separate:
Reporting quality is about what the paper describes.
Risk of bias is about whether the methods could systematically distort the result.
Match the threats to the design
Design | Main appraisal questions |
|---|---|
Randomised trial | Was allocation concealed, follow-up adequate and the intended comparison preserved? |
Cohort | Were exposure and outcome measured consistently, confounders addressed and losses explained? |
Case-control | Did the controls represent the source population? Was exposure measured the same way in cases and controls? |
Cross-sectional | Was sampling appropriate? Can the direction of the association be established? |
Qualitative study | Do the sampling, analysis, reflexivity and supporting extracts fit the research question? |
Systematic review | Were eligibility, searching, appraisal and synthesis specified and justified? |
Selection bias can arise before recruitment, during follow-up or through exclusions from the analysis.
Measurement bias happens when exposure or outcome is assessed systematically differently in the groups being compared.
Confounding is a competing explanation for an association. Statistical adjustment can only deal with the variables that were measured, and it depends on the causal assumptions made.
Read the result
Effect size tells you how big the finding is and which way it points, using a measure that suits the outcome.
Confidence intervals describe sampling uncertainty, assuming the model holds. A narrow interval does not rule out systematic bias.
Clinical importance depends on the absolute difference, the benefits and harms, and the follow-up period, not the p value alone.
Prespecification makes selective reporting easier to spot: compare the report with the registration, protocol and analysis plan.
Apply the findings with caution
Applicability depends on the patients studied, the setting, the comparator, how the treatment was delivered, the outcomes and the resources available.
Funding and competing interests are a reason to look closely at the design and reporting decisions. A declaration alone neither proves nor excludes bias.
A checklist helps you reach a reasoned judgement. It does not turn several serious weaknesses into an acceptable numerical score.