13 Research methods, statistics, critical review and evidence-based practice13.5 Diagnostic studies
13.5.7 Receiver operating characteristic curves
3 min read2 figures
Written by Resident Doctor Committee · Published
What the curve shows
A test scored on a continuous scale has a different sensitivity and specificity at each cut-off.
A receiver operating characteristic (ROC) curve plots these pairs across the cut-offs of a test, rising above a diagonal chance line. It is used to compare tests and to choose a cut-off.
Axis | Plots | Also called |
|---|---|---|
Vertical | Sensitivity | True positive rate |
Horizontal | 1 minus specificity | False positive rate |
The curve runs from the bottom left (0,0), where nobody tests positive, to the top right (1,1), where everybody does.
The diagonal is the chance line: a test on it gives positive results equally often in cases and non-cases.
The top-left corner is a perfect test, with sensitivity and specificity both 1.
To find the specificity (true negative rate) at a cut-off, subtract the x-axis value from 1: an x value of 0.19 means a specificity of 0.81.
Both axes are worked out separately within cases and within non-cases, so reading the curve needs no prevalence.
Moving along the curve
Moving up and to the right means a lower threshold for a positive screen: more people test positive, so sensitivity rises and specificity falls.
The cut-off plotted furthest right therefore has the lowest specificity.
In one example, 500 inpatients completed an alcohol questionnaire scored 0 to 40 and a reference interview, which found 100 cases and 400 non-cases.
Positive if score is at least | Sensitivity | Specificity | 1 minus specificity |
|---|---|---|---|
4 | 0.98 | 0.45 | 0.55 |
8 | 0.88 | 0.81 | 0.19 |
15 | 0.52 | 0.96 | 0.04 |
20 | 0.28 | 0.99 | 0.01 |
Area under the curve
The area under the curve (AUC) summarises overall diagnostic accuracy: how well the test separates people with and without the condition across all thresholds.
It equals the probability that a randomly chosen case scores higher than a randomly chosen non-case.
The AUC is independent of disease frequency.
The AUC measures discrimination, not calibration, and it does not give the positive predictive value.
AUC | Meaning |
|---|---|
Below 0.5 | Worse than chance |
0.5 | Chance: the curve lies on the diagonal and the test cannot discriminate |
0.5 to 0.6 | Fail |
0.6 to 0.7 | Poor |
0.7 to 0.8 | Fair |
0.8 to 0.9 | Good |
0.9 or more | Excellent |
1.0 | Perfect |
When several tests share one plot:
The curve that bows closest to the top-left corner has the largest AUC and discriminates best.
The curve that runs closest to the diagonal has the smallest AUC, a little above 0.5, and discriminates least well.
Choosing a cut-off
The optimal cut-off is the point nearest the top-left corner, where sensitivity plus specificity is greatest, not where either one peaks.
Youden's index is sensitivity plus specificity minus 1; in the example it peaks at a cut-off of 8 (0.88 plus 0.81 minus 1 gives 0.69).
Youden's index weights sensitivity and specificity equally, so the clinical purpose of the test can move the chosen cut-off.
Favour sensitivity when a positive result leads to something cheap, such as a brief alcohol intervention, and favour specificity when it leads to invasive tests, harmful treatment, a stigmatising label or detention.