13.5.2 Diagnostic test sensitivity, specificity and likelihood ratios

The 2x2 table

A diagnostic study compares an index test with a reference standard, often a structured diagnostic interview in psychiatry, in the kind of patients the test will be used on.

In the worked example below, 200 primary care patients complete a depression questionnaire and a diagnostic interview.

• Sensitivity, specificity and the false positive and false negative rates are read down the disease-status columns.

• Predictive values are read across the test-result rows.

• Accuracy and prevalence use the whole table.

Sensitivity and the false negative rate

• Sensitivity is the probability that a person who truly has the disorder tests positive: TP/(TP+FN) = 36/40 = 90%. True disease status fixes it.

• A sensitivity of 100% means there are no false negatives, so everyone with the condition tests positive. It says nothing about false positives.

• The false negative rate is 1 minus sensitivity, FN/(FN+TP) = 4/40 = 10%.

• The chance that a true case tests negative on every one of several independent tests is the false negative rate raised to the number of tests.

• When two tests share the same specificity, the one with higher sensitivity detects more true cases and misses fewer.

Specificity and the false positive rate

• Specificity is the proportion of people truly without the condition whom the test classifies as negative: TN/(TN+FP) = 128/160 = 80%. It holds whatever the prevalence.

• The false positive rate is 1 minus specificity, FP/(FP+TN) = 32/160 = 20%, read from the same disease-free column. It forms the x-axis of a receiver operating characteristic (ROC) curve.

• Two tests given to the same sample can differ in direction, one more sensitive and the other more specific, so both measures are calculated for each test.

Predictive values

• PPV is the chance that a positive result is right, and equals the post-test probability after a positive result.

• The denominator of NPV is all negative results, not everyone without the condition.

• The chance of having the condition despite a negative result, FN/(FN+TN), is 1 minus the NPV.

• Raising sensitivity, with specificity and prevalence held constant, raises both predictive values. NPV rises because missed cases leave the negative group, and PPV rises because extra true positives join an unchanged pool of false positives.

• Sensitivity is the probability of a positive result given disease, whereas PPV is the probability of disease given a positive result. Swapping them is the inverse fallacy.

• When disease is uncommon, even a small false positive rate produces many false positives, because non-cases greatly outnumber cases.

Tap to enlarge

Prevalence and accuracy

• Pre-test probability, before any result is known, is the prevalence in the patient's group: those with the condition divided by the whole sample (40/200 = 20%).

• Diagnostic accuracy is the proportion of everyone tested whom the test classifies correctly: (TP+TN)/total = 164/200 = 82%. It is a prevalence-weighted average of sensitivity and specificity.

Likelihood ratios

• LR+ is the true positive rate over the false positive rate, and it multiplies the pre-test odds after a positive result.

• An LR- well below 1 makes the condition much less likely after a negative result.

• Predictive values are probabilities that depend on prevalence, whereas likelihood ratios are multipliers of odds built from sensitivity and specificity.

• Likelihood ratios are not calculated from sample prevalence, but they can still vary across populations and disease spectra.

From pre-test to post-test probability

1. Pre-test odds = p/(1 - p) = 0.20/0.80 = 0.25.

2. After a positive result, odds = 0.25 x 4.5 = 1.125, so probability = 1.125/2.125, about 53%.

3. After a negative result, odds = 0.25 x 0.125 = 0.03125, so probability is about 3%.

Tap to enlarge

The positive screen therefore calls for further assessment rather than confirming depression.

Cut-offs

Sensitivity and specificity apply only at a stated cut-off.

• The lowest cut-off on a screening scale classifies the most people as positive, so it gives the highest sensitivity at the cost of specificity.

• On an ordinal score, sensitivity at a cut-off is found by cumulatively grouping everyone at or above the cut-off as test positive.

• Moving a cut-off changes sensitivity and specificity only through people who score at the value being crossed, so one measure can stay fixed while the other changes.

• When a condition is serious but curable if found early, and a confirmatory test follows, the screening cut-off is set low enough to catch every case, accepting more false positives.

Tap to enlarge

Combining tests

Screening programmes

Wilson and Jungner's principles (World Health Organization, 1968) include these requirements.

• The natural history of the condition, including its progression from latent or early stage to declared disease, must be adequately understood.

• There must be a recognisable latent or early symptomatic stage, a detectable pre-symptomatic window.

• The test must be acceptable to the population screened.

• There must be an accepted treatment for cases found.

The principles reject screening only the already symptomatic, treating a screening score as a diagnosis, one-off screening and ignoring cost.