13 Research methods, statistics, critical review and evidence-based practice13.5 Diagnostic studies
13.5.8 Validity of a measurement
3 min read
Written by Resident Doctor Committee · Published
Validity and reliability
Validity is whether an instrument captures the construct it was built to capture, whereas reliability is how consistently it measures.
Reliability is necessary but not sufficient for validity. A measure must be consistent before it can be valid, but consistency alone does not show that it measures the intended construct.
Face and content validity
Face validity is whether a test appears, at a glance, to measure what it claims, judged for example by prospective respondents. It helps acceptability but is the weakest form of validity.
Content validity is the systematic check that the items cover the intended domains, reviewed by experts and patients. The Hospital Anxiety and Depression Scale leaves out most somatic symptoms because they overlap with physical illness.
Shortening a rating scale trades some face validity for brevity and ease of use, because whole content areas may drop out. The short form can keep or improve its construct validity and its correlation with the full form.
Criterion validity
Criterion validity compares the measure with an accepted external reference.
Type | Compared with | How it is shown |
|---|---|---|
Concurrent | An established, validated measure of the same construct | Both are taken at the same time |
Predictive | A theoretically related outcome that occurs later | Scores are correlated with an outcome measured at a later date |
Criterion validity is only as good as its reference. Many psychiatric diagnoses have no biological gold standard, so clinical diagnosis or a structured interview serves as an imperfect one.
Construct validity
Construct validity is established by an accumulation of evidence that a measure behaves as the theory of its construct predicts. No single component establishes it alone.
Evidence | What it requires | Example |
|---|---|---|
Convergent | Strong correlation between different measures, or methods, of the same construct or trait | A negative symptom scale against the Scale for the Assessment of Negative Symptoms or the Positive and Negative Syndrome Scale negative subscale |
Discriminant (divergent) | Weak or near-zero correlation with a distinct or theoretically unrelated construct | Negative symptoms against depression |
Known groups | The expected differences between groups | Cognitive scores in dementia against controls |
Predicted relationships | Relations that fit the theory | Dependence scores that track withdrawal |
Convergent validity is the core of Campbell and Fiske's multitrait-multimethod approach.
Removing items that fit poorly with the theorised structure of a scale, found by Rasch analysis or confirmatory factor analysis, improves its construct validity.
Validity depends on setting
Validity holds for a particular population and use. A depression questionnaire that works in primary care may behave differently in dementia, and a screening tool is not automatically fit for diagnosis.
Differential item functioning is when people at the same level of the construct endorse an item at different rates across groups, so translated instruments and their thresholds need validating.
Validity of a study
Internal validity is confidence in the study's own causal conclusion.
Regression towards the mean, the drift of extreme scores back towards the average, can masquerade as a treatment effect and so threatens internal validity.
External validity is the generalisability of findings to other people, places and times. An unrepresentative sample is the chief threat to it.
Types of rating scale
Scale | How it works | Point to remember |
|---|---|---|
Visual analogue | A continuous unmarked line, such as 100 mm, between two anchors | Sensitive to small changes and usable without words, so suits poor readers, young children and adults with learning disability |
Thurstone (equal-appearing intervals) | A panel of judges sorts statements into piles, fixing each statement's weight in advance | The weight is the median pile value |
Guttman | Cumulative: passing a harder item implies passing every easier one | The total reproduces the pattern, and the coefficient of reproducibility tests the fit |
Likert-type | Ordered points, each defined in words, as in the Clinical Global Impression improvement item | Fixed verbal anchors make ratings comparable across raters and trial arms |
Named scales
The Warwick-Edinburgh Mental Wellbeing Scale (WEMWBS) measures positive mental wellbeing across the general population, with positively worded items and no ceiling on scores. It is not a tool to screen for, exclude or grade mental disorder.