13 Research methods, statistics, critical review and evidence-based practice13.15 Artificial intelligence in research
13.15 Artificial intelligence in research
2 min read1 figure
Written by Resident Doctor Committee · Published
New in the 2027 syllabus
The College added this topic for exams from January 2027.
What the models do
Artificial intelligence (AI) covers computer methods that carry out tasks such as prediction, classification, language processing and pattern recognition.
Machine learning estimates patterns from data. It does not rely only on rules that a programmer writes one by one.
Approach | What it learns from | Example research task |
|---|---|---|
Supervised learning | Labelled outcomes | Predicting later readmission |
Unsupervised learning | Structure in the data, with no outcome labels | Grouping similar records |
Generative models | Patterns that let it produce new outputs | Drafting text or proposing synthetic examples |
Prediction is not causal explanation. A variable can predict an outcome without being a useful treatment target.
Building and testing a model
Dataset | What it is for |
|---|---|
Training | Estimating the model's parameters |
Validation or tuning | Choosing model settings or comparing candidate models |
Held-out testing | Estimating performance once all choices are fixed |
External validation | Testing the fixed model in a different time, place or population |
Cross-validation repeatedly separates fitting from evaluation within the development data. Tuning settings may need an inner loop as well, or the performance estimate will be too optimistic.
Patient-level separation stops records from the same person ending up on both sides of a split, which would be inappropriate.
Data leakage happens when the model is built using information not available at the time of prediction. Examples are future outcomes, or test-data information used during preprocessing.
Overfitting happens when a model learns the noise in the development data. It then looks better on that data than on new observations.
Judging performance and bias
Discrimination is how well the model ranks or separates outcomes. Calibration is how closely the predicted probabilities match the frequencies actually observed.
Accuracy can look high for a rare outcome even when the model misses every case. So check sensitivity, specificity and predictive values too.
Subgroup performance can differ because of under-representation, biased labels, missing data or different clinical pathways.
Dataset shift happens when the population or the measurement process changes. Performance may then fall after the model is put into use.
Clinical usefulness depends on what happens when people act on the predictions. Statistical performance alone does not show this, so it needs its own evaluation.
Reporting and accountability
TRIPOD+AI guides transparent reporting of studies that develop and evaluate prediction models. Following it does not certify that a model is clinically safe.
Reproducibility needs clear records of data provenance (where the data came from), exclusions, preprocessing, model choices, uncertainty and evaluation methods.
Generative outputs can contain invented references and errors that sound plausible. Researchers must check factual claims and citations against the original sources.
Confidential information should be used only within approved data-handling arrangements. Consider what the model can access and what it retains.
Human accountability stays with researchers and organisations for study design, interpretation, reporting and the consequences of use.