13.10.10 Linear, multiple and logistic regression

Regression and correlation

• Correlation measures only the strength and direction of a linear association between two variables.

• Regression fits a best-fit line so that the value of an independent variable predicts the size of a dependent variable.

• The dependent (outcome) variable is the quantity being predicted, and it sits on its own on the left-hand side of the equation.

• A regression equation is worth building only when predictor and outcome are significantly correlated, positively or negatively. Causation, temporal order and a positive direction are not required.

Simple linear regression

The fitted equation y = a + bx predicts the mean of the dependent variable for a chosen value of the independent variable. It predicts only in the direction it was fitted, so it cannot be turned round to predict x from y.

Worked example: predicted depression score at 12 weeks = 12.4 + (0.18 x age in years).

• The slope (b) is the change in predicted outcome per unit of predictor: 0.18 points per year, or 1.8 points across ten years.

• The intercept (a) is the predicted outcome when the predictor is zero, which means little here because age zero lies outside the observed range.

• Centring age at the sample mean of 41 makes the intercept the predicted score at age 41 without changing the slope.

R squared

• R squared, the coefficient of determination, is the proportion of variance in the dependent variable accounted for by the regression model. It is the goodness-of-fit statistic.

• A low R squared does not rule out an important association.

Multiple linear regression

Multiple regression models a single dependent variable from two or more independent variables, and it is a parametric method.

• The outcome must be continuous, but the predictors need not be continuous or normally distributed.

• The model assumes linearity, independent residuals, constant residual variance and normally distributed residuals. It does not assume normally distributed predictors.

• Each coefficient gives the association with the outcome while the other predictors are held constant: each untreated month adds 0.55 points after adjustment.

• An adjusted coefficient is not a causal effect: harmful alcohol use carrying 3.10 extra points does not mean stopping alcohol lowers a person's score by 3.10.

• A standardised beta coefficient gives the direction of association for its predictor in standard deviation units, adjusted for the others. It is not a proportion of variance explained.

• Two predictors with significant coefficients of opposite sign are associated with the outcome in opposite directions, holding the other predictors constant.

• A categorical predictor with k categories enters as k - 1 indicators compared with a reference group, so five ethnic groups need four.

• Clustered data, such as patients sharing wards, need multilevel methods, because ignoring the dependence understates uncertainty.

Multicollinearity and choosing predictors

• Multicollinearity occurs when two highly correlated predictors, such as weight and body mass index, are entered together.

• The model cannot split their effects, so standard errors inflate and the individual coefficients lose significance or become unstable.

• Variance inflation factors help detect it.

Significance-based selection cannot tell confounders, mediators and colliders apart.

Choosing the model by outcome

Logistic regression

• Logistic regression models the log odds of a binary outcome: log(p / (1 - p)) = b0 + (b1 x x1) + ... + (bk x xk).

• Its sigmoid curve keeps predicted probabilities between 0 and 1, whereas a straight line can predict impossible values such as 1.3 or -0.2.

• Exponentiated coefficients are adjusted odds ratios: a coefficient of 0 gives an odds ratio of 1, a positive one gives above 1 and a negative one below 1.

• For example, a long-acting injectable rather than oral antipsychotic has a coefficient of -0.36, which is an odds ratio of 0.70 for readmission.

• The overall significance of the covariates together is tested with a likelihood ratio chi-square test of the model coefficients.

• An odds ratio overstates the risk ratio when the outcome is common: an odds ratio of 2.5 at a baseline risk of 40% means 62.5% in the exposed (risk ratio 1.56), not 100%.

Tap to enlarge

Confounders, mediators and colliders

Tap to enlarge

Hospital admission can be a collider when both cannabis use and psychotic symptoms lead to admission, so studying only inpatients distorts their association.

Tap to enlarge

Mediation analysis

• Mediation analysis separates the direct effect of an exposure on an outcome, with the mediator held constant, from its indirect effect through the mediator. The two can have opposite signs.

• In the Baron and Kenny causal steps approach, mediation requires the predictor-outcome association to weaken once the mediator is added, so the direct effect c' is smaller than the total effect c.

Path analysis

• Path analysis extends multiple regression to model hypothesised causal chains among several variables, allowing more than one dependent variable and both direct and indirect effects.

• Path coefficients, the values on the arrows of a path diagram, are standardised partial regression coefficients adjusted for the other arrows entering the same variable.

Factor analysis and principal component analysis

• Factor analysis has no dependent variable. It accounts for the correlations among many observed variables through a few latent dimensions.

• Liddle applied it to symptom ratings in chronic schizophrenia and derived psychomotor poverty, disorganisation and reality distortion.

• In principal component analysis, the first component's share of variance is its eigenvalue divided by the sum of all eigenvalues, which equals the number of standardised variables.

• A first-component share of about 90% or more shows that the variables largely measure one shared construct.