Week 9: Assumptions of Multivariable Linear Regression

Question 1

What is the outcome of linear regression?

Answer 1

Outcome is always continuous

Question 2

What types of variables can the explanatory variables be in linear regression?

Answer 2

Continuous or categorical

Question 3

How is a continuous explanatory variable interpreted?

Answer 3

As the explanatory variable increases by one unit, the outcome changes by the value of the coefficient

Question 4

How is a categorical explanatory variable interpreted?

Answer 4

The outcome changes by the coefficient’s value for the category of interest

Question 5

What are the assumptions of linear regression?

Answer 5

1. Normality of residuals
2. Linear relationship between outcome and explanatory variables
3. Constant variance (SD) of the outcome over x
4. Data independence

Question 6

What is a residual in regression?

Answer 6

The difference between the predicted value and the observed value of the outcome

Question 7

How can residuals be checked for normality?

Answer 7

By plotting kernel density (kdensity resid, normal) or using pnorm and qnorm plots

Question 8

What indicates heteroskedasticity in residual plots?

Answer 8

A fan or funnel shape in residuals vs. fitted values indicates unequal variance

Question 9

How do you check the linearity and equal variance assumptions in regression?

Answer 9

Plot residuals against fitted values; there should be no clear pattern or funnel shape

Question 10

What should you do if the linearity assumption is violated?

Answer 10

Consider adding a quadratic term or categorising the explanatory variable

Question 11

How can you very the independence assumption?

Answer 11

Ensure outcome data come from different individuals at one time point

Question 12

What can you do if multiple assumptions are violated (e.g., non-normality, non-linearity, heteroskedasticity)?

Answer 12

Transform the outcome variable to address all issues, but interpretation becomes more complex

Question 13

What transformations are commonly used for improving normality?

Answer 13

Logarithmic, square root, inverse, or power transformations

Question 14

What is a limitation of logarithmic transformation?

Answer 14

It cannot be used with variables that contain zero, unless a small constant (e.g., 0.1) is added

Question 15

How do transformations affect regression analysis?

Answer 15

They change the scale of coefficients and standard errors, leading to different results

Question 16

What is the interpretation of coefficients after a log transformation of the outcome variable?

Answer 16

Coefficients approximate percentage changes for a one-unit increase in the explanatory variable

Question 17

What are outliers in regression analysis?

Answer 17

Extreme values of the outcome variable with large residuals (positive or negative)

Question 18

What is leverage in regression analysis?

Answer 18

Observations with extreme values of the explanatory variable that can influence regression coefficients

Question 19

How can leverage points affect regression results?

Answer 19

They can pull the regression line toward them, distorting the fit

Question 20

What is collinearity in regression?

Answer 20

When explanatory variables are highly correlated, making it difficult to include both in the model

Question 21

What should you do if collinearity is present?

Answer 21

Choose only one correlated variable to include in the model

Question 22

Why is it better to keep a continuous outcome variable as it is?

Answer 22

It allows more explanatory variables to be included in the model and improves statistical power

Question 23

What is a rule of thumb for including explanatory variables?

Answer 23

One explanatory variable for every 10 observations; categorical variables count as the number of categories minus one (e.g., a variable with 4 categories counts as 3 variables)

Question 24

What are common methods for model building?

Answer 24

Include all variables, manual backwards selection, automated forward selection, backward selection, or stepwise selection

Question 25

What is manual backwards selection?

Answer 25

Start with all variables, remove the least significant, and refit the model until all remaining variables meet the significance threshold

Question 26

What are automated methods of model selection?

Answer 26

Backward selection, forward selection, and stepwise selection using predefined criteria for variable inclusion

Question 27

What are potential issues with automated model selection?

Answer 27

Different methods may lead to different final models, and some variable categories may be excluded

Question 28

What pattern should residuals follow in a residual vs fitted plot?

Answer 28

Residuals should sit evenly around 0 with no clear pattern

Question 29

What does a residual vs fitted plot reveal about non-linearity?

Answer 29

Residuals being positive for moderate fitted values and negative for small/large values indicates a curve in the relationship

Question 30

What happens to the assumptions as the dataset gets larger?

Answer 30

The assumptions become less critical

Question 31

What are limitations of dichotomising a continuous variable?

Answer 31

It reduces statistical power and limits the ability to include explanatory variables

Question 32

What is heteroskedasticity?

Answer 32

Unequal variance of residuals across values of the explanatory variable

Question 33

How do you identify outliers and leverage points?

Answer 33

Use scatter plots, stem and leaf plots, or examine residual and qnorm plots

Question 34

How can we get the residuals for each person in Stata?

Answer 34

<predict chosen_varname, resid>

Question 35

What’s the difference between pnorm and qnorm plots?

Answer 35

• pnorm is sensitive to non-normality in the middle range of the data (where it is likely there will be a lot of data)
• qnorm is sensitive to non-normality at the extremities of the data (where it is likely there will be less data). Non-normal residuals will show on this plot

Question 36

How do we check for normality on a residual vs fitted plot?

Answer 36

Residuals should sit evenly about 0
If they are densest elsewhere, this indicates that the residuals are not normal

Question 37

What should you always do when checking the linearity assumption?

Answer 37

Plot a scatter plot between outcome and explanatory variable(s) if continuous

Question 38

If we need to transform variables, what should be transformed first?

Answer 38

The outcome, then if necessary, the explanatory
If you are including a baseline value of an outcome as an explanatory, it makes sense for them to be on the same scale so transform them both

Question 39

On what kind of data do logarithmic transformations work best?

Answer 39

Right (positive) skewed data

Question 40

How can we address outliers and leverage points?

Answer 40

1. First check the data source and whether data have been entered correctly
2. If data are entered correctly, do not exclude data without good reason. You may want to analyse the data with and without the outlier(s) and/or leverage points to see how the models compare

Question 41

How do you compile an analysis plan?

Answer 41

1. Define research question(s) - check data you have will be able to answer your questions. Define H0 and H1
2. Check data - see how variables are distributed (histograms, check for outliers, tabulations)
3. Do analyses by an important categorical variable (if appropriate)
4. Consider whether you need to do regression analysis - decide which variables to include and check for multicollinearity
5. Model building
6. Make sure assumptions for regression analysis have been met

Question 42

What analytical methods can be used for two categorical variables?

Answer 42

Chi-square if assumptions met; Fischer’s exact test if assumptions not met

Question 43

What analytical methods can be used for a categorical and a continuous variable?

Answer 43

• Two samples t-test if two categories and assumptions met
• One way ANOVA is > 2 categories and assumptions met
• Non-parametric equivalents if assumptions not met

Question 44

What is forwards selection?

Answer 44

Starting with no variables in the model and adding variables one by one using a pre-specified criteria

Question 45

what is stepwise selection?

Answer 45

using a combination of forwards and backwards selection with pre-specified criteria for including and excluding variables