Week 6: Linear Regression

Question 1

What is the purpose of linear regression?

Answer 1

To describe the relationship between a continuous outcome and one or more predictors

Question 2

What does the slope (β1) in a regression equation represent?

Answer 2

The change in the outcome variable (y) for each unit increase in the predictor variable (x)

Question 3

What does the intercept (β0) in a regression equation represent?

Answer 3

The value of the outcome variable (y) when the predictor variable (x) is zero

Question 4

Write the general equation for simple linear regression

Answer 4

y = β0 + β1x
The equation of the straight line that best describes how the outcome (y) increases/decreases with the exposure (x)

Question 5

How are regression parameters (β0, β1) estimated?

Answer 5

The line is fitted with the shortest distance between points and line. Distances are called residuals
By minimising the sum of squared residuals, the differences between observed and predicted values

Question 6

What is H0 and H1 in testing the slope (β1)?

Answer 6

β1 = 0 (no relationship between x and y)

Question 7

How is the t-statistic for the slope calculated?

Answer 7

t = β1 / SE of β1 - where SE is the standard error of the slope

Question 8

What does a significant p-value for β1 indicate?

Answer 8

Evidence against H0, suggesting an association between x and y

Question 9

What is a 95% CI for β1?

Answer 9

Small sample sizes: β1 +- t* x SE, where t* is the critical value from the t-distribution
Large sample sizes: typically use 5% point of normal distribution (1.96) instead of t-distribution
The CI details the values between which the slope of the line could lie

Question 10

If a 95% CI for β1 does not include 0, what does it mean?

Answer 10

It indicates a statistically significant relationship between x and y

Question 11

How do you predict y for a given x?

Answer 11

Use the regression equation 𝑦^ = β0 + β1x

Question 12

Why should predictions not extrapolate beyond the range of x?

Answer 12

The relationship may not be linear outside the observed data range

Question 13

Why might β0 (intercept) not always be meaningful?

Answer 13

If x = 0 is outside the observed data range, β0 may not provide useful information

Question 14

How are binary categorical variables incorporated into regression?

Answer 14

By coding them as 0 and 1, representing the two categories

Question 15

What does β1 represent in a regression with a binary predictor?

Answer 15

The difference in the outcome between the two categories

Question 16

What is the purpose of multiple linear regression?

Answer 16

To examine the effect of multiple predictors on the outcome simultaneously

Question 17

How do you write the equation for multiple regression with two predictors?

Answer 17

y = β0 + β1x1 + β2x2, where x1 and x2 are predictors

Question 18

What is the adjusted R2 in a regression?

Answer 18

A measure of model fit that adjusts for the number of predictors

Question 19

What is centring a predictor variable?

Answer 19

Subtracting the mean of x from each value to create a new variable x_cent = x - x̄
The intercept would then be at the mean outcome

Question 20

Why centre a predictor variable?

Answer 20

To make the intercept (β0) interpretable as the value of y at the mean of x
E.g., it is not meaningful to predict y based on a body weight of 0

Question 21

Name key assumptions of linear regression

Answer 21

1. Linearity of relationship
2. Homoscedasticity (constant variance of residuals)
3. Normality of residuals
4. Independence of observations

Question 22

On a scatter plot, which variable goes on which axis?

Answer 22

Outcome on y-axis, exposure on x-axis

Question 23

What do the different regression lines represent?

Answer 23

• = no association
  / = positive association
  \ = negative association

Question 24

What does _cons represent in a regression output?

Answer 24

β0 (intercept)

Question 25

How would you calculate y if the coefficient for β0 (intercept) is 0.0857 and the coefficient for β1 (slope) is 0.0436? Assume x = 60
What would you conclude about the association between x and y?

Answer 25

y = 0.0857 + 0.0436 x 60 = 2.70
Per one unit increase in x results in a 0.0436 litre increase in y

Question 26

What are β0 and β1 subject to?

Answer 26

Sampling variation (β0 and β1 are estimates of population values of the intercepts and slopes)

Question 27

How is precision measured?

Answer 27

Standard errors (CIs)

Question 28

Outline the logic behind fitting a linear regression model if x is a categorical variable using the example of lung function (FEV) with sex (binary) as predictor:

Answer 28

1. FEV1 = β0 + β1(sex)
2. Ascertain values for each category (females = 0, males = 1)
3. Estimate mean FEV by sex
4. If sex = 0, FEV1 = β0 + β1 x 0
5. If sex = 1, FEV1 = β0 + β1 x 1
6. Fit regression model
7. Coefficient for _cons = mean FEV for girls = 1.54
8. For boys, mean FEV1 = 1.54 + 0.12 x 1 = 1.66
9. Boys have a larger mean FEV than girls (see mean difference and associated p value)
   Note: We would get the same results if we did an unpaired t-test