Linear & Multiple Regression

Question 1

Who is touted as the ‘Father of Behavioural Statistics’?

Answer 1

Francis Galton

Inspiringly, he is also known for not being very great at traditional statistics and had a severe breakdown when studying it at Cambridge.

Question 2

Describe:

The relationship between regression models and variance.

Answer 2

Regression models involve predictor variables that account for at least some of the variance seen in the outcome variable.

Variance can be thought of the extent to which values differ from the mean.

Regressions are a predictive tool for this.

Question 3

When trying to identify a question suited for a regression model, what key word should you look out for?

(In most cases)

Answer 3

Predict(s)

(e.g. “what predicts romantic interest on a date?”)

Question 4

Give the general equation for a regression model.

Answer 4

y’ = a + bx + e

y’ : outcome variable (what you are predicting).

a : the intercept (the mean value of y’ when all predictors are zero).

bx : predictor variable(s) (variables intended to explain variance seen in y’).

e : error (accounting for the fact no model is perfect).

Question 5

Why, in a regression model, is the intercept the value of the mean of the outcome variable?

Answer 5

When studying normally distributed populations, assuming a value will be close or equal to the mean should be correct more often due to chance alone.

The predictor variables should then ideally improve the accuracy of the prediction and account for variance in the outcome.

Question 6

List:

The FOUR assumptions of a regression model.

Answer 6

1. Independence of measurements.
2. Normally distribution of variables.
3. Linearity of predictor-outcome relationship.
4. Homoscedasticity of residuals.

Homoscedasticity refers to whether or not the residuals (error of the model) are random or not. There shouldn’t be a ‘systematic error pattern’.

Note this is what was tested in PSYC232, but some sources seem to contradict it.

Question 7

Explain:

Residual plots should display random variance.

And how does this relate to regressions?

Answer 7

If a residual plot (indicator of error) showed a non-random pattern, this would mean the predictor variables are NOT sufficiently predicting or explaining the variance in the outcome variable.

It likely means there are stronger predictors not being considered.

Question 8

What is the purpose of the model coefficient box in a (multiple) regression output?

Answer 8

It provides important information on which exact predictor variables improve accuracy in predicting the outcome.

Question 9

What is the purpose of the model fit box in a (multiple) regression output.

Answer 9

It gives holistic information on how well all of the variables in the model predict the outcome.

This is particularly denoted by ‘R’.

Question 10

What does R² represent in a (multiple) regression output?

Answer 10

The amount of variance in the outcome explained by the regression model.

This is an overall assessment, and does not tell you which variables contribute greater accuracy to others.

Having around or over 50% explained is considered impressive when studying human behaviour!

Question 11

How does the adjusted R² differ from R² in a multiple regression output?

Answer 11

It accounts for the number of predictors used in the regression model.

Having a larger number of predictors may lead to an increased chance of accidentally being more accurate, and so the adjusted R² reduces its value based on these factors.

Question 12

When would the adjusted R² value be the exact same as the R² value?

(In a regression model output)

Answer 12

When there is only one predictor variable used in the model.

Question 13

What is the general equation for calculating error in a (multiple) regression output?

Answer 13

1 - R²

In words, this means it accounts for the remaining variance not explained by the regression model.

Around 10 - 30% error is considered relatively normal for behavioural research.

Question 14

What is the format for presenting the overall model test from a regression output?

Answer 14

F (df₁ , df₂) = F , p-value

Note, the second F refers to the test statistic from the results.

Question 15

What does having a statistically significant regression model imply?

Answer 15

That, on average, the model is more accurate at predicitng the true value of a measurement compared to simply guessing the mean of the outcome variable.

Question 16

What does the estimate (B) of a model coefficient box in a multiple regression output represent?

Answer 16

The amount of change in the value of the outcome associated with one unit of change in the predictor variable.

Note that this is the unstandardised estimate.

It may also be thought of as the effect size.

Question 17

What is the purpose of a standardised estimate (β) of the model coefficient box in a multiple regression output represent?

Answer 17

It allows us to compare predictors’ effect sizes when they use different scales.

Question 18

What is ONE major benefit of using a multiple regression over just a linear regression?

Answer 18

Multiple regressions allow you to control for other predictor variables when analysing the change in the outcome associated with the focus predictor variable.

(e.g. analysing perceived attractiveness, controlling for gender - as the specific study found women were less attracted on average than the men were).

Question 19

True or False:

You can include categorical variables in a regression.

(e.g. gender).

Answer 19

True

However, you can only have two groups and you have to numerically label each category so there is a one unit difference between them (i.e. 0 and 1, or 1 and 2, etc.).

Question 20

Define:

Regression toward the Mean

Answer 20

A statistical phenomenon brought about by the fact extreme values tend to be followed by values that are closer to the mean.

This occurs when measuring something more than once, and can give the false impression of a trend occurring.

It makes sense, as an extreme value is typically less common, and so is unlikely to be followed by yet another extreme value.

An example of this may be seen with the so-called ‘2nd Album Syndrome’, which leads people to believe doing really well with your first album ‘causes’ you to do worse in your next. This is not the case, it is just unlikely to have another extreme outlier in ratings.

Many things in life tend to stabilise over time.

Question 21

What factors may lead to an increased pattern of regression toward the mean?

Answer 21

• The more extreme a sample/measurement is compared to the mean, the more noticeable the regression toward the mean.
• The less correlated variables are too.