Multiple linear regression

Question 1

Define

Unstandardised coefficient

Answer 1

represents the expected change that one unit change of the IV will have on the DV. Hard to interpret magnitude

Question 2

Define

Standardised coefficient (β)

Answer 2

represents the expected standard deviation change in the DV, for a standard deviation change in the predictor

Question 3

Define

Adjusted R^2

Answer 3

gives an estimate of R2 in the population taking into account the fact that the regression model might overfit your particular dataset

Question 4

Define

Sampling error

Answer 4

the discrepancy between sample and population

Question 5

Define

R^2 shrinkage

Answer 5

failure to replicate the R^2 on subsequent regression models

Question 6

Define

Cohen’s f2

Answer 6

a measure of effect size based on R^2; tell you the unique effect of one predictor variable

Question 7

Define

Homoscedasticity

Answer 7

all residuals in the model have equal variances

Question 8

Define

Heteroscedasticity

Answer 8

all residuals in the model have unequal variances

Question 9

Define

Multiple regression

Answer 9

a type of regression that is used to predict values of an outcome from several predictors

Question 10

Define

Multicollinearity

Answer 10

the presence of high intercorrelations between predictor variables

Question 11

Define

F-ratio

Answer 11

The overall significance of the regression equation; a significant value indicates that the equation predicts a significant proportion of the variability in the Y scores (i.e., more than would be expected by chance alone)

Question 12

Define

Dummy coding

Answer 12

the process of coding a categorical variable into dichotomous variables

Question 13

Define

Semi-partial (part) correlation

Answer 13

Measures the relationship between two variables, controlling for the effect that a third variable has on one of the others

Question 14

Definition

represents the expected change that one unit change of the IV will have on the DV. Hard to interpret magnitude

Answer 14

Unstandardised coefficient

Question 15

Definition

represents the expected standard deviation change in the DV, for a standard deviation change in the predictor

Answer 15

Standardised coefficient (β)

Question 16

Definition

gives an estimate of R2 in the population taking into account the fact that the regression model might overfit your particular dataset

Answer 16

Adjusted R^2

Question 17

Definition

the discrepancy between sample and population

Answer 17

Sampling error

Question 18

Definition

failure to replicate the R^2 on subsequent regression models

Answer 18

R^2 shrinkage

Question 19

Definition

a measure of effect size based on R^2; tell you the unique effect of one predictor variable

Answer 19

Cohen’s f2

Question 20

Definition

all residuals in the model have equal variances

Answer 20

Homoscedasticity

Question 21

Definition

all residuals in the model have unequal variances

Answer 21

Heteroscedasticity

Question 22

Definition

a type of regression that is used to predict values of an outcome from several predictors

Answer 22

Multiple regression

Question 23

Definition

the presence of high intercorrelations between predictor variables

Answer 23

Multicollinearity

Question 24

Definition

The overall significance of the regression equation; a significant value indicates that the equation predicts a significant proportion of the variability in the Y scores (i.e., more than would be expected by chance alone)

Answer 24

F-ratio

Question 25

# Definition the process of coding a categorical variable into dichotomous variables

Answer 25

Dummy coding

Question 26

# Definition Measures the relationship between two variables, controlling for the effect that a third variable has on one of the others

Answer 26

Semi-partial (part) correlation

Question 27

What does the unstandardised coefficient of this relationship mean?

Answer 27

-1.584 indicates that a 1 unit change in self-esteem is predicted to be associated with a 1.584 decrease in loneliness

Question 28

Why is it hard to interpret the magnitude of the effect unstandardised coefficients?

Answer 28

Coefficients depend on the scale of the predictors and the DVs If self-esteem ranges from 0 to 2, a 1 unit change is a huge difference; if the range is 0 to 100, a 1 unit change is very small

Question 29

What does the standardised beta coefficient tell you about this relationship?

Answer 29

In the current regression, β = -.257, meaning that a 1 SD unit increase in self-esteem is expected to predict a .257 decrease in loneliness

Question 30

Imagine that the variance of the DV is 5, and after calculating the regression, the variance of the residuals is 4. What is the model explained variance?

Answer 30

5 - 4 = 1

Question 31

What is the formula for R²? Imagine that the variance of the DV is 5, and after calculating the regression, the variance of the residuals is 4. What is R²?

Answer 31

R² = MEV/Total variance Model explained variance (MEV) = 5 - 4 = 1 Total variance = 5 R² = 1 / 5 = 0.2 = 20%

Question 32

What is considered a small, medium and large R²?

Answer 32

There are no fixed rules for what a small, medium, or large R2 is, but many people consider .04 (4%) as small, .09 (9%) as medium, and .25 (25%) as large

Question 33

What is an adjusted R²? What does it consider?

Answer 33

Gives an estimate of R² in the population. It takes into account sample size, the number of predictors, etc…

Question 34

Sampling error increases as sample size \_\_\_\_\_\_\_\_\_and as the number of predictors \_\_\_\_\_\_\_\_\_\_\_

Answer 34

Sampling error increases as sample size **decreases** and as the number of predictors **increase**

Question 35

Why is R² likely to overestimate the size of the effect in the population?

Answer 35

Sampling error increases as sample size decreases and as the number of predictors increase

Question 36

Shrinkage is best evaluated using a \_\_\_\_\_\_\_\_\_\_\_\_\_\_

Answer 36

Shrinkage is best evaluated using a **cross validation study**

Question 37

What is it called when a regression model does not work with a new data set?

Answer 37

Shrinkage

Question 38

What happens if there is a large discrepancy between R² and adjusted R²?

Answer 38

it indicates the regression model does not generalise well to the population

Question 39

When is Cohen's f2 useful?

Answer 39

Most useful when using a variant for multiple regression

Question 40

Age accounted for 10% of the variance in loneliness on its own Age + self-esteem combined accounted for 15% of variance in loneliness What is the f2 for self-esteem?

Answer 40

Difference between these two is self-esteem unique variance (15% - 10% = 5%) f2 for the unique effect of self-esteem would be: 𝑓2 = (.15 − .10) / (1 − .15) = .05 / .85 = .059

Question 41

What assumptions must be met to run a linear regression?

Answer 41

* Outcome variable must be continuous; predictors can be continuous or dichotomous * Predictors must not have zero variance * Independence * Linearity * Independent errors * Normally-distributed errors * Homoscedasticity

Question 42

What type of plot can we use to look for non-linearity in regression? What indicates that the assumption has been met?

Answer 42

Partial regression plot (i.e. residual and predicted) The absence of clear pattern, means assumption has been met

Question 43

How do we deal with non-linear data in regression?

Question 44

What does the independent errors assumption mean?

Answer 44

For any 2 observations, the errors or residual terms should be independent (i.e., uncorrelated) with one another Observations should be independent and there should not be any systematic relationship amongst the residuals

Question 45

What does the normally-distributed errors assumption mean?

Answer 45

Residuals for the regression model should be random and normally distributed, with mean = 0 Note: this does not mean the predictors have to be normally distributed – predictors do not need to be normally distributed (although, it does improve the chances of this assumption being met)

Question 46

What assumption does this scatterplot violate?

Answer 46

Homoscedasticity

Question 47

What should you report from a linear regression?

Answer 47

1. How you assessed the assumptions and whether they appear violated 2. Any attempts to address violated assumptions 3. Whether these attempts resolves assumption issues or if they were still violated 4. Descriptive statistics (e.g. mean and SD/median and IQR/frequency and percent) 5. Standardised coefficient, effect size, R², confidence intervals and p-value

Question 48

How is the mulitple regression equation different to the linear regression one?

Answer 48

Add b_kX_k, where k = number of predictor variable included In multiple regression b_k captures the unique relationship, conditional (adjusted) for all other predictors in the model

Question 49

Which variable is the best predictor?

Answer 49

The β value with the largest magnitude (ignoring the +/- signs) is the best predictor Thus, self-esteem is the best predictor of loneliness – indeed, it is the only significant predictor

Question 50

What does this R² of a multiple regression model tell you? What does it mean for this example?

Answer 50

R Square (R2 ) tells you the proportion of variability in the outcome variable that is accounted for by the predictor variables In our example, 7% of the variability in loneliness is accounted for by self-esteem and number of exercise days (when considered together)

Question 51

What statistics do we use to measure multicollinearity?

Answer 51

**Tolerance:** predictor variances with tolerances \< .10 are multicollinear with 1+ other predictors, which is concerning (i.e., you have a multicollinearity problem) **VIF:** VIF = 1/tolerance; predictor variables with VIF \> 10 are multicollinear with 1+ other predictors, which is concerning (i.e., you have a multicollinearity problem)

Question 52

What extra assumption does multiple regression have that linear regression doesn't?

Answer 52

Multicollinearity

Question 53

What should you do if you have an issue with multicollinearity?

Answer 53

Consider deleting one of the offending variables 2 Combine the variables with high intercorrelations into a single measure

Question 54

What does the semi-partial correlations tell us in this example?

Answer 54

The variability in loneliness uniquely accounted for by self-esteem = (-.261)2 \* 100 = 6.81% The variability in loneliness uniquely accounted for by exercise = (.058)2 \* 100 = 0.34%

Question 55

The overall significance of the regression equation can be evaluated by computing an \_\_\_\_\_\_\_\_

Answer 55

The overall significance of the regression equation can be evaluated by computing an **F-ratio**

Question 56

What does a significant F-ratio indicate?

Answer 56

A significant F-ration indicates that the equation predicts a significant proportion of the variability in the Y scores (i.e., more than would be expected by chance alone)