Hierarchical linear regression and model comparison

Question 1

Define

Hierarchical multiple regression

Answer 1

a special form of regression analysis in which more variables are added to the model in separate steps called “blocks.” This is often done to statistically “control” for certain variables, to see whether adding variables significantly improves a model’s ability to predict the criterion variable and/or to investigate a moderating effect of a variable

Question 2

Define

Nested models

Answer 2

refers to models where one model contains all the terms of the other, and at least one additional term

Question 3

Define

Non-nested models

Answer 3

models where neither can be obtained from the other by the imposition of appropriate parametric restrictions or as a limit of a suitable approximation

Question 4

Define

Covariates

Answer 4

characteristics (excluding the actual treatment) of the participants in an experiment

Question 5

Define

Log likelihood (LL)

Answer 5

measures the goodness of fit of a statistical model to a sample of data for given values of the unknown parameters

Question 6

Define

Akaike Information Criterion (AIC)

Answer 6

an estimator of in-sample prediction error and thereby relative quality of statistical models for a given set of data

Question 7

Define

Bayesian Information Criterion (BIC)

Answer 7

a criterion for model selection among a finite set of models

Question 8

Definition

a special form of regression analysis in which more variables are added to the model in separate steps called “blocks.” This is often done to statistically “control” for certain variables, to see whether adding variables significantly improves a model’s ability to predict the criterion variable and/or to investigate a moderating effect of a variable

Answer 8

Hierarchical multiple regression

Question 9

Definition

refers to models where one model contains all the terms of the other, and at least one additional term

Answer 9

Nested models

Question 10

Definition

models where neither can be obtained from the other by the imposition of appropriate parametric restrictions or as a limit of a suitable approximation

Answer 10

Non-nested models

Question 11

Definition

characteristics (excluding the actual treatment) of the participants in an experiment

Answer 11

Covariates

Question 12

Definition

measures the goodness of fit of a statistical model to a sample of data for given values of the unknown parameters

Answer 12

Log likelihood (LL)

Question 13

Definition

an estimator of in-sample prediction error and thereby relative quality of statistical models for a given set of data

Answer 13

Akaike Information Criterion (AIC)

Question 14

Definition

a criterion for model selection among a finite set of models

Answer 14

Bayesian Information Criterion (BIC)

Question 15

What is hierarchical multiple regression useful for?

Answer 15

Comparing the difference in R2 will show how much the predictor(s)/covariates uniquely contribute beyond the covariates/predictor(s)

Question 16

In hierarchical multiple regression, which outputs are the most useful? What do they show?

Answer 16

R-square change:

• R square of current model minus the R square of the previous model. Higher values indicate that the model explains more variance than the previous model

F change:

• Higher values indicate that the previous variables + additional variables have a greater effect size than the previous variables alone

Sig. F change:

• p < .05 indicates that the F change is significantly different to previous model

Question 17

What is the differene between nested and non-nested models?

Answer 17

Nested models are models where one includes all the variables of another model

This is not true for non-nested models

Question 18

True or False:

You can only run hierarchical regression on non-nested models if a large sample size was used

Answer 18

False

You can’t run hierarchical regression on non-nested models

Question 19

When comparing models, what do we use to determine which is better?

Answer 19

Log likelihood (LL)

Question 20

In linear models, the negative log likelihood is often the ______________

Answer 20

In linear models, the negative log likelihood is often the sum of squared deviations

Question 21

Most models work by __________the LL

Answer 21

Most models work by maximising the LL

Question 22

What question does log likelihood answer?

Answer 22

How likely is it that we would observe these data given some parameter estimate?

For example:

1. How likely is it that we would observe a score of 6, if the mean is 5?
2. How likely is it that we would observe a score of 10, if the mean is 5?

Question 23

What do we use to compare non-nested models?

Answer 23

Akaike Information Criterion (AIC)

Bayesian Information Criterion (BIC)

Question 24

What do AIC and BIC both measure?

Answer 24

Provides a penalty for more complexs model based on how many parameters it includes

Question 25

True or False:

A higher AIC/BIC is always better

Answer 25

False

Lower scores = better, so always pick the model with the lowest AIC/BIC