F13 Multilevel modeling I

Question 1

What is a hierarchical model?

Answer 1

A model with a nested datastructure.

Multilevel models are also called hierarchical linear models (HLM), mixed-effects models, or random-effects models.

Question 2

What two kinds of data structures have multiple relevant levels? Draw them

Answer 2

(1) Nested data structures. One level is contained within another (e.g. students in universities).

(2) Non-nested data structures. Different levels make sense in themselves and are independent from each other (e.g. airports and treatments)

Question 3

What is a key advantage of multilevel models compared to fixed effects

Answer 3

FE is power intensive and problematic with many groups and few observations pr. group - not many degrees of freedom + model is prone to outliers.

Question 4

What does multilevel models account for regarding the effect estimate that standard linear regression doesn’t

Answer 4

Possible heterogeneity in effect size. The
assumption of homogeneous effect magnitudes is often problematic. Very plausible in the real world.

Question 5

What is random effects?

Answer 5

A multilevel model. Opposite of fixed effects. Not the best formulation as the randomness isn’t clear.

Question 6

What is a random effects model?

Answer 6

If group-specific intercepts are modeled via a multilevel approach

Question 7

What is a random slope model?

Answer 7

If group-specific slopes are modeled via a multilevel approach

Question 8

What is a mixed or mixed effects model?

Answer 8

A model that combines both fixed effects and random effects.

Question 9

What are the two primary components of a multilevel model?

Answer 9

Units (i) and groups (j).

Question 10

What three combinations are possible with multilevel models and what is the regression line?

Answer 10

Varying:
Intercepts: y_i = α_(j[i]) + βx_i + ε_i

Slope: y_i = α + βx_(j[i]) + ε_i

Both: y_i = α_(j[i]) + βx_(j[i]) + ε_i

Question 11

What is the key difference between fixed and random effects?

Answer 11

We introduce the assumption that intercepts or slopes are normally distributed in random effects: α_j ~ Normal (μ_α , σ_α^2)

Question 12

What are hyperparameters?

Answer 12

Parameters that are not actually in the regression model but estimated ‘behind’ the model. Instead of estimating 100 unique intercepts we estimate two hyperparameters with random effects, which is way more efficient.

Question 13

What is an important assumption for random effects?

Answer 13

The coefficients from the multilevel estimate are normally distributed.

Question 14

Draw the intercepts from random effects, fixed effects and pooled regression.

Answer 14

Normal distribution, line with intercepts and one global intercept.

Question 15

How is observations pooled in random effects, fixed effects and pooled regression?

Answer 15

RE: Partial pooling
FE: No pooling
Regression: Complete pooling

Question 16

What is the simpel multilevel estimate? Explain parameters.

Answer 16

A weighted average between the specific average for group j and the overall average for all units.

(n_j/σ_y^2)y-bar_j + (1/σ_α^2)y-bar_all

divided by

(n_j/σ_y^2)+(1/σ_α^2)

(y_j): Average of group j
(y_all): Average of all units

σ_y^2: Unit specific error. The error term / residual. Unexplained variance Var(ε)
σ_α^2: Variance in intercept (hyperparameter)

Question 17

What happens with the multilevel estimate when there are many/few observations per group?

Answer 17

Many: The average of the group j will weigh more (outliers are less influential) relative to the overall average.

Few: We rely heavily on the overall average - the distinction between FE and RE is very clear here.

Question 18

What happens with the multilevel estimate when the residual of the model is big/small?

Answer 18

Big: There is a lot of variation that the model cannot explain (often because of outliers) - we would rather rely on the overall average and thus a larger set of observations.

Small: The model is very precise - we would rather rely on the group average

Question 19

What happens with the multilevel estimate when the variance of the random intercepts are big/small?

Answer 19

Big: Groups are not very comparable - we are less confident in weighting in the overall average.

Small: Groups are very similar and a lot of the intercepts are clustered around the mean. We can rely more on the overall average.

Question 20

When does the group j average weigh in more in the multilevel estimate?

Answer 20

With a lot of observations in the group
When model is precise (low residual)
When the groups are different (high variance in group intercept)

Question 21

When does the overall average weigh in more in the multilevel estimate?

Answer 21

With few observations in the groups
When a lot of variation in the model is unexplained (high residual)
When the groups are similar (low variance in group intercept)

Question 22

What is the variance ratio?

Answer 22

The relation between variation between and within groups

(σ_α^2) - between group / (σ_y^2 ) - within group

The higher the ratio the more relevant the model. If the ratio is only 0,04 then it would take more than 25 observations (1/0,04) to be pooled closer to the no-pooling estimate.

Question 23

How can the variance ratio be useful?

Answer 23

This ratio helps to assess how much of the variation in the outcome is explained by the grouping structure versus individual-level variation

Question 24

What is the intraclass correlation?

Answer 24

Bound between 0-1. 0=no between group variation. All groups have the same intercept. 1=no residual. Within a group units doesn’t differ. Everything is explained by the group

σ_α^2 / (σ_α^2 + σ_y^2)

Question 25

What are three advantages of multilevel modeling?

Answer 25

If I have very few observations within the group, I’m prone to mistakes with fixed effects. I multilevel I can rely on other groups to make the estimate (not just the specific group).

We need less statistical power because less power to groups specific intercepts and more pow-er to estimate the coefficient of interest.

More realistic in the real-world. There are probably differences between groups, but they probably vary around a specific mean. A lot of things in this world is normally distributed. Some units will gain more from a treatment and some less.

Question 26

When does multilevel modeling make sense?

Answer 26

A lot of groups with few observations.

Question 27

When does multilevel modeling NOT make sense?

Answer 27

Few groups with many observations

Question 28

Does multilevel modeling make sense with a lot of observations and groups?

Answer 28

Jan: Edge case. Apply both and see