introduction to multilevel modelling

Question 1

what are different names for multilevel modelling?

Answer 1

Hierarchical linear models, mixed models, random effects models and variance components models.

Question 2

what assumption is made by simple multiple regression that would not hold true if we had hierarchial data.

Answer 2

It assumes the data is independent, specifically that the residuals are uncorrelated with one another. If the data is in actual effect grouped and we haven’t taken this into account then this assumption will not hold.

Question 3

How can we account for grouping effects in data? What are the problems of each

Answer 3

1. include dummy variables for the groups
2. Include explanatory variables that measure group characteristics believed to influence outcomes in the model as additional predictors. Effectively control for their effects

For example, let’s say you are studying academic performance in students across different schools. You suspect that the school itself may have an effect on individual student performance. In this case, you can include variables that measure school characteristics (e.g., school funding, student-teacher ratio, school size) as additional predictors in your model.

Question 4

what happens if we don’t take into account the cluster effect in the model

Answer 4

The standard errors of the regression coefficients will be underestimated. Consequently, confidence intervals will be too narrow and p values will be too small. risks Type 1 error.

Question 5

if there is a clustered group effect in the analyses (e.g., n from same school having similar scores to n of diff schools) and is not accounted for. Which coefficinnts will this more severly impact?

Answer 5

Underestimation of standard errors is particularly severe for coefficients of predictors that are measured at the group level, eg, an indicator of whether a school is mixed or single sex

Question 6

if you are interested in only in controlling for clustering rather than exploring its effects what methods can we use vs if you are interested in exploring it.

Answer 6

If interested in exploring it – multilevel modelling is needed. Gives us correct standard errors.
If not and just treat the clustering as a nuisance for which to control – then 2 ways:
* Methods to adjust standard errors for design effects
* model the dependency between n in same group explicitly using marginal model

Question 7

Multilevel modelling enables researchers to investigate the nature of between-group variability, and the effects of group-level characteristics on individual outcomes. Is this true?

Answer 7

yes

Question 8

what is a model with dummy variables for groups called

Answer 8

fixed effects model

Question 9

hierarhcial data. What are the consequences of: Including a set of dummy variables for groups (a fixed effects model)

limitation of this?

Answer 9

Group is treated as a fixed classification, so the target of inference is restricted to the groups represented in the sample. I

if the number of groups is large, there will be a large number of additional parameters to estimate

Question 10

hierarhcial data. What are the consequences of: Fitting a single-level model with group-level predictors

Answer 10

High risk of Type I errors because standard errors of coefficients of group-level predictors may be severely underestimated. No estimate of the between-group variance that remains unaccounted for by the included group-level predictors.

Question 11

hierarhcial data. What are the consequences of: Correcting standard errors for design effects, or fitting a marginal model in which the dependency is modelled directly

Answer 11

The standard errors will be correct (properly adjusted for clustering), but unable to assess the degree of between-group variation.

Question 12

hierarhcial data. What are the consequences of: Multilevel modelling (random effects)

Answer 12

Correct standard errors and an estimate of between-group variance

Question 13

what is a typical two level model

Answer 13

Yij = b0 + b1x1ij + Uj + eij

Where,

Uj = N(0, sigma2, U)

EIJ = N(sigma2 , e)

Question 14

write the regression equation for
> simple regression model

multilevel data :
> fixed effects model (null model)
> random intercepts (null model
> random slope model

Answer 14

Simple: Yi = B0 +B1x1 + Ei

fixed: Yij = B0 + UJ + Eij

R intercepts: Yij = B0 + Uj + EiJ (Uj and Eij normally distributed)

R slope: Yij = B0 + b1xIJ + U0j + U1jx1ij + e0ij

Question 15

limitations of fixed effects approach?

Answer 15

When number of groups is large, there will be many extra parameters to estimate. (Only one in ML model.)
For groups with small sample sizes, the estimated group effects may be unreliable. (In a ML model residual estimates for such groups ‘shrunken’ towards zero.)

Question 16

what makes it a multilevel model

Answer 16

soon as you include predictor of the difference in mean from the overall average, to the group specific average

Question 17

what does the random intercept model help you see that teh simple regression model dowsnt

Answer 17

between-group effects

Question 18

Write out multilevel model for group means

Answer 18

YIJ = B0 + B1X1i + UJ + Eij

*Yij is the response for person I in group j.
*B0 – the mean score for all n across groups
*Uj – the difference between overall mean to the group specific mean
*Eij – the residual, the difference between the group mean (B0 + Uj) and the persons individual score (Yij)

Question 19

what does sigma2 U reflect in a multilevel model

Answer 19

Between group variance
- based on difference between group mean from overall mean

Question 20

what does sigma2 e reflect in a multilevel model

Answer 20

Within group, between individual variance
- Individual differences from group mean

Question 21

fixed vs random part of the multilevel regression equation

Answer 21

Fixed part
-Specifies the relationship between the mean of Y and exploratory variables
-B0 + B1XIJ
-Fixed part parameters: B0 +B1

Random part
-Contains the level 1 and 2 residuals
-Random part : UJ + eIJ
-Random part parameters: sigma2e sigma2u

Question 22

by including a grouping variable (UJ) how have we changed the structure of the data?

Answer 22

Now 2-level structure with individual (level 1) nested within group (level 2)

Question 23

variance partition coefficient (VPC)

Answer 23

measures the proportion of total variance that is due to differences between groups:

Question 24

measures the proportion of total variance that is due to differences between groups:

Answer 24

The variance at a certain level e.g., group divided by total variance (between group + within group variance)

Question 25

for simple multilevel models, what is the VPC equal to?

Answer 25

The intra-class correlation coefficient.

if the VPC is 0.2, for example, we would say that 20% of the variation is between groups and 80% within. The correlation between randomly chosen pairs of individuals belonging to the same group is 0.2.

Question 26

interpret a VPC of 0.4?

Answer 26

40% of the variation is between groups and 60% within.

Question 27

Comparing 2 models single level model vs multilevel model. How can we test the null hypothesis – that there are no group level differences?

Answer 27

Likelihood ratio test statistic is got from:
LR = -2 log L1 – (-2 log L2)
where L1 and L2 are the likelihood values of the single-level and multilevel models respectively and ‘log’ refers to the natural logarithm

Question 28

what is the LR test statistic compared against? How many degrees of freedom do we have?

Answer 28

Chi-squared distribution with the number of degrees of freedom equal to the number of extra parameters in the more complex model.
Have 1 additional parameter in this example (between group variance) so there is 1 degree of freedom.

Question 29

what is the difference between testing for group effects and estimating group effects (multilevel modelling)

Answer 29

Testing for group effects refers to determining whether there is evidence of systematic differences between groups. e.g., likelihood ratio test.

Estimating group effects, on the other hand, involves quantifying the magnitude and direction of the effects of the group-level predictors on the outcome variable

Question 30

what are the two ways we can estimate group effects?

Answer 30

1. Random effects approach
2. fixed effects approach

Question 31

what is the difference between random and fixed effects approach

Answer 31

fixed effects includes dummy coded variables for each group

random effects approach includes single parameter representing group variance

main difference is the group variables in fixed approach are treat as fixed parameter while in random effects they are treat as having a normal distribution summarized by a variance parameter

Question 32

how many residuals are there in the single vs multilevel model

Answer 32

In a single level model there is a single set of residuals. The difference between n score and the predicted score of the model.

In multilevel model, we have a 2 error terms. Total variance constructed by uj + eij (between + within group variance). How can we get the estimated residual for each?

Question 33

how do we calcualte a level 2 residual

Answer 33

Step 1: get the mean raw residual
step 2: multiply the mean raw residual by shrinkage factor

Question 34

what is the shrinkage factor?

Answer 34

• Between group variance
• Divided by between group variance + (within group variance / sample size in group)

Question 35

do we have the same shrinkage factor for every group e.g., uk, germany and france?

Answer 35

no

Different shrinkage factor for each group

Question 36

how do we get the level 1 residual

Answer 36

• Observed value (Yij)
• Minus the predicted value (BO+B1X1IJ)
• Minus level 2 residual

Question 37

How does the group sample size affect the shrinkage

Answer 37

smaller sample size = more shrinkage

Question 38

How does the within group variance affect the shrinkage

Answer 38

large variance = lot of shrinkage

Question 39

How does the between group variance affect the shrinkage

Answer 39

small variance = lot of shrinkage

Question 40

how do we present level 2 residuals

Answer 40

can use a caterpillar plot e.g., showing the level 2 residual and CI for each coutnry

Question 41

can the shrinkage factor be larger than 1

Answer 41

No the shrinkage facotor is always equal to or less than 1.

So when multiplying this by the mean raw residual the estimated residual is either equal to or less than the MRR.

Question 42

Why are shrinkage residuals also regarded precision-weighted estimates

Answer 42

These shrinkage residuals are also called precision-weighted estimates because we have taken their reliability into account in their estimation. Unreliable estimates with, for example, small nj will be shrunk towards the overall mean. Reliable estimates with a large nj will keep close to their raw mean value.

Question 43

what is the fixed effects approach to looking for group differences called?

Answer 43

Analysis of variance

Question 44

from the single level regresion equation

how many extra parameters do we have in a fixed vs random effects model measuring outcome score for 20 coutnries?

Answer 44

Fixed effects – 19 additional parameters (dummy coded for each)

Random effects – just one, the between group variance

Question 45

why is a random effects approach better if each group varied in sample size

Answer 45

*Fixed effects – generated group effects might be unreliable. Reliability of residual estimate for each group not taken into account
*Random effects - recognizes that there is little information for these groups by ‘shrinking’ their residual estimates towards zero, and therefore pulling their mean towards the overall mean

Question 46

What does it mean in terms of target of inference if fixed classifications are used vs random classifications?

Answer 46

If we are wanting to make conclusions about the specific thing chosen, e.g., treatment A vs treatment B, then variables are treat as fixed. We want to make inferences about those specific things.
In contrast if we are sampling from a wider population (e.g., schools) and the target of inference is on this wider population then random classifications are used.
Fixed effects approach prohibits generalisations made to groups beyond our sample. Random effects approach, the between group variance, is interpreted as the between group difference in the wider population.

Question 47

extend this random intercept model with no predoctors

to having 1 explanatory varibale

Yij = B0 + Uj + EJ

Answer 47

Yij = B0 + Uj + B1X1ij+ EiJ

Question 48

Yij = B0 + Uj + B1X1ij+ EiJ

random in tercept model

what is the overall relationship between x and y? and what is the intercept for group J

Answer 48

• Overall relationship between x and y represented by line: B0 + B1XIJ
• Intercept (start) for n in group J: B0 +UJ

Question 49

what is the random intercept model

Answer 49

Any model where the intercept of the group regression lines are allowed to take on different values from a distribution. the slope B1 is assumed to be the same for each group

Question 50

multilevel regression with single variable: what exactly is the intercept and slope for a dichotomos explanatory variable e.g., gender (coded as either 0 or 1)

Answer 50

B0 is the overall mean of Y for individuals with x=0.

BO +UJ = is the mean for individuals with x=1 in group J

the slope B1 is the difference in the mean for x=1 relative to x=0 (in any group).

Question 51

Q: what does the intercept of a model represent in single vs multilevel regression?

Answer 51

In a linear regression model, the intercept represents the value of the dependent variable when all independent variables are set to zero. It is the value of the dependent variable when there is no contribution from any of the independent variables.

However, in a random intercept model, intercept marks the DV for a group when all the independent variables are set to zero. It captures the average or typical value of the dependent variable for that particular group.

Question 52

why does ignoring clustering produce underestimated standard errors?

Answer 52

Because the analysis is conducted with the assumption that you have 5000 independent observations, all producing very similar results increasing precision.
In contrast you only have 100 independent observation so the similarity isnt as strong

Question 53

what is the number of independent observations called?

Answer 53

Effective sample size (ESS)

Question 54

in multilevel modelling, what does the effective sample size depend on

Answer 54

The degree of clustering (measured by the intra-class correlation or variance parition coefficient