Tutorial 1: Introduction & Linear Regression Model

Question 1

How does a OLS model (for all I = 1,…,N observations) look like?

Answer 1

y = X β + ε where….

• ….y = vertical vector = (y₁ … yₙ)’ = N x 1
• ….X = matrix of observations = N x K
• ….β = vertical vector = (β₁ … βₖ) = K x 1
• ….ε = vertical vector = (ε₁ … εₙ)’ = N x 1

Question 2

What are the OLS residuals?

Answer 2

e = ^ε = y − ŷ = y − X β̂

Question 3

Sum of squared residuals?

Answer 3

Σeᵢ² = e’e = (y - Xβ̂)’(y - Xβ̂)

Question 4

What’s the OLS estimator β̂ₒₗₛ?

Answer 4

arg min [β̂] (y - Xβ̂)’(y - Xβ̂) = (X’X)⁻¹X’y

Question 5

What is the OLS Estimator (conceptually)?

Answer 5

OLS estimator minimizes the sum of squared residuals

Question 6

What is K?

Answer 6

of parameters = # rows in β = # columns in X

Question 7

What is N?

Answer 7

sample size = # rows of X = # rows of Y = # rows of ε

Question 8

What is the relationship between K and N in order to be able to estimate β? What if that is not the case?

Answer 8

k =< N

Otherwise, other estimators like lasso or ridge must
be used.

Question 9

How can you interpret a coefficient?

Answer 9

how the linear prediction of Y changes if we increase variable x by one unit, holding the other variables fixed.

Question 10

What is the variance?

Answer 10

Variance (σ²):

measurement of the spread between numbers in a data set -> it measures how far each number in the set is from the mean and therefore from every other number in the set

Question 11

How does the variance of β̂ₒₗₛ look like (in matrix form)?

Answer 11

K × K variance-covariance matrix (K= number of parameters in the model):

Question 12

How can β̂ₒₗₛ be rewritten in terms of β?

Answer 12

Question 13

How can the variance of β̂ₒₗₛ be simplified?

Answer 13

Question 14

What is the variance of the error term in a heteroskedastic OLS model?

Answer 14

Variance of the error term is NOT constant across i , depends on xᵢ:

Question 15

What is the variance of the error term in a homoskedastic OLS model?

Answer 15

Variance of the error term is constant for all i , does not depend on xᵢ :

Question 16

What condition do we assume to hold for both homoskedastic and heteroskedastic models?

Answer 16

We always assume that error terms of two observations are not correlated:

Question 17

What are the consequences of heteroskedasticity?

Answer 17

Consequences of heteroskedasticity:

1. if the other OLS assumptions are fulfilled, ˆβ is still unbiased + consistent
2. ˆβ is not efficient (not BLUE any more), others like GLS will be better (= more efficient)
3. non-robust estimator Var(ˆβ|x) is not consistent estimator of Var(β|x) any more

Question 18

How can the variance of ˆβ be simplified with homoskedasticity?

Answer 18

Question 19

What is a consistent estimator of the variance of ˆβ under homoskedasticity?

Answer 19

Question 20

What are solutions to heteroskedasticity?

Answer 20

1. Heteroskedasticity-robust standard errors: only ˆβ will then still not be the most efficient estimator
2. GLS (= OLS on transformed data)

Question 21

How does heteroskedasticity-robust variance look like?

Answer 21

can not be simplied further. The non-robust standard errors will be inconsistent!

Question 22

What is the sandwhich estimator?

Answer 22

• proposed by White (1980)
• based on squared OLS residuals eᵢ²
• consistent under all forms of heteroskedasticity (also homoskedasticty!)
• = “hetereoskedasticity-robust standard errors”’

Question 23

What does it mean for an estimator to be consistent?

Answer 23

An estimator is consistent if, as the sample size increases, the estimates (produced by the estimator) “converge” to the true value of the parameter being estimated. To be slightly more precise - consistency means that, as the sample size increases, the sampling distribution of the estimator becomes increasingly concentrated at the true parameter value.

Question 24

What does it mean for an estimator to be unbiased?

Answer 24

An estimator is unbiased if, on average, it hits the true parameter value. That is, the mean of the sampling distribution of the estimator is equal to the true parameter value

Question 25

How can you conduct graphical analysis of heteroskedasticity?

Answer 25

Graphical analysis:

• Plot standardized/studentized residuals against explanatory variables
• In a multivariate model, also plot the standardized / studentized residuals against fitted values

Question 26

How can you detect heteroskedasticity?

Answer 26

Graphical analysis or White test

Question 27

What does the White test test?

Answer 27

Question 28

What does a hypothesis test for a single hypothesis test?

Answer 28

Question 29

How do you conduct the White test?

Answer 29

1. Take the squared residuals from a usual OLS regression eᵢ².
2. Regress eᵢ² on all linear, squared and interaction terms of all covariates
3. Calculate the test statistic = N * R² taking the R² from the auxiliary regression
4. Under H₀, the test statistic is asymptotically Chi-square distributed with q degrees of freedom. q= number of regressors in the auxiliary regression minus 1

Question 30

What is the t-statistic for a hypothesis test for a single hypothesis and how is it distributed?

Answer 30

the t-statistic is asymptotically t-distributed with N − K degrees of freedom

(N= number of observations, K= number of model parameters incl. constant)

Question 31

What does a hypothesis test for a single hypothesis on multiple parameters test?

Answer 31

Question 32

What is the t-statistic for a hypothesis test for a single hypothesis with multiple parameters and how is it distributed?

Answer 32

Question 33

What does a hypothesis test for multiple hypotheses on test?

Answer 33

where R is a J × K-matrix of restrictions, beta the K × 1-vector of parameters, and r is a J × 1-vector:

Question 34

What is the test statistic for a hypothesis test for multiple hypotheses? (where R is a J × K-matrix of restrictions, the K × 1-vector of parameters, and r is a J × 1-vector.)

Answer 34

Asymptotically F-distributed with J, N − K degrees of
freedom. V^ar(beta^) can be the robust or non-robust estimator for the variance-covariance matrix of beta^ .