Tutorial 1: Introduction & Linear Regression Model Flashcards
How does a OLS model (for all I = 1,…,N observations) look like?
y = X β + ε where….
- ….y = vertical vector = (y₁ … yₙ)’ = N x 1
- ….X = matrix of observations = N x K
- ….β = vertical vector = (β₁ … βₖ) = K x 1
- ….ε = vertical vector = (ε₁ … εₙ)’ = N x 1
What are the OLS residuals?
e = ^ε = y − ŷ = y − X β̂
Sum of squared residuals?
Σeᵢ² = e’e = (y - Xβ̂)’(y - Xβ̂)
What’s the OLS estimator β̂ₒₗₛ?
arg min [β̂] (y - Xβ̂)’(y - Xβ̂) = (X’X)⁻¹X’y
What is the OLS Estimator (conceptually)?
OLS estimator minimizes the sum of squared residuals
What is K?
of parameters = # rows in β = # columns in X
What is N?
sample size = # rows of X = # rows of Y = # rows of ε
What is the relationship between K and N in order to be able to estimate β? What if that is not the case?
k =< N
Otherwise, other estimators like lasso or ridge must
be used.
How can you interpret a coefficient?
how the linear prediction of Y changes if we increase variable x by one unit, holding the other variables fixed.
What is the variance?
Variance (σ²):
measurement of the spread between numbers in a data set -> it measures how far each number in the set is from the mean and therefore from every other number in the set
How does the variance of β̂ₒₗₛ look like (in matrix form)?
K × K variance-covariance matrix (K= number of parameters in the model):
How can β̂ₒₗₛ be rewritten in terms of β?
How can the variance of β̂ₒₗₛ be simplified?
What is the variance of the error term in a heteroskedastic OLS model?
Variance of the error term is NOT constant across i , depends on xᵢ:
What is the variance of the error term in a homoskedastic OLS model?
Variance of the error term is constant for all i , does not depend on xᵢ :
What condition do we assume to hold for both homoskedastic and heteroskedastic models?
We always assume that error terms of two observations are not correlated:
What are the consequences of heteroskedasticity?
Consequences of heteroskedasticity:
- if the other OLS assumptions are fulfilled, ˆβ is still unbiased + consistent
- ˆβ is not efficient (not BLUE any more), others like GLS will be better (= more efficient)
- non-robust estimator Var(ˆβ|x) is not consistent estimator of Var(β|x) any more
How can the variance of ˆβ be simplified with homoskedasticity?
What is a consistent estimator of the variance of ˆβ under homoskedasticity?
What are solutions to heteroskedasticity?
- Heteroskedasticity-robust standard errors: only ˆβ will then still not be the most efficient estimator
- GLS (= OLS on transformed data)
How does heteroskedasticity-robust variance look like?
can not be simplied further. The non-robust standard errors will be inconsistent!
What is the sandwhich estimator?
- proposed by White (1980)
- based on squared OLS residuals eᵢ²
- consistent under all forms of heteroskedasticity (also homoskedasticty!)
- = “hetereoskedasticity-robust standard errors”’
What does it mean for an estimator to be consistent?
An estimator is consistent if, as the sample size increases, the estimates (produced by the estimator) “converge” to the true value of the parameter being estimated. To be slightly more precise - consistency means that, as the sample size increases, the sampling distribution of the estimator becomes increasingly concentrated at the true parameter value.
What does it mean for an estimator to be unbiased?
An estimator is unbiased if, on average, it hits the true parameter value. That is, the mean of the sampling distribution of the estimator is equal to the true parameter value
How can you conduct graphical analysis of heteroskedasticity?
Graphical analysis:
- Plot standardized/studentized residuals against explanatory variables
- In a multivariate model, also plot the standardized / studentized residuals against fitted values
How can you detect heteroskedasticity?
Graphical analysis or White test
What does the White test test?
What does a hypothesis test for a single hypothesis test?
How do you conduct the White test?
- Take the squared residuals from a usual OLS regression eᵢ².
- Regress eᵢ² on all linear, squared and interaction terms of all covariates
- Calculate the test statistic = N * R² taking the R² from the auxiliary regression
- Under H₀, the test statistic is asymptotically Chi-square distributed with q degrees of freedom. q= number of regressors in the auxiliary regression minus 1
What is the t-statistic for a hypothesis test for a single hypothesis and how is it distributed?
the t-statistic is asymptotically t-distributed with N − K degrees of freedom
(N= number of observations, K= number of model parameters incl. constant)
What does a hypothesis test for a single hypothesis on multiple parameters test?
What is the t-statistic for a hypothesis test for a single hypothesis with multiple parameters and how is it distributed?
What does a hypothesis test for multiple hypotheses on test?
where R is a J × K-matrix of restrictions, beta the K × 1-vector of parameters, and r is a J × 1-vector:
What is the test statistic for a hypothesis test for multiple hypotheses? (where R is a J × K-matrix of restrictions, the K × 1-vector of parameters, and r is a J × 1-vector.)
Asymptotically F-distributed with J, N − K degrees of
freedom. V^ar(beta^) can be the robust or non-robust estimator for the variance-covariance matrix of beta^ .
