3- Classical linear regression Flashcards
What does the error term (u) capture in a regression?
All the effects on the dependent variable not in the explanatory variables
What are the betas in a regression?
The gradients associated with each of the theoretical parameters
What is always the first independent variable in a regression?
A constant β₀, it represents the y-intercept i.e. value of dependent variable when there’s no input
What is the population regression line?
The sum of the independent variables; the relationship that holds between x and y on average (without error term) y = β₀ + Xβ₁
Once x and y values are collected how is a line of best fit determined?
Minimising the sum of each error term squared
What is the error term for each observation plotted?
The vertical distance between the population regression line
What do the independent variable subscripts denote (xᵢₖ)?
The first letter is the number of observations and the second letter is the number of parameters
What are the 6 assumptions of the linear regression model?
-Linearity
-Identification condition
-Exogeneity
-Homoskedasticity
-Normality
-X can be fixed or random
Explain the linearity assumption
Linearity in parameters means the betas have index 1 so are not exponential
Explain the identification condition
Number of observations must be at least as great as the number of parameters
Explain the exogeneity assumption
Expected value of any u on X is zero E(uᵢ|X)=0 meaning no observation conveys any information about u
Explain the homoskedasticity assumption
Variance of the error term is constant across observations
Explain the normality assumption
Error terms are normally distributed
What are the 3 main properties of the OLS estimator?
-Unbiased
-Given variance covariance matrix
-Estimator is the best in that it has minimum variance
What does it mean that the OLS estimator is unbiased?
The expected value of all estimated betas will give their true value E(^βols)= β
What are the 2 steps to show the OLS estimator is unbiased?
- Sub in y to the OLS estimator expression and expand
- Take expectations, and show only β remains
What is the variance matrix of the OLS estimator var(βₒₗₛ)?
var(βₒₗₛ)=σ²(X’X)⁻¹
When a function is homoscedastic, what is E(uu’)?
σ²I
How can you prove the variance of the OLS estimator is var(βₒₗₛ)=σ²(X’X)⁻¹?
- Use equation var(B̂)=E[(B̂ - E(B̂))(B̂ - E(B̂))]
- Sub in OLS estimate and expectation and expand
What is the OLS estimate of variance?
E(û’û)/n-k
What is the Gauss-Markov theorem?
Unbiased linear estimators is the “Best”, in the sense that it has the minimum variance
What are the 3 conditions for the Gauss-Markov theorem?
- y=Xβ+u
- var(u) = σ²I
- X is full rank
What is the identity P?
X(X’X)⁻¹X’
What is the identity M?
I - P
What are 3 main properties of M?
-Square
-Symmetric
-Idempotent
How can you show X is uncorrelated to u?
E[(X’X)⁻¹X’u] = 0
What is the variance of a given error term var(uᵢ)?
E(u²) = σ²
If u has a multivariate normal distribution with zero mean, what is the distribution of uᵢ?
Univariate normal distribution
What happens to the estimated coefficients (β) when X is scaled up or down?
Coefficients are scaled in the opposite direction to compensate
What does the Beta variance covariance matrix look like?
Beta variances on principal diagonal and covariances in rest of elements
What is a trick to find the n?
n is the first element of X’X
What expression is always the error term estimate û?
û = (I - X(X’X)⁻¹X’)u
What is tr(E(û’û))?
nσ²
How can var(Xβ+u) be simplified?
var(u)
How do betas change with operators on y?
Multiplication is performed on all betas in the same direction, addition is also applied to the intercept
