HANDOUT 1 Flashcards
COV(X, Y) =
SUM (XI - XBAR)(YI - YBAR) / N-1
WHY IS DOF FOR COV N-1?
we only need either X bar or Y bar - NOT both
Why is COV not that useful?
It is NOT scale invariant - can make it as big or small as possible by changing units
CORR(x,y)=
= COV(X,Y) / SQRT VAR(X) VAR(Y)
Is correlation scale invariant?
YES
What is epsilon?
Ei = Yi - E(Yi I Xi) = Yi - alpha - beta(Xi)
True but unknown relationship between X and Y. What 2 parts?
Yi = alpha + beta(Xi) + Ei
Systematic & random components
4 CLRM ASSUMPTIONS
- E(Ei I Xi) = 0
- V(Ei I Xi) = sigma^2
- COV(Ei, Ej I Xi) = 0
- Ei I Xi - N(0, sigma^2)
ei =
the RESIDUALS
ei = yi - y hat = yi - a - bxi
OLS method
minimise the residual sum of squares to find the line of best fit.
MIN SUM ei^2
2 OLS conditions
- SUM ei = 0 - zero luck on average
2. SUM ei.xi = 0 - luck independent of x
What does ORTHOGONAL mean?
Independent e.g. SUM ei.xi = 0 means ei and xi are orthogonal.
If sum ei = 0 is NOT satisfied, how do we change the best fit line?
SHIFT it up/down
If sum ei.xi= 0 is NOT satisfied, how do we change the best fit line?
ROTATE it
estimate of alpha = a
a = y bar - b xbar
estimate of beta = b
b = sum (xi - xbar)(yi - ybar) / sum (xi - x bar)^2
COV(x,y) / Var(X)
estimate of sigma^2 = s^2
s^2 = sum ei^2 / DOF = RSS / DOF
= RSS / n - 2
properties of OLS estimators
B = best - min variance of b = efficient L = linear - a and b are linear functions of Ei which is normal so a and b normal U = unbiased E = estimators
Formula for b in terms of beta to prove unbiasedness
b = beta + sum wi.ei wi = (xi - xbar) / sum (xi - xbar)^2 E(b) = beta
What CLRM assumption do we need for unbiasedness?
- E(Ei I Xi) = 0
what CLRM assumptions do we need for min variance of b?
- E(Ei I Xi) = 0
- Var(Ei I Xi) = sigma^2
- COV(Ei, Ej I Xi) = 0
Formula for variance of b conditional on x
V(b I X) = sigma^2 / sum(Xi - Xbar)^2
What does V(b) depend on and how?
- Sigma^2: as variance of error term rises, variance of b rises as more uncertainty
- Var(X): more uncertainty in X = we have a huge data range = can pin down the slope more easily = smaller V(b)
distribution of b
Normal since b = beta + sum wi.Ei so linear function of Ei which is normal
BUT when we test b what distribution do we use and why?
b = normal
BUT sigma^2 unknown –> use s^2
Therefore normal –> t distribution
Distribution of s^2 and
(n - 2)s^2 / sigma^2 = Chi-squared n-2
confidence interval for beta formula
beta = [b +- CV.se(b)]
does the model have to be linear in x and y?
NO - only needs to be linear in parameters, can be non-linear in x and y
Interpretation of beta if linear in both x and y
Beta = expected change in Y for 1 unit increase in X i.e. marginal effect
Interpretation of beta if linear Y but not X
If g <= 0.1: B/100 = change in Y for 1% increase in X.
If g>0.1: B x ln(1 + g) = change in Y for g
= semi-elasticity
Interpretation of beta if non-linear Y but not X
If g<=0.1, Beta = expected growth rate of Y for 1 unit increase X
If g>0.1:
e^beta - 1 = expected growth rate of Y
100(e^beta - 1) = % change in Y for 1 unit rise in X = semi-elasticity
Interpretation of beta if non-linear Y & X
Elasticity: dy/dx = beta
Beta = proportionate change in Y / 1 proportionate change in X
= % change in Y / 1% increase in X
TSS identity
TSS = ESS + RSS
What is TSS?
TSS = sum(yi - ybar)^2
total variation in the dependent variable
What is ESS?
ESS = sum(yi hat - y hat bar)^2
variation in Y the model explains
What is RSS?
sum ei^2
Variation in Y the model fails to explain
Formula for R^2 and what is it?
R^2 = 1 - RSS/TSS = ESS/TSS
Measures the total variation in Y that the model explains. Between 0 and 1.
We use R^2 to…
Compare different linear models with the SAME dependent variable
How do we test if the model predicts new observations well?
hypothesis test whether en+1 is significantly different from 0 as we want E(en+1)=0
COV(a,b)=
Xbar sigma^2 / sum(xi - xbar)^2
V(en+1)=
sigma^2 [1 + 1/n + (xn+1 - xbar)^2/sum(xi -xbar)^2]
