Lecture notes 1 (Two variable regression) Flashcards
How do you calculate correlation?
What is a feature about it?
COV(X,Y) / sqrt V(X) V(Y)
It is scale free.
What is correlation?
The systematic relationship between two variables?
How do you calculate Covariance of X,Y
Show both methods
Sum of (xi- xbar)(yi-ybar) / n-1
E(y-meany)(x-meanx)
When can we view correlation as a causal relationship? (in context)
In clinical trials when you randomly assign different people to different groups.
What two components is this made of yi = α + βxi + εi ?
yi = α + βxi + εi
yi = α + βxi (systematic part)
εi is random
What are the 4 Classical Linear Regression assumptions in a model like:
yi = α + βxi + εi
- E(εi | xi) = E(εi) = 0 ∀i (so the error term is independent of xi).
- V (εi | xi) = σ^2 ∀i (error variance is constant (homoscedastic) – points are distributed around the true regression line with a constant spread)
- cov(εi, εj |xi) = 0 for i ̸= j (the errors are serially uncorrelated over observations)
- εi | xi ∼ N(0, σ^2) ∀i
How can we interpret something as a causal effect?
If E(εi | xi) = 0
What is the CLRM assumption of homoscadascity?
That all error terms must have a constant variance for every different explanatory variable value.
What does this CLRM assumption mean cov(εi, εj |xi) = 0 for i ̸= j
It shows the information about the ith person cannot predict the jth person.
Knowing the epsilon of person i will not help in predicting person j
What does the CLRM assumption mean εi | xi ∼ N(0, σ^2) ∀i
-The error terms take on a normal distribution
-Purely for mathematical convenience.
What does the CLRM assumption mean
E(εi | xi) = E(εi) = 0 ∀i
for every value of x the mean of epsilon is 0
The error terms are indepedent of x
When estimating a simple regression line for alpha, beta and standard deviation what are the estimates?
Alpha = a
Beta = b
Stdv = s^2
How does the OLS estimation work?
yi = α + βxi + εi
yi is real value
yhat is predicted value
for OLS you want to minimise difference between real and predicted value by choosing a and b.
-Add sqaured differences of each observation and minimise it.
-Optimal solution is to minimise the residuals sum of sqaures.
What is e?
e is the observed residual difference between yi and yhat
When using OLS graphically what are we doing?
We are looking at a perfect straight line then looking at how far actual data point is from straight line.
Square all these differences and then minimise them.
This would then find the straight line that is closest to all data points.
How do we estimate the variance in a regression
= S^2 Sample variance
1/ n-2 times by the sum of all residuals.
= RSS/ DOF
How do we work out OLS estimator for slope coefficient?
b = sum of (xi - xbar)(yi-ybar) / sum of (xi-xbar)^2
If you have an original model and then create a new model with all X times by 2 what happens to b?
How do you solve for this?
The new b is half the initial b
You sub in the relationship for new and old x.
To OLS formula.
Then also for this for intercept.
How are the estimates under OLS described?
Best - the OLS estimators have the minimum variance, such that V (b|x) ≤ V (b∗|x), where b* is any alternative unbiased estimator.
Linear - the OLS estimators are a linear function of the error term
Unbiasedness- the OLS estimators are such that: E(a|x) = α and E(b|x) = β
Estimators.
What must you look out for when interpreting coefficients?
WHAT TYPE OF MODEL IS IT?
DOES IT HAVE LOGS?
DO WE NEED TO TAKE e^ for exact if greater than 0.1
What is the value at which you can approximate log coefficients?
0.1 and less can be approximated.
What is the interpretation of coefficient with y = Blog(x)
A 1% increase in X causes a Beta/ 100 % increase in Y.
Or if greater than 0.1 use exact e
What is the interpretation of coefficient with Y = BX
A 1 unit change in X causes a Beta unit increase in Y
What is the interpretation of Cocefficient with log(y) = BX
When X increases by 1% Y increases by Beta x 100 %
when x is greater than 0.1 then exp(b) -1 x 100 corresponds to % change in Y.
How do you interpret the coefficient for a log log model?
log(y) = Log(x)
when X increases by 1%, Y increases by Beta%
No need to take exact as already in logs
What is another formula for the se(b)
all sqrt of s^2 / sum of xi - xbar squared
How do you perform a hypothesis test for slope coefficent?
- H0: B0
H1: B1 - Take significance level and DOF and find CValues
DOF = (N- nof restrictions on residuals) - We then calculate t = b - B0 / se(b)
- then compare to CV and either reject or accept H0.
What is forecasting?
What is the issue and how is it solved?
Regression models often give one data point prediction but these are unlikely to be correct.
Therefore, we use confidence intervals to predict a range of values in which it will fall.
How do you form a confidence interval?
Predicted value + / - CV x Standard error.
- What is the measure of goodness of fit and what equation explains it?
- Break down each part of the equation
Measure of goodness of fit = R^2 = ESS/TSS or 1 - RSS / TSS
Sum of i=1 to n (Yi - Ybari) = Sum of i=1 to n (Yhat - Ybarhat) + sum of i=1 to n of e^2
Sum of i=1 to n (Yi - Ybari) = Total variation in the data (TSS).
Sum of i=1 to n (Yhat - Ybarhat) = Variation in the predictor values. (ESS)
sum of i=1 to n of e^2 = Variation in the regression residuals. (RSS)
1.Is it correct to say a model with a high R^2 is a good model?
- What is a better way to measure model adequacy?
- No, it just means the model explains the total variation well.
Plus R^2 is quite an informal measure so cannot put too much emphasis.
- if the error terms are aligned with the CLRM.
Go on lectures notes 1 final page and explain all different parts of the stata output file.
When do you interpret an intercept?
-When the explanatory variable is a feasible number
