Lecture notes 1 (Two variable regression) Flashcards
How do you calculate correlation?
What is a feature about it?
COV(X,Y) / sqrt V(X) V(Y)
It is scale free.
What is correlation?
The systematic relationship between two variables?
How do you calculate Covariance of X,Y
Show both methods
Sum of (xi- xbar)(yi-ybar) / n-1
E(y-meany)(x-meanx)
When can we view correlation as a causal relationship? (in context)
In clinical trials when you randomly assign different people to different groups.
What two components is this made of yi = α + βxi + εi ?
yi = α + βxi + εi
yi = α + βxi (systematic part)
εi is random
What are the 4 Classical Linear Regression assumptions in a model like:
yi = α + βxi + εi
- E(εi | xi) = E(εi) = 0 ∀i (so the error term is independent of xi).
- V (εi | xi) = σ^2 ∀i (error variance is constant (homoscedastic) – points are distributed around the true regression line with a constant spread)
- cov(εi, εj |xi) = 0 for i ̸= j (the errors are serially uncorrelated over observations)
- εi | xi ∼ N(0, σ^2) ∀i
How can we interpret something as a causal effect?
If E(εi | xi) = 0
What is the CLRM assumption of homoscadascity?
That all error terms must have a constant variance for every different explanatory variable value.
What does this CLRM assumption mean cov(εi, εj |xi) = 0 for i ̸= j
It shows the information about the ith person cannot predict the jth person.
Knowing the epsilon of person i will not help in predicting person j
What does the CLRM assumption mean εi | xi ∼ N(0, σ^2) ∀i
-The error terms take on a normal distribution
-Purely for mathematical convenience.
What does the CLRM assumption mean
E(εi | xi) = E(εi) = 0 ∀i
for every value of x the mean of epsilon is 0
The error terms are indepedent of x
When estimating a simple regression line for alpha, beta and standard deviation what are the estimates?
Alpha = a
Beta = b
Stdv = s^2
How does the OLS estimation work?
yi = α + βxi + εi
yi is real value
yhat is predicted value
for OLS you want to minimise difference between real and predicted value by choosing a and b.
-Add sqaured differences of each observation and minimise it.
-Optimal solution is to minimise the residuals sum of sqaures.
What is e?
e is the observed residual difference between yi and yhat
When using OLS graphically what are we doing?
We are looking at a perfect straight line then looking at how far actual data point is from straight line.
Square all these differences and then minimise them.
This would then find the straight line that is closest to all data points.
How do we estimate the variance in a regression
= S^2 Sample variance
1/ n-2 times by the sum of all residuals.
= RSS/ DOF
How do we work out OLS estimator for slope coefficient?
b = sum of (xi - xbar)(yi-ybar) / sum of (xi-xbar)^2
If you have an original model and then create a new model with all X times by 2 what happens to b?
How do you solve for this?
The new b is half the initial b
You sub in the relationship for new and old x.
To OLS formula.
Then also for this for intercept.
How are the estimates under OLS described?
Best - the OLS estimators have the minimum variance, such that V (b|x) ≤ V (b∗|x), where b* is any alternative unbiased estimator.
Linear - the OLS estimators are a linear function of the error term
Unbiasedness- the OLS estimators are such that: E(a|x) = α and E(b|x) = β
Estimators.
What must you look out for when interpreting coefficients?
WHAT TYPE OF MODEL IS IT?
DOES IT HAVE LOGS?
DO WE NEED TO TAKE e^ for exact if greater than 0.1
What is the value at which you can approximate log coefficients?
0.1 and less can be approximated.
What is the interpretation of coefficient with y = Blog(x)
A 1% increase in X causes a Beta/ 100 % increase in Y.
Or if greater than 0.1 use exact e
What is the interpretation of coefficient with Y = BX
A 1 unit change in X causes a Beta unit increase in Y
What is the interpretation of Cocefficient with log(y) = BX
When X increases by 1% Y increases by Beta x 100 %
when x is greater than 0.1 then exp(b) -1 x 100 corresponds to % change in Y.
