Bivariate Linear Regression

Question 1

regression model for consumption

Answer 1

Add ε to

Y=β₀+β₁x + ε

β₀ is intercept
β₁is MPC
ε is random variable to represent other factors influencing consumption.

Question 2

When is a model linear

Answer 2

If it is linear in the parameters. Contains no unknown parameters

Question 3

Line of best fit formula

Answer 3

OLS (ordinary least squares)

Use estimated regression formula
Yi=β’₀ + β’₁xi + ε’i

Rearrange this to make residuals the subject, then square
ε’²i = min Σ(Yi - β₀ - β₁Xi)²

I.e square all the deviation/residuals, then add up.

I.e if n=28, used as example.

Question 4

Then, how to find minimum S

What values do we get for estimated β₀ and β₁

Answer 4

Differentiate β₀ and β₁ individually from
Min Σ(Yi - β₀ - β₁Xi)² and set =0 as we want to find minimum

To get…

β₀ = Ybar - β₁Xbar

β₁= …

Question 5

So now we know how to estimate β₁ and β₀,

Why do we need to know properties of OLS estimators

Answer 5

To see if the estimates we get for β₁ and β₀ are accurate

E.g we found B1 (MPC) is 0.7765, so per every £1 increase consumption by 0.77p. Is this accurate? We have to check properties

Question 6

Properties REQUIRED of OLS estimators (2)

Answer 6

Unbiased - centre of distribution
Efficiency - small variance

Question 7

How do we know OLS has these properties?

Answer 7

If certain conditions are satisfied, called classic linear regression assumptions. (CLRA)

SO WE USE CLRA TO ENSURE OLS IS UNBIASED AND EFFICIENT TO BE ABLE TO ESTIMATE B0 AND B1 ACCURATELY)

Question 8

Classical linear regression assumptions

Answer 8

1. Model is written as
   Y=β₀+β₁x + ε (and β₀ β₁ are unknown parameters/constants)
2. Explanatory variable X is fixed/non-stochastic (we can choose values of X in order to observe effects on Y
3. X is not a constant. It is variable, researchers adjust to observe values of Y!
4. Error (ε) has EV(mean) of 0
5. No 2 errors are correlated. Except in time-series models.
6. Each error has same σ² variance except in cross-sectional models. (HOMOSCEDASTIC)
7. Error is normally distributed. (Mean 0, variance σ²) allows us to do hypothesis testing.

Question 9

Theoretical assumption 1

Answer 9

Under classical linear regressions assumptions 1-4 holding,

OLS estimators are unbiased.

Question 10

Theoretical result 2

Answer 10

Under CLRA 1-6 holding,

OLS estimator is the best linear unbiased estimator (BLUE).

(MINIMUM VARIANCE, SO MOST EFFICIENT)

Question 11

Theoretical result 3

Answer 11

Under CLRA 1-7 holding

OLS estimator is the minimum variance unbiased estimator of linear AND non-linear estimators. (TR2 is just best LINEAR only)

Question 12

Proof for TA1 : estimator of slope parameter B^₁ is unbiased

Answer 12

Start with this
E(β^₁) = E(β₁+Σwiεi)

Simplify using technical appendix to make
= β₁ + ΣWi E(εi)

E(β₁)=β₁ using estimator properties
CLRA2 removes ΣWi from the expectations operator (since X is fixed)
CLRA 4 - ε mean is 0, therefore it leaves E(β^₁)= β₁ therefore unbiased

Question 13

Proof for why estimator of slope parameter B^₀ is unbiased

Question 14

Proof for lowest variance (BLUE)

Answer 14

CLRA 5 and 6 needed

Question 15

Learn 6* and 7* lowest variance formulas, we should end up getting that answer in the proof which proves TR2 and BLUE (lowest variance)

Answer 15

For estimated B₁,
Start with var (β^₁) = var (β₁ +Σwiεi)

Simplifies to var(Σwiεi) since β₁ is constant and thus no variance so =0

CLRA5 removes the correlation part to leave (as correl=0)
Σwi²var(εi)
CLRA6 - all ε has equal variances of σ²
E comes
σ²Σwi²

Then finally replace Wi with orignal X function to get 7* (would’ve got the lowest possible variance result)

Question 16

Random Regresssor

Answer 16

Rarely we actually have fixed X (CLRA2), so we have to consider X will be a random regressor

Question 17

Relationship between x and ε

Answer 17

We look at the value of ε given values of X (as random now)

We assume ε does not depend on value of X

Question 18

How is this expressed?

Answer 18

E (εi | Xi) = E(εi)

means conditional on X, but since we assume ε does not depend on X, it is the same

Question 19

What does this expression also show

Answer 19

Shows X and E are mean independent (X is strictly exogenous to E)

Question 20

What happens when we apply CLRA4 to the X and ε relationship

Answer 20

𝐸(𝜀𝑖|𝑋𝑖) = 0

Question 21

What does the zero conditional mean look like as an example

Answer 21

Assume ε represents innate ability and we are looking at wages.

So the average innate ability is the same regardless of given X (schooling years)

I.e whether it be 1 year or 20 years in education, ability stays the same.

However, if we think innate ability increases with schooling years, zero conditional mean does not hold.

Question 22

So now we need to adjust the classical assumptions to account for the random X and the zero conditional mean assumption.

Question 23

CLRA 1-6 accounting for random X

(No longer 7 assumption)

Answer 23

CLRA1 - the same except ε₁ is random

CLRA 2 - there IS variation in X variable (same)

CLRA 3 - Error has zero conditional mean assumption.

CLRA 4 -disturbances are conditionally uncorrelated, 𝑐𝑜𝑣(𝜀i, 𝜀j |𝑋) = 0 ,where 𝑖 and 𝑗 are time periods, because this only affects time-series models. (Same but just add |X)

CLRA 5 - Each E has finite CONDITIONAL VARIANCE 𝑣𝑎𝑟(𝜀𝑖|𝑋) = 𝜎2

CLRA 6 - E , conditional on X is normall distributed
𝜀𝑖|𝑋~𝑁(0, 𝜎2).

Question 24

Coefficient of determination - goodness of fit

Answer 24

R squared.

Shows how much of the total variation of Y is attributable to the regression line.

Question 25

When we consider the coefficient of determination , how does the regression change?

Answer 25

𝑌i = 𝛽^₀ + 𝛽^₁𝑋i + 𝜀̂i = 𝑌^ + 𝜀̂𝑖

Actual Y = 𝑌^ + error (Estimated value of y + deviation above or below the line)

Question 26

Total sum of squares

Answer 26

Σ(Yi -Ybar)²

Question 27

Explained sum of squares

Answer 27

Σ(Y^i-Ybar)²

Question 28

Residual sum of squares

Answer 28

Σε^i²

Same thing that OLS is trying to minimise! (Square all residuals and then sum up)

Question 29

R² formulas (2)

Answer 29

ESS/TSS

Or

1 -RSS/TSS

Question 30

What is the t statistic formula to work this out

Answer 30

T= βhat₁ - δ
/
√σ²/Σ(Xi-Xbar)² (or SE of β₁)

~ t(n-2) since 2 unknown parameters (β₀ and β₁)

But we also need to know formula for σ²
σ²= Sum of residiuals squared / n-2

Question 31

Error variance σ² formula

Answer 31

RSS/n-2

Question 32

Confidence bands

Answer 32

Bhati + tcv (seβ₁)