Econometrics 2: Bivariate Linear Regression 2.3 - Ordinary Least Squares Estimation

Question 1

Describe & explain the first-order conditions of OLS

Answer 1

OLS is the procedure that minimises the sum of squared residuals 𝑆 = ∑top right n bottom right i=1 𝜀̂bottom right𝑖 ^2, i.e.
we need to
min 𝑆 = min ∑top rightn bottom right i=1 𝜀̂bottom right𝑖 ^2 = min ∑(𝑌bottom right𝑖 − 𝛽̂hat bottom right0 − 𝛽̂hat bottom right1 𝑋Bottom right𝑖)^2
We know that to find the maximum or minimum of a function we need to differentiate and set to
0. This gives what we call the first-order conditions:
𝜕𝑆/𝜕𝛽̂hat0 = −2 ∑( 𝑌bottom right𝑖 − 𝛽̂hat0 − 𝛽̂hat1𝑋𝑖) = 0 (4)
𝜕𝑆/𝜕𝛽̂hat1 = −2 ∑ 𝑋𝑖( 𝑌𝑖 − 𝛽̂hat0 − 𝛽̂hat1𝑋𝑖) = 0 (5)
On solving these equations, we find that
𝛽̂hat0 = 𝑌̅bar − 𝛽̂hat1𝑋̅bar (6)
𝛽̂bar1 = ∑(𝑋𝑖−𝑋̅bar)(𝑌𝑖−𝑌̅bar) / ∑(𝑋𝑖−𝑋̅bar)^2 = ∑ 𝑋𝑖𝑌𝑖−𝑛𝑋̅bar𝑌̅bar / ∑ 𝑋𝑖^2−𝑛𝑋̅bar^2 (7)
These are the OLS estimators of 𝛽0 and 𝛽1 and as you can see, they are simply equations that
involve our data, 𝑋 and 𝑌. We can now see how we combine data with a statistical technique to
get estimates of the unknown parameters in the regression model. We have our data on variables
𝑋 and 𝑌. We have the statistical technique of OLS and this gives us the formulae with which we
can use the data in order to get our estimates, i.e. input the values of our dataset, 𝑋 and 𝑌, into our
estimator equations (6) and (7), and out pop two numbers, one an estimate of 𝛽0, the other an
estimate of 𝛽1