Chapter 4: The Simple Regression Model Flashcards
What is the simple linear regression model?
y = β0 + β1x + u
It is also called the two-variable linear regression model or bivariate linear regression model because it relates to the two variables x and y.
Y = dependant variable
X = independent variable
u = error term or disturbance in the relationship
β1 = slope parameter
β0 = intercept parameter/constant term
β0 and β1 are true unknown parameters we will estimate from the data.
B̂0 and B̂1 are our estimate for β0 and β1 from the data
What are the meanings of notations?
▶ Greek letters like β1 are the truth. They are part of the true Data Generating Process (DGP).
▶ Modified Greek letters like β1 are our estimate. They are what we think the truth is based on our data.
▶ Upper case english letters like X are random variable.
▶ Lower case english letters like x are actual data from our sample.
▶ Modified English letters like x̄ are calculation from our sample. They are what we do with our data (e.g
x̄ = 1/n Σn i = 1 xi )
▶ We can say that our estimate of the truth is that calculation, e.g. μ = x̄
What is an example of the regression model on house price and land size?
House price = β0 + β1landsize + u
So that y = house price and x = land size.
The researcher is interested in the effect of land size on the price of a house, holding other factors fixed. This effect is given by β1. The error term u contains factors such as the number of bedrooms, number of bathrooms, number of car parking spaces, whether the house has a pool and so on.
The coefficient β1 measures the effect of land size on the house price, holding other factors fixed: ∆house = β1∆landsize
What is an example of a regression model for a simple wage equation?
A model relating a person’s wage to observed education and other unobserved factors is:
Wag = β0 + β1educ + u
If wage is measured in dollars per hour and educ is years of education, then β1 measures the change in hourly wage given another year of education, holding all others factors fixed.
Some of those factors include labour force experience, innate ability, tenure with current employer, work ethic and numerous other things.
What does the linearity of the simple regression equation imply?
It implies that a one-unit change in x has the same effect on y regardless of the initial value of x.
However, this is unrealistic for many economic applications. E.g. in the wage-education example we might want to allow for increasing returns the next year of education has a larger effect on wages than did the previous year.
How are u and x related?
The average value of u does NOT depend on the value of x: E(u|x) = E(u)
What is the population regression function?
E(y|x) = β0 + β1x
The population regression function (PRF), E(y|x), is a linear function of x.
The linearity means that a one-unit increase in x changes the expected value of y by the amount of β1.
For any given value of x, the distribution of y is centred about E(y|x).
How do we derive the ordinary least squares (OLS) estimates of the parameters β0 and β1?
To do this we need a sample from the population. Let {(xi, yi): i = 1, …, n} denote a random sample of size n from the population. Because these data come from the simple regression equation, we can write:
yi = β0 + β1xi + ui for each i.
Here, ui is the error term for observation i because it contains all factors affecting yi other than xi.
Let ŷ = B̂0 + B̂1xi
What are OLSs used for?
When we want to draw a line, it allows us to choose where b0 and b1 go.
. ŷ = b0 + b1 * xi are the fitted values of yi given xi and our model (i.e. b0 and b1)
. ûi = ŷi - yi is the residual (i.e. error made by the model)
. ûiˆ2 > 0
What are the formulas for B̂0, B̂1, x̄ and ȳ?
B̂0 = ȳ - B̂1x̄
B̂1 = [∑n i=1 (xi - x̄) (yi - ȳ)] \ [∑n i=1 (xi - x̄)ˆ2]
Where
x̄ = 1/n ∑i xi
ȳ = 1/n ∑i yi
B̂0 and B̂1 are the OLS estimators of B0 and B1. Note that B̂1 = sample covariance (Y, X) / sample variance (X)
We would like to know the effect of attending seminars on grades. We have date on midterm test scores (out of 20) and number of tutorials attended from T1 2014.
We want to estimate: grade = B0 + B1tutorial + u
Tutorial Grade
6 15
5 14
3 8
2 6
1 7
6 13
4 10
x̄ = 1/n ∑i xi
x̄ = (6 + 5 + 3 + 2 + 1 + 6 + 4)/7 = 27/7 = 3.86
ȳ = 1/n ∑i yi
ȳ = (15 + 14 + 8 + 6 + 7 + 13 + 10)/7 = 73/7 = 10.42
Tute Grade x - x̄ y - ȳ (x-x̄)*(y-ȳ) (x - x̄)ˆ2
6 15 2.14 4.58 9.80 4.58
5 14 1.14 3.58 4.08 1.30
3 8 -0.86 -2.42 2.08 0.74
2 6 -1.86 -4.42 8.22 3.46
1 7 -2.86 -3.42 9.78 8.18
6 13 2.14 2.58 5.52 4.58
4 10 0.14 -0.42 -0.06 0.02
∑(x - x̄) * (y - ȳ) = 39.43
∑(x - x̄)ˆ2 = 22.86
Therefore
B̂1 = 39.43/22.86 = 1.73
B̂0 = 10.42 - 1.73 * 3.86 = 3.74
In turn, the estimated regression function is:
Grade = 3.74 + 1.73 * tutorial
What’s the interpretation of the following regression function: Grade = 3.74 + 1.73 * tutorial
Interpretation of B̂0 = 3.74
. We expect (on average) grade for a student who attended 0 tutorial to be 3.74 out of twenty.
Interpretation of B̂1 = 1.73
. We expect (on average) grades to increase by 1.73 for every additional tutorial attended.
What’s the interpretation of the following regression function: waˆge = −0.9 + 0.54educ
Wage: hourly wage in $
Educ: years of education
Interpretation of the intercept B̂0 = -0.9
. A person with no education has a predicted hourly wage of -$0.9
Interpretation of the slope B̂1 = 0.54
. One more year of education increases wage by $0.54 per hour on average
What’s the interpretation of the following regression function: sell = 34136 + 71landsize
Interpretation of intercept B̂0 = 34136
. A property with a land size of 0 will cost $34136 on average
Interpretation of the slope B̂1 = 71
. One more square meters will increase the property price by $71 on average
