07 Linear Regression

Question 1

σ

Answer 1

Population standard deviation

Question 2

s

Answer 2

Sample standard deviation:

An estimator of the population standard deviation

Question 3

s_y

Answer 3

s_y is the estimate of the population standard deviation for the random variable Y of the population from which the sample was drawn

Question 4

SE()

Answer 4

Standard error of an estimator:
An estimator of the standard deviation of the estimator

SE( ̄Y) = ˆσ_ ̄Y = s_y / √n

Question 5

μ

Answer 5

Population mean

Question 6

u

Answer 6

all other factors than X that affects Y

Question 7

synonyms for “dependent variable” and “independent variable”

Answer 7

dependent variable vs. independent variable
explained variable vs. explanatory variable
predicted variable vs. control variable
response variable vs. control variable
regressand vs. regressor

Question 8

A normally distributed variable (X) can be made standard normal by:

Answer 8

Z = (X - μ) / ( σ / root(n))

Question 9

The sample average is normally distributed whenever:

Answer 9

• Xi is normally distributed

- n is large (CLT)

Question 10

T variable

Answer 10

T = (X - μ) / (s_x / root(n))

Question 11

SLRM

Answer 11

Simple Linear Regression Model

Question 12

The sum of squared prediction mistakes over all n observations

Answer 12

sum[(Y - E(β0) - E(β1)X)^2]

Question 13

E(β0)

Answer 13

E(β0) = avg(Y) - E(β1) * avg(X)

Given by derivation++ of sum[(Y - E(β0) - E(β1)X)^2]

Question 14

E(β1)

Answer 14

E(β1) = sum[(X - avg(X)) (Y - avg(Y))] /
sum[(X - avg(X)^2]

Given by derivation++ of sum[(Y - E(β0) - E(β1)X)^2]

E(β1) = r_{XY} * s_Y / s_X

Question 15

If uˆi is positive, the line ____ Yi

Answer 15

If uˆi is positive, the line underpredicts Yi

Question 16

By the definition of uˆ and the first OLS first order condition the sum of the prediction error is …

Answer 16

By the definition of uˆ and the first OLS first order condition the sum of the prediction error is zero

Sum(û_i) = 0

Question 17

The sample covariance between the independent variable and the OLS residuals is …

Answer 17

The sample covariance between the independent variable and the OLS residuals is zero.

Question 18

The point … is always on the regression line (OLS)

Answer 18

The point (X ̄ , Y ̄) is always on the regression line (OLS)

Question 19

different goals of regression

Answer 19

Among others:

• Describe data set
• Predictions and forecasts
• Estimate causal effect

Question 20

Causality

Answer 20

Causality is the effect measured in an ideal randomized controlled experiment

Question 21

The OLS estimator is unbiased, consistent and has asymptotically normal sampling distribution if:

Answer 21

The OLS estimator is unbiased, consistent and has asymptotically normal sampling distribution if:
- Random sampling
- Large outliers are unlikely
- The conditional mean of u_i given X_i is 0:
E (u|X ) = 0
E(abil | educ = 8) = E(abil | educ = 16).

Question 22

The OLS estimator is ___, ____ and has _____ if:

• Random sampling
• Large outliers are unlikely
• The conditional mean of u_i given X_i is 0

Answer 22

The OLS estimator is unbiased, consistent and has asymptotically normal sampling distribution if:

• Random sampling
• Large outliers are unlikely
• The conditional mean of u_i given X_i is 0

Question 23

(OLS) When dealing with outliers one may want ______

Answer 23

When dealing with outliers one may want to report the OLS regression both with and without the outliers

Question 24

OLS is the most e cient (the one with the lowest variance) among all linear unbiased estimators whenever:

Answer 24

OLS is the most ecient (the one with the lowest variance) among all linear unbiased estimators whenever:

• The 3 OLS assumptions hold
• The error is homoskedastic

Question 25

TSS

Answer 25

Total sum of squares:
Sum[(Yi - avg(Y))^2]

TSS = ESS + SSR

Question 26

ESS

Answer 26

Explained sum of squares:

Sum[(^Yi - avg(Y))^2]

Question 27

SSR

Answer 27

Sum of squared residuals:

Sum[û_i^2]

Question 28

R^2

Answer 28

The regression R^2 is the fraction of the sample variance of Yi explained by Xi:

R^2 = ESS / TSS = 1 - SSR / TSS

R2 = 0 - none of the variation in Yi is explained by Xi
R2 = 1 - all the variation is explained by Xi, all the data points lie on the OLS line.
A high R2 means that the regressor is good at predicting Yi (not necessarily the same as a ”good” regression)

Question 29

SER

Answer 29

The standard error of the regression (SER) is an estimator for the standard deviation of the regression error u_i.

SER = SSR / (n - 2)

It measures the spread of the observations around the regression line.

Question 30

If the independent variable is multiplied by som nonzero constant c, then the OLS slope coeffi cient is _____

Answer 30

If the independent variable is multiplied by som nonzero constant c, then the OLS slope coeffi cient is divided by c.

Question 31

Homoskedasticity

Answer 31

The error u has the same variance given any value of the explanatory variable, in other words: Var(u|x) = 2

Homoskedasticity is not required for unbiased estimates, but it is an underlying assumption in the standard variance
calculation of the parameters. To make the variance expression easy the assumption that the errors are homoskedastic are added.

Question 32

The larger the variance of X, the ____ the variance of E(β1)

Answer 32

The larger the variance of X, the smaller the variance of E(β1)

Question 33

Var(E(β1))

Answer 33

Holy shit. (Appendix 4.3)

Question 34

s_xy

Answer 34

Sample covariance

1 / (n - 1) * sum{(Xi - avg(X))(Yi - avg(Y))}

Question 35

s^2_X

Answer 35

Sample variance of X

1 / (n - 1) * sum{(Xi - avg(X))(Xi - avg(X))}

Question 36

sample correlation coefficient

Answer 36

r_{XY} = s_{XY} / (s_X * s_Y)

Question 37

consistency

Answer 37

A variable is consistent if the spread around the true parameter approaches zero as n increases

Question 38

Normality assumption

Answer 38

The population error u is independent of explanatory variables and is Normal(0, σ^2)

• Whenever y takes on just a few values it cannot have anything close to a normal distribution.
• The exact normality of OLS depends on the normality of the error.
• If the βˆ is not normally distributed the t-statistic does not have t-distribution.
• The normal distribution of u is the same as the distribution of Y given X.
• In large samples we can invoke the CLT to conclude that the OLS satisfy asymptotic normality.

Question 39

E(β1) ∼

Answer 39

Normal[β1, Var(E(β1))]

Thus (E(β1) − β1) / std(E(β1)) ∼ Normal(0, 1)

This comes from:
• A random variable which is a linear function of a normally distributed variable is itself normally distributed.
• If we assume that u ∼ N(0, σ2) then Yi is normally distributed.
• Since the estimators βˆ and βˆ is linear functions of the Yi’s then the estimators are normally distributed.

Question 40

In general the t-statistics has the form:

Answer 40

t = (estimator - hypothesised value) / standard error of the estimator

Question 41

A coefficient can be statistically significant either because ____

Answer 41

A coefficient can be statistically significant either because the coefficient is large, or because the standard error is small.

Question 42

(OLS) Which standard errors are prefered?

Answer 42

Heteroskedasticity robust standard errors

In econometric applications the errors are rarely homoskedastic and normally distributed, but as long as n is large and we compute heteroskedasticity robust standard errors we can compute t-statistics and hence p-values and confidence intervals as normal.

Question 43

(OLS) Most often the violated assumption is ___

Answer 43

(OLS) Most often the violated assumption is the zero conditional mean assumption, X is often correlated with the error term.

Question 44

Sum[ (Xi - avg(X)) (Yi - avg(Y)) ] = Sum[ … ]

Answer 44

Sum[ Xi (Yi - avg(Y)) ]