17 Linear Regression

Question 1

Limitations of correlation coefficient

Answer 1

Doesn’t help us make predictions, it is only calculated for two variables

Question 2

What does Ei represent?

Answer 2

The error term

Question 3

Assumption of simple linear regression model

Answer 3

Xi are fixed (non random)

Ei and Yi are random variables

Question 4

Residuals of regression

Answer 4

Êi= yi- ŷi

Measure the vertical distance between the fitted line ŷi and the actual values of yi

Question 5

What is the intercept

Answer 5

B0

Question 6

What is the slope

Answer 6

B1

Question 7

OLS

Answer 7

Ordinary Least Squares. It works by fitting a line through the data minimising the sum of squared residuals

Question 8

What is the estimate of B1 equal to?

Answer 8

Cov(x,y)/var(x)

Question 9

What is the estimate of B0 equal to?

Answer 9

The mean of y - cov(x,y)/var(x) x mean of x

Question 10

Is the OLS estimator biased?

Answer 10

No because the expected value of the estimates of B0 and B1 is B0 and B1

Question 11

Which estimators are blue and have the smallest variance?

Answer 11

OLS estimators

Blue = best linear unbiased estimator

Question 12

What is the variance of the estimators of B0 and B1?

Answer 12

Zero, this shows they are consistent estimators

Question 13

What is coefficient of determination denoted as?

Answer 13

R^2

Question 14

What is the coefficient of determination

Answer 14

It calculates the proportion of the variation in the dependent variable that is explained by the fitted regression

Question 15

Total sum of squares (TSS)

Answer 15

The total squared variation of the yi values about their mean
TSS= sum of (yi-mean)^2

Question 16

Explained sum of squares ESS

Answer 16

The total squared variation of the fitted values ŷi about their mean
ESS= sum of (ŷi-mean)^2

Question 17

Residual sum of squares RSS

Answer 17

The total squared difference between the yi values and the fitted ŷi values
RSS= sum of (yi-ŷi)^2

Question 18

What is the TSS made up of?

Answer 18

TSS=ESS+RSS

Question 19

How can we work out the coefficient of determination?

Answer 19

R^2=ESS/TSS

R^2=1-RSS/TSS

Question 20

What values can R^2 take?

Answer 20

0<=R^2<=1

Question 21

What does it mean if ESS and R^2 are large?

Answer 21

The model is a good fit

Question 22

When is degrees fo freedom n-2?

Answer 22

When we are estimating two parameters

Question 23

As sample size decreases, what happens to the standard error and test statistic?

Answer 23

Standard error increases

Test statistic decreases

Question 24

When should we take inference from a hypothesis test?

Answer 24

When n>25, otherwise it is very hard to reject the null hypothesis and the power of the test is low

Question 25

When is the multiple linear regression used?

Answer 25

When one explanatory variable is insufficient to explain the variation of the dependent variable

Question 26

What will be the degrees of freedom when dealing with k+1 different parameters

Answer 26

n-1-k