Regression and Multiple Regression

Question 1

Regression

Answer 1

Regression estimates the relationship between one variable and a number of others. It is used for both description and prescription

Question 2

Steps

Answer 2

Scatterplots and Correlation analysis: feel for the direction and strength of the relationship

Model estimation: software

Diagnostic evaluation: evaluate validity and usefulness

Question 3

Simple Linear Regression

Answer 3

Tests relationships between 2 variables in a linear model
y = a + b x

Question 4

Residual Scatter

Answer 4

Residual (e): vertical distance of a point from a line. The difference between observed and predicted values of dependent variable.
Choose a line so that residual scatter is minimised

Question 5

R-Squared

Answer 5

In simple terms, R-squared tells you how well the independent variables in your model explain the variation in the dependent variable. However, it doesn’t tell you whether the coefficient estimates and predictions are biased, or whether the model is a good fit for the underlying data.

=1-s/sy

is a measure of how well the line fits the data
Standard error

The adjusted R squared penalises for low number of observations and high number of explanatory variables

S: the residual sum of squares (the sum of squared differences between the actual outcomes and the predicted outcomes)

Sy: represents the total sum of squares (the sum of squared differences between the actual outcomes and their mean).

Question 6

Residuals

Answer 6

residuals should be simple randomness remaining after deterministic part of variation in y has been modelled.

A non-random pattern would indicate there is something the model is not capturing that is leaking into the residuals.

Question 6

Testing Significance of variables

Answer 6

|t-stat| > 2 (1.96), reject the null hypothesis

t-stat = coefficient estimate/st.error

(p-val):
The probability of finding the observed relationship if the null hypothesis was true

Question 7

Good model

Answer 7

All coefficients: t-stat > 2, p-val<0.05

High adjusted R2

Satisfactory residuals

Equation makes sense

Question 8

Multiple Regression

Answer 8

a is the baseline
b is the increase in y resulting for a unit rise in x
Multicollinearity: A high degree of correlation between 2 or more explanatory variables
Cost is highly correlated with 1/Capacity and Years but these two are highly correlated with each other.
Including Years adds little information, hence model fit is no better