Part 7. Intro to Linear Regression

Question 1

Regression analysis

Answer 1

A tool for examining whether a variable is useful for explaining another variable.

i.e. whether earnings growth/cash flow growth helps explain the company’s value in marketplace.

Question 2

Sum of squares total (SST)

Answer 2

• A simpler exploration to try understand why each company’s ROA differs from mean ROA of 12.5%, we look at the sum of squared deviations of obs from mean to capture variations in return on assets (ROA) from their mean.

Question 3

Simple linear regression (SLR)

Answer 3

Method for relating dependent and independent variables through estimation of a relationship, where we have one independent variable.

Question 4

Multiple regression

Answer 4

Method for relating dependent and independent variables through estimation of a relationship, where we have more than one independent variable.

Question 5

Ordinary least squares (OLS) regression

Answer 5

The goal is to fit a line to observations on Y and X to minimise the squared deviations from line; using least squares criterion.

Question 6

Line of best fit

Answer 6

In simple linear regression, the estimate intercept, b0^, and slope b1^ are such that the sum of squared vertical distances from the observations to fitted line is minimised.

Question 7

Residual for ith observation, ei

Answer 7

This is how much the observed value of Yi differs from the Yi^ estimated using the regression line: ei = Yi - Yi^.

This refers to the true underlying population relationship, whereas the residual refers to fitted linear relation based on sample.

Question 8

Residuals

Answer 8

Represented by the vertical distances from the fitted line, therefore in the units of measurement represented by the dependent variable.

i.e. if dependent variable is in euros, the error term is in euros etc.

Question 9

Sample correlation, r

Answer 9

The ratio of the covariance to the product of the standard deviations.

Question 10

Slope

Answer 10

The change in dependent variable for one unit change in independent variable.

Question 11

Cross sectional regression

Answer 11

This involves many observations of X and Y for same time period, depending on regression model these observations could come from different companies, asset classes, investment funds etc.

Question 12

Time series

Answer 12

Use many observations from different time periods for the same company, asset class, investment fund, country or other entity depending on regression model.

i.e. monthly data from many years to test whether country’s inflation rate determine short term interest rates.

Question 13

Assumptions of simple linear regression model:

Answer 13

1. Linearity - the relationship between the dependent variable Y, and independent variable X is linear.
2. Homoscedasticity - the variance of regression residuals is the same for all observations.
3. Independence - the observations, pairs of Ys and Xs are independent of one another, implies regression residuals are uncorrelated across observations.
4. Normality - the regression residuals are normally distributed.

Question 14

Homoskedasticity

Answer 14

The variance of the residuals is the same for all observations.

Question 15

Heteroskedasticity

Answer 15

If residuals are not homoscedastic, if the variance of residuals differs across observations.

Question 16

Sum of squares regression (SSR):

Answer 16

The sum of the squared differences between the predicted value of the dependent variable, Yi^, based on the estimated regression line and mean of dependent variable Y-.

Question 17

Coefficient of determination (R^2):

Answer 17

The percentage of the variation of the dependent variable that is explained by the independent variable.

• measure used to evaluate goodness of fit.

Question 18

Standard error of estimate (se):

Answer 18

The absolute measure of distance between the observed values of the dependent variable, and those predicted from estimated regression.

The smaller the se, the better fit of the model.

Question 19

Measure of goodness fit of estimated regression:

Answer 19

1. Standard error estimate (se)

2. F-statistic

Question 20

Standard error of slope coefficient (sb1)

Answer 20

In simple linear regression, this is the ratio of the models standard error of the estimate (se) to the square root of the variation of the independent variable.

Question 21

Indicator/dummy variable

Answer 21

The case where it takes only the values 0 or 1 as the independent variable.

Question 22

Level of significance

Answer 22

Its always a matter of judgement.

• There is a 5% chance of rejecting H0, which is true (Type I error).
• Decreasing level of significance from 0.05 to 0.01 decreases the probability of Type 1 error, but also increases probability of Type 2 error - failing to reject H0, when it is false.

Question 23

p-value

Answer 23

The smallest level of significance at which H0 can be rejected.

• The smaller the p-value, the smaller the chance of Type 1 error (rejecting true H0), so greater likelihood the regression model is valid.

Question 24

The standard error of forecast depends on:

Answer 24

1. the standard error of the estimate, se.
2. the number of observations, n.
3. the forecasted value of independent variable, Xf, used to predict dependent variable and its deviation from estimated mean, X-
4. the variation of independent variable.