Ch 8 Notes

Question 1

8.1 Regression Models with Interaction Variables

Sample regression equation

Answer 1

𝑦̂=𝑏_0+𝑏_1 𝑥_1+𝑏_2 𝑥_2+…+𝑏_𝑘 𝑥_𝑘.

Question 2

8.1 Regression Models with Interaction Variables

Sometimes natural for the partial effect to depend on a predictor variable

Ex: Price of House

Answer 2

-Assumed that an additional bedroom results in the same increase in the house price regardless of the square footage

-Assumption may be unrealistic for larger houses, an additional bedroom often results in a higher increase in house prices

-Interaction effect between the number of bedrooms and square footage of the house

Question 3

8.1 Regression Models with Interaction Variables

Interaction Variables

Answer 3

-Capture effects by incorporating interaction variables in regression model

-Product of two interacting predictor variables

Question 4

8.1 Regression Models with Interaction Variables

Interaction Effect

Answer 4

Occurs when the partial effect of a predictor variable on the response depends on the value of another predictor variable

Question 5

8.1 Regression Models with Interaction Variables

Three types of interaction variables

Answer 5

1. Interaction between two dummy variables
2. Interaction between a dummy variable and numerical variable
3. Interaction between two numerical variables

Question 6

8.1: Regression Models with Interaction Variables

Regression Model with two dummy variables 𝑑_1 and 𝑑_2, and an interaction variable 𝑑_1 𝑑_2

𝑦= 𝛽_0+𝛽_1 𝑑_1+𝛽_2 𝑑_2+𝛽_3 𝑑_1 𝑑_2+𝜀

Answer 6

Estimated model: 𝑦̂=𝑏_0+𝑏_1 𝑑_1+𝑏_2 𝑑_2+𝑏_3 𝑑_1 𝑑_2

**Partial Effect of 𝑑_1 on 𝑦̂ is 𝑏_1+𝑏_3 𝑑_2, this depends on 𝑑_2.

𝑑_2=0: the partial effect of 𝑑_1 on 𝑦̂ is 𝑏_1

𝑑_2=1: the partial effect of 𝑑_1 on 𝑦̂ is 𝑏_1+𝑏_3

**The partial effect of 𝑑_2 on 𝑦̂ is 𝑏_2+𝑏_3 𝑑_1, this depends on 𝑑_1.

𝑑_1=0: the partial effect of 𝑑_2 on 𝑦̂ is 𝑏_2

𝑑_1=1: the partial effect of 𝑑_2 on 𝑦̂ is 𝑏_2+𝑏_3

Question 7

8.1: Regression Models with Interaction Variables

Regression Model with a numerical value 𝑥, a dummy variable 𝑑, and an interaction variable 𝑥𝑑

𝑦= 𝛽_0+𝛽_1 𝑥+𝛽_2 𝑑+𝛽_3 𝑥𝑑+𝜀

Answer 7

The estimated model is 𝑦̂=𝑏_0+𝑏_1 𝑥+𝑏_2 𝑑+𝑏_3 𝑥𝑑.

**The partial effect of 𝑥 on 𝑦̂ is 𝑏_1+𝑏_3 𝑑, this depends on 𝑑.

𝑑=0: the partial effect of 𝑥 on 𝑦̂ is 𝑏_1

𝑑=1: the partial effect of 𝑥 on 𝑦̂ is 𝑏_1+𝑏_3

**The partial effect of 𝑑 on 𝑦̂ is 𝑏_2+𝑏_3 𝑥, this depends on 𝑥.

Difficult to interpret because 𝑥 is numerical

Common to interpret this partial effect at the sample mean 𝑥̅

Question 8

8.1: Regression Models with Interaction Variables

Tegression model with two numerical variables, 𝑥_1 and 𝑥_2, and an interaction variable 𝑥_1 𝑥_2

𝑦= 𝛽_0+𝛽_1 𝑥_1+𝛽_2 𝑥_2+𝛽_3 𝑥_1 𝑥_2+𝜀

Answer 8

The estimated model is 𝑦̂=𝑏_0+𝑏_1 𝑥_1+𝑏_2 𝑥_2+𝑏_3 𝑥_1 𝑥_2.

The partial effect of 𝑥_1 on 𝑦̂ is 𝑏_1+𝑏_3 𝑥_2, this depends on 𝑥_2.

The partial effect of 𝑥_2 on 𝑦̂ is 𝑏_2+𝑏_3 𝑥_1, this depends on 𝑥_1.

The partial effects of both variables are difficult to interpret.

**Consider the partial effects at the sample means 𝑥̅_1 and 𝑥̅_2.

At 𝑥̅_1, the partial effect of 𝑥_2 on 𝑦̂ is 𝑏_2+𝑏_3 𝑥̅_1.

𝑏_3>0: the partial effect on 𝑦̂ will be greater (smaller) at values higher (lower) than〖 𝑥̅〗_1

Question 9

8.2 Regression Models for Nonlinear Relationships

Linear Regression

Answer 9

Often justified on the the basis of its computational simplicity

Question 10

8.2 Regression Models for Nonlinear Relationships

Implication of a simple linear regression model

Answer 10

If 𝑥 goes up by one unit, then the expected 𝑦 changes by 𝛽_1 regardless of 𝑥.

In many applications, the relationship between the variables cannot be represented by a straight line

Question 11

8.2 Regression Models for Nonlinear Relationships

Linearity Assumption

Answer 11

Places the restriction of linearity on the parameters and not on the variables

Question 12

8.2 Regression Models for Nonlinear Relationships

We can capture many interesting nonlinear relationships within the framework of the linear regression model

Answer 12

By simple transformations of the response and/or predictor variables

Question 13

8.2 Regression Models for Nonlinear Relationships

*In microeconomics, A firm’s average cost curve tends to be “U-Shaped”

Question 14

8.2 Regression Models for Nonlinear Relationships

Due to economies of scale, average cost cost 𝑦 of a firm

Answer 14

Initially decreases as output 𝑥 increases

As 𝑥 increases beyond a certain point, its impact on 𝑦 turns positive.

Other applications show the influence of the predictor variable initially positive but then turning negative, leading to an “inverted U-shape”

Question 15

8.2 Regression Models for Nonlinear Relationships

Quadratic regression model is appropriate when

Answer 15

Slope capturing the influence of 𝑥 on 𝑦, changes in magnitude as well as sign.

𝑦=𝛽_0+𝛽_1 𝑥+𝛽_2 𝑥^2+𝜀

Model can be extended to include multiple predictor variables

Question 16

8.2: Regression Models for Nonlinear Relationships

B_2 > 0

Answer 16

Slope up

Question 17

8.2: Regression Models for Nonlinear Relationships

B_2 < 0

Answer 17

Slope down

Question 18

8.2: Regression Models for Nonlinear Relationships

Use OLS to obtain

Answer 18

Sample regression equation
𝑦̂=𝑏_0+𝑏_1 𝑥+𝑏_2 𝑥^2

Cannot interpret 𝑏_1 in the usual way, focus on 𝑏_2.

The partial effect of 𝑥 on 𝑦̂ can be approximated by 𝑏_1+〖2𝑏〗_2 𝑥.

Question 19

8.2: Regression Models for Nonlinear Relationships

𝑦̂ reaches a maximum/minimum at

Answer 19

𝑥=(−𝑏_1)/(2𝑏_2 ).

Maximum when 𝑏_2<0

Minimum when 𝑏_2>0

Maximum/minimum reached when 2𝑏_2 is positive/negative

Question 20

8.2: Regression Models for Nonlinear Relationships

Use ____ to compare linear and quadratic models

Answer 20

R^2

Question 21

8.2: Regression Models for Nonlinear Relationships

Another common transformation that captures nonlinearity is based on

Answer 21

Natural Logarithim

Question 22

8.2: Regression Models for Nonlinear Relationships

Natural Logarithm

Answer 22

-Convert changes in a variable into percent changes

-Useful, many relationships are naturally expressed in terms of percentages

Ex: Income, House Prices, Sales

-Use intuitive and statistical measures to determine the appropriate form

Question 23

8.2: Regression Models for Nonlinear Relationships

In a log-log regression model

Answer 23

Both the response and predictor are transformed using natural logs:

ln⁡(𝑦)=𝛽_0+𝛽_1 ln⁡(𝑥)+𝜀

Relationship is a curve that depends on slope coefficient 𝛽_1

Question 24

8.2: Regression Models for Nonlinear Relationships

0 < B_1 < 0

Answer 24

Slope up

Question 25

8.2: Regression Models for Nonlinear Relationships

B_1 < 0

Answer 25

Slope down

Question 26

8.2: Regression Models for Nonlinear Relationships

𝛽_1 measures the approximate percentage change in 𝐸(𝑦) when

Answer 26

**When 𝑥 increases by 1%.

0<𝛽_1<1: positive relationship, 𝐸(𝑦) increases as a slower rate

𝛽_1>1: positive relationship, 𝐸(𝑦) increases as a faster rate

𝛽_1<0: negative relationship; 𝐸(𝑦) decreases as a slower rate

𝛽_1 is a measure of elasticity

Question 27

8.2: Regression Models for Nonlinear Relationships

Log Log Regression Model

Answer 27

Nonlinear in the variables but it is still linear in coefficients

Using an anti-log reverse transformation to each side has been known to underestimate the expected value of y

Predictions are made by 𝑦̂=𝑒𝑥𝑝(𝑏_0+𝑏_1 ln⁡(𝑥)+(𝑠_𝑒^2)⁄2), use unrounded coefficients.

Question 28

8.2: Regression Models for Nonlinear Relationships

Logarithmic regression model

Answer 28

Semi-log model that transforms only the predictor variable: y=𝛽_0+𝛽_1 ln⁡(𝑥)+𝜀

Attractive when only the predictor variable is better captured in percentages

𝛽_1∗0.01 measures the approximate change in 𝐸(𝑦) when 𝑥 increases by 1%.

Predictions are made by 𝑦̂=𝑏_0+𝑏_1 ln⁡(𝑥).

Question 29

8.2: Regression Models for Nonlinear Relationships

Exponential regression model

Answer 29

Semi log model that transforms only the response variable: : ln(y)=𝛽_0+𝛽_1 𝑥+𝜀.

𝛽_1∗100 measures the approximate percentage change in 𝐸(𝑦) when 𝑥 increases one unit

Predictions are made by 𝑦̂=𝑒𝑥𝑝(𝑏_0+𝑏_1 𝑥+(𝑠_𝑒^2)⁄2)

Question 30

8.2: Regression Models for Nonlinear Relationships

xy = B_0 + B_1X + E

Answer 30

Predicted Value: y^ = b_0 + b_1x

Estimated Slope coefficent: b_1 measures the change in y^ when x increases by one unit

Question 31

8.2: Regression Models for Nonlinear Relationships

ln(y) = B_0 + B_1ln(x) + e

Answer 31

Predicted Value: y^ = exp(b_0 + b_1ln(x) + s2e/2)

Estimated slope coefficient: b_1 measures the approximate percentage change in y^ when x increases by 1%

Question 32

8.2: Regression Models for Nonlinear Relationships

y = B_0 + B_1ln(x) + E

Answer 32

Predicted Value: y^ = b_0 + b_1ln(x)

Estimated Slope Coefficient: b_1 x 100 measures the approximate percentage change in y^ when x increases by one unit

Question 33

8.2: Regression Models for Nonlinear Relationships

ln(y) = B_0 + B_1x + E

Answer 33

Predicted Value: y^ = exp(b_0 + b_1x + s2e/2)

Estimated Slope Coefficient: b_1 x 100 measures the approximate percentage change in y^ when x increases by one unit

Question 34

8.2: Regression Models for Nonlinear Relationships

We cannot use adjusted 𝑅^2 to compare models that use 𝑦 as the response to models that use ln⁡(𝑦) as the response.

Compute the correlation between the observed and predicted values of each model: 𝑟_(𝑦𝑦̂ ).

Then use an alternative way to compute 𝑅^2=(𝑟_(𝑦𝑦̂ ) )^2.

Question 35

8.3 Cross-Validation Methods

All of the model evaluation measures thus far assess

Answer 35

predictability in the sample data that was used to build the model.

These measures do not help gauge how well an estimated model will predict an unseen sample

It is possible a model performs well with the sample data used for estimation, but then performs miserably once a new sample is evaluated

Question 36

8.3: Cross-Validation Methods

Overfitting occurs

Answer 36

**When an estimated model describes quirks of the data rather than the relationships between variables.

Model becomes too complex

Fails to describe the behavior in a new sample

Predictive power for new samples is compromised

Question 37

8.3: Cross-Validation Methods

Cross Validation

Answer 37

Measure of the predictive power of a model based on a set of data not used in estimation

**Technique that evaluates predictive models by partitioning the sample.

Training set: use to build/train the model

Validation set: use to evaluate/validate the mode

Use the root mean square error (RMSE) on the validation set.

𝑅𝑀𝑆𝐸=√(∑(𝑦_𝑖−𝑦̂_𝑖 )^2 /𝑛^∗ =√∑(𝑒_𝑖 )^2 /𝑛^∗

𝑦̂ is a true prediction for an observation in the validation set.

𝑛^∗ is the number of observations in the validation set.

RMSE will be lower for the training set than the validation set.

Question 38

8.3: Cross-Validation Methods

Holdout Methods

Answer 38

**Partitions the sample data set into two independent and mutually exclusive data sets.

Partition the sample into two parts: training and validation sets.

Use the training set to estimate competing models.

Use the estimates from the training set to predict the response in the validation set.

Calculate RMSE (or other performance measures) for each competing model. The preferred model will have the smallest RMSE.

We would like the model with the best performance in the training set to also have the best performance in the validation set

Question 39

8.3: Cross-Validation Methods

Conflicting Results are a sign of

Answer 39

overfitting using the training set