Single Variable Linear Regression

Question 1

We use regression analysis for two primary purposes:

Answer 1

Studying the magnitude and structure of the relationship between two variables.
Forecasting a variable based on its relationship with another variable.

Question 2

The structure of the single variable linear regression line is

Answer 2

ŷ =a+bx

ŷ is the dependent variable
X is the independent variable (i.e. it is plotted on the X axis), the variable we are using to help us predict or better understand the dependent variable.
aa is the y-intercept, the point at which the regression line intersects the vertical axis.
bb is the slope, the average change in the dependent variable yy as the independent variable xx increases by one.

Question 3

What is a prediction interval?

Answer 3

an interval around the point forecast that is likely to contain, for example, the actual selling price of a house of a given size. The center of the prediction interval is the point forecast.

Question 4

R2 measures …

Answer 4

the percent of total variation in the dependent variable, yy , that is explained by the regression line.

Question 5

What does R2 equal in single variable linear regression?

Answer 5

For a single variable linear regression, R2 is equal to the square of the correlation coefficient.

Question 6

How do we test whether the relationship between the dependent and independent variable is significant and whether the linear model is a good fit for the data?

Answer 6

We do this by analyzing the p-value (or confidence interval) associated with the independent variable and the regression’s residual plot.

If the coefficient’s p-value is less than 0.05, we reject the null hypothesis and conclude that we have sufficient evidence to be 95% confident that there is a significant linear relationship between the dependent and independent variables.

Question 7

What is p-value?

Answer 7

The p-value of an independent variable is the result of the hypothesis test that tests whether there is a significant linear relationship; that is, it tests whether the slope of the regression line is zero,

Question 8

P-value vs R2?

Answer 8

p-value and R2 provide different information. A linear relationship can be significant (have a low p-value) but not explain a large percentage of the variation (not have a high R2.)

Question 9

What does a low P-value show?

Answer 9

That a linear relationship can be significant

Question 10

What does low R2 show?

Answer 10

a large percentage of the variation

Lower R2 (0.70): A smaller portion of the variation in yy is explained by the regression line than in the previous graph.

Question 11

What does confidence interval indicate?

Answer 11

A confidence interval associated with an independent variable’s coefficient indicates the likely range for that coefficient.

Question 12

When do we use dummy variables?

Answer 12

To perform regression analyses using qualitative, or categorical, variables. To do so, we must convert data to dummy (0, 1) variables. After that, we can proceed as we would with any other regression analysis.

Question 13

What do we use scatter plots for?

Answer 13

for visualizing a relationship between two variables.

Question 14

What does correlation coefficient measure?

Answer 14

It is a value between -1 and 1 that measures the strength and direction (positive or negative) of the linear relationship between two variables.

Question 15

What do we use to predict a value?

For example, we may want to predict the price of a house on the basis of its size. How much can we expect to pay for a 1,200 square foot home?

Answer 15

Point forecast;

y = A + BX

y - dependent value (e.g. selling price)

a - intercept (the value of y when x is 0)

B - slope (average change when it increases by 1)

X - independent variable (one we are using to help us predict dependent variable)

Question 16

How do you forecast with EXCEL?

Answer 16

Another quick way to forecast is to use Excel’s FORECAST function:
=FORECAST(x, known_y’s, known_x’s)

Question 17

How do you calculate prediction interval in excel?

Answer 17

=T.INV.2T(probability, degrees_freedom)

probability is 1–confidence level, so for a 95% prediction interval, we would enter 0.05.

degrees_freedom is the number of degrees of freedom, in this case, n–2, where n is the sample size.

Question 18

What happens to prediction interval when confidence level increases?

Answer 18

the width of the prediction interval increases.

Question 19

Forecasting in Excel

Answer 19

=SUMPRODUCT(array1, [array2], [array3],…) i

Question 20

Residual Sum of Squares =

Answer 20

The Residual Sum of Squares is the amount of variation that is left unexplained by the regression line, that is, the sum of the squared differences between the predicted and observed values.

Question 21

Total Sum of Squares =

Answer 21

The Total Sum of Squares is the variance of yy, that is, the total variation in yy. The Total Sum of Squares equals the sum of the squared differences between the observed values of yy and the mean of yy. That is exactly what the graph shows.

Question 22

What does R2 measure?

Answer 22

R2 measures how closely a regression line fits a data set. it’s defined as a % of total variation in the dependent variable.

Question 23

Earlier in this module, we found that the correlation coefficient between house size and selling price is 0.86. What is the R2 of the best fit line that describes the relationship between selling price and house size?

Answer 23

0.74
Remember that for a single variable linear regression, R2 is the square of the correlation coefficient. Here, the correlation coefficient is 0.86, so R2=0.862=0.74.

Question 24

What are Quantitative (numerical) variables?

Answer 24

Variables that can be counted or measured and that are naturally represented as numbers.

Question 25

What are Qualitative (categorical) variables?

Answer 25

Variables that can be sorted or grouped into categories. Qualitative variables must be transformed into dummy variables

Question 26

The regression output table is divided into three main parts:

Answer 26

Regression Statistics table, the ANOVA table, and the Regression Coefficients table

Question 27

AS X INCREASES, Y INCREASES

What Slope is it?

Answer 27

Positive Slope

Question 28

AS X INCREASES, Y DOES NOT CHANGE.

What Slope is it?

Answer 28

Zero Slope

Question 29

AS X INCREASES, Y DECREASES.

What Slope is it?

Answer 29

Negative Slope

Question 30

Given the general regression equation, ŷ =a+bxy^=a+bx , which of the following describes ŷ y^ ?

Answer 30

The expected value of y - CORRECT
The dependent variable- CORRECT
The value we are trying to predict - CORRECT

Question 31

When analyzing a residual plot, which of the following indicates that a linear model is a good fit?

Answer 31

Random spread of residuals around the x-axis

A linear model is a good fit if the residuals are spread randomly above and below the x-axis.

Question 32

What indicates for a regression line’s slope indicates that the linear relationship is NOT significant at the 5% level?

Answer 32

If it contains zero. E.g. The range between -9.85 and 5.26 contains zero, which indicates that the linear relationship is not significant at the 5% level.