4. Multiple Linear Regressions

Question 1

What is the formula for calculating the T-statistic in coefficient testing?

Answer 1

t = β / SE(β), where β is the estimated coefficient and SE(β) is the standard error of the coefficient.

Question 2

How do you calculate R² in a regression model?

Answer 2

R² = SS_Reg / SS_Tot, where SS_Reg is the regression sum of squares and SS_Tot is the total sum of squares, representing the proportion of variance explained by the model.

Question 3

What is the formula for adjusted R², and why is it used?

Answer 3

R²_adj = 1 - ((n - 1) / (n - p)) * (SS_Res / SS_Tot), where n is the sample size, p is the number of predictors, SS_Res is the residual sum of squares, and SS_Tot is the total sum of squares. Adjusted R² accounts for the number of predictors, providing a more accurate measure of fit.

Question 4

How is the F-statistic calculated in an ANOVA table?

Answer 4

F = MS_Reg / MS_Res, where MS_Reg is the mean square of the regression and MS_Res is the mean square of the residuals, used to test the overall significance of the regression model.

Question 5

What does homoskedasticity mean in the context of a regression model?

Answer 5

Homoskedasticity means that the variance of the error terms is constant across all levels of the independent variables, a key assumption in regression analysis for unbiased estimates.

Question 6

How do you interpret a high R² in a multiple regression model?

Answer 6

A high R² indicates that a large proportion of the variance in the dependent variable is explained by the independent variables, suggesting a good fit for the model.

Question 7

Why is adjusted R² preferred over R² in multiple regression?

Answer 7

Adjusted R² adjusts for the number of predictors, providing a more accurate measure of fit by penalizing additional variables that don’t improve the model significantly.

Question 8

What does a T-statistic tell us about a regression coefficient?

Answer 8

The T-statistic tests whether a coefficient is statistically significantly different from zero, helping to determine if an independent variable has a meaningful impact on the dependent variable.

Question 9

How do you diagnose multicollinearity in a regression model?

Answer 9

Multicollinearity can be diagnosed by checking high Variance Inflation Factors (VIFs) for predictors, high correlations among predictors, or instability in coefficient estimates.

Question 10

How do you interpret a significant F-statistic in an ANOVA table for regression?

Answer 10

A significant F-statistic suggests that at least one of the independent variables significantly explains variance in the dependent variable, indicating an overall significant regression model.

Question 11

What is omitted variable bias in multiple regression?

Answer 11

Omitted variable bias occurs when a relevant variable is left out of the model, leading to biased and inconsistent estimates of the included coefficients.

Question 12

How does heteroskedasticity affect a regression model?

Answer 12

Heteroskedasticity, or non-constant variance of errors, can lead to inefficient estimates and unreliable standard errors, affecting the validity of hypothesis tests.

Question 13

How do you calculate the F-statistic for testing overall regression significance?

Answer 13

F* = MS_Reg / MS_Res, where MS_Reg is the mean square for the regression and MS_Res is the mean square of residuals, used to test if the model is statistically significant.

Question 14

How can you test for normally distributed errors in a regression model?

Answer 14

The normality of errors can be tested with visual tools like Q-Q plots or statistical tests such as the Shapiro-Wilk test, to validate regression assumptions.

Question 15

Why is the correlation of error terms important in regression?

Answer 15

Uncorrelated error terms are required for unbiased estimates. If errors are correlated, it suggests model misspecification or omitted variables, which may bias results.

Question 16

How would you interpret a T-statistic that is close to zero for a regression coefficient?

Answer 16

A T-statistic close to zero suggests that the coefficient is likely not statistically significant, indicating that the variable may have little or no linear impact on the dependent variable.

Question 17

How can multicollinearity be addressed in a multiple regression model?

Answer 17

Multicollinearity can be reduced by removing highly correlated predictors, combining correlated variables, or using regularization techniques like ridge or lasso regression.

Question 18

How do you interpret the results when both R² and adjusted R² values are low?

Answer 18

Low values for R² and adjusted R² indicate that the model explains very little of the variance in the dependent variable, suggesting either poor predictors or potential issues with model specification.

Question 19

How does one test for joint significance of regression coefficients using the F-statistic?

Answer 19

Use the F-statistic to test joint significance by comparing MS_Reg and MS_Res. A high F-value indicates that the set of predictors together significantly explains variation in the dependent variable, beyond what would be expected by chance.

Question 20

How would you detect and address heteroskedasticity in regression residuals?

Answer 20

Heteroskedasticity can be detected with tests like the Breusch-Pagan test or visualized through residual plots. To address it, one can use robust standard errors, transform the dependent variable, or apply weighted least squares regression.

Question 21

How do you calculate the predicted value of the dependent variable in a multiple linear regression model?

Answer 21

Use the equation Ŷ = β₀ + β₁X₁ + β₂X₂ + … + βpXp, where Ŷ is the predicted value, β₀ is the intercept, β₁, β₂, …, βp are the coefficients, and X₁, X₂, …, Xp are the predictor variables.

Question 22

How do you calculate the sum of squared residuals (SSR) in multiple linear regression?

Answer 22

SSR = Σ(Yi - Ŷi)², where Yi is the observed value and Ŷi is the predicted value from the model, summing over all observations.

Question 23

How do you interpret the coefficient βj in a multiple linear regression model?

Answer 23

βj represents the change in the dependent variable for a one-unit increase in the predictor Xj, holding all other predictors constant.

Question 24

What is the formula for calculating the variance of the residuals in multiple regression?

Answer 24

Variance of residuals = SSR / (n - p - 1), where SSR is the sum of squared residuals, n is the number of observations, and p is the number of predictors.

Question 25

How do you calculate the Variance Inflation Factor (VIF) for a predictor Xj?

Answer 25

VIFj = 1 / (1 - R²j), where R²j is the R² value obtained from regressing Xj on all other predictors. A high VIF indicates multicollinearity.

Question 26

How do you calculate the adjusted R² for a multiple regression model with p predictors?

Answer 26

Adjusted R² = 1 - [(n - 1) / (n - p - 1)] * (1 - R²), where n is the number of observations, p is the number of predictors, and R² is the unadjusted R² value.

Question 27

How do you calculate the standard error of a regression coefficient βj in multiple regression?

Answer 27

SE(βj) = √[σ² * (1 / (n - p - 1) * Σ(Xj - X̄j)²)], where σ² is the residual variance, Xj is the value of predictor j, and X̄j is the mean of predictor j.

Question 28

How do you calculate the F-statistic to test if a subset of k coefficients is jointly equal to zero?

Answer 28

F = [(SSRr - SSRur) / k] / (SSRur / (n - p - 1)), where SSRr is the sum of squared residuals from the restricted model, SSRur is from the unrestricted model, and k is the number of restrictions.

Question 29

How do you calculate the t-statistic for testing if a regression coefficient βj is significantly different from zero?

Answer 29

t = βj / SE(βj), where βj is the estimated coefficient and SE(βj) is its standard error.

Question 30

How do you interpret the partial correlation between a predictor Xj and the dependent variable Y?

Answer 30

The partial correlation measures the relationship between Xj and Y while controlling for the other predictors, showing how much of the variance in Y can be uniquely explained by Xj.

Question 31

How do you calculate the expected change in the dependent variable given changes in multiple independent variables?

Answer 31

ΔY = β₁ΔX₁ + β₂ΔX₂ + … + βₖΔXₖ, where each β represents the coefficient of the independent variable and ΔX represents the change in that variable.

Question 32

What is the formula for the T-statistic in coefficient testing?

Answer 32

t = β̂ / SE(β̂), where β̂ is the estimated coefficient and SE(β̂) is its standard error, used to test if the coefficient is significantly different from zero.

Question 33

How do you calculate the F-statistic for joint hypothesis testing?

Answer 33

F = MS_Reg / MS_Res, where MS_Reg is the mean square regression and MS_Res is the mean square residual, used to test if multiple coefficients are jointly significant.

Question 34

How is R² calculated, and what does it represent?

Answer 34

R² = SS_Reg / SS_Tot, where SS_Reg is the regression sum of squares and SS_Tot is the total sum of squares; it measures the proportion of variance in the dependent variable explained by the model.

Question 35

How is adjusted R² calculated, and why is it important?

Answer 35

R²_adj = 1 - ((n - 1) / (n - k - 1)) * (SS_Res / SS_Tot), where n is the number of observations, k is the number of predictors. Adjusted R² accounts for model complexity, penalizing for additional predictors.

Question 36

How do you calculate Mean Square Regression (MS_Reg) in an ANOVA table?

Answer 36

MS_Reg = SS_Reg / df_Reg, where SS_Reg is the regression sum of squares, and df_Reg is the degrees of freedom for regression.

Question 37

How do you calculate Mean Square Residual (MS_Res) in an ANOVA table?

Answer 37

MS_Res = SS_Res / df_Res, where SS_Res is the residual sum of squares, and df_Res is the degrees of freedom for residuals.

Question 38

How is the total sum of squares (SS_Tot) calculated in ANOVA?

Answer 38

SS_Tot = SS_Reg + SS_Res, where SS_Reg is the regression sum of squares, and SS_Res is the residual sum of squares.

Question 39

What does it mean if the variance of the error term is not constant (heteroskedasticity)?

Answer 39

If Var(εi) ≠ σ², heteroskedasticity is present, meaning the variance of errors changes with levels of the independent variables, affecting standard errors and inference.

Question 40

How do you calculate the standard error of the estimate?

Answer 40

SE = √(SS_Res / (n - k - 1)), where SS_Res is the residual sum of squares, n is the sample size, and k is the number of predictors, measuring the typical distance between observed and predicted values.

Question 41

What are robust standard errors, and when are they used?

Answer 41

Robust standard errors, like White and Newey-West standard errors, adjust for heteroskedasticity or autocorrelation in error terms, providing more reliable inferences when standard assumptions are violated.

Question 42

What is the formula for the vector of estimated coefficients in multiple linear regression?

Answer 42

β̂ = (XᵀX)⁻¹ XᵀY, where β̂ is the vector of estimated coefficients, Xᵀ is the transpose of the matrix of predictors, (XᵀX)⁻¹ is the inverse of the product of Xᵀ and X, and Y is the vector of observed values.

Question 43

What are the dimensions of the matrices Y, X, β, and e in multiple linear regression?

Answer 43

In a regression model with n observations and p predictors:

Y is an (n x 1) vector of observed values.
X is an (n x p’) matrix of predictors (where p’ = p + 1, including the intercept).
β is a (p’ x 1) vector of coefficients.
e is an (n x 1) vector of residuals (errors).