Linear Regression & Regression Diagnostics

Question 1

What is statistical modeling?

Answer 1

A process of using mathematical models to represent real-world data relationships.

It helps understand and predict outcomes.

Question 2

Define inferential modeling.

Answer 2

The use of statistical techniques to make predictions or inferences about a population based on a sample.

Common in regression analysis.

Question 3

What is linear regression?

Answer 3

A statistical method used to model the relationship between a dependent variable and one or more independent variables.

The equation is y = β₀ + β₁X + ε.

Question 4

What is simple linear regression?

Answer 4

A linear regression model with only one independent variable.

Example: Predicting house price based on square footage.

Question 5

What is multiple linear regression?

Answer 5

A linear regression model with two or more independent variables.

Example: Predicting sales based on advertising spend and product price.

Question 6

What does the slope in linear regression represent?

Answer 6

The rate at which the dependent variable changes with respect to the independent variable.

A positive slope indicates a positive relationship.

Question 7

Fill in the blank:

The y-intercept (β₀) represents __________.

Answer 7

The predicted value of y when x = 0.

Often not meaningful in some contexts.

Question 8

What is the main assumption of simple linear regression?

Answer 8

That there is a linear relationship between the dependent and independent variable.

Checked using scatter plots.

Question 9

What does the error term (ε) in regression represent?

Answer 9

The difference between the observed and predicted values.

Also known as residuals.

Question 10

What is the coefficient of determination (R²)?

Answer 10

A measure of how well the model explains the variance in the dependent variable.

R² values range from 0 to 1.

Question 11

What is Adjusted R²?

Answer 11

A modified R² that adjusts for the number of predictors in the model.

Unlike R², it penalizes adding unnecessary predictors.

Question 12

What is residual analysis in regression?

Answer 12

The process of analyzing the differences between observed and predicted values.

Helps detect model issues.

Question 13

What is a normal Q-Q plot used for?

Answer 13

Checking if residuals follow a normal distribution.

A straight 45-degree line suggests normality.

Question 14

What is homoscedasticity?

Answer 14

When residuals have constant variance.

Checked using residual vs. fitted plots.

Question 15

What is heteroscedasticity?

Answer 15

When residual variance is not constant.

Indicates a violation of homoscedasticity.

Question 16

How can you fix heteroscedasticity?

Answer 16

Apply transformations like:

• log, square root
• use weighted least squares regression.

Non-constant variance can distort regression results.

Question 17

What is multicollinearity?

Answer 17

When two or more independent variables are highly correlated.

Leads to unstable coefficient estimates.

Question 18

How do you detect multicollinearity?

Answer 18

Using Variance Inflation Factor (VIF).

VIF > 5 or 10 suggests high multicollinearity.

Question 19

How can you fix multicollinearity?

Answer 19

Remove one of the correlated variables, use PCA, or ridge regression.

Keeping highly correlated features can distort model interpretation.

Question 20

What is the normality assumption in regression?

Answer 20

Residuals should be normally distributed.

Checked using Q-Q plots or Shapiro-Wilk test.

Question 21

What is the Durbin-Watson test used for?

Answer 21

Detecting autocorrelation in regression residuals.

A value close to 2 suggests no autocorrelation.

Question 22

What is Cook’s Distance?

Answer 22

A measure to identify influential outliers.

Points with Cook’s Distance >1 may be problematic.

Question 23

What is Ridge Regression?

Answer 23

A regression technique that adds an L2 penalty to shrink coefficients.

Helps in handling multicollinearity.

Question 24

What is LASSO Regression?

Answer 24

A regression technique that adds a L1 penalty, shrinking some coefficients to zero.

Helps with feature selection.

Question 25

What is **Elastic Net Regression**?

Answer 25

A mix of **Ridge** and **LASSO** regression. ## Footnote Uses both L1 and L2 penalties.

Question 26

What is the purpose of **polynomial regression**?

Answer 26

To model a **non-linear relationship using higher-degree terms**. ## Footnote Example: y = β₀ + β₁X + β₂X² + ε.

Question 27

When should you use **logistic regression** instead of **linear regression?**

Answer 27

When predicting **a binary outcome**. ## Footnote Example: Predicting yes/no responses.

Question 28

What is **cross-validation** in regression?

Answer 28

A method to assess **how well the model generalizes to new data**. ## Footnote Common technique: k-fold cross-validation.

Question 29

How can **overfitting** be prevented in regression?

Answer 29

By using **regularization techniques** like *Ridge* or *LASSO*. ## Footnote Overfitting leads to poor generalization.

Question 30

What is the main goal of **regression diagnostics**?

Answer 30

To verify if **model assumptions are met**. ## Footnote Helps improve model reliability.

Question 31

What is the **F-test** used for in regression?

Answer 31

To check if **at least one predictor variabl**e is significantly contributing to the model. ## Footnote A low p-value (<0.05) suggests significance.

Question 32

What does a **small p-value** for a regression coefficient mean?

Answer 32

That the **predictor variable is significantly** contributing to the model. ## Footnote Typically, p < 0.05 is considered significant.

Question 33

What is **mean squared error (MSE)**?

Answer 33

The **average squared difference** between actual and predicted values. ## Footnote Lower MSE means better model performance.

Question 34

What is **root mean squared error (RMSE)**?

Answer 34

The **square root of MSE, measuring average prediction error in the same units as the dependent variable.** ## Footnote More interpretable than MSE.

Question 35

What does it mean if **residuals are large**?

Answer 35

The model’s predictions are **not** very accurate. ## Footnote Large residuals suggest potential outliers or a poor model fit.

Question 36

What is the **main assumption** of the least squares method?

Answer 36

That **residuals are normally distributed** and **have constant variance.** ## Footnote Used for estimating regression coefficients.

Question 37

Why should you be cautious about **extrapolating** in linear regression?

Answer 37

The model is **only valid** within the range of **observed data**. ## Footnote Predictions outside this range may be unreliable.

Question 38

What is **Bayesian regression**?

Answer 38

A type of regression that **incorporates prior beliefs** using probability distributions. ## Footnote Used when dealing with small datasets or uncertainty.

Question 39

What is **robust regression**?

Answer 39

A regression method that **reduces the influence of outliers.** ## Footnote More resistant to violations of normality and homoscedasticity.

Question 40

What is the main advantage of using **StatsModels** for regression?

Answer 40

Provides detailed **statistical summaries** and **diagnostics.** ## Footnote Offers better inference than Scikit-learn.