Chapter 5

Question 1

Backward Elimination

Answer 1

Backward elimination is a quantitative approach to identify the independent variable to include in a model. It starts with all the independent variables in the model first and each one is deleted one at a time if they are not significant. The process stops when all variables in the model are significant.

Question 2

Dependent Variable

Answer 2

The variable being predicted is referred to as the dependent (target) variable (Y).

Question 3

Dummy Coding

Answer 3

Dummy coding involves creating a dichotomous value from a categorical value.

Question 4

Feature Selection

Answer 4

Feature selection refers to identifying the optimal subset of features (independent variables) to explain a target variable. Feature selection can be done quantitatively or qualitatively.

Question 5

Forward Selection

Answer 5

Forward selection is a quantitative approach to identify the independent variable to include in a model. A separate regression model is created for each predictor and then variables are added one by one to determine which variables improve the model prediction.

Question 6

Independent Variable

Answer 6

The variables used to make the prediction are called independent variables (X) (also referred to as predictors or features).

Question 7

Linear Regression

Answer 7

Linear regression is a type of modeling that shows a relationship between the independent and dependent variables. It is represented by a straight line that best fits the data.

Question 8

Mean Absolute Error

Answer 8

Mean Absolute Error (MAE) measures the absolute difference between the predicted and actual values of the model.

Question 9

Mean Absolute Percentage Error

Answer 9

Mean Absolute Percentage Error (MAPE) is the percentage absolute difference the prediction is, on average, from the actual target.

Question 10

Multicollinearity

Answer 10

Multicollinearity is a situation where the predictor variables are highly correlated with each other.

Question 11

Multiple Regression

Answer 11

Multiple regression is used to determine whether two or more independent variables are good predictors of the single dependent variable.

Question 12

Numerical Variable

Question 13

Ordinary Least Squares

Answer 13

The ordinary least squares (OLS) regression method commonly referred to as linear regression minimizes the sum of squared errors.

Question 14

Overfitting

Answer 14

Overfitting occurs from an overly complex model where the results are limited to the data being used and are not generalizable—which means future relationships cannot be inferred, and results will be inconsistent when using other data.

Question 15

R^2

Answer 15

R^2 measures the amount of variance in the dependent variable that is predicted by the independent variable(s). The R^2 value ranges between 0 and 1, and the closer the value is to 1, the better the prediction by the regression model. When the value is near 0, the regression model is not a good predictor of the dependent variable.

Question 16

Regression Modeling

Answer 16

Regression modeling captures the strength of a relationship between a single numerical dependent or target variable, and one or more (numerical or categorical) predictor variables.

Question 17

Residuals

Answer 17

Residuals represent the difference between the observed and predicted value of the dependent variable.

Question 18

Root Mean Squared Error

Answer 18

Root Mean Squared Error (RMSE) indicates how different the residuals are from zero.

Question 19

Simple Linear Regression

Answer 19

Simple linear regression is used when the focus is limited to a single, numeric dependent variable and a single independent variable.

Question 20

Stepwise Selection

Answer 20

Stepwise selection is a quantitative approach to identify the independent variable to include in a model. Stepwise selection follows forward selection by adding a variable at each step, but also includes removing variables that no longer meet the threshold. The stepwise selection stops when the remaining predictors in a model satisfy the threshold to remain in the model.

Question 21

Test Data

Answer 21

A testing dataset is used to evaluate the final selection algorithm on a dataset unique from the training and validation datasets.

Question 22

Training Data

Answer 22

The training dataset is the data used to build the algorithm and “learn” the relationship between the predictors and the target variable.

Question 23

Validation Data

Answer 23

The validation data is used to assess how well the regression model estimates the target variable when compared to the actual values.