Linear Regression

Question 1

Backward Elimination

Answer 1

An iterative variable selection procedure that starts with a model with all independent variables and considers removing an independent variable at each step.

Question 2

Best subsets

Answer 2

A variable selection procedure that constructs and compares all possible models with up to a specified number of independent variables.

Question 3

Coefficient of determination

Answer 3

A measure of the goodness of fit of the estimated regression equation. It can be interpreted as the proportion of the variability in the dependent variable y that is explained by the estimated regression equation.

Question 4

Confidence interval

Answer 4

An estimate of a population parameter that provides an interval believed to contain the value of the parameter at some level of confidence.

Question 5

Confidence level

Answer 5

An indication of how frequently interval estimates based on samples of the same size taken from the same population using identical sampling techniques will contain the true value of the parameter we are estimating.

Question 6

Cross-validation

Answer 6

Assessment of the performance of a model on data other than the data that were used to generate the model.

Question 7

Dependent variable

Answer 7

The variable that is being predicted or explained. It is denoted by y and is often referred to as the response.

Question 8

Dummy variable

Answer 8

A variable used to model the effect of categorical independent variables in a regression model; generally takes only the value zero or one.

Question 9

Estimated regression

Answer 9

The estimate of the regression equation developed from sample data by using the least squares method.

Question 10

Experimental region

Answer 10

The range of values for the independent variables x1, x2, . . . , xq for the data that are used to estimate the regression model.

Question 11

Extrapolation

Answer 11

Prediction of the mean value of the dependent variable y for values of the independent variables x1, x2,… that are outside the experimental range.

Question 12

Forward selection

Answer 12

an iterative variable selection procedure that starts with a model with no variables and considers adding an independent variable at each step.

Question 13

Holdout method

Answer 13

Method of cross-validation in which sample data are randomly divided into mutually exclusive and collectively exhaustive sets, then one set is used to build the candidate models and the other set is used to compare model performances and ultimately select a model.

Question 14

Hypothesis testing

Answer 14

The process of making a conjecture about the value of a population parameter, collecting sample data that can be used to assess this conjecture, measuring the strength of the evidence against the conjecture that is provided by the sample, and using these results to draw a conclusion about the conjecture.

Question 15

Independent variable

Answer 15

The variable(s) used for predicting or explaining values of the dependent variable. It is denoted by x and is often referred to as the predictor variable.

Question 16

Interaction

Answer 16

The relationship between the dependent variable and one independent variable is different at different values of a second independent variable.

Question 17

Interval estimation

Answer 17

The use of sample data to calculate a range of values that is believed to include the unknown value of a population parameter.

Question 18

K-fold cross-validation

Answer 18

Method of cross-validation in which sample data set are randomly divided into k equal sized, mutually exclusive and collectively exhaustive subsets. In each of k iterations, one of the k subsets is used to build a candidate model and the remaining k - 1 sets are used evaluate the candidate model.

Question 19

Knot

Answer 19

The prespecified value of the independent variable at which its relationship with the dependent variable changes in a piecewise linear regression model; also called the breakpoint or the joint.

Question 20

Least squares method

Answer 20

A procedure for using sample data to find the estimated regression equation.

Question 21

Leave-one-out cross-validation

Answer 21

Method of cross-validation in which candidate models arerepeatedly fit using n - 1 observations and evaluated with the remaining observation.

Question 22

Linear regression

Answer 22

Regression analysis in which relationships between the independent variables and the dependent variable are approximated by a straight line.

Question 23

Multicollinearity

Answer 23

The degree of correlation among independent variables in a regression model.

Question 24

Multiple linear regression

Answer 24

Regression analysis involving one dependent variable and more than one independent variable where the relationship is depicted by a flat hyperplane

Question 25

Overfitting

Answer 25

Fitting a model too closely to sample data, resulting in a model that does not accurately reflect the population.

Question 26

p-value

Answer 26

The probability that a random sample of the same size collected from the same population using the same procedure will yield stronger evidence against a hypothesis than the evidence in the sample data given that the hypothesis is actually true.

Question 27

Parameter

Answer 27

A measurable factor that defines a characteristic of a population, process, or system.

Question 28

Piecewise linear regression model

Answer 28

Regression model in which one linear relationship between the independent and dependent variables is fit for values of the independent variable below a prespecified value of the independent variable, a different linear relationship between the independent and dependent variables is fit for values of the independent variable above the prespecified value of the independent variable, and the two regressions have the same estimated value of the dependent variable (i.e., are joined) at the prespecified value of the independent variable.

Question 29

Point estimator

Answer 29

A single value used as an estimate of the corresponding population parameter.

Question 30

Quadratic regression modrl

Answer 30

Regression model in which a nonlinear relationship between the independent and dependent variables is fit by including the independent variable and the square of the independent variable in the model

Question 31

Random variable

Answer 31

The outcome of a random experiment (such as the drawing of a random sample) and so represents an uncertain outcome.

Question 32

Regression analysis

Answer 32

A statistical procedure used to develop an equation showing how the variables are related.

Question 33

Regression model

Answer 33

The equation that describes how the dependent variable y is related to independent variables x_i and an error term

Question 34

Residual

Answer 34

The difference between the observed value of the dependent variable and the value predicted using the estimated regression equation

Question 35

Simple linear regression

Answer 35

Regression analysis involving one dependent variable and one independent variable.

Question 36

Stepwise selection

Answer 36

an iterative variable selection procedure that considers adding an independent variable and removing an independent variable at each step.

Question 37

T-test

Answer 37

Statistical test based on the Student’s t probability distribution that can be used to test the hypothesis that a regression parameter βj, is zero; if this hypothesis is rejected, we conclude that there is a regression relationship between the jth independent variable and the dependent variable.

Question 38

Training set

Answer 38

The data set used to build the candidate models.

Question 39

Validation set

Answer 39

The data set used to compare model forecasts and ultimately pick a model for predicting values of the dependent variable.