Linear Regression Flashcards
Backward Elimination
An iterative variable selection procedure that starts with a model with all independent variables and considers removing an independent variable at each step.
Best subsets
A variable selection procedure that constructs and compares all possible models with up to a specified number of independent variables.
Coefficient of determination
A measure of the goodness of fit of the estimated regression equation. It can be interpreted as the proportion of the variability in the dependent variable y that is explained by the estimated regression equation.
Confidence interval
An estimate of a population parameter that provides an interval believed to contain the value of the parameter at some level of confidence.
Confidence level
An indication of how frequently interval estimates based on samples of the same size taken from the same population using identical sampling techniques will contain the true value of the parameter we are estimating.
Cross-validation
Assessment of the performance of a model on data other than the data that were used to generate the model.
Dependent variable
The variable that is being predicted or explained. It is denoted by y and is often referred to as the response.
Dummy variable
A variable used to model the effect of categorical independent variables in a regression model; generally takes only the value zero or one.
Estimated regression
The estimate of the regression equation developed from sample data by using the least squares method.
Experimental region
The range of values for the independent variables x1, x2, . . . , xq for the data that are used to estimate the regression model.
Extrapolation
Prediction of the mean value of the dependent variable y for values of the independent variables x1, x2,… that are outside the experimental range.
Forward selection
an iterative variable selection procedure that starts with a model with no variables and considers adding an independent variable at each step.
Holdout method
Method of cross-validation in which sample data are randomly divided into mutually exclusive and collectively exhaustive sets, then one set is used to build the candidate models and the other set is used to compare model performances and ultimately select a model.
Hypothesis testing
The process of making a conjecture about the value of a population parameter, collecting sample data that can be used to assess this conjecture, measuring the strength of the evidence against the conjecture that is provided by the sample, and using these results to draw a conclusion about the conjecture.
Independent variable
The variable(s) used for predicting or explaining values of the dependent variable. It is denoted by x and is often referred to as the predictor variable.
Interaction
The relationship between the dependent variable and one independent variable is different at different values of a second independent variable.
Interval estimation
The use of sample data to calculate a range of values that is believed to include the unknown value of a population parameter.
K-fold cross-validation
Method of cross-validation in which sample data set are randomly divided into k equal sized, mutually exclusive and collectively exhaustive subsets. In each of k iterations, one of the k subsets is used to build a candidate model and the remaining k - 1 sets are used evaluate the candidate model.
Knot
The prespecified value of the independent variable at which its relationship with the dependent variable changes in a piecewise linear regression model; also called the breakpoint or the joint.
Least squares method
A procedure for using sample data to find the estimated regression equation.
Leave-one-out cross-validation
Method of cross-validation in which candidate models arerepeatedly fit using n - 1 observations and evaluated with the remaining observation.
Linear regression
Regression analysis in which relationships between the independent variables and the dependent variable are approximated by a straight line.
Multicollinearity
The degree of correlation among independent variables in a regression model.
Multiple linear regression
Regression analysis involving one dependent variable and more than one independent variable where the relationship is depicted by a flat hyperplane
Overfitting
Fitting a model too closely to sample data, resulting in a model that does not accurately reflect the population.
p-value
The probability that a random sample of the same size collected from the same population using the same procedure will yield stronger evidence against a hypothesis than the evidence in the sample data given that the hypothesis is actually true.
Parameter
A measurable factor that defines a characteristic of a population, process, or system.
Piecewise linear regression model
Regression model in which one linear relationship between the independent and dependent variables is fit for values of the independent variable below a prespecified value of the independent variable, a different linear relationship between the independent and dependent variables is fit for values of the independent variable above the prespecified value of the independent variable, and the two regressions have the same estimated value of the dependent variable (i.e., are joined) at the prespecified value of the independent variable.
Point estimator
A single value used as an estimate of the corresponding population parameter.
Quadratic regression modrl
Regression model in which a nonlinear relationship between the independent and dependent variables is fit by including the independent variable and the square of the independent variable in the model
Random variable
The outcome of a random experiment (such as the drawing of a random sample) and so represents an uncertain outcome.
Regression analysis
A statistical procedure used to develop an equation showing how the variables are related.
Regression model
The equation that describes how the dependent variable y is related to independent variables x_i and an error term
Residual
The difference between the observed value of the dependent variable and the value predicted using the estimated regression equation
Simple linear regression
Regression analysis involving one dependent variable and one independent variable.
Stepwise selection
an iterative variable selection procedure that considers adding an independent variable and removing an independent variable at each step.
T-test
Statistical test based on the Student’s t probability distribution that can be used to test the hypothesis that a regression parameter βj, is zero; if this hypothesis is rejected, we conclude that there is a regression relationship between the jth independent variable and the dependent variable.
Training set
The data set used to build the candidate models.
Validation set
The data set used to compare model forecasts and ultimately pick a model for predicting values of the dependent variable.
