Wronged Questions: Linear Models

Question 1

T/F: Error terms are considered to have a dimensionless measure.

Answer 1

False. The error term is not dimensionless. Since it is defined as ei = Yi - B0 - B1Xi, it has the same unit as the target variable.

Question 1

T/F: The error representation is based on the Poisson theory of errors.

Answer 1

False. The error representation is based on the Gaussian theory of errors. The error terms follow a Gaussian/normal distribution.

Question 2

T/F: Error terms are also known as disturbance terms.

Answer 2

True. The error representation is based on the Gaussian theory of errors. The error terms follow a Gaussian/normal distribution.

Question 3

T/F: Error terms are observable quantities.

Answer 3

False. There isn’t a specific definition for disturbance terms. The Frees text (page 31) states that error terms are also called disturbance terms.

Question 4

T/F: A model with a higher sum of squared errors has a higher total sum of squares compared to a model with lower sum of squared errors.

Answer 4

False. The total sum of squares for both models would be the same.

Question 5

T/F: The validation set approach is a special case of k-fold cross-validation.

Answer 5

False. Neither the validation set approach nor k-fold CV is a special case of each other.

Note that LOOCV is a special case of k-fold CV with k = n.

Question 6

T/F: The validation set approach is conceptually complex to implement.

Answer 6

False. The validation set approach is conceptually simple and easy to implement.

Question 6

T/F: Performing the validation set approach multiple times always yield the same results.

Answer 6

False. While performing LOOCV multiple times always yields the same results, this is not true for the validation set approach, where results vary due to randomness in the split.

Question 7

T/F: The validation error rate will tend to underestimate the test error rate.

Answer 7

False. One of the major drawbacks of the validation set approach is that it only uses the training dataset

LOOCV uses all the data so it doesn’t have this issue as much.

Question 8

T/F: The validation set approach has higher bias than leave-one-out cross-validation.

Answer 8

True. The LOOCV approach has lower bias than the validation set approach since almost all data is used in the training set, meaning it does not overestimate the test error rate as much as the validation set approach.

Question 9

T/F: The validation set approach is conceptually simple and straightforward to implement.

Answer 9

True

Question 10

T/F: The validation estimate of the test error rate can exhibit high variability, depending on the composition of observations in the training and validation sets.

Answer 10

True

Question 11

T/F: The model is trained using only a subset of the observations, specifically those in the training set rather than the validation set.

Answer 11

True

Question 12

T/F: Given that statistical methods typically perform worse when trained on fewer observations, this implies that the validation set error rate may tend to underestimate the test error rate for the model fitted on the entire dataset.

Answer 12

False

Question 13

T/F: The leverage for each observation in a linear model must be between 1/n and 1.

Answer 13

True

Question 14

T/F: The n leverages in a linear model must sum to the number of explanatory variables.

Answer 14

False. The leverages must sum to p+1, which is the number of predictors plus the intercept.

Question 15

T/F: If an explanatory variable is uncorrelated with all other explanatory variables, the corresponding variance inflation factor would be zero.

Answer 15

False. If an explanatory variable is uncorrelated with all other explanatory variables, the corresponding variance inflation factor would be 1.

Question 16

T/F: In best subset selection the predictors in the k-variable model must be a subset of the predictors in the (k+1)-variable model.

Answer 16

False. The predictors in the k-variable model do not need to be a subset of those in the (k+1)-variable model.

Question 17

T/F: In best subset selection, if p is the number of potential predictors, then 2^(p-1) models have to be fitted.

Answer 17

False. The correct number of models that need to be fitted is 2^p, not 2^(p-1).

Question 18

T/F: In best subset selection, the residual sum of squares of the k-variable model is always lower than that of the (k+1)-variable model.

Answer 18

False. The residual sum of squares for the k-variable model must be higher than or equal to that of the (k+1)-variable model (as long as the models are nested).

Question 19

T/F: In each step of best subset selection, the most statistically significant variable is dropped.

Answer 19

False. In each step of best subset selection, the least statistically significant variable is dropped.

Question 20

T/F: In high-dimensional settings, best subset selection is computationally infeasible.

Answer 20

True. In high-dimensional settings, the computational complexity of fitting all possible models makes best subset selection infeasible.

Question 21

se_b0

Answer 21

Question 22

se_b1

Answer 22

Question 23

se_hat(y) - used for estimators

Answer 23

Question 24

se_hat(y)_n+1

Answer 24

Question 25

Frees rule of thumb for identifying outliers

Answer 25

Observation is an outlier if the standardised residuals exceeds 2 in absolute value

Question 26

High leverage point

Answer 26

Observation that is unusual in the horizontal direction

Question 27

R^2 adj

Answer 27

Question 28

F statistic

Answer 28

Question 29

Variance-covariance matrix

Answer 29

(X^TX)^-1

Question 30

Mallow’s C_p

Answer 30

Question 31

AIC

Answer 31

-2ln(L)+2k

Question 32

BIC

Answer 32

kln(n)-2ln(L)

Question 33

Leverage formula

Answer 33

Question 34

Cook’s distance

Answer 34

Question 35

Breusch-Pagan test for heteroscedasticity

Answer 35

Question 36

LOOCV Error

Answer 36

Question 37

Centered variable

Answer 37

Variable resulting from subtracting the sample mean from the variable

Question 38

Scaled variable

Answer 38

Variable resulting from dividing a variable by its standard deviation

Question 39

Standardised variable

Answer 39

Variable resulting from first centering, then scaling the variable

Question 40

Ridge regression

Answer 40

Question 41

Lasso Regression

Answer 41

• performs variable selection
• yields more interpretable models

Question 42

Frees rule of thumb for high leverage points

Answer 42

[3(p+1)]/n

Question 43

Coefficient Matrix

Answer 43

Question 44

T/F: The best model by AIC will not also be the best model by 𝐶_p.

Answer 44

False. The best model by AIC will also be the best model by C_p.

Question 45

T/F: AIC, BIC, C_p, and R^2adj are not reliable when the model has been overfitted.

Answer 45

True

Question 46

List the cross validation techniques in order of least to most bias

Answer 46

LOOCV < k-fold < validation set

Question 47

List the cross validation techniques in order of most to least variance

Answer 47

LOOCV > k-fold > validation set

Question 48

F statistic using R^2

Answer 48

Question 49

Sum of Squares Regression (SSR)

Answer 49

Question 50

T/F: The standard error of the regression provides an estimate of the variance of y for a given x based on n-1 degrees of freedom.

Answer 50

False. The standard error of the regression provides an estimate of the variance of y for a given x based on n-2 degrees of freedom.

Question 51

T/F: In forward stepwise selection, if p is the number of potential predictors, then 2^p models have to be fitted.

Answer 51

False. Forward stepwise follows a more complex pattern. 2^p applies to best subset selection

Question 52

T/F: The predictors in the k-variable model must be a subset of the predictors in the (k+1)-variable model in forward stepwise selection.

Answer 52

True

Question 53

T/F: At each iteration, the variable chosen is the one that minimizes the test RSS based on cross-validation in forward stepwise selection.

Answer 53

False. At each iteration, the variable chosen is the one that minimizes the training RSS based on cross-validation in forward stepwise selection.

Question 54

T/F: Forward subset selection cannot be used even if the number of variables is greater than the number of observations.

Answer 54

False. Backward subset selection cannot be used if the number of variables is greater than the number of observations.

Question 55

T/F: The least squares line always passes through the point [bar(x), bar(y)].

Answer 55

True.

Question 56

T/F: The squared sample correlation between x and y is equal to the coefficient of determination of the model.

Answer 56

True

Question 57

T/F: The choice of explanatory variable x affects the total sum of squares.

Answer 57

False, because SST is not a function of x.

Question 58

T/F: The F-statistic of the model is the square of the t-statistic of the coefficient estimate for x.

Answer 58

True. This is true if both tests have the same set of hypotheses.

Question 59

T/F: A random pattern in the scatterplot of y against x indicates a coefficient of determination close to zero.

Answer 59

True

Question 60

Var(X+Y)

Answer 60

Var(X) + Var(Y) + 2Cov(X,Y)

Question 61

Var(X-Y)

Answer 61

Var(X) + Var(Y) - 2Cov(X,Y)

Question 62

T/F: As λ increases, the budget parameter increases.

Answer 62

False. An increase in λ actually corresponds to a decrease in the “budget” allowed for the coefficients’ magnitudes, not an increase.

Question 63

T/F: As λ decreases towards 0, the model becomes more biased.

Answer 63

False. As λ decreases towards 0, the model becomes less biased due to flexibility (and thus variance) increasing when λ decreases.

Question 64

T/F: Increasing the budget parameter decreases the variance of the model.

Answer 64

False. Increasing the budget parameter decreases λ, which results in an increase in variance.

Question 65

T/F: As λ decreases toward 0, the coefficient estimates become identical to those from ordinary least squares.

Answer 65

True. When λ is close to 0, the penalty effect diminishes, and the estimates approach those of the ordinary least squares, which has no penalty for coefficient size.

Question 66

T/F: A high λ value ensures that all coefficient estimates will be exactly zero.

Answer 66

False. A high λ value shrinks coefficients towards zero but does not ensure all are exactly zero

Question 67

T/F: Backward stepwise selection is computationally efficient compared to best subset selection.

Answer 67

True. Backward stepwise selection is more computationally efficient than best subset selection because it evaluates a smaller subset of models.

Question 68

T/F: Backwards stepwise selection cannot be used if the number of variables is greater than the number of observations.

Answer 68

True. Unlike the forward stepwise selection, backward stepwise selection cannot be used if the number of variables is greater than the number of observations.

Question 69

T/F: Backwards stepwise selection can be applied in a high-dimensional setting, unlike forward stepwise selection.

Answer 69

False. In high-dimensional settings, backward stepwise selection is not feasible, whereas forward stepwise selection remain feasible.

Question 70

T/F: At each step for backward selection, the variable to be dropped is the one whose absence causes the smallest decrease in the coefficient of determination.

Answer 70

True. The most statistically insignificant variable (the one whose absence causes the smallest decrease in the R^2) is dropped at each step of backward stepwise selection.

Question 71

T/F: If p is the number of potential predictors for backwards stepwise selection, then a total of 1+p(p+1)/2 models have to be fitted, which is the same as in the forward stepwise selection.

Answer 71

True.

Question 72

T/F: Shrinkage methods increase model rigidity, leading to better prediction accuracy when the rise in bias is offset by a larger reduction in variance.

Answer 72

True. Shrinkage methods, such as ridge and lasso, make the model less flexible than OLS, resulting in higher bias but lower variance. These methods improve prediction accuracy when the increase in bias is smaller than the decrease in variance.

Question 73

T/F: Shrinkage methods increase model rigidity, thus enhancing prediction accuracy when the rise in variance is offset by a larger reduction in bias.

Answer 73

False

Question 74

T/F: Shrinkage methods increase model flexibility, leading to better prediction accuracy when the rise in bias is offset by a larger reduction in variance.

Answer 74

False. They decrease model flexibility.

Question 75

T/F: Shrinkage methods increase model flexibility, thus enhancing prediction accuracy when the rise in variance is offset by a larger reduction in bias.

Answer 75

False

Question 76

T/F: The validation set approach tends to have higher bias in error estimates compared to k-fold cross-validation due to its dependency on a single split of the data.

Answer 76

True. The validation set approach may suffer from high bias if the validation split does not well-represent the entire dataset​​.

Question 77

T/F: LOOCV can be computationally expensive as it requires the model to be trained n times, where n is the number of observations.

Answer 77

True. LOOCV is computationally intensive because it involves training the model n times, each time leaving out a different single observation​​.

Question 78

T/F: K-fold cross-validation typically results in a better trade-off between bias and variance in error estimates compared to the validation set approach.

Answer 78

True. By repeatedly splitting the data into different subsets for training and validation, k-fold cross-validation provides a more robust estimate of model performance, striking a better balance between bias and variance in error estimates.

Question 79

T/F: With different random splits, the validation set approach generally provides more stable and less variable error estimates than k-fold cross-validation due to the larger size of the validation set.

Answer 79

False. With different random splits, the validation set approach can result in more variability and less stability in error estimates compared to K-fold CV, which benefits from averaging across multiple splits​​.

Question 80

T/F: K-fold cross-validation is generally less suitable for very small datasets compared to LOOCV, which uses all data points except one for training in each iteration.

Answer 80

True. LOOCV is particularly useful for small datasets because it maximally utilizes available data for training, while K-fold CV might not provide enough data in each training fold for very small datasets​​.

Question 81

T/F: Stepwise regression ensures the inclusion of all possible models, preventing the risk of data snooping.

Answer 81

False. Stepwise regression often involves data snooping, where fitting a large number of models to one set of data increases the chance of finding one that appears to fit well by chance.

Question 82

T/F: Stepwise regression automatically account for non-linear relationships and the presence of outliers and high leverage points in the data.

Answer 82

False. Stepwise regression typically does not account for non-linear relationships or the presence of outliers and high leverage points unless specifically designed to do so.

Question 83

T/F: Stepwise regression guarantees that the final model selected is the best among all possible models constructed from linear combinations of the predictors.

Answer 83

False. There’s no guarantee that the final model returned is the best one among all models that can be constructed, as only a subset of the possible models are considered.

Question 84

T/F: Stepwise regression relies on a variety of statistical measures, not just t-ratios, for determining which variables to add or remove.

Answer 84

False. Stepwise regression often relies heavily on t-ratios for adding or removing variables, rather than utilizing a broader array of statistical measures.

Question 85

T/F: Stepwise regression may overlook the best model because only a subset of all possible 2^k models are considered, where k is the number of predictors.

Answer 85

True. One of the primary drawbacks of stepwise regressions is that they consider only some of the 2^k possible models. Consequently, there’s a possibility that the best model, especially if it involves non-linear combinations of predictors or is otherwise outside the considered subset, might be missed.

Question 86

T/F: Not all multicollinearity problems can be detected by inspecting the correlation matrix.

Answer 86

True

Question 87

T/F: Severe multicollinearity reduces the accuracy of the estimates of the regression coefficients.

Answer 87

True

Question 88

T/F: The presence of multicollinearity always implies that the information provided by one variable is redundant in the presence of another variable.

Answer 88

False. The presence of multicollinearity does not always imply that the information is redundant. It is possible for two variables that are highly correlated to complement one another. This is the case of a suppressor variable, where a predictor increases the importance of another predictor.

Question 89

T/F: As a rule of thumb, when tolerance is less than 0.1 or 0.2, severe multicollinearity exists.

Answer 89

True. Tolerance is 1/VIF. Tolerances of 0.1 or 0.2 means that VIF is above 5 or 10

Question 90

T/F: The presence of severe multicollinearity makes it difficult to detect the importance of a variable.

Answer 90

True

Question 91

Two transformations that deal with heteroscedasticity

Answer 91

Logarithmic and square root transformation

Question 92

Correlation formula

Answer 92

Question 93

R^2 using covariance formula

Answer 93

Correlation formula ^2

Question 94

T/F: The standard least squares coefficient estimates are scale equivariant implying that regardless of how the j-th predictor is scaled, X_jB_j will remain the same.

Answer 94

True. On the other hand, ridge regression estimates are not scale equivariant. This is why scaling affects the result of ridge regression, and we often recommend variables to be scaled prior to performing ridge regression.

Question 95

T/F: Ridge regression’s advantage over least squares is rooted in the bias-variance trade-off because as the tuning parameter λ increases, the flexibility of the ridge regression fit increases.

Answer 95

False. As λ increases, flexibility decreases.

Question 96

As λ increases, the shrinkage of the ridge coefficient estimates leads to a substantial reduction in the variance of the predictions, at the expense of a slight increase in bias.

Answer 96

True. As λ increases, the flexibility of the ridge regression fit decreases, leading to decreased variance but increased bias.

Question 97

T/F: LOOCV is computationally prohibitive when used to validate least squares polynomial regression.

Answer 97

False. For polynomial or linear regression, LOOCV is computationally efficient because of the shortcut formula available. The shortcut formula enables the calculation of the estimated test error from just a single round of fitting, which makes the cost of LOOCV the same as a single model fit.

Question 98

T/F: k-fold cross-validation does not overestimate the test error rate as much as LOOCV.

Answer 98

False. k-fold CV, by using smaller training sets, tends to overestimate the test error more than LOOCV.

Question 99

T/F: 5-fold cross-validation requires more computational resources than 10-fold cross-validation.

Answer 99

False. 5-fold CV is more computationally efficient than 10-fold CV because it requires fewer runs.

Question 100

T/F: If the MSE of the full model containing all k predictors is an unbiased estimator of the true error variance, then Cp is an unbiased estimator of the test MSE.

Answer 100

True. The statement is about an unbiasedness property of Cp and reinforces the idea that selecting a model with a small Cp tends to lead to a model with a small test MSE.

Question 101

T/F: AIC and BIC are more general than Cp as they are applicable to linear, non-linear, and other general types of models fitted by maximum likelihood.

Answer 101

True.

Question 102

T/F: AIC and BIC provide an indirect estimate of the test error, while Cp provides a direct estimate of the test error.

Answer 102

False. All of the four model comparison statistics (adjusted R^2, Cp, AIC, BIC) aim to indirectly estimate the test error by adjusting the training error to account for model complexity. Direct estimation of test error can be achieved through resampling methods such as the validation set approach and cross-validation.

Question 103

T/F: Direct estimation of test error can be achieved through resampling methods such as the validation set approach and cross-validation.

Answer 103

True

Question 104

T/F: There are n-1 squared errors, one for each of the observation included in the fitting process for LOOCV.

Answer 104

False. There are n squared errors, one for each of the observation included in the fitting process for LOOCV.

Question 105

T/F: In the context of regression models with Gaussian errors, mallow’s Cp is an unbiased estimate of the test MSE if it is calculated using an unbiased estimate of σ^2.

Answer 105

True

Question 106

T/F: In the context of regression models with Gaussian errors, mallow’s Cp and Akaike information criterion are proportional to each other.

Answer 106

True

Question 107

T/F: In the context of regression models with Gaussian errors, a large value of Mallow’s Cp indicates a model with a high test error.

Answer 107

True

Question 108

T/F: Stepwise regression is designed to prioritize models based on non-linear combinations of predictors to accommodate complex relationships.

Answer 108

False. Stepwise regression does not consider models based on non-linear combinations of predictors, focusing instead on linear relationships.

Question 109

T/F: Stepwise regression effectively incorporates external knowledge or insights an investigator may have about the data.

Answer 109

False

Question 110

T/F: Stepwise regression guarantees that the model selected will not be influenced by outliers or high leverage points in the data.

Answer 110

False

Question 111

T/F: Stepwise regression primarily relies on t-ratios for adding or removing variables, which may not always be the most appropriate criterion.

Answer 111

True

Question 112

Studentised residuals

Answer 112

Question 113

Standardised residuals formula

Answer 113

Question 114

T/F: The residuals versus fitted values plot can be used to detect multicollinearity.

Answer 114

False

Question 115

T/F: Since irrelevant variables lead to unnecessary complexity in the resulting model, we can obtain a more easily interpretable model by setting the corresponding coefficient estimates to zero.

Answer 115

True

Question 116

T/F: The least squares approach is unlikely to yield any coefficient estimates that are precisely zero.

Answer 116

True

Question 117

T/F: Best subset selection automatically selects features or variables by excluding irrelevant variables from a multiple regression model.

Answer 117

True

Question 118

T/F: Partial Least Squares (PL) is a dimension reduction method.

Answer 118

True. Like principal components analysis (PCA), PLS is a dimension reduction method.

Question 119

T/F: After standardizing the predictors, PLS computes the first direction by setting its loadings to the coefficients from the simple linear regression of the response onto each original predictor.

Answer 119

True

Question 120

T/F: PLS identifies new features in an unsupervised way by approximating the original predictors, similar to principal components analysis.

Answer 120

False. Unlike principal components regression (PCR), PLS identifies new features in a supervised way; it uses the target variable to create new features that not only approximate the old features well, but also that are related to the target variable.

Question 121

T/F: In computing the first direction, PLS places the highest weight on the variables that are most strongly related to the response.

Answer 121

True. Since the slope coefficients for each simple linear regression are used for the first direction, a larger value indicates a stronger relationship with the target variable.

Question 122

T/F: The loadings for the first direction are proportional to the covariances between the response and each standardized predictor.

Answer 122

True. The loadings for the first direction are proportional to the correlations between the response and each predictor. Since the predictors are standardized, the correlations and the covariances are proportional to each other. Hence, the loadings are proportional to the covariances as well.

Question 123

T/F: Relationships among model deviations in SLR indicate a model misspecification issue.

Answer 123

True

Question 124

T/F: One option to handle heteroscedasticity in SLR is to use a logarithmic transformation of the dependent variable.

Answer 124

True

Question 125

T/F: If model deviations are associated with a variable, utilizing this information should enhance model specification in SLR.

Answer 125

True

Question 126

T/F: Departure from normality in the distribution of the deviations is a sign of model misspecification issue in SLR.

Answer 126

True

Question 127

T/F: K-fold CV entails randomly partitioning the observations into k groups, or folds, each of approximately equal size.

Answer 127

True

Question 128

T/F: One fold is designated as the validation set, while the model is trained on the remaining k folds.

Answer 128

False. One fold is designated as the validation set, while the model is trained on the remaining k-1 folds.

Question 129

T/F: The validation process is iterated k times, with a different fold of observations serving as the validation set each time, yielding k estimates of the test error.

Answer 129

True

Question 130

T/F: The k-fold cross-validation error estimate is determined by averaging the resulting mean squared error values obtained from the k validation iterations.

Answer 130

True

Question 131

T/F: Linear regression has the advantage of easy fitting by estimating only a small number of coefficients.

Answer 131

True

Question 132

T/F: Given a value for K and a prediction point x0, KNN regression identifies the K training observations that are closest to x0, and then estimates f(x0) using the distance-weighted average of all these K training responses.

Answer 132

False. KNN regression estimates f(x0) using the average of all the K training responses, not distance-weighted average.

Question 133

T/F: Ridge regression is less flexible than OLS and thus results in an improved prediction accuracy when its increase in bias is less than its decrease in variance.

Answer 133

True

Question 134

T/F: The linear model offers distinct advantages in inference and often proves to be surprisingly competitive with non-linear methods in addressing real-world problems when the true relationship between the predictor and response is approximately linear.

Answer 134

True

Question 135

T/F: If the true relationship between the target variable and the predictors is approximately linear, then the least squares estimates should be accurate, hence exhibiting low bias.

Answer 135

True

Question 136

T/F: If the number of observations is much larger than the number of variables, then the least squares estimates tend to have low variance and will perform well on test data when the true relationship between the target variable and the predictors is approximately linear.

Answer 136

True

Question 137

T/F: If the number of observations is not much larger than the number of variables, then there can be a lot of variability in the least squares fit, resulting in overfitting and poor predictions on unseen test observations when the true relationship between the target variable and the predictors is approximately linear.

Answer 137

True

Question 138

T/F: If the number of observations is less than the number of variables, then there is a unique least squares coefficient estimate when the true relationship between the target variable and the predictors is approximately linear.

Answer 138

False. If the number of variables is greater than the number of observations, then there is no longer a unique least squares coefficient estimate: the variance is infinite so the method cannot be used at all.

Question 139

T/F: The lasso tends to perform better than ridge regression when the response variable is a function of many predictors.

Answer 139

False. Ridge regression that tends to perform better when many predictors influence the response variable.

Question 140

T/F: Ridge regression has the advantage of forcing coefficient estimates to exactly zero.

Answer 140

False. Only Lasso can force coefficients to exactly zero.

Question 141

T/F: The penalty function in the lasso is a function of the l2
norm of the coefficients.

Answer 141

False. The lasso penalty is based on the l1 norm, not the l2 norm. In contrast, ridge penalty is based on the l2 norm.

Question 142

T/F: It is best to standardize predictors when using either ridge regression or the lasso.

Answer 142

True. Standardizing predictors ensures that the shrinkage penalty is applied uniformly across all predictor coefficients, which is important for both ridge regression and the lasso.

Question 143

T/F: The lasso tends to yield models that have better prediction accuracy than ridge regression.

Answer 143

False. Whether the lasso or ridge regression yields better prediction accuracy depends heavily on the specific dataset and situation.

Question 144

T/F: Studentised residuals in MLR should be realizations of a t-distribution.

Answer 144

True

Question 145

T/F: Studentised residuals in MLR should be realizations of a normal distribution.

Answer 145

False. Studentised residuals in MLR should be realizations of a t distribution.

Question 146

T/F: Studentised residuals in MLR have the same unit as the response variable.

Answer 146

False. Studentized residuals are comparable across different contexts because they are unitless.

Question 147

T/F: An observation with a negative studentized residual is likely an outlier.

Answer 147

False. A likely outlier is indicated by the magnitude of the studentized residual, not its sign.

Question 148

T/F: An observation with a large studentized residual is likely a high leverage point.

Answer 148

False. A high leverage point is identified using leverage, not studentized residual.

Question 149

(Large/small) R^2 and (big/small) t-statistics may suggest the presence of multicollinearity in multiple linear regression model.

Answer 149

Large R^2 and small t-statistics

Question 150

T/F: As K increases the test error usually decreases initially and then starts to increase.

Answer 150

True

Question 151

T/F: R^2 measures the linear relationship between y and the predictions hat(y).

Answer 151

True

Question 152

T/F: The training residual sum of squares will steadily decrease as λ increases for Lasso.

Answer 152

False. As λ increases the training error will increase because the flexibility has decreased.

Question 153

T/F: As λ increases for Lasso, the test residual sum of squares will remain constant.

Answer 153

False. The test error decreases initially, and then increases, following a U shape.

Question 154

T/F: As λ increases for Lasso, the variance will increase initially, and then finally decrease, following an inverted U shape.

Answer 154

False. The variance will steadily decrease.

Question 155

T/F: As λ increases for Lasso, the squared bias will steadily increase.

Answer 155

True.

Question 156

T/F: As λ increases for Lasso, the irreducible error will decrease initially, and then finally increase, following a U shape.

Answer 156

False. The irreducible error never changes.

Question 157

T/F: The validation set approach involves randomly dividing the available set of observations into two parts, a training set and a validation set.

Answer 157

True

Question 158

T/F: In validation set approach, the model is fit on the training set, and the validation error is estimated by applying the model to predict responses for the observed data in the validation set.

Answer 158

True

Question 159

T/F: The validation set approach can give highly variable results if the size of the validation set is not large enough.

Answer 159

True

Question 160

T/F: Validation set approach is typically repeated several times with different training-validation splits to reduce variability in the validation error estimate.

Answer 160

True

Question 161

T/F: The validation set error rate is skewed because the model has been optimized for the validation set.

Answer 161

False. The validation set error rate is not skewed because the model is fitted to the training set, not the validation set.

Question 162

T/F: LOOCV uses a single observation from the dataset for the validation each time, which leads to high variance in the error estimates.

Answer 162

True

Question 163

T/F: The estimate for test MSE in LOOCV is the average of the squared errors from each single validation.

Answer 163

True

Question 164

T/F: As λ increases, the sum of bj^2 increases for ridge regression.

Answer 164

False. The sum of bj^2 must decreases to minimise the SSE penalty. It is still possible for INDIVIDUAL bj’s to increase in absolute value as λ increases.

Question 165

T/F: As λ increases, it is not possible for an individual bj
to increase in absolute value for ridge regression.

Answer 165

False. It is still possible for INDIVIDUAL bj’s to increase in absolute value as λ increases.