7.2 - Supervised Learning Algorithms

Question 1

______ _______ models reduce the problem of ______ by imposing a penalty based on the number of features (i.e., _________ variables) used by the model.

Answer 1

Penalized regression;
overfitting;
independent;

Question 2

In a penalized regression model, the penalty value _________ with the number of independent variables (i.e., ______) used.

Answer 2

increases;

features;

Question 3

Imposing a penalty on the number of features makes a model more _________.

Answer 3

parsimonious;

Question 4

Penalized regression models seek to minimize the sum __ _____ _____ (___), as well as the magnitude of the _____ _____.

Answer 4

of squared errors (SSE);

penalty value;

Question 5

LASSO stands for what?

Answer 5

Least absolute shrinkage and selection operator.

Question 6

In addition to minimizing SSE, LASSO minimizes what?

Answer 6

The sum of the absolute values of the slope coefficients.

Question 7

LASSO automatically eliminates what?

Answer 7

The least predictive features of a regression model.

Question 8

In a LASSO model, the penalty term, referred to by the Greek letter ______, is the __________ that determines the balance between _________ the model and keeping it _________.

Answer 8

lambda;
hyperparameter;
overfitting;
parsimonious;

Question 9

In a LASSO model, you want both the _____ and _______ (the value of the penalty) to be low.

Answer 9

SSE;

lambda;

Question 10

A method related to LASSO that is used to reduce statistical variability in a high dimension data estimation problem is referred to as __________. This method forces the ______ ________ of nonperforming features toward _______.

Answer 10

regularization;
beta coefficients;
zero;

Question 11

What does SVM stand for?

Answer 11

support vector machine

Question 12

SVM is a linear classification algorithm that does what?

Answer 12

SVM separates the data into one of two possible classifiers (e.g., sell vs. buy).

Question 13

What does KNN stand for?

Answer 13

K-nearest neighbor.

Question 14

KNN is more commonly used in _________ (but sometimes in ________).

Answer 14

classification;

regression;

Question 15

KNN is used to classify an observation based on _________ to the observations in the __________ ________.

Answer 15

nearness;

training sample;

Question 16

In KNN, the researcher specifies the value of __, the __________, triggering the algorithm to look for the __ _________ in the sample that are closest to the new observation that is being _____________.

Answer 16

k;
hyperparameter;
k observations;
classified;

Question 17

As an example of KNN, if k=5, the algorithm will look for what?

Answer 17

The 5 nearest neighbors i.e., the 5 most similar observations).

Question 18

What are some investment applications of KNN?

Answer 18

Predicting bankruptcy; assigning a bond to a ratings class; predicting stock prices; creating customized indices;

Question 19

What does CART stand for?

Answer 19

Classification and regression trees.

Question 20

Classification trees (aka “____”) are appropriate when the ______ _____ is __________, and are typically used when the target is ________ (e.g., an IPO will be successful vs. not successful).

Answer 20

CART;
target variable;
categorical;
binary;

Question 21

Classification trees contain two types of _____: (1) __________ ____ and (2) ________ _____. They repeatedly divide the data until a _______ _____ is reached becomes the end of each branch.

Answer 21

nodes;
decision nodes;
terminal nodes;
terminal;

Question 22

What are some investment applications of CART?

Answer 22

detecting fraudulent financial statements (e.g., these are likely manipulators vs. these are likely not manipulators); selecting stocks and bonds;

Question 23

________ _______ is the technique of combining predictions from multiple models rather than from a single model.

Answer 23

Ensemble learning;

Question 24

The purpose of using _______ ________ models is that an individual model will have a certain error rate and will make _____ _________. But by ________ predictions from many models, you can reduce the _______.

Answer 24

ensemble learning;
noisy predictions;
averaging;
noise;

Question 25

In “________”, the ________ training set is used to generate “__” training data sets or “____” of data. Each new ___ is generated by ______ ________ with ___________. _________ helps to improve the stability of predictions and protects against overfitting the model.

Answer 25

bagging; 
original;
bags; 
random sampling;
replacement; 
Bagging:

Question 26

_______ _______ is a variant of classification trees whereby a large number of classification trees are trained using data __________ from the same data set.

Answer 26

Random forest;

bagged;