7.2 - Supervised Learning Algorithms Flashcards
______ _______ models reduce the problem of ______ by imposing a penalty based on the number of features (i.e., _________ variables) used by the model.
Penalized regression;
overfitting;
independent;
In a penalized regression model, the penalty value _________ with the number of independent variables (i.e., ______) used.
increases;
features;
Imposing a penalty on the number of features makes a model more _________.
parsimonious;
Penalized regression models seek to minimize the sum __ _____ _____ (___), as well as the magnitude of the _____ _____.
of squared errors (SSE);
penalty value;
LASSO stands for what?
Least absolute shrinkage and selection operator.
In addition to minimizing SSE, LASSO minimizes what?
The sum of the absolute values of the slope coefficients.
LASSO automatically eliminates what?
The least predictive features of a regression model.
In a LASSO model, the penalty term, referred to by the Greek letter ______, is the __________ that determines the balance between _________ the model and keeping it _________.
lambda;
hyperparameter;
overfitting;
parsimonious;
In a LASSO model, you want both the _____ and _______ (the value of the penalty) to be low.
SSE;
lambda;
A method related to LASSO that is used to reduce statistical variability in a high dimension data estimation problem is referred to as __________. This method forces the ______ ________ of nonperforming features toward _______.
regularization;
beta coefficients;
zero;
What does SVM stand for?
support vector machine
SVM is a linear classification algorithm that does what?
SVM separates the data into one of two possible classifiers (e.g., sell vs. buy).
What does KNN stand for?
K-nearest neighbor.
KNN is more commonly used in _________ (but sometimes in ________).
classification;
regression;
KNN is used to classify an observation based on _________ to the observations in the __________ ________.
nearness;
training sample;
In KNN, the researcher specifies the value of __, the __________, triggering the algorithm to look for the __ _________ in the sample that are closest to the new observation that is being _____________.
k;
hyperparameter;
k observations;
classified;
As an example of KNN, if k=5, the algorithm will look for what?
The 5 nearest neighbors i.e., the 5 most similar observations).
What are some investment applications of KNN?
Predicting bankruptcy; assigning a bond to a ratings class; predicting stock prices; creating customized indices;
What does CART stand for?
Classification and regression trees.
Classification trees (aka “____”) are appropriate when the ______ _____ is __________, and are typically used when the target is ________ (e.g., an IPO will be successful vs. not successful).
CART;
target variable;
categorical;
binary;
Classification trees contain two types of _____: (1) __________ ____ and (2) ________ _____. They repeatedly divide the data until a _______ _____ is reached becomes the end of each branch.
nodes;
decision nodes;
terminal nodes;
terminal;
What are some investment applications of CART?
detecting fraudulent financial statements (e.g., these are likely manipulators vs. these are likely not manipulators); selecting stocks and bonds;
________ _______ is the technique of combining predictions from multiple models rather than from a single model.
Ensemble learning;
The purpose of using _______ ________ models is that an individual model will have a certain error rate and will make _____ _________. But by ________ predictions from many models, you can reduce the _______.
ensemble learning;
noisy predictions;
averaging;
noise;
