B10 Improving Performance

Question 1

The process of adjusting a model’s parameters to identify the best fit is called _____________.

Answer 1

Parameter tuning

Question 2

Automated parameter tuning requires you to consider:

Answer 2

1. What type of machine learning model (and specific
   implementation) should be trained on the data?
2. Which model parameters can be adjusted, and how
   extensively should they be tuned to find the optimal
   settings?
3. What criteria should be used to evaluate the models to find the best candidate?

Question 3

The technique of combining and
managing the predictions of multiple
models is known as ____________.

Answer 3

meta-learning

Question 4

_______ and ________ are statistics that evaluate the performance of Classification models, while _______ or ________ are used for numeric models.

Answer 4

Accuracy; Kappa

R-squared;RMSE

Question 5

Cost-sensitive measures such as _______, _______, and ___________ can also be used to evaluate performance.

Answer 5

sensitivity, specificity, AUC

Question 6

The meta-learning approach that
utilizes the principle of creating a varied
team of experts is known as an
_______.

Answer 6

ensemble

Question 7

The _________ dictates how much of the training data

each model receives

Answer 7

allocation function

Question 8

The __________ governs how disagreements among the predictions are reconciled

Answer 8

combination function

Question 9

Some ensembles utilize another model to learn a combination function from various combinations of predictions. This is known as _______.

Answer 9

stacking

Question 10

The two main families of ensemble methods are:

1.
2.

Answer 10

Averaging methods

Boosting methods

Question 11

Ensemble methods provide a number of performance
advantages over single models:
-___________ to future problems.
- Improved performance on _____ or _______
datasets.
- The ability to synthesize data from distinct
domains.
- A more nuanced understanding of difficult learning
tasks.

Answer 11

Better generalizability

massive or miniscule

Question 12

Independently built models with their predictions averaged or combined by a voting scheme. They attempt to reduce the _______ of a ________. Examples include _________ and _________.

Answer 12

variance
single base estimator
Bagging methods
Random Forest

Question 13

___________ or __________ is a
technique that generates a number of
training datasets by __________
sampling the original training data.

Answer 13

Bootstrap Aggregating
Bagging
Bootstrap

Question 14

In the Bagging process:
1. The training datasets are used to
generate a set of models using a
\_\_\_\_\_\_\_\_\_\_.
2. The models' predictions are
combined using \_\_\_\_\_\_\_ (for
classification) or \_\_\_\_\_\_\_ (for
numeric prediction).

Answer 14

single learner
voting
averaging

Question 15

The Random Forest (or Decision
Tree Forest) learner focuses only on
ensembles of decision trees. It
combines the base principles of
\_\_\_\_\_\_\_ with \_\_\_\_\_\_\_\_ to add additional diversity
to decision tree models.

Answer 15

bagging

random feature selection

Question 16

Strengths of Random Forest?

Answer 16

-Performs well on most problems.
-Handles noisy or missing data as
well as categorical or continuous
features.
-Selects only the most important
features.
-Works for data with an extremely
large number of features.

Question 17

Weaknesses of Random Forest?

Answer 17

-Unlike a decision tree, the
model is not easily
interpretable.
-May require some work to tune
the model to the data.
-Increased computational
complexity.

Question 18

Sequentially built models which are combined to produce a powerful ensemble are referred to as _________.

Answer 18

Boosting Methods

Question 19

Boosting methods attempt to reduce the _____ of the
________. Examples include AdaBoost and
Gradient Tree Boosting.

Answer 19

bias

combined estimator

Question 20

Boosting is a technique that
sequentially boosts the performance
of weak learners in order construct a
\_\_\_\_\_\_ classifier as a linear
combination of simple \_\_\_\_
classifiers

Answer 20

strong

weak

Question 21

At each iteration of the Boosting process:
1. The resampled datasets are
constructed specifically to generate
\_\_\_\_\_\_\_\_\_ learners.
2. Each learner's vote is \_\_\_\_\_\_\_\_\_\_\_
on its past performance

Answer 21

complementary

weight based

Question 22

The ______________ learner works by sequentially
adding weak models which are trained using weighted
training data.

Each model is assigned a stage value which corresponds to how _______ it is against the training data.

Answer 22

Adaptive Boosting

Accurate

Question 23

AdaBoost Advantages?

Answer 23

-Boosting is a relatively simple
ensemble method to implement.
-Requires less parameter tuning
compared to other ensemble
methods.
-Can be used with many different
classifiers.

Question 24

AdaBoost Weaknesses?

Answer 24

-High tendency to overfit with
many weak learners.
-Rather slow training time.
-Sensitive to noisy data and
outliers.

Question 25

The _____________ learner is an
implementation of _________ decision trees
designed specifically for speed and performance.

Answer 25

Extreme Gradient Boosting

gradient boosted

Question 26

With gradient boosting, instead of assigning weights to
models at each iteration, subsequent models attempt to
predict the _______ of prior models using a gradient
descent algorithm to __________.

Answer 26

residuals

minimize loss