Chapter 9 Evaluate the Performance of Machine Learning Algorithms with Resampling

Question 1

WHAT ARE THE GENERAL WAYS OF EVALUATING AN ML ALGORITHM PERFORMANCE?

Answer 1

Make predictions for new data to which you already know the answers. (test set)
Use resampling methods that allow you to make accurate estimates for how well your algorithm will perform on new data. (cross validation)

Question 2

WHAT ARE FOUR DIFFERENT TECHNIQUES THAT WE CAN USE TO SPLIT UP OUR TRAINING DATASET AND CREATE USEFUL ESTIMATES OF PERFORMANCE FOR OUR MACHINE LEARNING ALGORITHMS?

Answer 2

ˆ Train and Test Sets.
ˆ k-fold Cross Validation.
ˆ Leave One Out Cross Validation.
ˆ Repeated Random Test-Train Splits.