Underfitting and Overfitting

Question 1

4 Reasons of Overfitting

Answer 1

Limited Training Data
Noisy Data
Model complexity
Low Regularisation

Question 2

2 Sign of Overfitting

Answer 2

Training error is low
Generalisation error is high

Question 3

3 Overfitting Remedy

Answer 3

Get more training data
Clean up the training data (noise).
Simplify or regularise the model

Question 4

3 Reason of Underfitting

Answer 4

The model is too simple to capture the underlying patterns
The model is constrained
Irrelevant data

Question 5

3 Underfitting Remedy

Answer 5

Feed better features to the learning algorithm
Reduce the constraints on the model
Select a more powerful model, with more parameters.

Question 6

2 Error Estimate for Model Selection

Answer 6

Optimistic Error Estimate - Not consider complexity
Pessimistic Error Estimate - Consider complexity

Question 7

Model Evaluation

Answer 7

To estimate performance of classifier on previously unseen data (test set)

Question 8

Holdout

Answer 8

Reserve k% for training and (100-k)% for testing.
Random subsampling - Repeated holdout

Question 9

Cross validation

Answer 9

Partition data into k disjoint subsets
K-fold - Train on k-1 partitions, test on the remaining one

Question 10

4 Step of Using Validation Set

Answer 10

Train multiple models with various hyper-parameters training set
Select model that performs best on validation set
Train the best model on the full training set
Evaluate final model on test set (estimate generalization error)

Question 11

2 Type of Fine Tune Model

Answer 11

Grid Search CV - Evaluate all combinations through a specified hyperparameter space.
Randomised Search CV - Evaluate a given number of random combinations