Regularization

Question 1

What is overfitting?

Answer 1

occurs when a model performs well on training data but not on new inputs

Question 2

What is dropout?

Answer 2

regularization technique that randomly removes nodes and their connections at every iteration, resulting in different sets of outputs and better generalization

Question 3

What problem does regularization solve in deep learning?

Answer 3

Improving model performance using training data AND new test data as the most common problem in deep learning is overfitting

Question 4

What are the different approaches to addressing overfitting?

Answer 4

1. Dropout
2. Augmentation
3. Early Stopping

Question 5

How does the dropout approach work?

Answer 5

At every iteration it randomly selects some nodes and removes them along with their incoming and outgoing connections

key points: each iteration has a different set of nodes and this results in a different set of outputs

Question 6

Why do dropout models perform better?

Answer 6

Captures more randomness and memorizes less of the training data which will lead to better generalizations and build a more robust predictive model

Question 7

How can dataset augmentation help train better models?

Answer 7

More data = better models and so by adding fake data you can synthesize more data by applying transformations on the existing dataset to synthesize more data

Question 8

What classification problem does dataset augmentation help the most with?

Answer 8

Object recognition because images are high dimensional and include an enormous range of factors of variation which many can be simulated

Question 9

Describe the concept of “early stopping”

Answer 9

Involves stopping training when the error on the validation set starts to increase, leading to a model with better validation and test set error