C2W1 Practical Aspects of Deep Learning

Question 1

Data should be divided into

Answer 1

Train (99%) / dev (0.5%) / test (0.5%)

Question 2

Bias/Variance

Answer 2

High bias - bad on train - algorithms fails to lear train data
High variance - good on train, bad on test.

Question 3

How to reduce bias

Answer 3

Bigger network / train longer

Question 4

How to reduce variance

Answer 4

More data / regularization

Question 5

Regularization L1 / L2

Answer 5

Just reduces weight of some weights, input parameter Alpha

Question 6

Dropout regularization

Answer 6

Drop random neurons during training on each sample. Do division of the remaining to adjust values. Don’t drop them when evaluating model.

Question 7

Other regularizations methods

Answer 7

Twist the image, early stopping (when dev error is optimal)

Question 8

Normalizing Inputs

Answer 8

Center the data with respect to the plot center (subtract the mean), x - mu
Reduce (normalize) the variance: x / sigma. Sigma is not an input parameter

Question 9

Weight Initialization for Deep Networks

Answer 9

There are different techniques, for example Xaviar initialization if activation function is tahn