W7 NN2 (eta optimization) Flashcards
1
Q
Different ways of using training data to train classifier?
A
Batch, mini batch, Stohastic Gradient
2
Q
What is problem with using same eta on all weights?
A
To much diversity in NN
3
Q
Running Average(exponenital moving average)
A
4
Q
What is ADAM?
A
Momentum rule, adaptive gradient
5
Q
Momentum rule?
A
6
Q
Adagrad
A
7
Q
Ways of regularization in NN
A
Weight decay, Early stopping, Pertubation, Dropout, Batch Normalization
8
Q
Batch normalization
A
