05 Deep learning methods*

Question 1

categorical encodings

Answer 1

1. Integer
   - red = 1, green=2, blue=3…
2. dummy
   - red= 00
   - green = 01
   - blue=10 …
3. one hot
   - table format
   - r, g, b
   1,0,0
   0, 1, 0

Question 2

k class classification

Answer 2

• use softmax to calculate probability
• use cross entropy to calculate loss

Question 3

stochastic gradient descent

Answer 3

• random starting weights
• attempt to find global minimal error
• slower (but more efficient memory)
• minibatches can be used per step (more stable)

Question 4

momentum

Question 5

nesterov

Question 6

learning rate decay

Answer 6

• start with big learning rate
• impose a decay to reduce learning rate by the number of epochs
• decay = (1/ (1+learn_rate . epoch_no.)) . decay0