w7 l1 deep learning nlp

Question 1

what are some problems with forward nerual networks

Answer 1

limited context and theres no way to incorporate the history of previosuly made decisions

Question 2

whats an altertiative to fnns

Answer 2

recurrent neural networks

Question 3

what is the key difference between fnns and rnns

Answer 3

the input to the hidden layer at time t is expanded with the value from the previous hidden layer at t-1

Question 4

how is the rrn aritecture impacted by the expeansion of the current input with the previous hidden layer

Answer 4

learning an extra parameter, matrix of weights for the hidden layer from the previous time point

check formula in slides

Question 5

what are the problems with rnns

Answer 5

focus too much on immediate context

or competly ignore the immediate context

Question 6

what are rnns with a memory unit called and why is it good to have a memoery unit

Answer 6

Exploding and vanishing gradient problems are fixed by “storing”
data using a memory unit.
RNNs that have a memory unit are called Long Short Term Memory
(LSTM) Networks