Recurrent Networks

Question 1

Mention an example of a sequence-to-vector rnn

Answer 1

Genre classification of a song

Question 2

Mention an example of a vector-to-sequence rnn

Answer 2

Image captioning

Question 3

Mention an example of different input and output sequence length rnn

Answer 3

Speech interpretation/machine translation

Question 4

Name all three gating mechanism of LSTMS

Answer 4

input gate (blocking writing), forget gate and output gate (blocking reading)

Question 5

How do we create deep RNNs?

Answer 5

Multilayer perceptron: n hidden layers

Stacking. I.e stacked RNNs. We take the states of a rnn layer and use it as input into a new RNN.

Question 6

What is the memory, compute and serial steps for feed forward, backprop and backprob using target?

Answer 6

1. O(1), O(T), O(T)
2. O(T), O(T), O(T)
3. O(1), O(T), O(1)

Question 7

Why is it useful to process data sequentially?

Answer 7

some Information come in sequential settings, like speech recognition.
An immediate response, limited bandwidth, limited computational power, limited storing capabilities.

Question 8

What aspect of a RNN is it a LSTM improves upon?

Answer 8

Learning long-term dependencies, accounting for exploding/vanishing gradients.