RNN

Question 1

What are RNNs?

Answer 1

Recurrent neural networks

Output of one neuron feeds back into another neuron (even itself)

Question 2

What are some properties of RNNs?

Answer 2

Has short term memory
Can’t use regular back prop - back prop through time or truncated back prop through time
Exploding/vanishing gradients
Not that useful… by themselves
Good for temporally dependent tasks - we can think of music as temporally dependent - say we play the 1 and we’re in a major key - our next chord/note now depends on the major 1 to preserve consonance - Basically non MDPs

Question 3

What are the types of RNNs?

Answer 3

LSTM models
Self supervised models
World Models
Generative Adversarial Networks

Question 4

What are the components of an LSTM node?

Answer 4

Cell memory
Forget filter
Input filter
Output filter

Question 5

What are self supervised models?

Answer 5

Learn through observation
Combo of supervised and non supervised = given labelled training data but makes observations about the underlying structure of said data implicitly

Question 6

How do SS models learn?

Answer 6

By removing pieces of training data and attempt to predict what those pieces should have been - for a video remove some frames