Recurrent Neural Networks Flashcards

You may prefer our related Brainscape-certified flashcards:
1
Q

When is a neural network recurrent?

A

When neurons in the network can form loops (1 sends to 2 and 2 sends to 1).

Note: a single artificial model neurons can also be recurrently connected to itself.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

What are Hopfield Networks?

A

Recurrent neural networks where the weights are symmetric and the activity of units is either on or off.

Hopfield networks can learn patterns via Hebbian Plasticity.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

Can a Random Recurrent Neural Network have spontaneous dynamics?

A

Yes.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

What are the content-addressable memories (CAM)?

A

Memories that can be retrieved based on incomplete/partial stimuli.

Thus, in the case of CAM memories, your mother’s phone number is not retrieved by providing the address of the memory location in the storage, but rather, memory uses the input information (name of your mother) to retrieve the number. This is an example of an auto-associative CAM.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

How is the term ‘energy’ used in Hopfield networks?

A

Hopfield networks have a scalar value associated with each neuron of the network that resembles the notion of energy. The sum of these individual scalars gives the “energy” of the network.

If we update the network weights to learn a pattern, this value will either remain the same or decrease, hence justifying the name “energy.”

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

What is the function of a Hopfield network?

A

A Hopfield network retrieves patterns stored as fixed-point attractors. It is a model of associative memory.

It is a ‘content addressable memory’. Input patterns that look sufficiently similar to a stored memory will evoke the trained pattern.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

What are the assumptions underlying Hopfield networks?

A
  1. The activation of each neuron is either +1 or -1
  2. All of the connections are reciprocal (wij = wji)
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

What are the possible states of a neuron in a Hopfiled network?

A

It is a binary neuron (Mc Culloch-Pitts)

(0 or 1) or sometimes (-1 1)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

How are Hopfield networks trained?

A

Via Hebbian Learning: neurons that are co-active during a pattern are more strongly connected.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

Why do recurrent neural networks produce spontaneous dynamics?

A

Because of recurrent connections previous outputs become inputs again, and so on in a sequence.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

What are time-series?

A

In general, a sequence of numbers representing a certain variable acquired through time.

In the case of neural network activity, a sequence of activations values.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

How do we train read-outs from attractors in Echo State Networks?

A

Via linear regression of the output weights

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

What kinds of tasks are typical for an RNN?

A

In general RNN’s are competent in tasks where the output has history dependence.

Sequence Learning (e.g. word sequences), Sequence Generation, Sequence Classification (e.g., Language Parsing), Sensorimotor Transformations (for robot control).

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

What is an Echo-State Network?

A

A Recurrent Neural Network with Random weights, where the output is trained to produce desired time-series.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

What is a dynamical repository?

A

It is a network that can have many different attractors that can be accessed via multiple parameterizations.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

What is a discrete-time recurrent neural network?

A

A network that processes input in a stepwise fashion.

It produces discrete states.

17
Q

In Hopfield networks, the Hebbian learning rule learns patterns as …

A

fixed point attractors.