Week 5: Memory Matrices

Question 1

Memory Matrices are different but similar version as compared to the

Answer 1

Hopfield network

Question 2

Memory matrices is another toy model that helps us

Answer 2

think about memory

Question 3

Memory matrices as compared to Hopfield network: (2)

Answer 3

brings us closer to mapping associative memory function onto the hippocampus

A more realistic model compared to Hopfield

Question 4

A feed-forward single layer neural network can be drawn as in hetro-association

Answer 4

Question 5

Wiring diagram of memory matrices has two sets of

Answer 5

neurons: circle (input neurons) and triangles (output neuron)

Question 6

Wiring diagram of memory matrices, red arrow

Answer 6

Red arrow are axons
so output of the blue neurons go along the axon

Question 7

Wiring diagram of memory matrices: black line

Answer 7

Black line is dendrite of triangular neurons which make contact of axons of the circular input neurons

Question 8

Wiring diagram of memory matrices: black box

Answer 8

Black box is the synaptic connection that the axon makes onto the dendrite of the receiver (output) neuron

Question 9

In memory matrices the activation and weights are either

Answer 9

0 or 1

Question 10

In wiring diagram of memory matrices, Y is and X is (2)

Answer 10

Y is receiver neuron
X is input neuron

Question 11

In memory matrices, we let the neurons learn by

Answer 11

changing the weights to be maximum between the current weight and product of input and output

Question 12

We impose some pattern on x and also impose a pattern on y diagram so it causes in the diagram

Answer 12

So x1 neuron is active,
x2 is not
x3 is active
x4 and x5 are not:

So y1, y3 and y4 is active.

Question 13

What happens when imposing patterns on x and y neurons in memory matrices according to our learning rule? - (3) ‘hetero-association’ example

Answer 13

x1 and y1 are both 1 and if weight was 0 before so 1 x1 = 1 and 1 is maximum value so turn weight to 1 so synapse has been learned

If I give an input x1 then give contribution to activation in y1,y3 and y4.

Same for x3 and other synapses stay 0

Question 14

Hetroassociation - (3)

Answer 14

imposing pattern on x and y and network learns input and output assocations

one pattern (x) can generate another (y)

the input can generate the previous output based on having associated them together before with synaptic connections

Question 15

Memory matrices auto-association is where

Answer 15

we can also associate the pattern with itself using recurrent connections

Question 16

Memory matrices a recurrent feed-back neural network can be drawn as: (auto-association)

Answer 16

Question 17

In memory matrices auto-association diagram

Answer 17

each neuron has an axon that connects to dendrites that connects to itself as well as neighbouring neurons that represent the input

Question 18

In memory matrices auto-association activation of x and weights:

Answer 18

xi = 0 or 1
wij = 0 or 1

Question 19

In memory matrices, we impose a pattern on the triangular neurons

what happens if you impose a different pattern? (2)

Answer 19

in which they will learn the synpases and maintain the input (i.e., state)

If we impose a different pattern then they will learn a different set of connections

Question 20

Learning rule generally (also for auto-association/hetro-association etc..)

Answer 20

Question 21

Output of Y in memory matrices (hetero-association/auto-association is)

Answer 21

Threshold or divide by no of active inputs so yi is 0 or 1

Question 22

Auto-assocation is when the network

Answer 22

learns to associate a pattern of activity with itself

Question 23

Detonator synapses in auto-association function (2)

Answer 23

These synapses are labelled detonator synapses

Need them to impose a new pattern of activity to be learned, while ignoring the feedback from the current pattern

Question 24

Memory matrices is similar to Hopfield auto-associative network but (3)

Answer 24

connection weights are 0/1 and don’t need to be symmetric

Connection weights only increase (with pre and post synaptic activity)

Neuron activation values are 0/1 (not -1/1)

Question 25

Memory matrices perform

Answer 25

pattern completion and error correction like Hopfield entwork

Question 26

Memory matrices prone to

Answer 26

Memory matrices prone to

interference

Question 27

Diagram of memory matrices worked example labelled (3) hetro-association

Answer 27

synapses and black synapses are 1 and empty squares mean synapses are 0

input neuron x and output neuron y

If i impose a pattern x1 and y1 then learn given connections then impose a pattern x2 and y2 and learn other connections etc…

more patterns I present the more synapses turned on.

Question 28

Diagram of memory matrices worked example for pattern 1 in red

Answer 28

Question 29

Diagram of memory matrices worked example for pattern 2 in purple hetro-association

Answer 29

Question 30

We can take diagrams out and treat it as a matrix of connection weights:

Answer 30

Question 31

We get correct recall so we take x3/pattern 3 in hetro-association - (2)

Answer 31

we multiply it by the cornnection matrix and divide by 3 (number of active cells)

Then we get the pattern 100110 in y3 neuron

Question 32

Pattern completition so we give x3 but mistakenaely turn one off

so x3 is 001001 instead of 001011 - (2)

hetro-association

Answer 32

if we multiply this by the connection matrix and divide by number of active (2) we get the correct output:

100110 = y3

Question 33

We also get saturation in adding another pattern like x4 (011100) gives y 4 but we get interference

hetero-association

Answer 33

as we filled the memory matrices with too many synapses and pattern 3 can not be recalled correctly anymore so y3 not same as actual y3

Question 34

Pattern completion in auto-assocative network:

Answer 34