Week 5: Memory Matrices Flashcards
Memory Matrices are different but similar version as compared to the
Hopfield network
Memory matrices is another toy model that helps us
think about memory
Memory matrices as compared to Hopfield network: (2)
brings us closer to mapping associative memory function onto the hippocampus
A more realistic model compared to Hopfield
A feed-forward single layer neural network can be drawn as in hetro-association
Wiring diagram of memory matrices has two sets of
neurons: circle (input neurons) and triangles (output neuron)
Wiring diagram of memory matrices, red arrow
Red arrow are axons
so output of the blue neurons go along the axon
Wiring diagram of memory matrices: black line
Black line is dendrite of triangular neurons which make contact of axons of the circular input neurons
Wiring diagram of memory matrices: black box
Black box is the synaptic connection that the axon makes onto the dendrite of the receiver (output) neuron
In memory matrices the activation and weights are either
0 or 1
In wiring diagram of memory matrices, Y is and X is (2)
Y is receiver neuron
X is input neuron
In memory matrices, we let the neurons learn by
changing the weights to be maximum between the current weight and product of input and output
We impose some pattern on x and also impose a pattern on y diagram so it causes in the diagram
So x1 neuron is active,
x2 is not
x3 is active
x4 and x5 are not:
So y1, y3 and y4 is active.
What happens when imposing patterns on x and y neurons in memory matrices according to our learning rule? - (3) ‘hetero-association’ example
x1 and y1 are both 1 and if weight was 0 before so 1 x1 = 1 and 1 is maximum value so turn weight to 1 so synapse has been learned
If I give an input x1 then give contribution to activation in y1,y3 and y4.
Same for x3 and other synapses stay 0
Hetroassociation - (3)
imposing pattern on x and y and network learns input and output assocations
one pattern (x) can generate another (y)
the input can generate the previous output based on having associated them together before with synaptic connections
Memory matrices auto-association is where
we can also associate the pattern with itself using recurrent connections
Memory matrices a recurrent feed-back neural network can be drawn as: (auto-association)
In memory matrices auto-association diagram
each neuron has an axon that connects to dendrites that connects to itself as well as neighbouring neurons that represent the input
In memory matrices auto-association activation of x and weights:
xi = 0 or 1
wij = 0 or 1
In memory matrices, we impose a pattern on the triangular neurons
what happens if you impose a different pattern? (2)
in which they will learn the synpases and maintain the input (i.e., state)
If we impose a different pattern then they will learn a different set of connections
Learning rule generally (also for auto-association/hetro-association etc..)
Output of Y in memory matrices (hetero-association/auto-association is)
Threshold or divide by no of active inputs so yi is 0 or 1
Auto-assocation is when the network
learns to associate a pattern of activity with itself
Detonator synapses in auto-association function (2)
These synapses are labelled detonator synapses
Need them to impose a new pattern of activity to be learned, while ignoring the feedback from the current pattern
Memory matrices is similar to Hopfield auto-associative network but (3)
connection weights are 0/1 and don’t need to be symmetric
Connection weights only increase (with pre and post synaptic activity)
Neuron activation values are 0/1 (not -1/1)
Memory matrices perform
pattern completion and error correction like Hopfield entwork
Memory matrices prone to
Memory matrices prone to
interference
Diagram of memory matrices worked example labelled (3) hetro-association
synapses and black synapses are 1 and empty squares mean synapses are 0
input neuron x and output neuron y
If i impose a pattern x1 and y1 then learn given connections then impose a pattern x2 and y2 and learn other connections etc…
more patterns I present the more synapses turned on.
Diagram of memory matrices worked example for pattern 1 in red
Diagram of memory matrices worked example for pattern 2 in purple hetro-association
We can take diagrams out and treat it as a matrix of connection weights:
We get correct recall so we take x3/pattern 3 in hetro-association - (2)
we multiply it by the cornnection matrix and divide by 3 (number of active cells)
Then we get the pattern 100110 in y3 neuron
Pattern completition so we give x3 but mistakenaely turn one off
so x3 is 001001 instead of 001011 - (2)
hetro-association
if we multiply this by the connection matrix and divide by number of active (2) we get the correct output:
100110 = y3
We also get saturation in adding another pattern like x4 (011100) gives y 4 but we get interference
hetero-association
as we filled the memory matrices with too many synapses and pattern 3 can not be recalled correctly anymore so y3 not same as actual y3
Pattern completion in auto-assocative network:
