Task 5-competitive learning, Hebbian learning rules, autoassociator, pattern associator, recurrent network

Question 1

AUTOASSOCIATION

Answer 1

• AIM: PREPRODUCE THE SAME OUTPUT (AS LEARNED) EVEN IF INPUT IS NOISY OR DEGRADED
• AS EXTERNAL INPUT IS RECEIVED, THE OUTPUT OF EACH UNIT
  IS RECEIVED BACK TO ALL OTHER (RECURRENT CONNECTIONS) UNITS (INTERNAL INPUT)

does not need external teacher
learning occurs through weight change

Question 2

HEBBIAN LEARNING RULES

Answer 2

NEURONS THAT FIRE TOGETHER WIRE TOGETHER”
- IF 2 NEURONS ARE ACTIVE AT THE SAME TIME, THE WEIGHTS BETWEEN THESE TWO INCREASES (LIMITATION: WEIGHTS CAN ONLY INCREASE)LIMITATION: WEIGHTS GROW WITHOUT BOUNDS (SOLUTION: NEO HEBBIAN LEARNING), INSENSITIVE TO CS-US INTERVAL (LOW PSYCHOLOGICAL/NEUROLOGICAL) PLAUSIBILITY

Question 3

PATTERN ASSOCIATOR

Answer 3

• LEARNING BY ASSOCIATING ONE STIMULUS WITH ANOTHER
• GENERALIZATION: HAPPENS DURING RECALL; IF RECALL CUE IS SIMILAR TO PATTERN THAT IS ALREADY LEARNED, IT WILL PRODUCE A SIMILAR RESPONSE
• Fault tolerance
• Prototype extraction-if more then one pattern possible it will produce that with most activation ( competitive learning, ACT
• Needs external teacher / autoassociator does not need external teacher
• Difference to autoassociator: recurrent connections, doe not have same input as output, need extrenal teacher

Question 4

RECURRENT NETWORK

Answer 4

ACTIVATION FROM THE OUTPUT UNITS ARE FED BACK INTO THE

INPUT NEURONS

Question 5

hippocampus input

Answer 5

Receives input from the PARAHIPPOCAMPAL GYRUS
and ENTORHINAL CORTEX

These areas receive input from all ASSOCIATION AREAS (parietal, temporal and frontal lobes)
Hippocampus has info from different sensory pathways which have already been processed

Question 6

hippocampus output

Answer 6

The hippocampus projects back to the cortical
areas which provide input to the hippocampus
There are connections with SUBLIMBIC STRUCTURES

Question 7

INFORMATION PROCESSING occurs in 3 stages:

Answer 7

DENTATE GYRUS:
Get input from form pPerforant path
Form inhibitory interconnections between cells • In the gyrus
Output is carried by the mossy fibres to
• cells in the ca3 region
2. CA3 (cornu ammonis): Output branches
One branch forms a set of recurrent connections, synapsing back to the dendrites of other ca3 cells
Other branch (schaeffer collateral) carries the output to ca1 cells
3. CA1 (cornu ammonis):
Input from schaeffer collaterals from ca3
Output leaves the hippocampus and returns to the neocortical areas which provide the hippocampal performant path inputs

Question 8

sparse input

Answer 8

Any input pattern excites only few CA3 cells
Different input patterns are likely to activate different sets of CA3 neurons
A sparse input enables an autoassociator to store more memories
What limits the number of memories that can be stored is the number of inputs per neuron cannot be increased beyond 20.000 in the brain

The best way to maximize capacity is to ensure a sparse representation at input
The perforant path dentate granule cell system acts as a competitive learning network
Competitive learning removes redundancy: output from the DG system will be less correlated & more categorised than the inputs to it from the perforant path
Overlapping signals on the perforant path will be separated before they reach CA3
The role of the DG-mossy fibres is to maximise the separation of patterns reaching the CA3 autoassociation system

Question 9

Auto association in CA3

Answer 9

An episodic memory requires arbitrary sets of concurrent activities to be associated quickly and stored as one event which can be retrieved by a partial cue consisting of a sub-component of the memory
CA3
CA1 processing route

The ability to recall a complex memory with a cue which is a sub-component of the whole is a property of autoassociative memory

new event to be memorized would be represented as a firing pattern of CA3 pyramidal cells

The pattern would be stored using associatively modifiable synapses on the recurrent connections

Subsequently retrieval of a whole representation could be initiated by the activation of some part of it

Area CA3 could act as an autoassociative memory due to the recurrent connections

The hippocampus is able to quickly create “snapshots” of episodes through Hebb-like learning which create cues
Each event or episode would necessarily be stored as simple associations between the different cortical inputs which occurred during it
The collection of events forming an episodic memory must be kept separate from other episodes, even if they are somewhat similar, so that what happened on a single occasion can later be recalled correctly
This type of memory formation may be achieved by relatively large synaptic changes which will overwrite other information
ability of such a store to hold a particular piece of information will decline with time as it becomes: overwritten in its turn by the large synaptic changes accompanying the storage of new episodes
Individual memories may be difficult to extract from such a store, but the resultant knowledge may last indefinitely

Question 10

stages of information processing

Answer 10

1. Competitive learning: DG 2.
2. Autoassociation:CA3
3. Competitive learning:CA1
4. Pattern association: between C1 and entorhinal cortex

Question 11

THE ROLE OF CA3 RECURRENCE

Answer 11

If the recurrence of CA3 units is not allowed, then the firing is no closer to the originally learnt pattern that it is cue, and consequently the recall at later stages is worse

Question 12

instar /outsatar

Answer 12

INSTAR – neurodes receives large number of stimulus signals coming from outside its boundaries inwardly pointing star of incoming stimuli

OUTSTAR – neurodes sends its output to large number of other neurodes
outwardly radiating star of output signals moving out from the neurodes
Every neurode in every neural network is thus simultaneously both the center of an instar and the center of an outstar

Each instar receives stimuli from outstar neurode and from
corresponding position in external stimulu

Question 13

pROBLEM WITH HEBBIAN LEARNING

Answer 13

Weights only increase in strength Hebb’s Law only allows strength of a connection to increase under proper circumstances

Question 14

differential hebbian learning

Answer 14

propose variation on Hebb’s Law

The connection strength changes according to change (difference) in receiving neurode’s activation and change in the incoming stimulus signal

Question 15

‘QUANTITATIVE CIRCUIT’’

Answer 15

captures the essential magnitude of the relations between the elements
• Principle purpose of the quantitative circuit is to understand the functional consequences of the circuit, which are given by the details of the connectivity

Question 16

canal circuit

Answer 16

shows the average pattern of connections between the different neuronal types
that make up the circuit
• Does not take into account any quantitative aspects and shows only the most essential
functional connections of the circuit

Question 17

‘‘PETER’S RULE’’

Answer 17

neurons interconnect in proportion to the contribution to the neuropil of their dendrites & axonal synaptic boutons
Does not hold universally wherever there is a selectivity in the connections between types of
neurons, there is an exception to Peter’s rule

Question 18

just enough

Answer 18

‘‘Just-enough’’ means that the total effect of input projections is also capped by the small number of synapses that are available to drive the target neuron from any particular source, such as thalamus or another cortical area
The input projections are critically small
This feature has particular significance in relation to the pathways between cortical areas, which are thought to provide the essential ‘‘top-down modulation’’ or ‘‘bottom-up driving’’ of their targets