Problem 4

Question 1

Hebbs law

Answer 1

Suggests that if 2 neurons on either side of a synapse are activated simultaneously, the strength of that synapse is selectively increased

Question 2

Weight

Answer 2

Is determined by the degree of activity between the 2 neurons

e.g.: activate simultaneously = increase; activate separately = reduce

Question 3

Strong positive vs strong negative weights

Answer 3

1. Strong positive weights
   - -> nodes that tend to be both positive/ both negative at same time
2. Strong negative weights
   - -> nodes that tend to have opposites

Question 4

There are certain drawbacks on the hebbian learning method .

Name them.

Answer 4

1. Interference
   - -> the number of associations which can be stored before they begin to interfere on each other are limited
2. Saturation
   - -> when the weights keep increasing, all units will be active when one presents an input to the network

Question 5

Neo-Hebbian learning

Solution to Saturation

Answer 5

Involves the “forgetting” term which allows you to forget a little bit of the weight every time

Question 6

Differential Hebbian learning

Answer 6

Interested in whether the neurons behave the same way

–> not interested in whether neurons are active at the same time

Question 7

Drive reinforcement theory

Answer 7

Builds upon differential hebbian learning and introduces time + time difference

–> in order to capture classical conditioning, we have to account for the prediction of CS based on US

Question 8

Pattern associator

Answer 8

Refers to a particular kind of network, which is presented with pairs of patterns during training

Question 9

What are the results after successful learning of the pattern associator ?

Answer 9

1. Recalls one of the patterns at output when the other is presented at input
2. Responds to novel inputs by generalizing from its experience it had with similar patterns

Question 10

Name the advantages of the pattern associator NWs.

Answer 10

1. Tolerance of noisy input
2. Resistance to internal damage
3. Ability to extract a central prototype from similar examples

Question 11

Explain the NS model on the basis of the “taste and sight of chocolate”.

Answer 11

Suggests that during learning in the NS, 2 patterns are presented simultaneously

1. Representing the taste of chocolate
   - -> reaching the dendrites via unmodifiable snaypses
2. Representing the sight of chocolate
   - -> via modifiable synapses

THUS: learning takes place via the modification of synapses

Question 12

CN model 
(Connectionist network model)

Answer 12

Like in the NS, this model suggests that during learning 2 patterns are presented simultaneously

1. P1 must be produced as an output unit
2. P3 is presented to input units

THUS: pattern association takes place by modifying the strength of the connections between input + output units

Question 13

In the CN model there are several terms that are equivalent to the terms used in the brain.

Name the equivalent terms for the following:

1. Axon
2. Dendrite
3. Synaptic strength

Answer 13

1. Input line
2. Output unit
3. Weight

Question 14

Hebb rule for weight change

Answer 14

delta wij = eai aj

–> rule is in multiplicative form as in order for a synapse to increase in strength both pre + postsynaptic activity must be present

Question 15

e

Answer 15

Refers to the learning rate constant which specifies how much a synapse alters in any one pairing of the two patterns

Question 16

ai

Answer 16

Refers to the activity of element i in pattern 1

Question 17

aj

Answer 17

Refers to the activity of element j in pattern 2

Question 18

Pattern associators have several important properties.

Name 6 of them.

Answer 18

1. Generalization
   - -> if a recall cue is similar to an already learnt pattern, the program produces a similar response to the new pattern
2. Fault tolerance
   - -> even if some synapses on neuron i are damaged, net input might still be a good approximation
3. Distributed representations
   - -> knowing the state of most elements to know which elements are represented
4. Prototype extraction + noise removal
5. Speed
   - -> recall is fast
6. Interference
   - -> not a bad thing

Question 19

Autoassociative memories

Answer 19

Are capable of retrieving a piece of data upon presentation of only partial info from that piece of data

e.g.: Hopefield NWs, capable of remembering data by observing a portion of that data

Question 20

Hopefield NWs can take on 2 different forms.

Name them.

Answer 20

1. Asynchronous
   - -> one unit is updated at a time
2. Synchronous
   - -> all units are updated at the same time

Question 21

Autoassociator

Answer 21

Refers to a particular from of pattern associator which is trained with the delta rule

–> its aim is to reproduce the same pattern at output that was present at input

Question 22

What are the 3 most advantageous assets of autoassociators ?

Answer 22

1. Store independent memories on the same set of connections
2. perform well with incomplete/noisy input
3. automatically form prototypical instances of categories

Question 23

Recurrent connections

Answer 23

Refer to connections whereby the output line of each unit is connected to the dendrites of the other units

–> present in autoassociators

Question 24

Competitive learning

Answer 24

Refers to a variant of Hebbian learning

–> here output units are in competition for input patterns

Question 25

Why do these competitive neurons compete with each other in competitive learning?

Answer 25

To see which one of them is the most similar to that particular input vector

–> thus the more strongly one particular unit responds to an incoming stimulus, the more its shuts down the other members of the cluster

Question 26

What is the basic structure of competitive learning models ?

Answer 26

A set of hierarchically layered units, with each layer connecting with layer above it

–> connections within are inhibitory; between = excitatory