Lecture 9 - Adaptive Control

Question 1

most ____ skills improve with practice

Answer 1

sensorimotor

Question 2

Proof that VOR learns to adjust

Answer 2

VOR restores retinal-image stability when vision is altered by lenses - magnifying glasses take ~ 30 min for VOR to grow stronger - minimizing glasses makes VOR grow weaker - glasses that flip the world upside down take about a week for VOR to adjust (VOR changes direction)

Question 3

what happens to saccades when eye muscle is damaged?

Answer 3

saccades miss target and drift

Question 4

if eye muscle damage isn’t too sever, what happens to saccades?

Answer 4

neural adaption restore saccade accuracy and eliminate drift, even if the muscle is damaged

Question 5

controller must know plant well, but…

Answer 5

we do not have to be born with accurate knowledge of our plant - plant changes through life - controller learns

Question 6

How do control networks adjust themselves to improve performance?

Answer 6

1. error-driven learning
2. Learning by perturbation
3. Gradient-descent learning

Question 7

controllers learn the properties of their plants based on…

Answer 7

sensory feedback (learn from trying, examples…)

Question 8

what is the aim of learning?

Answer 8

minimize average error (aka risk, aka expected loss)

Question 9

error (e) =

Answer 9

y - y*

Question 10

loss (L) =

Answer 10

|e|²/2

Question 11

What does the learner want?

Answer 11

minimize error & average loss E(L)

want both these to get 0

Question 12

risk depends on…

Answer 12

probabilities of different situations (not every input has the same loss)

Question 13

how does the brain estimate risk?

Answer 13

Learning by perturbation

Question 14

learning by perturbation

Answer 14

make small changes to the weights and accept the ones that reduce error;

w_i + η

If |e_pert| < |e|, then the perturbed weights are accepted

Question 15

each decision to accept or reject perturbed weights is based on…but…

Answer 15

single input z, but overtime the neuron samples many inputs and the OVERALL risk is reduced

Question 16

What is the learning algorism that samples many weights and calculates the error every time?

Answer 16

weight perturbation

Question 17

What happens when η is too big? Too small?

Answer 17

big: learn faster, not accurate (can’t zero in)
small: learn slower, very accurate (steps are small)

Question 18

weight perturbation formula

Answer 18

w_i ← w_i + η(random_i - 0.5)

Question 19

Gradient-descent learning

Answer 19

find the minimum on a weight by weight plot

Question 20

equation for gradient-descent learning

Answer 20

Δw = -ηez

Question 21

What is Widrow-Hoff learning?

Answer 21

Gradient-descent learning

Question 22

Advantage of Widrow-Hoff learning

Answer 22

compute which way to go instead of guess (like perturbation learning does)

Question 23

Why is WH learning faster?

Answer 23

exploits knowledge of the network: neuron is linear and error is a linear function of the neruon’s out put signal

Question 24

learning rate of WH depends on …

Answer 24

η

too big: unstable, may overstep low ground

too small: slow

Question 25

how to learn nonlinear functions?

Answer 25

1. weight perturbation (but too slow!!)
2. error-backpropagation (generalized WH)

Question 26

backprop

Answer 26

gradient-descent algorithm that trains layer networks of non-linear cells;

each cell in each layer computers its own aL/aw and sense info upstream;

given appropriate signals from all its downstream cells, a neuron can compute its own aL/aw

Question 27

argument against backprop

Answer 27

takes precise communcation between layers to computer aL/aw for synapses deep in the network -> possibly not possible in the brain?

Question 28

what came about from backdrop being too complex?

Answer 28

shallow learning

Question 29

shallow learning

Answer 29

• linear output cells
• outer layer of neurons is adjustable
• all other synapses are frozen (weights don’t change)

Question 30

curse of dimensionality

Answer 30

number of cells & synapses rise exponentially with dimension of the functions input space

Question 31

how to cope with curse of dimensionality?

Answer 31

• have lots of synapses (seeds, spores, sperm, immune system, granuel cells in cerebellum)
• prior knowledge