Topic 7a: Basal Ganglia- pathways dopamine, reinforcement learning

Question 1

Is the basal ganglia old or new system?

Answer 1

“Old” system
Predates the neo-cortex – still has important connections to brainstem
Exerts influence on motor and prefrontal areas via thalamus
Exerts influence on limbic circuits
Filtering our actions

Question 2

What does the Basal Ganglia consist of?

Answer 2

Consists of several separate nuclei:
- Striatum (caudate, putamen, nucleus accumbens)
- Globus pallidus- interna (GPi) and externa (GPe)
- Subthalamic nucleus (STN)
- Substantia nigra pars compacta (SNc)

Question 3

What is the direct and indirect pathway

Answer 3

Direct:
- Striatum > GPi > Thalamus
Net excitatory

Indirect:
- Striatum > GPe > STN > GPi > Thalamus
Net inhibitory

Question 4

What pathway is excitatory and inhibitory?

Answer 4

Direct pathway - net excitatory
Indirect pathway - net inhibitory

Question 5

What decides whether to take the direct or indirect pathway?

Answer 5

The SNc -Substantia nigra

Regulated by dopamine

Question 6

Neuronal loss in substantia nigra pars compacta (SNc)

Answer 6

Parkinson’s Disease

Question 7

What does a weak direct pathway cause?

Answer 7

Akinesia - symptom that causes a person to lose the ability to move their muscles on their own
Action plans aren’t able to be selected efficiently and ones that are are slow due to inhibition from the indirect pathway

Question 8

What causes Parkinson’s disease?

Answer 8

Degeneration of dopamine neurons in substantia nigra

Neurological / psychiatric disease caused (largely) by alteration in a single neurochemical – dopamine

Disordered signals sent to SMA: motor disorder

Question 9

What are the symptoms of Parkinson’s Disease?

Answer 9

The three cardinal symptoms of PD are:
1. Absence/slowness of movement (akinesia/bradykinesia)
2. Stiffness or rigidity
3. Tremor “at rest” (much reduced with active movement)

Other major symptoms include:
Gait and postural disturbances
Depression
Speech and swallowing problems
Mental confusion
Sleep disturbances
Loss of sense of smell

Question 10

What are the treatments for Parkinson’s disease?

Answer 10

• Levodopa (L-DOPA: a precursor for dopamine)
  Transplants:
• Bone marrow derived stem cells
• Embryonic stem cells
  (adult brain cells)
  Neuro-surgery to rebalance connections between striatum and SMA:
• Thalamotomy- surgery on thalamus
• Pallidotomy - surgery on globus pallidus internus
• Deep brain stimulation

Question 11

What can Long term L-DOPA cause

Answer 11

Tilts the balance of direct/indirect back toward direct
Strong direct / weak indirect pathway -> hyperkinesia
Basal Ganglia pathology:
loss of striatal neurons or of STN
Huntingdon’s
Hemiballism
Tourette’s

Question 12

Topic 7 Pathways and dopamine summary

Answer 12

• The basal ganglia form a series of loops between cortex, striatum, pallidum, back to cortex via thalamus
• Function to serve action selection
• Circuit relies on dopamine
• Strong links to common neurological conditions

Question 13

What other cortical basal ganglia loops are there

Answer 13

• Motor
• Oculomotor
• Executive/associative
• Emotion/motivation

Question 14

Basal ganglia and eye movement

Answer 14

Basal ganglia act as “filter” to add contextual information onto reflex control of eye movements – generalized inhibition of the sup. colliculus except for target of interest

Question 15

What are the different learning systems?

Answer 15

Cortex: learning relationships

Basal Ganglia: reinforcement learning

Cerebellum: learning through error corrections

Question 16

What is the role of the basal ganglia?

Answer 16

Acts as a filter/selector with broad cortical input and motor output
Inputs: sensory-motor, limbic and cognitive
Outputs: excitatory and inhibitory modulation of thalamus, modulation of cortical state (esp. SMA)
Selection of appropriate behaviours?
Self-initiation of behaviours

Question 17

What is reinforcement learning?

Answer 17

Learning is based on reward or punishment (good/bad) rather than on error (too much, too little, left, right)

The reinforcement signal does not directly provide information about how to improve performance (contrast with cerebellum & climbing fibre signals)

Reward based learning - try to choose strategy to maximize future rewards

Question 18

What is Instrumental conditioning/operant conditioning?
What is reinforcement learning?

Answer 18

Skinner box – rat, box, lever and reward

Behaviours are strengthened or weakened based on the consequences

Reinforcement learning is inspired by instrumental conditioning and uses AI to learn by taking feedback from the environment (remember the mouse game in the lecture)

Question 19

What are rewards?

Answer 19

Natural reinforcers (water, food, sex)
Rewards stimulate release of dopamine (DA)
Animals will work to self-administer DA
Substances of abuse increase DA release in nucleus accumbens (NAcc, ventral striatum)

Question 20

Neural basis of reward

Answer 20

Reward causes release of dopamine
Animal presses lever for intra-cortical electric stimulation (ICS) of dopamine neurons
Most reliable ICS sites include DA fibres:
mesolimbic: Ventral Tegmental Area (VTA) to NAcc (ventral striatum)
mesocortical: VTA to frontal cortex
nigrostriatal: SNc to striatum

Question 21

Mechanism of action

Answer 21

ICS causes dopamine release in NAcc / striatum / cortex (rewarding effect)
Potentiates the glutamate transmission in cortex
Leads to strengthening of cortical inputs to striatum
Long term potentiation (LTP) of synapes
Reinforces behaviour over long-term. Can lead to habit forming – actions are performed even when they are no longer rewarding (addiction).

Question 22

Topic 7 Reinforcement learning Summary

Answer 22

The basal ganglia form a series of loops between cortex, striatum, pallidum, back to cortex via thalamus
Action selection – They function to select behavioural actions, depending on sensory-motor and motivation status
There are strong links to several common neurological disorders
They are involved in reinforcement learning via dopamine