Topic 7a: Basal Ganglia- pathways dopamine, reinforcement learning Flashcards
Is the basal ganglia old or new system?
“Old” system
Predates the neo-cortex – still has important connections to brainstem
Exerts influence on motor and prefrontal areas via thalamus
Exerts influence on limbic circuits
Filtering our actions
What does the Basal Ganglia consist of?
Consists of several separate nuclei:
- Striatum (caudate, putamen, nucleus accumbens)
- Globus pallidus- interna (GPi) and externa (GPe)
- Subthalamic nucleus (STN)
- Substantia nigra pars compacta (SNc)
What is the direct and indirect pathway
Direct:
- Striatum > GPi > Thalamus
Net excitatory
Indirect:
- Striatum > GPe > STN > GPi > Thalamus
Net inhibitory
What pathway is excitatory and inhibitory?
Direct pathway - net excitatory
Indirect pathway - net inhibitory
What decides whether to take the direct or indirect pathway?
The SNc -Substantia nigra
Regulated by dopamine
Neuronal loss in substantia nigra pars compacta (SNc)
Parkinson’s Disease
What does a weak direct pathway cause?
Akinesia - symptom that causes a person to lose the ability to move their muscles on their own
Action plans aren’t able to be selected efficiently and ones that are are slow due to inhibition from the indirect pathway
What causes Parkinson’s disease?
Degeneration of dopamine neurons in substantia nigra
Neurological / psychiatric disease caused (largely) by alteration in a single neurochemical – dopamine
Disordered signals sent to SMA: motor disorder
What are the symptoms of Parkinson’s Disease?
The three cardinal symptoms of PD are:
1. Absence/slowness of movement (akinesia/bradykinesia)
2. Stiffness or rigidity
3. Tremor “at rest” (much reduced with active movement)
Other major symptoms include:
Gait and postural disturbances
Depression
Speech and swallowing problems
Mental confusion
Sleep disturbances
Loss of sense of smell
What are the treatments for Parkinson’s disease?
- Levodopa (L-DOPA: a precursor for dopamine)
Transplants: - Bone marrow derived stem cells
- Embryonic stem cells
(adult brain cells)
Neuro-surgery to rebalance connections between striatum and SMA: - Thalamotomy- surgery on thalamus
- Pallidotomy - surgery on globus pallidus internus
- Deep brain stimulation
What can Long term L-DOPA cause
Tilts the balance of direct/indirect back toward direct
Strong direct / weak indirect pathway -> hyperkinesia
Basal Ganglia pathology:
loss of striatal neurons or of STN
Huntingdon’s
Hemiballism
Tourette’s
Topic 7 Pathways and dopamine summary
- The basal ganglia form a series of loops between cortex, striatum, pallidum, back to cortex via thalamus
- Function to serve action selection
- Circuit relies on dopamine
- Strong links to common neurological conditions
What other cortical basal ganglia loops are there
- Motor
- Oculomotor
- Executive/associative
- Emotion/motivation
Basal ganglia and eye movement
Basal ganglia act as “filter” to add contextual information onto reflex control of eye movements – generalized inhibition of the sup. colliculus except for target of interest
What are the different learning systems?
Cortex: learning relationships
Basal Ganglia: reinforcement learning
Cerebellum: learning through error corrections
What is the role of the basal ganglia?
Acts as a filter/selector with broad cortical input and motor output
Inputs: sensory-motor, limbic and cognitive
Outputs: excitatory and inhibitory modulation of thalamus, modulation of cortical state (esp. SMA)
Selection of appropriate behaviours?
Self-initiation of behaviours
What is reinforcement learning?
Learning is based on reward or punishment (good/bad) rather than on error (too much, too little, left, right)
The reinforcement signal does not directly provide information about how to improve performance (contrast with cerebellum & climbing fibre signals)
Reward based learning - try to choose strategy to maximize future rewards
What is Instrumental conditioning/operant conditioning?
What is reinforcement learning?
Skinner box – rat, box, lever and reward
Behaviours are strengthened or weakened based on the consequences
Reinforcement learning is inspired by instrumental conditioning and uses AI to learn by taking feedback from the environment (remember the mouse game in the lecture)
What are rewards?
Natural reinforcers (water, food, sex)
Rewards stimulate release of dopamine (DA)
Animals will work to self-administer DA
Substances of abuse increase DA release in nucleus accumbens (NAcc, ventral striatum)
Neural basis of reward
Reward causes release of dopamine
Animal presses lever for intra-cortical electric stimulation (ICS) of dopamine neurons
Most reliable ICS sites include DA fibres:
mesolimbic: Ventral Tegmental Area (VTA) to NAcc (ventral striatum)
mesocortical: VTA to frontal cortex
nigrostriatal: SNc to striatum
Mechanism of action
ICS causes dopamine release in NAcc / striatum / cortex (rewarding effect)
Potentiates the glutamate transmission in cortex
Leads to strengthening of cortical inputs to striatum
Long term potentiation (LTP) of synapes
Reinforces behaviour over long-term. Can lead to habit forming – actions are performed even when they are no longer rewarding (addiction).
Topic 7 Reinforcement learning Summary
The basal ganglia form a series of loops between cortex, striatum, pallidum, back to cortex via thalamus
Action selection – They function to select behavioural actions, depending on sensory-motor and motivation status
There are strong links to several common neurological disorders
They are involved in reinforcement learning via dopamine
