Lecture 26 - Reward System Dysfunction

Question 1

Brain system

Answer 1

A collection of structures in the brain that work together to perform a common function

Examples: Visual system, auditory system, vestibular system, somatosensory system, motor system, reward system

Systems can work alone or in concert with other systems to bring about changes in behaviour – analysis of how this is achieved is called systems neuroscience

Question 2

Brain network

Answer 2

Subtly different to a ‘network’ in that a network loosely refers to the structure of circuits that connect areas of neurons together

Question 3

Two distinct types of brain system

Answer 3

systems with a specific function

systems with a generic function

Question 4

Systems with a specific function

Answer 4

Processing visual information from the eye:
retina -> thalamus -> visual cortex -> cortex visual association areas

Generating fine movements of fingers:
motor association/planning areas in prefrontal and frontal cortex -> hand area in primary motor cortex

Question 5

System with a generic function

Answer 5

• Systems that process information received from regions performing widely differing specific functions to perform a particular behaviourally important operation
Usually a global function that uses all the information from different regions

• [….] generic functions are required for most/all tasks: e.g. attention, perfecting skills, memories, reward processing [can’t easily answer ‘where’?]

Question 6

How is the reward system and its network a generic processing system?

Answer 6

It is a generic system because it integrates information from manyareas serving different functions and SELECTS a behavioural response: e.g. stay or go?; repeat what you just did or don’t; or do something different.

Prefrontal cortex has an input pathway into the nucleus accumbens which from then goes to the ventral pallidum to the thalamus to the cortex but there are also other inputs into the nucleus accumbens that are a part of the lambic

Fear from the amygdala for example is all integrated with the motivation you have to do something to lead to an output

Caudate/putamen goes to the globus pallidus and the nucleus accumbens goes to the ventral pallidum (ventral system goes to the ventral pallidus)and from there to the thalamus and then back to the thalamus which also forms a loop which is more about associative learning rather than motivation/limbic functioning i.e. learning new skills and habits

Question 7

The mesolimbic dopamine circuit

Answer 7

The mesolimbic dopamine circuit

• Inputs from hippocampus to integrate memory of ‘context’
• Inputs from amygdala to integrate memory of an emotional e.g. fearful situation

Mid brain to limbic system

Question 8

Nigrostriatal dopamine circuit

Answer 8

The nigrostriatal dopamine circuit

• Reinforces sensory-motor associations
• Important in learning new skills
• Important in forming habits - also disordered in addiction

Substantia nigra to striatum (takes dopamine to the caudate/putamen)

E..g driving once you have practice lots the stimulus drives the movement rather than having to learn each step of how to respond

Question 9

Much of the reward networks involves the

Answer 9

basal ganglia

Question 10

Reward networks and basal ganglia - Dorsal circuit (sensorimotor)

Answer 10

1) Striatum (caudate,
putamen- sensorimotor),

2) Globus pallidus (sensorimotor),
3) Substantia nigra (sensorimotor dopamine - nigrostriatal),
4) (Subthalamic nucleus)

Question 11

Reward networks and basal ganglia - Ventral circuit (limbic)

Answer 11

1) nucleus accumbens
(limbic)

2) Ventral pallidum (limbic)
3) Ventral Tegemental area (limbic dopamine - mesolimbic)
4) Subthalamic nucleus

Question 12

How does the basal ganglia part of the reward system select between the options of what to do?

Answer 12

We have competing ‘motivations’ BUT only one set of muscles to respond with!

How might we ensure that the right behaviour is selected in a particular circumstance?

Basal ganglia tries to decide how the right behaviour is selected in a particular circumstance

Question 13

The basal ganglia and selecting the most important thing

Answer 13

The basal ganglia - different parts select the most appropriate / most important movement, sensation, idea, plan, memory, emotion, in a given situation, based on experience

Has different loops

Question 14

Basal ganglia and its loops

Answer 14

Motivation and emotion - Limbic loop where we want to think about motivation and emotion, whether it is appropriate to do this particular behaviour in a particular situation

Cognitive: ideas/memories/plans - this cognitive loop is really here to decide what the plan is to do in a specific example, when do you choose a particular strategy in a particular scenario

Sensorimotor stimuli/actions - Loop that involves the stimulus occurring and deciding what action to form in response to it

Question 15

How do the Basal Ganglia help us learn from experience about what the best action is to do in particular circumstances?

Answer 15

Reinforcement learning –
learning the strategy to be taken in a given situation that maximises the chance of a successful outcome
-> strategy is stored for later selection
-> reinforcement makes that strategy more likely to be selected in a similar situation in future
-> It’s all about dopamine: positive reinforcement releases dopamine

Question 16

A dopamine signal can be measured

Answer 16

in the striatum of the basal ganglia

Question 17

Dopamine is released in response to rewarding stimuli such as ..

Answer 17

stimuli associated with food or liquid (pulse is relatively short, matter of seconds)
stimuli assoicated with sex (anything you know that leads to sex will become rewarding and lead to dopamine release in these areas)

Question 18

Release of dopamine into the striatum _________ cortical activity (an ‘action’) can increase the strength of synapses from the cortex

Answer 18

AFTER

Question 19

Release of dopamine into the striatum AFTER cortical activity (an ‘action’) can increase the strength of synapses from the cortex

Answer 19

Synaptic plasticity at
corticostriatal synapses
controls what is learnt for later selection

Input coming into the basal ganglia is like an action and then if something nice happens like a reward occurs and dopamine is bought into this area to strengthen the pathway that goes right the way round, what dopamine is doing is that it is working on the connections between the cortex and the striatum/basal ganglia and these synapses are called corticostriatal synapses

Can record from cells in the striatum, these are cells that are going to go from the striatum to the pallidum and these are called spiny projection neurons as they have lots of spines (lots of sites of connections to other neurons) and they project out, looking at the strength of the synapses from the cortex to this neuron and recording from there we can measure a baseline response after hitting it will a little pulse
The spiny projection neuron stimulates and pumped one dopamine on which has reinforced the circuit and caused long term potentiation

baseline synaptic response, when action is followed by dopamine this synapse with the baseline is strengthen, dopamine released in response to positively reqarding stimulus following a cortical ‘action’ leads to long term potentiation

Question 20

Release of dopamine into the striatum AFTER cortical activity (an ‘action’) can increase the strength of synapses from the cortex - Principles of Positive Reinforcement

Answer 20

Addition of a pleasant stimulus to a behaviour equation

The dog sits FIRST - an ‘action’ that presents cortical activity to the striatum
The reward FOLLOWS the ‘action’ so the PRECEDING action is reinforced
What happens if you give the dog the reward first then try to teach him to sit?

Question 21

How do we know that dopamine release is reinforcing?

Answer 21

Because animals will learn the action required to activate their own dopamine pathways: Intracranial self-stimulation (ICSS)

Do an experiment where we put a stimulating electrode in the brain of an animal to activate the dopamine circuit and when we do this we give them a lever that will turn on the stimulator and give them a reward and you see what happens, animals learn very quickly the action required to stimulate their dopamine pathway which is called intracranial self-stimulation

Stimulating electrode implanted in an area near dopamine neurons in a rat for example

Intracranial self-stimulation of the dopamine cells in the substantia nigra – directly activates the reward pathways so animals learn to ‘select’ the action of pressing the lever

Question 22

Brain disorder examples that involve dysfunction of dopamine and the reward system and affect the basal ganglia’s ability to correctly select …

Answer 22

Motor (execution of movement) - Parkinson’s disease
Sensorimotor (stimulus-response) - Addiction (sensory problem e.g. seeing the needle produces response
Cognitive (planning actions) - Obsessive Compulsive Disorder
Limbic (emotion/memory) - Schizophrenia/Psychosis (abnormal associations that do not make sense to others in the world)

Question 23

In Parkinson’s disease dopamine is lost in …

Answer 23

the basal ganglia

Question 24

PD and radiolabbeled DOPA which is dopamine in the basal ganglia

Answer 24

The nigrastriatal degenerates much more than the mesolimbic and lose dopamine in the caudate/putamen

PET scans and give them flurodopa and radiolabel it so that it is like a precursor for dopamine so it will go in to the area that makes dopamine which is mostly the caudate/putamen

Patient - quite a reduction in the area compared to the controls, asymmetric which could indicate that someone is in quire an early stag, Fewer dopamine fibres to uptake the DOPA due to the neurodegeneration.

Question 25

What is dopamine usually replaced by in PD?

Answer 25

Dopamine is usually replaced by L-DOPA which is turned into dopamine and floods the system, losing natural dopamine timing

Question 26

L-Dopa

Answer 26

L-DOPA administration:
- prolonged slow dopamine fluctuations over hours instead of seconds

L-Dopa given through the mouth and then in the brain it gets converted into dopamine which then kind of floods the system

Lost reward timing … High levels of dopamine at the wrong times
means strengthening of synapses that reinforce actions that are not useful –> ‘DISORDERED’ MOTOR REINFORCEMENT

Question 27

Dopamine like drugs

Answer 27

Dopamine-like drugs are also used that directly activate receptors for dopamine
Dopamine like drug

directly activates receptors in the basal ganglia, activates receptors throughout the brain

These can
overdrive reward system and can reinforce addictive behaviour –>‘DISORDERED’ SENSORIMOTOR REINFORCEMENT

Addictive behaviour such as hyper sexuality, gambling, shopping addiction etc

• 52 year old Didier Jambart awarded $NZ 300K in 2012 by GlaxoSmithKline for use of a dopamine-like drug for Parkinson’s disease that resulted in him gambling and seeking sex with gay men

Question 28

PD and reinforcement and addiction

Answer 28

In Parkinson’s disease there is a loss of the ability to select normal learnt movements

dopamine treatments can cause reinforcement and selection of abnormal
movements or pleasurable addictive behaviours

What about addiction that is not caused by dopamine drugs taken for Parkinson’s disease? Is it different?

Question 29

Addiction

Answer 29

Repeated use of an addictive substance that persists despite serious harmful consequences to an individual and significant problems in their life - interferences with social/occupational functions

Addiction is thought of as a chronic relapsing disorder, with periods of abstinence followed by relapse

Diagnostically, the term Substance Use Disorder is preferable since it has fewer negative connotations than the term “addiction”

Question 30

Substance Use Disorder and Dependence

Answer 30

Efforts to stop substance use are largely ineffective, with relapse occurring without treatment in 80% of individuals

Physical dependence - is a state which has occurred when:
• increased experience leads to reduced drug action (tolerance)
• cessation of drug-taking leads to intense physiological disturbances (withdrawal) - ‘extinction’ of the sensorimotor habit of drug taking

Psychological dependence - a condition which develops when:
• taking a drug produces a pleasurable state;
• - a person is motivated to take the drug in order to maintain a pleasurable state or to avoid dicomfort
- abnormal limbic drive for addictive substances

Question 31

Physical dependence

Answer 31

Physical dependence - is a state which has occurred when: (probably caused LTP of these circuits and other circuits in the body)
• increased experience leads to reduced drug action (tolerance)
• cessation of drug-taking leads to intense physiological disturbances (withdrawal) - ‘extinction’ of the sensorimotor habit of drug taking (therefore leading to the frustration like seen in mice during the extinction phase)

Question 32

Psychological dependence

Answer 32

Psychological dependence - a condition which develops when:
• taking a drug produces a pleasurable state;
• - a person is motivated to take the drug in order to maintain a pleasurable state or to avoid dicomfort
- abnormal limbic drive for addictive substances

Question 33

Tolerance

Answer 33

Physical dependence

increased experience leads to reduced drug action (tolerance)

Question 34

Withdrawl

Answer 34

Physical dependence

cessation of drug-taking leads to intense physiological disturbances

Question 35

Natural history of substance use

Disorder/Addictions

Answer 35

See poster

Question 36

We have learnt a lot about patterns of substance use and brain mechanisms from self-administration studies in rats

Answer 36

Self-administration training (Drug is available) - Escalation - plasticity from reinforcement

Extinction (drug is NOT available) - Persistence and erratic drug seeking

Testing (Drug is Not available) - Cue induced relapse (drug priming, drug cues, stress)

Question 37

Self-administration training (Drug is available)

Answer 37

Self-administration training (Drug is available) - Escalation - plasticity from reinforcement

Learning about the association between the lever press and the drug and the lever pressing therefore escalates and then starts to drop off because the mouse becomes tolerant

Question 38

Extinction (drug is NOT available)

Answer 38

Extinction (drug is NOT available) - Persistence and erratic drug seeking

Turn off supply and leave lever there is initially lots of responses and then it extinguishes, persistsnce that the animals show, memory for it is still there

Question 39

Testing (Drug is Not available)

Answer 39

Testing (Drug is Not available) - Cue induced relapse (drug priming, drug cues, stress)

E.g. light going on during the trials and then turn that light on and the animals will immediately start pressing which is called priming (starts a craving)

Question 40

The common pathway of addictions is

Answer 40

enhanced dopamine release

see poster

Question 41

Radiolabelled cocaine brings to dopamine sites in the striatum

Answer 41

After 3-4 mins start seeing overall increase and you see the increases in frontal areas and mostly around the striatum which shows that this is where the cocaine wants to end up in the areas that are rich with dopamine terminals, after about 9-10 minutes it starts to fade away and this is how long the high is too

Question 42

Dopamine in the striatum is associated with a drug ‘high’

Answer 42

Measured binding of a radiolabelled dopamine-like molecule to dopamine receptors
Take exogenously which was labelled that would bind to receptors as agonists in the caudate/putamen

• (Left) drug users given a placebo drug (no drug abuse properties) - radiolabelled molecule can easily access dopamine receptors in striatum
  Therefore comes out bright, doesn’t work on the dopamine system
• (Right) participants given a dose of methylphenidate (which has drug abuse properties) -causes widespread dopamine release - radiolabelled molecule has fewer places to bind because of effect of the drug
  Causes widespread dopamine release from inside which means that the radio labelled molecule has fewer places to bind because of the drug’s effect therefore decreases the binding of the radio labelled molecule
• Measured amount of dopamine release after methylphenidate as difference in labelled molecule signal
• (Bottom graph) The larger the difference in the signal, the more dopamine was released, the greater the ‘high’ the user experiences

Question 43

Repeated substance use induces lasting changes in the dopamine system

Answer 43

Repeated drug taking means that less of the radiolabelled molecule can bind to the dopamine transporter (DAT) (compared to a control person)

• Suppression of access to dopamine sites by normal dopamine release - ?may underlie intense craving
  Because the drugs there but you have no more sites so you tend to crave more
• Some of these changes can thankfully be reversed by abstinence from drug taking
  Abstinence of 14 months can help it go back to normal (meth abuser 14 month abstinence looks similar to control)

Normal control and give them a radiolabelled molecule and there are plenty of spaces for it to bind on because there is not a lot of dopamine around and in a meth abuser who uses it a lot even after a month off it there are much fewer places for the radiolabelled molecule to bind, reduced binding in chronic users

Question 44

Addiction and its highjacking of the brains reward/positive reinforcement system

Answer 44

addictive substances induce abnormally high dopamine release throughout the brain that can lead to:

• abnormal sensorimotor associations that drives habitual action of taking substances in a specific situation (physical dependence)
• abnormal limbic drive (motivation/memory) for addictive substances (psychological dependence / craving) - limbic drive causing a craving or psychological dependence by activating this system
• abnormal cognitive activation (action planning) to drive substance seeking, without regard for the consequences - Planning to get drunk or high is what this is regarding