Section One - The dopamine hypothesis of reward

Question 1

Definition of learning

Answer 1

The process of forming associations, representations, and predictions about future rewards based on past experiences.
(Berridge & Kringelbach, 2008)

Question 2

What is the law of effect

Answer 2

Behavior is selected based on its consequences.
Pleasant outcomes → Behavior increases
Unpleasant outcomes → Behavior decreases
Responses are adjusted according to their immediate consequences.

Question 3

Classical conditioning

Answer 3

Learning through association
Conditioned stimulus (CS) comes before (precedes) the unconditioned stimulus regardless of behaviour.
Example = Pavlov’s dog (Bell > food)

Question 4

Operant conditioning

Answer 4

Learning through consequences
reinforcement or punishment depends on behaviour
Skinners box (lever > food)

Question 5

Skinner’s box experiment

Answer 5

Initially, the rat presses the lever rarely.
Once lever pressing is rewarded, frequency increases.
Skinner argued that human and animal behavior is largely driven by reward.

Question 6

what is a reward

Answer 6

an active process

Question 7

evolutionary perspective of rewards

Answer 7

Many rewarding stimuli increase the probability of gene propagation

Question 8

types of reward

Answer 8

primary (natural) = essential for survival e.g. food , water
secondary (conditioned) = acquired through association e.g. money, grades

Question 9

excitatory neurotransmitters

Answer 9

chemical messengers that increase the likelihood of a neuron firing an action potential
depolarise the postsynaptic neuron
Example: glutamate, acetylcholone (main)

Question 10

Inhibitory neurotransmitters

Answer 10

chemical messengers that decrease the likelihood of a neuron firing an action potential
hyperpolarise the postsynaptic neuron
Example: GABA, glycine (main)

Question 11

key brain areas in reward processing

Answer 11

nucleus accumbens (NAc) → mediates reward motivation
Ventral Tegmental Area (VTA) → Source of dopamine projections.
Prefrontal Cortex (PFC) → Involved in decision-making.
Lateral Habenula (LHb) → Modulates aversive learning

Question 12

neural projections in reward system

Answer 12

Axons project from the VTA to the NAc, forming the mesolimbic dopamine pathway.
These projections are conserved from rats to humans

Question 13

why use rats in experiments

Answer 13

functionally similar to humans
conditioned responses are similar
allows for experimental control

Question 14

Olds & Milner (1954)

Answer 14

implanted electrodes in rats’ nucleus accumbens.
Rats repeatedly returned to areas where they received stimulation.
When trained with a lever, rats preferred stimulation over almost any other behaviour.
= brain has specialised reward centres

Question 15

Examples of reward research techniques

Answer 15

Deep Brain Recordings – Measure neural activity in reward-related regions.
Brain Stimulation – Direct electrical or optogenetic activation.
Optogenetics – Uses light-sensitive proteins to control neurons with high precision

Question 16

Self stimulation

Answer 16

follows a dose-response curve
Higher stimulation → Increased reward value → More pursuit of stimulation

Question 17

Drugs that affect dopamine and influence self stimulation

Answer 17

amphetamine = increases DA = enhance reward value
pimozide =DA antagonist so decreases DA = reduces reward value

Question 18

Glutamatergic Inputs into the nucleus accumbens

Answer 18

amygdala, hippocampus, prefrontal cortex

Question 19

Dopaminergic input into the nucleus accumbens

Answer 19

Ventral tegmental area (VTA)

Question 20

Nucleus accumebns structure

Answer 20

shell - related to pleasure processing
core - goal driected behaviours

Question 21

GABAergic projections (inhibitory) from the nucleus accumbens

Answer 21

sent to ventral pallidum then substantia nigra
affect motor systems

Question 22

Where do you record levels of neurotransmitter from

Answer 22

ventral tegmental area and NAc shell/core because it is the only dopaminergic pathway

Question 23

EEG technique + cons

Answer 23

Measures on scalp
Doesnt work for humans as NAc too deep

Question 24

fMRI + cons

Answer 24

Blood based response
slower than other techniques - delayed
poor spatial resolution
brain area too small
hard to pass ethics

Question 25

Electrophysiology + cons

Answer 25

records populations of brain cells in deep structures good spatial and temporal resolution Access both action potentials and synpatic activity Hard to differentiate between different afferent projections (amygdala, hippocmpus etc). Makes it hard to distinguish what causes a synaptic event

Question 26

Microdialysis technique

Answer 26

- enables to collect samples of small molecules - Mouse is able to do behavioural tasks as requires chronic implantation of dialysis probe - Involves a small conveyor belt (semipermeable membrane) that allows small molecules like dopmaine to pass through while keeping larger molecules out. - ACSF (artificial cerebrospinal fluid) is continuously pumped to mimic the brains natural environment - neurotransmitters diffuse into the probe from surrounding extracellular fluid. They move in and out based on concentration gradients

Question 27

temporal resolution

Answer 27

how precisely a method can detect changes over time

Question 28

spatial resolution

Answer 28

how precisely a method can pinpoint activity in the brain e.g from small areas like neurons

Question 29

pros of microdialysis

Answer 29

- probe is small, displaces limited tissue - membrane gives physical barrier between perfusate and tissue (reduces introduction of bacteria) - analyte depletion is less than other techniques meaning can sample dopamine without affecting normal neurotransmission

Question 30

cons of microdialysis

Answer 30

- limited time resolution - midbrain dopamine firing is time locked to stimuli (milliseconds) but NAc DA levels remain elevated for some time (minutes) - can deplete solutes around the probe - could affect responsiveness of substance being tested

Question 31

Hernandez and Hoebel 1988 basic results - operant behaviour

Answer 31

animals learned lever pressing releases food pellet when a light is turned on. Measured dopamine before and during food availability = Increase in dopamine

Question 32

Hernandex and Hoebel 1988 - dopamine levels afterwards (operant)

Answer 32

dopamine levels remained elevated for 40-100 minutes after feeding stopped because of long term potentiation

Question 33

Hernandez and hoebel 1988 conclusion on dopamine (operant)

Answer 33

concluded dopamine facilitates effects on memory encoding and emoitonal/appetitive states

Question 34

Hernandez and Hoebel 1988 basic results - stimulation induced feeding

Answer 34

when food not available, similar response to when it is due to stimulating the lateral hypothalamus which is associated with feeding and appetite

Question 35

Hernandex and Hoebel 1988 - basic conclusion on stimulating feeding

Answer 35

potential appetitive circuit (LH > VTA > NAc > hedonic/pre-motor output).

Question 36

Hernandez and Hoebel 1988 - drugs results

Answer 36

both cocaine and amphetamine increased dopamine

Question 37

Tanda et a 1997 - experiment

Answer 37

recorded NAc DA levels during intravenous drug injections tested the psychoactive compound in cannabis (Δ⁹-THC), a synthetic cannabinoid antagonist and heroin each condition was pre-treated with saline (control), an antagonist that blocks cannabinoid effects or an antagonist that blocks opioid effects (Naloxone)

Question 38

Tanda et al 1997 - findings

Answer 38

- psychoactive compound (Δ⁹-THC) and synthetic cannabinoid agonist increase NAc DA levels - antagonist that blocks cannabinoid effects blocked effects of everything but heroin - opioid anatagonist (Naloxone) blocks everything

Question 39

Tanda et al 1997 - conclusions

Answer 39

- Heroin increases dopamine via opioid receptor activation. - Cannabinoid and opioid systems interact, but are distinct in how they modulate DA release. - These findings help explain why both cannabis and opioids can be addictive—they both increase dopamine levels in the reward pathway.

Question 40

Tanda et al 1997 - brain

Answer 40

when injected into VTA, heroin and Δ⁹-THC act to increase extracellular DA by suppressing inhibitory GABAergic interneurons in the VTA This can be blocked by antagonising (Blocking) opioid receptors on VTA interneurons

Question 41

Ranaldi et al 1999 - basic concept of study

Answer 41

lab animal learn pressing a lever supplies amphetamine/cocaine behaviour is regulated by loading (initial pressing of lever frequently) and maintenance phases (adjusting lever pressing to maintain steady drug intake) If drug dose increases, animal compensates by pressing more

Question 42

Ranaldi et al 1999 - findings

Answer 42

DA levels measured during maintenance, extinction and reinstatement of amphetamine

Question 43

Ranaldi et al 1999 - Relationship Between DA Levels & Drug Seeking

Answer 43

Rats self-administered amphetamine only when NAc DA levels dropped. Drug intake was not time-dependent, meaning rats pressed based on DA levels, not habit

Question 44

Ranaldi et al 1999 - extinction phase

Answer 44

drug removed Da declined, pressing increased = motivation to seek drug was DA dependent

Question 45

Ranaldi et al 1999 - reinstatment

Answer 45

injection of amphetamine single non-contingent amphetamine infusion restored DA levels and restarted exploratory behavior

Question 46

Ranaldi et al 1999 - satiety effect

Answer 46

When DA levels were already high, additional amphetamine had little reinforcing effect. Suggests a saturation point

Question 47

Medial forebrain bundle

Answer 47

white matter tract that flows between VTA and Nac

Question 48

Hedonic hotspots

Answer 48

areas that change pleasure responses to a stimuli

Question 49

Testing hedonic spots

Answer 49

Give animal a stimulus like juice Manipulate activity within hedonic spots and measure behavioural response (e.g. sticking out tongue)

Question 50

Draw picture of afferent projections

Answer 50

Answer: lecture 2 below subheading 'what is required for learning to occur'

Question 51

pre-motor pathway from NAc

Answer 51

dorsolateral ventral pallidium substantia nigra pars reticulata dorsal striatum

Question 52

Reward/emotion pathway from NAc

Answer 52

ventromedial ventral palliduim substantia innominate (part of amygdala) lateral hypothalamus

Question 53

3 dopamine pathways

Answer 53

mesolimbic mesocortical nigrostriatal

Question 54

mesolimbic pathway

Answer 54

VTA → NAc (key role = addiction/pleasure-seeking) Dysfunction = addiction, compulsions

Question 55

mesocortical pathway

Answer 55

VTA → PFC (Key role = executive functions) Dysfunction = Sz, depression, ADHD

Question 56

nigrostriatal pathway

Answer 56

Substantia nigra → dorsal striatum (Key role = motor coordination and voluntary movement) Dysfunction = parkinsons disease

Question 57

normal dopamine transmission

Answer 57

1. dopamine (vesicular) released for presynaptic neurons in nucleus accumbens 2. DA binds to DA receptors on postsynaptic neuron 3. sodium binds to dopamine transporter allowing reabsorbption of excess dopamine into presynaptic neuron

Question 58

Dopamine

Answer 58

acts through G-coupled protein receptors (D1,D2) D1 like - activates adenylyl cyclase increasing intracellular cocnentration of cyclic adenosine monophosphate cAMP ^excitability D2 like - inhibits adenylyl cyclase decreasing intracellular concentration of cAMP and reduced excitability

Question 59

DA transmission with cocaine

Answer 59

1. cocaine blocks dopamine transporter preventing reuptake of dopamine as sodium cannot bind 2. excess dopamine remains in synaptic cleft 3. this = prolonged dopamine signalling and overactive mesolimbic pathway = addictive

Question 60

DA transmission with amphetamine

Answer 60

1. amphetamine acts as a substrate for dopamine transporters 2. it enters the cell via DAT 3. reduces dopamine reuptake = more dopamine in the synaptic cleft 4. amphetamine targets VMAT2 (vesicular monoamine transporter 2) which prevents dopamine being stored in the cleft = dopamine accumulates in the cytoplasm 5. alters the concentration gradient forcing DAT to work in reverse leading to excessive postsynaptic dopamine receptor activation

Question 61

difference between cocaine and amphetamine

Answer 61

cocaine needs dopamine release amphetamine when induced will get rid of dopamine in a way that is independent from any input from the VTA as it just interfered with what is at synaptic level

Question 62

DA transmission with opioids

Answer 62

- GABAergic VTA interneurons regulate dopaminergic VTA neurons - Opioids target u-opioid receptors (MORs) which are found exclusively on GABAergic interneurons in the VTA - These interneurons normally inhibit dopaminergic neurons - When opioids bind to MORs they open potassium channels and GABA interneuorns hyperpolarise LESS GABA = LESS INHIBITION OF DOPAMINERGIC NEURONS = more dopamine (euphoria)

Question 63

DA transmission with cannabis

Answer 63

Endocannabinoids mediate DA release dopmaine-producing neurons in the VTA synthesise and release: AEA (Anandamide) and 2-AG (2-Arachidonoylglycerol) which bind to cannabinoid receptors (CB1) they reduced the inhibition of dopamine neurons Cannabis (THC) mimics 2-AG and acts on CB1, mediating robust inhibition of GABA input to DA

Question 64

DA transmission with nicotine

Answer 64

nicotine causes direct excitation of DA i VTA by activating ACh nicotinic receptors

Question 65

Reward prediction error

Answer 65

If an action produces an unpredicted positive outcomes, it is more likely to be repeated No further learning takes place when a reward in entirely predicted by sensory cues - can be applied to Pavlovian and operant conditioning

Question 66

Positive RPE

Answer 66

when the reward is better than expected = more dopamine release

Question 67

Negative RPE

Answer 67

when the actual reward is worse than expected = less dopamine

Question 68

Zero RPE

Answer 68

reward matches expectation = no change in dopamine

Question 69

Schultz monkey tests

Answer 69

record extracellular activity from midbrain DA neurons (VTA & SN) while monkeys performed behavioural tasks Naive monkey required to touch level following presentation of a signal light which gives desirable juice reward

Question 70

Reading raster plots

Answer 70

represents firing of neurons at a single time Horizontal rows = single trial aligned to stimulus onset -500 = before event 0-500 = after event

Question 71

before and after learning of reward - DA neurons

Answer 71

Before learning - no reward is predicted so large dopamine spike (conceptual reward prediction error) After learning - conditioned stimulus predicts a reward No reward prediction error. Dopamine occurs at presentation of CS

Question 72

RPE and blocking

Answer 72

learning only happens when there is a surprise, and food is already expected due to CS1, CS2 (light) is blocked from being learned.

Question 73

How blocking works

Answer 73

1. neutral stimulus (tone) paired with unconditioned stimulus (food), leads to conditioned response (salivation) 2. second stimulus (light) paired with tone 3. second stimulus (light) should trigger salivation but blocking stops this as tone already fully predicts the reward

Question 74

Stepwise transfer

Answer 74

- transfer of DA signal to earliest predicting CS - DA responds at the time of the CS which predicted the time-discounted reward, even if the actual reward is still far away and the brain calculates future rewards and assigns value to the first predictive cue

Question 75

RPE and risk aversion

Answer 75

monkeys show nonlinear preferences for rewards When rewards are low/medium, animals seek risk when reward is high, animals are risk averse

Question 76

midbrain DA and risk

Answer 76

- Medium risk gambles associated with: Moderate midbrain DA activity associated with cue Increased midbrain DA activity at time of reward -Higher risk gambles associated with: High midbrain DA activity associated with cue Reduced midbrain DA activity at time of reward

Question 77

Limitation of RPE

Answer 77

timing of DA response may be too quick to carry meaningful information Recent evidence shows the midbrain DA response comprises of 2 phases: unselective transient phase that cues for any potential rewarding salient object and a second phase that codes for reward value of a stimulus

Question 78

Fast scan cyclic voltammetry

Answer 78

↑ DA with positive RPE. ↓ DA with negative RPE. symmetrical encoding - brain treats positive and negative rewards equally

Question 79

conditioned response and DA

Answer 79

NAc integrates cortical/subcortical input, synapsing onto VTA-targeted neurons. Coincidence of cortical & DA activity → stronger conditioned responses. DA release is required for corticostriatal plasticity (Lerner & Kreitzer, 2011). Higher DA release (positive RPE) = higher likelihood of learning & memory formation

Question 80

Action selection in basal ganglia

Answer 80

NAc projects to the basal ganglia (BG) to initiate motor programs for reward pursuit. Motor programs are selected based on the strongest bid for activity. Positive RPE = stronger bid in BG = increased likelihood of reward-seeking behavior.