Module 2

Question 1

What is non-associative learning?

Answer 1

• a process in which an organism’s behavior toward a specific stimulus changes over time in the absence of any evident link to (association with) consequences (types: habituation, sensitization, dishabituation)
• ex. habituation: gradual decrease of responding the more stimulus is presented

Question 2

What is associative learning?

Answer 2

behavioral change that accompanies the presentation of two or more stimuli at the same point in time or space
– classical/pavlovian conditioning and instrumental conditioning

Question 3

What is instrumental learning?

Answer 3

a type of learning in which behaviors are strengthened or weakened by their consequences

Question 4

What are the three key elements of instrumental learning?

Answer 4

1. The environment
2. The instrumental behaviour
3. The consequence

Question 5

What factors can influence instrumental learning?

Answer 5

• Timing of the reward delivery
• Rules of reward delivery
• Type of rewards
• Other stimuli associated with rewards

Question 6

Who is Thorndike and what did he study/conclude?

Answer 6

• Edward Thorndike is an American learning theorist:
• Devised the puzzle box to study learning
• Cat learned to press the lever to escape over many trials
• Concluded that a connection is formed between the lever (S) and the response (R) –> S-R learning –> law of effect
• Learning is incremental, not insightful (Insightful learning primary the case for higher learning)

Question 7

What is stimulus-response learning?

Answer 7

• S-R learning focuses on the direct association between stimuli and responses. It posits that learning occurs when a specific stimulus consistently elicits a specific response.
• The learner forms a connection between a stimulus (S) and a response (R) without considering the consequences of the response

Question 8

What is the law of effect?

Answer 8

• This principle emphasizes the role of consequences in learning.
• If a response in the presence of a stimulus is followed by a satisfying event, the association between the stimulus and the response is strengthened; if a response in the presence of a stimulus is followed by an annoying event, the association is weakened
• Argues for connectionism in learning.
• It suggests that the strength of a stimulus-response association is influenced by the satisfaction (or dissatisfaction) that follows the response.

Question 9

What is the limitation of SR learning?

Answer 9

Generalization: The example of monkeys transferring tool manipulation between hands illustrates a form of generalization. In this case, the monkeys learn to apply their skills with tools in a flexible way across different contexts, which suggests that their learning involves more than just a simple stimulus-response association.

Question 10

Who is B.F. Skinner and what did he study?

Answer 10

B. F. Skinner studied learning from a pure behaviourist perspective
* Designed many innovative tools to study learning
* Coined the term “Operant”: Operates on the environment
Trained rats to run down a runway to goal box where there’s reward à trying to see if rat learns that they can be rewarded for running to goal, they would run faster. If performance increases this would indicate learning

Question 11

What is the operant chamber/skinner box?

Answer 11

Operant chamber also called “Skinner box”
Operant chamber and cumulative recorder
* In the box there are objects (e.g. lever) the rats can interact with
* Tones and light for training stimuli.
* Shaping: teaching rat to hit lever for sugar reward
* Cumulative recorder: allows for continuous recording of free ongoing instrumental
behaviouràroll of paper that runs continuously and corresponds to time. With every response performed by rat, pen will move continuously to one direction until l it resets. Slope = how many actions over unit if time -à rate of response

Question 12

What is reinforcement/punishment/positive/negative?

Answer 12

Reinforcement: behaviour increases when it produces an appetitive stimulus
Punishment: behaviour decreases when it produces an aversive stimulus
* Stimulus here is the consequence (e.g. foot shock, sugar pellet) (reinforcer vs punisher =whether behaviour increases or decreases)
Positive and negative refers to “contingency” (presence or absence) * Positive: action leads to presentation of a stimulus
* Negative: action leads to removal of a stimulus
e.g. neg reinforcer –> something bad taken away

Question 13

What is the difference between reward and reinforcer?

Answer 13

Reward vs reinforcer: attractive and motivational property vs behaviour facilitator

Question 14

What is a continuous reinforcement schedule (CRF)?

Answer 14

every response leads to reinforcer delivery

Question 15

What is a partial reinforcement schedule (PRF)?

Answer 15

Partial reinforcement schedule (PRF):
* Ratio schedule: reinforcers are delivered based on the number of times a response occurs (e.g. FR10= animal gets reward every 10 lever presses)
* Interval schedule: reinforcers are delivered based on the time elapsed after which a response occurs
* Fixed vs variable:
- Fixed: the number of responses or the time has to elapse is certain
- Variable: the overall average is known, but the number/time for each reinforcer delivery is uncertain.

Question 16

How do the different schedules of reinforcement produce stereotypical response patterns?

Answer 16

VR: steady and robust responding
FR: post-reinforcement pause + ratio run
VI: steady and stable responding
FI: fixed-interval “scallop. After reinforcement, animals stop responding. As time approaches to when they predict the next reinforcer will be coming, they start to increase responding
— Typically, VR leads to the strongest responding: because its unpredictable. To get most rewards they need to respond a lot

Question 17

What is extinction?

Answer 17

A conditioned response diminishes due to lack of reinforcement
* Is a learning process itself – actions no longer produce rewards (During extinction it’s not about unlearning. Its new learning that suppresses old learning)
* Adaptive: saves energy by reducing unnecessary behaviour
*Extinction rate is affected by previous reinforcement schedules
– Slot machines operate on VR schedules: hardest to extinguish!

Question 18

What are primary vs secondary reinforcers?

Answer 18

Primary reinforcers: often biologically essential like food, water, sex etc.
Secondary reinforcers: stimuli that are previously paired with a primary reinforcer that becomes reinforcing in its own right, also known as conditioned reinforcers.
* e.g., sound of the lever, clicker, vouchers

Question 19

What are the four different functions generally ascribed to secondary (conditioned) reinforcers?

Answer 19

1. Reinforcing of new learning response
2. Establishing and maintaining schedules of reinforcement
3. Maintaining of behaviour during extinction
4. Mediating delays between response and delivery of reinforcement

Question 20

What is temporal contiguity and what was the study by Grise?

Answer 20

• Temporal contiguity: how soon the reinforcer follows the response
• Immediate reinforcer delivery leads to maximum learning (experimentally determined as ~0.5s)
  Grise (1948) removed all reinforcers, whereas Wolf (1964) kept a secondary reinforcer –> What’s the relationship between learning and delay:
• Adding a delay to reinforcer delivery discounts the reinforcing effect
• Delay of up to 10 seconds –> there’s virtually no learning if reinforcer is delivered 10
  sec after
• Secondary reinforcer (tone kept in): learning somewhat protected by presence of secondary reinforcer
  » In one condition, the primary reinforcer (food) was delayed after the behavior (like pressing a lever).
  » In another condition, while there was a delay in the primary reinforcer, a secondary reinforcer (the tone) was still present
  » When the tone (secondary reinforcer) was kept in the experiment, learning was somewhat protected from the negative effects of the delay. Even if the food (primary reinforcer) was delayed, the tone provided a cue that helped maintain the connection between the behavior and the expected reward.

Question 21

How are superstitious behaviours accidental reinforcement learning?

Answer 21

• When a random behaviour is accidentally reinforced by an appetitive outcome that comes in close temporal contiguity
• Highlights the natural tendency to draw causal links between events
• Reinforced pigeons on Fixed time schedule (will get a reward after certain amount of time- not contingent on action) à over a half of the pigeons’ developed stereotypical
  behaviour (thought doing this behaviour would get them reward)

Question 22

Definition of rewards as reinforcers:

Answer 22

Rewards are often called reinforcers because a response followed by a reward strengthens the association between certain environmental conditions (stimuli) and the response.
* Response theories
* Motivational theories

Question 23

Definition of rewards as incentives:

Answer 23

the anticipation or expectancy of reward arouses incentive motivation

Question 24

What was Skinner’s definition of a reinforcer and why is it practical?

Answer 24

• Skinner’s definition of reinforcers focuses on the functional aspect: any stimulus following a response that increases the probability of that response’s recurring is a reinforcer.
• This approach has considerable practical utility because it is often difficult to determine what will be a good reinforcer for a given person in a given situation.

Question 25

What is Premack’s principle?

Answer 25

Core argument: probability of a response could be low or high
Premack (1959): responses occur at different probability. What is reinforcing is relative, not absolute, and is dependent on the probability of the responses.
* Rats deprived of water or running wheel: the deprived one can be used to reinforce the other.
* Found the same patterns in children: eating candy and playing pinballs can reinforce each other depending on what they naturally prefer. The kids who like to play Ping Pong will use it as reinforcer for eating candy (if they prefer playing Ping Pong they have to eat candy to play ping pong –> increases probability of eating candy). Whatever’s being deprived, they will preform action to get to it (e.g Eating food becomes highly probable behaviour with deprivation – deprivation as reinforcing response. OR making food really good [ larger reward- reinforcing response] also increases motivating probability of eating.)
Premack’s principle: if two responses are arranged in an operant conditioning procedure, the more probable response will reinforce the less probable response; the less probable response will not reinforce the more probable response.

Question 26

What does response theory argue?

Answer 26

Response theory argues that responses have different probabilities and you can manipulate the probability to achieve learning outcome

Question 27

What is momentary probability?

Answer 27

Momentary probability: the probability of the behaviour at a given time in a given situation. May reflect the “value” of the behaviour. Can be manipulated by deprivation or size of the reward. Eating food becomes highly probable response with deprivation. Or better incentive

Question 28

What are the motivational theories of reinforcers?

Answer 28

Based on the homeostatic model: drive arises from need and energizes behaviour to reach goal.

Question 29

What is drive-reduction theory?

Answer 29

Drive reduction theory (Hull, 1943): any behavioural outcomes that reduce the drive is reinforcing.
Need reduction and drive stimulus reduction:

Question 30

What is the evidence for drive-reduction theory?

Answer 30

• Miller & Kessen (1952): a hungry rat either drank the milk or had the milk directly injected into
  the stomach during T-maze learning. Drinking milk was a better reinforcer.
• Injecting milk was more reinforcing than injecting saline solution.
• If only need reduction is true, then drinking or injecting milk should be equally reinforcing.
• Drinking milk reduced both drive stimulus intensity and need, whereas injecting milk only
  reduced need. Saline reduced neither.

Question 31

Rewards act as reinforcers are based on what assumptions (i.e. are rewards necessary for learning)?

Answer 31

The idea that rewards act as reinforcers rests on two assumptions:
* Learning is an associative process
* The role of rewards is to form and/or strengthen associations
If learning is not associative or reinforcement is not necessary for learning to occur, then these theories are challenged…

Question 32

What is the evidence against drive-reduction theory?

Answer 32

Evidence against drive reduction:
* events that do not reduce drive are still reinforcing * Self-stimulation of the brain (Olds & Milner, 1954) * Monkeys work for “sensory experience

Question 33

What is latent learning?

Answer 33

Tolman & Henzik (1930): hungry rats learned to run maze. Either rewarded or not.
- Reward made animals run the maze faster, but non-rewarded rats still completed the maze.
- The introduction of reward immediately improved the performance of previously non-rewarded rats. change is too quick to be explained by learning –> suggests that learning had already occurred gradually but hadn’t manifested in their performance
–> Latent learning: learning had occurred but was not manifested until reward was introduced.

Question 34

What is latent extinction?

Answer 34

Extinction of a previously rewarded response can occur without performance of the response in the absence of reward.
* Rat learns to run down runway to goalbox for reward. If rat is first placed in empty goalbox, and then allowed to run down runway to empty goalbox (extinction), they extinguish faster.
* Suggesting that the rat is not expecting the reward.
– these finding suggest that perhaps rewards working as reinforces that change behaviour isn’t true. Perhaps rather what animal anticipates; whether or not there will be reward à rewards operate as incentive motivation (external stimuli). Depending on desire and anticipation of reward being there, performance will be changed. Explains why things can be reinforcing even if there’s no biological need à

Question 35

What are theories of incentive motivation?

Answer 35

Alternative view to rewards as reinforcers: the anticipation or expectancy of reward arouses incentive motivation, a drive state which prompts us to engage in activities that lead to rewards.
* Theories of incentive motivation attempt to capture the way in which objects and events in the environment can acquire high motivational value and drive behaviour, even in the absence of a clear biological need.
- This theory argues that anticipation, or how much value is assigned to rewards
(subjective motivation), will influence performance (rather than learning). Argues against the fact that motivation is just about biological need

Question 36

What are incentive shifts?

Answer 36

Animals respond “better” (faster, more vigorously, more accurately) for bigger rewards
* Do animals learn better or are they more motivated to perform?
* Support for incentive motivational hypothesis: animals learn to respond for either small or large reward and then shifted from small to large or from large to small
*Performance changes to appropriate levels, but changes too fast to be explained by learning.
Experiment: 2 groups of rats
1. Initially trained with smaller reward then got switched to larger reward
2. Initially trained with larger reward then got switched to smaller reward.
* Group 2 initially (when receiving large reward) had a higher responding ceiling than group 1
* Once group 2 switched from large to small reward= reduced performance, below expected
* Once group 1 was switched from small to large reward = = enhanced performance, above expected
Animals over and underperform → they are subjectively assigning value to rewards which influence performance

Question 37

What are contrast effects?

Answer 37

Contrast effects: the enhancement or diminishment, relative to normal, of perception, cognition and related performance as a result of immediately previous or simultaneous exposure to a stimulus of greater or lesser value in the same dimensions
Not just the absolute value, but the subjectively assigned value of the reward influence behaviour.
Crespi (1944)- hypothesis: animals develop different emotional reactions to the shifts – anticipatory and actual

Question 38

What are positive contrast effects?

Answer 38

• Positive contrast effect: shifting from a small reward to a large reward leads to more responding than the naïve responding to the large reward (performance is even better than expected)

Question 39

What are negative contrast effects?

Answer 39

• Negative contrast effect: shifting from a large reward to a small reward leads to less responding than the naïve responding to the small reward (performance becomes even worse than it should be/ would be at baseline)

Question 40

What is the incentive motivational view on deprivation and what studies are consistent with this view?

Answer 40

Incentive motivational view on deprivation: deprivation does not directly energize the behaviour, rather, it increases incentive motivation by making anticipated incentives more attractive/valuable.
* Tolman (1948): only hungry animals show latent learning – for sated animals, food has no/little
incentive value.
* Alliesthesia (Cabanac, 1979): hunger makes food a better incentive, thirst makes water a better
incentive etc., reflecting the palatability of the reward.

Question 41

How is activation (arousal) part of emotion?

Answer 41

Activation (arousal) is a part of emotion:
* As the physiological response that accompanies emotional experience * * As a dimension of emotion (dimensional theory of emotion)

Question 42

How is arousal hypothesized to energize behaviour?

Answer 42

Arousal, like drive, is hypothesized to energize the behaviour.
* Duffy (1934): energy mobilization, emphasizing the autonomic system arousal
* Provided a physiological account of drive (drive is hypothesized as a “state”)
* Diverges from drive theory in several ways

Question 43

What is the reticular formation and reticular activating system?

Answer 43

• The RF is a complex network that integrates information. Knows the external and internal environment
• Receives all sensory inputs except olfactory (limbic system)
• Integrating and coordinating activities in CNS and PNS
  Reticular activating system (RAS):
• Ascending RAS:cortical“preparation”
  *Descending RAS: motor/muscular tone-up
  Highly neuromodulatory: DA, NE, 5-HT, histamine, acetylcholine, and neuropeptides.
• Some brain stimulation and lesion studies support the role for attention/consciousness/arousal
• Pharmacological modulation of RAS activity also lends support that this part is important for how much arousal in NS

Question 44

What is the inverted U-function of arousal?

Answer 44

• From a motivational perspective, there needs to be an optimal level of arousal for behaviour.
  The inverted-U function of arousal:
  Note the similarity between the optimal levels of arousal and stress in performance. No or low performance: is it due to no learning, no motivation, or other reasons?

Question 45

How does EEG activity correlate with arousal?

Answer 45

EEG activity correlates with arousal states (Beck Ch. 6)
* Alpha wave: awake and relaxed
* Beta wave: awake and activated

Question 46

What was the brightness discrimination task study and what’s the Yerkes dodson law?

Answer 46

Brightness discrimination tasks (black vs white) with different levels of shocks as punishment:
* Easy task: large contrast, best performance with high level of shocks (high arousal)
* Medium task: intermediate contrast, best performance with medium level of shocks
* Hard task: minimal contrast, best performance with low level of shocks (low arousal)
- Yerkes-Dodson Law: Cognitively difficult or intellectually demanding tasks may require a lower level of arousal for optimal performance to facilitate concentration, whereas tasks demanding stamina or persistence may be performed better with higher levels of arousal, presumably to increase and maintain motivation for “boring” tasks

Question 47

How does caffeine increase arousal (study by Anderson) and what theory does it support?

Answer 47

• Caffeine blocks adenosine receptors (GPCRs) in the nervous system.
• Anderson et al. (1994): gave undergrads different levels of caffeine and observed performance on easy and difficult language tasks.
• Found that performance of an easy task was enhanced by higher levels of caffeine, whereas higher caffeine levels impaired performance of a difficult task.
• Support the inverted-U function

Question 48

What is the Easterbrook hypothesis?

Answer 48

• Easterbrook (1959) hypothesized that arousal levels alter attentional/perceptual processes, and thus, affect performance.
• Attention works like a “spotlight”, and increased arousal decreases the range of attention
• Relevant (R) and irrelevant (Ir) cues coexist and are simultaneously processed
  »> In this scenario, there are more R than Ir in the task. An increased arousal level leads to more relevant cues in the periphery to be ignored, therefore, performance is impaired by failing to attend to some Rs.
  »> In this scenario, there are more Ir than R in the task. An increased arousal level leads to more irrelevant cues in the periphery being eliminated, and performance is enhanced by restricting the range of cues.
• Performance is modulated by arousal levels but is also relative to the task itself
  »> At low arousal levels you can pay attention to grey and white circle. When arousal increases, it narrows down spotlight, so you can only pay attention to the center.
• Task that requires global attention, high arousal bad
• Task that requires selective attention= high arousal good
• Influential idea in research into eyewitness reporting in criminology/forensic psychology.

Question 49

What is the evidence for the Easterbrook hypothesis?

Answer 49

Anderson & Revelle (1982): undergrads were asked to proof-read articles while provided with varying amount of caYeine:
* Two types of errors: intra-word (typo/misspelling) and inter-word (wrong syntax/grammar) * Test how sensitive subjects are to diYerent types of errors with diYerent arousal levels
* Aroused individuals had a lower detection rate of inter-word errors, which required a wider range of cue utilization; the detection rate of intra-word errors was unaYected by caYeine
Also, Mendelsohn & Griswold (1964; described in Beck Ch. 6 p. 169) supports the Easterbrook hypothesis.

Question 50

What is the drive theory vs activation theory?

Answer 50

Both theories attempt to address an intensity dimension of motivation, but with key diYerences:
Drive theory argues that it is optimal to maintain a minimal level of drive, since the deviation from homeostasis is the smallest.
Activation theory argues that it is optimal to maintain a somewhat intermediate level of arousal, since performance is more reliable at this optimal level and is aYected at lower or higher arousal levels.
* Drive reduction focuses on meeting the biological need; however, arousal may not always be directly relevant to a biological need.
* Suggesting that animals under certain circumstances will seek out stimulation to maintain such an optimal level of arousal.

Question 51

What did Berlyne mean by “ludic behaviour”?

Answer 51

• The term ludic behaviour refers to activities we engage in primarily for enjoyment, curiosity, or intellectual stimulation, rather than for survival or fulfilling an immediate biological function. Berlyne suggests that these activities are not driven by direct biological needs (such as eating or reproducing) but rather by a desire for engagement, challenge, or intellectual satisfaction.
• In essence, ludic behaviour is about seeking experiences that are enjoyable, intellectually stimulating, or emotionally engaging without a clear biological payoff. People are drawn to stimuli that are novel, uncertain, complex, or conflict-ridden, and the amount of stimulation in the environment (load) and an individual’s ability to handle it (screening ability) will affect how they engage with and enjoy those experiences.
  Berlyne’s theory helps explain why we seek out entertainment, puzzles, art, or philosophy—it’s the pleasure of the unknown, the complex, the surprising, and the intellectually challenging, which stimulate curiosity, thought, and creativity

Question 52

What are the qualities people look for in stimuli when engaging in ludic behaviour?

Answer 52

• When people engage in ludic behaviour, they often seek certain qualities in the stimuli they encounter. These qualities make the experience stimulating, engaging, or enjoyable:
• Novelty: surprise, incongruity, unfulfilled expectations
• Uncertainty: the amount of information carried by a stimulus, ambiguity
• Conflict: multiple responses aroused at the same time
• Complexity: number of distinguishable elements, dissimilarity of elements

Question 53

What is meant by environmental load and what are screeners vs non-screeners?

Answer 53

Environmental load refers to the amount of stimuli present in a given environment.
High load: A high-load environment has a lot of stimuli with lots of complex or novel elements. For example, an art gallery with many detailed and varied pieces of art or a science lab with complex experiments would represent high load.
Low load: A low-load environment is simpler, with fewer stimuli or elements. For example, sitting in a quiet room with minimal decoration or a simple, repetitive task might represent low load.
- Screeners and non-screeners refer to different styles of dealing with stimulation:
Screeners (sometimes called high tolerance for stimulation) are people who can handle and enjoy environments with a high load of complex and novel stimuli.
Non-screeners (sometimes called low tolerance for stimulation) are more likely to become overwhelmed by high loads of stimulation and prefer simpler, more controlled environments.
For example, a person who enjoys attending a chaotic music festival with many diverse performances and a lot of sensory input (loud music, bright lights, large crowds) might be a non-screener, while someone who prefers a quieter, more focused environment like reading a book in a calm library might be a screener

Question 54

How does different arousal levels produce different affects (moods)?

Answer 54

Berlyne (1970) hypothesized that diYerent arousal levels produce diYerent eYects. * Positive aYect is aroused by stimuli that produce medium level of arousal
* Animals thus seek out medium arousal stimuli
* Stimuli too far above or below the optimal level are aversive
* Motivated approach and avoidance behaviour to stimuli that do not restore homeostasis * Positive aYect also reinforces the behaviour without having to restore homeostasis
Animals seek out stimuli by things that generate an optimate level of arousal → motivated to approach or avoid different stimuli depending on arousal levels they induce

Question 55

How do the aesthetics of stimuli impact arousal?

Answer 55

• Smith & Dorfman (1975): The complexity of visual stimuli changes the emotional or arousal response as we are repeatedly exposed to them.
  » First Exposure: When you first see a complex visual pattern, it might arouse a high level of curiosity or excitement because it’s unfamiliar and stimulates novelty, but it might also cause confusion or a sense of being overwhelmed if it’s too complex to easily process.
  » Repeated Exposure: Over time, your emotional or arousal response to the pattern can change. You might adapt to the complexity—either finding it more engaging (if you begin to understand or appreciate the elements of the pattern) or losing interest (if it becomes predictable or you’ve seen it too often).
  » Increased Familiarity: As the visual pattern becomes more familiar, the initial arousal may drop, and instead, the stimuli might elicit a more stable, calm appreciation or reduced interest (depending on how engaging you find the stimulus over time).
• Vitz (1966): Aesthetic appreciation (e.g., in tone sequences) differs depending on a person’s expertise or familiarity with the subject.
  » Expert Viewers: People who are musically trained or art experts have a more refined ability to understand and appreciate the nuances of a tone sequence, such as subtleties in harmony or rhythm. Their aesthetic appreciation may be more complex and informed, leading to a higher level of enjoyment or appreciation for what might seem like a complex or subtle piece of art.
  » Novice Viewers: On the other hand, those without expertise might have a more general or basic reaction. They may not notice the technical details or depth that experts do, and their appreciation might be more based on simple pleasure or emotional response, rather than a detailed understanding of the composition.
• Humour and Comedy: Comedy arouses us in a pleasant way because of its ability to surprise us, challenge expectations, and provide relief, making us feel good.
  » Humour and comedy are enjoyable because they arouse us in a pleasurable way. The response to humour often includes feelings like surprise, incongruity, and relief. When something is funny, it typically violates expectations or creates an unexpected resolution to a situation, which can be mentally stimulating and pleasurable.
  » Humour is often linked with pleasant emotional arousal—not in the sense of stress, but in terms of a stimulating response that creates positive feelings. This type of arousal can also be seen as a form of mild cognitive dissonance—where the joke sets up a certain expectation and then breaks it in an amusing way, creating surprise and a sense of release when the punchline is revealed.

Question 56

How is goal-directed action more sophisticated than S-R behaviour/learning?

Answer 56

Stimulus-response habit: S-R learning relies on procedural memories i.e., memory for the motor program that has been strengthened via associations with the reward (pressing lever because of habit). May be the foundation of habit.
Goal-directed behaviour uses declarative memories i.e. press the lever beacause representations of the food and of how nice it is. Declarative memories influence and are influenced by motivation.

Question 57

How does goal-directed action involve multiple processes?

Answer 57

As the purpose of goal-directed behaviour is to achieve the desired outcome through performing the chosen behaviour, animals must be able to:
* Form direct S-R associations (instrumental conditioning through reinforcement)
* Evaluate the value of the outcome as an instrumental goal (incentive motivation)
* Evaluate the hedonic value of the outcome (which is dissociable from incentive motivation)
* Decipher and encode of the action-outcome contingency
* Process physiological and environmental cues

Question 58

What is the hedonic value system and instrumental incentive value system?

Answer 58

• Hedonic value system: how much animal is willing to consume the reward
• Hedonic value can be changed by satiation/devaluation etc. (if you keep feeding them the same things they’ll get full or bored- no longer consume it)
• Changes in hedonic value directly affect consumption of reward.
• reflects how much the animal likes the reward.
• Instrumental incentive value system: how much the animal is willing to work for the reward, i.e., to what extent the outcome of an action is a desirable goal.
• reflects how much the animal wants the reward.
• animals must learn through experience that a change in the hedonic value of the food changes its incentive value, e.g., that food is not worth working for when they are not hungry

Question 59

How does devaluation change goal-directed action performance (Belleine & Dickinson et al.,)

Answer 59

Experimental paradigm explained:
Group 1: injection with a delay (no devaluation) and expecting water
Group 2: injection without a delay (devaluation) and expecting water
Group 3: injection without a delay (devaluation) and expecting sugar (with sugar exposure as a reminder before Test 1)
*injection makes them sick
* At the beg of test 1, they were given a reminder of the drinking apparatus; either a drop of water or drop of sugar (made them expect to either get water or sugar depending on reminder cue)
In Devaluation, the devalued group expected to associate sugar with illness, devaluing sugar.
* In Test 1, groups (1&2) not expecting sugar pressed the lever equally frequently; Group 3 expecting sugar pressed the lever less frequently (didn’t want to get the sugar).
* In Test 2, re-exposure to sugar led the devalued groups (2&3) to press the lever less frequently, i.e., they have learned that sugar is not as nice and is not worth working for.
*
This result suggests that, during re-learning, animals experienced the discomfort associated with sugar and that this experience reduced incentive value of the outcome and, hence, performance
Some animals learn incentive value of sugar decreased- reflected in behaviour Goal directed= modulated by changing hedonic value

Question 60

How is habitual responding not affected by devaluation (Holland, 2004)?

Answer 60

Holland (2004): repeated instrumental conditioning reduces the impact of devaluation, i.e., animal will keep pressing the lever for food reward even after it has been associated with illness.
* Devaluation still leads to decreased incentive value, however, performance is unaYected * Suggests no role for incentive learning
* Repeated instrumental conditioning procedure leads to strong S-R connections
* Habit and goal-directed action operate diYerently
→action not guided by change of incentive, it’s driven by habit

Question 61

What areas of the brain are responsible for goal-directed action and SR/habitual action?

Answer 61

Devaluation and lesion studies in rodent models suggest that instrumental conditioning
involves two systems distinct neural substrates:
* incentive learning (goal-directed system): PFC + dorsomedial striatum * S-R learning (habit system): dorsolateral striatum

Question 62

What study found another responsible for goal-directed action in humans?

Answer 62

Valentin et al. (2007): examined the neural substrates of goal-directed component in humans.
* Trained participants to acquire food reward with a higher probability, then “devalued” the reward by satiation.
* Participants self-reported less pleasantness for the devalued food (hedonic value change), and reduced their choice for high-probability reward (incentive value change).
* fMRI found that areas of the orbitofrontal cortex showed increased activity before devaluation, and decreased activity after devaluation, suggesting its role for encoding incentive value.

Question 63

What does ICSS do?

Answer 63

• Mesocorticolimbic DA system: VTA DA neurons projecting to frontal cortex and NAcc.
• ICSS causes dramatic DA increases within the NAcc – a key site for reward-related learning and
  addiction.
• DA agonists increase ICSS whereas DA antagonists suppress it.
• Lesions of the mesocorticolimbic DA system disrupt ICSS.

Question 64

How does DA signalling respond to expectation of reward?

Answer 64

Electric recording from DA neurons in behaving monkeys.
* Before learning, DA neurons fire when the reward (US) occurs.
* US is unsignalled, therefore, unexpected, at this point.
* Once CS-US association is established, DA neurons now fire to the CS. DA neurons do not fire to the reward anymore.
* If an expected reward is missed, DA neurons pause firing –> negative change
→ Suggest DA signalling is responding to the expectation

Question 65

How does manipulating the probability of CS affect DA signalling?

Answer 65

Manipulated the probability of CS preceding US:
* At p=0.0, there will always be no US (fully predictable)
* At p=0.25, 3 out 4 times there will be no US
* At p=0.5, 1 out 2 times there will be US (least predictable)
* At p=0.75, 3 out 4 times there will be US
* At p=1.0, there will always be US (fully predictable)
DA neurons also ramp up their firing after CS presentation if US delivery is less certain. Maximal “ramping” is seen when p=0.5, i.e., maximal uncertainty. (Fiorillo et al. 2003)

Question 66

How does fast/slow dopamine transmission mediate different functions?

Answer 66

Dopamine transmission comes in diYerent timescales that mediate diYerent functions in the brain.
– Fast Dopamine Transmission (subseconds):
This occurs rapidly, in a very short time window (fractions of a second).

It is primarily localized in areas like the ventral tegmental area (VTA) and substantia nigra—regions in the brain involved in reward and movement.

Fast dopamine transmission is critical for reward processing and prediction errors—the process by which the brain updates its expectations about future rewards based on whether or not they occur (e.g., “I expected a reward, but it didn’t happen, so I need to adjust my behavior”). This helps the brain learn from outcomes and optimize future behavior.

Prediction Error Responses: Fast dopamine transmission responds to reward prediction errors. If something good happens when you weren’t expecting it, you get a burst of dopamine. If it doesn’t happen when you expect it, dopamine levels dip. This helps the brain learn and adapt.

Example: If an animal presses a lever and is expecting food but doesn’t get it, the brain registers a negative prediction error through dopamine. Conversely, if the animal presses the lever and gets food (which it was expecting), dopamine levels are higher, signaling that the reward has been achieved.

– Slow Dopamine Transmission (seconds to minutes):

This occurs over longer periods of time (seconds to minutes) and involves a more sustained or continuous release of dopamine.

Slow dopamine transmission doesn’t appear to be as directly related to immediate reward processing. Instead, it influences general brain functions, such as mood regulation, motivation, cognitive control, and attention.

This slower form of dopamine transmission affects processes like ongoing motivation or the sustained effort needed to achieve long-term goals.

Tonic Dopamine Transmission: This is the continuous or baseline release of dopamine in the brain, which influences things like overall mood and motivation, rather than reacting to specific rewards. It can be thought of as the brain’s “general state of readiness” or motivation level.

Question 67

What is causal action-outcome contingency and how does dopamine play a role?

Answer 67

• action-outcome contingency: animals must understand that their actions result in the outcome (e.g., detection of causal contingency between pressing the lever and food pellet being delivered), and that the outcome is a goal that is desired by the animal.
• Fast dopamine transmission (which is tied to reward prediction errors) is involved in helping the brain encode this cause-effect relationship.
  » when an animal presses the lever, dopamine transmission helps the brain recognize that the action (lever press) is causally linked to the outcome (getting food). This understanding allows the animal to learn from experience and adjust its behavior accordingly.