Chapter 7 - Instrumental Conditioning: Motivated Mechanisms

Question 1

How is instrumental conditioning behaviour achieved in the brain?

Answer 1

There exists a mechanism to represent the magnitude and valence of a reward
- it monitors how responses influence the delivery of reinforcement

Question 2

What is neuroeconomics?

Answer 2

The brain is designed to maximize reinforcement (profit) while minimizing effort (cost)
- provides an explanation for choice behaviour

Question 3

What brain system codes for neuroeconomics?

Answer 3

Dopamine in the mesocorticolimbic pathway

Question 4

Describe the role dopamine plays in motivation and addition?

Answer 4

• Addiction is a disorder of motivation
• All drugs of abuse increase dopamine in the striatum and nucleus accumbens
• Dopamine activity increased in the striatum of monkeys when reinforcement was given
• dopamine predicts the availability of reinforcers and instigates actions to acquire it

Question 5

Where is the origin of the dopamine pathway?

Answer 5

In the ventral tegmental area

Question 6

What does activity in the amygdala indicate?

Answer 6

Reward magnitude and valence

- activated with pleasure and aversive stimulus

Question 7

What does increased striatal activity indicate?

Answer 7

Approach or do not approach behaviour

- associated with craving scores in addicts

Question 8

What is the orbitofrontal cortex important for?

Answer 8

Decision making

- executive functions like long term planning and moral functions, last decision maker

Question 9

What are the lateral and medial orbitofrontal cortices associated with?

Answer 9

Medial OFC - activated in response to reinforcing outcomes
Lateral OFC - activated in response to aversive outcomes
- damaged OFC - outcome value not used in decision making

Question 10

What two systems guide motivation?

Answer 10

1. habit learning

2. executive function

Question 11

Describe habit learning

Answer 11

• influenced by lower brain structures (e.g. striatum and amygdala)
• uses prediction of next available reinforcer (and errors in prediction) to guide behavior; we like something, so we go towards it

Question 12

Describe the executive function system

Answer 12

• Controlled by the OFC
  
  • this is why healthy adults are better decision makers than children or animals
• filters the go/no-go desire

Question 13

Describe the brain activity of a heroin addict

Answer 13

There is less OFC activity, but more striatal than control

- indicates a lack of restraint of reward seeking behaviour

Question 14

What is the first thing that motivates instrumental behaviour?

Answer 14

The associative structure of instrumental conditioning - how these aspects become associated with each other
- focus on individual responses and their stimulus antecedents and outcomes

Question 15

What kind of approach do all of the associative structures focus on?

Answer 15

Molecular approach - in the moment responses

Question 16

What three factors produce or contribute to instrumental responses?

Answer 16

1. stimulus
2. response
3. outcome
   - aka 3 term contingency

Question 17

What was the first theory of why instrumental conditioning would work?

Answer 17

Thorndike’s Law of Effect
- the pleasant reward/outcome would cause a greater likelihood of instrumental behaviour and the opposite for an annoying response/outcome

Question 18

What was the problem with the S-R learning association?

Answer 18

It tells us nothing about the outcome

• the reinforcer (O) serves to stamp in the S-R association
• not learning about O or S-O or R-O
• more relevant for habit learning because you aren’t thinking about the outcome

Question 19

How much of human behaviour is habitual?

Answer 19

about 45%

Question 20

What is necessary in the formation of an association?

Answer 20

The outcome

Question 21

What is the S-O association also called?

Answer 21

The reward expectancy

- in certain contexts, we are more likely to get a reward (this is the same as classical conditioning)

Question 22

According to Hull and Spence, what two factors motivate the instrumental response?

Answer 22

1. S-R association
   - the stimulus comes to evoke the response directly
2. S-O association
   - response is motivated by expectancy of reward

Question 23

What is the modern two-process theory?

Answer 23

S-O association (pavlovian learning) > conditioned, central emotional state (positive or negative based on the reinforcer) > response

Question 24

According to the modern two-process theory, what is important in affecting response?

Answer 24

Emotions

Question 25

Does expectancy of a reward produce more instrumental behaviour?

Answer 25

Yes

Question 26

What does PIT stand for?

Answer 26

Pavlovian Instrumental Transfer

- test of whether Pavlovian conditioned emotions motivate instrumental behaviour

Question 27

What two brain systems are active in PIT?

Answer 27

1. amygdala
2. ventral striatum
   - only when CS+ was present

Question 28

Does classical conditioning influence instrumental behaviour via a positive or negative emotional state (based on reinforcer valence) or do subjects acquire specific expectations of the reinforcer?

Answer 28

There is evidence of specific expectations

Question 29

What is the R-O association?

Answer 29

Response-outcome association

Question 30

Do animals learn that behaviours produce outcomes?

Answer 30

Yes

Question 31

How can we test is the R-O association is occurring?

Answer 31

Changing the value of the reward

Question 32

Describe the hierarchical S(R-O) association?

Answer 32

• S activates R (habitual behaviour)

- S also activates R-O association (learning to differentiate responses in different contexts)

Question 33

What is the second process that motivates instrumental behaviour?

Answer 33

Response allocation

Question 34

How is response allocation different from the associative structure of instrumental conditioning?

Answer 34

Because it takes a molar view

Question 35

What does a molar view entail?

Answer 35

How performing one response limits other activities/redistributes activities
- looks at overall consequences

Question 36

What is Thorndike’s definition of a reinforcer?

Answer 36

A stimulus that produces a satisfying or annoying state of affairs

Question 37

What is Skinner’s definition of a reinforcer?

Answer 37

A stimulus or outcome that increases the response that caused that stimulus to become available

Question 38

What is the problem with both Thorndike’s and Skinner’s definitions of a reinforcer?

Answer 38

They don’t help us predict what will become a reinforcer, or if something will become a reinforcer in a given situation
- they describe the relationship between a behaviour and consequence

Question 39

What helps guide instrumental behaviour?

Answer 39

Conditioned emotional responses

Question 40

What is Sheffield’s definition of a reinforcer?

Answer 40

Reinforcers are species specific consummatory responses

• these are not stimuli per se but the responses that we enjoy making
• ex. eating as opposed to the food itself as being the reward

Question 41

What is the consummatory response theory?

Answer 41

• species typical consummatory behaviours are a critical feature of reinforcers
• they are involved in the completion of an instinctive behavioural sequence
• this was the first theory that proposed that reinforcers were anything other than stimuli*

Question 42

What is another definition of a reinforcer?

Answer 42

Reinforcers are high probability responses

Question 43

What is the Premack principle?

Answer 43

• Difference in response probability is critical for reinforcement
• any behaviour more likely to be performed will reward for a low probability event
• reinforcement occurs when the instrumental act allows access to a more preferred (or more likely) behaviour

Question 44

What is the differential probability principle?

Answer 44

If the low probability event (L) produces a high probability event (H), then H reinforces L, but if H predicts L, H will not reinforce L
** only goes from low to high

Question 45

What did Timberlake and Allison propose?

Answer 45

Restricting behaviour and making it contingent on something is sufficient to produce reinforcement

Question 46

What is the response deprivation hypothesis?

Answer 46

every behaviour has a preferred level and once access to that behaviour is restricted, then we will perform another behaviour to get it back

Question 47

What is the main motivation in producing instrumental behaviour?

Answer 47

Maintaining or reaching homeostasis

- behavioural mechanisms can support homeostasis

Question 48

What is the behavioural bliss point?

Answer 48

Every organism has an optimal distribution of possible activities
- preferred level of activity for each activity

Question 49

What is the minimum deviation point?

Answer 49

The minimum unpreferred behaviour in order to get the maximum amount of preferred behaviour

Question 50

What is the reinforcement effect?

Answer 50

• happens by making contingency necessary
• increase in occurrence of instrumental response above the level of that behaviour in the absence of the response-reinforcer contingency
  ex. study time increases more than it would occur normally if watching TV is contingent upon studying

Question 51

What does behavioural economics outline?

Answer 51

How the value of the reward changes as a function of the price
- the price of something can be time dependent as well

Question 52

What are the three main characteristics of elasticity?

Answer 52

1. substitutes (ex. coke vs. pepsi)
   - if a good alternative is available
2. independents
   - switching over to an unrelated product
3. complements
   - related items will drop at the same rate (ex. salsa and tortilla chips)