Lecture 6 - Instrumental conditioning basics

Question 1

What is instrumental behaviour?

Answer 1

Behavior that occurs because it was previously effective in producing certain consequences

goal-directed

some aspect of the individual’s behavior is instrumental in producing a significant stimulus or outcome

Question 2

What was Thorndike’s law of effect?

Answer 2

if a response R in the presence of a stimulus S is followed by a satisfying event, the association between the stimulus S and the response R becomes strengthened. If the response is followed by an annoying event, the S–R association is weakened.

what is learned is an association between the response and the stimuli present at the time of the response

Question 3

Modern Approaches to the Study of Instrumental Conditioning

What are Discrete-Trial Procedures?

Answer 3

each training trial begins with putting the animal in the apparatus and ends with removal of the animal after the instrumental response has been performed

Question 4

Modern Approaches to the Study of Instrumental Conditioning

What are Free-Operant Procedures?

Answer 4

free-operant procedures allow the animal to repeat the instrumental response without constraint over and over again without being taken out of the apparatus until the end of an experimental session.

Question 5

Free-Operant Procedures

What is Magazine Training?

Answer 5

preliminary steps for establishing lever-press responding in a laboratory rat.
First, the rat has to learn when food is available in the food cup.
The sound of the food-delivery device is repeatedly paired with the release of a food pellet into the cup.
After enough pairings of the sound of the food magazine with food delivery, the sound elicits a classically conditioned approach response

Question 6

Free-Operant Procedures

What is response shaping?

Answer 6

After magazine training

At this point, food is given if the rat does anything remotely related to pressing the lever. For example, at first the rat may be given a food pellet each time it gets up on its hind legs anywhere in the experimental chamber.

Once rearing over the lever has been established, the food pellet may be given only if the rat touches and depresses the lever. Such a sequence of training steps is called response shaping

Question 7

What are the steps for Successful shaping of behavior?

Answer 7

First, you have to clearly define the final response you want the trainee to perform.

Second, you have to clearly assess the starting level of performance, no matter how far it is from the final response you are interested in.

Third, you have to divide the progression from the starting point to the final target behavior into appropriate training steps or successive approximations

The execution of the training plan involves two complementary tactics: reinforcement of successive approximations to the final behavior and withholding reinforcement for earlier response forms.

Question 8

Can shaping result in new behaviour?

Answer 8

In teaching the rat to press the bar, we are not teaching new response components. Rather, we are teaching the rat how to combine familiar responses into a new activity. Instrumental conditioning often involves the construction, or synthesis, of a new behavioral unit from preexisting response components that already occur in the organism’s repertoire

Question 9

Instrumental Conditioning Procedures

What is the term for a pleasant event and an unpleasant stimulus?

Answer 9

A pleasant event is technically called an appetitive stimulus. An unpleasant stimulus is technically called an aversive stimulus.

Question 10

What are the types of instrumental conditioning procedures?

Answer 10

Positive reinforcement
(punishment) Punishment
Negative reinforcement (escape or avoidance)
Negative punishment (omission training/differential reinforcement of other behavior)

Question 11

Fundamental Elements of Instrumental Conditioning
What are the ways The Instrumental Response determines the result of positive reinforcement procedures?

Answer 11

Behavioral Variability Versus Stereotypy
Relevance or Belongingness in Instrumental Conditioning
Behavior Systems and Constraints on Instrumental Conditioning

Question 12

Fundamental Elements of Instrumental Conditioning
The Instrumental Response

Behavioral Variability Versus Stereotypy

How can the criteria for reinforcement also be defined in terms of more abstract dimensions of behavior?

Answer 12

e.g., novelty

The behavior required for reinforcement can be defined as doing something unlike what the participant did on the preceding four or five trials
response variability

Question 13

Fundamental Elements of Instrumental Conditioning
The Instrumental Response

What is Relevance or Belongingness?

Answer 13

“A behavior cannot be reinforced by a reinforcer if it is not naturally linked to that reinforcer in the repertoire of the animal”

e.g., yawning to be let out of a puzzle box

Question 14

Fundamental Elements of Instrumental Conditioning
The Instrumental Response

What are Behavior Systems and Constraints on Instrumental Conditioning?

Answer 14

According to behavior systems theory, when an animal is food deprived and is in a situation where it might encounter food, its feeding system becomes activated, and it begins to engage in foraging and other food-related activities.

The effectiveness of the procedure in increasing an instrumental response will depend on the compatibility of that response with the preexisting organization of the feeding system.

responses that become more likely when the animal is hungry are readily reinforced with food, whereas responses that become less likely when the animal is hungry are difficult to train as instrumental responses.

Question 15

Fundamental Elements of Instrumental Conditioning
What are the ways The Instrumental Reinforcer determines the result of positive reinforcement procedures?

Answer 15

Quantity and Quality of the Reinforcer
Shifts in Reinforcer Quality or Quantity - positive and negative behavioral contrast effects

Question 16

Fundamental Elements of Instrumental Conditioning
What are the ways The Response–Reinforcer Relation determines the result of positive reinforcement procedures?

Answer 16

temporal relation
- temporal contiguity
response–reinforcer contingency

Question 17

What is a secondary, or conditioned, reinforcer?

Answer 17

a conditioned stimulus that was previously associated with the reinforcer. Verbal prompts in coaching, such as “good,” “keep going,” and “that’s the way” are conditioned reinforcers that can provide immediate reinforcement for appropriate behavior

Question 18

What is a marking procedure?

Answer 18

mark the target instrumental response in some way to make it distinguishable from the other activities of the organism

facilitates learning with delayed reinforcement

Question 19

How can superstitious behavior occur?

Answer 19

accidental, or adventitious, reinforcement

high contiguity rather than contingency

Question 20

Reinterpretation of the Superstition Experiment

What are terminal and interim responses?

What did Staddon and Simmelhag find?

Answer 20

terminal - some of the responses occurred predominantly toward the end of the interval between successive reinforcers

interim - somewhere near the middle of the interval between food deliveries

Which actions were terminal responses and which were interim responses did not vary much from one pigeon to another. Furthermore, Staddon and Simmelhag failed to find evidence for accidental reinforcement effects.

Question 21

Explanation of the Periodicity of Interim and Terminal Responses

How did Staddon and Simmelhag explain terminal and interim responses?

Answer 21

Staddon and Simmelhag (1971) suggested that terminal responses are species-typical responses that reflect the anticipation of food as time draws closer to the next food presentation. By contrast, they viewed interim responses as reflecting other sources of motivation that are prominent early in the interfood interval, when food presentation is unlikely.

Question 22

What is the learned-helplessness EFFECT?

Answer 22

exposure to uncontrollable shock disrupted subsequent learning

Question 23

What is the learned-helplessness HYPOTHESIS?

Answer 23

The learned-helplessness hypothesis assumes that during exposure to uncontrollable shocks, animals learn that the shocks are independent of their behavior—that there is nothing they can do to control the shocks. Furthermore, they come to expect that reinforcers will continue to be independent of their behavior in the future.

First, the expectation of lack of control reduces the motivation to perform an instrumental response. Second, even if they make the response and get reinforced in the conditioning phase, the previously learned expectation of lack of control makes it more difficult for the subjects to learn that their behavior is now effective in producing reinforcement.

Question 24

Alternatives to the Helplessness Hypothesis

What is the activity deficit hypothesis?

Answer 24

animals show a learning deficit following exposure to inescapable shock because inescapable shocks encourage animals to become inactive or freeze

However, it cannot explain instances in which exposure to inescapable shock disrupts choice learning

Question 25

Alternatives to the Helplessness Hypothesis

What is the attention deficit hypothesis?

Answer 25

exposure to inescapable shock reduces the extent to which animals pay attention to their own behavior

Question 26

Alternatives to the Helplessness Hypothesis

Stimulus relations in escape conditioning

Why is exposure to escapable shock not nearly as bad as inescapable shock?

Answer 26

Some of the response-produced stimuli experienced at the start of the escape response are called shock-cessation feedback cues.

Other response-produced stimuli experienced as the animal completes the response, just after the shock has been turned off at the start of the intertrial interval are called safety-signal feedback cues

Therefore, contextual cues of the chamber in which shocks are delivered are more likely to become conditioned to elicit fear with inescapable shock