Lecture 6 - Instrumental conditioning basics Flashcards
What is instrumental behaviour?
Behavior that occurs because it was previously effective in producing certain consequences
goal-directed
some aspect of the individual’s behavior is instrumental in producing a significant stimulus or outcome
What was Thorndike’s law of effect?
if a response R in the presence of a stimulus S is followed by a satisfying event, the association between the stimulus S and the response R becomes strengthened. If the response is followed by an annoying event, the S–R association is weakened.
what is learned is an association between the response and the stimuli present at the time of the response
Modern Approaches to the Study of Instrumental Conditioning
What are Discrete-Trial Procedures?
each training trial begins with putting the animal in the apparatus and ends with removal of the animal after the instrumental response has been performed
Modern Approaches to the Study of Instrumental Conditioning
What are Free-Operant Procedures?
free-operant procedures allow the animal to repeat the instrumental response without constraint over and over again without being taken out of the apparatus until the end of an experimental session.
Free-Operant Procedures
What is Magazine Training?
preliminary steps for establishing lever-press responding in a laboratory rat.
First, the rat has to learn when food is available in the food cup.
The sound of the food-delivery device is repeatedly paired with the release of a food pellet into the cup.
After enough pairings of the sound of the food magazine with food delivery, the sound elicits a classically conditioned approach response
Free-Operant Procedures
What is response shaping?
After magazine training
At this point, food is given if the rat does anything remotely related to pressing the lever. For example, at first the rat may be given a food pellet each time it gets up on its hind legs anywhere in the experimental chamber.
Once rearing over the lever has been established, the food pellet may be given only if the rat touches and depresses the lever. Such a sequence of training steps is called response shaping
What are the steps for Successful shaping of behavior?
First, you have to clearly define the final response you want the trainee to perform.
Second, you have to clearly assess the starting level of performance, no matter how far it is from the final response you are interested in.
Third, you have to divide the progression from the starting point to the final target behavior into appropriate training steps or successive approximations
The execution of the training plan involves two complementary tactics: reinforcement of successive approximations to the final behavior and withholding reinforcement for earlier response forms.
Can shaping result in new behaviour?
In teaching the rat to press the bar, we are not teaching new response components. Rather, we are teaching the rat how to combine familiar responses into a new activity. Instrumental conditioning often involves the construction, or synthesis, of a new behavioral unit from preexisting response components that already occur in the organism’s repertoire
Instrumental Conditioning Procedures
What is the term for a pleasant event and an unpleasant stimulus?
A pleasant event is technically called an appetitive stimulus. An unpleasant stimulus is technically called an aversive stimulus.
What are the types of instrumental conditioning procedures?
Positive reinforcement
(punishment) Punishment
Negative reinforcement (escape or avoidance)
Negative punishment (omission training/differential reinforcement of other behavior)
Fundamental Elements of Instrumental Conditioning
What are the ways The Instrumental Response determines the result of positive reinforcement procedures?
Behavioral Variability Versus Stereotypy
Relevance or Belongingness in Instrumental Conditioning
Behavior Systems and Constraints on Instrumental Conditioning
Fundamental Elements of Instrumental Conditioning
The Instrumental Response
Behavioral Variability Versus Stereotypy
How can the criteria for reinforcement also be defined in terms of more abstract dimensions of behavior?
e.g., novelty
The behavior required for reinforcement can be defined as doing something unlike what the participant did on the preceding four or five trials
response variability
Fundamental Elements of Instrumental Conditioning
The Instrumental Response
What is Relevance or Belongingness?
“A behavior cannot be reinforced by a reinforcer if it is not naturally linked to that reinforcer in the repertoire of the animal”
e.g., yawning to be let out of a puzzle box
Fundamental Elements of Instrumental Conditioning
The Instrumental Response
What are Behavior Systems and Constraints on Instrumental Conditioning?
According to behavior systems theory, when an animal is food deprived and is in a situation where it might encounter food, its feeding system becomes activated, and it begins to engage in foraging and other food-related activities.
The effectiveness of the procedure in increasing an instrumental response will depend on the compatibility of that response with the preexisting organization of the feeding system.
responses that become more likely when the animal is hungry are readily reinforced with food, whereas responses that become less likely when the animal is hungry are difficult to train as instrumental responses.
Fundamental Elements of Instrumental Conditioning
What are the ways The Instrumental Reinforcer determines the result of positive reinforcement procedures?
Quantity and Quality of the Reinforcer
Shifts in Reinforcer Quality or Quantity - positive and negative behavioral contrast effects
Fundamental Elements of Instrumental Conditioning
What are the ways The Response–Reinforcer Relation determines the result of positive reinforcement procedures?
temporal relation
- temporal contiguity
response–reinforcer contingency
What is a secondary, or conditioned, reinforcer?
a conditioned stimulus that was previously associated with the reinforcer. Verbal prompts in coaching, such as “good,” “keep going,” and “that’s the way” are conditioned reinforcers that can provide immediate reinforcement for appropriate behavior
What is a marking procedure?
mark the target instrumental response in some way to make it distinguishable from the other activities of the organism
facilitates learning with delayed reinforcement
How can superstitious behavior occur?
accidental, or adventitious, reinforcement
high contiguity rather than contingency
Reinterpretation of the Superstition Experiment
What are terminal and interim responses?
What did Staddon and Simmelhag find?
terminal - some of the responses occurred predominantly toward the end of the interval between successive reinforcers
interim - somewhere near the middle of the interval between food deliveries
Which actions were terminal responses and which were interim responses did not vary much from one pigeon to another. Furthermore, Staddon and Simmelhag failed to find evidence for accidental reinforcement effects.
Explanation of the Periodicity of Interim and Terminal Responses
How did Staddon and Simmelhag explain terminal and interim responses?
Staddon and Simmelhag (1971) suggested that terminal responses are species-typical responses that reflect the anticipation of food as time draws closer to the next food presentation. By contrast, they viewed interim responses as reflecting other sources of motivation that are prominent early in the interfood interval, when food presentation is unlikely.
What is the learned-helplessness EFFECT?
exposure to uncontrollable shock disrupted subsequent learning
What is the learned-helplessness HYPOTHESIS?
The learned-helplessness hypothesis assumes that during exposure to uncontrollable shocks, animals learn that the shocks are independent of their behavior—that there is nothing they can do to control the shocks. Furthermore, they come to expect that reinforcers will continue to be independent of their behavior in the future.
First, the expectation of lack of control reduces the motivation to perform an instrumental response. Second, even if they make the response and get reinforced in the conditioning phase, the previously learned expectation of lack of control makes it more difficult for the subjects to learn that their behavior is now effective in producing reinforcement.
Alternatives to the Helplessness Hypothesis
What is the activity deficit hypothesis?
animals show a learning deficit following exposure to inescapable shock because inescapable shocks encourage animals to become inactive or freeze
However, it cannot explain instances in which exposure to inescapable shock disrupts choice learning
Alternatives to the Helplessness Hypothesis
What is the attention deficit hypothesis?
exposure to inescapable shock reduces the extent to which animals pay attention to their own behavior
Alternatives to the Helplessness Hypothesis
Stimulus relations in escape conditioning
Why is exposure to escapable shock not nearly as bad as inescapable shock?
Some of the response-produced stimuli experienced at the start of the escape response are called shock-cessation feedback cues.
Other response-produced stimuli experienced as the animal completes the response, just after the shock has been turned off at the start of the intertrial interval are called safety-signal feedback cues
Therefore, contextual cues of the chamber in which shocks are delivered are more likely to become conditioned to elicit fear with inescapable shock
