Chapter 10- Scheduled Of reinforcement

Question 1

A rule describing the delivery of reinforcers for a behaviour

Answer 1

Schedule of reinforcement

Question 2

The distinctive rate and pattern of responding associated with a particular reinforcement schedule

Answer 2

Schedule effects

Question 3

A reinforcement schedule in which a behaviour is reinforced each time it occurs

Answer 3

Continuous reinforcement CRF

Example: a rat receives food every time it presses a lever

Question 4

Any of several reinforcement schedules in which a behaviour is sometimes reinforced

Answer 4

Intermittent schedule

Also called partial reinforcement

Question 5

A reinforcement schedule in which every nth performance of a behaviour is reinforced

A behaviour is reinforced when it occurred a fixed Number of times

Answer 5

Fixed ratio schedule FR

Example: every third lever press is reinforced

Question 6

What response pattern does a six-ratio schedule typically generate

Answer 6

Animals on fixed ratio schedules perform at a high rate, often punctuated by short fathers after reinforcement

Question 7

A pause in responding following reinforcement; associated primarily with FI and FR schedules

Answer 7

Post-reinforcement pauses

Question 8

The rate at which a behaviour occurs once it has resumed following reinforcement

Answer 8

Run rate

Increasing the ratio of lever presses to reinforcers does not change how rapidly a rat presses once it has resumed lever pressing, but does increase the length of the breaks the rat takes after each reinforcement

Question 9

A reinforcement schedule in which, on average, every nth performance of a behaviour is reinforced

Answer 9

Variable ratio schedule, VR

Example: in a VR5 schedule, reinforcement may occur after 2 to 10 lever presses, but the overall average will be one reinforcement for every five presses

Most gambling games are based on variable reinforcement schedules in which the payoffs occur after a variable and unpredictable number of responses

Question 10

What response pattern does a variable ratio schedule typically generate?

Answer 10

Produces steady performance at run rates similar to comparable FR schedules. If post-reinforcement pauses occur, they usually appear less often and are for shorter periods then in a comparable FR schedule. Post reinforcement causes are strongly influenced by the size of the average ratio and by the lowest ratio

Question 11

A reinforcement schedule in which a behaviour is reinforced the first time it occurs following a specified interval since the last reinforcement

Answer 11

Fixed interval schedule, FI

Example: a pigeon on an FI 5” schedule will have food delivered into its tray the first time it pecks the disk, but for the next five seconds, disc pecking produces no reinforcement. Then, at the end of the five second interval, the very next disc peck is reinforced

Example: baking in the oven, waiting for a bus, studying

Question 12

Identify the response pattern that a fixed interval schedule typically generates

Answer 12

Produces post reinforcement pauses and a scalloped-shaped cumulative record.

Question 13

A reinforcement schedule in which a behaviour is reinforced the first time it occurs following an interval since the last reinforcement, with the interval varying around a specified average

Answer 13

Variable interval schedule, VI

In a VI 5” schedule, the average interval between reinforced pecks is five seconds.

Examples, human hunters lying in wait for game.

Question 14

Identify the response pattern that a variable interval schedule typically generates

Answer 14

Produce high, steady run rates, higher than FI schedules, but not so high as comparable FR and VR schedules

Question 15

A reinforcement schedule in which reinforcement is contingent on the continuous performance of a behaviour for a fixed period of time

Answer 15

Fixed duration schedule, FD

Example: the child who is required to practice playing a piano for half an hour, at the end of practice, and provided the child has practised the entire time, he receives a reinforcer such as a cookie

Question 16

A reinforcement schedule in which reinforcement is contingent on the continuous performance of a Behaviour for a period of time, with the length of time varying around an average

Answer 16

Variable duration, VD

Example: the child who is practising piano, any given session might and after 30 minutes, 55 minutes, 20 minutes, or 10 minutes. On average, the students will practice for half an hour before receiving the milk and cookies, but there is no telling when the reinforcers will appear

Question 17

A reinforcement schedule in which a behaviour is reinforced only if a specified period of time has elapsed since the last performance of that behaviour

Answer 17

Differential reinforcement of low rate, DRL

Example: a rat might receive food for pressing a lever, but only if five seconds have elapsed since the last lever press. The interval begins each time the behaviour is performed.

In a DRL 5” schedule, a pigeon that pecks a disk receives reinforcement only if five seconds have elapsed since the last disc peck

Question 18

Identify the response pattern that a differential-reinforcement-of-low-rate schedule typically generates

Answer 18

Produces extremely low rates of behavior. Sometimes results in the performance of a series of behaviours that are quite a relevant to reinforcement, and this behaviour may be superstitious

Question 19

A form of differential reinforcement in which a behaviour is reinforced only if it occurs at least a specified number of times in a given.

Answer 19

Differential reinforcement of high rate, DRH

Example: a pigeon might be required to pack a disc five times in a 10 second period. If it text fewer than five times during that period, it receives nothing

Question 20

Identify the response pattern that a differential reinforcement of high rate schedule typically generates

Answer 20

Can produce extremely high rates of behavior, higher than any other schedule

Question 21

A reinforcement schedule in which reinforcement is delivered independently of behaviour at fixed intervals

Answer 21

Fixed time schedule, FT

Example: in an FI 10” schedule, a pigeon may receive food after a 10 second interval, but only if it pecks a disk. Whereas in an FT 10” schedule, the pigeon receives food every 10 seconds whether it pecks the disk or not

Not common outside the laboratory. Unemployment compensation and welfare payments come close to meeting the definition of fixed time schedules

Question 22

A reinforcement schedule in which reinforcement is delivered at varying intervals regardless of what the organism does

Answer 22

Variable time schedule, VT

The only difference between VT schedules and FT schedules is that in VT schedules the reinforcer is delivered at intervals that vary about some average

Have been used to establish superstitious behaviour. Example, periodically a sportfishermen gets lucky and the use of a particular lure or kind of bait or method of casting is coincidentally reinforced

Question 23

The procedure of gradually increasing the number of responses required for reinforcement

Answer 23

Stretching the ratio

Also referred to as thinning the schedule

With the procedure known as stretching the ratio, successive approximations of the desired behaviour are reinforced, it is essentially the same shaping process used to shape any new behaviour

The experimenter gradually sheets persistence in this way

Example: when card sharks and pool hustlers sometimes let their competitors win frequently during the early stages of play and then gradually win more and more of the games

Question 24

Disruption of the pattern of responding due to stretching the ratio of reinforcement too abruptly or too far

Answer 24

Ratio strain

Example: workers who complain about being overworked and under paid and who shirk their responsibilities

Question 25

The density or frequency of a reinforcement schedule is a continuum. What extremes are on either end?

Answer 25

At one extreme we find continuous reinforcement, an FR schedule in which every single occurrence of a behaviour is reinforced. At the other extreme we find extinction, a schedule in which a behaviour is never reinforced

Question 26

The tendency of a behaviour to be more resistant to extinction following partial reinforcement than following continuous reinforcement.

Answer 26

Partial reinforcement affect or PRE

In a rat lever pressing study, the thinner the reinforcement schedule before extinction, the greater number of lever presses during extinction

Question 27

Why does the author state that the partial-reinforcement effect is paradoxical?

Answer 27

It is paradoxical since the law of effect implies that the unreinforced lever presses that occur during an intermittent schedule should weaken the tendency to press, not make it stronger.

Question 28

The proposal that the PRE occurs because it is harder to discriminate between intermittent reinforcement and extinction than between continuous reinforcement and extinction

Answer 28

The discrimination hypothesis

Example: it takes longer to discriminate between extinction and NFR 30 schedule then it does to discriminate between extinction and FR1 schedule

Question 29

The proposal that the PRE occurs because non-reinforcement is frustrating and during intermittent reinforcement frustration becomes an S+ for responding

Answer 29

Frustration hypothesis

The thinner the reinforcement schedule during training, the higher is the level of frustration when the rat finally receives food

Question 30

The proposal that the PRE occurs because the sequence of reinforced and non-reinforced behaviours during intermittent reinforcement becomes an S+ for responding during extinction

Answer 30

Sequential hypothesis

Attributes the PRE to differences in the sequence of cues during training
Extinction proceeds rapidly after continuous reinforcement because an important cue for performing is missing

The thinner the reinforcement schedule, the more resistant the rat will be to extinction, since a long stretch of non-reinforced lever pressing has become the queue for continued pressing. In other words, the rat performs in the absence of reinforcement because, in the past, long strings of non-reinforced presses have reliably preceded reinforcement

Question 31

The proposal that the PRE is due to differences in the definition of a behaviour during intermittent and continuous reinforcement

Answer 31

Response unit hypothesis

In an FR 2 schedule, where one press does nothing, but two presses produces food, we should not think of this as press-failure, press-reward, but rather as press-press-reward. The unit of behaviour being reinforced is two lever presses.

When responses are defined in terms of the units required for reinforcement, the total number of responses during extinction declines as the reinforcement schedule gets thinner. Behaviour on intermittent reinforcement only seems to be more resistant to extinction because we have failed to take into account the response units required for reinforcement

Question 32

A complex reinforcement schedule in which two or more simple schedules alternate, with each schedule associated with a particular stimulus

Answer 32

Multiple schedule

Abbreviation: MULTI in front of the different schedules

Example: a pigeon is reinforced for packing on an FI 10” schedule when a red light is on, but on AVR 10 schedule when a yellow light is on. The two reinforcement schedules alternate, with the changes indicated by changes in the colour of the light

Question 33

A complex reinforcement schedule in which two or more simple schedules, neither associated with a particular stimulus, alternate

Answer 33

Mixed schedule

MIX FI 10” VR 10 schedule: disc pecking might be reinforced on an FI 10” schedule for 30 seconds and then on VR 10 schedule for 60 seconds, but there is no clear indication that the schedule has changed

Question 34

A complex reinforcement schedule that consists of a series of simple schedules, each of which is associated with a particular stimulus, with reinforcement delivered only on completion of the last schedule in the series

Answer 34

Chain schedule

Question 35

A complex reinforcement schedule that consists of a series of simple schedules, with reinforcement delivered only on completion of the last schedule in the series. The simple schedules are not associated with different stimuli

Answer 35

Tandem schedule

Question 36

A complex reinforcement schedule in which reinforcement is contingent on the behaviour of two or more individuals

Answer 36

Cooperative schedule

Two pigeons receive food by picking a disk when the two of them have packed a total of 20 times. One might pack the disc at the rate of 10 times a minute while the other packs at 40 times a minute. As soon as the total number of packs reaches 20, they each receive a few pieces of green

Question 37

A complex reinforcement schedule in which two or more simple schedules are available at the same time

Answer 37

Concurrent schedule

A pigeon may have the option of picking a read disc on a VR 50 schedule, or packing a yellow disc on a VR 20 schedule. The concurrent schedule involves a choice

Question 38

The principle that, given the opportunity to respond on two or more reinforcement schedules, the rate of responding on each schedule will match the reinforcement available on each schedule

Answer 38

Matching law

Question 39

Why is it a poor strategy to switch back-and-forth between two different ratio schedules of reinforcement? Or, why is it a good strategy to stay with one ratio schedule?

Answer 39

It makes sense to identify the more reinforcing schedule as quickly as possible and remain loyal to it. Switching back-and-forth between two ratio schedules is pointless. You should discriminate between which schedule is more dense

Question 40

Why is it a good strategy to switch between two different interval schedules of reinforcement?

Answer 40

There will be periods during which lever pressing is useless. Some of this time could be spent pressing the lever on the other schedule. It therefore makes sense for the animal to devote most of its effort to the more dense schedule but occasionally press the lever on the thinner schedule

Question 41

What is Herrnstein’s formula that predicts choice in a two-choice situation? Describe each of the terms in the equation, and explain the overall meaning of the equation

Answer 41

Ba/Ba + Bb = rA/rA + rB

Ba and Bb represent two behaviors, behaviour A and behaviour B, and rA and rB represent The reinforcement rates for behaviours A and B, respectively.

This equation is merely a reformulation of the matching law

Question 42

What is Herrnstein’s formula that predicts choice in a multiple-choice situation? Describe each of the terms in the equation, and explain the overall meaning of the equation

Answer 42

Ba/Ba + Bo = rA/rA + rO

Ba represents the particular behaviour we are studying, and Bo represents all other behaviors, rA represents the reinforcers available for Ba, and rO represents the reinforcers available for all other behaviours

This formula has less predictive value then the formula for the two-choice situation, because it is not possible to specify all the behaviours that may occur, nor all the reinforcers those acts may produce

Reminds us that behaviour is a function of the reinforcers available for any behaviour that might occur, not merely the reinforcers available for the behaviour that interests us at the moment

Question 43

Cite and recognize original examples of the matching law describing human behaviour

Answer 43

A farmer may devote most available farmland to a crop that produces a nice profit under typical weather conditions, and planting a smaller area in a less profitable crop that does well under adverse weather conditions.

When we spend more time at a high-paying job then at a low-paying one

When college students devote more time to a five credit course then to a one credit course, since a high-grade in the former pays better than a high-grade in the latter

Question 44

What type of reinforcement schedule maintains gambling behavior? Explain how and why “early wins” and “near misses” can lead to compulsive gambling. If you were the owner of a gambling enterprise, how could you use this knowledge to make more money?

Answer 44

The payoff in most games of chance resembles variable ratio schedules of reinforcement, and such schedules can produce high rates of behaviour that are highly resistant to change

Momentary variations in schedules that lead to early wins and near misses can lead to compulsive gambling because the person is reinforced early on

If I owned a gambling enterprise, I would put my machines on a variable ratio schedule, and have people win during their first few bets, and after that have them almost win every once in a while

Question 45

The use of reinforcement schedules, among other techniques, to study economic principles

Answer 45

Experimental or behavioural economics

Question 46

Describe the ways in which reinforcement schedules have been studied in experimental or behavioural economics

Answer 46

Economists know that when the price of a luck Sharee item rises, the consumption of that item declines. But when the price of an essential item, such as food, rises, there is little change in consumption. The same phenomenon has been demonstrated in rats. Rats will work for psychoactive drugs, a luxury, but increases in the price of the drug, the number of lever presses required for a dose, usually results in decreased consumption; yet large increases in the price of food, an essential, do not lower consumption substantially

Psychiatric patients who earn tokens for performing various tasks and exchanges them for cigarets and other items are given a choice between activities for which tokens are available (ex. Doing laundry), and other activities for which reinforcers are available (watching tv). The distribution of the tokens among patients resembled the United States population. Those in the top 20% held a total of 41% of all tokens, while those in the bottom 20% held only 7%.

Question 47

Explain how Goldberg and Cheney set up an experimental analogue for malingering. What implications does this animal study have for human behavior?

Answer 47

Tested the idea that operant behaviour associated with chronic pain may be maintained by reinforcement after the pain has ceased. Pairs of rats on a cooperative schedule where one was exposed to a mild electric shock resulted in an abrupt reduction in the amount of work done by the rat in pain. These rats continue to press the lever, but at a much lower rate than during baseline.
After the shock was terminated, the rats continue to work at a lower pace, only gradually increasing their share of the work load. This is interesting because the slower rate of work reduced the amount of food both rats received. Although the partner read could take up the slack, it necessarily took longer to reach the 50 lever press is required for reinforcement when one rat did little Work

Suggests that there is good reason to believe that people made malinger if others are willing to press the lever more often to make up for someone who appears to be hurting. Malingering may occur even though everyone, including the malingerer, loses by it

Question 48

What criticisms have been made of research in reinforcement schedules?

Answer 48

Argue that the schedules of reinforcement studied in the laboratory are artificial constructions not found in the real world

Complain that schedules research generally produces trivial findings

Complain that reinforcement schedules reveal considerably more about rats and pigeons then they do about people.

Question 49

Why is examining the effects of schedules preferable to explaining behaviour in terms of personality characteristics and state of mind?

Answer 49

These kinds of explanations merely name the behavior to be explained. Identifying the kinds of reinforcement schedules that produce these behaviours is a considerable advance

Question 50

What other advantages accrue from research into schedules of reinforcement

Answer 50

When the goal is to discover rules that describe the way the environment affects behaviour it is difficult if not impossible to discover such rules unless the experimenter simplifies the environment as experimenters do with research into schedules of reinforcement

Allows us to answer questions that might be difficult to answer otherwise

Gives us a more scientific way of accounting for differences in behaviour

The only reason that studies with humans sometimes reveal patterns of behaviour different from those obtained with animals may be because human subjects often receive instructions about what they are to do which have a powerful effect on human behaviour

Provides a very good way of testing the effects of variables on behaviour

Question 51

How can reinforcement schedules be used as a baseline to study the effects of different independent variables on behavior?

Answer 51

Example: researchers trained rats to press a lever to get access to an exercise wheel and later administered cocaine in varying amounts 10 minutes prior to running the rest. The cocaine had no detectible influence on the pattern of behaviour until the researchers reached a dosage of 16 mg per kilogramme of body weight. Add to this level, the scalloped pattern of the FI schedule began to deteriorate. In much the same way, researchers used schedules as a basis for comparing the effects of alcohol and cocaine on human performance. Schedule performance can provide a baseline for evaluating the effects of toxins, diet, sleep deprivation, exercise, brain stimulation, and many other variables

Question 52

Provide and recognize original examples of learning in which no new behaviour is acquired

Answer 52

Example: an increase in the rate of behavior.
A pigeon that turns in counterclockwise circles at the rate of three or four a minute may learn to make 10 or 15 turns a minute

Example: a reduction in the rate of behaviour
A bird that turns counterclockwise 10 times a minute can learn to make one turn a minute

Example: a change in the pattern of performance as well as the rate
The cook learns to avoid opening the oven in the first few minutes but to check on the cookies more and more often during the last few minutes of baking time