Ch. 10

Question 1

concurrent schedule of reinforcement

Answer 1

consists of the simultaneous presentation of two or more independent schedules, each leading to a reinforcer.

The organism is thus allowed a choice between responding on one schedule versus the other.

Choice between concurrent VR schedules is easy because an exclusive preference for the richer alternative clearly provides the better payoff.

concurrent VI schedules —> It is so systematic, in fact, that it led to the formulation of what is known as the matching law.

Question 2

matching law

Answer 2

holds that the proportion of responses emitted on a particular schedule will match the proportion of reinforcers obtained on that schedule (note that it is proportion of responses and reinforcers and not number of responses and reinforcers).

The matching law therefore predicts a consistent relationship between the proportion (or percentage) of reinforcers obtained on a certain alternative and the proportion (or percentage) of responses emitted on that alternative.

Matching appears to be a basic principle of choice behavior, applicable to a variety of situations and species.

principle of matching may underlie various aspects of human social interaction.

Question 3

matching law expressed as a formula

Answer 3

R_A / R_A + R_B = S^R_A / S^R_A + S^R_B
Or

R_A / (R_A + R_B) = S^R_A / (S^R_A + S^R_B)

• R represents the number of responses emitted
• S^R is the number of
  reinforcers earned
• the subscripts A and B refer to the two schedules of reinforcement.
• Thus, R^A is the number of responses emitted on schedule A,
• R^B is the number of responses emitted on schedule B,
• S^R_A is the number of reinforcers earned on schedule A,
• S^R_B is the number of reinforcers earned on schedule B.

Question 4

R_A / (R_A + R_B)

Answer 4

Therefore, the term to the left of the equal sign:

indicates the proportion of responses emitted on schedule A. It is the number of responses emitted on schedule A divided by the total number emitted on both schedules.

Question 5

S^R_A / (S^R_A + S^R_B)

Answer 5

The term to the right of the equal sign:

indicates the proportion of reinforcers earned on schedule A. It is the number of reinforcers earned on schedule A divided by the total number earned on both schedules.

Question 6

Cher ami matching example

Answer 6

given a choice between responding on a VI 20-sec versus a VI 60-sec schedule of reinforcement.

1st assumption one should make is that she will go back and forth between the two alternatives in order to pick up all or almost all of the available reinforcers on each schedule _{which is in fact what hungry pigeons readily learn to do.}

She will earn about 3 times as many reinforcers on the VI 20-sec schedule _{(which provides an average of three reinforcers per minute)} as on the VI 60-sec _{(which provides an average of one reinforcer per minute).}

Imagine that during a 60-minute session, this is precisely what she does: she earns all of the 180 reinforcers that became available on the VI 20-sec schedule during that one hour session and all of the 60 reinforcers that became available on the VI 60-sec schedule.

insert these values into the right-hand (reinforcer) side of the equation:

SR V1 20s/ (SR V1 20s + SR V1 60s) = 180/ (180 + 60) = 180/240 = .75

This means that the proportion of reinforcers that Cher Ami obtained from the VI 20-sec schedule during that session was .75.

In other words, 75% (or three-fourths) of the reinforcers that she earned were obtained from the VI 20-sec schedule, which also means that the remaining 25% (or one-fourth) of the reinforcers were obtained from the VI 60-sec schedule

Therefore, as expected, Cher Ami obtained three times as many reinforcers from the VI 20-sec schedule as from the VI 60-sec schedule (75% vs. 25%).

Question 7

Cher Ami example part 2

The question now is, did Cher Ami match the proportion of responses she emitted on each alternative to the proportion of reinforcers she obtained from each alternative?

Answer 7

To answer this question, we have to look at the actual number of responses she emitted on each alternative.

Suppose, for example, that during that one-hour session, she emitted 2,240 responses on the VI 20-sec schedule and 775 responses on the VI 60-sec schedule.

She clearly emitted more responses on the richer schedule, but did she closely match proportion of responses to proportion of reinforcers obtained?

To determine this, we need to insert those response numbers into the left-hand (response) side of the equation.

R VI 20s:/(R VI 20s + R VI 60s) = 2,240/(2,240 + 775) = 2,240/3015= .74

This shows that Cher Ami emitted .74 (or 74%) of her responses on the VI 20-sec schedule, which closely matches the proportion of reinforcers, 75 (or 75%), that she obtained on the VI 20-sec schedule.

Thus, as predicted by the matching law, the proportion of responses that Cher Ami emitted on the VI 20-sec schedule closely matches the proportion of reinforcers that she obtained on that schedule.

_{Note that she did not have to distribute her responses in this way in order to pick up all of the available reinforcers. For example, she could just as easily have obtained all of them by distributing her responses equally between the two alternatives. But she didn’t do that; instead, in keeping with the matching law, she closely matched the proportion of responses she emitted on an alternative to the proportion of reinforcers she obtained on that alternative.}

Question 8

If asked to calculate what the matching law predicts concerning the expected proportion of responses emitted on two concurrent VI schedules of reinforcement, you should:

Answer 8

(1) assume that the pigeon (or rat or person) will earn all of the available reinforcers on each schedule during the session, and

(2) insert the expected number of reinforcers earned on each schedule into the right hand (reinforcer) side of the matching equation.

This will give you the expected proportion of reinforcers that will be earned on each schedule.

According to the matching law, this proportion will also be the predicted proportion of responses emitted on each schedule.

• Note that you do not need to bother with the left-hand (response) side of the equation in making this prediction.

You use that side of the equation only if you have already run such a session and have actual response numbers to work with.
— this will then enable you to confirm whether the pigeon did in fact match the proportion of responses emitted to the proportion of reinforcers earned on each alternative.

Question 9

undermatching

Answer 9

the proportion of responses on the richer schedule versus the poorer schedule is less different than would be predicted by matching (to remember this, think of undermatching as less different).

Undermatching can occur when there is little cost for switching from one schedule to another.

Question 10

changeover delay or COD.

Answer 10

In experimental studies of matching, the act of switching from one schedule to another results in a changeover delay
a short period of time that must pass before any response can produce a reinforcer.

_{It is as though when the pigeon switches from one key to another, the first peck on the new key is simply a statement of intent that says, “I now want to try this key.” Then there is a two-second delay before any peck can actually earn a reinforcer.}

_{Without a COD, a pigeon will eventually learn to alternate pecks back and forth on each key, catching each reinforcer as soon as it becomes available. Only when a slight cost for switching is added to the situation does the pigeon spend more time on the richer alternative.}

Question 11

foraging situation is analogous to?

Answer 11

COD

This experimental procedure seems analogous to foraging situations in which an animal has to travel a certain distance from one food patch to another.

Question 12

overmatching,

Answer 12

the proportion of responses on the richer schedule versus the poorer schedule is more different than would be predicted by matching (to remember this, think of overmatching as more different).

Overmatching can occur when the cost of moving from one alternative to another is very high.

Question 13

Bias from matching

Answer 13

Occurs when one response alternative attracts a higher proportion of responses than would be predicted by matching, regardless of whether that alternative contains the richer or poorer schedule of reinforcement.

Bias in matching can, therefore, be used to indicate degree of preference for different reinforcers.

_{For example, suppose that our two schedules are VI 20-sec and VI 60-sec, and that we alternate which schedule is associated with a red key versus green key.}

_{The matching law predicts that the proportion of responses on the red key should be .75 when the richer VI 20-sec schedule is presented on it, and .25 when the poorer VI 60-sec schedule is presented on it.}

_{But if the proportions instead turned out to be .85 when the VI 20-sec schedule is presented on it and .35 when the VI 60-sec schedule is presented on it, then bias has occurred.}

_{The pigeon is emitting 10% more responses on the red key than predicted by matching, both when it is the richer alternative and when it is the poorer alternative. (Of course, this also means that the pigeon is emitting 10% fewer responses on the green key.)}

Question 14

As with the phenomenon of behavioral contrast

Answer 14

the matching law reminds us that operant behavior should often be viewed in context.

The amount of behavior directed toward an alternative is a function of the amount of reinforcement available on that alternative as well as the amount of reinforcement available on other alternatives.

This notion has important implications for everyday behavior.

Question 15

melioration theory

Answer 15

the distribution of behavior in a choice situation shifts toward those alternatives that have higher value regardless of the long-term effect on the overall amount of reinforcement.

This shifting will cease at the point where the two outcomes are approximately equal in terms of costs versus benefits.

Question 16

Why would an animal or person not shift their behavior toward the higher values alternative.

The problem is that this tendency to move toward the higher valued alternative can sometimes result in a substantial reduction in the total amount of reinforcement obtained.

1st way this can occur

Answer 16

First, an alternative might not require as much responding as one is distributing toward it to obtain all of the available reinforcers.

Question 17

Why would an animal or person not shift their behavior toward the higher values alternative.

The problem is that this tendency to move toward the higher valued alternative can sometimes result in a substantial reduction in the total amount of reinforcement obtained.

2nd way this can occur

Answer 17

A second problem with melioration is that overindulgence in a highly reinforcing alternative can often result in long-term habituation to that alternative, thus reducing its value as a reinforcer.

This means that our enjoyment of life may be greatest when we do not have all that we desire.

Question 18

Why would an animal or person not shift their behavior toward the higher values alternative.

The problem is that this tendency to move toward the higher valued alternative can sometimes result in a substantial reduction in the total amount of reinforcement obtained.

3rd way this can occur

Answer 18

which is that melioration is often the result of behavior being too strongly governed by immediate consequences as opposed to delayed consequences.

Question 19

Behavioral approaches largely __ the concept of willpower as an explanation for self-control.

Answer 19

Reject

Question 20

Skinner on Self-Control

Answer 20

_{viewed self-control, or “self-management,” not as an issue of willpower but as an issue involving conflicting outcomes.}

An event can lead to both positive outcomes and negative outcomes.

Skinner also proposed that managing this conflict involves two types of responses: a controlling response that serves to alter the frequency of a controlled response.

Question 21

Skinners controlling response:

Physical Restraint

Answer 21

With this type of controlling response, you physically manipulate the environment to prevent the occurrence of some problem behavior.

Question 22

Skinners controlling response:

Depriving and Satiating

Answer 22

Another tactic for controlling your behavior is to use the motivating operations of deprivation and satiation to alter the extent to which a certain event can act as a reinforcer.

Question 23

Skinners controlling response:

Doing Something Else

Answer 23

To prevent yourself from engaging in certain behaviors, it is sometimes helpful to perform an alternate behavior.

Question 24

Skinners controlling response:

self-reinforcement and self-punishment

Answer 24

A self-control tactic that might seem obvious from a behavioral standpoint is to simply reinforce your own behavior.

_{Although Skinner suggested that this might work, he also noted a certain difficulty with it.}

_{In the typical operant conditioning paradigm, the reinforcer is delivered only when the appropriate response is emitted. The rat must press the lever to receive food}

_{In the case of self-reinforcement, however, this contingency is much weaker. To use Martin and Pear’s (1999) terminology, what is to stop you from “short-circuiting” the contingency and immediately consuming the reward without performing the intended behavior?}

_{A similar problem exists with the use of self-punishment.}

_{some evidence that, for many people, self-delivered consequences are more effective when the person perceives that other people are aware of the contingency, suggesting that the social consequences for attaining or not attaining the intended goal are often an important aspect of self-reinforcement and self-punishment procedures}

_{Bandura believes that self-reinforcement and self-punishment can work for people who are likely to feel guilty if they violate standards that they have set for themselves.}

Question 25

From a temporal perspective, self-control problems arise from

Answer 25

the extent to which our behavior is more heavilv influenced by immediate consequences.

Question 26

Self-control is shown by choice of a ___ reward over a ___ reward. It can also be shown by choice of a ___ punisher over a ___ punisher.

With respect to choice between rewards, the opposite of self-control is called ___ , which is demonstrated by choice of a ___ reward over a ___ reward.

Answer 26

1. Larger later
2. smaller sooner
3. smaller sooner
4. larger later
5. impulsiveness,
6. smaller sooner
7. larger later

An additional problem in self-control situations is that the delayed consequences tend to be less certain than the immediate consequences.

Question 27

Children who are ___ successful at a delay of gratification task generally keep their attention firmly fixed on the desired treat.

While waiting for dessert, Housam imagines that the Jell-O looks like wobbly chunks of glass.

By contrast, Ruby views the Jell-O as, well, Jell-O. Between the two of them,
___ is less likely to get into trouble by eating the Jell-O before being told that it is okay to do so.

This is because delay of gratification can be enhanced by thinking about the desired reward in ___ rather than ___ terms.

Answer 27

1. Least
2. Housam
3. abstract
4. concrete

Question 28

The Ainslie-Rachlin model

Answer 28

is based on the finding that as a reward becomes imminent, its value increases more and more sharply yielding a “delay curve” (or delay function) that is upwardly scalloped.

_{At an early point in time, when both rewards are still distant, the larger later reward (LLR) is clearly preferred. As time passes, however, and the smaller sooner reward (SSR) becomes imminent, its value increases sharply and comes to outweigh the value of the LLR.}

If confronted by a choice between one food pellet available in 10 seconds and two food pellets available in 15 seconds, a rat would likely choose the latter. But if 9 seconds were allowed to pass before the rat could make that choice, then it would likely choose the former

Question 29

Preference reversal

Answer 29

occurs when the smaller sooner reward becomes imminent, and its value rises above the value of the other reward.

Given that this type of preference reversal occurs, the question arises as to whether anything can be done about it. Two alternatives suggest themselves:

(1) raise the value of the LLR at early points in time and

(2) lower the value of the SSR when it becomes imminent.

Question 30

Changing the Shape of the Delay Function for the Larger Later Reward

Answer 30

Preference reversal occurs because the LLR has low value at long delays; that is, its delay curve is deeply scalloped.

If the delay curve were less deeply scalloped - meaning that the value of the LL did not decline so drastically as a function of delay- then it would stand a better chance of outweighing any temptations that arise along the way.

Question 31

Herrnstein (1981) suggested several variables that can affect the shape of a delay function.

Answer 31

1. there appear to be innate differences in impulsivity between species.
2. There may be individual differences, with some individuals being more impulsive than others.
3. People generally become less impulsive as they grow older.
4. People generally become less impulsive after repeated experience with responding for delayed rewards.
5. The availability of other sources of reinforcement may also serve to reduce impulsiveness.
6. we can more easily maintain responding for a distant goal by setting up an explicit series of subgoals.

Question 32

commitment response

Answer 32

(also known as a recommitment response) can be defined as an action carried out at an early point in time that serves either to eliminate or greatly reduce the value of an upcoming temptation.

Question 33

Behavioral contracting,

Answer 33

in which a person formally arranges to attain certain rewards for resisting temptation or receive certain punishers for yielding to temptation, essentially operates on this principle.

The contract is negotiated with the therapist at an early point in time, before encountering the temptation.

The contingencies outlined in the contract serve to reduce the attractiveness of the tempting alternative.

Of course, in some circumstances, it might even be possible to completely eliminate the tempting alternative.

Question 34

small-but cumulative effects model

Answer 34

each individual choice on a self-control task has only a small but cumulative effect on our likelihood of obtaining the desired long-term outcome.

This model highlights the potential usefulness of establishing clear rules for distinguishing between acceptable and unacceptable behaviors, since the point at which impulsive behavior becomes harmful is not clear.

Question 35

limited resource model of self-control

Answer 35

(also referred to as the strength or ego-depletion model), self-regulatory behaviors are fueled by a type of limited internal energy.

A key proposition of the model is that this internal resource which is what most people would call “willpower” is depleted by tasks that require mental or emotional effort, such as studying, solving puzzles, or resisting a temptation.

This in turn results in a decrease in our ability to self-regulate, as indicated by a significant reduction in performance on subsequent tasks or by an increased likelihood of impulsive behavior.

_{reported that ego-depleting tasks significantly reduced blood glucose levels, and that the usual decrement in performance following such tasks could be reversed with the consumption of a sugary solution. These results suggest that glucose is the underlying factor that fuels a person’s willpower and self-control.}