Chap 13 Flashcards by Zaya Esterberg

Concurrent Schedule

Any situation in which two or more reinforcement schedules are presented simultaneously can be called a

How well did you know this?

Not at all

Perfectly

Herrnstein Experiment

Pigeon chamber with 2 response keys (red on left, white on right).
Grain reinforcer
*Schedule of two keys independent
The proportion of responses on the left key equaled, or matched, the proportion of reinforcers delivered by the left key.

How well did you know this?

Not at all

Perfectly

Herrnstein Matching Law

(B1)/(B1 + B2) = (R1)/(R1 + R2)
* B1 is number of responses of type 1 (left key)
* B2 is number of responses of type 2 (right key).
* R1 is the number of reinforcers obtained making response 1.
R2 is the number of reinforcers obtained making response 2.
* States that in a 2 choice situation, the proportion of responses directed toward one alternative should equal the proportion of reinforcers delivered by that alternative.

How well did you know this?

Not at all

Perfectly

Studies such as Davison and Baums show that

When the environment introduces rapid changes in the choice alternatives, animals can adapt with rapid changes in their choice responses.

How well did you know this?

Not at all

Perfectly

The matching law has been applied with reasonable success in a wide range of experiments with both animals and humans.

True

How well did you know this?

Not at all

Perfectly

Conger + Killeen experiement

College kids + 3 people who agree with one person more
Found that percentage of time the participants spent talking to each confederate approximately matched the percentage of verbal reinforcers delivered by that confederate.

How well did you know this?

Not at all

Perfectly

W.M. Baum listed 3 ways that the results of experiments have deviated from strict matching.

Under matching: response to proportions are consistently less extreme than reinforcement proportions.
* Can occur if subjects develop a habit of rapidly switching back and forth between the two options, a pattern that could be accidentally reinforced if food was delivered immediately after a switch.
* animals may occasionally attribute a reinforcer to the wrong response.
Over matching: response proportions are more extreme than the reinforcement proportions.

How well did you know this?

Not at all

Perfectly

1.5 changeover delay

A penalty for switching between keys.
A pigeon had to make 2 or more consecutive responses on the same key before collecting a reinforcer, thereby making the adventitious reinforcement of switching behavior less likely.

How well did you know this?

Not at all

Perfectly

Miller bias modification of (13-1)

(B1)/ (B1 + B2) = = (Q1R1)/(Q1R1 + Q2R2)

*Q1 and Q2 stand for the qualities of the reinforcers available on the two keys.
* States that a pigeons behavior is determined by both the rate of reinforcement and the quality of reinforcement delivered by the different schedules.

How well did you know this?

Not at all

Perfectly

Another variable that can affect preference is the

Amount or size of each.

EX: If one key delivers 2 food pellets as a reinforcer and the other deliver only one, this should certainly affect a subject’s choices.

How well did you know this?

Not at all

Perfectly

Herrnstein assumed

That there are always some built in reinforcers available for performing other behaviors such as grooming, exploring, and resting.

*Whereas the experimenter can control the number of food reinforcers, the built-in reinforcers for non pecking behaviors are out of the experimenters controls, and they occur at a fairly constant rate.

How well did you know this?

Not at all

Perfectly

Baum and Rachlin amount modification of (13-1)

(B1)/(B1 + B2) = (A1)/(A1 + A2)
* A1 and A2 are the amounts of reinforcement delivered by the two alternatives.

How well did you know this?

Not at all

Perfectly

Measure All behaviors in units of time (equation 13-4)

(T1)/(T1 + T2) = (R1)/(R1 + R2)

*Let T1 represent the time spent key pecking and T2 the total time spent in all other behaviors so these two = the total session time.
* R1 is the rate at which food reinforcers are delivered by the VI schedule, and R2 is the equivalent reinforcing value of all the built in reinforcers.

How well did you know this?

Not at all

Perfectly

Martens and Houk

Showed that equation13-4 nicely predicted the relation between the behavior of a girl who had a mental disability and the amount of reinforcement delivered by her teacher.

How well did you know this?

Not at all

Perfectly

Herrnsteins theory can be stated quite simply:

An operant response must compete with all other possible behaviors for the individuals time.

*As the reinforcement for the operant response increases, the individual will devote more and more time to this behavior.

ALTHOUGH

It is impossible to predict how a reinforcer will affect a behavior without taking into account the context, that is, the other reinforcers that are simultaneously available for other behaviors.

How well did you know this?

Not at all

Perfectly

Descriptive statements

These can be extremely useful as they can help us to predict and control future events.

EX: statement that water increases in volume when it freezes is simply a description - it does not explain why this expansion occurs.

How well did you know this?

Not at all

Perfectly

Explanations

A statement that attributes this expansion to the crystalline structure that hydrogen and oxygen molecules form when in a sold state can be called a explanation
It is a theory about the molecular events that underlie this phenomenon.

How well did you know this?

Not at all

Perfectly

Herrnstein theory of choice behavior

States that animals exhibit matching behavior because they are built to do so.
In any choice situation, an animal might measure the value of the reinforcement it receives from each alternative (where “value” encompasses such factors as the rate, size, and quality of the reinforces), and the animal then might distribute its behavior in proportion to the values of the various alternatives.

How well did you know this?

Not at all

Perfectly

Studies with both animals and people have found that they always choose

The smaller VR schedule in situations, never the larger.

How well did you know this?

Not at all

Perfectly

Herrnstein and Vaughan melioration equation (13-5).

States that animals will invest increasing amounts of time and or effort into whichever alternative is better.
(R1/B1) = (R2/B2)
emphasizes the fact that at the point of matching, the ratio of reinforcers received to responses produced is equal for both alternatives.
States that if these ratios are not equal, the animal will shift its behavior toward whichever alternative currently has the higher reinforcer: response ratio, until it reaches a point where the two ratios are equal.
R1= 2 times R2
B1 = 2 times B2

*Predicts that matching behavior will occur in concurrent VI schedules.

How well did you know this?

Not at all

Perfectly

The predictions are the same for any pair of VR schedules:

The individual should eventually respond exclusively on the schedule with the more favorable reinforcer:response ratio.

*In summary: The principle of melioration correctly predicts matching behavior in a choice between two VI schedules, and it predicts exclusive preference for the better of two VR schedules.

How well did you know this?

Not at all

Perfectly

Why are the matching theory and melioration theory viewed as competitors for optimization?

The principles of matching and melioration do not always lead to the optimal choices

*To put simply, matching behavior proportions to reinforcer proportions is not always the best strategy.

How well did you know this?

Not at all

Perfectly

Optimization Theory

States that people tend to make decisions that maximize their satisfaction.
Predicts preference for the better of the two VR schedules- this behavior maximizes reinforcement and minimizes effort.
An animal in this type of situation will try different ways of distributing its behaviors, and the animal will eventually settle on the response distribution that maximizes the overall rate of reinforcement.

How well did you know this?

Not at all

Perfectly

Supporters of optimization theory propose

That although the matching law may provide satisfactory description of behavior in these situations, optimization theory actually provides an explanation of matching behavior.

How well did you know this?

Not at all

Perfectly

Rachlin and colleagues proposed

That with any typical concurrent VI schedules, matching behavior will maximize the rate of reinforcement.
With concurrent VI schedules, it just so happens that the maximum rate of reinforcement can be obtained by matching.

How well did you know this?

Not at all

Perfectly

Herrnstein and colleagues obtained results that

Favored the matching law over optimization theory.

How well did you know this?

Not at all

Perfectly

Results of studies:

Although optimization theory predicted that the bird should always make about 50 % of their responses on each key, the birds consistently hows a preference foe whichever key delivered more reinforcers, as predicted by the matching law. But by doing so, they slowed dow the VI clock and exactly the opposite of what optimization theory predicted should happen.

How well did you know this?

Not at all

Perfectly

Optimization theory predicts

*That animals should make most of their responses on the VR schedule, because most of the responses on any VI schedule are wasted, whereas every response on a VR schedule brings the animal closer to reinforcement

BUT

Several experiments with animals failed to support this prediction, but the results were consistent with the predictions of the matching law.

How well did you know this?

Not at all

Perfectly

Because they deal with an individuals overall distribution of responses over long periods of time:

All three of these theories can be classified as molar theories.

How well did you know this?

Not at all

Perfectly

Some researchers now believe that more complete explanations of choice behavior will be found in molecular theories, which attempt to predict the moment-to-moment behavior and which assumes that short-term consequences have large effects on choice.

True

Momentary maximization theory

States that at each moment, an organism will select whichever alternative has the highest value AT THAT MOMENT.

The value of an alternative will usually depend on many factors:

The size and quality of reinforcer
The individuals state of deprivation

and more

Strategies of momentary maximization and overall optimization may lead to very different decisions.

True

Experiment for momentary maximization

Modified roulette wheel- probability of winning is .1 for all doors.
*Once a dollar is deposited behind a door, it will remain there until you collect it.
*There will never be more than one dollar behind a door at one time.

*Theory predicts that the player will choose whichever alternative has the higher probability of reinforcement on each trial.

Momentary maximizing theory predicts about a subjects behavior on concurrent VI schedules:

Predicts that there should be an orderly and cyclical pattern to an animals moment-by-moment choices.

Momentary maximizing theorist predict:

Animals will show at least some tendency to choose the alternative that has the higher probability of reinforcement.
Matching behavior is simply an incidental by-product of an animals orderly moment-by-moment choices.

Nevin concluded that

The momentary maximizing theory DOES NOT provide the correct explanation of why animals match, because matching behaviors sometimes occurs even when the animals choices appear to be random from moment to moment.

Hinson and Staddon experiment

They showed that their pigeons could follow a momentary maximizing strategy if they used a fairly simple rule:

If schedule 1 delivers, for example, three times as many reinforcers as schedule 2, you should check schedule 2 if the time since you last checked it is more than three times longer than the time since you last checked schedule 1.

They showed that their pigeons behaviors were by no means perfect from the standpoint of momentary maximization theory, but a majority of their responses did follow this rule.

Not all molecular theories of choice assume that animals follow the principle of momentary maximization.

True

Baum and colleagues proposed:

That on concurrent VI schedules, animals follow a pattern that they call FIX AND SAMPLE.

Analysis: Animals respond mostly on the better alternative but occasionally make short visits to other alternative.
Visits to better alternative are very long of it delivers a large percentage of reinforcers (90%) but not as long if it delivers only slightly more reinforcers than the other alternative (60%).

*He as presented evidence that this pattern of fixing and sampling at the molecular level leads to matching at the molar level.

Hybrid Theories

They assume that both molar and molecular variables affect choice.

EX: Fantiono’s delay-reduction theory

Fantiono’s delay-reduction theory

Include basic idea of matching law, but in addition it assumes that animals choices are directed toward whichever alternative produces a greater reduction in the delay to the next reinforcer.

Herrnstein and Mazur argued

That this tendency to switch preferences over time in self-control choices is one of the strongest pieces of evidence against optimization theory.

*If people followed this strategy that optimized their satisfaction in the long run, they would consistently choose one alternative or the other.

Regardless of which theory of choice proves to be the most accurate, no one can dispute the more general assertion of molecular theories that short-term factors have a large effect on choice behavior.

True

Indifference point

A combination of delays and amounts that the student finds equally preferable.

Self-control choice

Situation involving a choice between a small, proximal reinforcer and a larger but more distant reinforcer.

Important characteristic: Ones preferences may exhibit systematic changes over time.

Delay Discounting

Self-control choices illustrate quite dramatically how the strength or value of a reinforcer decreases as its delay increases. The effect is called:

Precommitment

The individual makes a decision in advance, which is difficult or impossible to change at a later time.

One main finding

The rates of delay discounting are different for different people

Green, Fry, and Myerson found

Rates of delay were fastest for 12 yr olds, slower for 20 yr old college students, and slowest for adults in their 60s.

George Ainslie - The Anslie-Rachlin Thoery

Assumes that the value of a reinforcer decreases as the delay between making a choice and receiving the reinforcer increases (with greater delays between choice and the delivery of the reinforcer, its value decreases).

*Assumes that an individual will choose whichever reinforcer has the higher value at the moment the choice is made.

EX: Because the proximity of sleep is now greater than that of the good grade, the student chooses to more immediate reinforcer.

Major difference between self-control situations described by the previous section and the following animal research is:

The time scale involved
With pigeons, rats, and other animals, a delay of a few seconds can often make the difference between self-control and impulsiveness.

Green, Fischer, Perlow, and Sherman experiment of preference reversals we would expect if the Ainslie-Rachlin theory is correct:

A peck at a red key gave 2 seconds of grain. Peck at green key gave 6 seconds of grain. But, short delay between peck and delivery of reinforcer. 2 second delay = 2-second reinforcer and same for 6.
*Birds showed impulsive behavior on nearly every trial, choosing 2-second reinforcer.
Inconsistent with optimization theory.
*Shift in preference when both reinforcers are farther away is what A-R model predicts.

Ainslie pre-commitment experiment:

Pigeons had to choose between 2 seconds of food delivered immediately and 4 seconds of food delivered after short delay. Pigeons chose immediate food 95% of time.
*However, pigeons had opportunity to peck a green key that served as a pre-commitment to larger delayed reinforcer. When key was green, both reinforcers were several seconds away.
3 of ten pigeons learned to use pre commitment option.

When the alternatives in a self-control situation are punishers rather than reinforcers, they have the reverse effect on choice.

True

Grosch + Neuringer expierment

A pigeon could either wait 15 seconds and then eat preferred grain or peck a key and receive a less preferred type of grain immediately.
They found that they would wait for this reinforcer on about 80% of the trials.
*Experimenters then showed food and pigeons became much more impulsive (waited for preferred type of grain for only 15% of trials.)
*Also found that pigeons were more likely to wait for delayed reinforcer if they had the opportunity to engage in some specific activity during delay.

Researches have found that the tendency to wait for a large delayed reinforcer is related to a

Childs age
Childs IQ
Other factors

Children who had “responsive, cognitively stimulating parent-toddler interactions” at age 2 tended to be less impulsive at age 6.

True

Mischel found that

A Childs behavior can be influenced by observational learning.

Self control choices are heavily influenced by observational learning

True

Mischel, Shoda, and Rodriquez found that

Self control ability at age 4 was correlated with abilities to concentrate on a task, to pursue goals, to resist temptation, and to tolerate frustration.

Self reinforcement

Individual who delivers his or her own reinforcers for the appropriate behaviors.

*Problem = easy to cheat.

Complimentary strategy of self reinforcement

is to make the value of the impulsive option lower by attaching some form of punishment.

Rule- governed behavior

Basic idea that people can be taught to use verbal rules to guide their choices toward the larger, delayed reinforcer.

It has been suggested that selectively thinking about the large, delayed reinforcer can forestall and impulsive action.

True

There is evidence that giving people training to improve their short term memories can lead to greater self-control, presumably because it enhances their ability to think about future rewards.

True

An interesting fact about choices involving uncertain consequences is that sometimes people seem to prefer a risky alternative, and sometimes they prefer a safe alternative instead. The same has been found for animals.

True

Caraco, Martindale, and Whittam expierment

If bird went to one site it would receive one millet seed every time. If it went to another, it had a 50% chance of getting two or a 50% of getting none.
*Found that if trials followed one another rapidly, the birds preferred the single.
*However, if trials had longer delays, birds preferred 50% chance of getting 2 seeds.

*Conclusion: If food is abundant animals will have no need to take risk, if food is scarce animals will take risk.

Humans who need to earn a certain amount of money also tend to be risk-prone when resources are scarce and risk-averse when resources are plentiful.

True

March and Shapira propsed:

Individuals + groups also have aspiration levels and their level of risk taking may depended on how close they are to their goal.

Rachlin proposed:

Preference for gambling is based on possibility of obtaining an immediate reward.
Buying a lottery ticket is an attractive option for some people for the same reason that VR or VI schedules produce steady and persistent responding in the laboratory: In both cases, there is a chance that a reinforcer will be delivered almost immediately.

When a person makes an impulsive choice in a self-control situation, the person is acting against his or her long-term interests.

True

The Tragedy of the Commons - Garrett Hardin

Commons was a grassland owned by the village where cows grazed freely.
Outcome may be more animals than people could support. Overgrazing then causes scarce grass, erosion, and destroyed commons.

*Hardin concluded- herder will experience a net gain by adding the additional cow to the herd, and by adding a second cow, and so on.
* Each man is locked into a system that compels him to increase heard without limit, in a world that is limited.

“Freedom in a commons brings ruin to all”.

Hardin + Platt: suggested several ways real world tragedy can be averted:

A society can decide to make it difficult or impossible for individuals to act selfishly (pass laws that make it illegal to litter, pollute air, or kill endangered species).
Attach a punisher to the small, immediate alternative or attach an additional reinforcer to the large, delayed alternative (make expensive to park on city streets, or use parking garage.
Not underestimate the capacity of human beings to attend to and be influenced by the long-term consequences of their behaviors for society.
Assert behaviors that benefit others so that they can be inherently reinforcing for many people.

Brainscape's Knowledge GenomeTM

Chap 13 Flashcards

Brainscape's Knowledge Genome^TM