exam 2

Question 1

Operant Behavior

Answer 1

• Pavlovian responses are passive; they happen to you (reflexes, autonomic reactions).
• Operant behavior is active; it consists of the things you do.
Pavlovian responses are elicited(by stimuli) Operant behavior is emitted(by an organism)
• B. F. Skinner coined the term “operant behavior” to characterized the behavior studied by his influential predecessor, Edward Thorndike.
• Many of Skinner’s early ideas were derived from Thorndike’s work, but Skinner developed his own terminology and philosophical perspective (radical behaviorism).

Question 2

Thorndike: The Law of Effect

Answer 2

• Thorndike’s research emphasized the effects on learning of the consequences that follow behavior.
• Law of Effect: Behavior followed by favorable outcomes will increase in frequency; behavior followed by undesirable outcomes will decrease.
• Therefore, he was the first scientist to systematically study operant behavior, although he called the changes that occurred trial-and error learning.

Question 3

Skinner’s operant learning is not S-R (reflex) learning

Answer 3

• Early behaviorists like John Watson used the reflex terminology of stimulus-response (S-R) psychology to account for all behaviors.
• From this perspective, stimuli force responses much like meat in a dog’s mouth forces salivation.
• Using the reflexive, S-R approach leads to awkward, mechanistic explanations of voluntary, goal-oriented behaviors.
• Example on board of Clark Hull’s S-R chaining
– Every point in a maze is a stimulus that drives an organism reflexively to the reinforcement at the end of a maze.

Question 4

Molecular vs Molar Behaviorism

Answer 4

Watson and Hull’s early Molecular approach:
All behavior, even operant behavior,is made up of S-R chains.Complex behavior = more complex chains.
Tolman & Skinner’s molar approach:
Behavior is made up of learned,whole, integrated acts that can be extended over time in complex ways. No need to describe or account for every twitch: just describe the final behavioral outcome—the rat runs to the end of the maze more rapidly than before.

Question 5

S-R vs R-S learning

Answer 5

• Skinner agreed that classical (Pavlovian) conditioning could be characterized as S-R learning.
• But not all learning was S-R learning.
• Operant conditioning might be better described as R-S learning
• Complex behaviors are emitted first (R)
• This response leads to an environmental outcome (S), which then determines whether R is likely to be repeated.

Question 6

The Selection of Operant Behavior

Answer 6

• Skinner preferred to describe operant conditioning in terms borrowed from Darwin’s natural selection.
• The principle of selection of behavior states that an individual emits behavior that produces effects, consequences, or outcomes.
• Based on these consequences,some behaviors increase in an organism’s behavioral repertoire while other behaviors decline or become extinct.
• (Analogous to physical traits increasing or decreasing in a population due to natural selection)
• Try something (behave); keep what works (i.e. what brings reinforcement)

Question 7

Methodological Advances

Answer 7

• Discrete Trial Procedures: Behavior of an organism ends a trial (e.g., escape from box, or finding the end of a maze ends a trial; start again)
– Requires researchers to be present during task.
• Skinner introduced “Free Operant Procedures” : Ongoing operant behavior can be “freely” repeated any number of times; rates of responding automatically recorded by a cumulative recorder.
– Researchers can do other things while behavior is recorded automatically.

Question 8

Operant Conditioning

3

Answer 8

• Any procedure through which an operant behavior becomes stronger or weaker depending on the consequences of that behavior.
–Strength of behavior refers to its frequency, rate, probability of occurrence, intensity, or persistence.

Question 9

Contingency

Answer 9

• In operant conditioning, we say that the strength of an operant behavior is contingent upon its consequences.
–Recall that contingency implies an if-then relationship.
• In operant conditioning, if I do this, then a certain consequence follows.
• If behavior, then consequence.

Question 10

Operant Conditioning Procedures

Answer 10

• Reinforcement: providing consequences that increase or maintain the strength of an operant behavior.
• Punishment: providing consequences that reduce the strength of an operant behavior.

Question 11

Environmental Consequences

Answer 11

• Consequences of a behavior can be described in terms of whether an environmental stimulus is presented or removed as a result of a behavior.
–Sometimes our actions result in something becoming present or available to us.
–Think (+) (addition)
–Sometimes our action result in something being removed or taken from us.
–Think (−) (subtraction)

Question 12

Two types of Reinforcement

Answer 12

Positive Reinforcement: A response is strengthened by adding a stimulus or increasing its intensity
(e.g., + food, praise)
The stimulus is called a positive reinforcer.
Negative Reinforcement: A response is strengthened by subtracting a stimulus or decreasing its intensity
(e.g.,—shock, nagging)
The stimulus is called a negative reinforcer.

Question 13

Positive reinforcers as rewards

Answer 13

• Positive reinforcers are often things we seek out, things that bring us pleasure or satisfaction, things that might be called rewards for our behavior.
• Sometimes positive reinforcement is called reward learning.
• Skinner objected: “People are rewarded, but behavior is reinforced.”
• What is positively reinforcing must be determined empirically: does its presentation increase the strength of behavior?

Question 14

Negative reinforcement and escape

Answer 14

• Negative reinforcers are often things we find aversive, things we would normally avoid.
• Sometimes negative reinforcement is called escape-avoidance learning.
• We will do something more often (strengthen a behavior) to avoid or escape something we don’t like.
• As with positive reinforcement, the stimuli that act as negative reinforcers must be determined empirically for a particular species or organism.

Question 15

Punishment

Answer 15

Any stimulus or outcome that decreases the strength of an operant behavior.

–Positive Punishment: Decreases the likelihood of a behavior by presenting (adding) a stimulus (usually aversive).

–Negative Punishment: Decreases the likelihood of a behavior by removing (subtracting) a stimulus (usually a desirable one).

Question 16

Kinds of Reinforcers: Primary & Secondary

Answer 16

• Primary reinforcer: any reinforcer that is not dependent on another reinforcer for its reinforcing properties.
–Often biologically based; innately or naturally reinforcing for members of a species.
–Food, water, sexual stimulation, or relief from pain, cold, or other aversive conditions.
• Secondary Reinforcer: Any reinforcer that has acquired its reinforcing properties through its association with other reinforcers.
–Also called “conditioned reinforcers”
• e.g., pair a buzzer with food (classical conditioning)
• Then the buzzer can be used to reinforce operant behavior (such as pressing a bar).

Question 17

Generalized Reinforcers

Answer 17

• Any secondary reinforcer that has been paired with a variety of other reinforcers.
• Money: Paired with almost every imaginable reinforcer in consumer societies.
• In prisons, cigarettes are generalized reinforcers.
• In some institutions, tokens are generalized reinforcers (token economies).

Question 18

Kinds of Reinforcers

Answer 18

–Contrived (“extrinsic”) reinforcers: Any reinforcing event that has explicitly been designed to modify behavior.
• Almost all reinforcements in research and applied behavior analysis.
–Natural (“intrinscic”) reinforcers: Any reinforcing event that follows automatically, i.e., naturally from a behavior.
• Pleasure from succeeding on a task.
• Scratching an itch.
• Reading for intrinsic pleasure (natural) vs. reading to earn a star in class (contrived).

Question 19

Social Reinforcers

Answer 19

• Social reinforcers are consequences that involve personal communication from others
– Praise, touch, compliments
– Many mammals respond to these social reinforcers( Limbic System): “good dog” in a happy, warm tone. ‘
• Social reinforcers are easy to deliver, can be applied immediately, and are not very disruptive

Question 20

Shaping & Chaining

Answer 20

• By definition, reinforcement must come after a behavior is emitted (a reinforcer is a consequence of the behavior).

Question 21

Shaping

Answer 21

• Shaping involves reinforcing successive approximations to the desired (target) behavior.
– Hot, warmer, cold

Question 22

Chaining

Answer 22

• A behavior chain is a connected (and usually ordered) sequence of distinct behaviors
• e.g., Cooking
• Once a chain has been learned, it can be considered one extended act.

Question 23

Chaining Methods

Answer 23

– 1st Step: Task Analysis—identify each link in the chain and their order

– Forward chaining begins with the first link in a task; when a link is mastered, the next link is added as a new requirement for reinforcement. And so on.
• If any link does not readily occur, use shaping.

– Backward chaining begins with the last link and works backward toward the first link.

Question 24

Prompting

Answer 24

Prompts are any stimuli that help an organism to initiate the desired behavior.

Some prompts are simply verbal or visual cues or reminders (sticky notes, or saying “Wash your hands.”)

Modeling (showing how) is another type of prompting.

Physical (or manual) prompting means physically guiding the organism to engage in the correct behavior.

e.g., If a dog simply will not spontaneously sit, push its rear end down, then reinforce.

Question 25

Variables Affecting Reinforcement

Answer 25

• Contingency
• Contiguity
• Characteristics of Reinforcers
• Behavior Characteristics
• Motivating Operations

• Other odds and ends

Question 26

Contingency

Answer 26

• Remember? If A, then B

• In operant conditioning, contingency refers to the degree of correlation between a behavior and its consequences.
• The rate of learning depends on the degree to which a behavior is followed by a reinforcer.

More contingency, more learning

• Smaller reinforcers with high contingency are generally better at producing learning than larger reinforcers with low contingency.
• Organisms will often work for a smaller “sure thing” than for a larger “long shot.”

Question 27

Contiguity

Answer 27

• In operant conditioning, contiguity refers to the time between a behavior and its consequences.
• In general, the faster a reinforcer is delivered after a behavior, the stronger the learning.

Shorter delay, faster learning

• If a delay is too long, other behaviors might occur before reinforcement.

Question 28

Characteristics of Reinforcers: Size

Answer 28

• All else being equal, larger reinforcers (e.g., more food) strengthen behavior more than smaller reinforcers.
• $1 vs. $100 after each A on report card.
• However, there are diminishing returns: as the size of the reinforcer increases, the rate of learning (while increasing) becomes less pronounced.
• i.e., a nonlinear relationship between size and learning.

Question 29

Characteristics of Reinforcers: Quality

Answer 29

• Species, and individuals within a species, often have different “tastes.”
• e.g. Food is not interchangeable; some reinforce better than others.
• Rats prefer bread or milk over sunflower seeds: bread and milk strengthen behavior more than seeds.
• Reptiles do not respond to social reinforcers.

Question 30

Behavior Characteristics

Answer 30

• Behavior that is a part of a normal repertoire for a species also affects reinforcement:
–Birds that peck for seeds in the wild (e.g., pigeons) learn to peck a disk for reinforcers much more readily than do birds of prey (e.g., hawks).

Question 31

Motivating Operations

Answer 31

• Also called Establishing Operations
• MOs are anything that establishes conditions that improve the effectiveness of areinforcer.
–e.g. Thorndike’s work: animals were deprived of food to motivate them to find a way out of the puzzle box.
–Food deprivation established food as an effective reinforcer.
• Water (make thirsty)
• Social contact (keep isolated from others)
• Physical stimulation (sensory deprivation/boredom)
• Rest (make them tired)
• Warmth (keep in cold room)

Question 32

Alternative reinforcers

Answer 32

• Reinforcement is weaker if there are alternative reinforcers easily available in the environment.
–A rat might not work hard for a particular type of food if there are other types lying around “for free.”

Question 33

Extinction in Operant Conditioning

Answer 33

• In classical conditioning, if the CS (bell) is continually presented without the US (food), you get extinction: the CS will no longer elicit salivation.
• In operant conditioning, you find a similar process:
• If you withhold the reinforcing consequences of a behavior that has been reinforced in the past, the behavior will be weakened.

Question 34

Another extinction phenomenon…

Answer 34

• When extinction begins, organism will begin to vary the previous behavior
–“try something different.”
• This can be useful in shaping: if the behavior gets “stuck” at one point, try withholding reinforcement.
• The organism will often begin variations in their behavior, perhaps offering a better approximation to the final target behavior.

Question 35

Extinction and Emotion

Answer 35

• During the initial period of extinction, an organism will frequently increase its emotional behavior,especially aggression.
• Example: Rats who have been frequently reinforced with food for pressing a lever.
• Stop delivery of the food: Rats will become aggressive, biting at the lever, attacking other nearby rats.

Question 36

Extinction-Induced Aggression

Answer 36

• Most organisms will attack when they are shocked, prodded, or subjected to other aversive stimuli.
• Apparently, removal of positive reinforcement is an aversive event.
– Extinction periods from previous reinforcement can been used as punishments to weaken other behaviors.

Question 37

Spontaneous Recovery

Answer 37

• During extinction, it is typical for a behavior to continue decreasing until it reaches a pre-reinforced level or ultimately ceases.
• However, a phenomenon commonly associated with extinction is the reappearance of the behavior, after a delay,even though it has not be reinforced.
• This is called“spontaneous recovery,” just as in classical conditioning.
• Spontaneous recovery is short-lived and limited if the extinction procedure remains in effect.

Question 38

Resurgence

Answer 38

• Condition Behavior A through reinforcement, then extinguish the behavior by withholding the reinforcement.
• Now condition Behavior B using the same reinforcer, then start to extinguish it.
• What happens?
• Subject engages in Behavior A again.

Question 39

Resistance to Extinction

Answer 39

• The rate at which a behavior is extinguished by non-reinforcement depends on many factors.
• Continued responding during extinction is termed resistance to extinction.
• As a rule of thumb, any factor that strengthens reinforcement might also produce resistance to extinction.
• The number of times a behavior has been reinforced in the past affects resistance to extinction.
• A behavior with a long history of reinforcement will be more resistant to extinction than a behavior with fewer reinforcements.
– Number of bar presses during extinction varies by number of previous reinforcements.

Question 40

Effort and Resistance to Extinction

Answer 40

• If a response requires greater effort, it may decrease more quickly during extinction than a response requiring less effort.

Question 41

Guidelines for the Use of Extinction for Eliminating Unwanted Behavior

Answer 41

• Identify & withhold all sources of reinforcement for the unwanted target behavior.
• Withhold Consistently
• Consistency is important to all behavior change procedures; it is absolutely essential to extinction.
• Use Instructions (with children)

Question 42

Guidelines for Extinction

• Expect Common Extinction Phenomena:

Answer 42

–The behavior might get worse, not better, at first.
• Extinction bursts will occur
–The behavior might come back briefly.
• Spontaneous recovery will occur
–Aggressive and immature behavior might appear
• Resurgence and extinction-induced aggression
• These are indicators that extinction is working
–so don’t get dismayed, and stay with the plan.

Question 43

Olds (1958)

Answer 43

• Using the electrodes again, Olds rigged a Skinner box so that depressing a lever automatically produced the mild stimulation of the brain.
• The rats were soon pressing the lever at a very fast rate, as many as 700 times in an hour.
• When allowed, rats would repeatedly depress the lever for over two days, until they collapsed from exhaustion.
• They would seldom eat, preferring the stimulation to food or water.

Question 44

Getting Info from one neuron to another: Neurotransmitters

Answer 44

Neurotransmitters (chemicals) released from the sending neuron, travel across the synapse and bind to receptor sites on the receiving neuron, thereby influencing it to generate an action potential.

Question 45

Dopamine

Answer 45

• Dopamine is released by almost everything that we experience as pleasurable— –Eating
–Winning a fight
–Sex
–Most addictive drugs, but cocaine produces especially high dopamine release.
–Dopamine is particularly high when a “reward” is unexpected.
–As rewards become more predictable, less dopamine is released (diminishing returns in reinforcement?)

Question 46

Effects of blocking dopamine

Answer 46

• Wise et. al (1978): rats trained to routinely press a lever for food.
• After this, rats divided into 3 groups:
–Control + Continued Reinforcement
–Control + No Reinforcement
–Primozide+ Reinforcement

–Primozide is an effective dopamine blocker: blocks dopamine receptor sites.

Question 47

Theories of Reinforcement

Answer 47

How does a reinforcer produce an increase in the probability of the reinforced response?

Two Approaches
• 1. Reinforcers are certain kinds of stimuli
–Drive Reduction Theory
• 2. Reinforcers are certain kinds of behaviors (responses)
–Relative Value Theory and the Premack Principle
–Behavioral Bliss Points & the Relative Deprivation Hypothesis

Question 48

Drive Reduction Theory

Answer 48

• Clark Hull
– Organisms attempt to maintain a state of physiological equilibrium, or homeostasis
– Deviations from homeostasis produce biological drives that are aimed at bringing about homeostasis
• examples: hunger,thirst
– Reinforcers are stimuli that reduce biological drives
• examples: food to a hungry organism, water to a thirsty one
– Accounts for effectiveness of motivating operations (e.g. deprivation) and primary reinforcers(such as food) in learning.
– Also accounts for satiation: a well-fed rat will not be reinforced by food.
– Why? Its drive is already reduced

Question 49

Reinforcers as Responses

Answer 49

• Some theories conceptualize a reinforcer as a response (not a type of stimulus).
– Food is not a reinforcer, the act of eating is.
• David Premack’s Relative Value Theory
• What makes eating a reinforcing response?
– Not a physiological drive state.
– A particular operant behavior (like eating) is a reinforcer if it has a high baseline probability of occurrence.
– We might say that high baseline behaviors are preferred behaviors, or have a high relative value
• Behaviors can be ranked by preferences: observe the whole repertoire of behaviors. Rank by probability of occurring (preference).

Question 50

Premack Principle

Answer 50

• Organisms will work at a less preferred behavior for the opportunity to engage in a more preferred behavior.
– aka Grandma’s Law:
– Eat your spinach first (less preferred), then you can go out and play.

Question 51

Behavioral Bliss Point

Answer 51

• The preferred frequency of a behavior (its relative value in Premack’s terms) is called the behavioral bliss point for that behavior
• With no restrictions, how much time would you spend:
– Working
– Studying
– Social networking
– Watching TV
– Housework
– The time you would spend on each activity is your bliss point for each activity.

Question 52

The Response Deprivation Hypothesis

Answer 52

• The response deprivation hypothesis is that any behavior can be used as a reinforcer if the behavior is first constrained to be below your bliss point.
• Even initially very low probability behavior can be turned into a reinforcer if the behavior is constrained to be below your bliss point.
• How often would you like to scratch your nose?
• Scratching your nose is a low frequency behavior.
• But if your hands are tied, you are still below your (very low) bliss point.
• You might work really hard to be freed to scratch your nose.
• (working hard for “forbidden fruit”)

Question 53

Schedules of Reinforcement

Answer 53

• A schedule of reinforcement is a set of rules for delivery of reinforcement
–Different schedules lead to different rates and patterns of behavior, which are called schedule effects.
–One distinction we can make is between continuous and intermittent (or partial) reinforcement schedules.

Question 54

continuous reinforcement

Answer 54

(CR) schedule provides reinforcement every time a response occurs.
–CR produces very rapid learning, but it also yields the fastest extinction after the reinforcement is terminated.
–(Extinction can also be considered a schedule of reinforcement in which no behavior is reinforced.)

Question 55

Schedules of Reinforcement

Answer 55

• Most schedules of reinforcement are intermittent(partial) rather than continuous. In other words, some responses are reinforced and others are not.
• This is more like the real world (you win some, you lose some).
• We can distinguish between two types of intermittent schedules:

– Ratio schedules: the delivery of reinforcement depends on the number of responses given by the individual.
• Fixed vs. Variable
– Interval schedules: delivery of reinforcement depends on the amount of time that has passed since the last reinforcement.
• Fixed vs. Variable

Question 56

fixed-ratio schedule

Answer 56

provides reinforcement only after a definite (“fixed”) number of correct responses have been made.
– e.g., a laboratory rat being reinforced for hitting a lever after every 5 hits is being reinforced on an FR-5 schedule.
–The local coffee shop gives you a card that says if you buy 9 coffee drinks you will get the 10th beverage for free:
– FR-9

Question 57

Fixed Ratio

Answer 57

• In FR schedules, there is high performance, but there is often a temporary pause in behavior immediately following reinforcement
. • This is called a post-reinforcement pause.
– The higher the order of the ratio (more work required before reinforcement), the longer the pause.
– The pause will be longer for an FR-100 schedule than for an FR-20 schedule.

Question 58

Why the pause? (fixed ratio)

Answer 58

• Some have argued that fatigue causes the pause—after all that work, one gets tired.
• But some schedules produce even greater effort, yet there is no pause.
–The going explanation is that hard work is aversive, and we escape by avoiding the behavior for a while.
–Similar to procrastination—we put off studying not because we are tired, but because studying can be unpleasant.

Question 59

After the pause… (fixed ratio

Answer 59

• After the post-reinforcement pause, regardless of its duration, the organism returns to a steady rate of responding.
–Called a “break-and-run” pattern –Take a break, but take off running hard afterward
• So an FR-100 schedule produces a longer pause than FR-20, but the rate of responding, once begun, is the same in both schedules.

Question 60

variable-ratio schedule

Answer 60

provides reinforcement after a variable number of correct responses, working out to an average number in the long run.
• For example, in a VR-5 schedule, reinforcement sometimes follows a single response, sometimes after 8 responses, etc., but the average is about 5 responses.
• Fishing, golfing, “spare change?”
• Slot machines, like all gambling, provide a particularly compelling form of variable ratio reinforcement to the player

Question 61

Variable Ratio

Answer 61

• Produces very rapid learning, as fast (or faster) than fixed ratio schedules.
• However, the post-reinforcement pause is eliminated (or greatly reduced in duration).

Question 62

fixed-interval schedule

Answer 62

provides reinforcement for the first response made after a specific time interval (since the last reinforcement).
– A person who is paid every two weeks is reinforced for work on a fixed interval schedule (FI-2 weeks).
• i.e., if you are still working two weeks after your last paycheck, you will get another pay check
– If you receive your local newspaper at the same time every day, you are reinforced for looking on a fixed interval schedule (FI-24 hr)
• i.e., you look 24 hrs since you last got your paper.

Question 63

Fixed Interval

Answer 63

–Immediately after reinforcement, there is a reduction in the rate of responding.
–But as the time interval for reinforcement approaches, there is very rapid responding until the reinforcement is delivered.
–Called a “scallop-shaped” pattern
–No matter how long the time interval, the organism will begin increased responding as the payoff time approaches.
– Checking the oven when baking (if the recipe says bake 10 minutes, you will not check after 2-3 minutes, but you will start checking frequently around 8-9 minutes)
– More studying as scheduled exams approach (to make sure you are reinforced with a good grade).

Question 64

variable-interval schedule

Answer 64

provides reinforcement after a variable amount of time has elapsed since last reinforcement (but with an average time).
• If your newspaper delivery person is very inconsistent about delivery times, showing up one day at 5:00AM, the next day at 7:30AM, etc., but on average at 6:15AM, your paper is delivered on a variable interval schedule.

Question 65

Variable Interval

Answer 65

• Rates comparable to (or sometimes higher than) fixed interval, but no scalloped effects.
• i.e., a smooth pattern of responding

Question 66

Fixed Time Schedule

Answer 66

– Fixed time (FT) schedules: A reinforcer is delivered after a specific time regardless of behavior.
•In an FT-10s schedule, a rat gets food every 10 seconds, regardless of what it does.
– Can produce superstitious behavior: a behavior that happened to occur at the time of delivery might increase in frequency (although the reinforcement actually does not depend on the behavior).
» If a coach wears a hounds-tooth hat to a game and wins, he might begin wearing the “lucky” hat in the future.
» Occasional reinforcement (wins) might strengthen the behavior.
– Unemployment payments are similar to FT schedules if the check is provided every month regardless of the behavior of the recipient.

Question 67

Stretching the Ratio

Answer 67

• Organisms can be trained to work very hard for very little payoff
– Rats can be reinforced to press a bar hundreds of times for a single reinforcer.
• This is accomplished by a form of shaping called “stretching the ratio.”
•Start with high reinforcement rate; slowly decrease (or “stretch”) the reinforcement rate.
•e.g., Begin with FR-3 schedule, then increase to FR-5, then FR-10, FR-20, FR-30, and so on, perhaps up to FR300.

Question 68

Stretching the Ratio 2

Answer 68

• If the schedule is stretched too quickly or too far, signs of extinction may be observed
– Might see an extinction burst, followed by lower rate of responding.
– Might see aggression
– May see an increase in other responses
• This is known as ratio strain.
• If this happens, retreat to a denser (higher rate) reinforcement schedule.

Question 69

Partial Reinforcement Effect

Answer 69

• PRE: Behaviors on intermittent (partial) reinforcement schedules are more resistant to extinction than behaviors on continuous reinforcement.
• PRE is counter intuitive: we hold on to learning better when reinforced less.

Question 70

Concurrent Schedules

Answer 70

• In real life, operant conditioning is rarely a matter of being offered only one source of reinforcement.
• Instead, individuals typically must choose between alternative sources of reinforcement, and the reinforcements offered might require different types of behavior.
– Studyinggood grades
– Internetentertainment, social reinforcement
– Workpay, skill development
– Choices: What will you do most often?

Question 71

Choice in the lab: Concurrent Schedules

Answer 71

• Two or more basic schedules are operating independently at the same time for two or more different behaviors
• The organism has a choice of behaviors and schedules.

Question 72

Concurrent Schedules 2

Answer 72

•All else being equal, when different reinforcement schedules are provided for each of the concurrent responses:
– The response receiving the higher frequency of reinforcement will increase in rate
– The response requiring the least effort will increase in rate •Pressing a lever vs climbing a wall to press a lever
– The response providing the most immediate reinforcement(contiguity) will increase in rate

Question 73

The Matching Law

Answer 73

• Behavior on concurrent schedules follows Robert Herrnstein’s Matching Law.

• Herrnstein has an equation to represent the matching law. For 2 behaviors reinforced on two schedules, the equation is
B1/(B1+B2) = R1/(R1+R2)

– The proportion of behavior devoted to a choice is the same as the proportion of reward offered.

Question 74

Conger and Killeen (1974) Study of Matching in Humans

Answer 74

• They asked volunteers to participate with two others (confederates) in a discussion session on drug abuse.
• While the volunteer was talking, the two confederates sat on either side and intermittently expressed approval on different schedules.
• They found that the relative amount of time the volunteer looked at each confederate matched the relative frequency of verbal approval delivered by that confederate.

Question 75

ALWAYS REMEMBER

Answer 75

Punishment always weakens a behavior; lowers the likelihood of response.

Positive punishment by adding a (usually aversive) stimulus.

Negative punishment by removing a (desired) stimulus.

Question 76

Note on Negative Punishment

Answer 76

• Two types of negative punishment:
• Response cost: take away a specific reinforcer
– Money
– TV viewing
– Computer time
• Time Out: Remove all reinforcers For a while
– Keep it brief (1 minute per year of age)
– Make sure the time out location is not more reinforcing than the alternatives (“Go to your room” might be better than sitting at the dinner table.)
• With all punishment in humans, give an explanation of what is being punished.
– Sometimes kids don’t know exactly what they did to deserve punishment

Question 77

Measuring the Effects of Punishment

Answer 77

• A widely used measure of the effects of punishment on a behavior is the called the suppression ratio.
• Suppression Ratio: a percentage of the responses that would be expected without the punishment.
– A suppression ratio of 90% would indicate a small drop in the rate of behavior (due to the punishment).
• i.e., the behavior is still occurring 90% of the time that it would occur without punishment.
– A suppression ratio of 30% would indicate a large drop in the rate of behavior

Question 78

Does punishment work?

Answer 78

• Both E. L. Thorndike (Law of Effect) and B.F. Skinner said “no.”
• Both Skinner and Thorndike were influential researchers, so their dismissal of punishment as a way to control behavior was taken seriously for many years.
• Parents and teachers were discouraged from using punishment.
– Not because it was cruel, but because it was thought to be ineffective.

Question 79

Skinner’s early work on punishment

Answer 79

Phase 1:
Rats were reinforced with food for bar pressing on a VI schedule until a high rate of pressing was obtained.
Phase 2:
On two successive days, rats were placed on extinction (no reinforcement regardless of behavior).
During the first 10 min of extinction on day 1, one group of rats was punished for each bar press (paw slapped); the other,control group,was not punished.
Rats then simply observed for rest of day 1 and day 2

Question 80

Results of Skinner’s Experiment on Punishment

Answer 80

Punishment suppressed responding while it was being administered,but responding began to increase shortly afterward.
By the end of day 2, the punished rats ended up making as many responses overall during extinction as the unpunished controls.

Question 81

Thorndike and Skinner were wrong about the effects of punishment

Answer 81

• Punishment has subsequently been shown to effectively suppress behavior, but many variables affect the strength of punishment.
• Contingency
• Contiguity
• Intensity
• Introductory Level of Punishment
• Nature of Previous Reinforcement
• Deprivation Level

Question 82

Contingency (punishment)

Answer 82

• The effect of punishment in weakening a behavior depends on contingency between the behavior and the punishment. • If a rat is punished (e.g., shocked) every time it presses a lever, but not otherwise, then pressing will be less likely to occur.

Question 83

Contingency (punishment) Boe and Church

Answer 83

• Rats trained to press a lever for food, then put on extinction for 20 min.
– During first 15 min.,
• Some rats (punishment group) were shocked only when they pressed the lever.
• Some rats were shocked at random, regardless of what they did (non-contingent shock group).
• Some rats (controls) were never shocked.
• Percentage of time pressing the lever was recorded for all three groups.

Question 84

Contingency (punishment) Boe and Church (results)

Answer 84

• Boe & Church (1967)
– The rats who were given non-contingent shocks did not differ from the control rats who were under extinction.
– The punished rats had dramatically lower rates of responding than both of the other groups.
• So, punishment is effective when it consistently follows the behavior.

Question 85

Contiguity (punishment)

Camp et al., 1967

Answer 85

• The time interval between a behavior and a punishing consequence has a strong effect on the rate of responding.
• Camp et al. (1967)
– Rats were periodically reinforced for pressing a lever, but were sometimes shocked for the same behavior.
• Some shocked immediately (0 second delay)
• Some shocked after 2 seconds delay
• Some after 30 seconds delay
• Suppression ratios calculated for the 3 conditions.

Question 86

Contiguity (Camp et al., 1967)

Answer 86

• Suppression Ratio measured for all three groups.
– For the immediate shock group, pressing dropped to less than 10% of expected responding.
– For the 2 sec delay condition, it was between 1020% of expected.
– For 30 sec delay, pressing was around 40% of expected.
• So, punishment is more effective when it immediately follows the behavior.

Question 87

Intensity of Punishment

Answer 87

• In general, the stronger the punishment (more aversive), the faster the behavior will decline.
• Very mild punishments have little effect on behavior.
– Thorndike’s “wrong.”
– Skinner’s “slap on the wrist”
• Much research shows that stronger punishments
(e.g., greater voltage of shocks) can greatly suppress behaviors.

Question 88

Introductory Level of Punishment 1

Answer 88

• How can you find the right intensity of punishment that will suppress a behavior?
• You could start with strong punishment and gradually weaken it, or
• You could start with mild punishment and gradually increase the intensity.
– This question has implications for strategies for real life punishment: cheating policies, tax evasion, etc.
• Should you start small and let possible consequences “sink in”?
• Or should you hit ’emhard first time so that don’t forget it?

Question 89

Introductory Level of Punishment 2

Answer 89

• Research shows:
– If you start with weak punishment, organisms tend to continue responding even as the intensity of punishment increases.
– Often punishments that normally would be successful if they had been applied from the start will not work if preceded by weak punishments.

Question 90

Introductory Level of Punishment 3

Answer 90

• Real life punishments often follow this weak-at first pattern.
• Traffic violations: I’ll just give you a warning this time. Next time a ticket. Next time suspended license.
• Same with children: stern look; scolding; swat on the backside; beat the tar out of them.

Question 91

Nature of Previous Reinforcement

Answer 91

• The frequency, amount, and quality of reinforcement history will affect punishment in obvious ways.
– Behaviors will persist if there was, or continues to be, a “good source” of reinforcement available for the behavior.
• Behavior will also persist if there is no other source of reinforcement.
– A rat will continue to press a lever, in spite of shocks, if food can’t be obtained any other way.

Question 92

Deprivation Level

Answer 92

• Organisms that are deprived of a primary reinforcer (food or water) will be less influenced by punishment than a satiated organism.
– Hungrier rats will press a lever longer in the face of shocks.
• Azrinet al. (1963)
—varied weight of rats with food deprivation (60-85% normal weight).
• Observed rate of lever pressing for food when pressing was also accompanied by shock.

Question 93

Theories of Punishment

just the names

Answer 93

• Disruption Theory
• Two-process Theory
• One-process Theory

Question 94

Disruption Theory

Answer 94

• Skinner and Guthrie (and others) thought that instinctive, often emotional, reactions to aversive events were simply incompatible with the reinforced behavior.
• A rat will jump, freeze, or even run when shocked.
• These behaviors interfere with steady lever pressing.
• This is sometimes called response competition: an instinctive response is competing with an operant response.
• Therefore lever pressing will temporarily decline during punishment.

Question 95

Problems with Disruption Theory

Answer 95

• Response competition alone is insufficient to make punishment effective.
• When punishment is contingent on behavior instead of just co-occurring at random, it is much more effective.
• Random shocks do not produce the same punishing effects of contingent shocks, even when the shocks occur with the same frequency.

Question 96

Two Process Theory

Answer 96

• Two-process theory states that punishment involves both classical (Pavlovian) conditioning and operant conditioning.
– When a rat is shocked for pressing a lever, the lever becomes a conditioned stimulus for pain and fear.
• Lever : Shock = fear/pain

– As a result, the lever becomes an aversive CS, and the rat will avoid it, thus decreasing the rate of lever pressing.
– Avoidance (of the feared lever) is then negatively reinforced.

Question 97

Two Process Theory 2

Answer 97

• This is a popular theory, but there are inconsistent research findings.
• In some studies, organisms no longer show fear of the lever, but continue to stop pressing anyway.
• Also: Although a rat might stop pressing a lever, it does remain exposed to it to some degree.
• Over time, with no shocks, fear of lever should extinguish.
• When fear is extinguished, the rat should start pressing again.
• But when shocked, the fear and avoidance return.
• Should see an oscillating pattern of responding over time.
• This is not observed.

Question 98

One Process Theory

Answer 98

– Punishment does not involve classical conditioning, only operant conditioning.
• This was Thorndike’s original view, before he decided that punishment actually had no effect on behavior.
– Originally, he argued that punishment was simply the mirror image of reinforcement.
– His theory was that reinforcement strengthens a neural connection between a behavior and an event.
– Punishment weakens a neural connection between a behavior and an event.
– This was called the “Negative Law of Effect”

Question 99

One Process Theory 2

Answer 99

• Problem with Thorndikes “neural weakening” theory:
– Skinner’s finding that suppression of behavior is sometimes only temporary contradicts this.

– Why would a weakened neural bond regain its strength over time under conditions of extinction?

Question 100

One Process Theory

Answer 100

• Supporters of one-process theory today disregard neurological speculation, adopting relative value theory.
• They claim that if punishment is just a mirror image of reinforcement, then the Premack Principle should apply to punishment as well as reinforcement.
• Original Premack: Eat your spinach, then you can eat your cake.

Question 101

Premack and Punishment

Answer 101

• If punishment is just another form of operant conditioning, then a less preferred behavior should be able to punish a more preferred behavior.
• Premack for Punishment: If you eat your cake, I’ll make you eat a plate of spinach.
• Much research confirms that the Premack Principle applies to punishment as well as to reinforcement.
• Mazur (1975):Hungry rats will reduce the rate pressing a lever for food (a preferred behavior) if the rat is also made to run in a wheel (less preferred) after eating.

Question 102

Problems with Punishment

Answer 102

• Some common side effects of punishment:
• Punishment is aversive and threatening, so it can lead to fight or flight responses.
• Flight: Escape/Avoidance
• People and other organisms will simply avoid the punisher or the punishing situation.
• Related: Conditioned fear of punisher
• Do you want kids to always be afraid of you?
• Fight: Aggression (as with extinction)
– Attacking, revenge, etc.
– After being shocked, rats will attack other rats and objects.

Question 103

More Problems with Punishment

Answer 103

• Apathy: Sometimes an organism will simply stop behaving (period) if punishment is severe enough.
– Seligman’s learned helplessness theory of depression.
• Imitation of Punisher (via observational learning):
– Punishment can become a tradition.
– Punitive managers have subordinates who become punishers.

Question 104

Benefits of Punishment

Answer 104

• If punishment is immediate, consistent, and strong enough, it actually works.
– This can be useful with seriously destructive behaviors, such as assault or self harm (head banging, scratching face)
– Downside: because it is effective, it might be overused.
• Severe spanking will stop misbehavior, so parents might use severe spanking for everything. (Abuse)

Question 105

Other benefits of punishment

Answer 105

• Sometimes leads to an increase in pro social behavior as a side effect (not just decrease in antisocial behavior)
–Being nicer, friendlier, apologizing, etc
–This might reflect innate appeasement behaviors to help repair and restore relationships

Question 106

Alternatives to Punishment

Answer 106

• Response Prevention
– For seriously harmful actions, physical restraint might be necessary
• Remove any eliciting stimuli associated with misbehavior
– Sometimes misbehavior occurs only in certain situations or in the presence of specific stimuli.
– How does your son behave around different friends?
– If he misbehaves only when he’s hanging out with Johnny, keep him away from Johnny.

Question 107

Differential Reinforcement of Alternative Behaviors (DRA).

Answer 107

– Combines extinction of unwanted behavior with reinforcement of other behaviors
– So, you don’t punish the unwanted behavior, you simply don’t reinforce it (e.g., ignore it).
– But you also reinforce more appropriate behaviors.
– DRI–differential reinforcement of incompatible behavior
• Reinforce alternative behaviors that are incompatible with the unwanted behavior.
• Rather than punish running around the room in class, strongly reinforce sitting still