Lecture 6 - Operant Conditioning

Question 1

Operant condition

Answer 1

(also known as instrumental conditioning and trial-and-error learning)

is associating a voluntary
behavior (‘operation on the environment’) with an outcome.

some action the animal chooses to do is associated with an outcome

Question 2

Law of Effect

Answer 2

Animals learn that a behavior (or class of similar behaviors)
predicts a particular outcome and seek the outcome by performing a particular behavior 

Behaviors with good outcomes increase; behaviors with bad outcomes decrease.

(Thorndike, 1911)

Question 3

Discrete trial paradigm

(Thorndike,
1911)

Answer 3

Cat opens the puzzle box and is reinforced with food reward.

Cat learned that flipped the switch was responsible for it getting out and getting food: So that escape behavior becomes more likely (and faster) in the future.

discrete because every-time the cat got out that was one trial and for each new trial the cat had to be put back in

Question 4

B. F. Skinner

free-operant paradigm

Answer 4

refined Thorndike’s method to allow the animal to respond repeatedly

==> allowed the animal to control the rate of responding ==> animal controls when they get the reward (food)

• SKinner Box

Question 5

Skinner Box

Answer 5

little contraption, everything was automated: counted the number times the lever was pressed, counted the number of times the reward was provided

made it easy to measure this activity over time

instead of recording trials you’re recording behaviors over time

• Behaviors could be automatically recorded
in a Skinner box – count number of behaviors and outcomes.

Question 6

Acquisition

Answer 6

reinforcing behavior: giving reward for every time the rat presses the lever

the amount of responses goes up

Question 7

extinction

Answer 7

it keeps pressing the lever but no food comes

if you stop reinforcing the behavior then the behavior starts to go away

amount of responses decreases

Question 8

Basic elements of the free-operant paradigm:

Answer 8

• discriminative stimulus (S)
• behavioral response (R)
• outcome (O)

S –> R –> O

Question 9

Through repeated trials, the animal learns that the outcome is contingent upon

Answer 9

the appropriate response.

Question 10

discriminative stimulus (S)

Answer 10

that helps you select
the appropriate behavior (e.g. rat can see the lever).

the animal has to be able to ID something in the environment that it’s operating on

Question 11

behavioral response (R

Answer 11

or class of similar responses,

is performed in response to the stimulus (e.g. rat pushes lever with either paw).

Question 12

outcome (O)

Answer 12

follows that either reinforces or punishes the behavior (e.g. rat gets food, good outcome).

Question 13

reinforcers

Answer 13

Outcomes that increase the likelihood of the behavior

primary reinforcers

secondary reinforcers

Question 14

primary reinforcers

Answer 14

meet some innate need (e.g. food, water, sleep, and sex).

Note that these are not always reinforcing (i.e.
you won’t work for water if already satiated).

Question 15

Secondary reinforcers

Answer 15

have no intrinsic value, but predict or are associated with primary reinforcers (e.g. money, good grades, gold stars, etc.).

something by itself has no value but through some kind of association it’s learned that this other thing is valuable

Question 16

punishers

Answer 16

Outcomes that decrease the behavior

primary punisher

secondary punisher

Question 17

Primary punisher

Answer 17

Pain (shock), nausea, loud noises, social disapproval (?), loss of freedom (jail).

basically just aversive things

Question 18

Secondary punisher

Answer 18

Monetary fines, demerits, bad grades, etc.

Question 19

You are about to press a button on your iClicker. When
you see that you got the correct answer to the question,
that acts as a ______________.

Answer 19

Secondary reinforcer

Question 20

positive (+) conditioning

Answer 20

If an outcome/consequence is added, if you’re given an outcome as a result of your behavior

this has nothing to do with “good” or “bad.”

Question 21

negative (-) conditioning.

Answer 21

If an outcome/consequence is removed, something is taken away

this has nothing to do with “good” or “bad.”

Question 22

Positive reinforcement

Answer 22

when you want to increase the behavior (reinforce) and you do it positively

animal rewarded for doing a behavior –> given something to make the behavior more likely

response increases (reinforcement)+ consequence is added (positive)

Question 23

Negative Reinforcement (escape/avoidance)

Answer 23

response increases (reinforcement) + consequence is removed (negative)

want the behavior to increase but take something away ==> if you do something I want, I’ll take away a “bad thing”

Question 24

Positive punishment

Answer 24

Response decreases (punishment) + Consequence is added (positive)

when you don’t want a behavior and you add something (electric shock)

Question 25

Negative punishment (omission)

Answer 25

Response decreases (punishment) + Consequence is removed (negative)

I don’t want you to do a behavior so I take something away (money, privileges, etc…)

“No more T.V. for you!”

Question 26

Positive reinforcement example

Answer 26

Eat all your vegetables –> get some dessert.

“do something I want you to do and I give you something”

Question 27

Positive punishment example

Answer 27

Scratch the couch ==> get sprayed with water;

tease your sibling ==> parental scolding.

Question 28

Negative Reinforcement example

Answer 28

Shut off the alarm clock (aversive stimulus) ==> removal of an aversive stimulus;

- arm does flailing motion
- next morning you're more likely to make that same movement
- reinforcement of behavior that takes away an aversive stimulus 

take ibuprofen ==> reduce a headache.

- next time you have a headache you're more likely to grab that medicine again
- reinforces that behavior of taking the medicine
- you're not getting anything, something is being taken away (an aversive stimulus - a headache)

Question 29

Negative punishment example

Answer 29

Commit armed robbery ==> loss of freedom (jail).

Question 30

timing and context in operant conditioning

Answer 30

are critical for forming the association.

critical for how effective it’s going to be

Question 31

If the outcome is delayed….

Answer 31

… the association is not learned as well.

So, punishing your dog for something it did an hour ago is probably not very effective…

Question 32

any kind of reinforcement to be effective needs to come

Answer 32

almost immediately

Question 33

Reinforcement schedules

Answer 33

(i.e. how often you get the outcome)

how providing an outcome, on what timing, how frequently, how reliably, how that can affect the rate at which the associations are learned.

how often and how reliably you get the outcome: going to affect the rate of learning and the effectiveness over time

Question 34

continuous reinforcement

schedule

Answer 34

When you get a reward after every behavior:

every time the rat presses the lever it gets a reward: no break in the reward: everytime you perform the action you get the outcome

Question 35

partial reinforcement

schedule

Answer 35

anything that isn’t a continuous reinforcement schedule

the outcome follows
less than 100% of the time

Question 36

variable-ratio schedule

Answer 36

A powerful form of partial reinforcement schedule

steep learning curve: if you don’t know when it’s coming you just keep banging away at the lever

you don’t get the outcome every time, but you get it about every 5 or 10 times - but you can’t predict it (unknown) – the exact timing can’t be predicted.

gambling!!! - the payout is variable

most effective and has the highest curve of learning

Question 37

fixed ratio

Answer 37

every fifth time you perform the action you get the reward

rats: 5 responses and a pause (a plateau)

Question 38

Sheldon gave Penny chocolate each time she did

something to please him. What kind of paradigm is this?

Answer 38

Positive reinforcement

Question 39

Sheldon sprayed water on Leonard when he disagreed.

What kind of paradigm is this?

Answer 39

Positive punishment

Sheldon wants Leonard to do it less (punisher) and he is adding something (positive)

something is being added to the situation and he wants him to not perform that behavior again

Question 40

Reinforcers and punishers can be

Answer 40

equally effective at
producing behavior in laboratory conditions (controlled conditions); however, punishers can experience problems in the real world.

Question 41

Problems with punishers?

in the real world when you can’t control those discriminative stimuli or timing as easily

Answer 41

1. If you punish a behavior, you may encourage
   cheating/circumvention. (“Don’t’ speed” becomes “Don’t get caught speeding”.)

2. Concurrent reinforcement may undermine the punishment. (Student punished for talking in class may be reinforced with
approval by other students.)

3. Punishment can lead to more variable behavior. (If a specific behavior is decreased, what replaces it?)
   
   • if you punish a child for jumping on the couch, then they may start jumping on the bed (doesn’t get rid of the class of behaviors)
4. The initial intensity of the punishers needs to be fairly intense (otherwise you may get habituation).
5. Punishment can lead to stress and anxiety, which is associated with other undesirable behaviors. (creates states that aren’t conducive for encourage the behaviors you want)

Question 42

How do animals get trained to do complex (and sometimes stupid) things?

Answer 42

• You can’t simply reinforce a complex behavior as it may not be done accidentally.
• Use chaining (chained learning) to create a series of reinforced behaviors that build on each other (start with something simple and keep adding one step at a time till you get something that looks much more complex)

squirrel on waterskis

S (See platform) –> R (Stand on platform) –> O (Food reward)

S (See handle) –> R (Stand on platform + place paws on handles) –> O (Food reward)

Question 43

Operant vs. classical conditioning

Answer 43

Classical conditioning
• Passive: environment works on animal.
• UStimulus evokes a
response.
• Animal learns that the CS predicts the US.
• Typically simple associations.

Operant conditioning:
• Active: animal operates on environment.
• A behavioral response
produces an outcome.
• Animal learns that behavior predicts an outcome.
• More flexible and powerful, producing more complexity.

However, the two often work together (e.g. primary and secondary reinforcers can become associated classically).

Question 44

Evaluating situations to ID what kind of conditioning or paradigm

Answer 44

is it passive or active?

what’s being associated?

the more complex a behavior the more likely it’s operant conditioning

Question 45

Brain-based models for operant conditioning

Answer 45

any instance of operant conditioning involves the interaction of several neural systems.

Question 46

Law of effect

Answer 46

origins of operant conditioning

states that animals make associations between voluntary behaviors and
contingent outcomes.

Question 47

_____ make a behavior more likely

Answer 47

reinforcers

Question 48

_____ make a behavior less likely

Answer 48

punishers

Question 49

Both reinforcers and punishers can be due to

Answer 49

intrinsic preferences
(primary) or learned associations with intrinsic preferences (secondary).

Question 50

When you add something to the outcome (give a treat or shock), that is

Answer 50

positive

Question 51

When you take away something (pain or freedom),

that is

Answer 51

negative

Question 52

_____ may not always be as effective as reinforcers in the real world, but are equally effective in the lab.

Answer 52

punishers