Task 6 - Instrumental Conditioning

Question 1

Operant conditioning

Answer 1

process whereby organisms learn to make or refrain from making certain responses in order to obtain or avoid certain outcomes
example: Thorndikes puzzle box

Question 2

Reinforcement

Answer 2

this process of providing an outcome for a behaviour that increases the probability of that behaviour

Question 3

when deciding whether paradigm is operant or classical

Answer 3

• -> focus on the outcome
• when the outcome happens regardless –> classical
• when the outcome only happens by chance (if one does something) –> operant

Question 4

Free-operant paradigm

Answer 4

animal could operate the apparatus freely, whenever it chose (f.e. when Thorndike added a return ramp to his puzzle box)`

Question 5

Discrete trials paradigm

Answer 5

trials were controlled by the experimenter

Question 6

Skinner box

Answer 6

he devised the cage – with a trough in one wall through which food could be delivered automatically

Question 7

Cumulative recorder

Answer 7

A learning curve drawn by a pen that moves across a roll of paper at a steady rate, increasing its vertical height by a fixed amount for every response of an organism, such as a lever press by a rat in a Skinner box or a peck by a pigeon of an illuminated plastic key – f.e. odometer in the car

Question 8

Discriminative Stimuli

Answer 8

stimuli that signal whether a particular response will lead to a particular outcome
–> they help the learner discriminate or distinguish the conditions where a response will be followed by a particular outcome

Question 9

Discriminative Stimuli

Answer 9

stimuli that signal whether a particular response will lead to a particular outcome
–> they help the learner discriminate or distinguish the conditions where a response will be followed by a particular outcome

Question 10

Shaping (??)

Answer 10

in which successive approximations to the desired response are reinforced

Question 11

Chaining (–> backward chaining)

Answer 11

technique in which organisms are gradually trained to execute sequences of discrete responses

• related technique to shaping
• -> sometimes more effective to train the steps in the reverse order

Question 12

Chaining (–> backward chaining)

Answer 12

technique in which organisms are gradually trained to execute sequences of discrete responses

• related technique to shaping
• -> sometimes more effective to train the steps in the reverse order

Question 13

Reinforcer

Answer 13

is a consequence of behavior that leads to increased likelihood of that behavior in the future

Question 14

Primary reinforcers

Answer 14

they are of biological value to the organism, and therefore organisms will tend to repeat behaviors that provide access to these things
- examples: Food, water, sleep, the need to maintain a comfortable temperature, and sex

Question 15

Drive reduction theory (Clark Hull)

Answer 15

proposed that all learning reflects the innate, biological need to obtain primary reinforcers
–> complication: primary reinforcers are not always reinforcing

Question 16

secondary reinforcers

Answer 16

reinforcers that initially have no biological value, but that have been paired with (or predict the arrival of) primary reinforcers (can be as strongly encouraging as primary enforcers)
– Example: money

Question 17

Token economies

Answer 17

often used in prisons, psychiatric hospitals, and other institutions where the staff has to motivate inmates or patients to behave well and to perform chores such as making beds or taking medications

• tokens function in the same way as money does in the outside world
• Animals as well will work for secondary reinforcers

Question 18

negative contrast:

Answer 18

organisms given a less-preferred reinforcer in place of an expected and preferred reinforcer will respond less strongly for the less-preferred reinforcer than if they had been given that less-preferred reinforcer all along
– F.e. the monkey that throws the cucumber because it is the less preferred food, once he saw the grapes

Question 19

Punishment

Answer 19

the process of providing outcomes for behaviour that decrease the probability of that behaviour – the response decreases

Question 20

Punishers or negative outcomes

Answer 20

common punishers for animals include pain, confinement, and exposure to predators (or even the scent of predators)

Question 21

Four most important factors that determine how effective the punishment will be

Answer 21

1. Punishment leads to more variable behaviour.
2. Discriminative stimuli for punishment can encourage cheating
3. Concurrent reinforcement can undermine the punishment
4. Initial intensity matters

Question 22

Differential reinforcement of alternative behaviors (DRA)

Answer 22

A process – rather than delivering punishment each time the unwanted behaviour is exhibited, it’s possible to reward preferred, alternate behaviours

Question 23

Reinforcement schedules

Answer 23

the rules determining when outcomes are delivered in an experiment

Question 24

Timing affects learning

Answer 24

Normally, immediate outcomes produce the fastest learning

Delays undermine the punishments effectiveness, and may weaken learning

Question 25

Response consequence delay

Answer 25

the longer one waits to punish something/someone the less the association will be made between the punishment and the …

Question 26

Self-control

Answer 26

an organism’s willingness to forego a small immediate reward in favor of a larger future reward

Question 27

Positive (reinforcement)

Answer 27

positive does not mean good → instead it means added

Question 28

Positive reinforcement

Answer 28

the desired response causes the reinforcer to be added to the environment

Question 29

Positive punishment

Answer 29

an undesired response causes a punisher to be added to the environment

Question 30

Negative reinforcement

Answer 30

behaviour is encouraged (reinforced) because it causes something to be subtracted from the environment – over time the response becomes more frequent
– sometimes called avoidance training

Question 31

Negative punishment

Answer 31

something is subtracted (negative) from the environment, and this subtraction punishes the behavior
– sometimes called omission training

Question 32

Negative (reinforcement)

Answer 32

negative does not mean bad, it means subtraction in a mathematical sense

Question 33

Reinforcement / Punishment
Positive / Negative
(Definition)

Answer 33

the terms reinforcement and punishment describe whether the response increases (reinforcement) or decreases (punishment) as a result of training. the terms positive and negative describe whether the outcome is added (positive) or taken away (negative)

Question 34

Partial reinforcement schedules

Answer 34

patterns in which an outcome follows a response less than 100 percent of the time
– Example: Becky has to clean her room seven days in a row to obtain her weekly allowance (seven responses for one reinforcement)

Question 35

Four types of partial reinforcement:

Answer 35

1. Fixed-ratio (FR) schedule
2. Fixed-interval (FI) schedule
3. Variable-ratio (VR) schedule
4. Variable-interval (VI) schedule

Question 36

1. Fixed-ratio (FR) schedule

Answer 36

In operant conditioning, a reinforcement schedule in which a specific number of responses are required before a reinforcer is delivered; for example, FR 5 means that reinforcement arrives after every fifth response

Question 37

Postreinforcement pause

Answer 37

In operant conditioning with a fixed-ratio (FR) schedule of reinforcement, a brief pause following a period of fast responding leading to reinforcement.
It just happens – the animal takes a break – the longer the organism is doing the response the longer the pause will be

Question 38

2. Fixed-interval (FI) schedule

Answer 38

an FI schedule reinforces the first response after a fixed amount of time

Question 39

3. Variable-ratio (VR) schedule

Answer 39

a VR schedule provides reinforcement after a certain average number of responses
–> as a result, there is a steady, high rate of responding even immediately after a reinforcement is delivered, because the very next response just might result in another reinforcement

Question 40

4. Variable-interval (VI) schedule

Answer 40

a VI schedule reinforces the first response after an interval that averages a particular length of time – VI schedules tend to produce higher rates of responding than FI schedules (more reinforcing than the fixed-ratio)
The interval schedules are better than the ratio

Question 41

Concurrent reinforcement schedules

Answer 41

in which the organism can make any of several possible responses, each leading to a different outcome
– Linked to behavioural economic –> how they use their time and resources

Question 42

Matching law of choice behaviour

Answer 42

the principle that an organism, given a choice between multiple responses, will make a particular response at a rate proportional to how often that response is reinforced relative to the other choices

Question 43

Behavioural economics

Answer 43

the study of how organisms allocate their time and resources among possible options
– economic theory predicts that each consumer will allocate resources in a way that maximizes her “subjective value,” or relative satisfaction. (in microeconomics, the word utility is used instead of subjective value.) the value is subjective because it differs from person to person
Pigeon could either get a reinforcer after a minute or two pellets after 2 min

Question 44

Bliss point

Answer 44

the particular allocation of resources that provides maximal subjective value to an individual
- Changes depending on context

Question 45

Premack principle

Answer 45

The theory that the opportunity to perform a highly frequent behavior can reinforce a less frequent behavior; later refined as the response deprivation hypothesis.

• • Example: if you have been studying for several hours straight, the idea of “taking a break” to clean your room or do the laundry can begin to look downright attractive
• • Rats want to run on their wheel

Question 46

Response deprivation hypothesis

Answer 46

a refinement of the Premack principle stating that the opportunity to perform any behaviour can be reinforcing if access to that behaviour is restricted → want something because you can’t have it

Question 47

Basal ganglia

Answer 47

collection of ganglia (cluster of neurons) information from the sensory cortex to the motor cortex can also travel via this indirect route
One part of the basal ganglia is the dorsal striatum – which can be further subdivided into the caudate nucleus and the putamen

Question 48

dorsal striatum

Answer 48

receives highly processed stimulus information from sensory cortical areas and projects to the motor cortex, which produces a behavioral response
– Plays a critical role in operant conditioning, particularly if discriminative stimuli are involved
Rats with lesions of the dorsal striatum can learn operant responses (e.g., when placed in a skinner box, lever-press R to obtain food O). But if discriminative stimuli are added (e.g., lever-press r is reinforced only in the presence of a light sd), then the lesioned rats are markedly impaired – similar to people that have a disruption to the striatum due to Parkinson’s disease or huntington’s disease
→ the dorsal striatum appears necessary for learning SD → R associations based on feedback about reinforcement and punishment

Question 49

Orbitofrontal cortex

Answer 49

appears to contribute to goal-directed behavior by representing predicted outcomes

• • receives inputs conveying the full range of sensory modalities (sight, touch, sound, etc.) and also visceral sensations (including hunger and thirst), allowing this brain area to integrate many types of information;
• • outputs from the orbitofrontal cortex travel to the striatum, where they can help determine which motor responses are executed

Question 50

?? is this also right ??

Answer 50

First projects from the sensory cortex (stimulus) to → the orbitofrontal cortex (prediction) → then to the basal ganglia (SD → R association)→ then to the striatum (motor learning)→ then to the motor cortex (reaction)

Question 51

wanting and liking in the brain

Answer 51

later studies identified that rats would work for electrical stimulation in several brain areas, including the ventral tegmental area (VTA)

Question 52

Ventral tegmental area (VTA)

Answer 52

a small region in the midbrain of rats, humans, and other mammals – produces dopamine (wanting something) – can stimulate the VTA to get same effect as a reinforcer

Question 53

“pleasure centers”

Answer 53

some researchers inferred that the rats “liked” the stimulation, and the VTA and other areas of the brain where electrical stimulation was effective became informally known as “pleasure centers.”

Question 54

Anhedonia hypothesis

Answer 54

the incentive salience hypothesis proves this wrong – that wanting and liking is the same thing and that dopamine is for both

Question 55

Hedonic value

Answer 55

the subjective “goodness” of a reinforcer, or how much we like it

Question 56

Motivational value

Answer 56

meaning how much we “want” a reinforcer and how hard we are willing to work to obtain it

Question 57

Incentive salience hypothesis

Answer 57

The hypothesis that dopamine helps provide organisms with the motivation to work for reinforcement – states that the role of dopamine in operant conditioning is to signal how much the animal “wants” a particular outcome—how motivated it is to work for it

Question 58

Endogenous opioids

Answer 58

brain chemicals that are naturally occurring neurotransmitter-like substances (peptides) with many of the same effects as opiate drugs

Question 59

how do “wanting” and “liking” interact

Answer 59

Possible way that the two brain systems (of liking and wanting) interact: differences in the amount of endogenous opioid released, and in the specific opiate receptors they activate, may help determine an organism’s preference for one reinforcer over another

Question 60

Pathological addiction

Answer 60

a strong habit that is maintained despite harmful consequences

addiction may involve not only seeking the “high” but also avoiding the adverse effects of withdrawal from the drug. in a sense, the high provides a positive reinforcement, and the avoidance of withdrawal symptoms provides a negative reinforcement—and both processes reinforce the drug-taking responses

Question 61

Behavioural addictions

Answer 61

are addictions to behaviour, rather than drugs, that produce reinforcements or highs, as well as cravings and withdrawal symptoms when the behaviour is prevented
– Perhaps the most widely agreed-upon example of a behavioral addiction is compulsive gambling

Question 62

Detoxification

Answer 62

taking a different drug instead – like drinking alcohol free beer

Question 63

Extinction

Answer 63

if response R stops producing outcome o, the frequency of r should decline

Question 64

Distancing

Answer 64

avoiding the stimuli that trigger the unwanted response

Question 65

Differential reinforcement of alternate behaviours (DRA)

Answer 65

reinforce yourself for example with a spa day if you didn’t use the drug or punish yourself if you did use the drug

Question 66

Delayed reinforcement

Answer 66

whenever the smoker gets the urge to light up, she can impose a fixed delay (e.g., an hour) before giving in to it

Question 67

most effective treatments

Answer 67

combine cognitive therapy (including counseling and support groups) with behavioral therapy based on conditioning principles—and medication for the most extreme cases

Question 68

Protestant ethic effect (NOT sure if this is right)!!!

Answer 68

delusional – would rather work for their food then get it freely – you think that you do something for an effect– vs habit slip

Question 69

Reward prediction hypothesis (ask about this again)

Answer 69

the firing of dopamine

Question 70

Reward prediction hypothesis (ask about this again)

Answer 70

the firing of dopamine

– the phasic activity of dopaminergic neurons in the midbrain signals a discrepancy between the predicted and currently experienced reward of a particular event