Exam 2: Chapter 4-6

Question 1

Know the classical conditioning (CC) associative learning paradigm and be able to identify the following in an example: US, CS, UR, CR.

Answer 1

Classical conditioning represents an association between the CS and the US

example:
Pavlov Dog

Innate reflex:
US –> UR

US: saliva
UR: dog food

Training:
CS: bell ringing before food

Conditioned:
CS –> CR

Question 2

CC represents a LEARNED association between the _____ and the _____.

Answer 2

CS and US

Question 3

CC leverages what kind of behavior?

Answer 3

Innate, AKA reflexive, AKA involuntary!

Question 4

Understand the difference between appetitive conditioning (approach) and aversive conditioning (avoidance)

Answer 4

• Appetitive Conditioning: new reflex prepares to obtain the US
• Aversive Conditioning: new CS->CR reflex helps avoid noxious US

Question 5

Be able to explain the eyeblink conditioning experiment, understanding that the CR can occur before the US in anticipation.

Answer 5

US: a puff of air
UR: blink eyes
CS: tone
CR: blink eyes before air puff

Initially, CS does not do anything but after pairing US and CS the bunny/human will produce CR before the US.

Question 6

What is a conditioned compensatory response? How does this maintain homeostasis? How does this contribute to tolerance?

Answer 6

Conditioned compensatory response is a CR that is the opposite of the UR, helping to balance/correct for the US-UR reflex.

I- nject adrenaline (US) -> heart rate increase (UR)
- Repeat procedure in same testing chamber (CS).
- Eventually, CS comes to produce a decrease in heart rate (CR) that helps maintain homeostasis (balance) against expected adrenaline injection.
- We observe this as tolerance, as the testing chamber evokes a CR that weakens the overall effects of the drug.

Question 7

What is extinction? What does it mean to say it can occur “in context”?

Answer 7

Breaking the association betweenthe CS and US can extinguish thenew CS->CR reflex:
- Present the CS alone repeatedly.
- Initially, CS evokes strong CRs.
- With repetition, however, CS becomesless effective, similar to beginning oftraining.

Extinction doesn’t erase the CS-US connection, just inhibits it:
- Stress, new context, and/or passage of time can make the CS effective again!

Question 8

What is compound conditioning? Overshadowing? Blocking?

Answer 8

• When two cues appear together in a conditioning experiment, a paradigm known as compound conditioning occurs.
• One stimuli has the potential to overshadow another stimuli if they occur together. Overshadowing occurs when a more salient cue within a compound acquires far more share of attention and learning than the less salient one.
• Blocking demonstrates that classical conditioning occurs only
• when a cue is BOTH a useful and a non-redundant predictor of the future.

Question 9

Be able to differentiate between the Rescorla-Wagner (US modulation) and Mackintosh models (CS modulation) of CC.
Which is better for explaining error-correction learning? Latent inhibition?

Answer 9

• US modulation – in which learning changes processing of the US (e.g., Rescorla-Wagner Model) EXPECTATION/SURPRISE
  –> Error-correction learning: Errors on each trial lead to small changes in performance that seek to reduce the error on the next trial.
• CS modulation – which postulates that the stimulus enters into an association which is determined by how the CS is processed (e.g., Mackintosh Model) LIMITED ATTENTION
  –> Sometimes there is a reduction in learning about a stimulus if there has been prior exposure to that stimulus without any consequence. This is latent inhibition.

Question 10

Which interstimulus interval (timing) is best for CC learning? What happens when the US is presented before the CS? Understand delay vs. trace conditioning.

Answer 10

• Delay conditioning, is the best learning, where CS is continuous presented until US is presented for a shorter period of time
• “Backwards” conditioning: US then CS, no learning!
• Trace conditioning is CS is presented before the US but only the same amount of time

Question 11

What does it mean to say that some associations are easier to make than others? What is a conditioned taste aversion (CTA)?

Answer 11

Some associations are innately easier to make:
- When tone + taste paired with poison, only taste provokes CR
- When tone + taste paired with shock, only tone provokes CR

Seems we have some innate preferences for forming associations that can override statistical correlations
- E.g., conditioned taste aversion: poison group, tone + taste –> poisoning, tone = ?; shock group: tone + taste –> shock, taste = ?

Question 12

What experiment (conducted by Thompson in the 1980s) showed that CC partly occurs in the cerebellum?

Answer 12

• In the 1980s, Thompson and associates discovered that eyeblink conditioning in rabbits depends on the cerebellum.

Question 13

What two locations in the cerebellum are most likely to store the CS-US association? What evidence do scientists have to implicate these areas?

Answer 13

CS–US association may be stored in:
- Purkinje cells of the cerebellar cortex.
- Cerebellar interpositus nucleus (eyeblink CR pathway).

• Recordings from thecerebellum before andafter learning have helpeddemonstrate where thememory is stored …
  
  • Shutting off the Purkinje inhibition (double negative!) to the interpositus enables the CS to generate CRs. (shown in Electrophysiological Recording in the Cerebellum)

Question 14

What happens to associative learning if there is damage to the interpositus nucleus? What if there is damage to the Purkinje cells?

Answer 14

• Interpositus nucleus damage = destroy and prevent CRs
• Purkinje cells damage = affect timing of CR learning

Question 15

What role does the hippocampus play in CC? (hint: eliminates latent inhibition and other paradigms that depend on changes in the CS – i.e., Mackintosh model vs. R-W model).

Answer 15

• CS modulation (Mackintosh) occurs in thehippocampus and medial temporal lobe
• US modulation (Rescorla-Wagner) occurs in the cerebellum

Question 16

Understand the role of classical conditioning and tolerance in drug addiction.

Answer 16

• Drug tolerance increases with use in the same drug-taking context (situation-specific).
• Even subtle changes (e.g., to drug taste) can overrun tolerance and increase drug effects.
  
  • Increases possibility overdose

In first-time dose rats, large heroin dose led to:
- 96 percent fatal overdose.
In rats with small heroin dose before larger dose in a different location:
- 64 percent fatally overdose.
In rats with small dose before larger dose in the same location:
- Only 32 percent overdose.

Question 17

How can cue-exposure therapy help addicts kick the habit? What happens to the association between the CS-CR in cue-exposure therapy?

Answer 17

• Bouton (2000) suggests that therapists conduct cue-exposure therapy:
  
  • In different contexts, including home.
  • Over varying time lengths.

The association between CS-CR in cue-exposure therapy is being put through extinction

Question 18

What role does extinction play in reducing addiction?

Answer 18

The extinction between the association of CS-CR plays a role in cue-exposure therapy

Question 19

Review Thorndike’s Law of Effect from Chapter 1.

Answer 19

when you want a behavior to increase use reinforcement, and when you want a behavior to decrease use a punishment

Question 20

Define and understand the relationships between the discriminative stimulus (SD), the response (R) and the outcome in operant (instrumental) conditioning (O).

Answer 20

• In context, discriminative stimulus (SD), Response (R) produces Outcome (O): SD -> R -> O

Question 21

What does the SD indicate about contingencies?

Answer 21

• Discriminative Stimulus (S): tell us which contingencies are in effect

Question 22

What is a habit slip? What evidence indicates that the response (R) is not just a rote motor program?

Answer 22

Some SD -> R behavior is so strong that it happens automatically, no matter what other options are available.

This is called a habit slip.

• If you’ve ever woken up late and jumped up to get ready for class only to remember that it is Saturday, then I am talking about you.

Initially, it was thought to be a rote motor program.

• However, if normal motor program is blocked, animal will use other methods to achieve same ends.

R is not a single behavior but a class of behaviors producing an effect. Some cognitive psychologists would call it a goal or intention.

Question 23

What is the effect of reinforcement on behavior? Punishment? What is the difference between a positive or negative consequence?

Answer 23

• Reinforcement increases behavior
• Punishment decreases behavior

Positive Reinforcement
- If it leads to positive effects, do it more
Negative Reinforcement (escape)
- If it ends/avoids a negative effect, do it more

Positive Punishment
- If it leads to negative effects, do it less

Negative Punishment (omission)
- If it ends/avoids a positive effect, do it less

Question 24

Define the acquisition and extinction phases of OC.

Answer 24

Training – contingency is introduced: If S, R -> O

Acquisition – animal discovers contingency, rate of R increases

Extinction – contingency is eliminated (R -> __), rate of R decreases

Question 25

Be able to fully describe the Skinner box experimental paradigm, including how you can shape complex behaviors (chaining) with successive approximations.

Answer 25

• Note that animal is ‘free’ in the chamber, no experimenter intervention (free-operant paradigm Shaping
• Shaping through successive approximation builds a complex R incrementally

Chaining
- Chaining builds complex R sequences by linking together SD->R->O conditions

Shaping and chaining can be used together to train animals to complete incredibly complex behaviors
Both techniques require skill and patience from the trainer
- Keep animal motivated and interested
- Select proper training sequence–can’t move too fast

Question 26

What are the major differences between OC and CC? Be able to identify which is being used to promote learning in example situations. look over chapter examples of differences between OC and CC

Answer 26

Classical Conditioning:

• Environment operates on the animal’s innate behavior
• Stimulus evokes Response (SD->R?)
• Stimulus followed by O
• Animal learns CS (SD) predicts US (O)

Operant Conditioning:

• Animal operates on the environment
• Stimulus evokes a response to produce an outcome (SDRO)
• Animal connects context, behavior, and outcome

Question 27

What is the difference between a discrete-trial paradigm and a free-operant paradigm?

Answer 27

Have to repeat trials over and over, resetting animal and device.
I.e., trials were controlled by the experimenter.
- This is known as a discrete-trials paradigm.

• Note that animal is ‘free’ in the chamber, no experimenter intervention (free-operant paradigm)

Question 28

What are primary and secondary reinforcers? What is the drive reduction theory? What does it mean to say that reinforcers can have negative contrast?

Answer 28

• A stimulus, such as food, water, sex, or sleep, that has innate biological value to the organism and can function as a reinforcer, is known as a primary reinforcer.
• Drive reduction theory states that organisms have innate drives to obtain primary reinforcers and that learning is driven by the biological need to reduce those drives.
• A stimulus (such as money or tokens) that has no instrinsic biological value but that has been paired with primary reinforcers or that provides access to primary reinforcers is known as a secondary reinforcer.
• negative contrast - water to sugar water, no longer want water as a reward

Question 29

What are four things about punishment that can make it less effective than reinforcement? How can differential reinforcement of alternative behaviors help?

Answer 29

Punishment can be effective, but …
1. Punishment leads to more variable behavior.
2. Discriminitive stimuli for punishment can encourage cheating.
3. Concurrent reinforcement can undermine the punishment.
4. Initial intensity matters.

• Rather than delivering punishment every time, use differential reinforcement of alternative behaviors (DRA): a method to decrease the frequency of unwanted behaviors by instead reinforcing PREFFERED alternate behavior.

Question 30

How does timing of reinforcement affect learning in OC?

Answer 30

• Closer time between behavior and consequence, better the learning

Question 31

Know detail regarding the four main schedules of reinforcement (FR, FI, VR, VI) including some examples. What is a post-reinforcement pause? Which schedule creates the “strongest learning”? What did Skinner mean by “strongest learning/behavior”?

Answer 31

Fixed Ratio (FR) - Every X Responses produces 1 Outcome
- ex. ice cream stamp card
- If 1 R = 1 O, then continuous reinforcement schedule
- If 2 R = 1 O, then partial reinforcement schedule

• Post-reinforcement pause:(flat line): time out fromresponding after each reward

Variable Ratio (VR) - Every X Rs produces 1 O, but X changes with each reinforcer
- ex. gambling
strongest learning

Fixed Interval (FI) - After Y seconds, 1 R produces 1 O
example: paycheck, looking at the clock
–> behavior before interval expires has no consequences

FI scallop:
- At beginning of interval, little/no responding
- Increases to rapid rate of responding before interval expiration

Variable Interval (VI) - After Y seconds, 1 R produces 1 O, but Y changes after each O. Behavior before interval expires has no consequence.
- ex. checking your texts

• Skinner meant that the “strongest learning/behavior” are the hardest to extinguish

Question 32

Know the matching law of “choice” behavior, behavioral economics, and bliss point.

Answer 32

• Matching law of choice behavior—response rates to concurrent schedules often correspond to the rate of reinforcement for each schedule.
• Behavioral economics—the study of how organisms distribute their time and effort among possible behaviors.
• Bliss point—the ideal distribution for the organism; provides maximum subjective value.

Question 33

What is the Premack Principle (AKA the response deprivation hypothesis)?

Answer 33

The Premack Principle: Responses as Reinforcers
- By restricting the ability to execute a response, you can make the opportunity to perform that response reinforcing.

Question 34

What role does the dorsal striatum play in OC? Of what structure is the dorsal striatum a part? If the dorsal striatum is lesioned, what happens to OC associative learning?

Answer 34

• Within the basal ganglia, the dorsal striatum (caudate nucleus and putamen) seems to play an important role in SD–> R learning.
• No learning, however, with discriminative stimuli (if light on, then lever press gets food)
• Though, simple RO relationships are learned correctly (e.g., press lever, get food)

Question 35

What role does the orbitofrontal cortex play in OC?

Answer 35

The orbitofrontal cortex (seen at left) may be an important brain substrate for storing R->O associations.
- helps organisms choose particular responses based on the expected outcomes

Question 36

What is the difference between wanting( (motivation) and liking (hedonics)? Can you have one without the other?

Answer 36

• Only when “wanting” and “liking” signals are both present will the arrival of the reinforcer evoke responding and strengthen the SD-> R value.
• Motivational value: (Da)
  how much we “want” a reinforcer and how hard we are willing to work to obtain it
• Hedonic value: (μ)
  the subjective “goodness” of a reinforcer, or how much we “like” it

example: No matter how much we may “like” chocolate cake, most of us will not be very motivated to obtain more if we have just eaten three large slices!
- we can have like without motivation —> satiated and want without like —> addiction

Question 37

What is one of the “pleasure centers” in the brain? What neurotransmitter is used by this center to stimulate powerful reinforcement?

Answer 37

• One of the “pleasure centers” is the ventral tegmental area (VTA) in the brainstem.
  
  • The VTA is the center for dopamine neuromodulation.
• VTA stimulation = powerful reinforcement

Question 38

How does the VTA signal wanting and by stimulating what other regions of the brain with what neurotransmitter? [hint: SNc (SD -> R); orbitofrontal cortex (R -> O); DA]. What is the incentive salience hypothesis of wanting?

Answer 38

Wanting:

Some dopamine (DA) neurons in the VTA have axons that extend to the
- substantia nigra pars compacta (SNc) which is part of the basal ganglia/striatum (SD  R)
- orbitofrontal cortex (R  O)

Makes this an ideal place to start looking at how the brain signals motivation.

• Incentive Salience Hypothesis—dopamine motivates learners to work for reinforcement.

Liking:

Endogenous opioids (endorphins) may mediate hedonic value or “liking.”

• Opiates (heroin, morphine) bind to the brain’s natural opiate receptors.
• Opiates may provide information about “liking” that helps stimulate VTA’s “wanting” system.

Question 39

How does the brain signal liking and with what chemicals?

Answer 39

• Endogenous opioids (endorphins) may mediate hedonic value or “liking.”
• Opiates (heroin, morphine) bind to the brain’s natural opiate receptors.
• Opiates may provide information about “liking” that helps stimulate VTA’s “wanting” system.

Question 40

What two regions in the brain are thought to be involved in punishment signaling?

Answer 40

• Insular cortex (purple) (–> taste center of the brain (gustation, disgusting)) and specifically the dorsal posterior insula is a brain region that helps us determine subjective disliking of painful physiological and psychological stimuli.
• The dorsal anterior cingulate cortex (dACC; orange) may help determine the motivational value of punishers (or no reward… or less reward… Remember negative contrast?) (specifically pain in general)

Question 41

Understand the difference between pathological and behavioral addiction. What initiates addiction? What maintains it?

Answer 41

• Pathological addiction- a strong habit maintained despite harmful consequences.
  
  • involves craving a high “euphoria” and avoiding withdrawal
  • seeking pleasure involves positive reinforcement
  • avoid pain involves negative reinforcement
• Although liking a drug may help initiate addiction, the incentive salience hypothesis suggests that addiction is maintained by “wanting” a drug.
• Behavioral addiction - addiction to certain behaviors, rather than drugs.
  -Produces euphoria.
  
  • Understanding drug addiction may help understand/treat behavioral
    addictions.
• damaging insula –> eliminate addiction

Question 42

What happens at the dopaminergic synapses with methamphetamines? Cocaine? How about with naltrexone?

Answer 42

• cocaine stops the reuptake of dopamine –> get the euphoric
• amphetamines causes you to release more dopamine in the synapse –> get more dopamine
• naltrexone inhibits dopamine production, can help treat heroin addicts and compulsive gamblers

Question 43

What cognitive-behavioral treatments can help with addiction?

Answer 43

(Cognitive) behavior therapies:
- e.g., extinction, distancing, reinforcement of alternative behaviors (DRA), delayed reinforcement (imposing a fixed delay)
- Based on instrumental conditioning principles

Question 44

What is generalization?

Answer 44

Generalization: transferring past experiences to new situations

• generality: applying the past broadly; generalizing
• learned specificity –> a change in the generalization gradient for less generality

Question 45

What is discrimination?

Answer 45

Discrimination: the perception of differences between stimuli

• specificity: applying the past narrowly; discrimination
• discrimination training - providing two different consequences for stimuli initially treated by an animal as similar