Learning theories

Question 1

What is the associative learning theory?

Answer 1

• It accounts for complex phenomena on the basis of a few simple principles
• Suggests that there are nodes (neurones) which respond to stimuli from the environment
• It assumes that if there are co-activation of nodes then an association can be created between the nodes
• Learning consists of the establishment of associations between the mental representations of the CS and the UCS
• Assumes that contiguity between events would be enough to guarantee learning
• But Rescorla (1968) found that contiguity (pairing) does not lead to conditioning
• CS informative value depends on the relative probability of the occurrence of the UCS in the presence and the absence of the CS
  — Poor learning = when the UCS is equally likely in the presence than in the absence of the CS
  — Good conditioning = the UCS only occurs in the presence of the CS
• Contiguity between stimuli does not guarantee learning, but the surprise of the UCS may have some relevance
• Certain types of causes are more likely to produce certain types of effects: the cue-to-consequence effect
  — Garcia and Koelling (1966)
  — Animals received training where they drank a sweet solution in the presence of a light-noise compound
  • This was followed by an internal sickness or an external shock
  — They were then tested with the sweet solution, and also with plain water in the presence of the light-noise
• Conditioning depends on the CS receiving the animal’s full attention
• Conditioning results in the growth of an association between internal representations of the CS and US
  — The strength of this association determines the strength of the CR
  • Called associative strength
  — The greater the change in associative strength on a trial, the stronger the CR will be on the next trial
• The rate of conditioning was directly related to the CS intensity
  — But it did not influence the ultimate level of responding
• Conditioning with a compound CS
  — Rescorla and Wagner (1972) proposed that the associative strengths of the individual stimuli will be added together
  Rescorla (1988)
• We now know that arranging for 2 well-processed events to be contiguous need not produce an association between them
  — Failing to arrange contiguity does not prevent associative learning
  — Contiguity can lead to conditioning = it depends on the amount of information that the UCS gives the CS = conditioning is dependent upon the informational relation on which the groups differ
  — Conditioning occurs whenever one arranges a temporal relation among the events, regardless of the other properties of the events
• Conditioning can sometimes be slow, but many modern conditioning preparations routinely show rapid learning
• If the value of a reinforcer is changed after conditioning has been completed, subsequent responding to its associated CS will also change accordingly
• The response observed to a CS often depends not only on the US but also on the perceptual properties of the CS itself
  — 2 different signals of the same US may evoke quite different responses
• Wasserman, Franklin and Hearst (1974)
  — Positive relationship (CS → US) = approach CR
  — No relationship (CS/US) = no behavioural change
  — Negative relationship (CS → no US) = withdrawal CR

Question 2

What are the 3 dimensions of learning?

Answer 2

• Conditions:
• – Rescorla = contingency = a stimulus will acquire the properties of a CS only if it is informative about the occurrence of the US
• – Garcia = associations between a CS and a US will establish if they are similar or biologically relevant
• – Kamin = conditioning depends upon the salience or intensity of the CS and the US; a stimulus CS will associate only with surprising US
• Contents
• Effect on behaviour

Question 3

What is blocking?

Answer 3

• Kamin (1969) = a blocking study that uses conditioned suppression
  — 1st stage = group E, but not C, was given pairings of a noise with shock
  — 2nd stage = both groups received identical training for a number of trials in which a compound composed of the noise and a light was paired with shock
  — 3rd stage = test trials involving the light by itself
  — Results = virtually no evidence of a conditioned response in group E, whereas one of considerable strength was recorded for group C
  • Suggests that the original training with the noise was somehow responsible for preventing, or blocking, learning about the light during compound conditioning
  • If conditioning was merely dependent on the pairing of a CS and a US, then there would have been effective
  • Kamin proposed that group E learned that the noise predicted the shock in the first stage, which led to the light being followed by an unsurprising US during the 2nd stage
• Suggests that conditioning depends upon the surprise of the US

Question 4

What is inhibitory conditioning?

Answer 4

Pearce (2008)

• Inhibitory conditioning = training in which a conditioned stimulus signals the absence of an unconditioned stimulus
• It can be difficult to determine if the animals have learned anything about the stimulus
• Retardation test = a test for conditioned inhibition by assessing if a stimulus is slow to acquire excitatory properties when it is paired with an unconditioned stimulus
• Summation test = a test for conditioned inhibition by assessing if a stimulus will weaken responding elicited by a conditioned excitor when they are presented together

Question 5

What is overshadowing?

Answer 5

Pearce (2008)

• The disruption of conditioning with 1 stimulus because of the presence of another stimulus
• Mackintosh (1976): overshadowing depends on the intensity or salience of the competitor stimulus
• – A salient stimulus is more likely to overshadow a weak competitor

Question 6

What is alpha conditioning?

Answer 6

Pearce (2008)

An increase in the strength of the unconditioned response to a CS as a result of pairing it with an US

Question 7

What is extinction?

Answer 7

Pearce (2008)
- Training in which a conditioned stimulus is presented by itself after it has been paired with an unconditioned stimulus, resulting in a progressive weakening of the conditioned response

Question 8

What is the CS-US contingency?

Answer 8

Pearce (2008)
The degree to which the US occurs during the CS
- Positive contingency = the US is more likely in the presence than the absence of the CS
- Negative contingency = US is more likely in the absence than the presence of the CS
— This will result in inhibitory learning
- Zero contingency = the US is equally likely during the presence and the absence of the CS
- Exerts an important influence on the associative strength acquired by a CS during excitatory conditioning

Question 9

Evaluate the Rescorla-Wagner model

Answer 9

Pearce (2008)

• Not all the effects associated with blocking and overshadowing are consistent with it
• The model does not adequately account for the role of surprise in conditioning
• Inhibition is conceptualized as negative associative strength, which may not be a sufficient explanation
• The model has been of value by prompting the development of alternative theories of learning

Question 10

What is the orienting response?

Answer 10

Pearce (2008)
- OR = a measurement of the attention that animals pay to a stimulus
- Novel stimuli often elicit an OR/an investigatory reflex that allows the animal to investigate any changes that occur in its environment
- It has been suggested that the conditionability of a stimulus is not solely determined by its intensity, but by the attention it receives
— The attention it receives may be less if the stimulus is familiar vs novel
- Support provided by Kaye and Pearce (1987)
— 2 groups of rats were placed into a conditioning chamber containing a light bulb and a food dispenser
— For the first 12 sessions, nothing happened for group novel, whereas for group familiar the bulb was illuminated for 10s at a time at intervals in each session
— Both groups were then given a single pre-test session in which the light was occasionally illuminated for 10s
— The strength of the OR in the pre-test session was considerably more vigorous in group novel than in group familiar
— All subjects were then conditioned with the light serving as a signal for food
• Conditioning was more rapid in group novel than in group familiar = this effect is known as latent inhibition (Lubow, 1973) = the reduction in effectiveness of pairing a CS with an US, as a result of prior exposure to the CS
• Latent inhibition can develop in the absence of an expenctancy of a US
• Latent inhibition has been found to disrupt inhibitory conditioning

Question 11

What is the Pearce-Hall (1980) model?

Answer 11

• Based on the supposition that animals need to attend to a stimulus only while they are learning about its relationship with its consequences
• – Hence this model suggests that controlled attention will be directed most to those stimuli that need to be learned about, i.e. those that are surprising

Question 12

What is the neural basis of associative learning?

Answer 12

Pearce (2008)

• An enhancement of the ease with which 1 neuron can excite another
• If it is assumed that the degree of similarity between 2 stimuli determines the number of sensory neurons that they both excite, then this account correctly predicts that the amount of stimulus generalization will be determined by the similarity of the test and training stimuli

Question 13

What evidence is there for the suggestion that the CS retrieves a memory of the US, which is then responsible for generating the CR?

Answer 13

Colwill and Motzkin (1994)

• Hungry rats in a conditioning chamber were occasionally presented with a tone that signaled food pellets and a light that signaled sucrose solution
• The training resulted in subjects performing a CR during either the tone or light of approaching the magazine where the 2 outcomes were delivered
• The rats then received training in which they were allowed to consume food and sucrose in different sessions
• – Consumption of food was followed by an injection of lithium chloride (poison) to condition an aversion to it, but consumption of sucrose solution was not
• After several sessions of this training, rats were extremely unwilling to eat the food, but they willingly consumed the sucrose solution
• The rats were again presented with the tone and light in the conditioning chamber, but on this occasion neither stimulus was followed by an outcome
• – The rats were much more willing to approach the magazine during the light than the tone

Question 14

What is stimulus-stimulus learning?

Answer 14

Pearce (2008)

Associations can develop between 2 stimuli, even when neither of them has any unconditioned properties

Question 15

What is serial conditioning?

Answer 15

Pearce (2008)
Training in which 2 or more conditioned stimuli are presented in sequence and followed by a single unconditioned stimulus

Question 16

What is sensory preconditioning?

Answer 16

Pearce (2008)

Training in which 2 stimuli, A and B, are presented together before B is paired with an unconditioned stimulus

Question 17

What is second-order, or higher-order, conditioning?

Answer 17

Pearce (2008)

• Training in which 2 stimuli, A and B, are presented together after B has been paired with an unconditioned stimulus
• With the right amount of training, a CR can be observed during CS2 even though this stimulus itself is never paired with the US

Question 18

What evidence is there to suggest that animals anticipate the rewards for which they are responding?

Answer 18

• If a rat is trained first with 1 reward which is then changed in attractiveness, there is a remarkably rapid change in its performance on subsequent trials
• Elliott, 1928:
  — Found that the number of errors in a multiple-unit maze increased dramatically when the quality of reward in the goal box was reduced
  — The animals were so dejected by the change that they made more errors than a control group that had been trained throughout with an attractive reward
• Tolman (1932) argued that findings such as these indicate that rats form R-unconditioned stimulus associations as a result of instrumental conditioning
  — They are assumed to learn that a response will be followed by a particular outcome
• Support for this theory from Colwill and Rescorla (1985) = devaluation training
  — First stage = rats were able to make one response (R1) to earn one reinforcer (US1) and another response (R2) to earn a different reinforcer (US2)
  • Lever pressing or pulling chain
  • Food pellets or sucrose solution
  — 2nd stage = aversion to US1 was formed by allowing rats free access to it and then injecting them with a mild poison (LiCl)
  • Rats completely rejected US1 when they were presented with it again
  — Test trial: subjects were able to make either of the responses but no reinforcers were delivered
  • Lead to a decline in R2, but R2 was still performed more vigorously than R1
• Several S-R theorists have pointed out that the anticipation of reward could have been based on CS-US associations instead
• Re-examination of Colwill and Rescorla (1985) study:
  — After the devaluation treatment there was still a tendency to perform R1, even though the reinforcer was never delivered and even though the rat would reject US1
  — This would be expected if the original training resulted in the growth of an S-R connection, because it doesn’t allow an animal to anticipate the reward it will receive

Question 19

What is the response-reinforcer contingency?

Answer 19

Pearce (2008)
The degree to which the occurrence of the reinforcer depends on the instrumental response
- Positive contingency: the frequency of the reinforcer is increased by making the response
- Negative contingency: the frequency of the reinforcer is reduced by making the response
- Zero contingency: the frequency of the reinforcer is unaffected by making the response

Question 20

What is the molar theory of reinforcement?

Answer 20

Pearce (2008)
- The assumption that the rate of instrumental responding is determined by the response-reinforcer contingency
Thomas (1981)
- Experiment to test molar theory
- Rats in a test chamber containing a lever were given a free pellet of food once every 20s, even if they did nothing
- At the same time, if they pressed the lever during any 20s interval then the pellet was delivered immediately, and the pellet at the end of the interval was cancelled
— Subsequent responses during the remainder of the interval were without effect
— Hence the rats received 3 pellets of food a minute regardless of whether or not they pressed the level
- According to the molar point of view, the rate of making this response should not increase since level pressing did not increase the rate of food delivery
- The mean rate of responding eventually rises to a reasonably high rate for the rats, which contradicts the prediction from the molar theory

Question 21

What is the Premack principle?

Answer 21

Pearce (2008)

The proposal that activity A will reinforce activity B, if activity A is more probable than activity B

Question 22

What does the animal learn during Pavlovian conditioning?

Answer 22

• The law of effect (Thorndike, 1911)
  — Predicts an association between the CS and the response unit
  — Conditioning depends exclusively on the establishment of associations between a stimulus and a response (S-R)
  — As a result of conditioning the animal will automatically respond in the presence of the CS without anticipating the presentation of the food-reward
  — Thorndike proposed that a stimulus would serve as a reinforcer if animals were willing to respond in order to receive that stimulus
• Stimulus-stimulus associations = a challenge to the law of effect
  — Sensory preconditioning: Rizley and Rescorla (1972)
  — The sensory preconditioning phenomenon shows that:
  • A stimulus which was never paired with the UCS acts as an effective CS
  • Animals can associate stimuli which are not biologically relevant, in the absence of strong unconditioned responses
  • The behaviour of the animals is controlled by an associative chain: CS1 → CS2 → US
• Response-stimulus associations
  — Reinforcer devaluation: Colwill and Rescorla (1985)
  — Demonstrates that:
  • Animals can associate a response with a reward
  • Animals can anticipate the consequences of their behaviour
• Animals are able to encode hierarchical relationships

Question 23

What are some assumptions of the Rescorla-Wagner model?

Answer 23

• Learning results in the establishment of associations between the representations of stimuli in the long term memory
• An association will be established if the CS and the US are processed together in the short term memory
• If the CS activates the US in the long term memory and in the short term memory, this results in the display of the conditioned response
• Stimuli can only be activated in one way in short term memory: associatively or directly
• In the presence of 2 CSs, the associative strength of the individual stimuli will be added together
• A US associatively activated does not have capacity to support new learning

Question 24

What is the Rescorla-Wagner model (1972)?

Answer 24

• A mathematical model of classical conditioning
• Learning takes place when there is a discrepancy between what is expected and what actually happens
  — Actual event, the US, is represented as λ
  — US present: λ = 1, US omitted: λ = 0
  — Expectation of the US: V (predictive value)
  — 0λ, negative surprise: λ-V = 0-1 = -1
• The expectation of the US changes with experience
• The change in the value of V in a particular trial is represented as ΔVn
  — ΔVn = λ-ΔVn-1
  — But this does not work: learning would take place in one single trial
  • Learning would be all-or-none, rather than proceed gradually
  Kamin and Shaub, 1963
• There is a second parameter that determines learning: the salience (importance) of the CS and the US
• The salience is represented as α, of fixed value between 0 and 1
• Learning, ΔV, depends upon α and the surprise of the US
  — ΔVn = α (λ-ΔVn-1 )
  — Assuming that learning depends on the CS salience and the surprise of the US, the model elegantly accounts for gradual learning
  — Vn = Vn-1 + ΔVn

Question 25

Describe overshadowing in terms of the Rescorla-Wagner model

Answer 25

• In compound conditioning (AB → US), we have to sort out the V for A and B
• In every trial, the surprise of the US will be determined by the sum of the VA and VB
• ΔVA = αA [λ-Vtotal ]
• – This formula elegantly models the overshadowing effect
• – Vtotal = 1

Question 26

Describe blocking in terms of the Rescorla-Wagner model

Answer 26

• Stage 1 (A → US): ΔVAn = α (λ-ΔVAn-1 )

- Stage 2 (AB → US): ΔVBn = αB (λ-Vtotal)

Question 27

What are some problems with the Rescorla-Wagner model?

Answer 27

• Spontaneous recovery
  — By allowing negative surprise, the model accounts for the development of inhibition
  — Inhibition allows for a simple and elegant explanation of extinction, characterized as “unlearning”
  — But the model fails to account for the spontaneous recovery effect after extinction of a CR
• Latent inhibition
  — Pre-exposure to the CS retards conditioning
  — Lubow (1965) = experiment shows that something happens during the pre-exposure to the CS that impairs subsequent conditioning
  — Within the R-W model, the contribution of the CS to the learning process is determined by its salience, α
  • α is a fixed parameter that cannot change according to Rescorla and Wagner (1972)
  — Given that α doesn’t change, no differences are predicted between the pre-exposed and the control groups
  — R-W = expectation of the US reduces its effectiveness
  • The same could apply for the CS during the pre-exposure phase of a latent inhibition experiment
• The context plays an important role in learning
  — The context where stimuli are presented can become a predictor of the stimuli
  — Wagner (1981): context can activate a representation of a stimulus in A2
  — Wagner’s theory, by assuming a marginal level of activation (A2) can account for phenomena which fall out of the scope of the R-W model:
  • Habituation
  • Latent inhibition
  — Association context-stimulus accounts for habituation
  • Exposure to the context activates the shock in A2, reducing its processing and producing a weak response
  — Association context-stimulus also accounts for latent inhibition:
  • Exposure to the context activates the light in A2, reducing its processing and making it difficult to associate the CS and US
  — Wagner’s reformulation allows for changes in the processing of the CS
  • It accounts for habituation and latent inhibition
  • It also predicts that, given that latent inhibition depends upon the association between the context and the CS, the phenomenon should be context dependent (context-dependent latent inhibition)

Question 28

What is Lashley’s model of learning theory? (1943)

Answer 28

• Stage 1 = training size discrimination task
• Stage 2 = training size-shape discrimination task
• Test = equal size shapes
• The animals responded at random during the test
• Suggests that animals paid attention to the size and ignored other dimensions that were also relevant (shape)
• Animals pay attention to stimuli perceived as relevant and ignore stimuli perceived as relatively poor predictors of outcomes

Question 29

What is Mackintosh’s model of learning theory? (1975)

Answer 29

• Core idea = the animal that rapidly detects signals for relevant events is more likely to survive than one that ignores these stimuli
• Animals are likely to pay more attention to stimuli that predict important events
• ΔVcs = (processing of CS) (processing of US)
• To become an effective CS, a stimulus should be perceived as a relevant predictor of the US
  — The salience or associability of a novel stimulus is relatively high, and will increase if it is perceived as a good predictor of its consequences
  — A stimulus perceived as irrelevant will lose salience
• CS processing depends on its salience or associability, α, which is a non-fixed feature of the CS
  — α adopts values between 0.05 and 1
  — A new CS will have relatively high associability, α = 0.8
  — Experience will:
  • Increase αCS if perceived as a good predictor of the US
  • Decrease αCS if perceived as a poor predictor of the US
• Supported by Le Pelley et al. (2005)
  — Human participants are presented with trials in which a virtual patient experiences different reactions to different combinations of foods
  • Strawberry + grapes → allergy (Ax → O1)
  • Banana + grapes → diarrhoea (Bx → O2)
  — A and B are good predictors of outcomes 1 and 2
  — X (grapes) is a poor predictor of those outcomes
  — ABCD = good predictors of O1 and O2
  • V and x = irrelevant
  — Trials:
  • Av → O1
  • Bv → O2
  • Cx → O2
  • Dx → O1
  • Ax → O3
  • Cv → O3
  — Test = prediction of O3 = participants were more confident that AC (relevant in stage 1) produce O3 than vx (irrelevant in stage 1)
  — This result is based upon the predictive history of the stimuli
  — Assuming αAC=0.8 and αVX=0.8
  • αAC will increase to ~1
  • αVX will decrease to ~ 0.05
  • VAC = 1 (1-0)= 0.5
  • VVX = 0.05 (1-0)= 0.05

Question 30

What is the Hall-Pearce effect? (1979)

Answer 30

• Phase 1
  — Pre-exposed group = A –> shock
  — Control group = B –> shock
  — Assuming αA=0.8 and αB=0.8
  — Both αA and αB will increase as they reliably predict the outcome (shock)
  — After phase 1, αA≈1 and αB≈1
• Phase 2:
  — Pre-exposed group = A –> shock = αA≈1
  • ΔVA = 1 (1-0) = 1
  • ΔVcs = αcs (λ-V)
  — Control group = A –> shock = αA=0.8
  • ΔVA = 0.8 (1-0)= 0.8
  — We can expect faster learning in group pre-exposed than control
  — But in reality:
  • Group pre-exp showed impaired learning
  • According to that, αA decreased rather than increased in Phase 1
  • This result challenges the main assumption of Mackintosh’s theory

Question 31

Describe the Pearce-Hall model (1980)

Answer 31

• They assume a processor of limited capacity
  — Biologically relevant stimuli (the US) will have granted access to the processor
  — Other stimuli will have restricted access to ensure that only stimuli needed for learning would gain access
• Stimuli that fully predict their consequences will be processed automatically without attentional waste.
• Stimuli with low predictive value (followed by surprising or unpredicted events) will attract the attention and will be fully processed (controlled processing)
• ΔVcs = σcs (λ-V)
  — Processing of the CS depends on σCS
  • σ adopts values between 1 and 0.5
  — That is, the better a CS predicts an outcome (the US), the less attention it will attract
  — This is the opposite to the attentional parameter, α, put forward by Mackintosh (1975)
  Supported by Kaye and Pearce (1984)
• Pre-test = all groups receive light stimulus only
• Test:
  — None group = light stimulus, no food
  — Continuous = light stimulus always accompanied with food
  — Partial = food is sometimes given with the light
• The orientation response (OR) measures the attention to a stimulus
• A novel light stimulus elicits an OR from the rats: σLight = 0.8
• Repeated presentations of the light result in habituation of the OR; σLight=0.5
• Groups None and Continuous
  — L is a good predictor of outcome
  — Attention will gradually decrease
  • σLight = 0.8 → σLight = 0.5
• Group Partial
  — L is a bad predictor of outcome (food and no food in alternated trials)
  — Attention will be kept high
  — σLight = 0.8 → σLight = 0.8 - 1
• Attention to L decreased when it was a good predictor of its consequences and was maintained high when its consequences were unpredictable

Question 32

What is the hybrid attentional model?

Answer 32

• Proposed by Le Pelley, 2004
• Combines the attentional processes put forward by Mackintosh (1975) and Pearce and Hall (1980)
• Assumes the Rescorla-Wagner model (processing of the US)
• Assumes 2 parameters which modulate the processing of the CS
  — α = salience parameter characterised by Mackintosh (1975)
  — σ = salience parameter characterised by Pearce and Hall (1980)
• VA= αA σA (λ-V)
  — α adopts values between 0.05 and 1
  • Determines which stimuli get access to the learning mechanism
  — σ adopts values between 0.5 and 1
  • Determines the level of processing of the stimulus by the learning mechanism
• Using Le Pelley et al. (2005) as an example:
  — AC good predictors of outcomes:
  • αAC increases
  • σAC decreases
  • αAC≈1
  • σAC≈0.5
  • ασ ≈ 0.5
  • V= α σ (λ-V) .: VAC= 1 x 0.5 (1-0)=0.5
  — VX poor predictors of outcome
  • αVX decreases
  • σVX increases
  • αVX≈0.05
  • σVX≈1
  • ασ ≈ 0.05
  • V= α σ (λ-V) .: VVX= 0.05 x 1 (1-0)=0.05
• Using the Hall-Pearce effect (1979) as an example
  — Initial level of αA and σA = 0.8
  — Phase 1 (Pre-exp)
  • αA increases
  • σA decreases
  • αA≈1
  • σA≈0.5
  • αA * σA ≈ 0.5
  — Phase 2
  • V= α σ (λ-V)
  • Pre-exposed
  • VA= 1 x 0.5 (1-0)=0.5
  • Control
  • VA= 0.8 x 0.8 (1-0)=0.64
  — The hybrid model correctly predicts the Hall-Pearce effect