Everything Flashcards
Reversal Learning (Task 7)
Choose between 2 stimuli and get either monetary reward or loss Probalistic Task and Deterministic Task
- Contingencies reversed
Results for Control
- greater activity in switch trials (AFI = cognitive control, lOFC = updating, MCC=switch and error monitoring)
- greater activity in stay trials
(vmPFC less deactivated = correlated with value)
Results for Lesion:
- no greater activity in lOFC (amygdala causally involved)
- no correlation with vmPFC for expected value
- no difference to control for response
Implications:
amygdala crucial for modulation of feedback processing
vmPFC and expected reward = missing affective tag leads to weaker correlation
Murrays Model
IT/PRh, Amygdala, OFC, vmPFC, preSMA and ACC
- object identity and affective value are processed in parallel
- value updating in the OFC coming from amygdala
DeMartino Study
Gain frame leads to loss aversion
Loss frame leads to gambling
- amygdala activity aligned with framing
- PreSMA and aMCC counter framing (conflict monitoring)
- OFC and vmPFC rational acting
Sumner: Masked Prime Paradigm
Aim: PreSMA and FEF are involved in unconscious suppression of stimulus evoked motor plans
Presentation of prime followed by masking stimulus
Compatible trials and incompatible trials
Results healthy pp:
- below 100ms: priming facilitation of task
- above 100ms: NCE (automatic inhibition), slower for congruent trials
Implication:
1st subconscious motor activation
2nd NCE automatic inhibition of motor activation
Results lesions pp:
- disrupted NCE
Implication
SMA and FEF involved in automatic inhibition of unwanted motor activation externally elicited
Affordances
Potential use of an object based on its properties, opportunities for possible actions
E.g. doorknob
Ventral Stream
What? visual inputs = mental representations
Dorsal Stream
How? navigation and control of skilled action
Dorsodorsal stream
grasp component
online controlling of action directed at currently visible stimuli
Ventrodorsal stream
use component
skilled tool action
slower elicit but maintains longer
Binkowski Experiment
Participants initiation time was measured for conflict objects and non conflict objects
Conflict objects
Use - function action : slower RT
Grasp - structural action: only slower when participants performed use task before
Non conflict:
Ras were faster
Spatial Delayed Response Task
See what happens during delay
(not just maintenance of sensory items)
Fixate on a central cross, target appears, target disappears, make saccade to target location
manipulations monkeys had to respond to the opposite orientation (180 degrees)
Results: Different firing during delay depending on task rule, sustained activity reflects the transformation from sensory input to response
Millers Model of PFC Function
Meaningful behaviour is the output of complex, learned stimulus response mapping rules
Internal hidden units representing intervening stages of processing (between cues and response/voluntary action)
Different set of cues activates different responses
Example ringing phone
at home or someone else’s house
Biased Competition Model
neurons compete for activation, visual attention biases this competition in favour of task relevant stimuli and attenuates activity for task irrelevant stimuli
Wisconsin card sorting task
sort based on shape, color, number or symbols
LFC damage: can learn initial rule but struggle to switch to new sorting rule
S-R associations, can not flexibly switch between rules
Lateral PFC more than WM
Not only maintenance of sensory input but transformation into response code, focus on use of information
Matching and nonmatching task
lPFC maintains representation and use
Cells fire differently during delay according to specific rule in operation
Match or no match rule indicated by juice drop or low tone
Xu experiment
IFJ (part of lPFC) showed synchronisation with FFA for faces and PPA for houses
- reflects attentional biases and task relevant perception
-modulation by top down attention
stream of pictures with faces or houses = one back images
Crosstalk/overlap in dorsal medial striatum
dmstriatum as a hub , allowing for integration of motivational, cognitive and motor information
Feedforward organisation
Interconnections between striatum and dopamine neurons (VTA) are organised in a feedforward fashion from ventral to more dorsal areas
- reinforcement learning facilitating goal directed behaviour to cognitive engagement adjusting mistakes and fine-tuning to automation of motor actions
- from motivational/limbic, to cognitive/associative to motor loop
example: learning to play piano
Cools Experiment, set switching
rapid switching between tasks (letter or number naming)
- color of the card indicated the rule
- crosstalk condition: letter and digit presented = need to infer appropriate rule and inhibition of competing rules
results: parkinson patients showed impairment in rule switching trial with cross talk
- controlled for WM, rule based learning, general slowing as confounders
Lee: T maze Task
Stimulation of dopamine receptors changes your tendency of direction, but did not predict outcome just changed the tendency
Activity in the direct pathway (D1) votes for contralateral orientation
Activity in the indirect pathway (D2) votes against contralateral orientation (but for ipsilateral)
Example for loop interaction
(chocolate)
motivational loop: high dopaminergic signal for chocolate
amygdala evaluates emotional significance (affective memories)
cognitive loop: PFC processes LT consequences, rational evaluation, parietal cortex may assess availability and effort required
motor loop: prepares motor plan
1) nucleus accumbens
2) caudate nucleus
3) putamen
Blocking Paradigm
Blocked predictor does not elicit a negative prediction error as a response to no reward and elicits a positive prediction error when reward was is shown
Stimulus A has already been associated with a reward through prior conditioning
When new stimulus B is introduced along with A, aB does not effectively learn to predict reward
Stimulus A is fully predicting the reward on its own
Inhibition Paradigm
stimulus X becomes associated with the absence of reward (conditioned inhibitor)
- when the expected outcome is contradicted with a delivery of response after x it leads to a positive prediction error
Dopamine Signal in Reward Uncertainty
Coding of reward uncertainty is distinct from reward prediction error
highest sustained activation for CS with P=0.5
- when there is a chance but not certainty, dopamine neurons fire at a steady rate
Learning Based on Dopamine
How and where?
happens through short and long term modification of synaptic transmission in the striatum. These changes can lead to optimised prediction of rewards following occasion until prediction errors no longer occur.
transmitted in the striatum, a part of the brain. These changes help the brain better predict rewards over time until there are no more mistakes in predicting those rewards.
remember prediction error
Why are the actual rewards often higher/lower than the expected reward
Dopamine neurons do not respond to proportional size of rewards but to size of errors in reward prediction
Wilhuhn Cocaine self administration
measured dopamine release in the VMS and DLS
daily 1h sessions of r 3 weeks
active key (cocaine) and nonactive key (nothing)
phasic dopamine release in the VMS during early self-administration and diminished in amplitude for week 2 and 3
phasic dopamine in DLS not present in early self-administration but appears in week 2 and 3 (progressively emerges)
recreational to habitual use
development of phasic dopamine in DLS depends on VMS
motivational addiction to behavioral addiction
Sensory Specific Satiety
Rushworths Model
lOFC - value assignment
mOFC - value comparison
ACC - action value comparison
ACC vs MCC
value related vs motor related
which option is preferred, which action is selected
Reversal Learning Task - Camille
vmPFC vs MCC lesions
stimulus vs action value
card deck vs wrist movement
associations and learned and relearned (contingencies were changed)
Affordance Competition Hypothesis
Action selection and action specification occur simultaneously and behaviour is a constant competition for most relevant action
- sensory info is not collected to have a complete picture of environment but in an action dependent manner
Competition
different brain areas
Competition between potential actions via biasing by inputs from different sources
- by lPFC: topdown control, based on task rules
- by basal ganglia controls action based on expected reward
- by parietal activity: decision making factors
cells with different movement preferences mutually inhibit each other
Fronto parietal system
set of loops spanning over central sulcus, each loop processes info relating to separate aspect of movement
smell a cup of coffee (sensory cortex)
knowing its lunchtime (PFC)
Traditional filter models of attention
Top-Down Control: higher brain areas, such as PFC, generate priority signals that guide attention. These signals bias the sensory cortices to process certain stimuli over others.
Sensory Filtering: Attention acts as a filter at the level of the sensory cortices, allowing only a subset of sensory signals to reach higher levels of processing. This selective processing is influenced by goals, expectations, and relevance.
The posterior sensory brain structures (visual or auditory cortices) are directly influenced by attention signals that filter incoming sensory data based on salience and relevance.
- neurons in sensory areas modulate firing depending on how attention is allocated
LIP is thought to contain a salience map
New view of attention
Value Based Decision Making: Attention is not the initial filter, instead it arises from the processes involved in value-based decision making. =basal ganglia
State Estimation: The brain first estimates the most appropriate state for the current situation. This state estimation process involved learned associations and current goals, integrating sensory and other data.
Basal Ganglia assess and assign value to different potential states, effectively deciding which state should dominate. This dominant state then influences what the sensory cortices should prioritize and process.
state
cooperation of learned associations between inputs outputs and prior knowledge
each state is a template for a particular situation and dictates what to attend to
First over the finish line task
pp observe motion of two balls moving from different starting points and different speeds towards a finish line, after short time ball disappears, pp have to predict
- received feedback on performance or no feedback
- error rate was kept 37%
- or x on 26% of trials
results
- experiment 1
inhibition of habenula = disinhibition of VTA
- correct trials, increased
dopamine response, facilitating learning
- less inhibition of VTA which inhibit aMCC so no ERN but excitation of VS and putamen (reward)
error trials, opposite effect
decreased dopamine response impairs learning
=ERN can reflect either omission of reward OR higher uncertainty during errors
Experiment 2
correct trials with x had similar activation patterns to error trials
error trials with x lower habenula activity less inhibition of VTA which inhibits aMCC so no ERN
MCC activated by error with negative feedback, = error monitoring
ERN
occurs 50-100ms post error
ERN indicates a worse than expected result
anti learning function
disinhibition of aMCC
- feedback signal
error detection and internal error monitoring
-by informative feedback :(
- action outcome associations
Action Selection and acition specification
What to do?
- based on external sensory information about objects and internal information about current needs
How to do?
- based on external sensory information: info about spatial relationships among objects relative to configuration of own body
ERN as a feedback signal
depression of dopamine system leads to disinhibition of MCC which elicits ERN
- ERN also reflects emotional response to errors due to connection to limbic regions (amygdala)
- leads to behavioural adjustment due to updates action outcome associations
