Decision Making Flashcards
What is one of the most interesting things about how we make decisions?
-how we compare things that dont have a quantifiable value
What is most cognitive neuroscience research about decision making based on?
behavioral economics and animal models from behaviorism where the value or likelihood of reward can be quantified
In an decision what do we have to represent in order to come up with a set of feasible actions?
represent our environment (external states) and our own states (internal states)
How do we make decisions in three broad phrases?
we evaluate potential actions
select an action
and evaluate the outcome of the action
What do we get the chance to do with the outcome of a decision?
we have a chance to learn something from the outcome
In the weather prediction task what could learning potentially influence?
presentation of cards in the deck
predicted reward contingencies for the cards
selection of rain or shine based on those contingencies
Why is it harder to study decision making in real life?
do not get to see the direct outcome of a decision immediately
What does valuation overlap with in the brain?
areas involved in habit learning and some of the same reward circuitry is active
What was seen and done in the study in which Roitman and Shalden trained monkeys on coherent dot motion display?
-told the monkeys to make a saccade to the target when you have decided what direction the dots are moving in
-single unit activity was measured in the LIP
-actviity ramps up over time
What does the LIP represent in the context of the dorsal attentional system?
-represents both the location of the stimulus you are monitoring for and the target location of eye movement
How come in the Roitman and Shalden study even if there was 0% coherence for the motion of the dots the monkey still made a saccade in a certain direction?
the monkey might be gathering evidence in a way that favors one side
LIP neurons show sustained activity associated with what?
associated with endogenous attention to specific retinotopic locations
-LIP is representing the motor target of eye movement and location in space where the stimulus occurs
How do attention related regions represent in the context of decision making?
represent both actions and internal and external states relevant to deciding among actions
Where in the brain does activity correspond to accumulating evidence for a perceptual decision?
dorsal attentional stream
What is valuation?
what is the value of each action given internal and external states
What are the three different valuation systems?
-pavlovian
-habitual
-goal directed
What is an appetitive and avoidant example of pavlovian valuation system?
appetitive - eat all food on the plate - reward obtained food
avoidant - cross street upon seeing dangerous person punishment avoided - potential harm
What is an appetitive and avoidant example of habitual valuation system?
appetitive - morning cup of coffee; reward - stimulant
avoidant - drive usual route to work; punishment avoided - traffic
What is an appetitive and avoidant example of goal directed valuation system?
appetitive - movie selection reward - entertainment
avoidant - go for a run - punishment avoided - obesity
Wha kind of conditioning is habit learning?
operant conditioning - associating behavior with reward not stimulus with reward - is a form of nondeclarative learning
What kind of conditioning is pavlovian learning?
classical conditioning where you associate a reward with a stimulus
How is goal directed learning?
it is longer term and more abstract and conlifct in goal directed evaluation system and more simple brain evaluation system
In pavlovian learning what is the value assigned based on?
an evolutionarily appropriate response
The habit learning system is highly conserved evolutionarily what regions of the brain is based on?
basal ganglia or striatum
midbrain dopaminergic nuclei vta or SNc
medial prefrontal cortex
What is the value assigned based on in goal directed learning?
computes new values based on the evaluation of new rewards - a more dynamic system in response to new information
What region of the brain is associated with goal directed system?
dorsolateral prefrontal cortex
What are the valuation systems and the limits of rational economic models?
risk and uncertainty - expected value vs unexpected utility and prospect theory
time - immediate rewards and rewards offered at a delay are evaluated differently
What are some of the assumptions we make in regards to how people make rational decisions about buying and selling?
-they represent value accurately
-they have accurate information
-they make decisions that optimize value
-there are issues thought with assuming humans are rational though
What is the st petersburg paradox?
-bernoulli was interested in gambling game involving coin flipping
-you pay a fixed amount to play the game
-heads you win
-the prize on each flip is proportional to the likelihood of winning n times
-keep flipping until it comes up tails
-how much would you pay to play
-the paradox is that the expected value is essentially infinite
s-o people should be willing to pay huge sums to play but are not
-the expected value - sum the product of the probability of reward and likelihood of reward for all possible outcomes
What relationship must we assume between value and utility to get more reasonable results?
a logarithmic relationship between value and utility
-have a logarithmic terms for potential gain and loss
-cost to play
and current wealth where people with more money feel the cost les for a game like this
Since log scales are a basic feature of perceptual systems what is another example of a log scale?
the just noticeable difference between two stimuli varying in intensity gets larger as the value of the intensity increases
-utility is not linearly related to value
What does it mean to estimate utility in a reference dependent way?
do not treat values of gains and losses as equivalent
-bad at estimating probabilites at lower end of scale
What is prospect theory?
prospect theory relates subjective value or utility to objective value as a complex function
-the function is steeper below zero which means people are loss averse
-the function is slightly diffreent across individuals but general shape is the same
-subjective probability follows a weighting function
-people overestimate likelihood of infreuqenct events and underestimate likelihood of high frequency events
-to compute expected uiltiy we apply utility function and probability weighting then compute the sum of products as before
How did tom et al investigate expected utility in the reward system?
presented participants with a decision on each trial
-possible win
-possible loss
-full range of win loss combinations tested
-50/50
-participants had top decide whether to accept or reject the bets on each trial
-no win loss feedback was given during the imaging session
-big win and small loss and small win and big los are easy decisions and when both the loss and win are small or large the decision is more difficult
-the loss range is smaller than win range whcih is consistent with prospect teheory
-gains half as valuable as losses so needed to gain twice more than you lose to play - people take this bet 50/50 of the time
IN WHAT REGIONS OF THE BRAIN IS THEIR ACTIVITY RELATED TO THE REPRESENTATION OF EXPECTED UTILITY OF EACH TRIAL?
STRIATUM AND VENTROMEDIAL PFC
-OTHER RESEARCHERS HAVE FOUND THAT SYSTEMS INVOLVED IN NEGATUVE EMOTIONS LIKE FEAR ARE MORE STRINGLY ASSOCIATED WITH POTENTIAL LOSSES
FOUND ESTIMATED LOSS AVERSION NAD ACTIVITY IN VENTRAL STRIATUM TO BE STRONGLY CORREALTED
What is delay disocunting?
people prefer the larger sum in the future but the smaller sum when it is immediately available
-there is a premium in rewards that are available immediately discount rewards that are available at a delay
How did mcclure et al investigate the immediacy premium?
gave people choices that varied in delay (days to weeks to month) and value (5 dollars vs 40 dollars)
-participants told would get one choice at end of experiemtn
what regions of the brain respond more strongly when a reward is immediately offered?
striatum and medial pfc - dopaminergic system
What set of reions was sjown to have no sensitivity to imemdiacy but are strongly actuvated by the task in general?
-visual cortex and pre supplementary motor areas
-the lateral pfc and ofc are thought to be related to goal direction evaluation
-the posterior parietal region - aka dorsal attentional network is involved in accumulating evidence for response
When mcclure et al focused on more difficult decisions in whcih the more delayed rewrad was 5-25% more than the less delayed reward what did they see?
the areas associated with goal direction value were more actuve during difficult decisions than easy decisions
How are expected utility and surprise related?
evaluation involves comparison between expected and observed outcomes
-outcome evaluation is negative compared to expected utility
-expected utility or predicted reward is important - a pair of comfortable socks is better than nothing but it is less than what is hoped for
How is expetec utility ties to activity in the vta or dopaminergic midbrain area and is behaving in accordance with the temporal difference learning function?
-recordings n the vta during classical conditioning (which is different from habit learning because a reward is paired with a stimulus not with a repsonse or a stimulus associated response)
-learning is about the pairing of a conditional stimulus with a primary reward which is a squirt of juice
-learning is also paired with expected value and the vta represents this
In the neuronal recodings in vta of a monkey what was seen in actviity when there was no cs and the reward is there?
this is an unexpected reward the vta responds when the reward is presented
After training the monkey as learned that the cs predicts a reward at constant delay so what is the neuronal activity seen in the vta in the monkey when there is a cs and a reward occurs?
-the reward signal is seen when the cs is presented there is no reward signal to the reward
After training the monkey as learned that the cs predicts a reward at constant delay so what is the neuronal activity seen in the vta in the monkey when there is a cs and NO reward occurs?
if the cs is presented and the reward is withheld get response to cs but get a dip in dopaminergic activity when the reward should have occurred
What is learning?
update the representation valuation and action selection processes
What is reward learning in general driven by?
differences between expected and observed outcomes
-temporal dofference leanring function
What is the temporal difference learning function?
reward prediction error (temporal difference) = (actual reward at this time + (discount factor * predicted reward in the future) - predicted reward at this time
IF GET NO CS AND GET A REWARD = ACTUAL REWARD IS POSITIVE PREDICTED REWARD AT THIS TIME IS ZERO AND PREDCITED REWARD IN FUTURE IS ZERO SO GET POSITIVE TEMPORAL DIFFERENCE
IF SEE JUST THE CS - ACTUAL REWARD IS ZERO PREDICTED AWARD IN THE FUTURE IS POSIITVE AND PREDICTED REWARD AT THIS TIME IS ZERO SO POSITIVE OVERALL
IF SEE CS AND REWARD - ACTUAL REWARD IS POSITIVE AND PREDCITED REWARD AT THIS TIME IS POSITIVE NEGATIVE AND PRECITED REWARD IN FUTURE IS POSITIVE
IF SEE CS AND NO REWARD - PREDCITED REWARD IN FUTURE IS ZERO AND PREDCITED REWARD AT THIS TIME IS NEGATIVE AND ACTUAL REWRAD AT THIS TIME IS ZERO - GET NEGTATIVE VALUE
WHAT DISTRUBITED SYSTEM REFLETCS EXPECTED UTILITY?
DA NEURONS IN THE FRONTAL CORTEX THE MIDBRIAN AND BAAL GANGLIA
WHAT AREAS OF THE RIAN ARE ASSOCIATED WITH COMPUTING EXPECTED UTILITY WHICH IS RATHER HABIT LEARNING OR PAVLOVIAN BASED?
MEDIAL PFC AND MIDBRAIN
-LOSS AND GAIN FUNCTION
-TEMPORAL DISCOUNTING
WHAT AREAS OF THE BRAIN ARE AASSOCIATED WITH GOAL DIRECTED VALUATION?
LATERAL PFC
WHAT IS REWARD LEARNING DRIVEN BY AND WHAT IS THAT DRIVEN BY?
REWARD LEARNING ID DRIVEN BY PREDICTION ERROR WHCIH IS DRUVEN BY SURPRISE RELATIVE TO WHAT WAS PREDICTED Y THE VALUATION SYSTEM
When does risky decision making peak?
adolescence (the social capacity to engage in these behaviors also changes so there is a social component in addition to the behavioral)
What are some attributes associated with risky decision making?
-facilitates independence
-encourages exploration away from caregivers
-increased rates of sexual promiscuity
-high incidence of addiction
-reckless driving
What is the neural explanation for the risk behavior of adolescents?
the pfc continues to develop throughout adolesence
-this is in contrast to other brain regions which approach their adult state as measured by relative proportion of grey to white matter by late childhood
What higher congitive functions is the pfc involved in?
decision making
impulse control - can have go and no go stimulus to measure impulsivity in the lab
future planning
goal directed behavior
appreciation of future outcomes
If children have an even more less developed pfc that why do they not seek out the same risks in decision making as adolescents?
reward regions are hyper excoyable in adolescents
Initial models of adolecsent behavior psoted that teens engage in risky behavior because they do not have a fully capable pfc howevre this is actually not due to insuffiecnt knowledge or intellectual abilities?
in laboraory studies adolesents to no differ from adults in
-knowledge of the dange of risky behavior
-ability to perceive risks
-feelings of invulnerability
-logical reasoning abilities
-psychscoial maturity is declined in adolsecnts compared to adults but intelletucal ability is the same
What study conducted by Galvan et al showed that reward regions are hyper excitable in adolescents?
participants
children - 7-11
adolescents - 13-17
adults - 23-29
simple right and left perceptula decision
-difrenet pirate accessoires implicityly associated with different amounts pf reward
-some of the medial pfc and the NAcc have a u shaped developemntal function in repsonse to reward where it peaks in adolescents - this is in response to feedback about the value of the reward not the response to the cue
How was it shown in the galvan et al study that the size of the response is modulated ore strongly by the size of the reward in adolescents than in either children or adults?
the loss of aversion might be weaker because the gain function is so tuned up
- adolescents have a steeper gain function than adults
-large and medium rewards gave greater response enad small reward is negatuve response
-
What to reward responses to near misses in adults associated with?
the desire to continue in a gambling task
Does reward hypersensitivity in teens extend to near misses?
the slot machine paradigm includes two different kinds of loss trials
-near misses XXY and full misses XYZ
-adolescents show a stronger response to reward in the striatum than either adults or children
-reward sensitivity n adolescents is often observed under conditions where adults and children show no striatal activity
response to near misses are also much stringer in adolecsents
Why might responses to near misses being much stronger in adolescents be possibly adaptive?
-this is important for learning new skills or adapting to a new environment or contingencies
How does teen drivers riskiness of death change with the passenger in the car?
adult decreases
other teen increases the more teens you add
What is the stoplight driving game?
stoplight is 3 seconds and miss stoplight and can get no delay or get into a crash and get a 6 second delay
What was seen in teens when it came to the eprcent of risky decision. make and the presence of a friend?
presence of friend led to riskies decisions and worse outcomes for teens than in young adults and adults
-friend was in the control room
ACtivity in what region of the brain increases over development?
the lateral pfc increases over development irrespectuve of peers
-adolescents are less likely to activate this region at all during the task compared to young adults or adults
In reward circuitry was there a change in adolecents in teh presence of peers?
yes for adolecents only the activity was greater in the presence of peers than when alone
What are the effects of daily stress in the teenage brain?
teens and adults are equally stressed however better self control in adults whereas stress has a greater effect on teens self control abilities
-unde high stress adolescents are less likely too undergo successful inhibition and are more impulsive and the successful inhibition data is 1-impulsivity datar
What is the extent of the engagement of the lateral pfc in teens versus adults under stress?
less engages in teens under stress for lataeral pfc than adults whcih is more engaged under stress
What is the relevance for juvenile justice?
pfc cortex debelopment - teends are more vulnerbale to negatuve influences like peer pressure
-limbic circuitry development - heightened capapcity for change
What is the relevance for mental health?
many mental health conditions have onset in adolesence and can be due to imbalance between subcortical and midline regions and the lateral pfc
What does the neurobio of decision making involve?
-both reward driven pavlovian habit learning and goal directed valuation systems
What does the relatuve timing of pavloviam and habit learning and goal directed habit systems inlfuence?
risky decision in making teens
-reward systems is u shaped function (peaks in adolescence) and pfc is linear with developemnt with age
