Lecture 4 - Decision making (area MT) Flashcards
WHY DECISION MAKING FOR AI
▪ Humans are extremely good at integrating evidence across multiple sources to derive decisions.
▪ Mirroring human-like decisions will lead to better integration and acceptance of AI systems
(explainable AI)
▪ Brain reading (communication with locked in patients) can benefit from better understanding of the underlying neural processes (compare to blindly applying algorithms to data).
PERCEPTUAL DECISION MAKING
▪ Perceptual decision making is studied in the labs to understand general decisions.
▪ A nice and simple recipe (probably too simple) for decision making could be described as:
- Representation of sensory evidence
- Integration of sensory evidence
- Selection of motor response
- Performance and error monitoring
REPRESENTATION OF SENSORY EVIDENCE
▪ The middle temporal visual area (MT or V5) is a region of extrastriate visual cortex.
▪ MT neurons have large receptive fields, which might
provide a basis for determining whether a number of
points (parts of an object) are moving together.
▪ MT neurons are responsive to the direction and speed of moving stimuli.
▪ MT neurons are topographically arranged.
▪ MT neurons are arranged in “direction hypercolumns” within MT cortex.
▪ Motion coherence is the ability to detect coherent
motion (things moving in the same direction).
- The sensitivity to motion coherence is assessed by measuring
the ratio of ‘signal’ to ‘noise’ dots required to determine the
coherent motion direction. The required ratio is called the
motion coherence threshold.
- The amount of evidence that you get (the amount of motion
coherence) is mirrored in the firing rate of MT.
- Neurons have a preferred direction. If the motion coherence
is in the null direction (not preferred), then the motion
coherence will not be detected and the neurons will stay silent.
- Area MT is causally involved in the perception of motion
in both macaques, and humans. Therefore a lesion in MT will result in complete failure to detect motion coherence and electrical stimulation of MT can change the perception of motion (generate a bias for a direction).
INTEGRATION OF SENSORY EVIDENCE
▪ A decision threshold (decision bound) is the
point where your brain has enough evidence to
draw a conclusion about a moving object (what
direction it is moving in). The cumulative
quantity of evidence required to trigger
commitment.
▪ When making a decision, MT has areas that
listen in to both of the directions and its
integrating evidence from both. When one area is
stronger than the other, you get more and more
certain over time, that the object is in fact moving in the suspected direction.
▪ The drift rate is how fast you accumulate evidence of what direction an object is moving in.
The strength of the evidence entering the decision process.11 | P A G E
o The evidence is not in a direct line because there is noise in the system. The bigger the
standard deviation around the slope, the more noise there is.
▪ Reaction time distributions can be modelled using sequential sampling models.
o The drift rate, starting point (to create starting bias), non-decision time, decision
threshold and standard deviation are parameters that can be changed in such a model.
o By running a model with different parameters, you can create a distribution of
reaction times.
o The drift rate scales with stimulus strength and the bounds are set to achieve a balance
between speed and accuracy demands. Depends on the situation: how sure or fast you
want to be.
Lateral Intraparietal Sulcus (LIP)
is likely responsible for the
decision making process. When a monkey was tested on deciding what
direction a dot was moving in, neurons in this area exhibited activity in
the delay period before saccadic
eye movements (quick
movement of the eyes towards a
new fixation point).
o LIP neurons have
saccade target fields
(encoding future
saccade directions).
o Activity in LIP relates
to the decision, even at
0% coherence. Slope
varies with evidence.
▪ LIP neurons for triggering a
saccade into a specific direction
will be excited by MT neurons that perceive movement into the
same direction. They will be inhibited by MT neurons that
perceive movement into the opposite direction. Therefore,
decisions are phrased as a competition of two opposing
alternatives.
o With time, and given a signal, the integral will develop
more and more clearly and a decision will be made (by
triggering a saccade into the respective direction), as the
firing rates scale with the evidence.
o This is not the exact truth, it is merely a model of the truth!
GENERAL MODEL FOR DECISION MAKING
This is a general model, not limited to dot-motion (MT) and saccade responses (LIP).
DECISIONS: INTERMEDIATE SUMMARY
▪ When the subject is supposed to react as fast as possible, an average threshold emerges.
▪ Activity in MT directly reflects stimulus properties (scaling with coherence and direction).
▪ Variations in MT activity correlate with the decision formed.
▪ Activity in LIP is increasing/decreasing during stimulus presentation. The slope of LIP
activation reflects the strength of integrated evidence
▪ LIP activity is maintained in the absence of the stimulus, up to the behavioural decision.
▪ The activity of LIP neurons does not only reflect stimulus properties, but also reflects the
behavioural decision.
▪ A straightforward model can predict the interaction of MT and LIP based on the integration of
activation differences.
SELECTION OF MOTOR RESPONSES
▪ The Frontal Eye Field (FEF) is responsible for
saccadic eye movements for the purpose of visual
field perception and awareness, as well as for
voluntary eye movement.
o It is mainly used for more complex or
abstract task scenarios (e.g. if you need to
turn your eyes to the left, while the visual
cue is on the right → anti-saccade task)
▪ Gold & Shadlen conducted an experiment in
which they stimulated the FEF and forced a
saccadic eye movement on the monkey to the
right. As the monkey saw the stimulus actually
appeared at the top, he voluntarily makes a saccade
to the target. This makes the influence of a nascent
decision on an imposed action measurable.
▪ FEF is constantly aware of what the other systems
(LIP / MT) are doing. This means that this
oculomotor system is informed about the evolving
decision, not just the final outcome.
▪ FEF is only for eye movements! It
does not have anything to do with
other parts of your body such as
hand movements. There is likely
another brain area doing the same
thing as FEF but then for hand
movements. The key to decisionmaking is that the right part of
your brain is updated on the
decision that needs to be made,
depending on the task.
▪ FEF can integrate the task
instruction into evidence and
already transform this into what
the appropriate response will
likely be (which leads to
separation). It can prepare the
right reference frame of 13 | P A G E
responses. Yet if FEF has no way to prepare itself (e.g. when the task instruction appears after
the onset of the stimulus), then there is no preparation effect and no separation.
o Decision and behavioural response share a common level of neural organization only
when associated with a specific predictable movement.
PERCEPTUAL DECISIONS IN THE HUMAN BRAIN
▪ Two criteria for an area integrating sensory evidence:
o Larger signal (larger
evidence) in clear
stimuli than the signal
given by the degraded
stimuli.
o Activation should
correlate with
difference in sensory
areas (FFA & PPA).
▪ Many areas show evidence accumulation,
dependent on response modality. And across many species, many similar brain areas and
effects can be found. This suggests the decision-making model is cross-species.
o Reach: medial intraparietal area (macaque), motor cortex (human). Saccades: LIP,
FEF.
