PSY2002 W6 Multisensory Integration 2 (L)

Question 1

What are some benefits of normative models ?

Answer 1

Normative models can give us “optimal” solutions to problems.
A normative model applied to multisensory integration allows us to calculate the optimal way to integrate information

Question 2

Is sensory information integrated optimally?

Sceintific evidence

Answer 2

Need to know uncertainty for optimal integration: Can be hard to estimate and Easier in sensory perception than cognitive reasoning.

Calculations can be intractable or take a long time: Heuristics are suboptimal, but fast and ‘Good enough’ solutions often satisfactory. e.g.: heuristics in reasoning (availability heuristic.

Question 3

What is Sensory conflict?

Answer 3

Different senses might provide conflicting information about a sensory stimulus. This conflict needs to be resolved.

Question 4

What is normative vs process models?

Answer 4

Example: Find shortest path between Endcliffe and Cathedral Court
Normative model: Compare length of ALL routes, pick shortest (A)

Process model: Choose between small number of salient options

Question 5

What are examples of sensory conflicts?

Answer 5

Sitting in the train, train outside the window starts moving, “are we moving?” Proprioception: there is no movement. Yet we feel movement because visually movement would look the same.

Our perception of the sound changes depending on what we are reading – sensory conflict

Question 6

What is the McGurk Effect?

Answer 6

person say dadada if you are look at the person you will hear dadada but when you are looking at the person you will hear vahvahvah.

Question 7

What is sensory uncertainty due to?

Answer 7

Perceptual limits, e.g. visual resolution determined by spacing of photoreceptors in the fovea

Neural noise, e.g. synaptic noise

Cognitive resource limits, e.g. attention

Question 8

How to test conflicts between vision and touch?

Answer 8

Task: Judge size of object by vision and touch.
Setup: Looks small (vision), Feels big (touch).
Vision dominates perceived object size: visual capture

Question 9

Is there are strict sensory hierarchy?

Answer 9

No. Audition can dominate vision.

Experiment: Report number of visual flashes
Setup: Auditory beeps (1-4) played during flashes, participants were told to ignore the beeps and only count the flashes.

Results: Number of auditory beeps determines reported number of visual flashes

Question 10

What is modality precision hypothesis?

Answer 10

Modality with highest precision (lowest uncertainty) chosen depending on task:

Spatial task → Vision
Temporal task → Audition

Question 11

What if precision of sensory modality changes? - task/method

Ernst & Banks 2002

Answer 11

Task: Judge heigh of bar
Setup: discrepancy between vsiual and haptic inputs, cna change visual and tactile uncertainty independently.
Vision: virtual reality setup: change height of bar, modify uncertainty by adding visual noise.
Haptics: force feedback device, change height of bar.
Vision and haptic input can be in conflict!
Method: present vritual bar with sensory conflict (haptic=/=visual) compare against bar without. determine point of subjective equivalence. Manipulate sensory uncertainty of visual feedback

Question 12

What if precision of sensory modality changes? - results

Ernst & Banks 2002

Answer 12

No added visual noise: Perception of bar height biased towards visual input

Increased visual noise: Perception of bar height now determined by both visual and haptic inputs.

High visual noise: Perception of bar height now determined by haptic inputs

Question 13

What is a normative model?

Answer 13

how a problem should be solved (‘optimal’ solution). Based on theory. Can establish bounds (“this is the best we could possibly do”)

Question 14

What is a process model?

Answer 14

how a problem is actually solved, based on data.

Question 15

What is the question and answer of a normatic model?

Answer 15

Q: How should we solve the problem of sensory integration?
A: Pick integration method that minimizes sensory uncertainty

Question 16

Explain the Haptic => Integrated signal & Vision => Integrated signal

Answer 16

Haptic [haptic uncertainty]
Vision[visual uncertainty]

=> Integrated signal [smallest possible combined uncerainty]

Question 17

What are the different integrating probabilities?

Answer 17

1. High variance wide distribution high sensory uncertainty
2. Narrow distribution, low variance with low sensory uncertainty
3. Sensory conflict
4. Combined estimate has lower uncertainty (smaller variance than either estimate alone.
5. High uncertainty (haptic noise is high) Low uncertainty (visual feedback – narrow distribution) – sensory conflict
6. Combined estimate is biased towards the visual estimate as this has lower uncertainty than the haptic estimate.

Question 18

What rules does human perfomance follow?

Answer 18

Optimal sensory integration rules. Humans act in accordance with this model and therefore integrate sensory information optimally.

Question 19

Are probabilities encoded in the brain?

Answer 19

Uncertainty needs to be represented in some way: confidence signal? Representation of full probability distribution?

Little direct electrophysiological evidence, but several plausible shemes proposed:
Mean + Variance -|-
Full distribution |||
Samples ..::.: .

Question 20

What is a correspondence problem?

Answer 20

External space [vision(dog) = audition(dog)] (One or 2 dogs)

External space [vision(dog)=/=audition(cat)] one dog and a cat or a dog who sounds like a cat?

Question 21

What does a normative model predict?

Answer 21

A normative model predicts the optimal solution for the sensory integration problem

Question 22

What model does humans act in accordance with?

Answer 22

Normative model

Take sensory uncertainty into account when integrating information
Minimize overall sensory uncertainty