L16 - The Senses in Context

Question 1

How to integrate sound and vision?

Answer 1

• The binding problem: where diff part of brain that are responsible for certain aspects of vision e.g motion but when you see something = they all combine together across the senses

Question 2

How do we integrate things across space?

Answer 2

• To solve the cross-modal binding problem is to have one sense (vision) to dominate another
• When you see and hear, you can see in different places and hearing in different places
• Vision tells brain where things are and the rest follows e.g illusions (not always true)

Question 3

What are examples of vision dominating senses?

Answer 3

• Ventriloquism: sound coming from the person’s throat sounds like its coming from the puppet
• Pseudophone: swaps ears around: trumpet on one ear goes over the ear and listens for sound on other ear (changes time and intensity of sounds because of johns lectures) = hear things coming from the wrong direction but when eyes are open it does not matter about the cues to the auditory system as you know where fingers are clicked or if person is talking

Question 4

What is a study looking at ventriloquism aftereffect?

Answer 4

• Expose people where vision and hearing where slightly displaced physically and if it caused an effect
• IF vision dominates, the displacement causes the hearing to get distorted so there is no conflict = shows vision is dominant
  STUDY:
• Train people to the offset of speakers and a light which did not line up - show 2500 trials and show attenuation by identifying which sound was louder
• Before adaptation, measured accuracy of auditory adaptation by identifying a noise using head/nose position
• Then after the adaptation, they remeasured auditory localisation
• When no adaptation effect = pre-training and post-training is the same as there was no offset
• When there was training and offset appear shifted 8 degrees to the right because auditory system has adapted

Question 5

What is a study looking at vision and touch

Answer 5

• Traditional view = vision is always dominant
• Some argue that touch is a big part of space perception - might not be vision that dominates
• STUDY: how big things are perceived as
• Presented physical objects and put them above a cloth so you could not see hand but feel it through cloth instead and look at it = had to make tactile and visual and visuo-tactile judgements about its size BUT put a compressive lens on that makes it look smaller
• 3 conditions: only seen, only felt, and see+felt condition
• Measured perceived size using drawing: uses both systems, visual matching or tactile matching e.g array of objects and which one it was or which one it felt like
• When you just saw it with compressive lens = object looked smaller
• When you felt it = feels bigger
  When you look compressed and feeling at right width = perceive it as smaller regardless of whichever perceived measure you use = shows vision dominance

Question 6

What is the reverse ventriloquism study?

Answer 6

• Vision does not always dominate as exps tend to bias vision = can create conditions where it is not biased
• Have two types of intervals: standard (always centre) and probe (left/right/centre) and task is to identify which interval appeared more rightward
• Probe and standard can interchange
• Called two-interval forced choice = 50% of time you will say probe is left/right , when probe is obviously left/right it will be 0/100% but when you make the threshold small = makes perception harder
• Results = psychometric function = s-shaped graph, when probe is slightly off-centre = not as certain
• When you increase the size of the cues, the shifting is harder to notice = curve of slope is shallower = shallower the curve in individuals = less reliable/precise judgment

Question 7

How did Alais and Burr change the reverse ventriloquism?

Answer 7

• Changed the interval to the one that appeared more leftward AND whether it was a blob/click
• Unimodal condition - test vision OR hearing
• Click = small blobs causes steep slope = good at judging position, big blobs causes shallow curves

Question 8

What was the bimodal condition?

Answer 8

• Standard and probe have flash and click = need to identify where the click/flash and whole object is
• When it lines up = perceived position of audiovisual object
• Can look at psychometric function = looking at where the psychometric function is at 50/50 = called point of subjective equality (PSE) = probe position is neither left/right = when standard and probe are in the same position
• Have standard but click is on left and flash is on right = when they come together you ask where the audio-visual object was = IF vision dominant, you take the side of the vision and etc. (depends on strength of ventriloquist effect)
• PSE when things appear in the same place = shifted to the right = called normal ventriloquism where vision kind of dominates
• When you replace big blob with small blob = if ventriloquism reverses and moves from vision to hearing, where robe and standard are same = probe is on the left = psychometric shifts left and gets shallower
• When vision is on the right and small blob, curve shifts to right, when vision is on left, shifts to the left
• Medium blob = no ventriloquism occurs
• Because we can reverse ventriloquism shows vision is not always dominant
• Large blobs are better for hearing

Question 9

Space vs Time in integration (when is audio more don ex)

Answer 9

1) When you see a single flash + single click = perceive single flash
- When you see single flash + double-click = perceive double flash = shows audio is more dominant
2) When two points cross each other with no sound/click = looks like the objects are pass through each other BUT when there is a click = perceived to bounce away = shows vision is disturbed by hearing

Question 10

Does vision always dominate hearing?

Answer 10

• No: most reliable cue is given more weighting
• In space: visual cues are more reliable as light travels in straight lines, but sounds bounce whereas retina preserves location
• Time: auditory cues are more reliable as vision has poor temporal resolution, hearing has high temporal resolution
• Reliability/precision can be changed by various properties e.g size and contrast
• Domination only occurs when one cue is much more precise than the other

Question 11

What is the McGurk effect?

Answer 11

• Ba is repeated but the shape of mouth changes and it makes it sound like different words