Multimodal integration Flashcards
True or False: It is important for an organism to be able to bring to gather information from the different sensor system
True
What is Multimodal integration?
Integration of information different sensory modalities
Multimodal integration is important for a wide variety of activities
An example is __________.
Motor Control
_________ behaviors response should be able to activated not only a single type of sensory stimulus but for example either the sound or the sight (or both) of a predator
Escape
__________ –> requires information from our vestibular system –> helps us to know which way is up
and
from the __________________ –> sensory in our joint tell us the positions of our legs
and
even helped from our ______________ –> helps us to know whether our heads are straight or learning
Posture
Somatosensory system
visual system
Multimodal integration is important for a wide variety of activities
Another example is __________.
Commiucation
____ greeting each other uses many modalities of sensory information.
When they meet they will
bark providing _____________
wag their tails providing ____________
sniff each other providing _________________
Dogs
Auditory information
Visual information
Olfactory information
__________ also use information of several modalities for communication
Although much of our communication is through
having speech –> ___________
replying on seeing people’s expression to know the emotional content of what they are saying –> _______________
(important to know if somebody is serious or just kidding)
Humans
Auditory
Visual
We also use visual information during communication by ________________.
reading people’s lips
What is the McGurk effect?
How to demonstrate the effect?
Illustrates our ability to read lips
To demonstrates the effect –> scientist took a videotape of someone’s face as they were saying GA GA GA
On a soundtrack of video –> they put the person’s voice saying BA BA BA
In this case, the visual information (GA) is in conflict with the auditory information (BA)
Most adults (98%) think they are hearing DA –> so-called _____________ where the D is a result of _______________.
In reality, you are hearing the sound BA while you seeing the lip movements GA
Most adults (98%) think they are hearing DA –> so-called Fused response where the D is a result of audio-visual illusion
In reality, you are hearing the sound BA while you seeing the lip movements GA
Neurons located in _____________ regions increase their firing rates when they receive stimuli from multiple modalties.
Multimodal integration
Such cells can detect _________________ as long as low intensity stimuli from two or more modalities are present at the same time and place.
very low intensity stimuli
Which species have shown a very good ability to locate objects in auditory space, in fact, localize prey even better if they combine auditory and visual information?
Owls
Where does the integration of information occur in the owl’s equivalent of the superior colliculus?
(hint: midbrain)
Optic tectum
What is the major problem in multimodal integration?
What two sensory systems are difficult to combine?
There is a major problem in how to combine
the visual
and
auditory information in the tectum.
Why are the visual and auditory systems hard to combine in the optic tectum?
The two sensory systems use different coordinate systems to map where objects are located
What kind of systems does the auditory system uses?
(Hint: ear is attached to the head)
Head-center coordinate system
True or False: The owl’s auditory tectum is a multimodal area.
False
It is not a multimodal area
True or False: Owl’s auditory tectum will see a sound straight ahead of us in the real world (at 0 elevations, 0 azimuths) will excite cells at the location in the auditory map where the zero is azimuth line crosses the zero isoelevation line.
True
What happens when we turn our heads?
(Hint: The information about IID and ITD that are used by the brain to localize sounds in space depending on the location of the sound with respect to our ears.)
The sound has not changed its location in the world, but it has changed its location with respect to our ears.
If we turn our head 10 degrees to the right wherein the auditory tectum will cell respond?
10 degree to the left
Now with our head turned, where in the real world would we have to place the stimulus (for example, speaker) in order to get cells at 0,0 points in the tectum to respond?
10 degrees to the right in the real world
Summary of the auditory system
So the locations on the map in the auditory tectum do not correspond to places in the real world but rather to __________________.
places with respect to where our head is
What system does the visual system use?
An eye-centered or retinotopic coordinate system
If we are looking straight ahead and our head is pointed straight ahead, wherein the tectum map will cell respond to an object located straight ahead?
(0 azimuth, 0 elevation)
At 0, 0
Shown as A in the figure
If we turn our heads but keep our eyes looking at the object where will cells respond to the object?
(side-eye)
At 0 elevation and 0 azimuth
Point A on the image
(eye does not move)
If we keep our head stationary but turn our eyes 10 degrees to the right where will cells respond?
-10, 0 –> 10 degree to the left
Shown as point B in the figure
(eye moves)
Summary of the visual system
So cells in retinotopic map do not care where objects are in real-world or where they are with respect to our heads but rather where an ________________________.
the object is with respect to where our eyes are looking
True or False: The different coordinate systems used by the visual and the auditory systems we have a problem combing information from the two systems into one coherent map.
True
If an object is directly in front of us and our eyes and head are facing straight ahead where will cells be activated in the two maps?
Visual A; Auditory A
If we turn our eyes 10 degrees to the right without moving our head where will cells be activated in the two maps?
Visual: B, Auditory: A
This creates a problem in our brain because our _________ is now telling us that the object is in a different place from where our _____________ is telling us the object is located.
Visual system
Auditory System
The _______________ is the brain location which has to solve this problem between the auditory and visual system in the brain.
(Hint: the midbrain)
Superior colliculus
(optic tectum)
Superior colliculus receive information about ______________________
and
needs to __________________.
Also receives information about ______ and ________ positions
Superior colliculus controls motor behavior called ________, which is the behavior of turning our eyes and head toward the location of auditory or visual or multimodal stimuli.
locations of both auditory and visual stimuli
integrate that information
eye and head
orienting
Recap of the problem
The visual maps in superior colliculus are in _______________ like visual maps in other brain areas.
retinotopic or eye-coordinate coordinates
In order to match this map, we would expect to find that the auditory map would also be in __________________ and the auditory receptive fields would have to move when we move our eyes in order to keep the two maps in the register.
Eye coordinate
A _________ between auditory and visual information occurs in the superior colliculus
mismatch
When the eyes move, the auditory receptive fields in the superior colliculus (do/do not) move, but they _____ about 1/2 as far as the visual receptive field move.
Therefore there remains a mismatch between auditory and visual information.
do
move
True or False: This mismatch is probably the reason that the orienting behavior controlled by the superior colliculus is so strong.
True
When we ______to a stimulus, we move our head and eyes toward it so that it is now straight ahead of both our head and our eyes.
orient
True or False: A normal adult owl with prism glasses shift everything it looks at 23 degrees to the right, the adult animal show changes in behavior adjusting to this situation and when the glasses are removed from an adult animal it can’t orient correctly to visual cues.
False:
Adult animals show no changes in behavior
and
when glasses are removed it can orient correctly to visual cues.
In young animals, plasticity can occur during the _________.
critical period
True or False: In the Critical period, When they a young, Over time, the auditory map in the optic tectum adjusts to being in register with the visual maps.
True
True or False: Young owl makes auditory mistakes 23 degrees to the right as well as visual mistakes. This is an example of the McGurk effect, which shows the primary role played by the visual system.
True
_____ shows that the visual system is primary even in animals with the good auditory system.
Bats
Bats use a system called ___________ to find their preys.
Echolocation
To use echolocation, the bate _______________ when it hunts and listens for the _________- of it noices bounchng off of it prey.
continually makes sounds
echos
