Multi-sensory integration Flashcards
Give an example of a function of multisensory integration (cat, dog example)
Detecting when danger is approaching. For a cat, the visual image of a dog may elicit 1 action potential in a neuron, the sound of a dog may also elicit one. Both together, however, may produce many in a peak response and elicit appropriate behaviour.
Give other generalised functions of multi-sensory integration
Perceptual binding; combining of these different percepts to represent a concept. Possibly also consciousness according to certain theories but this is more debated.
Give an example of one early brain area which integrates sensory information
The superior colliculus (tectum in other vertebrates) integrates sensory modalities quite early. It is the earliest stage for processing visual information and bypasses the LGN in a different route coming directly from the eyes.
How is SC different to many of the other areas associated with integration?
The other regions are typically in much more modern areas. This could suggest that association cortices were previously confined to the more ancient regions (SC); now mostly processed in these more modern areas. The tectum of smaller creatures takes up much more of the brain than in primates such as humans.
Describe some cortical regions for multi-sensory integration (5)
PRR (MIP): Visual/ auditory/ tactile information
LIP: Visual/ auditory
STS: Vocalisation/ audiovisual congruence
VIP: Visual/ auditory/ tactile/ vestibular
VLPFC: Audiovisual congruence/ vocalisation
Look at docs for map
What three senses are integration studies most focused on?
Visual, auditory and tactile information but there is obviously also gustatory and olfactory
Briefly describe the recent history of visual/ auditory integration history
It was known that there was multi-sensory integration among higher order areas. There was some evidence foud for direct connections between the primary areas however there is debate around this. There was activity found in the visual cortex following sound, however this was found not to be true but actually resulting from a reaction to the sound (e.g differences in vision following movement). More evidence has been found since, however, for the integration of auditory information absent from these reactions. Moreover, axonal tracing studies indicate that numerous direct connections exist between primary auditory and visual cortex, suggesting that auditory-visual cross-talk is present before the associative stage.
Apart from the general understanding of how we build our percept and possible roles it plays in consciousness, why else is multi-sensory integration research important?
Important for people with brain damage, PTSD, restoring functions, plasticity etc. Due to the role our senses play in these ailments and phenomena.
How is the relationship between primary areas and association areas different in primates and rodents? What implications does this have?
Primary areas in mice take up most of the brain, in humans these are more peripheral. Association areas have expanded a lot in our brains. It is likely that the mechanisms are conserved but we cannot make claims about individual association areas based on animal models.
How have certain ‘streams’ been quite conserved across rodents and primates?
There is still quite. a clear dorsal and ventral stream in rodents
What is the spatial relationship between unimodal and multisensory neurons in rodents?
There are higher concentrations of multisensory neurons around the borders of these primary areas, with more unimodal neurons inside them. Pink areas = (mouse) dorsal stream; blue = (mouse) ventral stream in notes.
What are the markers of multisensory neurons?
When two different sensory stimuli elicit a stronger response than the strongest individual stimuli. This is a typical effect called multi-sensory enhancement.
Response depression can also be see in that the presence of two sensory stimuli elicits weaker activation than the activation evoked by the weakest of the two sensory stimuli, this is not often observed.
(see docs)
How is the effect of response enhancement further divided into two effects
The effects on activation of auditory (A) and visual (V) stimuli are summed (A+V). If the presence of both stimuli is higher than the strongest individual signal but not higher than the summation then the effect is described as sub-additive. If the increase in activation is higher than this summation then it is described
Assignment
What is meant by the double flash illusion and what does it demonstrate?
A brief sequence of two sounds played during a single visual flash leads to the perception of two flashes; show that audition and vision have strong perceptual bonds
Are the auditory to visual or visual to auditory connections stronger?
Recent studies indicate that auditory to visual connections are much stronger than their reciprocals, and that they provide inputs to visual cortex that can modulate visually driven activity
Mouse auditory cortex encodes a wide variety of acoustic features, what are the most well studied?
Spectral content and temporal features such as modulations of frequency or intensity are the most well-studied, and
intensity variations occurring at sound onsets and offsets are particularly salient auditory features.
How might you measure how auditory information is mapped onto the visual cortex in mice?
The question of how sound frequency information would map onto visual cortex is a difficult one, because of the lack of perceptual and ethological data on the particular frequency cues that could potentially be associated with particular visual stimuli in mice
In contrast, temporal coincidence is known to be used for perceptually assigning auditory and visual stimuli to the same object and is implicated in the double flash and ventriloquist illusions. Detection of temporal coincidence involves determining when sounds begin and end, and therefore might implicate neurons that encode particular intensity envelope features such as onsets and offsets.
Also, covariations of the size of a visual input and sound intensity envelope are important for binding looming and
receding auditory-visual stimuli. This suggests that there could be preferential cross-talk between some intensity envelope features and visual information.
Describe the kind of features different neurons in the auditory cortices respond to
Envelope features such as onsets, offsets and sustained temporal dynamics are encoded in separate cells with further selectivity for different sound amplitudes. Some neurons respond only to high amplitude, ‘loud’ sound onsets, whereas others only respond to low amplitude, ‘quiet’ onsets, and neurons responding to offsets and sustained phases are also tuned to precise intensity ranges. Some neurons also encode combinations of these features.
Back to lecture : *
How can we mathematically compare the activations elicited by the two sensory modalities? (2)
Interactive index (or multisensory
enhancement index):
ii = ( ( AV - max(A, V) ) / ( max (A, V) ) ) x 100
Mean statistical contrast:
msc = sum([AVi - (Ai + Vi)]) / n
When the combined component is different to the individual components there is an effect.
What relationship is observed between unisensory stimuli and multisensory integration?
Weak unisensory stimuli => large multisensory integration
Strong unisensory stimuli => small multisensory integration
E.g For unisensory neurons providing information to an integration neuron; When the dog is far and there is some auditory and olfactory information, there may be a four fold increase in the multisensory neurons from the small activation present in the unisensory neurons. When the dog is right there, the activation in the unisensory neurons might be strong and there will be some enhancement however this might be around x2.
What name is given to this relationship between stimuli strength and multisensory activation?
Inversive effectiveness = enhancement is higher for weaker stimuli
What is observed from the receptive fields of multisensory neurons?
Multisensory neurons have a
similar receptive field in all
modalities; A sensory cue falling in the overlapping portion of the
receptive field usually leads to a
stronger multi sensory
enhancement
Why could this overlapping of fields be important in multisensory integration?
Important in orientating your body towards the given stimulus.
Is multisensory integration different from unisensory integration?
No, in multi-sensory neurons weakly effective visual and auditory stimuli (represented by the electronic traces) are integrated to produce multisensory enhancement. In this case, the enhanced combined response exceeds the sum of the component responses (see docs) and was therefore superadditive. It is important to note that, in principle and in practice, enhanced responses could be superadditive, additive or subadditive.
In unisensory visual neurons with overlapping visual fields, pairing the visual stimulus with another visual stimulus yields a subadditive interaction that fails to meet the criterion for enhancement. This is due to surrounding suppressing receptive fields. These representative samples are exemplary of the characteristic differences between the neural computations that underlie multisensory and unisensory integration.