Brainscape's Knowledge GenomeTM

Browse over 1 million classes created by top students, professors, publishers, and experts.


See full index

PSYC - year 3 Perception > lecture 4 - multisensory integration > Flashcards

lecture 4 - multisensory integration Flashcards

1
Q

definitions
-multisensory
-cross-modal
-integration

A

Multisensory: More than one modality is used in perception
Cross-modal: Interactions between different

modalities → one sense affects perceptions provided by a different sense

Integration: Merging information from different modalities into a unified percept

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

how is perception multisensory

A

Perception is multisensory in natural interactions with the environment

  • Information from different senses can either be complementary or redundant/ overlapping
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

multisensory information ________ reliability of the percept and provides a more complete representation of the world

A

Multisensory information increases
reliability of the percept and provides a more complete representation of the world
(+ increases resistance to interference)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

different modalities can provide convergent information about…
-what does this mean for the CNS

A

Different modalities can provide convergent information about the same external event/properties

  • CNS has to disentangle cases where stimulation of different senses is unrelated and where it is related
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

simple heuristics for integration
(3) (how do we know information from 2 senses belong together)

A

1) Temporal correlation:
Stimulation of different modalities occurs at, (roughly) the same time.

2) Spatial congruency:
Stimuli in the different senses come from approximately the same location

3) Inverse effectiveness:
Reduced benefit of multisensory integration the stronger the unimodal signal of a cross-modal
cue is → Multisensory response is stronger when one stimulus by itself is quite weak

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

what is inverse effectiveness

A

Degree to which a multisensory response exceeds the response of the most effective modality specific
stimulus component declines as the effectiveness of the modality-specific stimulus component increases

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

inverse effectiveness on a neural level
-what is the point of this

A

example of one neuron and its reaction to a weak stimulus

Multi-modal neurons in Superior Colliculus (SC) (relevant for rapid orienting of attention):

-when neuron reacts to JUST visual input there is a weak respons, same as when it reacts to auditory input and has a weak response

  • Spikes produced by combination of visual and auditory event (5) is larger than the individual
    neural spikes in response to visual (1) and auditory stimuli (2)
  • Superadditivity of spike counts:
    Multisensory response is greater than the sum of uni-sensory responses
  • Sum usually only larger for weak inputs (near threshold) → aids detection of weak stimuli
    → speeds up behavioural responses
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

inverse effectiveness
-photos of the dog , 1,2,3

A

1) dog far away from window, weak auditory stimuli, and periphery
2)

Superadditivity: Both cues are weak – response exceeds the sum of the separate inputs

  1. Additivity: As cues become stronger unisensory responses become stronger → integrated response is not different from the sum of the responses to each component
  2. Subadditivity: Combined input is smaller than the sum of the two uni-sensory inputs (but still exceeds the largest single input response)
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

neural mechanisms
subcortical areas : the superior colliculus
-where is it located
-how many multinsensory neurons
-do neurons overlap

A

Located in the mid-brain –
important for orienting
behaviour and fast motor
reactions

  • High(est) proportion of
    multisensory neurons
    (extensively studied)
  • Neurons show overlapping
    spatial maps for visual, auditory
    and somatosensory modalities
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

neural mechanisms
-cortical areas
-how many multisensory neurons and where
- what did the studies of this area focus on

A
  • Multisensory neurons are
    found in most areas – often in combination with unimodal neurons

-Even in areas previously considered modality specific (e.g., neurons in visual cortex respond to tactile cues, and neurons in primary auditory cortex are activated by
visual lip movements

Studies in primates primarily focussed on posterior parietal cortex (converging information from visual, vestibular, tactile
and auditory system

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

cross modal integration
-how is input from two senses combined perceptually?

A

Different modalities are combined to yield the best estimate of the external properties

The modality that provides more reliable information is given more weight (greater reduction in uncertainty)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

cross modal integration examples

A

→ e.g., vision strongly influences auditory localisation
(ventriloquist effect) – vision is spatially more accurate

→ audition can dominate vision in temporal properties,
e.g., auditory flutter drives perception of visual flicker (the two flash illusion

→ Modality appropriateness hypothesis

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

what is semantic congruency

A

Semantic congruency (consistent
meaning of two stimuli) Semantic congruency refers to the degree of similarity or compatibility between two pieces of information, such as words or stimuli, based on their meaning or semantic content.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

semantic congruency _______ multinsensory stimulus integration and…

A

Semantic congruency (consistent
meaning of two stimuli) strengthens
multisensory stimulus integration and corresponding behavioural
performance

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

semantic congruency and its affect on speed of participants responses

A

Semantic congruency of visual and
auditory stimuli affects the speed of
participants responses
→ faster target detections when
visual stimulus is accompanied by a
semantically congruent sound

two visuals of- teleophone and frog
two sounds of teleophone and frog
-did a combination- press a button whenever you see or hear the telephone
-if you only see the teleophone (visual stimulus) you have a reaction time of jus under 5 ms, and if you only hear it there is similar reaction time
-incogruent : see the teleophone and hear the frogor vice versa (slow reaction time)
-

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

crossmodal illusions
-what is the parchment-skin Illusion

A

-complementary information improves the reliability of our perception
-sound modifies tactile sensations
-enhanced high frequency feedback makes the skin feel drier
-temporal coincidence required

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

incongruent information can result in unexpected precepts due to _______ _______

A

sensory interactions eg audio-visual illusion of the MCGurk effect)
-you listen to auditory stimulus ‘ba’, have a visual stimulys/actor pronouncing ‘ga’
-usually the perception is a fusion response (da / tha)

18
Q

explain the parchment skin illusion in depth
exp

A

-relationship between auditory information and skin perception
-sound modifies tactile sensations

deprivation of one modality can modify the development and integration of remaining modalities

-enhanced high frequency feedback makes the skin feel drier
-temporal coincidence required

19
Q

parchment skin illusion in blind people

A

Less susceptible to the illusion – ability to ignore irrelevant auditory input in the tactile task

20
Q

parchment skin illusion ,
vision, audition and touch
study on blind people
-what does this show

A

-study
-control
-10 early onset blind participants
-had to rate their tactile perception on scale -5 to +5

-if high frequencies accentuated (makes it feel rougher), works well in sighted participants

Robust illusion in sighted humans
* 7 of the early blind participants were not/ minimally susceptible to the illusion
* Only 3 of the early blind participants showed small effects in the expected direction

  • Multisensory perception might not be innate but is – at least to some extent - based on experiences during early
    development → following visual deprivation extensive cross-modal changes occur (re-organisation of perceptual system)
21
Q

Cross modal plasticity in the Cerebral cortex
study by hamilton et al 2000
-what does this case suggest

A

Case of blind woman (born blind) who lost ability to read Braille (code) following bilateral occipital lesions (usually processing vision) following stroke in primary visual cortex (processes vision area) but didn’t loose any other tactile sensitivity , just specific to reading braille

  • Occipital cortex involved in decoding spatial and tactile information for Braille reading -
  • Suggests that there may be a critical period of susceptibility for the recruitment of the occipital cortex for haptic information processing (in congenitally blind)
22
Q

Cross-modal Plasticity in the Cerebral Cortex
TMS – Study
Cohen et al., (1997),

A

TMS briefly disrupts the electric activation patterns of the neurons in the cortical area it is applied to. short bursts of magnetic stimulation applied to certain areas of the brain

  • Blind Braille readers and sighted participants (blindfolded) who had to identify embossed Roman letters
  • Occipital stimulation (visual area) disrupted Braille reading in blind participants but not tactile discrimination in sighted participants (but disrupted their visual performance)
  • Visual cortex recruited for somatosensory processing in early blind
23
Q

Cross-modal Plasticity after Sensory Deprivation mechanism
-1 Reorganisation in areas associated with the deprived modality

A

if you don’t have vision, the occipital cortex seems to reorganise process

Primary sensory areas are able to process information from remaining modalities

  • Sensory inputs shape the functional architecture of the brain
  • Reorganisation likely to be limited to early-onset (sensitive period- when brain is still elastic)
  • Caution:
    Difficulty to clearly distinct between
    primary brain areas and neighbouring areas (usually multi-modal) – small spatial resolution of TMS, PET and MRI it might not be clear where the reorganisation in primary areas occurs
24
Q

Cross-modal Plasticity after Sensory Deprivation mechanism
2. Reorganisation of multi-modal
areas in the cortex:

A

Behavioural compensation for missing modality is mediated by enhanced
recruitment of multi-modal areas

  • Reorganisation in multi-modal areas also for late-onset deprivations
  • Example: Enhanced recruitment of
    posterior STS (area of multi-modal
    integration) in deaf individuals when attending to moving visual displays
25
Q

perceptual distinction between one’s own body and the environment is critical for….

what does body ownership include

A

Perceptual distinction between one’s own body and the
environment is critical for our survival and part of human
conscious experience

  • Body Ownership includes feeling the skin stretching around
    joints and digits, feeling the coolness/ warmness on the skin,
    feeling the tension from muscles and tendons etc. (all combined
26
Q

is body ownership a single sense

A

Sense of Body Ownership is multisensory in nature (cannot be
reduced to a single modality) – not really a “sense” but a complex/multisensory perception

27
Q

what is the rubber hand illusion
- 3 way interaction between vision, touch & proprioception

A

-participant has one hand in front and one hand behind the occlude. put a rubber hand in place of the real hand so it looks as if participants hand is there
-eventually after touching the rubber hand with the same objects etc as the real hand behind the occluder, participant starts to thinkit’s their own hand

28
Q

how is the rubber hand illusion objectively quantified

A
  • Questionnaires and Rating scales
  • Proprioceptive Drift (reported hand
    location changes)
  • Physiological responses (skin
    conductance) in response to
    perceived threat to the rubber hand
29
Q

perceptual rules of body ownership
-temporal synchrony
-spatial rules

A
  1. Temporal synchrony: if visual and tactile stimulation are mismatched illusion disappears (~ 300-500 ms)
  2. Spatial rules: e.g., distance between real hand and rubber hand (peri-personal space as constraint), identical direction of
    visual and tactile strokes, matching
    orientation & postures of hands
30
Q

multinsensory integration is the key mechanism in perceiving _____ ______

A

body ownership

Perceptual Rules are similar to principles of multisensory integration

  • Sense of body ownership is governed by the same principles as multisensory perception
    → multisensory integration is the key mechanism in perceiving body ownership
31
Q

what is peripersonal space (spatial rules rubber hand illusion)
-when does the illusion strength decreases

A

Space immediately surrounding our bodies in which objects can be grasped and manipulated

Illusion strength
decreases at distances > ~30 cm (peripersonal space)

32
Q

Perceptual Rules of Body Ownership (RHI) cont
-Tactile Congruence Rule:
- Humanoid Shape Rule

A

Tactile Congruence Rule: Tools that touch real and rubber hand must be similar in texture and geometric features (subtle incongruences possible)

  1. Humanoid Shape Rule: Rubber hand must resemble a human hand in shape and structure (colour and material less critical) → e.g., ownership observed for realistic prosthetic hands or even images
    of human hands
33
Q

Congruent pattern of ______ _______ ________drive the perceptual phenomenon

A

Congruent pattern of multi-sensory signals drive the perceptual phenomenon!

34
Q

Multisensory integration of body signals- where does this happen

A

there are Cortex areas specifically dedicated to multisensory integration of body related
signals in peripersonal space

meta analysis : areas in ventral premotor cortex and intraparietal sulcus that respond with greater activation for combined/congruent visuotactile stimulation than for unimodal visual or haptic stimulation or
incongruent stimulation

Certain areas in your brain get super excited when you see and feel something at the same time, like a toy and a high-five together. They respond more strongly to this combined experience than to just seeing or feeling something alone, or when the two don’t match. Your brain loves it when things happen together

Recent studies on limb ownership:
* Consistent activation in area EBA
of the ventral stream (processes body postures etc)

35
Q

generally body ownership is associated with activation in ________________ areas in ______ and ______ lobes

A

Generally, body ownership is associated with activation in multisensory areas in frontal and parietal lobes

36
Q

multisnesory integration : neuroimaging
graphs of intraparietal and premotor areas
-results

A

graphs show response of these areas to stimulation of the rubber hand illusion
-people in scanner, real hand is occluded, rubber hand in front of them
Manipulation of temporal congruency (synchrony) and spatial congruency (hand orientation)

Largest activation when stimulation was both spatially and temporally congruent
* Activation in intraparietal cortex is additive
* Response in the pre-motor cortex is super additive

37
Q

entire body is perceived as ___ ____

A

Entire body is perceived as one’s own (not just the
stimulated parts) → requires multisensory perceptual binding

38
Q

does the RH illusion work on the rest of the bdoy

A

illusion adheres to the same
perceptual rules as the RH-illusion

  • Illusion seems to relate to similar
    activation of brain patters as the
    RH-Illusion (increased activity in
    ventral premotor cortex, intraparietal cortex and LOC)
  • Activation in ventral premotor
    cortex correlates with the strength
    of the illusion
39
Q

activation in _______ ______ cortex correlates with the strength of the illusion

A

Activation in ventral premotor
cortex correlates with the strength
of the illusion

40
Q

-individual differences in illusions

A

Degree of illusory experience varies
(+about 30% of population are immune to the induction of the RH-Illusion)

  • Factors influencing individual differences are largely unknown
  • Multi-sensory account predicts that immunity should relate to how visual, tactile and proprioceptive information is weighted by the brain
  • People that rely more on proprioceptive information (e.g., dancers, gymnasts etc.) maybe more resistant to the illusion
41
Q

potential clinical applications

A

Projection of ownership to advanced hand prostheses: Indication that synchronised
brushing of participant’s stump and the fingers of a prosthetic hand produces RH-Illusion in some (~ 30%) amputees

  • Projection of ownership to simulated bodies in VR: People can maintain ownership of a virtual
    hand as long as its movements are temporally and spatially congruent with movements of real hand (allow paralysed people to “own a virtual
    limb” in virtual and mixed reality applications)

PSYC - year 3 Perception (8 decks)

Brainscape helps you reach your goals faster, through stronger study habits.
© 2024 Bold Learning Solutions. Terms and Conditions