body schema and multisensory integration Flashcards
moravec’s paradox
reasoning (high-level cognitive) is relatively easy to program computers to do
e.g. can give them all possible scenarios in a game of chess - other skills like walking they cannot sense all the unknown variables
perception (low-level) is difficult to program
e.g. can’t walk well as they don’t have a body schema
define body schema
representation of the positions of body parts in space → updated during body movement
mostly non-conscious
define body image
not same as body schema
how you imagine your body, how you feel about your body
mostly visual
can have affective response to it
characteristics of body schema (7)
modular
different body parts processed separately in different brain regions
spatially coded
representation of position of each body part in external space
updated with movement
automated and continuous tracking of body posture
adaptable
changes when the body changes, long-term, gradually over time, tools
interpersonal
other’s actions are represented within the same body schema
understanding of other peoples action in relation to own
mirror neurons
coherent
spatial continuity
resolves perceptual conflicts (source of illusions)
don’t want to think there are multiple possible realities and so combines them to make one most logical reality
pinocchio illusion: muscle in arm is stimulated so your arm feels like its moving away from you
so when holding your nose it feels very far away
supramodal integration
transcends sensory modality
combines input from multiple sensory sources:
vision, sound, touch, and proprioception
peripersonal space
space immediately surrounding out bodies
special awareness of this space
objects here can be grasped and manipulated immediately
it isn’t fixed - can be extended
e.g. holding a stick so you can push things from a distance - becomes an extension of your arm
peripersonal space in brain - study in monkeys
have monkey by a table - baseline visual response to things on that table
as the arm is extended, the area around it will have heightened responses to it
* more response in brain to things in that areas than beyond it
this occurs when they are holding a tool too - further extends the area of peripersonal space
disorder of body image
autotopagnosia
body schema not affected as they can still perform actions with the unconscious awareness of where their arm is for movement - action execution is generally preserved
inability to locate body parts
loss of spatial unity of body
can name body parts but not relative order
e.g. know what a head is but couldn’t tell you it is next to the neck
disorder of body schema
ideomotor apraxia
inability to execute actions - especially related to tools
e.g. knows what a key is but cannot mimic the action of unlocking a door
body image is generally preserved
disorder of body schema and image
alice in wonderland syndrome
distorted size perception
microsomatognosia = body parts appear smaller
macrosomatognosia = body parts appear larger
- can affect how they move → wont jump in a room as they think they will hit the ceiling
- it is not linear - can walk into a room and then think your head will touch the ceiling even though you just walked through the door
temporal order judgement task
stimulate hands one then the other - random order
participant responds which hand was stimulated first
sometimes stimulated at same time - so they guess left or right
2 conditions - hands crossed or hands normal
results
more uncertain with arms crossed - give wrong answer more often especially with smaller gap between each stimulation (more ambiguous)
shows body schema is updated with movement and effects perception
development of body schema study
baby with uncrossed feet vs crossed
squeeze one foot and see which foot they look at
results
4 months old - no effect of perception
less accurate with age - as perception causes conflict
body schema develops around 5-6 months
multisensory integration
touch + vision + audition = multisensory integration → for coherent perception
e.g. know that the cup you see and the one can feel is the same object - coherence
vision = eye-centred/retinal = location of visual stimulus on the retina
audition = head-centred = location of sound source with respect to ears
touch = body-centred = location of tactile stimulus on skin
* need to combine all of these frames of references into a single map of the world
frame of reference issues
e.g. knowing “left” to one person is “right” to another
constantly updating frame of reference to understand world
coordinate transformations
e.g. looking at a dog as it moves
if you look with eyes and don’t move head, the perception from both is different
e.g. head is pointing forwards but eyes are pointing left
need to know about alignment to combine these to know true direction of the dog
alignment of eye-to-head (orientation of eyes) and head-to-body (orientation of head)
convert between these using knowledge of position and orientation of body parts (body schema)
cross-modal integration - lab study
method
* participants hold a box with two fingers
* given tactile stimulation (buzz) and respond which finger was stimulated (up or down)
* measured reaction time
* visual stimulation (up or down) also presented on same or different hand
* not relevant to the task, but interferes with performance
* can be congruent or incongruent
congruency effect = reaction time for incongruent minus reaction time for congruent response
in = 700ms
con = 638ms
congruency effect = 62ms
results
found congruency effect for both hands
evidence for cross-modal integration
this effect is smaller but still there when on the other hand
cross-modal integration with arm crossing - lab study
change mapping of body schema by crossing arms
effect of visual distracter moves with the hand (e.g. still tactile on right hand)
body schema mediates integration between vision and tactile stimulation
almost no congruency effect with visual on non-tactile hand → same hand = 67, other hand = 3
shows cross-modal interactions are mediated by body schema
cross-modal integration with tool use crossing - lab study
incorporation of the tool into body schema during use - mediate multisensory integration/localisation
hold a stick in each hand with the lights on the end of the sticks, tactile stimulation is still to fingers
cross-modal congruency effects apply during tool use
- uncrossed condition → same = 74, other = 54
- crossed condition = no crossing of arms → only tools are crossed → same = 79, other = 37
same delay effects seen as if arms were crossed → shows integration into body schema of tools
neural mechanisms with body schema
visuotactile interactions - same neurons that respond to both visual and tactile stimuli in many cortical and some subcortical areas
sensory conflicts - why study it
finding limits of perception to discover the mechanisms behind it
mind as a black box - can find out rules within by trying different inputs and seeing the outputs
brain tries to make one coherent reality - resolves conflicts sometimes incorrectly
sensory conflict examples - visual and proprioception (1)
the train next to you moves and for a moment you feel you’re moving too
visually, movement would look the same even without proprioception - then the conflict is resolved quite quickly
sensory conflicts examples - visual and sounds (2)
videos where you hear whatever you read at the time
McGurk effect
hear a different sound when you watch someone mouth the word vs not
* saying “ba” but mouthing “da” so with eyes closed you hear “ba” (auditory stimuli) and with eyes open you hear “da” (visual stimuli)
* visual stimulation changes what you hear
reasons for sensory uncertainty (3)
- perceptual limits - e.g. visual resolution depends on photoreceptors on fovea
- neural noise - none of our senses are fool proof, never 100% clear what is being transmitted - synaptic noise = random noise going on at same time which interrupts the signal
- cognitive resource limits - by attending to something, the signal is stronger and so could take over what we experience - e.g. told to attend to other object during McGurk effect experiment, therefore the effect was reduced as they didn’t look at the face - attention as an amplification of signal
multisensory integration and attention - what effects what
attention can affect how multisensory signals are integrated (top-down influence)
* e.g. McGurk effect
multisensory signals can also influence attention
* e.g. have to find the vertical or horizontal bar amongst others
* one condition - auditory signal played at same time a horizontal or vertical bar was presented
* improved reaction speed by seconds (massive difference)
cyclical → multisensory integration is both pre-attentive and depends on attention
theories of sensory conflicts (3 - list)
- visual dominance
- modality precision
- weighting based on uncertainty
sensory conflict - visual dominance lab study
task = judge size of an object by vision and touch (object on a blanket with hand underneath to feel it)
object looks small but feels big (altered the look of an object - viewed through lens making it look smaller
results = visual dominance - object size is determined by vision more than by touch
idea that visual cortex takes up a lot of space relative to other senses - therefore assume dominance
modality precision theory of sensory conflicts
vision will sometimes dominate - but not on all tasks
* spatial task = visual dominance
* temporal task = auditory dominance
* hearing is more sensitive to time than vision
modality precision lab study
task = report number of visual flashes on the screen (very short flashes)
auditory beeps (1-4) played during flashes
results = interested on conditions when one flash was shown with multiple beeps, more flashes were reported as seen
this is a temporal task - counting things in time not in space
weighting based on uncertainty
vision will sometimes dominate - but not on all tasks
* spatial task = visual dominance
* temporal task = auditory dominance
hearing is more sensitive to time than vision
changing sensory uncertainty in one sense
do we dynamically adapt? or assume constant uncertainty for a given sense/task?
weighting based on uncertainty - lab study set up and method
task = judge the height of a bar
create discrepancy between visual and haptic input - can change visual and tactile uncertainty independently
setup
* wear goggles, in a “virtual reality” setup - changed what they see and changes the height of the bar
* add visual noise - makes it less clear what they see - causes conflict between vision and touch
method
manipulating size:
* show virtual bar with sensory conflict (is 5cm but looks 6cm)
* compare this against one without conflict (5.5cm) - which one is taller/shorter
* determine point of subjective equality (PSE) = point at which people cannot distinguish which object is taller - have to guess at this point
* then manipulate sensory uncertainty of visual feedback - add in visual noise
weighting based on uncertainty - lab study results
60mm = visual bar height
50mm = haptic bar height
no visual noise = PSE - 58mm
perception biased towards visual input
67% visual noise = PSE - 57mm
perception biased towards visual input - only slight change
100% visual noise = PSE - 55mm
perception both visual and haptic - between the two - decreased visual dominance with increased uncertainty
200% visual noise (yellow) = PSE - 52mm
perception biased towards haptics
normative vs process model of sensory conflicts
normative model
how it should be solved - optimal solution
based on theory - not real life
can establish bounds - best we could do so how close do humans get to this standard
process model
how a problem is actually solved
based on data from real life
comparison of these two models to see how optimally humans act
LOOK AT END OF NOTION
about graphs with signals of different levels of uncertainty
high vs low uncertainty
two conflicting senses of equal uncertainty
two conflicting sense of unequal uncertainty
normative vs process model of sensory conflicts - results from study
humans compute very similarly to the optimum
therefore use these uncertainty ideas in real life
graph is very similar to the optimum/predicted graph
MLE
maximum likelihood estimation
normative model which allows us to calculate the optimal way to integrate information
* more weight on inputs with low sensory uncertainty
* uncertainty is reduced after integration
* this models how humans act
correspondence problems
issue of determining whether two signals come from the same source - common cause
use statistical properties of signals
looks for correlation over space and time
neural correlates and uncertainty
recent research has found some - more research is needed
