Embodiment - Language 5-6 Flashcards
Learning outcomes
To understand the experiential simulations approach to language comprehension.
To examine evidence that perceptual processes are involved in comprehension.
To examine evidence that motor processes are involved in language comprehension.
To understand individual differences in
What is the Traditional view?
Meanings are represented as features, usually described in terms of other words.
Much of the learning of words consists in relating them to objects and actions, which have manifestations in the real world, and are accessible through perception and action.
However, meanings need to be grounded in the physical, social and emotional world.
– Symbol grounding problem (Harnad, 1990).
– Chinese dictionary example.
What is the Embodied view?
That linguistic meaning is grounded in bodily activity, such as the brain‘s systems for representing perception and action.
So in summary, the experiential simulations view of comprehension holds that comprehenders mentally simulate the described objects, states, and events during comprehension
What is an example of the Embodied view?
For example, in the case of a sentence such as, “You take the dog for a walk”, the comprehender might form a visual representation or mental image of the dog, a somatosensory representation of the dog tugging on the lead, and the motor representation of taking the dog for a walk. This is because from your experience of walking a dog, you learn to associate these words with particular bodily experiences.
Which perceptual features are represented?
Visual orientation (Stanfield & Zwaan, 2001)
Object shape (Zwaan, Stanfield, & Yaxley, 2002)
Visibility (Yaxley & Zwaan, 2007)
*See Zwaan and Pecher (2012) on MOODLE, for an overview
How do people mentally represent:
“The pencil is in the cup”? - Traditional
Stanfied & Zwaan (2001)
Traditional view (amodal symbols):
[IN[PENCIL,CUP]] - traditional assumption had been that the human mind manipulates abstract, arbitrary, and amodal symbols and that this combination of manipulation and symbolic representations constituted cognition. That is, language was represented in terms of symbols and rules – so a bit like a computer might represent something.
How do people mentally represent:
“The pencil is in the cup”? - Embodied
Participants should mentally represent the orientation of the pencil, based on their experiences.
Visual Orintation has been studied by?
(Stanfield & Zwaan, 2001) Sentence-picture verification task:
In their study, they used a sentence-picture verification task. In this task, participants read a sentence, such as “John put the pencil in the cup” and were then shown a picture. Their task was to indicate whether the object depicted in the picture had been mentioned in the sentence.
Half of the time the picture matched the orientation implied by the sentence
Object shape has been studied?
(Zwaan, Stanfield, & Yaxley, 2002)
Zwaan et al. conducted a further study examining object shape. This was also a sentence-picture verification task. So in this task, participants would read a sentence such as “Mary saw the egg in the eggbox”. If they have created a mental simulation of an egg in an eggbox while reading this sentence, then their mental image should be of an egg still in its shell. After reading the sentence, they were then shown a picture that would either match…or mismatch with this mental representation.
The task was to indicate whether the item represented by the picture had been mentioned in the previous sentence.
How has Visibility been studied? (Yaxley & Zwaan, 2007)
“Through the clean goggles, the skier could easily identify the moose.Through the fogged goggles, the skier could hardly identify the moose.”
comprehenders activate perceptual information about described objects, such as their shape and orientation. However, the activation of perceptual aspects of text is not limited to the attributes of described objects. The environment in which the objects appear can also affect relevant perceptual information.
Yaxley and Zwaan investigated this issue in a further experiment. In this study, participants performed a sentence–picture verification task in which they read sentences about a person viewing an object (e.g., a moose) in different conditions of visibility (e.g., clean vs. fogged goggles). Participants then verified whether a subsequently pictured object was mentioned in the previous sentence. The time that participants took to do this was shorter when the resolution of the pictured object and the resolution implied by the sentence matched than when they did not. These results suggest that the degree of visibility implied in linguistic context can influence object recognition. These data suggest that readers mentally simulate the visibility of objects during language comprehension. Thus, the simulation of linguistic descriptions is not limited to the activation of intrinsic object properties (e.g., object shape), but also invokes the perceptibility of objects that is implied by the environmental context.
Eye-tracking (Wassenburg & Zwaan, 2010)
Phase 1: Word-picture verification task
Toothbrush - 700ms - 200ms
Does the picture match the word?
Phase 2: Filler task (mental rotation task – 15-20 mins)
Eye-tracking experiment
Eye Tracking results
Phase 3 was then an eye-tracking experiment, in which participants’ eye movements were monitored when they read sentences which mentioned the objects that were used in Phase 1 (for example, a toothbrush). Specifically, participants read sentences in which the implied orientation of the object would either ‘match’ or ‘mismatch’ with the orientation of the object as seen in Phase 1. For example, if participants had seen a picture of a vertical toothbrush in Phase 1, then the sentence “Aunt Karin finally found the toothbrush in the cup” would match in orientation, whereas the sentence in which she found the toothbrush “in the sink” would be a mismatch. In contrast, if they had seen a picture of a horizontal toothbrush, the sentence containing the phrase ‘in the sink’ would be a match, and ‘in the cup’ would be a mismatch.
From the graph on the slide showing reading times, you can see that reading times in the mismatch condition are significantly longer than reading times in the match condition in the critical region of text that indicated the orientation of the target object (e.g., in the sink), whereas reading times in other regions do not differ between conditions.
This suggests that previous exposure to a picture of an object in a particular orientation affected reading times for phrases that described that particular object. These findings support and extend previous results that I have discussed by showing that not only does language affect the processing of visual information, but visual memory also influences language processing.
ERPs (Coppens, Gootjes, & Zwaan, 2012)
Three studies (presented as unrelated experiments):
Expt 1: Word-picture verification task
Ironing board
Does the picture match the word?
Expt 2: Emotional word stroop task (15 mins)
Coppens et al. then conducted a subsequent study, in which they examined object shape rather than object orientation, and used ERPs rather than eye-tracking. The design of the study was very similar to the one that we have just looked at – there were again three separate parts. In the first part, participants were again given a word-picture verification task, where they would be presented with a word such as ‘ironing board’, and then presented with a picture of an ironing board that could be either upright, or folded up in shape. In this study, the second task (or filler task) was an emotional word Stroop task, again, to try and draw participants’ attention away from the goals of the study.
