Multimodal Communication Flashcards
traditional view of language
language is speech and text, and of the mind. gesture is communication, not language
multimodal view of language
F2F communication that involves the use of the body to convey meaning
what are gestures?
communicative hand movements that represent information about the world
- universal
- accepted as correct information when conflicting with speech
- children begin gesturing before speaking
types of gestures:
- iconic
- metaphoric
- beat
- deictic
- pantomimes
- emblems
- signs
iconic gestures
when something directly maps onto the thing it represents; requiring speech to be understood
metaphoric gestures
same as iconic, but metaphors are used to describe concepts
beat gestures
produced in the tempo of speech, may not have meaning
deictic gestures
indicating or pointing directly to something
pantomimes
whole-body representational gestures
emblems
describe conventional gestures with precise meaning, which may be culturally determined
signs
part of a fully conventionalised language system
obligatory presence of speech
beat gestures
iconic/metaphoric gestures
optional presence of speech
deictic gestures
emblems
obligatory absence of speech
pantomimes
signs
linguistic properties absent
pantomimes
beat gestures
iconic/metaphoric gestures
some linguistic properties present
deictic gestures
emblems
linguistic properties present
signs
not conventionalised
pantomimes
iconic/metaphoric gestures
partly conventionalised
beat gestures
deictic gestures
fully conventionalised
emblems
signs
global and synthetic semiosis
iconic/metaphoric gestures
beat gestures
deictic gestures
global and analytic semiosis
pantomimes
segmented and synthetic semiosis
emblems
segmented and analytic semiosis
signs
who do people gesture for?
themselves and for the benefit of others
- blind children gesture as much as sighted children
- fewer gestures are produced when the listener cannot see them, but they still occur
why do people gesture?
- helps them think better
- improves problem solving and mental rotation
what happens when people are unable to gesture?
results in lexical access problems, showing it helps people speak better
what can single gestures convey?
information about:
- object
- location
- path
- manner of movement
gesture and linguistic relativity
- Aymara speakers see the past in front and the future is behind
- Guugu Yimithirr speakers use absolute cardinal direction which results in an incredible sense of geographic orientation
what is iconicity?
the meaningful relationship between human experience and linguistic form
sign languages use iconicity much ____ than spoken languages
more
as the hands are better than the voice at mapping meaning and form together
arbitrariness
no connection between word form and meaning
this has dominated approaches to language studies
signers’ RTs can be affected by:
- age-of-acquisition
- imageability
- concreteness
- phonological complexity
- group
how can iconicity affect language processing?
provides faster automatic access to meaning, for both production and comprehension, e.g., Bouba-Kiki effect
what is there universal sensitivity to?
iconic mappings, which can facilitate processing:
- bouba-kiki has cross-linguistic reliability
- nonsense words with more continuants (m/b/) are recognised faster in a curvy frame vs stops in a spiky frame
what is iconicity used alongside?
prosody, as there is speedup on fast verbs and slowdown for slow verbs
- also seen in pitch and object size
why is iconicity critical during spoken language acquisition?
children learn to map words onto meaning, and connections are clearer when words have large iconicity interaction
language learning is not entirely iconic, as __________ ________ and ___________ are also involved in acquisition of sign language
neighbourhood density
frequency
what does iconicity affect?
both language processing and language acquisition in signers
resilient properties of language
those that are relatively easy to learn and overdetermined
fragile properties of language
more difficult to learn, and development is sensitive to changes in input, learner, and learning environment
how many languages are there?
around 7000 spoken languages and a couple hundred signed languages
what are home signs?
a gestural communication system created by deaf children in hearing families, as they do not have any sign language input
resilient features of home signs:
- gestures form a lexicon
- lexical items are composed of parts to form a morphological system
- lexical items combine to form structured syntactic systems
- lexical markers can modulate sentence meaning
- various pragmatic functions
evidence of home signs:
birth of nicaraguan sign language in the 1970s when deaf children created a new pidgin language by exchanging idiosyncratic home signs
