Visual Speech Perception Flashcards
What is the difference between lipreading and speechreading?
Lipreading is analytically watching the lip movements to extract speech information - relies solely on visual cues from the speaker’s lip movements
Speechreading utilizes visual, auditory, gestural, and contextual cues to understand speech - involves gathering communication cues by scanning the upper and lower face (umbrella term that includes lipreading, contextual cues, auditory closure skills, etc.)
What are the limits of lipreading?
Only about 40% of sounds are visible on the lips (the ones that are performed at the front of the mouth)
Some complexities reduce visibility to only 20%
What are the complexities that limit lipreading?
Visibility of sounds (relies on cues coming from place of articulation, voiced/manner cues are invisible, vowels are less visible but louder)
Visemes and homophones (many sounds and words look alike on the face)
Co-articulation and stress effects (appearance of a word varies based on how they’re spoken)
Rapidity of speech (sequencing sounds occur faster than the eye can resolve them)
Speaker effects (mouth movements vary by talker)
What are visemes?
Sounds that look identical when produced
Examples: /b/ bat, /m/mat, /p/ pat
What are homophenes?
Words that look identical on the mouth when produced
Like the auditory concept of a homophones (sale/sail) , but similarities are visual rather than auditory
~56% of words are visually homophenous
Does the appearance of a viseme change in relation to the sounds surrounding it and the syntactic stress patterns?
Yes
Refers to coarticulation
Example: boot and beet
Will patterns of stress also impact how the viseme appears?
Yes
Whadja DO yesterday?
What did YOU do yesterday?
Can the human eye only register 9-10 visemes per second?
Yes
We speak at a rate of:
~15 phonemes/ second
4 to 5 syllables per second
150 to 250 words/ minute
Word boundaries are hard to identify at typical speaking rates
Lipreaders have little time to ponder what was said when one-word ends, and the next begins
Cannot take in as many sounds as there are being produced through speech
What are the different individual variations that can affect how speech visually looks?
Degree of mouth opening
Facial animation
Accents
What factors may be predictive of lipreading abilities?
Working memory
Visual word decoding
Lexical identification speed
Phonological processing
Verbal inference/auditory closure skills
Pre-lingual onset of hearing loss
*All relate to processing at the level of the auditory processing cortex - don’t need to memorize
What are lipreading skills assessments?
Performed using live voice
A clinically useful assessment: run speech tests as auditory only, followed by auditory/voiced to identify degree of improvement
Voiceless assessments supply a true measurement of LR skills
Face-to-face presentations
Distance 5 to 10 feet
Lighting: no shadows, viewing angle 0 to 45 degrees
What are some examples of lipreading skills assessments?
Utley lipreading test
Denver quick test
Craig lipreading test
Is there a test of speechreading?
Yes
WSU sentence test
Do word recognition tasks improve when assessments include audio and visual information?
Yes
Example: Audio only - 50%
Visual only - 20%
AV - 90%
Does auditory and visual information automatically combine to form one unified percept?
Yes
A ‘percept’ is a mental impression developed as a result of the process
We’re not exactly sure how it occurs but what we see influences what we hear
known as the McGurk Effect
Does the mcgurk effect prove that everyone lipreads?
Yes
It’s disturbing when AV cues are out of sync with each other
Visual cues improves intelligibility in noise
Facial cues make it easier to understand cognitively complex passages
fMRI mapping lights up the primary auditory cortex when inaudible speech with facial cues are viewed
*proves that everyone lipreads
Does auditory visual integration require a significant amount of cognitive processing?
Yes
What is the lexical neighborhood activation model (NAM) of integration?
Spoken words and visual representation of words activate a set of lexical candidates automatically
Sparce neighborhoods (word groups which contain few words that sound and/or look the same)
Dense neighborhoods (word groups which contain many words that sound and/or look the same)
Does processing speed slow down in dense lexical neighborhoods?
Yes
And it speeds up in sparse neighborhoods
Does automatic AV integration help shift dense neighborhoods to sparse neighborhoods?
Yes
Multiple auditory or visual possibilities exist for DENSE word populations
Possibilities reduce with integration of the two sensory inputs
This shifts the word to sparser lexical populations
Knowledge of lexicon further reduces word options (can you hand me a fort doesn’t make sense)
Context quickly narrows down the possible word options
Does speechreading improve AV integration by adding additional cues?
Yes
Speechreading is ‘reading’ the visual clues of a spoken message
It still involves viewing movements of the lips, the tongue, the lower jaw
The eyes, the eyebrows provide prosodic cues
Visual closure skills are assisted by available auditory cues, attending to the speakers’ gestures, and situational context
What are some factors that enhance AV integrations during patient counseling?
Residual hearing (the more audible the signal is, the easier speechreading becomes)
Familiarity of grammatical structure assists understanding (language is redundant - more info in a complete sentence than is essential; assisted by the rules of the language you learned as a child)
Awareness of context improves success in speechreading
Viewing angle (90 degrees significantly reduces identification, 0 degrees is ideal)
How you feel influences speechreading abilities
How can you simplify syntactic structure?
Get to the point. Use concrete, clear, and concise phrases (add more periods)
Choose common words
Don’t speak above a person’s level (that’s just rude anyway)
Use Clear Speech speaking techniques
*Also make sure non-verbal gestures match your message so they’re not a distraction
Does distance influence the speechreading process?
Yes
5 to 6 feet is ideal
Minimal difference noted at 18 feet
At 23 feet, there’s a 16% decrease is visual recognition
At 100 feet, only 11% of the common nouns were recognized
What are other general factors that impede speechreading?
Mumbles
Doesn’t look at me when talking
Chews gum
Unusual accent
Speech impediment
Smiles too much
Uses no facial expressions
Shouts
High-pitched voice
Talks too fast
Complicated sentences
Beard/mustache
Wears dark glasses
What are some formal trainings?
Lipreading Practice
Seeing and Hearing Speech - Sensimetrics
Lipreading.org course
Read My Quips
What are some informal trainings?
Watching TV with closed captions
Enable captions for video calls and virtual meetings
Practice with CPs who speak clearly
