Lecture 14: Language Flashcards
Mental lexicon
mental store of information that
includes semantic (word meaning), syntactic (word
combinations), and word form (visual or sound
patterns, spelling) information
Average adult speaker knows about 10,000-20,000 words and can easily recognize or produce about 3
words/sec: so, alphabetically ordered, dictionary-type organization could not work.
Other differences:
Words can be forgotten and added
More frequently used words more easily accessed
Symbol grounding problem
if words are defined by other words, one must know the meaning of some
words in advance
How is the mental lexicon organized?
In contrast to the standard dictionary model, access to a word in our mental
lexicon is affected by its relation to other words
way around Symbol grounding problem
One way around this problem: if some concepts are not
defined by other words, but are “grounded” by
interactions
with the environment, e.g., meaning of “pull” or “kick”
could be grounded by motor action
The Hub-and-Spoke Model
Amodal semantic ‘hub’ (anterior temporal lobe)
Grounded/embodied semantics (sensory/motor systems)
the model stores semantic information in various regions involved in sensory and bodily processes (the spokes) and these connect to a central amodal semantic syste (hub)
Two contrasting models for organization of mental
lexicon in relation to producing words.
Levelt’s Discrete Stages Model
Dell’s Interactive Stages Model
Levelt’s Discrete Stages Model
Model can account for basic tip of the tongue (TOT) phenomenon:
lemma activated, and activation fails to spread to next stage of lexeme
retrieval
Studies of TOT (Caramazza) show that people have access to purely
syntactical information (e.g., grammatical gender in Italian speakers)
without phonology
Model can account for basic tip of the tongue (TOT) phenomenon:
lemma activated, and activation fails to spread to next stage of lexeme
retrieval.
BUT: The same studies show that it is also possible to access
phonology (first phoneme) w/out grammatical gender
Interactive models of language processing reject the unidirectional flow of information in the Levelt stage model. why?
activation at later stages able to
influence what happens at earlier stages, because
there is some parallel processing
Dell’s Interactive Stages Model
Similar stages to Levelt’s but includes bidirectional or
interactive activation
Reprinted from Levelt
(1999).
Lemma selection influenced by both
phonological and semantic information
Can account for “mixed” speech errors, e.g., saying “oyster”
for “lobster” – error reflects both phonological and semantic
information
In interactive models like Dell’s, access to a word in the mental lexicon
affected by its relation to other words on a number of dimensions
auditory neighborhood of a word
number of similar
sounding words – more specifically, number of words that differ from
target by only a single phoneme.
Phoneme
smallest unit of sound that makes a difference for meaning:
ex, “L” and “R” in English: Late and Rate have different meanings (L
and R represented by a single phoneme in Japanese)
Neighborhood effect
we are slower to identify words with a large than small “auditory neighborhood”, i.e., more words differ on only a single phoneme.
Semantic/associative relations especially important.
Models of Knowledge as Semantic Networks
Words that have strong associative or semantic relations are closer
together in the network (e.g., car and truck) than are words that have
no such relation (e.g., car and clouds), as shown in studies of
semantic priming (e.g., car primes truck but not clouds).
(Semantically related words are colored similarly in the figure, and
associatively related words (e.g., firetruck–fire) are closely
connected)
-Semantic network models often include categorical organization
-Could brain damage destroy a particular category within the mental
lexicon?
-Is it possible to lose your ability to name specific categories of
objects?
Elizabeth Warrington and colleagues in London reported that some patients showed category-specific deficits
e.g., had little or no difficulty pointing to
or naming pictures of living things, but
had great difficulty pointing to or naming
man made objects such as tools
Other patients showed the opposite
pattern (i.e., double dissociation).
These patients had category-specific
deficits in conceptual/semantic
knowledge, but others have category-
specific naming deficits with intact
conceptual knowledge
Locations of brain lesions that are
correlated with selective deficits in
naming persons, animals, or tools
Actual averaged lesion data are
displayed for patients that had person
-naming (top), animal-naming
(middle), or tool-naming (bottom)
deficits.
The colors indicate the percentage of patients
with a given deficit whose lesion is located in the
indicated area. Red indicates that most patients
had a lesion in that area; purple indicates that
few had a lesion in that area.
Damasio et al. (1996,
Nature)
Similar Pattern of Results from Healthy
Subjects in an Early Neuroimaging Study (PET)
Naming persons activated
mostly the temporal pole,
naming animals activated
mostly the middle portion of
the inferior temporal gyri, and
naming tools activated mostly
the posterior portions of the
inferior temporal gyrus
Why are category-specific deficits in comprehension
and naming observed
Warrington & Shallice (1984): Differences in processing of
sensory perceptual information (most relevant for distinguishing
among
living things) vs. functional information (most relevant for
distinguishing nonliving things, e.g., manmade objects such as
tools)
Problems (see Ward):
Patients with selective deficits for living things don’t have more
difficulty answering sensory vs. functional questions about animals
or objects.
Some patients who have difficulties with sensory properties don’t
show expected category-specific impairments
Others have argued for the organization of brain into distinct
categories, but not a strict sensory-functional dichotomy –
e.g., Caramazza & Shelton (1998) argued that there may be
hardwired categories.
Still being debated, but evidence for some type of category-specific
organization is strong.
evidence toward categorical organization of semantic knowledge
Evidence from category-specific naming disorders and
neuroimaging studies point toward categorical organization of
semantic knowledge, and indicates that anterior temporal lobe is
associated with impairments in naming living things, more posterior
regions with impairments in naming manmade objects.
Sound
pressure
waves caused by
vibration
Sound waves vary in
frequency and
amplitude
Two Special Problems of Speech Perception
Lack of sharp boundaries
Segmentation
Lack of sharp boundaries
Written words/sentences have sharp physical boundaries, but spoken words/sentences don’t.
Ex Speech waveform of a single word can appear like two
words because of embedded silence
Segmentation
Spoken sentences often lack clear boundaries between
words because they are frequently coarticulated (i.e., ends and
beginnings
are united).
Ex Speech waveform of “What do you mean?”
Speech perception system attempts to solve these problems of speech perception by relying on
cues from
Prosody (tone of voice)
* Syllable stress
* Formant frequencies: Complex sound waveforms that carry the most
critical information about speech
* Different phonemes/sounds differ in 2 critical formant
frequencies (F1 and F2)
* By putting together different combinations of F1 and F2 formants,
you can create understandable synthetic speech!
formant frequencies
Formants are frequency peaks in the spectrum which have a high degree of energy. They are especially prominent in vowels. Each formant corresponds to a resonance in the vocal tract (roughly speaking, the spectrum has a formant every 1000 Hz). Formants can be considered as filters.
Even simple sounds are comprised of complex
waveforms containing formant frequencies
Gunnar Fant (1919-2009)
Pioneer in the development of
synthesized speech using formant
“How are you?…I love
you…”
Speech synthesis with just two formant frequencies.
Erik Ramsey: Locked-In
-Car accident in 1999, stroke in brain stem, “locked in”
-Can’t move any part of his body, except his eyes- Moving eyes is exhausting
-Awake and intelligent, can feel
-Hadn’t spoken since 1999 – his stroke disconnected
“motor plans” formulated in
cortex from subcortical
motoneurons necessary to
produce speech.
-Can think of/imagine speech sounds, just can’t produce
A Neural Implant Designer
Phillip Kennedy
-Neurologist who designs
electrodes to use as neural
implants for brain-machine
interface
-Implanted an electrode in
Ramsey’s brain: left premotor
cortex (speech planning area;
localized in Erik via
fMRI)
-Electrode could wirelessly
transmit information from
surrounding neurons.
-Collected extensive neural
data, gathered when
Kennedy’s team asked
Ramsey to imagine speaking
specific words.
-But they could not decode the
data.
-In 2006, Kennedy contacted
BU researcher Frank guenther
Cognitive Neuroscientist Working on a
Computational Model of Speech Processing
and Production
Key idea in model: speech output areas
represent intended speech sounds in
terms of formant frequencies.
Used his computational model to generate design of decoder software that
could translate information about neural
activity from the electrode in the
premotor speech planning area into
formant frequencies.
Output of decoder drives a speech
synthesizer – would this allow Erik
Ramsey to learn to control the
Schematic of the Brain-Machine Interface
Erik Ramsey: Not Totally Locked-
In
After he learned to imitate, Guenther et al. examined
whether Erik could alter a synthesized sound (by
changing the neural signals that drive the BMI) to a
slightly different vowel than what he heard (e.g., hears
“UH” as in “hut”, instructed to produce “OO” as in “hoot”).
Across 25 sessions with real-time feedback, Erik
showed significant improvement:
Average hit rate in producing 3 target vowel sounds
increased from 45% in first session to 70% in final
session, including 89% in final block of final session
Anatomy/Neuropsychology of
Language
Left lateralization of language
Language is extremely complex; we don’t know how many psychologically defined functions of
language map onto the brain
No animal model
Early clues from brain damage and disease
But they support overly simplified model of
language
evidence of left lateralization of Language: Silent word generation task:
Silent word generation task:
Generate words beginning with the letter presented,
and articulate words “silently and entirely”
50 Left-handed subjects
50 Right-handed subjects
Limited Language in the Right Hemisphere
-knows object names,
some simple semantic
knowledge
-could read simple
sentences
-e.g., “The boy that was hit by the girl cried” is too
hard
Aphasia
-deficit in language comprehension or
production following brain damage or disease
problems in spoken/written language
-occurs in approximately 40% of stroke
primary aphasia
problems in language due to direct disruption of language processing
system
secondary aphasia
problems in language
resulting from memory problems
broca vs wernicke
Classical model of language processing (Lichtheim).
Damage to the area that stores auditory word images
produces Wernicke’s aphasia. Damage to the area that
stores motor word images produces Broca’s aphasia. The
arrows indicate the direction of information flow.
locations of broca vs wernicke
“Motor image area”= Broca’s area: left inferolateral prefrontal
“Auditory image area”= Wernicke’s area: posterior third of left superior
temporal gyrus
Broca’s Area
Non-Fluent
Aphasia
Broca concluded that speech production brain area is in the left inferior frontal lobe
As a result of a lesion in Broca area, there is a breakdown between one’s thoughts and one’s language abilities. Thus, patients often feel that they know what they wish to say but are unable to produce the words. That is, they are unable to translate their mental images and representations to words
TMS to Broca’s Area Produces
Transient Non-
fluent Aphasia
Some problems with the classical view of
Broca’s aphasia as a non-fluent aphasia
produced by damage to the left inferior
prefrontal cortex
does the brain rely on a single source of information for word recognition
multiple features of the acoustic signal (varying in temporal duration and psycholinguistic size) contribute to word contribution, not just phonemes
phoneme
any of the perceptually distinct units of sound in a specified language that distinguish one word from another, for example p, b, d, and t in the English words pad, pat, bad, and bat.
syllable
a unit of pronunciation having one vowel sound, with or without surrounding consonants, forming the whole or a part of a word; e.g., there are two syllables in water and three in inferno.
prosody
melodic aspects of spoken language (intonation to indicate a question, emotion)
mechanism by which spoken word recognition takes place: cohort model
The cohort model is based on the concept that auditory or visual input to the brain stimulates neurons as it enters the brain, rather than at the end of a word.
for example: on hearing the sound e, all words beginning with that sound become active (cohort of words) but when more information is revealed the cohort whittles down to fewer words until a point is reached where the evidence is consistent with one word (uniqueness point)
uniqueness point
The uniqueness point is that point, measured from the beginnning of the word, at which a word diverges from all other words in the lexicon.
imageability
semantic property of a word that relates to the extent to which a word’s meaning can evoke sensory images
N400
reflects a negative peak at around 400 ms after the onset of a word
the N400 wave is an event-related brain potential (ERP) measured using electroencephalography (EEG). N400 refers to a negativity peaking at about 400 milliseconds after stimulus onset. It has been used to investigate semantic processing, which may be dysfunctional in schizophrenia.
network model of semantic memory organization
Allan Collins and Ross Quillian developed the network model of semantic memory organization in the late 1960s. This network model indicates that nodes of information (categories) are connected to each other through strong and weak links. Priming allows for our memory to ready associated information for retrieval.
con: not all concepts have clear hierarchies
semantic dementia is linked to atrophy of the
temporal pole
evidence for hub and spoke model
patients with semantic dementia can categorize pictures accurately when exemplars are typical like categorizing a dog as a animal but they struggle with atypical category members like categorizing an ostrich as a bird
when asked to select semantic features they choose the typical category answer (they would match green with carrot because more vegetables are green)
agrammatism
Deficits in processing grammatical aspects of language
* Can understand sentences with simple grammatical structure
“The boy ate the cookie.”
* Have trouble understanding sentences with more complex rules (i.e., who
hit whom?)
“The boy was hit by the girl.”
* Broca’s area shows increased activity during processing of
grammatically complex sentences à might reflect higher working
memory demand
halting speech production devoid of function words like of, at, the, and
naming persons activates
temporal pole
naming animals activates
middle portion of inferior temporal gyri
naming tools activates
posterior portions of inferior temporal gyrus
Why are category-specific deficits in comprehension
and naming observed?
Warrington & Shallice (1984): Differences in processing of sensory-perceptual information (most relevant for distinguishing among living things) vs. functional information (most relevant for
distinguishing nonliving things, e.g., manmade objects such as tools).
Problems:
Patients with selective deficits for living things don’t have more difficulty answering sensory vs. functional questions about animals or objects.
Some patients who have difficulties with sensory properties don’t show expected category-specific impairments
Evidence from category-specific naming disorders and
neuroimaging studies point toward
categorical organization of
semantic knowledge, and indicates that anterior temporal lobe is associated with impairments in naming living things, more posterior
regions with impairments in naming manmade objects
Neighborhood effect:
We are slower to identify words with a larger auditory
neighborhood (i.e., more words that differ on only a single phoneme)
- Auditory neighborhood: The number of similar sounding words (i.e., differ from target
by a single phoneme; late/rate/hate) - Phoneme: Smallest unit of sound that makes a difference in meaning (e.g., L and R <- late vs. rate)
Our knowledge of words is organized into a semantic network,
where words that are more closely related are represented more
closely together in this network
* Evidence from studies of
semantic priming
If subjects have to make a decision about a list of words, they are
faster at making a decision about a word if the previously presented
word is semantically related (e.g., “car” primes “truck”, but not “cloud”)
semantic network
Our knowledge of words is organized into a semantic network,
where words that are more closely related are represented more
closely together in this network
* Evidence from studies of semantic priming
how are semantic networks organized
Semantic networks are categorically organized
* Words can be categorized by certain semantic properties (e.g., living,
non-living)
* Can brain damage destroy a particular category of words within the
mental lexicon?
* Evidence from category-specific word deficits
Category-Specific Deficits
Warrington (1970): Studied patients who had category-specific deficits for
conceptual/semantic knowledge about certain categories of words:
* These patients were fine at pointing to/naming pictures of living things
* BUT, they had great difficulty in pointing to/naming non-living, man-made objects
like tools
* Other patients showed the opposite pattern (double dissociation)
Category-specific deficit is NOT the same thing as visual agnosia
Visual agnosia: Cannot recognize the object they are presented with, but can tell you
other conceptual/semantic knowledge about the object (e.g., can’t recognize a picture
of a telephone, but can tell you that it rings and you use it to call people)
- Category-specific deficit: Can tell you that 2 objects are the same (i.e., can match 2 pictures of telephones), but don’t have access to what that object is for (not a problem of object recognition– deficit is in conceptual knowledge)
Category-Specific Deficits brain location
- Damasio et al., 1996: Locations of brain lesions are correlated with
selective deficits in naming people (mostly anterior temporal lobe),
animals, or tools (mostly posterior temporal lobe) à anterior to
posterior gradient for living to non-living things
Similar pattern in PET study of healthy subjects: Naming people
activated mostly temporal pole (anterior portion of temporal lobe),
naming animals activated middle portion of inferior temporal gyri,
and naming tools activated mostly posterior portions of the
inferior temporal gyrus
category-specific deficits due to brain damage can be
attributed to differences in the processing of sensory/perceptual information vs processing of functional information
Sensory and perceptual information is most relevant for distinguishing between living things …deficits arise from damage to more anterior temporal regions (closer to IT cortex, important for
object perception)
Functional information is most relevant for distinguishing between nonliving things, such as
tools à deficits arise from damage to more posterior temporal-parietal regions (important for
sensorimotor functions)
- Problems with the sensory-functional explanation:
- Patients with selective deficits for living things don’t have more
difficulty answering sensory vs. functional questions about animals or
objects - Some patients who have difficulties with sensory properties don’t have
the expected category-specific impairments - Compromise: Organization of information within semantic memory network is a
distributed network of specialized clusters
category-specific deficits due to brain damage can be
attributed to differences in the processing of functional information
Functional information is most relevant for distinguishing between nonliving things, such as
tools à deficits arise from damage to more posterior temporal-parietal regions (important for
sensorimotor functions)
using formats for locked in patient
Phillip Kennedy & Frank Guenther: We can use the concept of formant
frequencies to create synthetic speech in a locked-in patient (Eric Ramsey)
* Can think of/imagine speech sounds, but cannot produce them
Implant electrodes in left premotor cortex (speech planning area) and measure
activity when asked to imagine speaking words
* But… had trouble decoding data from neurons
* Key idea: Speech output areas represent intended speech sounds in terms of
formant frequencies
* Build a decoder that translates neural activity from premotor neurons into formant
frequencies, and output can drive a speech synthesizer
Eric Ramsey could imitate and alter synthesized sound based on formants– and
showed significant improvement with practice!
Lichtheim’s Classical Model of Language Processing
There is an auditory area that stores information about word
sounds that you hear (Wernicke’s area)
* There is a speech programming (motor) area that involves the
motor component of word output, necessary for speaking (Broca’s
area)
These 2 areas are connected by a fiber tract: the arcuate
fasciculus (AF)
Take away point: The association of Broca’s area solely with motor output of speech in Lichtheim’s classical model is too simplistic
However, damage to Broca’s area doesn’t always result in Broca’s
aphasia
- Dronkers et al. (1996): Imaging study showed that out of 22 patients with
damage to Broca’s area, only 10 had Broca’s aphasia à damage to other white
matter and subcortical structures is also important
Broca’s aphasia not only involves problems in speech production, it
can also involve comprehension problems
Wernicke’s area:
Speech comprehension brain area in superior
temporal gyrus (junction between parietal and temporal lobes)
* Wernicke thought this area is involved in auditory storage of words
- Wernicke’s aphasia (fluent aphasia):
- Problems in word comprehension and producing meaningful
sentences (i.e., can’t choose the right words, can’t monitor verbal
output) - Fluent speech, but nonsensical sounds or sentences (“word salad”)
Wernicke’s Aphasia: Re-evaluated
There is not a perfect association between Wernicke’s aphasia and
damage to Wernicke’s area (out of 70 patients with aphasia, 7 had
brain damage outside Wernicke’s area)
* Suggests there has to be damage to the surrounding posterior temporal lobe
regions, or damage to white matter paths that connect this area to other parts
of the brain
Wernicke’s aphasia not only involves problems in speech
comprehension, it can also involve speech production problems
* Some patients can produce made-up words and have naming
problems
Although Lichtheim’s classical model says that Wernicke’s aphasia
reflects loss of linguistic knowledge, which impairs
comprehension, there has been evidence against the classical view
* Wernicke’s aphasics show intact semantic priming on a lexical
decision task (i.e., is this a word or non-word?) despite problems with
explicitly comprehending the semantic relationship between words
* Example: “doctor” primes “nurse” even though patients fail to
comprehend relationship between the two when explicit judgment is
required
Possible explanation: Perhaps Wernicke’s aphasia involves a
problem with processes that normally provide access to linguistic
information for use in real time
* Evidence favors the idea that patients can’t integrate words into the
context of a sentence quickly enough to allow normal comprehension
Broca’s and Wernicke’s Areas
evidence rejects the simplistic distinction that Broca’s area =
output and Wernicke’s area = comprehension
* To some degree, both areas are involved in both output and comprehension
We can think of these areas a 2 nodes in a larger left hemisphere
language processing network that is critical for both comprehension
and production
* Damage to the entire network produces global aphasia (problems with both comprehension and production)
Are Broca’s and Wernicke’s areas involved only in heard/spoken
languages, or do they serve a more general linguistic function?
* Evidence from American Sign Language (ASL)
fMRI study of 3 groups while processing sentences in either written
English or ASL
1. Normally hearing, monolingual native English speakers who didn’t
know ASL
2. Congenitally deaf individuals whose native language was ASL and
learned English late and imperfectly
3. Normally hearing “native signers” who were born to deaf parents and
learned both ASL and English as their native language
What would activity in Broca’s and Wernicke’s areas look like for
these 3 groups when reading English sentences and viewing ASL
sentences?
When reading English sentences, Broca’s and Wernicke’s areas were active for the hearing group and to some extent native signers, but not for congenitally deaf
* When viewing ASL sentences, Broca’s and Wernicke’s areas were active for deaf and native signers, but not for hearing subjects
these areas might serve a more general linguistic function (hard-wired for language)
- Describe Lichtheim’s classical model of language processing.
Where are Broca’s and Wernicke’s areas, and according to the classical model, what
roles do they play?
Describe symptoms of Broca’s and Wernicke’s aphasias. How are these aphasias
usually tested?
Explain why Lichtheim’s classical model is too simplistic of a view of Broca’s and
Wernicke’s areas. What is contradictory evidence for this model?
Are Broca’s and Wernicke’s areas strictly for heard/spoken languages? Describe
evidence related to this point.
