Word Recognition Flashcards
COHORT’s assumptions (per TRACE)
- Uses first sound or first CV to determine words in initial candidate set (“cohort”).
- Eliminates words from cohort as successive phonemes arrive.
- Phoneme-to-word inhibition
- Words can also be eliminated based on semantic content, but initial cohort is determined by acoustics.
- Word recognition occurs when there is a single item ileft n candidate set.
- Word recognition can influence phoneme identification after the word has been recognized.
COHORT person
Marslen-Wilson, circa 1980
Good evidence for COHORT
In gating, the uniqueness point usually matches the participant’s acceptance/certainty point.
Confirms COHORT’s prediction that words are recognized when just one item remains in cohort.
COHORT’s problems (per TRACE)
- Cannot cope with distortion or underspecified onsets
- No way to recover items that were removed from cohort
- Want to reject “present” for pleasant but accept “blacelet” for bracelet.
- Assumes listeners always knows where a word’s onset is.
COHORT’s three stages of word recognition
- Access (when the bottom-up perceptual input first activates lexical representations)
- Selection (narrowing down the activation candidate set)
- Integration (retrieve and integrate semantic/syntactic details)
COHORT
- accesses all items consistent with input and
- evaluates the multiple words in parallel,
- using information as it becomes available.
COHORT’s segmentation strategy
Segmentation is implicit.
Utterance onset marks onset of first word.
Offset of each word marks onset of next word.
Controversial features of TRACE
Spatializing time, so lots of units are duplicated.
Assuming interactive effects between layers.
Activation in TRACE
It’s continuous, based on how acoustic/phonetic features map onto lexical representations.
Supports partial activation rhymes because they are bottom-up matches.
Interactivity in TRACE
- Top-down connections from lexical items to phonemes.
- The top-down connections from phonemes to feature detectors are usually disabled.
Competition in TRACE
Temporally overlapping units in phonemic and lexical layers inhibit one one another.
How TRACE models time
Units in phoneme and lexical layers are repeated every few time slices.
It spatializes time.
Coarticulation in TRACE
Input phoneme’s features are spread over 11 steps, but the centers of adjacent input phonemes are 6 steps apart.
TRACE’S acoustic features
- Acute
- Burst
- Consonantal
- Diffuse
- Power
- Vocalic
- Voiced
Each with nine levels of activation, each with a feature detector at every timestep.
So there would be Voice0, Voice1, …, Voice8 feature detector units at each step.
Shortlist’s main idea
- At each phoneme time-step, a shortlist of matching words is generated
- The words in the shortlists that overlap each other compete with each other via lateral inhibition
- Separates lexical access (shortlist formation based on match scores) from competition (overlapping words across the lists have to compete with each other)
Shortlist’s scoring system
- 1 point for each matching phoneme
- -3 points for each mismatching phoneme
- Strong mismatch penalty will keep mostly onset-matching items in the shortlist.
- Rhymes will only appear in the list
- when shortlist is sparse and
- when there have been multiple matching phonemes to overcome initial mismatch.
- There is a shortlist at each phoneme timestep, consisting of words with top match scores.
Simple recurrent networks
- Learning network
- Layers for input units, hidden units, context units, output units
- Context units are exact copy of last time steps hidden units
- Hidden units combine information from input and previous state.
- Interactive in the sense that the context interacts with the input units.
- Recurrence is self feedback
Overtraining in SRNs
- If you train an SRN until error asymptotes, it will not show rhyme effects
- If you train until each target reaches from recognition threshold, rhyme effects will remain intact.
- Adults learning novel neighborhoods look like these SRNs (Magnuson et al. 2003)
Localist representations
One unit for each word.
Competing units compete as their activation changes over time.
Distributed representations
- All items are represented by a shared set of units.
- Competition shows up in the blend of hidden representations.
- Makes predictions priming effects of ambiguous inputs activates a blend of competitors.
Distributed Cohort Model
SRN but two output layers: phonological and lexical semantics
Distributed representation of phonological and semantic features in the hidden units
Cross-Modal Semantic Priming
- Castle-candy-sweet priming
- Castle activates its cohort (including candy) which in turn activate semantic cohorts (including sweet).
- Works for cohorts but not for rhymes
- Cohorts don’t get much inhibition during start of word
- When rhymes finally have perceptual support, the onset-cohort are strongly inhibiting them
Naming / Repetition / Shadowing
- Play a wordlike stimulus. Listener repeats word.
- Measure accuracy and response latency.
- Word frequency and neighborhood density influence outcome measures.
- Unlike real recognition
- bc it’s post-perceptual
- bc listener may pay close attention to the speech sounds (without semantic processing).
Lexical Decision
- Play a wordlike stimulus. Listener decides whether stimulus is a word or not.
- Measure accuracy and response latency.
- Sensitive to
- word frequency (more false rejects of less common words)
- neighborhood density (probably, slower to make decision when stimulus overlaps with a lot of words).
- Unlike real recognition bc
- it’s a post-perceptual judgment about stimulus.
- Also a listener can make decision w/o knowing meaning of word (I can recognize tamp but I don’t know what it means).
Gating
- Early gates have a wide variety of guess (many words activated)
- Uniquess point: When there is only one completion of a word
- Recognition point: When correct guessing occurs
- Recognition can precede uniqueness, especially for higher frequeny words
DAS similarity
- Neighborhood consists of all words that differ by Deletion, Addition, or Substitution of just one phoneme
- Treat rhymes as just as neighborly as cohorts.
Cohort similarity
Neighborhood consists of all words that share first 1-2 phonemes
Graded similarity metric
- Neighborhood based on phonetic features
- cat/pat (1 feature diff) more similar than cab/bat (2 feature diff)
- bull/veer are neighbors (3 feature diff)
Frequency-Weighted Neighborhood Probability
- Frequency weighted similarity score: Similarity of candidate to input times candidate’s log frequency: FWSS(t,x) = f(t) * S(t,x)
- Neighborhood probability is a candidate’s FWSS divided by sum of FWSS of all other words in lexicon
- For DAS similarity, S(t,x) is 1 or 0, so it reduces to FWNP(t) = f(t) / ∑ f(w)
- Proportion of neighborhood frequency contributed by the word t
- Frequency-weighting applies a prior probability
- If 2 words matched on neighborhood size, more frequent one is easier to recognize
- If 2 words matched on frequency, one in less dense neighborhood is easier to recognize
Principles of spoken word recognition
- Multiple activation: Multiple candidate words are activated
- Activation of candidate word is based on
- Similarity: degree of fit between speech and candidate
- Priors: prior probability of word
- Compeition: Competition among candidates leads to recognition
How to test model against data
- Establish linking hypothesis between model and data
- Figure out if success/failure is due to theory, implemenation (input representation, number of units, etc), model parameters, or linking hypothesis
How to model frequency in TRACE
- Post-lexical (adjust choice rule that transforms activations)
- Resting activations
- Manipulate phoneme-to-word weights (effect of frequency is proportional to amount of word heard)
All very similar results (Dahan et al., 2001) but bottom-up weight scheme had best fit to data.
Arguments for feedback (as in TRACE)
- We see lexical effects on sublexical tasks
- Makes model robust to noise
- Implicitly encodes sublexical probabilities (diphone etc)
- TRACE more accurate and faster if feedback on
- Top-down knowledge can guide perceptual recalibration
- Interactive activation systems can perform optimal Bayesian inference
Arguments against feedback (in TRACE)
- Lexical effects can be captured if perceptual and lexical info integrated postlexically
- No way to increase information in raw signal, so best bottom-up signal is best guess
- Predicts perceptual hallucinations
- But listeners to show lexically influence perceptions, like phoneme restoration or failing to recognize some mispronunciations
Magnuson, Dixon, Tanenhaus, and Aslin, 2007
- Eyetracking
- Demonstrated competition effects even though competitors were not displayed
- Manipulated frequency, neighborhood density (one phoneme DAS), and cohort density
- High frequency: Early, continuous facilitating effect
- High cohort density: Early, continuous inhibitory effect
- High neighbor density: Early facilitating, late inhibitory effect (rhymes kick in)
- Competitor set dynamically changes as word unfolds
Dahan, Magnuson, Tanenhaus, and Hogan, 2001
- Subcategorical phonetic mismatches
- Ne(ck)t slower than ne(p)t in eyetracking
- Marslen-Wilson and Warren, 1994, found no RT difference for these kind of data.
- Concluded part of “neck” could not be inhibiting “net”
- Here, eyetracking data shows the predicted difference.
- Compatible with lateral inhibition
Phonotactics in TRACE
Implicit
More frequently a phoneme or n-phone appears in TRACE lexicon, the more top-down feedback it gets
Hannagan, et al., 2013
- Implemented an activation model inspired by TRACE but without the spatial duplication of time
- Used string kernels
- Achieves spatial invariance for visual word recognitions, so generalize to time.
- Layers
- Input phonemes over time
- 1-Phone and 2-Phone levels
- Word level (lateral inhibition here)
- /d/o/g/ activates /do/, /dg/, /og/ but not /gd/
- Proof of concept
Temporal problems for spoken word recognition models
- Temporal order over phonemes: dog vs. god
- Multi-token independence problem: do vs. dude
- Temporal order over words: man bites dog vs. dog bites man
- Segmentation problem
Phenomena simulated by TRACE
- Lexical effects
- Main Ganong effect: Xlug as /p/, Xlood as /b/
- Time pressure eliminates effect
- Late lexical effects (targeX as /t/). Context has stronger effect for ambiguous phonemes late in word.
- No lexical effects for unambiguous phonemes
- Phoneme monitoring data
- No lexical effect on RT for word-initial targets (respond when you hear word starting with /g/)
- Lexical effect on RT for later targets (faster on secreT vs. gulduT)
- Lexical conspiracy: Phonotactic effects from lexical statistics
- Xluly (X in p-t continuum). Interprets as “Xl” early on then shifts to “tl” by end, a phonotactically illegal sequence.
- Simulating cohort and rhyme competition in eyetracking
- Sensitivity to subcategorical mismatches
- Lexical basis for segmentation
The Shortlist Models
- Shortlist A, Merge, Shortlist B
- Differentiates time-specific tokens and time-invariant words
- Generates a shortlist of tokens and lexical lattice on the fly
- How could lattice be wired on the fly?
- Core theoretical Assumptions
- Prelexical to Lexical paths are feedforward only
- Candidate selection is based on matching and mismatching information
- Implicit segmentation via competition among candidates
similarity in neighborhood activation model
- Global similarity
- Static set of competitors
- Recognition RT related frequency-weighted neighborhood probability
- More neighbors, harder.
- More frequent neighbors, harder.
- Naming, decision, recognition in noise allow global similarity to matter
why input to spoken word models should not be phoneme units
They overlap too much.
Phoneme certainty is never 100% at any one point in time.
some ways to make non-phoneme input to word recognition models
- Fake acoustics
- Real acoustics
- Simulate output of prelexical processing
Shortlist B
- Norris and McQueen 2007
- Most radical move: Abandons activation entirely
- What is an activation, anyway?
- No inhibition among competing hypotheses
- Competition reflected in differing path probabilities
- Assumes word recognition is optimal, in a Bayesian sense
- As ambiguity increases, prior beliefs matter more.
- Priors require us to encode frequency.
- Likelihood functions are more important than similarity metrics
