Lecture 8 - From HMMs to End-to-End Systems

Question 1

In a Large Vocabulary Speech Contunuous Recognition, how many words are there?

Answer 1

80,000 - 100,000

Question 2

When creating an ASR, some questions you might ask in designing would be is is constrained, natural speech, small or large vocabulary?

Explain the difference between a small and large vocabulary

Answer 2

Small vocabulary
- Isolated word, each word dedicated acoustic model

Large vocabulary
- Model at sub-word level
- Acoustic models for each phoneme
- Words recognised as sequences of models concatenated.

Question 3

What is the disadvantage of HMFCCs?

Answer 3

HMFCCs are not noise robust.

Question 4

When using HMMs for ASR, what can it be considered for?

Answer 4

HMMs can be considered as the acoustic model

Question 5

Explain the difference between small and large vocabulary when using HMMs as the acoustic model

Answer 5

1. Small vocabulary
   - Word-level HMM
2. Large vocabulary
   - Phone-level HMM (40 monophones)

• 2-state HM is used to model a phoneme.
• Words built from phonemes.

Question 6

A problem for HMMs in ASR is that given an observation sequence, how to compute what is the most likely state sequence to produce that observation sequence

What is the solution to this?

Answer 6

Using the viterbi algorithm.

The viterbi algo defines best score along a single path, at time t, that accounts for the first t observations and ends in state Si

Question 7

What are the challenges in an ASR system?

Answer 7

1. Atypical speakers (i.e. children, speech impediments)
2. Colloquiums, um, er, coughs
3. Noise - incorporate visual information
4. emotion and intent
5. Limits of current approaches -> use of DL

Question 8

What is the McGurk effect?

Answer 8

We don’t perceive speech just from sound, it is audio-visual.

The shape that the mouth makes also has an influence on the type of sound being perceived.

Audio ‘ba’ + video ‘fa’ perceive ‘fa’

Audio ‘ba’ + video ‘ba’ perceive ‘ba’