06 - Automatic Speech Recognition

Question 1

What makes ASR difficult (name two)?

Answer 1

Size, speaker, acoustic environment, style, accent/dialect, languages (amount of data etc.)

Question 2

What is ASR built from in two concepts (think front end and back end)?

Answer 2

A front-end feature extraction and a back-end classification.

Question 3

ASR is not a sequence2sequence problem. True or false?

Answer 3

False

Question 4

What is the main trend in the third generation (3G) of ASR?

Answer 4

Hierarchical modelling of speech.
AM –> LM –> PM (using HMM and GMM)

Question 5

What is the main trend in the fourth generation (4G) of ASR?

Answer 5

End2end. Direct mapping from acoustics to words/characters.

Question 6

What is the most important metric in ASR?

Answer 6

WER = Word Error Rate

Question 7

What is the equation for the Word Error Rate (WER)?

Answer 7

WER = (I + S + D) / N

Simple:
I = Insertion
S = Substitution
D = Deletion

I: The number of insertion errors, which represents the additional or extra words inserted in the output compared to the reference (ground truth).
S: The number of substitution errors, which indicates the words that are substituted or replaced in the output compared to the reference.
D: The number of deletion errors, which represents the words that are missing or deleted in the output compared to the reference.
N: The total number of words in the reference (ground truth) transcript.

Question 8

What are the two building blocks of the ‘conventional’ build of an Acoustic Model in the ASR?

Answer 8

It is the HMM (Hidden Markov Models) and the GMM (Gaussian Mixture Models)

Question 9

What are the three basic HMM problems?

Answer 9

1. Evaluation/scoring (forward algorithm)
2. Decoding (viterbi algorithm)
3. Training (baum-welch algorithm)

Question 10

What does the ‘Forward Algorithm’ do?

Answer 10

It permits computing the probability P(X|Ø) based on dynamic programming in O(N^2) time.

Question 11

What is the ‘Viterbi Algorithm’ used for?

Answer 11

It is used to find the best state sequence.

Question 12

What does the ‘Baum-Welch algorithm’ do?

Answer 12

It makes use of the forward probabilites and the backward probabilities to estimate the state occupation probability.

Question 13

What probability does the ‘Acoustic Model’ calculate?

Answer 13

P(X|W). That is, the probability of a vector sequence, X given an utterance, W.

Question 14

What does the language Model calculate the probability of?

Answer 14

P(W), that is. The probability of a given utterance, W.

Question 15

What is CFG?

Answer 15

Context Free Grammar

Question 16

When do we use CFG instead of n-gram?

Answer 16

When the sentences or phrases are complete and non-complex. Very simple phrases for example.

Question 17

When do we use n-grams instead of CFG (context free grammar)?

Answer 17

When there are comlex and maybe incoherent senteces or phrases. It is used when word sequence probabilities are important, e.g. in text prediction and speech classification.

Question 18

What do we use the ‘Pronunciation model’ for?

Answer 18

To find the optimal, W. That is utterance.

Question 19

We cannot perform continuous speech recognition with HMM’s. True or false?

Answer 19

False. We can and do.

Question 20

What does ‘LVCSR’ stand for?

Answer 20

Large Vocabulary Continuous Speech Recognition

Question 21

When we turn our attention to more modern approaches, the observation likelihood used in the ‘conventional’ methods is replaced by?

Answer 21

A (scaled) neural network posterier.

Question 22

Some disadvantes of using Neural Networks until 2012 were?

Answer 22

It was computationally expensive (still is) - moreso back then.
Neural network systems were less complex before then.

Question 23

What are the differences (progress) in Automatic Speech Recognition after 2012?

Answer 23

We have deeper and wider networks and our computers and GPU’s allow for much faster training.

Question 24

Time-delay Neural Networks (TDNN’s) model richer content. True or false?

Answer 24

True.

Question 25

Since GMM/HMM was proposed the approaches were more?

Answer 25

Statistical.

Question 26

Hierarchical (3G) systems is often split into 3 (sometimes more) concepts. These are types of models, and are commonly what kind of models?

Answer 26

Acoustic model, language model (n-grams) and pronunciation model. Also a decoder sometimes.

Question 27

We see a huge improvement in Large Vocabulary Continuous Speech Recognition (LVCSR) since 2012 due to what?

Answer 27

The positive improvement and impact of deep learning.

Question 28

Newer deep learning ASR systems replace the acoustic model of a hierarchical/statistical system by what?

Answer 28

A deep neural network. This gives better accuracy and context modelling.