04 - Speech Pattern Classification

Question 1

What is meant by linguistic information?

Answer 1

Information that is explicitly in or almost uniquely inferable from the written message.

Question 2

What is meant by paralinguistic information?

Answer 2

Information that is not inferable from the written message, but is added by the speaker to complement the linguistic information. Attitude, intonation, etc. “I am SOO excited!”

Question 3

What is meant by Nonlinguistic information?

Answer 3

Information about other factors such as age, gender, idiosyncrasy (personal traits), physical emotion. In general, conditions that are not related to the linguistic contents and cannot be controlled by the speaker. Example: Crossing your arms in protest.

Question 4

Speech Pattern Classification refers to?

Answer 4

The extraction of information from a speech such as language, accent etc. and to take an input and convert it into a sequence of class labels.

Question 5

In speech classification we normally divide into 3 models. These are?

Answer 5

Acoustic model, language model, pronunciation model

Question 6

The 2 main blocks of the speech pattern classifications process are?

Answer 6

Feature extraction and classification.

Question 7

What is the local region of analysis?

Answer 7

This is the framing of the data.

Question 8

What is the global region of analysis?

Answer 8

This is the functionals. For example the mean, median, max etc.

Question 9

What does ASR stand for?

Answer 9

Automatic Speech Recognition

Question 10

What is the segmental region of analysis?

Answer 10

This is the phonemes, voiced/unvoiced, word, etc.

Question 11

Spectral features in feature extraction are?

Answer 11

Classical speech (ASR) features, spectral measures. This is most likely the MFCC’s.

Question 12

Prosodic features in feature extraction are?

Answer 12

Pitch, energy, formants, timing, articulation etc.

Question 13

What are the delta-coefficients?

Answer 13

Delta features (first-order derivatives) provide information about the rate of change of the acoustic features over time. They are obtained by computing the differences between consecutive frames of the acoustic features. Delta features capture the dynamics of the speech signal and can help in modeling the transitions between different phonetic units.

Question 14

What are the double-delta coefficients?

Answer 14

Double-delta features (second-order derivatives) provide information about the acceleration or curvature of the acoustic features. They are computed by taking the differences between consecutive delta features. Double-delta features capture the changes in the rate of change of the acoustic features and can provide additional temporal information beyond the delta features.

Question 15

When do we use the Cepstral Mean (Variance) Normalization (CMN/CMVN)?

Answer 15

We use it in the pre-processing before conducting the actual analysis and we do it to reduce variation in various channels.

Question 16

Prosodic features refers to 4 things. These are?

Answer 16

Fundamental frequency (F0): mean, median, pitch contour etc.
Energy: shimmer, energy contours, voice level etc.
Duration: Speech rate, ratio of duration of voiced/unvoiced regions etc.
Formants: first to fourth formants, bandwidths etc.

Question 17

There are also some time-domain features, which were also prominent in subject 3 on speech signal representations. These can be for example?

Answer 17

Zero Crossing Rate (ZCR)
Autocorrelation
Attack (duration, slope)
Temporal energy centroid

Question 18

Why is it normal that we use Voice Activity Detection (VAD) in feature pre-processing?

Answer 18

The presence of silence in the training data can corrupt the model. And likewise, silence in the test data will degrade the decision.

Question 19

How do we apply Voice Activity Detection (VAD)?

Answer 19

Energy thresholds (as we did in the second lab).
Waveform and spectrum analysis based on pitch and harmonic detection or maybe even ZCR.
Or finally, based on statistical models.

Question 20

We can use some methods to manipulate our features to improve training time such as dimensionality reduction. Two of these measures are?

Answer 20

Principle Component Analysis (PCA) and Linear Discriminant Analysis (LDA).

Question 21

A classical approach to speech pattern recognition is the Gaussian Mixture Model (GMM). The GMM is a particular case of which model?

Answer 21

The Hidden Markov Model(s). HMM.

Question 22

Why is the Gaussian Mixture Model (GMM) so useful?

Answer 22

Single gaussians cannot model. We need several gaussians and to mix them in order to model. In theory, we can fit anything given enough components.

Question 23

Which method do we use to estimate GMM parameters?

Answer 23

The Expectation-Maximization (EM) algorithm.

Question 24

After using a method to estimate the GMM parameters, what do we then do?

Answer 24

Compute the log-likelihood of the sequence of features.

Question 25

What can we use Speaker Recognition (SR) for?

Answer 25

Speaker identification, access to closed spaces, verification

Question 26

In Speaker Recognition (SR) tasks we often need to evaluate measures. We can do this using test trials. We have target trials and imposter trials. Each trial requires 2 outputs which are?

Answer 26

Actual decision (true/false) and a likelihood score (confidence)

Question 27

There are two types of decision errors in Speaker Recognition tasks. These are?

Answer 27

Missed detections (Miss). These are the percentage of target trials rejected incorrectly. Normally called False Reject Rate (FRR). False alarms (FA). These are the percentage of imposter trials accepted incorrectly. Often called False Accept Rate (FAR)

Question 28

What is the 'Equal Error Rate' (EER) a measure of?

Answer 28

It is when the False Alarms (FA or FAR) and the Missed Detections (Miss or FRR) cross each other. It is often used as a performance measure.

Question 29

What can we use a Detection Error Tradeoff (DET) plot for?

Answer 29

It allows us to look at the tradeoff between Miss/FFR (x-axis) and FA/FAR (y-axis) to see the benefits of choosing a threshold. Kind of like a ROC-curve.

Question 30

In Speaker Recognition (SR) there are 4 stages between the speech signals input and the results (output). These are?

Answer 30

Feature Extraction (front end), Representation, Variability compensation, classification (back end)

Question 31

Features for speaker recognition should be PRS which is?

Answer 31

Practical, Robust and Secure

Question 32

Variablity compensation in Speech Recognition refers to?

Answer 32

The changes in channel effects between training and successive detection attempts. This is microphones, the environment etc.

Question 33

In Speaker Recognition we often use a GMM-UBM. What are the advantages of using this?

Answer 33

The Gaussian Mixture Model-Universal Background Model. The advantages is that it needs less data, only updates seen events, and keeps correspondence between means and fast scoring.

Question 34

In recent time, research have focused on developing more robust systems to session variability. Why?

Answer 34

Because session variability is one of the largest challenges to practical use of SR systems. Think voice activation in a noisy environment for example.

Question 35

What is the supervector concept?

Answer 35

The supervector concept takes each components from a GMM-UBM model, which represents a set of acoustic features, and concatenates the mean vectors of these features into a single SUPER vector. This enables very effective modelling and discrimination.

Question 36

After the GMM-UBM the GMM-Support Vector Machine (GMM-SVM) method was introduced. Why was this a huge success?

Answer 36

SVM's perform a nonlinear mapping from a high-dimensional input space and thus is more efficient/fast and will improve modelling and discrimination.

Question 37

What is the identity-vector (i-vector)?

Answer 37

The i-vector, which stands for "identity vector," is a concept in speaker recognition that represents speakers as points in a high-dimensional space. It captures speaker-specific information by modeling the variations in the mean supervector of a speaker's utterances. The i-vector approach is beneficial because it provides a compact and robust representation of speaker characteristics, enabling efficient modeling of speaker variability and facilitating speaker recognition tasks even in challenging conditions such as limited enrollment data and mismatched recording conditions. The i-vector concept has shown promising results and has become a popular technique in the field of speaker recognition.

Question 38

In 2018, x-vectors were introduced as an extension of the i-vector. Why are these so successful?

Answer 38

X-vectors improved upon the i-vector approach by incorporating deep neural networks (DNNs) to extract speaker embeddings. X-vectors leverage the power of deep learning to capture more discriminative speaker representations, leading to enhanced speaker recognition performance, especially in scenarios with large speaker variability and challenging acoustic conditions.

Question 39

What is the state-of-the-art in speaker verification?

Answer 39

The ESCAPA-TDNN. A deep residual convolutional neural network. It has less than 2% EER (Equal Error Rate).

Question 40

What is the state-of-the-art in speaker diarization?

Answer 40

It is the VBx system. A variational Bayers hidden markov model with x-vectors. It has a Diarization Error Rate (DER) of about 5%.

Question 41

What is the state-of-the-art in Speech Emotion Recognition?

Answer 41

A 2D CNN LSTM. Specifically the best one is currently the EmoDB.