2020 A2

Question 1

Contrast hard and soft allocation EM algorithms with one another and provide an example of
each.

Answer 1

Hard Allocation - associates each feature vector with only one missing label, being the most probable one (Viterbi re-estimation)

Soft allocation - probabilistically/partially associates the feature vector with multiple labels in accordance with one’s probabilistic knowledge of the situation. (Baum Welch re-estimation)

Question 2

Suppose you have fitted a GMM to data, and someone then points out there should be temporal structure in your data. How can you use your GMM to initialize an HMM

Answer 2

1. Create an HMM with the number of states equal to the mumber of mixture components in the GMM model.
2. Assign the parameters found for each Gaussian mixture component to the Gaussian density functions associated with the stage of the HMM.
3. State transition probabilities can be determined by analyzing the responsibilities of the data set.
4. The HMM should now be initialised. Perform EM to optimised the HMM for the given data set.

Question 3

Give a situation where a cyclic topology would be appropriate

Answer 3

Any repetitive pattern

Question 4

Suggest a sensible approach to initialise the training of a cyclic HMM from a data set.

Answer 4

(ans)
The initialization of the values can have a big impact on the performance of an HMM. So different methods can produce vastly different results.

Ones can use either the EM algorithm or the Viterbi Algorithm.

Question 5

It is common practice in machine learning to partition data sets into three portions for analysis.
One of these portions, the validation set, is used for selecting hyperparameters. Give the
hyperparameters for the following models:
i) logistic regression
ii) HMM.

Answer 5

logistic regression - Regulation parameter.

HMM - Number of states.