EM-Algorithm

Question 1

What is the general idea if the EM-algorithm

Answer 1

1. fill in missing values by estimated values
2. estimate parameters given the filled in values
3. re-estimate missing values with new parameter estimates
4. re-estimate parameters etc….

iterate until convergence of parameters

Question 2

What are the missing data mechanisms taken into account by EM-algorithm

Answer 2

MAR and MCAR

Question 3

What does the EM-algorithm consist of?

Answer 3

Expectation step and a maximisation step

Question 4

One advantage and one disadvantage of EM-algorithm

Answer 4

+ will reliably converge

- rate of convergence can be too slow when large fraction of missing information

Question 5

What does the E-step do?

Answer 5

Calculates the conditional expectation of complete data log likelihood given the observed data and current estimates of parameters

Question 6

What does the M-step do?

Answer 6

Performs maximum likelihood estimation on the (conditional) expected log likelihood found in E-step

Question 7

Formal definition of the E-step

Answer 7

Q(θ, θ(t)) = Eymis[ l(θ | Yobs, Ymis) | Yobs, θ(t) ]

Question 8

Formal definition of the M-step

Answer 8

θ(t+1) = arg max(θ) Q(θ | θ(t)

Question 9

What does the GEM-algorithm do differently?

Answer 9

θ(t+1) not chosen to globally maximise Q(θ | θ(t) ) but rather to ensure Q(θ(t+1) | θ(t) ) is greater than Q(θ(t-1) | θ(t))

Question 10

Explain stability / monotonicity in terms of EM-algorithm

Answer 10

Each step in the EM-algorithm leads to an observed likelihood that is greater than or equal to the previous likelihood

log ( L( θ(t+1) | Yobs) ) ≥ log ( L( θ(t) | Yobs) )