Language Models

Question 1

Language Models

Answer 1

How likely is this word sequence?

Probabilistic models of the likelihood of a string occurring

Compute the maximum likelihood estimate
(get probabilities by estimating relative frequency from the text)

For longer sequences, the counts are too sparse, so use the chain rule and the Markov assumption to estimate the probabilities

Question 2

P (“the chair hopped away”)

Answer 2

Chain Rule:
P(“the”) * P(“chair”|”the”) * P(“hopped”|”the chair”) * P(“away” | “the chair hopped”)

Markov Assumption:
P(“away” | “the chair hopped”)
= P(“away”|”hopped”) if using bigrams
= P(“away”|”chair hopped”) if using trigrams