Probabilistic Model

Question 1

What is a sample space?

Answer 1

The set of possible outcomes of a random experiment

Question 2

What is an event?

Answer 2

A subset of the sample space. A collection of outcomes

Question 3

When is an event said to occur?

Answer 3

If the outcome of the random experiment is a member of the event set

Question 4

How is relevance determined by a probabilistic retrieval model?

Answer 4

Using the probability that a user who likes d would enter query q
Relevance(q,d) = p(q|d)

Question 5

What is the assumption made with the probabilistic model?

Answer 5

A user is formulating their query based on an imaginary relevant document

Question 6

What is a statistical language model?

Answer 6

Represents a probability distribution over word sequences.
Ex: p(“Today is Wednesday) = 0.001 but p(Today Wednesday is) = 0.000000001

Question 7

What is a language model?

Answer 7

A probabilistic model that estimates the likelihood of a sequence of words based on patterns observed in training data. Higher probabilities are given to more likely word sequences

Question 8

What is the unigram language model?

Answer 8

A language model that generates text one word at a time, with each word being chosen independently from a distribution of words

Question 9

How is probability of a phrase generated using a unigram language model?

Answer 9

Multiply the probabilities of the individual words

Question 10

How is the probability of a word determined with a unigram language model?

Answer 10

Based on frequencies within a corpus of text that is relevant to the topic in question

Question 11

What is the maximum likelihood estimator?

Answer 11

A method for estimating the probabilities of words in a Unigram LM
P(w|d) = (c(w,d))/|d|

Question 12

What is the issue with the maximum likelihood estimator?

Answer 12

It doesn’t account for unseen words. These are words which may be relevant but do not appear in the doc

Question 13

What is topic modelling?

Answer 13

A technique used to identify the main themes or topics present in a collection of documents. Can be solved using language models. If you have a new document that frequently uses words with high probabilities under a certain topic, it probably belongs to that topic

Question 14

What is association analysis?

Answer 14

Determining which words are semantically related to others. It analyzes the probabilities and patterns of word occurences.
Uses the probability of a searched word to find words similarly occuring

Question 15

How can we use the maximum likelihood estimator on a multi-word query?

Answer 15

Get the maximum likelihood estimate for each word and multiply them together

Question 16

What is the issue with using maximum likelihood estimator on a multi-word query?

Answer 16

If there is a query that doesn’t appear, the entire likelihood goes to 0

Question 17

How can we solve the issue with the maximum likelihood estimator?

Answer 17

Instead computing query likelihood which is how likely we are to observe a specific query form a doc model

Question 18

How do you compute query likelihood?

Answer 18

Multiply the probabilities of finding each word within the document and rank by highest likelihood