Lecture 8 - Probabilistic models

Question 1

what are the two interpretations of probability

Answer 1

1. Frequentist interpretation
2. Bayesian interpretation

Question 2

What is frequentist interpretation of probability

Answer 2

Probabilities represent long run frequencies of events.

Question 3

What is bayesian interpretation

Answer 3

Probabilities are used to quantify our uncertanty about something and are related to information rather than repeated trials.

Question 4

what are the 4 components of bayes rule

Answer 4

• Prior P(Y)
• Posterior P(Y|X)
• Likelyhood P(X|Y)
• Evidence P(X)

Question 5

what would the 4 components represent in the spam vs ham example

Answer 5

• Y is Ham/Spam
• X representing features (words in email)
• P(Y) is the Prior probability of recieving a spam and ham email in general
• P(X) is the Evidence probability of a specific email with certain words

Question 6

When should i use likelyhood and when should i use posteriori

Answer 6

Use likelyhoods if you want to ignore the prior distrubution or assume it uniform, and posteroir probabilities otherwise.

Question 7

posteroir ordds = likelihood ratio x prior odds

Answer 7

true

Question 8

What re the two types of proabbility density functions of the Gaussian distribution

Answer 8

• Uni-variate
• Multi-variate

Question 9

What is maximum-likelyhood

Answer 9

Maximum-likelihood decision rule for classification using the likelihood ratio:
LR(x)= P(x|+)/P(x|-)

Question 10

What is MLE

Answer 10

Maximum likelihood estimation is a method of estimating the paramters of a probability distribution by maximising a likelihood function, so that under the assumed statistical model the observed data is most probable.

Question 11

What is multivariate Bernoulli distribution

Answer 11

Models whether or not a word occurs in a document. The joint distibution over the bit vector.

Question 12

What is the Multinomial distibution

Answer 12

Models how many times the word exists. Joinst distribution over a count vector: a histogram of the number of occurrences of all vocabulary words in a document.

Question 13

what is the Naive Bayes assumption

Answer 13

Naive Bayes assumption features are independent from each other. Multinomial and Multivariate Bernoulli models assume that variables are drawn independently from the same categorical distribution

Question 14

Is naive bayes assumption true?

Answer 14

NO
In general words are not drawn independently from a dictionary of words.