Naive Bayes

Question 1

Compare Naive Bayes to other methods (2)

Answer 1

• Training is very fast
• Naive Bayes computes its parameters in closed form by counting

Question 2

What’s the Naive bayes assumption?

Answer 2

Question 3

What’s wrong with the naive bayes assumption?

Answer 3

The features might not be independent!

Question 4

What’s the naive bayes recipe for closed form MLE?

Answer 4

Question 5

What does maximizing likelihood accomplish?

Answer 5

• There is only a finite amount of probability mass (i.e. sum-to-one constraint)
• • MLE tries to allocate as much probability mass as possible to the things we have observed…
• …at the expense of the things we have not observed

Question 6

What are the different Naive Bayes models and what kind of features do they correspond to?

Answer 6

Question 7

What does Naive bayes do?

Answer 7

Provide a framework for generative modeling

Question 8

What’s the overall process for Naive bayes? (3)

Answer 8

• Choose p(xm | y) appropriate to the data (e.g. Bernoulli for binary features, Gaussian for continuous features)
• • 3. Train by MLE or MAP 4.
• Classify by maximizing the posterior

Question 9

What kind of model is logistic regression?

Answer 9

Discriminative classifier

Question 10

Describe discriminative classifier (3)

Answer 10

– Example: Logistic Regression
– Directly model the conditional: p(y|x)
– Learning maximizes the conditional likelihood

Question 11

Describe generative classifiers (4)

Answer 11

Question 12

Compare naive bayes and logistic regression in terms of

• error
• model efficiency
• assumptions
• number of examples needed to estimate the parameters

Answer 12

• If model assumptions are correct: Naive Bayes is a more efficient learner (requires fewer samples) than Logistic Regression
• If model assumptions are incorrect: Logistic Regression has lower asymptotic error, and does better than Naïve Bayes
• Naïve Bayes makes stronger assumptions about the data but needs fewer examples to estimate the parameters

Question 13

What do we know about the features of naive bayes?

Answer 13

Features x are assumed to be conditionally independent
given y. (i.e. Naïve Bayes Assumption)

Question 14

What does the naive bayes assumption say, in words?

Answer 14

Features x are assumed to be conditionally independent
given y? (double check this)

Question 15

Compare Naive bayes and logistic regression wrt assumptions made about the featuers

Answer 15

Question 16

Compare Naive bayes and logistic regression wrt learning (MAP Estimation of Parameters)

Answer 16

Question 17

Compare Naive bayes and logistic regression wrt Learning (Parameter Estimation)

Answer 17