7-Naive Bayes

Question 1

What is the naive assumption in naive bayes?

Answer 1

All features (attributes) are conditionally independent on the class y

Question 2

What is the naive bayes algorithm?

Answer 2

y^ = argmax (y) P(y) * Product from i to n P(x_n | y)

Question 3

What are the naive bayes assumptions?

Answer 3

Features are conditionally independent
Instances are independent
Distribution of test data is the same as training data

Question 4

How do we manage unseen discrete features in naive bayes?

Answer 4

If any term hasn’t been seen before, we can:
- Epsilon smoothing: Set a small epsilon value
- Laplace smoothing: Add a pseudo count , (a + count())/ (Ma + count()), where Ma is the total count

Question 5

How do we manage continuous features with zero variance in naive bayes?

Answer 5

Ignore feature

Add small smoothing value to standard deviation