Lecture 6 - Naïve Bayes

Question 1

Classification & clustering

Answer 1

Both result in a categorisation of records into one or more classes based on their values

Question 2

Classification

Answer 2

• Trains a model that allows classifying new records to one of the classes
• Assumes the existence of predefined classes

Question 3

Clustering

Answer 3

• Divides the records into clusters
• Records with high similarity reside inside a cluster and records of two clusters are dissimilar

Question 4

Example of classification & clustering of e-mails

Answer 4

Classification: does e-mail go to inbox or spam?

Clustering: e-mail to work, friends or family folder?

Question 5

Why do we need classification?

Answer 5

• Organising documents is hard work
  
  • Route email messages into folders
  • Route help-desk inquiries to correct staff
  • Place documents in predefined categories/topic hierarchy
• Decided about (predefined) user interests/skills/…
  
  • User modelling
  • Instead of using human-authored expert system, let computer to induce rules or models from log data

Question 6

Classification: Learning & applying

Answer 6

Question 7

Classification Techniques

Answer 7

• Naïve Bayes
• Nearest Neighbor
• Decision Trees
• Support Vector Machines
• Logistic regression
• Deep learning
• Ensemble classification
• …

Question 8

Naïve Bayes example

Answer 8

Question 9

Naïve Bayes Classifier

Answer 9

Question 10

Bayes theorem

Answer 10

Bayes rule is a standard formula for inverting conditional probabilities

Question 11

Naïve Bayes Assumption

Answer 11

Naive conditional independence: assume that all features are independent given the class label y

Question 12

Laplace Smoothing

Answer 12

• Having a probability zero is problematic, because it wipes out all information in other probabilities

Solution:

Laplace Smoothing, or Correction, or Estimator

• Incorporates a small-sample correction in every probability computation
• Increase the numerator/denominator
• Thus, no probability will be zero

Question 13

Lecture Summary

Answer 13

Naïve Bayes is not so Naïve:

• Its beauty is in its simplicity
• Ability to handle categorical variables directly
• Computational eefficient
• Good classification performance, especially when the number of predictors is very large

Negative aspects:

• Requires a very large number of records to obtain good results
• Independence assumption may not hold for some attributes