Classification

Question 1

In Supervised Learning, what is the goal?

Answer 1

To find a function f̂ that maps input x to output y, based on training data (xi, yi)

Question 2

What is the difference between Classification and Regression in Supervised Learning?

Answer 2

Classification: y belongs to discrete classes (e.g., binary or multiclass)
Regression: y is a continuous value

Question 3

What is the probabilistic view of Supervised Learning?

Answer 3

Specify a model P(X, Y | Θ), estimate parameters θ, and predict output using the estimated model

Question 4

What is a Generative classifier?

Answer 4

A classifier that models the joint distribution P(X, Y) and uses Bayes’ rule for prediction

Question 5

What is the Naive Bayes assumption?

Answer 5

Features xi are independent, conditionally on the class

Question 6

How does the Naive Bayes classifier represent documents in text classification?

Answer 6

Using a bag of words model, where each document is represented by a binary vector of word presence/absence

Question 7

What is the curse of dimensionality in classification?

Answer 7

As the number of features increases, the number of parameters needed grows exponentially (2^d - 1 per class)

Question 8

How does Gaussian Naive Bayes differ from a full multivariate Gaussian classifier?

Answer 8

Gaussian Naive Bayes assumes the covariance matrix Σc is diagonal (features are independent given the class)

Question 9

What are the steps for classifying using MAP (Maximum A Posteriori) in a Naive Bayes classifier?

Answer 9

See hand written paper.

Question 10

What are two methods for estimating parameters in Naive Bayes?

Answer 10

Maximum Likelihood Estimation (MLE) and Maximum A Posteriori (MAP) estimation