SVM

Question 1

What is the general idea of SVM? What does the distance from the support vector mean?

Answer 1

To create a support vector to act as a barrier, separating the data into +1 or -1 regions. The distance from the support vector corresponds to the confidence of the prediction.

Question 2

What is the definition of the margin in SVM?

Answer 2

The distance to the closest data point from the decision barrier.

Question 3

How can a dataset that’s not linearly separable be handled with SVM?

Answer 3

We can introduce penalty to the weight function we wish to minimize.

Question 4

How do we maximize the margin in SVM?

Answer 4

We minimize the weights so min(1/2)(W^2) or min(1/2)(W^2+C) if penalty.

Question 5

How do we handle multi-class classification with SVM?

Answer 5

We can do this in 2 ways:

1) Learn 3 classifiers + vs. {o, -} - vs. {o, +} o vs. {+, -}
2) Learn 3 sets of weights simultaneously