Support Vector Machines

Question 1

What is the length of the projection of x onto w if w is a unit vector?

Answer 1

w^Tx

Question 2

What is the margin?

Answer 2

Distance between the decision boundry (hyperplane) and the closest training point

Question 3

How is the modulus of a vector w written?

Answer 3

|| w ||

Question 4

What is || w ||?

Answer 4

sqrt(w^Tw)

Question 5

If the hyperplane is defined as wx + w₀ = 0, what is the distance from the origin to the hyperplane?

Answer 5

b = - w₀ / ||w||

Question 6

What is the perpendicular distance from a point x to the hyperplane w^Tx + w_o = 0?

Answer 6

(1 / ||w||) |w^Tx + w₀|

Question 7

What is the value of the margin under the constraint min_i | w^Tx_i + w₀ | = 1

Answer 7

1 / ||w||

Question 8

What is maximizing 1 / ||w|| the same as?

Answer 8

minimizing ||w||²

Question 9

What is the SVN optimization problem?

Answer 9

min_{<strong>w</strong>} ||w||²

such that y_i(w^Tx + w₀) >= 1 for all i

Question 10

What are the 2 good properties of the optimal weight parameters (for SVM’s)?

Answer 10

• They are linear function of the input and class labels
• Solution is sparse (optimal hyperplane determined by just a few examples)

Question 11

What are support vectors?

Answer 11

The few training examples that determine the hyperplane

Question 12

What is the problem if the data is not linearly seperable for SVM’s?

Answer 12

The optimization problem has no solution

Question 13

What can we add to solve the problem if the data is not linearly seperable (for SVM’s)?

Answer 13

Slack variables

Question 14

What is the SVM optimization problem (with slack variables)?

Answer 14

Minimize:

Question 15

What is k (power of slack variable) usually set to (SVM)?

Answer 15

1

Question 16

What is C in SVM optimization problem with slack variables?

Answer 16

Trade-off parameter, how important are the slack variables

Question 17

What does this measure (SVM’s)?

Answer 17

How well we fit the data

Question 18

Why does adding slack variables increase the number of support vectors?

Answer 18

Every non-zero slack variable adds a support vector (because slack puts the point on the margin)

Question 19

What is a kernal function?

Answer 19

k(x_i, x_j) = phi(x_i)^T phi(x_j)

Takes the feature vectors, transforms them into new feature space and takes the dot product

Question 20

Whats special about kernal functions (compared to basis expansion)?

Answer 20

Can work with infinite dimensions

Question 21

What is the form of a polynomial kernal function?

Answer 21

k(x_i, x_j) = (x_i x_j)^d

Question 22

What is the form a gaussian radial basis function kernal function?

Answer 22

k(x_i, x_j) = exp(-||x_i - x_j||² / c)

Question 23

What does || w || mean?

Answer 23

The magnitude of vector w

Question 24

What do slack variables do (SVM)?

Answer 24

Move misclassified points to the margin