Lecture 6

Question 1

What causes irregular boundries?

Answer 1

Irregular distribution, imbalanced training sizes, and outliers

Question 2

What causes misclassifications?

Answer 2

unoptimized decision boundries

Question 3

Support Vectors

Answer 3

a subset of vectors that support or determine the boundry

Question 4

What is the goal of support vector machines?

Answer 4

to learn a boundary that leads to the largest margin (buffer) from points on both sides

Question 5

What points of the data set have an influence on the decision boundry (when using an svm)?

Answer 5

Only the support vectors. any point that isn’t a support vector has no influence and can be messed with while having no affect on the decision boundary.

Question 6

What data points do SVMs use to compute predictions?

Answer 6

Only the support vectors, not the whole training set

Question 7

What dimension is a decision boundary for a dataset with 2 features?

Answer 7

It is the third dimension and creates a 2D plane

Question 8

Where is the decision function equal to zero?

Answer 8

On the decision boundary

Question 9

When are inputs labeled as ‘undefined’?

Answer 9

When they lie between the margins i.e. between -1 and 1

Question 10

What is the goal of decision functions for SVMs?

Answer 10

to maximize the margin between the data points and the hyperplane

Question 11

How can optimal values of w and b be found?

Answer 11

through optimization via projective gradient descent

Question 12

What do your graph and results look like when parameters w and b are optimized?

Answer 12

The algorithm correctly classifies the training examples and the margin is maximized

Question 13

In a margin-based classifier, what happens to the margin when the weight vector w gets smaller?

Answer 13

The margin gets larger

Question 14

In hard margin SVMs, do we minimize ‘1/2 abs(w)^2’ or ‘abs(w)’ and why?

Answer 14

1/2 abs(w)^2 because abs(w) is not differentiable at w = 0

Question 15

What do you do when the data is not linearly separable?

Answer 15

Introduce slack variables and allow “error” in classification

Question 16

What does the data have to look like for you to not be able to use a hard margin SVM?

Answer 16

A margin can’t cleaning split the data without leaving, for example, a blue piece on the red side. In other words, if blue pieces separate a red piece from the other red pieces.

Question 17

What are the two conflicting elements of soft margin SVMs?

Answer 17

1. making the slack variables as small as possible to reduce the margin violations and
2. making w^T * w as small as possible to increase the margin

Question 18

What does a kernel function map?

Answer 18

It maps the low dimensional data to high dimensional space

Question 19

What type of function is the kernel function?

Answer 19

a similarity function

Question 20

What’s the relationship between the weight vector w and the margin?

Answer 20

The smaller the weight vector w, the larger the margin

Question 21

When using RBF kernel in SVM what does a high Gamma value signify?

Answer 21

The model would consider only the points close to the hyperplane for modeling.

Question 22

Gamma parameter

Answer 22

The gamma parameter in SVM tuning signifies the influence of points either near or far away from the hyperplane. For a low gamma, the model will be too constrained and include all points of the training dataset, without really capturing the shape. For a higher gamma, the model will capture the shape of the dataset well.

Question 23

Disadvantages of SVMs

Answer 23

It is not suited to larger datasets as the training time with SVM’s can be high

It is less effective on noisier datasets with overlapping classes

It was originally designed as a 2-class classifier