SVM Flashcards
What is the margin in the SVM algorithm
The margin is the distance from the line to the nearest point, and the SVM algorithm tries to maximize it. This also maximizes robustness of the algorithm.
SVM first considers classification errors and then maximizes the margin.
What is the Kernel Trick
The kernel trick consist in mapping features from a low dimensional input space where they are not linear separable, to a high dimenssional space. And then, having a better linear separation solution in the new high-dimenssional space, the boundary is mapped again to the original feature space.
What are the hiperparameters in SVM
The kernel, C and Gamma.
What is the impact of ‘gamma’ parameter in a SVM with linnear kernel?
The ‘gamma’ parameter actually has no effect on the ‘linear’ kernel for SVMs.
What does the ‘C’ parameter do in a SVM?
C parameter control the tradeoff between smooth decision boundary and classifying training points correctly
Does a large ‘C’ mean you expect a smooth boundary or that you will get more training points correct.
Having a bigger ‘C’ means haing more training points correct, and could lead to an overfitting in training data.
What does ‘gamma’ parameter do?
Gamma defines the influence of a point based on the distance from this point to de boundary.
Low values mean each point has a far reach and all are important to define the boundary.
High values mean that the boundary will only depend on the closest points.
With a high value of ‘gamma’ parameter in SVM do you expect the decision boundary to be smother or more jagged?
The decision boundary with a high value of ‘gamma’ would be more jagged, as the closest points to the boundary would have big weights and would be important to classify them correctly.
When is good to use SVMs and when is not? Why?
It’s good to use in complex domains where there is a clear margin of sepparation.
They don’t perform so well in large datasets because training time is cubic in the size of the datase, nor in datasets with lots of noise (very overlapping classes).
