SVM and Neural Networks Flashcards
What are the key attributes of an SVM?
- Sparseness
- Convexity
Briefly, how does SVM work?
We want to separate data linearly.
- Define a margin (distance from line to the closest point
on each side)
- Maximize the margin, which will give you the largest
possible separation
- Support Vectors are the data points pushing on the
margin
What can you do about noisy datasets in SVM?
Introduce a slack variable in the margin. This makes the margin be soft instead of hard. If the margin is hard, no noise/outliers are allowed and the data can be mis-represented.
What can you do about non-linear datasets?
Can always map them to a higher dimension where it is linearly separable.
What is the motivation behind neural networks?
Based on human neurons and how they operate. Humans can solve complex problems quickly despite our neurons having a slow transmission time.
What is a perceptron?
The building block of a neural network, based on a neuron. Performs as a binary classifier if given a n-dimensional vector as input.
What is a MLP (Multi Layered Perceptron)?
Several layers of perceptrons connected. They function as universal approximators, and allow modeling of non-linear discriminant functions.
What is underfitting/overfitting?
Underfitting is when you have not trained the network enough, and it has issues recognizing things it should.
Overfitting is when you have overtrained, and the network only recognizes things it has previously trained on.
What is back-propagation?
The basic method of a simple neural network.
Three phases:
- Forward propagation (basically sends data forwards)
- Backward propagation (sends data backwards)
- Weight updating (looks at the data and makes
changes based on results)
The convergence velocity depends on three things:
- Complexity of the problem
- Complexity of approximating function
- Learning rate
