Models Flashcards
What are the two approaches to creating a probabilistic model?
- Directly modelling the posterior p(f|x)
- Construct the likelihood and prior, and then compute the posterior using Bayes theorem
p(f|x) = p(x|f) p(f) / p(x)
Why are probabilistic models sometimes preferred over deterministic models?
Probabilistic models give us an output with a quantified uncertainty
What are some advantages of a deterministic model over a probabilistic model?
It is cheaper to train, requires less memory and resources.
Using a linear classification model, what is a non-probabilistic method we can use?
A separating hyperplane
Using a linear classification model, what is a probabilistic method we can use?
Logistic regression or soft-max for multi-class output
What is a non-parametric model?
A model with no parameters to optimise.
e.g. a K-NN model
What is a parametric model?
A model with parameters to optimise
What is the decision boundary in a classification model?
When classifying we take the output of our model and threshold on some value t.
The decision boundary is when the output of our model is equal to t.
f(X) = t
f(X) >= t CLASS 1
f(X) < t CLASS 2
How do we deal with non-linear data?
We find a transformation such that the data points are linearly separable.
And then we apply the linear model to the points in this new space.
What is the Linear Basis Function Model?
This is where we have a non-linear basis function for each feature.
After applying the basis function to each feature and weighting, we can then apply a linear model.
What does the inner product of two vectors represent?
This is a measure of how similar two vectors are.
How much is one vector pointing in the direction of the other?
What is the kernel method/trick?
This avoids directly defining a mapping function from a non-linear space to linearly separable space.
It does this by computing the inner product function of the transformed points directly.
What is the function for logistic regression?
Y = [ e^x ] / [ 1+ e^x ]
