Chapter 5: Loss Functions Flashcards
what is a loss function
describes how well a model fits training data
what are other names for loss function
error
cost
objective
what do we do with the loss function
aim to minimise or maximise it
give 5 loss functions
sum of squares error
mean squared error
hinge loss
cross entropy loss
likelihood, log likelihood
describe sum of squares error
1/2 (sum (Yi - yi)^2)
how do we calculate classification error
classes are calculated using thresholding
we use one hot encoding for multi class classification
what is regularisation
it prevents overfitting by drawing attention away from the error function
by adding a regularisation term to the error function
give a regularised least squares equation
normal squares + param/2 w^t w
give l1 regularisation
least squares + param/2 sum of dimensions |wj|^1
give l2 regularisation
least squares + param/2 sum of dimensions |wj|^2
what is another name for l2 regularisation
lasso
what is another name for l2 regularisation
ridge regression
give mean squared error
1/N (sum to N (Yi - yi)^2)
what is hinge loss
models the classification error
give hinge loss
sum to N( max(0, 1- Yi*yi))
