Cost Functions, Performance Measures and Optimization Flashcards
What is the Cost Function?
The cost function is also called the loss function.
It is the difference between the predicted values and the ground truth, y.
What is the cost function used for regression?
We call it RMSE. Root-Mean-Square-Error
What is the cost function that we use to classify?
We have used a modified RMSE that is modified so that it returns either 0 or 1 by multiplying by the ground truth
What is regularization?
Regularization is something we use to prevent overfitting in our model.
We add some information to the cost function to change the algorithm
What is overfitting?
Overfitting is when the learning rate is too high so that the algorithm will over adjust when near the best fitting vector.
It could also describe when the model has been trained with too much training sets so that it learns the noise and errors of the data as features.
With classifiers, usually, how many nodes should there be in each layer as we go towards the output layer?
There should be fewer and fewer nodes.
e.g if there are 10 input nodes there should be fewer nods in layer 1 and so on.
Which parameters affect the learning
number of hidden layers, number of nodes in those layers, number of training iterations. Biasses
What is a hyperparameter?
A hyperparameter is the parameters that affect learning, but now how it learns. So.. learning rate and regularization parameters.
What are the performance measures for regression models?
RMSE or Mean Absolute error.
What is the performance measures for classification
Accuracy
F1 scores
Youden’s Index
precision
What is the accuracy performance measure for classification models?
It measures how accurate your model is at predicting and classifying correctly.
“The proportion of true results among the total number of cases”
What is the issue with accuracy performance measure?
It might not be useful.
If we predict how many asteroids we can just say zero and be right 99 % of the time. But the model is not very useful then.
which activation functions would you use for classification?
sigmoid or tanh
When is accuracy a good performance measure to use?
When we want to classify almost evenly distributed categories.
It does not make sense to use in very imbalanced cases.
What is the precision performance measure?
It is how often your model predicts truly correct.
How often your model predicts a case correctly
When do we use the precision performance measure?
When we want to be very precise with our predictions.
when it is important that the cases that are predicted are right. compared to when you just want to be statistically right
What is the F1 performance measure?
The F1 measure is the harmonic mean.
It considers the precision, sensitivity and accuracy
What is the accuracy and precision of the following classification, in the case 99 % of asteroids will miss the earth
Our model predicts that no asteroids will hit the earth
The accuracy is 99 %.
The precision is 0 - because there are no cases correctly predicted. e.g. measured a specific asteroid and then predicting that case.
We have not successfully predicted the true positives.
If you are a police inspector and you want to catch criminals, you want to be sure that the person you catch is a criminal (Precision) and you also want to capture as many criminals (Recall) as possible. The F1 score manages this tradeoff.
okay thanks
What is Youden’s Index?
It is a performance measure that takes into account specificity and sensitivity. It can yield 1 or 0.
1 when there is no false positives or false negatives. (perfect test)
0 when the test gives equal weight to positive and negative results (equal amount of wrong and right predictions.)
