L8 - Gradient Descent and Classifier Performance

Question 1

1. What is fitting a model?

Answer 1

1. The process of finding parameters such that the model fits to to the data.

Question 2

2. In classification, what are 2 sub-optimal approaches to fitting a model?

Answer 2

1. Random Sampling Values
   
   2. Grid Search

Question 3

3. Through what process can we improve parameter fitting?

Answer 3

1. Learning
   
   2. Minimise error and maximise fit

Question 4

4. When parameter fitting, what is the goal?

Answer 4

1. To minimise error of the model
   
   2. I.e find the minimum error point

Question 5

5. When establishing the minimum error using a Generic Parameter Fitting model, when do we stop?

Answer 5

1. When the error starts increasing again.

Question 6

6. In Deterministic Parameter Fitting, when do we stop?

Answer 6

1. When we reach N steps since everything is determined.

Question 7

7. Describe the Stochastic Parameter Fitting method…

Answer 7

1. Pick a random point P1 and calculate L2 norm
   
   2. Pick a random point P2 near P1 and calculate L2 norm
   3. Repeat until L2 norm is less than an error threshold or whether N steps can been taken.

Question 8

8. What is the purpose of the gradient descent algorithm?

Answer 8

1. Iteratively find the minimum of a function or model.
   
   2. In this context, to find the minimum error of the classification model.

Question 9

9. When does the gradient descent algorithm stop?

Answer 9

1. When the error is 0

Question 10

10. Describe the steps of the gradient descent algorithm

Answer 10

1. Start at random point P1
   
   2. Calculate loss of P1
   3. Choose P2 in the direction where the loss has the steepest gradient
   4. Repeat until loss value is below a threshold or N steps have been taken

Question 11

11. Why do we want to minimise the loss function of a model?

Answer 11

1. In order for the model to better fit the data

Question 12

12. What is Precision? And what does it mean if a model has High Precision?

Answer 12

1. Positive Prediction Value -> I.e Those labeled as positive is indeed positive
   
   2. Confusion matrix shows few number of False Positives

Question 13

13. What is Recall? And what does it mean if a model has High Recall?

Answer 13

1. Recall is sensitivity -> Out of all the actual positive instances, how many did the model correctly identify?
   
   2. Measures how well the model avoids FN’s.
   3. Model is good at finding positive instances, but may not be precise.

Question 14

14. What are the equations for Precision and Recall?

Answer 14

1. Precision = TP / ( TP + FP )
   
   2. Recall = TP / ( TP + FN )

Question 15

15. What does it mean if a model has High Recall and Low Precision?

Answer 15

1. Most positive instances are classified, but there’s likely to be many false positives.

Question 16

16. What does it mean if a model has Low Recall and High Precision?

Answer 16

1. Misses a lot of positive instances, but the TP’s predicted are likely correct.

Question 17

17. What is the F1 measure?

Answer 17

1. Score that combines the Precision and Recall measures.

Question 18

18. In a classification model, what is the difference between Specificity and Sensitivity?

Answer 18

1. Sensitivity -> Measures the models ability to identify positive instances, but may not be precise.
   
   2. Specificity -> Measures TN accuracy.

Question 19

19. What does the ROC curve show?

Answer 19

1. Combines specificity and sensitivity to show the trade off between TP and FP rate

Question 20

20. What is the threshold of the ROC curve? How do we decide it?

Answer 20

1. Threshold is hyper parameter that is set to create new TP and FP values.
   
   2. Threshold is a design choice -> Asks the question, which mistake is worse, a false positive or a false negative?

Question 21

21. Regarding TP and FP, what is the perfect classifier?

Answer 21

1. High TP and Low FP

Question 22

22. What is the ideal value of Area Under Curve?

Answer 22

1. Close to 1 as possible -> Higher TP rate

Question 23

23. How is a ROC curve created?

Answer 23

1. Iteratively move threshold across the classification graph, calculating the sensitivity and specificity at each point.
   
   2. Plot the results on a scatter graph.
   3. The result is the ROC curve

Question 24

24. What is the main metric that can be calculated from the ROC curve?

Answer 24

1. Area under curve -> Tells us ability to calculate True Positives.