lecture 10: performance issues

Question 1

we have learnt the building blocks of a learning algorithm, what is the next missing step?

Answer 1

validating the accuracy of the algorithm in terms of predicting novel data based on the limited data we have.
to do this, we can use a train, validation and test split of the data, and then cross validate

Question 2

what are the common partitioning folds for TVT splitting

Answer 2

2,4,5 and 10 fold CV

Question 3

what is the purpose of splitting data three ways?

Answer 3

the test set now contains data that the algorithm has never seen before, so if the model performs well on predicting the test set, we can say that the model generalises well

Question 4

what are the 2 most common evaluation metrics used to measure performance of regression learning algorithms?

Answer 4

mean square error and mean absolute error

Question 5

what are the evaluation metrics for classification learning algorithms

Answer 5

confusion matrix for binary classification, cost matrix for binary classification, decision error trade off, area under curve for receiver operating characteristic curve, gini coefficient

Question 6

what are some of the trade-offs involved with achieving high software quality

Answer 6

computational efficiency, maintainability, flexibility, extensibility, usability etc.