Lecture 2 Flashcards
What is needed to assess the performance of a vision system?
Ground truthed data
How should ground truth data be generated?
Several experts reaching a consensus
What should be the relationship between data used for developing a system, and that used for testing it?
Disjoint - can’t use same data
How do almost all vision systems operate?
Take in images and yield labelled results
What are the 4 possible outcomes/statuses of a label?
True positive
False positive
True negative
False negative
What is an ROC curve?
Receiver-operating characteristic curve:
Plots false positive rate against true positive rate as a tuning parameter is varied
What does a flat line represent on an ROC curve?
Random results; a poorly performing algorithm
How do you compare the performance of two algorithms?
Statistical test - McNemar’s test
How does McNemar’s test work?
Draw a table of Success and Failure for the two algorithms:
Alg 1 Alg 2 S F S Nss Nfs F Nsf Nff
Then Chi2 = (|Nsf-Nfs|-1)^2
(Nsf+Nfs)
If Chi2 > 1.96 the difference in performance is significant at the 5% level
How does content-based image retrieval work?
Looks at histograms, similar histograms indicate a match
Can use correlation coefficient on histograms; the closer the result to 1, the closer the image matches
What is the range of values produced by correlation coefficient?
-1 <= r <= 1
Give two example graphs for correlation coefficient (values -1 and 1)
What is the formula for correlation coefficient?
How does a confusion matrix work?
Expected along top, actual along side, plot number of occurrences
