Unit 3 - Evaluation And Visualization Of IR

Question 1

What is performance evaluation?

Answer 1

The measure of how much the retrieved documents are relevant to the users query.

Question 2

Categories of the results obtained from a information retrieval system:

Answer 2

Relevant and retrieved
Relevant and not retrieved
Non-relevant and retrieved
Non-relevant and not retrieved

Question 3

What are relevant items?

Answer 3

Document which are actually useful to the user.

Question 4

What is precision?

Answer 4

Ratio of number of relevant and retrieved documents with the total number of retrieved documents

P = (A / (A + C))x100%

Question 5

What is recall?

Answer 5

Ratio of number of relevant and retrieve documents to the total number of relevant documents in the database.

R = (A /(A + B))x100%

Question 6

Problem with precision and recall

Answer 6

They are inversely proportional to each other
Need of detailed knowledge about The relevant items in the database

Question 7

Mean reciprocal rank (MRR)

Answer 7

It is the rank aware evaluation metric i.e. it considers the relevance as well as the rank of the documents

Comprises of binary relevance based metric

Question 8

Algorithm of MRR:

Question 9

What is F-measure?

Answer 9

Measure that combines the recall and precision.
Also known as harmonic mean.

Question 10

Normalise discounted cumulative gain (NDCG)

Answer 10

It is a measure of ranking quality.

Question 11

Assumption that needs to be kept in mind for NDCG

Answer 11

Highly relevant documents are more useful than moderately relevant documents which are in turn more useful than irrelevant documents 

Question 12

To find NDCG, calculate

Answer 12

Cumulative gain
Discounted cumulative gain
Ideal discounted communicative gain
Normalised discounted cumulative gain