Interpreting Measures

Question 1

Kendall’s tau

Answer 1

Correlation for Nominal data

Question 2

Spearman’s rho

Answer 2

Correlation for ordinal data

Question 3

Correlation

Answer 3

Degree of association between two sets of data

Does not:
Tell us the extent of agreement
Give a sufficient measure of reliability

Question 4

Percent agreement

Answer 4

The extent to which observers agree in their ratings

Does not consider agreement that would occur by chance - Need similar frequencies in each category

Question 5

Kappa Statistic

Answer 5

Indicates the proportion of agreement beyond that of expected chance (does not tell if there are random differences or systematic differences)

Used to express reliability for nominal or ordinal data

Only agreement beyond that expected by chance can be considered agreement

Represents ―average rate of agreement for the entire set of scores (doesn’t tell you where the discrepancies lie)

Examples: Inter-rater, Intra-rater, test-retest reliability

Question 6

Kappa Scores

Answer 6

Range from –1 to 1

1: perfect agreement
0: agreement no better than if raters guessed
Negative K: agreement worse than expected by chance

Interpretation is always dependent on situation
>.80 is ―excellent
>.60 is ―substantial
.40–.60 is ―moderate
<.40 is ―poor to fair