8. Observational Methods 2 - Coding and Reliability

Question 1

A coding scheme is a m____ i____

Answer 1

measuring instrument

Question 2

The outcome of the application of a coding scheme is a set of q____ data, and those data are then a____

Answer 2

quantitative, analysed

Question 3

Principles of measurement:
1. There is no such thing as p____ measurement
2. Measurement are more or less a____. A____ is about c____ or v____ measurement (the degree to which measurements a____ c____ the phenomena of interest)
3. Measurements are more or less p____. P____ is about e____ (the degree to which measurements are r____ or r____)

Answer 3

1. perfect
2. accurate, accuracy, correct, valid, actually capture
3. precise, precision, exactitude, reliable, replicable

Question 4

Intra-observer reliability means the s____ observer codes the same behaviour record at different times

Answer 4

same

Question 5

Inter-observer reliability means that d____ observers i____ cded the same behavioural record either at the same time or at different times

Answer 5

different, independently

Question 6

No strict standard currently exists for how much of a behavioural record to independently code, for purposes of reliability, but typically it is around __% to __% off the record.

Answer 6

15% - 20%

Question 7

Consensus estimates are based on the a____ that two or more coders can come to e____ a____. This is typically used on n____ data, where different codes represent q____ d____ ideas

Answer 7

assumption, exact agreement
nominal, qualitatively distinct

Question 8

Consistency estimates are based on the a____ that it is u____ for two or more coders to interpret a scale i____, but that the coders will c____ c____ phenomena with their understanding of the scale. Typically used on o____-to-c____ scales

Answer 8

assumption, unnecessary, identically, consistently classify, ordinal-to-continuous scales

Question 9

Percent agreement does not correct for agreement by r____ c____.

Answer 9

random choice

Question 10

Cohen’s kappa us the proportion of agreement after c____ for agreement by r____ c____

Answer 10

corrections, random chance

Question 11

Correlation coefficient does not take into account v____ between c____

Answer 11

variance, coders

Question 12

Cronbach’s alpha corrects for v____ between c____ and can e____ r____ for more than two coders

Answer 12

variance, coders, estimate reliability

Question 13

Codes should be m____ e____ (each behaviour can only be put into one category) and e____ (all possible behaviours have categories)

Answer 13

mutually exclusive, exhaustive