Using Interobserver Agreement to Assess Behavioral Measurement

Question 1

What is IOA?

Answer 1

Interobserver agreement (IOA) refers to the degree to which two or more independent observers report the
same observed values after measuring the same events.

Question 2

What are the benefits and uses of IOA?

Answer 2

1. Competence of new observers 2. Detect Observer drift 3. Confirms definition was clear and system not too difficult. 4. Decreases likelihood variability is due to observers.

Question 3

What do the four benefits of IOA lend?

Answer 3

Proactive observer accountability, believability of data.

Question 4

What are the requisites for obtaining valid IOA measures?

Answer 4

Two observers use the same code and measurement system, observe and measure the same participants and events, and observe and record the behavior independent of any influence from one other.

Question 5

What is the most common convention for reporting IOA in ABA?

Answer 5

Percentage of agreement.

Question 6

What are the various methods for calculating interobserver agreement for data obtained by event recording?

Answer 6

Total IOA, mean count-per-interval IOA, exact count-per-interval IOA.

Question 7

What is total count IOA?

Answer 7

Total count IOA is
expressed as a percentage of agreement between the total number of responses recorded by two observers and is calculated by
dividing the smaller of the counts by the larger count and multiplying by 100, as shown by this formula: Smaller count —– X 100 = total count IOA %

Question 8

What is a consequence of total IOA?

Answer 8

Does not show that they measured the same events at the same count.

Question 9

What is mean count per interval IOA?

Answer 9

1. Divide total observation time into smaller counting times 2. Record the number of occurrences of the behavior within each interval 3. Calculate agreement within each interval 4. Use agreements per interval as basis for calculating IOA for total observation period.

Question 10

Give the formula for mean count per interval IOA.

Answer 10

(Int 1 IOA + Int 2 IOA + Int N IOA )/ n intervals = mean count-per-interval IOA %

Question 11

What is exact count-per-interval IOA?

Answer 11

the
percentage of total intervals in which two observers recorded
the same count.

Question 12

What is the formula for exact count per interval IOA?

Answer 12

(Number of intervals of 1OO% IOA)/
n intervals
X 1OO = exact count-per-interval IOA %

Question 13

What is trial-by-trial IOA?