RM - Reliability Flashcards
Inter-observer reliability
The extent to which there is agreement between two or more observers involved in observations of a behaviour.
Reliability
Is consistency - The consistency of measurements. We would expect any measurement to produce the same data if taken on successive occasions.
Test-retest reliability
The same test or interview is given to the same ppts on two occasions to see if the same results are obtained.
What are observations a form of?
Measurement.
What happens in observations?
A researcher is recording (i.e. measuring) what people (or animals) are doing. The researcher will keep a record of events observed using a set of behavioural categories.
How do researchers record data in observations?
The researcher will keep a record of events observed using a set of behavioural categories.
What are ways of ensuring reliability of results from observations?
Observer should repeat the observations a second time (e.g. by watching a video recording).
Have two or more observers making separate recording and then compare these records.
What would the results of the second observation be like if the observations are reliable?
Then the second set of observations should be more or less the same as the first set.
Why is the observer repeating the observations a second time (e.g. by watching a video recording) not always a good way to make sure data is reliable?
There is a problem as the observer may have been bias.
What is a better way of testing reliability than an observer repeating observations for a second time?
Have two or more observers making separate recording and then compare these records.
What is the extent to which the observers agree on the observations they record called?
Inter-observer reliability.
What can inter-observer reliability be calculated as?
A correlation coefficient for pairs of scores. A result of 0.8 or more suggests good inter-observer reliability.
What correlation coefficient score shows that something has good inter-observer reliability?
0.8
How can reliability/the score for inter-rater reliability be improved (in observational techniques)?
- Make the behavioural categories clearer (as it may be that the behavioural categories were not operationalised clearly enough, so one observer interpreted an action as ‘hitting’ whereas another interpreted it as ‘touching’).
- It may be that some observers just need more practice using the behavioural categories so they can respond more quickly.
What are questionnaires and interviews an example of?
Self-report techniques.