Reliability Flashcards
What is reliability?
-If results are reliable they’re said to be consistent
-If a study was repeated using same methods, design and measurements you’d expect similar results
How can observations be unreliable?
-If observer is biased or not well trained to spot the target behaviour
-Result = observer providing their own interpretation of target behaviour and so other observers may observe it differently
What is inter-rater/observer reliability?
-Measuring extent to which different observers achieve similar results when observing and scoring same participants
-They would record their data individually, then sets of data obtained from each observer are correlated to est degree of similarity in scores
-Behaviours can also be recorded on video/audio tape and watched back
-Observer reliability is achieve if significant positive correlation (above +0.8) between scores of observers
How can reliability be improved using inter-rater reliability?
Observer training = observers should all be trained thoroughly in the techniques they are required to use
Behaviour categories can be operationalised:
-Definitions of key terms involved should be clear and understood fully
-This ill involve using fully operational behavioural categories so ‘aggressive behaviour’ is broken down into specific measurable categories such as ‘shouting’, ‘kicking’
How can self-report techniques be tested to assess reliability?
Test re-test method:
-Used to assess consistency of a test or questionnaire over time
-Involves presenting same participants with same test or questionnaire on 2 different occasions, with no feedback after the first
-The time interval between presentation must be selected carefully - if too short may remember answers, if too long then may have changed in a way relevant to the test or questionnaire
-Both tests are assessed using exact same scoring criteria
-Correlational techniques are used to indicate consistency of scores - one exceeding +0.8 indicates reliable responses
How can reliability be improved in self-report methods?
Reducing complexity/ambiguity of questions:
-Low reliability in a test may be because test items are ambiguous so may be interpreted differently
-Replacing some of the open Qs with closed, fixed choice alternatives would solve this
Using same interviewer:
-If this is not possible or practical, all interviewers must be trained properly e.g. one is not asking questions that are too leading or ambiguous when another is not
Use of structured interviews/ pre-determined Qs:
-Interviewer’s behaviour is more controlled by fixed Qs
-Interviews more free-flowing would be less likely to be reliable
How is reliability within experiments different to observations or self-report methods?
More about precise replication of a particular method rather than demonstrating reliability of a finding
How can the reliability of experiments be assessed?
Test-retest method
How can the reliability of experiments be improved?
Standardisation of procedures:
-Lab experiments often described as reliable as researcher cab exert strict control over many aspects of the procedure
-This contol is more often achievable in a lab than field exp
