Reliability

Question 1

What are ways we can assess reliability?

Answer 1

Test-Retest
Inter-observer reliability
Measuring reliability

Question 2

Outline test-retest as a way of assessing reliability

Answer 2

Administering same test/questionnaire to same person on different occasions
If test/questionnaire is reliable, then results obtained are same/similar each time administered
Most commonly used in questionnaires, psych tests & interviews
Must be sufficient time between test & retest so person can’t recall original answers

Question 3

Outline inter-observer as a way to assess to reliability

Answer 3

Observers watch same event, but record data independently, then correlated to assess reliability

Question 4

How is reliability measured?

Answer 4

Correlational analysis:
In both test-retest & inter-observer reliability, two sets of scores are correlated

Question 5

How do we improve reliability for questionnaires?

Answer 5

Should be measured using test-retest
Comparing 2 sets of data should produce correlation exceeding +.80
Questionnaire producing low test-retest reliability = some items to be ‘deselected’/rewritten

E.g. some questions are complex, so could be interpreted differently by same person on different occasions. Solution may be replace some open questions with closed questions, reducing complex

Question 6

How do we improve reliability for interviews?

Answer 6

Use same interviewer each time
If this isn’t possible, each interviewer must be properly trained, e.g. no leading questions
This is more easily avoided in structured interviewers where interviewer’s behaviour controlled by fixed questions. Unstructured interviews more free-flowing & thus less reliable

Question 7

How do we improve reliability of observations?

Answer 7

Ensuring behavioural categories properly operationalised, they’re measurable & self-evident
Categories shouldn’t overlap
If not, different observers can have different judgements of what to record & have inconsistent data
If reliability low = observers need further training or discuss selections with each other

Question 8

How do we improve reliability in experiments?

Answer 8

Procedure is focus of reliability
To compare performance of different parties procedures must be same throughout
Thus, experimenter concerned about standardised procedures