Reliability

Question 1

Reliability

What does reliability equal?

Answer 1

If a particular measurement is repeated and the same result is obtained then that measurement is described as being reliable.

Reliability= consistency

Question 2

Test-Retest

Answer 2

Test the same person twice.

The same test or questionnaire is given to the same person (or people) on two or more different occasions.

If the test or questionnaire is reliable the results should be the same (or very similar) each time it is administered

Question 3

Inter-Observer

Answer 3

Compares observations from different observers.

In an observation, two or more observers compare their data by conducting a pilot study- a small scale trial run of the observation to check that observers are applying behavioural categories in the same way.

Observers should watch the event, or sequence of events, but record their data independently

Question 4

Measuring Reliability

Answer 4

Reliability is measured using a correlation.

In test-retest and inter-observer reliability, the two sets of scores are correlated.

The correlation coefficient should exceed +.80 for reliability

Question 5

Improving the Reliability of Questionnaires

Answer 5

Rewrite the questions.
Questionnaires that produce low test-retest reliability may need some items to be deselected or rewritten.

The researcher may replace some open questions (which can be misinterpreted) with closed, fixed choice alternatives which may be less ambiguous

Question 6

Improving the Reliability of Interviews

Answer 6

Improved training.

The best way of ensuring reliability in an interview is to use the same interviewer each time.

If this is not possible, all interviewers must be trained (eg so they can avoid questions that are leading or ambiguous)

Question 7

Improving the Reliability of Experiments

Answer 7

Standardised procedures.

Lab experiments are often described as being the most reliable because of the strict control over many aspects of the procedure, such as the instructions that the participants receive and the conditions within which they are tested

Question 8

Improving the Reliability of Observations

Answer 8

Operationalisation of behavioural categories.

Behavioural categories should be measurable.

Categories should not overlap and all possible behaviours should be included.

If categories are overlapping or absent, different observers have to use their own judgment in deciding what to record and where, and may end up with inconsistent records

Question 9

Two Methods of Assessing Reliability

Answer 9

Test-retest

Inter-observer