reliability

Question 1

reliability

Answer 1

• measure of consistency, if a particular measurement is made twice and produces same result then it is reliable
• rulers should find the same measurement for an object every time, unless it is broken
• if there is a change in the measurement, we would attribute that change to the object rather than the ruler
• if a psychology test or measure assessed something on a particular day (ie. intelligence), we would expect the same result on a different day unless the thing itself had changed
• psychologists are more interested in abstract concepts such as attitudes, memory, IQ etc.

Question 2

ways of assessing reliability - test retest

Answer 2

• involves administering the same test or questionnaire to the same person or people on different occasions
• if the test is reliable then results should be the same or at least very similar every time
• most commonly used with questionnaires and psychological tests but can also be applied to interviews
• must be sufficient time in between test and retest to ensure that, for example, participants cannot recall their answers to a survey but not so long that their attitudes, opinions or abilities may have changed
• with questionnaires or tests, the two sets of scores would be correlated to ensure they are similar
• if the correlation is significant and positive then the reliability is assumed to be good

Question 3

ways of assessing reliability - inter-observer reliability

Answer 3

• in observational research, one observer’s interpretation of events may be different to another’s (subjectivity, bias and unreliability)
• recommendation is that observers conduct their observations in teams of at least two
• inter-observer reliability must be established, for example by using a pilot study to ensure that behavioural categories are being applied in the same way
• data collected should be correlated to assess reliability

Question 4

ways of assessing reliability - measuring reliability

Answer 4

• measured using a correlational analysis
• correlation coefficient should exceed +.80 for reliability

Question 5

improving reliability - questionnaires

Answer 5

• a questionnaire that produces low test-retest reliability may require some items to be deselected or rewritten
• for example, if some questions are complex or ambiguous, they may be interpreted differently by the same person on different occasions
• one solution might be to replace open questions with closed, fixed-choice alternatives

Question 6

improving reliability - interviews

Answer 6

• best way to ensure reliability is to use same interviewer every time
• if this is not possible all interviewers must be trained so that, for example, no one is using questions that are too leading or ambiguous
• this is more easily avoided in structured interviews

Question 7

improving reliability - observations

Answer 7

• reliability can be improved by making sure that behavioural categories are operationalised and that they are measurable and self-evident
• categories should not overlap and all possible behaviours should be listed
• if categories are not created well, observers have to make their own judgements on what to record
• if reliability is low, researchers may need further training in using categories and may wish to discuss with one another

Question 8

improving reliability - experiments

Answer 8

• in order to compare performance of different participants, procedures must be the same every time
• in terms of reliability, concerned about standardised procedures