21. Reliability (A2) Flashcards
What is reliability?
Refers to how consistent the findings from an investigation or measuring device are. A measuring device is said to be reliable if it produces consistent results everytime it is used.
When may a measuring device produce different results but still be considered reliable?
When the object or person in the measuring device is measuring has changed in some way.
What are the ways of assessing reliability?
Test retest
Inter-observer reliability
What is test retest reliability?
A method of assessing the reliability of a questionnaire or psychological test by assessing the same person on two separate occasions. This shows to what extend the test produces the same answers – is consistent & reliable.
How much time should be left between the two tests in the test retest reliability situation?
Enough time so that the participant will be unable to recall his responses but not so much time that their opinions and attitudes may have changed significantly.
What is done to the data from the two tests check the reliability?
In the case of the questionnaire the two sets of scores would be correlated to make sure they are similar, if the correlation is positive and significant there is good reliability.
What is inter-observer reliability?
The extent to which there is agreement between two or more observers involved in observations of behaviour. This is measured by correlating the observations of two or more observers. The general rule is if the total number of agreements divided by the total number of observations is greater than 80 the data has high inter-observer reliability.
How many one check that a study is likely to have good inter-observer reliability before it’s conducted?
Running a pilot study of an observation may allow people to check that observers are applying behavioural categories in the same way.
How would the results from an inter-observer reliability test be used to check for reliability?
Like the test retest method of the data collected would be correlated to assess reliability a similar method would also apply to content analysis and interviews if there conducted by different people.
How can we improve the reliability of questionnaire?
Questionnaire that produces low test retest reliability may need some of the items and questions to be rewritten, complex or ambiguous questions may need replacing as will open questions. Fixed choice alternatives could replace them.
How can we improve the reliability of interviews?
The best way of ensuring reliability is to use the same interview each time. If this is not possible all interviewers must be properly trained so as not to ask leading or ambiguous questions structured interviews with fixed questions are generally more reliable.
How can we improve the reliability of experiments?
Lab experiments are generally reliable because of the strict control the research has over the procedure this control is more achievable in lab and field studies. In field studies precise replication of a particular method improve the reliability of findings.
How can we improve the reliability of observations?
Properly operationalising behavioural categories will improve the reliability of observations as will making behavioural categories measurable and self evident. Categories should not be overlapping and all possible behaviours should be covered under a checklist this removes the need to the individual judgements on what to record.
