Reliability Flashcards
what is reliability?
reliability = consistency
it refers to how much we can depend on a particular measurement
a study/measurement is reliable if it is repeated and the same results are obtained more or less every time (i.e. it is consistent)
OBSERVATIONAL TECHNIQUES
assessing the reliability of observational techniques
reliability can be assessed by making the observer repeat the observations a second time
the observations are reliable if the second set of observations are more or less the same as the first
although using the same observer twice may not be very effective as the observer might be biased and this may be why the observations are the same both times
therefore, a better way to assess reliability is to get two or more observers to make separate observations then compare their records
the extent to which the observers agree on the observations they record is called inter-observer reliability
this can be calculated as a correlation coefficient — a result of 0.80 or more suggests good inter-observer reliability
OBSERVATIONAL TECHNIQUES
improving reliability of observations
behavioural categories can be improved in order to improve inter-observer reliability…
- it may be that the behavioural categories were not operationalised clearly enough — so one observer interpreted a behaviour as one thing while another interpreted it as something else, therefore behavioural categories need to be made more explicit
- it may be that some observers need more practice using behavioural categories so they can respond more quickly, therefore more training could be given to observers
OBSERVATIONAL TECHNIQUES
inter-observer reliability
the extent to which there is agreement between two or more observers involved in observations of a behaviour
worked out by plotting data from observers on a scattergram and calculating a correlation coefficient which may be done using a statistical test
SELF REPORT TECHNIQUES
reliability of self report techniques
researchers need to ensure that the self report methods being used are reliable and that they would obtain the same set of answers or the same score on a test every time
it needs to be ensured that any change in score or result is due to the person, not the test
SELF REPORT TECHNIQUES
assessing reliability of self report techniques: test-retest reliability
the test-retest method can be used to assess the reliability of self report measures
involves the same test being given to the same participants on two different occasions to see if the same results are obtained
usually there is a short interval between tasks, such as a week or two, so that people don’t remember their answers
if the measure is reliable, the outcome should be the same every time
the scores for each person can be compared using correlation
SELF REPORT TECHNIQUES
assessing reliability of self report techniques: inter-interviewer reliability
a researcher could assess the reliability of one interviewer by comparing answers obtained on one occasion with answers from the same person with the same interviewer a week later
or the researcher might want to assess the reliability of two interviews by having them interview the same person to see if similar answers are obtained
SELF REPORT TECHNIQUES
improving reliability of self report techniques
one way to improve reliability is to reduce ambiguity
low reliability in a psychological test may be because some test items are ambiguous so people have different answers
some people might interpret a question as asking for factual information whereas others might think the question is asking about emotions and will respond with their own feelings and personal experiences
in such cases, questions on the questionnaire need to be re-examined and rewritten
EXPERIMENTS
reliability of experiments
the DV in an experiment is often measured using a rating scale or behavioural categories
reliability in an experiment may be concerned with whether the method used to measure the DV is consistent
for example, the study by Bandura et al (1963) where the DV was the aggressive behaviour of the children which was assessed by observing their behaviour in a room full of toys and using behavioural categories such as verbal imitation
a study by Rutter et al (2010) on Romanian orphans used IQ scores as one of the DVs
EXPERIMENTS
improving reliability of experiments: standardisation
the procedures are often repeated for different participants
it is important that the procedures are exactly the same each time because otherwise we cannot compare the performance of participants
for this reason, procedures are standardised (made the same for every participant)
if another researcher wishes to repeat the experiment, they also need to use exactly the same procedures
