RM - Reliability

Question 1

Inter-observer reliability

Answer 1

The extent to which there is agreement between two or more observers involved in observations of a behaviour.

Question 2

Reliability

Answer 2

Is consistency - The consistency of measurements. We would expect any measurement to produce the same data if taken on successive occasions.

Question 3

Test-retest reliability

Answer 3

The same test or interview is given to the same ppts on two occasions to see if the same results are obtained.

Question 4

What are observations a form of?

Answer 4

Measurement.

Question 5

What happens in observations?

Answer 5

A researcher is recording (i.e. measuring) what people (or animals) are doing. The researcher will keep a record of events observed using a set of behavioural categories.

Question 6

How do researchers record data in observations?

Answer 6

The researcher will keep a record of events observed using a set of behavioural categories.

Question 7

What are ways of ensuring reliability of results from observations?

Answer 7

Observer should repeat the observations a second time (e.g. by watching a video recording).

Have two or more observers making separate recording and then compare these records.

Question 8

What would the results of the second observation be like if the observations are reliable?

Answer 8

Then the second set of observations should be more or less the same as the first set.

Question 9

Why is the observer repeating the observations a second time (e.g. by watching a video recording) not always a good way to make sure data is reliable?

Answer 9

There is a problem as the observer may have been bias.

Question 10

What is a better way of testing reliability than an observer repeating observations for a second time?

Answer 10

Have two or more observers making separate recording and then compare these records.

Question 11

What is the extent to which the observers agree on the observations they record called?

Answer 11

Inter-observer reliability.

Question 12

What can inter-observer reliability be calculated as?

Answer 12

A correlation coefficient for pairs of scores. A result of 0.8 or more suggests good inter-observer reliability.

Question 13

What correlation coefficient score shows that something has good inter-observer reliability?

Answer 13

0.8

Question 14

How can reliability/the score for inter-rater reliability be improved (in observational techniques)?

Answer 14

• Make the behavioural categories clearer (as it may be that the behavioural categories were not operationalised clearly enough, so one observer interpreted an action as ‘hitting’ whereas another interpreted it as ‘touching’).
• It may be that some observers just need more practice using the behavioural categories so they can respond more quickly.

Question 15

What are questionnaires and interviews an example of?

Answer 15

Self-report techniques.

Question 16

What are examples of self-report techniques?

Answer 16

Questionnaires and interviews.

Question 17

What are self-report techniques?

Answer 17

Research methods where the individual tells a researcher what they think or feel.

Question 18

What psychological tests are similar to self-report techniques?

Answer 18

IQ tests or personality tests.

Question 19

What is test-retest reliability used to assess?

Answer 19

The reliability of a psychological test or other self-report measure.

Question 20

What ways of assessing reliability of self-report techniques can be used?

Answer 20

Test-retest reliability.

Inter-interviewer reliability.

Question 21

How is test-retest reliability used to assess reliability of self-report techniques?

Answer 21

The test/questionnaire designer gives the test to a group of people and then gives the same group of people the same test a second time. Usually there is a short interval between the tests, such as a week or two, so that people don’t remember their answers. If the measure is reliable the outcome should be the same every time.

Question 22

How are the scores compared when assessing reliability of self-report techniques?

Answer 22

Using correlation.

Question 23

What is inter-interviewer reliability used to assess?

Answer 23

The reliability of one interviewer or the reliability (consistency) of two interviewers

Question 24

How is inter-interviewer reliability used to assess reliability of self-report techniques?

Answer 24

In the case of interviews, a researcher could assess the reliability of one interviewer by comparing answers on one occasion with answers from the same person with the same interviewer a week later.

Or, the researcher might want to assess the reliability (consistency) of two interviewers using the same method as with two observers.

Question 25

What way of improving reliability of self-report techniques can be used?

Answer 25

Reduce ambiguity.

Question 26

Why might there be low reliability in a psychological test?

Answer 26

Because some test items are ambiguous so people give different answers. For example, a question might be: What are your thoughts about dieting?' Some people might interpret this as being a question asking for factual information, and provide facts about dieting, whereas others might think the question was about emotions and respond with their own feelings about their efforts at dieting.

Question 27

How is reducing ambiguity used to improve reliability of self-report techniques?

Answer 27

Some test items are ambiguous so people give different answers. In such cases test items (or questions on a questionnaire) need to be re-examined and rewritten.

Question 28

How is the dependent variable in an experiment often measured?

Answer 28

Using a rating scale or behavioural catergories.

Question 29

Give examples in research of how the dependent variable in an experiment is measured using a rating scale or behavioural categories.

Answer 29

- Bandura et al. (1963) study on aggression using a Bobo doll. The dependent variable was the aggressive behaviour of the children, This was assessed by observing their behaviour in a room full of toys and using behavioural categories such as verbal imitation. - Rutter and Sonuga-Barke (2010) study on Romanian orphans used IQ scores as one of their dependent variables.

Question 30

What is reliability of experiments concerned with?

Answer 30

Whether the method used to measure the dependent variable is consistent, i.e. the observations or the self-report method. (The same is true of how co-variables in correlations are measured.)

Question 31

How can reliability of experiments be improved?

Answer 31

By standardisation.

Question 32

How can standardisation improve reliability of experiments?

Answer 32

It is important that the procedures are exactly the same each time (i.e. reliable) when repeated for different ppts because otherwise we can't compare the performance of ppts. For this reason procedures are standardised. If another researcher wishes to repeat the experiment, they also need to use exactly the same procedures.