Validity And Reliability: Observations And Inter-rater Reliability

Question 1

What key word is used to define reliability

Answer 1

Consistency

Question 2

What is external reliability

Answer 2

Consistency over time
—> get same results across different occasions with the same person

Question 3

In order to check if results are consistent over time we use the ______ _______

Answer 3

Test-retest method

Question 4

What is the test-retest method

Answer 4

Giving the same test on a 1st and 2nd occasion to the same person, and checking if results are the same

Question 5

What are some of general principles that can be addressed to improve external reliability

Answer 5

• standardisation: standardises instructions + procedures = all participants have same experience
• control over extraneous variables: so that different extraneous variable don’t cause a lack of consistency
• measurement: use an objective scientific standardised measuring tool so that there’s consistency in measurement

Question 6

How can researcher bias be linked to reliability, how do we control it?

Answer 6

• researcher bias = no consistency over time
• Double blind / getting in independent researchers

Question 7

What is inter-rather reliability

Answer 7

Consistency between observers (agreeing on behaviours observed)

Question 8

How do we check for inter-rather reliability

Answer 8

• observations should be correlated
• should be more than 80% agreement on the observed behaviours = data has inter-rather reliability

Question 9

To improve the inter-rather reliability of an observation we want the observers to what? And how might we achieve this?

Answer 9

• consistently mark the same behaviours
  1. Pilot study: see what types of behaviours occur so we can develop operationalised behavioural categories, so that observers will be more consistent
  2. Train observers on using the behavioural checklist