0.14. Reliability and validity

Question 1

What is the definition of reliability?

Answer 1

• Is the tool consistent?
• Replicability, if you can do the study again and get the same results, it’s reliable

Question 2

Ways to assess reliability: Test-retest (experiment)

Answer 2

• do the test, wait a given time period and do it again, if reliable (0.80 co-efficient) the results should be the same/ similar

Question 3

Ways to assess reliability: Inter-rater reliability (interview) & Inter- observer reliability (observation)

Answer 3

• 2 or more raters/ observers
• agree behaviour categories (operationalised)
• sit and record separately
• at the end, they correlate the data, looking for 0.80 co-efficient to be reliable

Question 4

What is reliability measured in?

Answer 4

• correlation (by scattergram)
• always looking for a 0.80 co-efficient to be reliable (80% similar)

Question 5

Ways to improve reliability

Answer 5

• questionnaires: use closed/ fixed choice questions
• interviews: use a trained interviewer as they won’t give off cues through body language, won’t use ambiguous questions or leading questions or use the same interviewer
• observations: operationalise behaviour categories and they must not overlap
• experiments: use standardised procedures: ppts do the same thing in the same order - like in Milgram

Question 6

What is the definition of validity?

Answer 6

• measuring what you intend to measure

Question 7

What is internal validity?

Answer 7

• is the research tool measuring what it should- a pilot study should check this

Question 8

What is external validity?

Answer 8

• ecological: can the results apply to other settings?
• temporal: can the results apply over time?
• population: can the results apply to different groups of people?

Question 9

Ways to assess validity: concurrent validity

Answer 9

• do your study, then correlate against the data from a pre-industrial study
• if 0.80 co-efficient, your tool is valid

Question 10

Ways to assess validity: face validity

Answer 10

• on the face of it, does it measure what it intends to?
• ask and expert if it does

Question 11

Ways to improve validity

Answer 11

• questionnaire: add lie scales into questions or ensure confidentiality for honesty or if only measuring one question, add more in to reduce demand characteristics (distracts them)
• interviews: use a trained interviewer as they won’t give off cues through body language, won’t use ambiguous questions or leading questions or use the same interviewer
• observations: operationalise behaviour categories
• experiments: use standardised procedures- ppts do the same thing in the same order -> add a control group to compare to