Self Reports: Assessing And Dealing With Validity

Question 1

What might stop a questionnaire/interview accurately finding out what it sets out to find out

Answer 1

• people don’t know how they think/feel
• pop validity = low in questionnaires (only specific type of people do them)
• interviewer bias - interviewer’s expectations might influence the interviewees answers
• leading Qs
• forced choice Qs with limited answers may make participant answer in a way that they don’t actually feel
• interviewer bias in interpreting the open Qs answers
• Ambiguous answers
• interviewer effect - social desirability bias + influenced by gender/appearance of interviewer

Question 2

What are some of the solutions to the problems of Questionnaires/interviews

Answer 2

• make it forced choice
• do an interview instead
• do a questionnaire instead
• get an independent interviewer in, who is unbiased
• reword problematic Qs
• remove problematic Qs
• make it open Qs

Question 3

What are the 3 most appropriate types of validity related to self reports

Answer 3

• face validity
• content validity
• concurrent validity

Question 4

What is face validity

Answer 4

• does a self report look like its measuring what the researcher intended it to

Question 5

What is content validity

Answer 5

• does the self report measure what it intended to (assessed by an expert in that field)

Question 6

What is concurrent validity

Answer 6

• compares performance of the self-report with other well established validated ones. If they produce similar outcomes = valid

Question 7

How could you assess how valid something is from a face validity

Answer 7

Researcher looks at questionnaire and checks if the Qs look like they are going to measure what they say they will

Question 8

How could you assess how valid something is from a content validity

Answer 8

• researcher looks to see if Qs are relevant (and not off topic), then get an expert to check if the Qs are valid

Question 9

How could you assess how valid something is from a concurrent validity

Answer 9

Pilot study, give participants both your questionnaire and another published questionnaire: see if they’ll produce similar results

Question 10

Why might a questionnaire not be reliable

Answer 10

• people won’t always interpret scales the same way
• people giving socially desirable answers will lower consistency and reliability

Question 11

Why might an interview not be reliable

Answer 11

• interviewer bias: direct (through tone/gestures/leading Qs) and indirect (types of Qs used)
• semi-structured interview will be difficult to repeat due to the fact they’re different each time
• difference in style/gender/personality of interviewer will create different answers

Question 12

How can you assess if a self-report has high or low internal reliability

Answer 12

• Split half method
  1. Randomly select half the test items and placing them on form A and placing the other half on form B
  2. Therefore you have 2 forms of the same test
  3. To have good reliability, the scores from these tests should at least be 80% in agreement

Question 13

How can you deal with it if a self-report has low internal reliability

Answer 13

Remove/change problematic Qs

Question 14

How to assess if a self-report has high or low external reliability

Answer 14

• Test-retest method
  1. Give someone same questionnaire / interview to the same person on 2 separate occasions
  2. With a sufficient gap for the chance to forget
  3. If both yield same results = reliable

Question 15

How to deal with it if a self-report has low external reliability

Answer 15

1. Rewrite problematic Qs
2. Train interviewer

Question 16

What ethical issue might be raised when carrying out a self report

Answer 16

• risk of psychological harm (personal/invasive Qs)
• valid consent
• confidentiality (anonymity)
• risk to the participant’s values, beliefs, relationships, status or privacy

Question 17

What should researchers do when carrying out a questionnaire/interview to uphold ethical guidelines

Answer 17

• give consent statement (e.g. contains warnings for sensitive topics that are brought up)
• give the right to withdraw — make this clear
• do a debrief — check for consent/their wellbeing
• anonymity — keep data within the study (not sharing it)