Validity

Question 1

Researcher bias

Answer 1

values and opinions of researcher are strong, so they can affect objectivity

This risk can be addressed through having more than one researcher interviewing respondents

Question 2

Respondent bias

Answer 2

values, opinions, memory, honesty, etc. of respondents.

This risk can be addressed through asking same questions to multiple respondents

Question 3

Recollection bias

Answer 3

arises when questions asked refer to an old event, as our memory is not reliable and with time we forget the details or our memories change.

This risk can be addressed by asking same questions to multiple respondents

Question 4

Construct validity

Answer 4

whether operationalization of construct is good / sound / reasonable OR whether it measures what it is intended to measure.

Ways to address this risk:
- Operationalize constructs using documents and literature
- Multiple sources of data – triangulation of sources
- Multiple informants – triangulation of informants

Question 5

Internal validity

Answer 5

does data support developed theoretical ideas and conclusions OR is there strong link between data and generated theory.

Ways to address this risk: Pattern matching or explanation building

Question 6

External validity

Answer 6

to which extent findings are generalizable beyond particular research context.

Ways to address this risk: Replication logic in multiple case studies

Question 7

Quality of questionnaire

Answer 7

quality of questions (content validity – do questions cover what researchers intend to cover; construct validity – do the construct measure what they intend to measure; clarity of instructions for respondents.

Ways to address this risk is the use of previously developed scales and ways to operationalize constructs OR checks of clarity of instructions and questions (pre-test) OR Cronbach alpha calculation for multi-item constructs (this is reliability check!)

Question 8

Sample quality and sampling bias

Answer 8

sample is not representative of the population.

Ways to address this risk: construction of good sample frames (if random sampling is used ) OR reflection on representativeness of sample in non-random sampling

Question 9

Low response rate and non-response bias

Answer 9

that respondent who have not answered are systematically different from the received answers, so there is a risk to generalizability of findings to the population/external validity.

Ways to address this risk: sending reminders about surveys OR analysis of and reflection on this bias in the report

Question 10

Response incompleteness and missing values

Answer 10

incomplete results mean not all variables were properly captures in the survey, so analysis is not fully possible.

Ways to address this risk: discarding incomplete responses OR including attention checks in the questionnaire OR questions to evaluate knowledgeability of respondents