Quality in qualitative research

Question 1

Why is quality important?

Answer 1

• An understanding of quality is vital in academic work. It guides what researchers do and how research outputs are used
• As students – should this study make a contribution to my argument?
• As clinicians, commissioners, policy makers, publishers, funders – should this study be published, funded, shape a policy or service?
• As researchers – how can I ensure that my study yields credible and useful findings

Question 2

How is quality assessed in quantitative research?

Answer 2

• Positivist-empiricist, hypothetico-deductive, quantitative psychological research:
• Reliability
• Internal validity
• Criteria rely on an assumption of objectivity
• Research paradigm aims to limit bias – i.e. any deviation from objective truth or fact
• Applying to qualitative research can be problematic

Question 3

Regarding quality in qualitative research what is it important to have criteria that do?

Answer 3

Important to have criteria that evaluate how well a study meets the goals of Qualitative research

Question 4

What are the different types of quality criteria for qualitative research?

Answer 4

 Quantitative scoring systems
 Criteria for specific types of analysis
 Flexible criteria for qualitative research
 Each study evaluated within unique context of the research problem

Question 5

What did Yardley do (2000) regarding quality in qualitative research?

Answer 5

• Proposed a set of flexible principles for evaluating the quality of a qualitative study, whilst remaining sensitive to the diversity of qualitative approaches:
• Sensitivity to context
• Commitment and rigour
• Transparency and Coherence
• Impact and importance

Question 6

What is sensitivity to context?

Answer 6

• Awareness of broader context that the research is conducted in
   The relevant literature and previous related empirical work
   ‘Common sense’ concepts and assumptions (e.g. philosophical stance)
   Socio-cultural setting (of all participants, including the researcher)

Question 7

What are features of commitment and rigour?

Answer 7

• completeness of data collection
• completeness of analysis
• triangulation
• validation

Question 8

What should you aim for with commitment and rigour?

Answer 8

Should aim for a complete interpretation that addresses all of the variation and complexity observed in the data, and produces high quality themes. Can require prolonged engagement with the data and iterative cycles of analysis phases

Question 9

What are things to keep in mind when considering the completeness of data collection?

Answer 9

 There is no magic number
 Samples are purposive – collect enough data to address the question
 Aim for data saturation – the point at which no new ideas are drawn from the data (concept taken from grounded theory)

Question 10

What are common problems with the completeness of analysis?

Answer 10

 Themes paraphrase data without providing analytic narrative
 Themes are summaries of interview questions or accounts from a single interviewee
 Themes are unrelated, overlap, vague, not consistent with data examples
 Alternatives are unconsidered (e.g. alternative interpretations of the data or negative cases within the data)

Question 11

What is triangulation?

Answer 11

• Combining methods of data collection and analysis to gain a multi-layered understanding of the research topic
   Might involve gathering data from various sources (e.g. patients, doctors, nurses)
   Might involve combining analytic approaches

Question 12

What is validation and what are the different types?

Answer 12

checking the interpretation of the data with others can increase the credibility findings
 Peer verification – analysts working together to check that interpretations are plausible, consistent with the data, communicated clearly
 Respondent verification – study participants reviewing analysis findings to comment on fit between analysts interpretation and their experiences

Question 13

What should transparency and coherence consider?

Answer 13

• auditability
• reflexivity
• Findings present a coherent narrative that is consistent with the quotations presented in themes
• Discussion links findings to existing knowledge
• Good fit between research question and the philosophical perspective adopted, and the method of investigation and analysis undertaken

Question 14

what is auditability?

Answer 14

 Method details, every aspect of the data collection process, the rules used to code data (e.g. sematic vs latent) how stages of the analysis progressed
 Findings present excerpts of the textual data so the readers can themselves discern the patterns identified by the analysis

Question 15

What is reflexivity?

Answer 15

 Discussion of the experiences or motivations which led the researcher to undertake a particular investigation. Their assumptions, intensions and actions
 Papers often include a reflexive statement

Question 16

What are the features of impact and importance?

Answer 16

• Discussion (and abstract) explains why the findings are important and their potential impact
   Theoretical impact – e.g. how should current theories be updated in light of the findings?
   Practical impact -e.g. how should services and policies be updated in light of the findings?
   Social-cultural impact – e.g. what do the findings indicate about how we think about social problems?
• Can also be demonstrated by discussing transferability
   The degree to which the results of qualitative research can be transferred to other contexts or settings with other respondents
   The researcher facilitates the transferability judgement by a potential user through thick description e.g. full description of the participants and socio-cultural setting
   But ultimately the ‘user; of the research decides whether the findings are transferrable to their research problem

Question 17

If reviewing a paper what should you consider?

Answer 17

 Have they explained their method and analysis in enough detail?
 Have they been transparent about their assumptions (e.g. provided a reflexive statement)?
 Is their analysis convincing?
 Why are the study findings relevant to your project/ essay?

Question 18

What is mixed methods research?

Answer 18

• A systematic approach to data collection and analysis that integrates different sources of data and qualitative and quantitative analytical procedures with the intention to engage multiple perspectives in order to more fully understand complex social phenomenon (Creamer and Reeping, 2020)
• Embraces two different analytical logics:
   An exploratory/ hypotheses generating one
   A confirmatory/ hypotheses confirmation one
• Produces findings that are greater than what can be produced separately by its parts (Fetters and Molina-Azorin, 2017)

Question 19

What is the prevalence of mixed methods research?

Answer 19

• 1992-2016 percentage of empirical mixed methods publications in all fields slightly more than 18% (Timans, Wouters, and Heilbron, 2019)
• Lower in psychological research
   7% of the empirical articles in top-tier APA journals reporting on basic research integrated qualitative and quantitative methods (Alise and Teddlie, 2010)
   13.7% in psychology journals with applied focus (Powell et al., 2008)

Question 20

What are the purposes of mixing methods?

Answer 20

• triangulation
• complementarity
• development
• initiation
• expansion

Question 21

What is triangulation?

Answer 21

Different methods used to examine the same phenomenon, increasing confidence in conclusions reached

Question 22

What is complementarity?

Answer 22

Different methods investigate different aspects/ dimensions of same phenomena to deepen/ broaden interpretations and conclusions

Question 23

What is development?

Answer 23

Results from one method used to inform development of other method

Question 24

What is initiation?

Answer 24

Different methods used to investigate different aspects/ dimensions of same phenomenon

Question 25

What is expansion?

Answer 25

Different methods used to assess different phenomena to expand scope/ range of study

Question 26

What is considered in the design of mixed method research?

Answer 26

The order and dominance of Qual-Quant

Question 27

What would the mixed method be if it was equal and concurrent?

Answer 27

QUAL & QUANT

Question 28

What would the mixed method be if it were equal and sequential?

Answer 28

QUAL QUANT

QUANT QUAL

Question 29

What would the mixed method be if it were dominant and concurrent?

Answer 29

QUAL & quant

QUANT & qual

Question 30

What would the mixed method be if it were dominant and sequential?

Answer 30

QUANT qual
Quant QUANT
QUAL quant
Qual QUANT

Question 31

What is the purpose of having a preliminary qualitative method for a principle quantitative method study?

Answer 31

Purposes: smaller qualitative study helps guide data collection in a principally quantitative study:
- Can generate hypotheses, develop content for questionnaires and interventions

Question 32

What is the purpose of having a preliminary quantitative method for a principle qualitative method study?

Answer 32

Purposes: smaller quantitative study helps guide the data collection in a principally qualitative study
- Can guide purposive sampling, establish preliminary results to pursue in depth etc

Question 33

What is the purpose of having a qualitative follow up after a principle method quantitative study?

Answer 33

Purpose: smaller qualitative study helps evaluate and interpret results from a principally quantitative study
- Can provide interpretation for poorly understood results, help explain outliers ect

Question 34

What is the purpose of having a quantitative follow up after a principle method qualitative study?

Answer 34

Purposes: smaller quantitative study helps evaluate and interpret results from a principally qualitative study
- Can generalise results to different samples, tests elements of emergent theories etc

Question 35

What should you consider when thinking about design decisions?

Answer 35

• Led by research question and current knowledge on a research topic, commonly:
• Quantitative for validating existing knowledge and assumptions
• Qualitative for exploratory insights