Quality in qualitative research Flashcards
Why is quality important?
- An understanding of quality is vital in academic work. It guides what researchers do and how research outputs are used
- As students – should this study make a contribution to my argument?
- As clinicians, commissioners, policy makers, publishers, funders – should this study be published, funded, shape a policy or service?
- As researchers – how can I ensure that my study yields credible and useful findings
How is quality assessed in quantitative research?
- Positivist-empiricist, hypothetico-deductive, quantitative psychological research:
- Reliability
- Internal validity
- Criteria rely on an assumption of objectivity
- Research paradigm aims to limit bias – i.e. any deviation from objective truth or fact
- Applying to qualitative research can be problematic
Regarding quality in qualitative research what is it important to have criteria that do?
Important to have criteria that evaluate how well a study meets the goals of Qualitative research
What are the different types of quality criteria for qualitative research?
Quantitative scoring systems
Criteria for specific types of analysis
Flexible criteria for qualitative research
Each study evaluated within unique context of the research problem
What did Yardley do (2000) regarding quality in qualitative research?
- Proposed a set of flexible principles for evaluating the quality of a qualitative study, whilst remaining sensitive to the diversity of qualitative approaches:
- Sensitivity to context
- Commitment and rigour
- Transparency and Coherence
- Impact and importance
What is sensitivity to context?
- Awareness of broader context that the research is conducted in
The relevant literature and previous related empirical work
‘Common sense’ concepts and assumptions (e.g. philosophical stance)
Socio-cultural setting (of all participants, including the researcher)
What are features of commitment and rigour?
- completeness of data collection
- completeness of analysis
- triangulation
- validation
What should you aim for with commitment and rigour?
Should aim for a complete interpretation that addresses all of the variation and complexity observed in the data, and produces high quality themes. Can require prolonged engagement with the data and iterative cycles of analysis phases
What are things to keep in mind when considering the completeness of data collection?
There is no magic number
Samples are purposive – collect enough data to address the question
Aim for data saturation – the point at which no new ideas are drawn from the data (concept taken from grounded theory)
What are common problems with the completeness of analysis?
Themes paraphrase data without providing analytic narrative
Themes are summaries of interview questions or accounts from a single interviewee
Themes are unrelated, overlap, vague, not consistent with data examples
Alternatives are unconsidered (e.g. alternative interpretations of the data or negative cases within the data)
What is triangulation?
- Combining methods of data collection and analysis to gain a multi-layered understanding of the research topic
Might involve gathering data from various sources (e.g. patients, doctors, nurses)
Might involve combining analytic approaches
What is validation and what are the different types?
checking the interpretation of the data with others can increase the credibility findings
Peer verification – analysts working together to check that interpretations are plausible, consistent with the data, communicated clearly
Respondent verification – study participants reviewing analysis findings to comment on fit between analysts interpretation and their experiences
What should transparency and coherence consider?
- auditability
- reflexivity
- Findings present a coherent narrative that is consistent with the quotations presented in themes
- Discussion links findings to existing knowledge
- Good fit between research question and the philosophical perspective adopted, and the method of investigation and analysis undertaken
what is auditability?
Method details, every aspect of the data collection process, the rules used to code data (e.g. sematic vs latent) how stages of the analysis progressed
Findings present excerpts of the textual data so the readers can themselves discern the patterns identified by the analysis
What is reflexivity?
Discussion of the experiences or motivations which led the researcher to undertake a particular investigation. Their assumptions, intensions and actions
Papers often include a reflexive statement
What are the features of impact and importance?
- Discussion (and abstract) explains why the findings are important and their potential impact
Theoretical impact – e.g. how should current theories be updated in light of the findings?
Practical impact -e.g. how should services and policies be updated in light of the findings?
Social-cultural impact – e.g. what do the findings indicate about how we think about social problems? - Can also be demonstrated by discussing transferability
The degree to which the results of qualitative research can be transferred to other contexts or settings with other respondents
The researcher facilitates the transferability judgement by a potential user through thick description e.g. full description of the participants and socio-cultural setting
But ultimately the ‘user; of the research decides whether the findings are transferrable to their research problem
If reviewing a paper what should you consider?
Have they explained their method and analysis in enough detail?
Have they been transparent about their assumptions (e.g. provided a reflexive statement)?
Is their analysis convincing?
Why are the study findings relevant to your project/ essay?
What is mixed methods research?
- A systematic approach to data collection and analysis that integrates different sources of data and qualitative and quantitative analytical procedures with the intention to engage multiple perspectives in order to more fully understand complex social phenomenon (Creamer and Reeping, 2020)
- Embraces two different analytical logics:
An exploratory/ hypotheses generating one
A confirmatory/ hypotheses confirmation one - Produces findings that are greater than what can be produced separately by its parts (Fetters and Molina-Azorin, 2017)
What is the prevalence of mixed methods research?
- 1992-2016 percentage of empirical mixed methods publications in all fields slightly more than 18% (Timans, Wouters, and Heilbron, 2019)
- Lower in psychological research
7% of the empirical articles in top-tier APA journals reporting on basic research integrated qualitative and quantitative methods (Alise and Teddlie, 2010)
13.7% in psychology journals with applied focus (Powell et al., 2008)
What are the purposes of mixing methods?
- triangulation
- complementarity
- development
- initiation
- expansion
What is triangulation?
Different methods used to examine the same phenomenon, increasing confidence in conclusions reached
What is complementarity?
Different methods investigate different aspects/ dimensions of same phenomena to deepen/ broaden interpretations and conclusions
What is development?
Results from one method used to inform development of other method
What is initiation?
Different methods used to investigate different aspects/ dimensions of same phenomenon
What is expansion?
Different methods used to assess different phenomena to expand scope/ range of study
What is considered in the design of mixed method research?
The order and dominance of Qual-Quant
What would the mixed method be if it was equal and concurrent?
QUAL & QUANT
What would the mixed method be if it were equal and sequential?
QUAL QUANT
QUANT QUAL
What would the mixed method be if it were dominant and concurrent?
QUAL & quant
QUANT & qual
What would the mixed method be if it were dominant and sequential?
QUANT qual
Quant QUANT
QUAL quant
Qual QUANT
What is the purpose of having a preliminary qualitative method for a principle quantitative method study?
Purposes: smaller qualitative study helps guide data collection in a principally quantitative study:
- Can generate hypotheses, develop content for questionnaires and interventions
What is the purpose of having a preliminary quantitative method for a principle qualitative method study?
Purposes: smaller quantitative study helps guide the data collection in a principally qualitative study
- Can guide purposive sampling, establish preliminary results to pursue in depth etc
What is the purpose of having a qualitative follow up after a principle method quantitative study?
Purpose: smaller qualitative study helps evaluate and interpret results from a principally quantitative study
- Can provide interpretation for poorly understood results, help explain outliers ect
What is the purpose of having a quantitative follow up after a principle method qualitative study?
Purposes: smaller quantitative study helps evaluate and interpret results from a principally qualitative study
- Can generalise results to different samples, tests elements of emergent theories etc
What should you consider when thinking about design decisions?
- Led by research question and current knowledge on a research topic, commonly:
- Quantitative for validating existing knowledge and assumptions
- Qualitative for exploratory insights
