Final Exam (Week 8-10) Flashcards
How is evaluating a qualitative study different from quantitative study?
Requires different approaches than criteria used to evaluate quantitative research
No measures or instruments to assess (researcher is the instrument)
No intervention or treatment to find threats to internal validity
Qualitative evaluation approaches
Trustworthiness (can you trust the study?)
Methodological coherence (is the method coherent?)
Consolidated criteria for reporting qualitative research (is the criteria consistent?)
Relativistic approach to characterizing traits (are we being subjective?)
Ethics (ethical?)
What is an ethics evaluation approach?
Consider merits of research in an ethical way
Foundation of all research
An ongoing process
Researchers need to ensure respect for peoples, concern for welfare, and equity
What is a trustworthiness evaluation approach
Convincing an audience that a study is worth paying attention to and worth taking account of (you can trust this study)
Four aspects:
Neutrality = findings based on the participants’ meaning and experiences (researcher isn’t adding in their own)
Truth value = how true are the findings?
Consistency = dependability of a study (it’s not a one-off)
Applicability = extent to which we can transfer the findings to other contexts (is it useful)
What is similar to trustworthiness in a quantitative study?
Controlling for threats to internal validity
It’s a starting place to evaluate qualitative research
Other similar terms include rigour and validation
Strategies to evaluate trustworthiness
- Researcher reflexivity (researcher positions themselves)
- Triangulation (using a variety of sources, perspectives, and methods)
- Purposeful sampling (info-rich participants who can best inform the question)
- Prolonged engagement (lots of time with participants in the field, building relationships)
- Member check (participants review data or study interpretations and can add/alter/delete so we ensure accuracy of records)
- Rich, thick descriptions (generating thorough, descriptive data and presenting them in a rich manner)
- Peer debrief (professional ‘peer’ pushes researcher to critically reflect on study)
- Audit trail (researchers maintain detailed description of entire process, record EVERYTHING, someone external to the study examines various components of the study)
- Presents negative/discrepant info that counters main study findings (both sides of the story)
What is methodological coherence?
Coherence among philosophical assumptions, research questions, study design, data generation, data analysis, and interpretation
Indicator of quality research
Imperative to have a well-planned research design
What is COREQ
Consolidated criteria for reporting qualitative research
Three domains: research team + reflexivity, study design, analysis + findings
total of 32 items
Purpose is to be a guide to inform researchers of important aspects to include in their research (bc qual can be open-ended), identifies variety of details where qual research can be evaluated
What is a relativist approach
Basically it is up to researchers (and audience) to evaluate and determine merits of a study
Characterizing traits that may allude to quality of research are dependant on
Context of study (time, occasion, purpose)
Fluid and dynamic
We avoid fixed terms and checklists, criteria to deem qual research as “good” is relative
Flow chart ‘spiral’ of qualitative data analysis
Collect data -> organize data -> read data -> code data ->generate themes -> describe and interpret -> represent findings
Types of interviews
One-on-one or group interviews
Types of one-on-one data generation
- Complete participant (engaging as a participant)
- Participant as observer (engaging as researcher and participant, more noticeable)
Written docs PUBLIC, visual and audio sources can be art objects, photographs
Types of observations in group data generation
- Observer as participant (researcher first, less engaged as a participant)
- Complete observer (no interaction, not seen or noticed)
Written docs PRIVATE, visual and audio sources can be music, videos/film
Includes focus groups, sharing circles, talking circles
What is the most common method of data generation in qual studies? Why?
Interviews, because of its relational nature
Important to build rapport
Lets researchers understand participant’s views and meanings they attach
Three phases of interviews
Intro - builds rapport, intro to each other, what the topic will be, any ethical things that they need to know
Questioning - Interview guide can be structured, semi-structured, or unstructured (conversational)
Closing - End on a positive note, see if they have questions/comments, communicate next steps, thank them
What is observation and what does it look like
Researchers go into natural setting to better understand topic (external validity), use of senses (sight, touch, hearing)
Field notes are often used to record interviews
What should goals of data analysis align with?
It should align with research question/purpose
- What do we want to do, define a construct, describe participants’ experiences, develop a theory?
Inductive vs Deductive data analysis
Inductive: Researchers identify themes/theory from data
Deductive: Making inference based on an existing framework or starting list of categories
It is possible to have a combo!! Ex. start with a framework but allow data to form emergent categories
What do we mean when saying qualitative data analysis is immediate?
Analysis begins at the beginning of the research process
Begins the moment the investigator starts thinking about the research (since he/she is the tool), need to reflect on their role as part of the analyzing process
It’s meaning-making at the onset of investigation
What do we mean when saying qualitative data analysis is ongoing?
Researchers engage in analysis throughout the process, not just at one moment
New info may challenge previous interpretations
Researcher reflexivity is continuous
What do we mean when saying qualitative data analysis is spiral?
Data analysis isn’t a fixed linear approach, researchers flow through the process in analytic circles, often returning to earlier steps as new insights and reflections emerge
Without embracing the spiral nature, much insight can be missed
The fluidity of this process is the heart of qual research
What is the 6 step approach to qualitative data analysis?
- Organize and prepare the data
- Read or look at all the data
- Start coding all the data
- Generate descriptions or themes
- Decide how the findings will be represented
- Interpret the findings
Organizing and preparing the data includes
Transcribing interviews (many challenges, like intonation and things that are unique to human speech)
Type field notes
Scan images
Create files
Potential for LOTS of data
Reading/looking at all the data includes
Read and re-read, requires a lot of patience but still important
- Immersion in entire database (overall sense of info before breaking it into parts)
Make margin notes, so general thoughts, ideas, key concepts
Reading/looking at all the data includes
Read and re-read, requires a lot of patience but still important
- Immersion in entire database (overall sense of info before breaking it into parts)
Make margin notes, so general thoughts, ideas, key concepts
Coding all the data includes
Systematically organize and reduce data into meaningful chunks
Assign names for chunks
Various methods (ex. sticky notes, highlighters, comments on a doc, spreadsheets, etc.)
Generating descriptions or themes includes
Combining codes (chunks) from step 3 into broader categories, provide brief descriptions of each theme
Need to tell the story that best represents data
Usually 5-7 themes generated
Deciding how the findings will be represented includes
Basically how to best SHARE the findings
Journal articles (can be limiting! page limits, may miss intended audience)
Conference presentations
Reports
Policy briefs
Interpreting the findings includes
Making sense of data and going beyond themes to a larger meaning (ex. what is the bigger story, the essence, how do the themes relate?)
Data analysis tips
Slow down
Look at commonalities and differences
Remember that we only have a piece, not a whole
Be aware of challenges (multiple researchers, dominant voices, different assumptions or perspectives)
Need to rely on brains and not just specific techniques to analyze (even though there are systematic ways)
What is qualitative data
Used when researchers generate non-numerical data
What is the correct data analysis method?
There isn’t any one correct one, but the 6-step method is very commonly used
2 ways theory can be used
Theoretical lens: theory shapes the question(s), participants, data generation, and analysis
Interpretive framework: Use theory to interpret findings
Data collection vs generation
Collection: picking berries, it exists outside of research, researcher’s job is to use the best methods to pick them
Generation: berry pie, creating knowledge is an active process of research, and is co-constructed by us and participants
Challenges with interviews
Co-operation is crucial
Participants may be unwilling or unable to share everything the researcher wants to explore
Participants might have good reasons to lie
Researcher might not understand local language/customs
Characteristics of a structured interview
Specific SET and ORDER of questions, no flexibility and often has closed-ended questions (researcher-administered survey with options)
Interviewer plays a neutral role, rapport and interested listening
Responses often recorded onto a coding scheme
Characteristics of a semi-structured interview
Interview guide with room to discuss other topics not on the guide
Not intended to standardize every interview
More flexibility (can follow any interesting avenues that emerge)
Characteristics of an unstructured interview
Conversation with a purpose
NO interview guide, just guiding topic for conversation
Common in narrative research
Open-ended questions should be:
Open-ended, use exploratory verbs (WWWWW), non-directional, and have a single focus (no “and”)
Probe and follow-up questions are important
What does an open-ended question do?
Doesn’t assume a bias for any angle for the interviewee
Types of questions we should have
Main guiding questions
Probe questions (ex. share an example of…)
Follow-up questions
What tone should we set for the interview?
A kind and interested discussion
What questions should we ask first?
General questions to break the ice, then ask more specific ones
Ask questions to confirm previous statements
Difference between private and public written documents
Public: policies, newspaper articles, historical archives
Private/written documents from participants: logbooks (ex. exercise log), personal diaries or letters, performance reports
What is visual data and why is it useful?
Photographs, drawings, mapping, diagrams, or videos can provide valuable insight into a phenomenon
Cool because not all participants are comfortable expressing themselves through words
What is researcher reflexivity?
The researcher understanding what the see and what they don’t
Considering and embracing how their assumptions can impact the study
Keeping a research journal can help
What things could change depending on strategy of inquiry?
Sampling, data generation, and data analysis
Strategies of inquiry examples
Narrative (story, represents broad experiences, unstructured interviews, observation and journaling)
Ethnography (focus on culture, immerse into culture, observation interviews, and docs like poetry and art)
Phenomenology (examining a phenomenon, lived experiences, many in-depth interviews, observations, field notes, diaries)
Case study (complexity and distinctiveness of a specific case, specific to time and place, extensive methods like interviews, visual methods, docs, observations)
- Can be instrumental case study (issue of interest), intrinsic case study (complexity of case), or collective case study (multiple cases)
Grounded theory (generates theory from data)
Qualitative description (comprehensive description and summary of a phenomenon, less interpretation, describes phenomenon in everyday language, can take on hues/tones of other designs, many one-on-one interviews, data saturation so no new information will surface)
Nonprobability sampling techniques
Convenience sampling
Quota sampling (create sample that has similar proportions to population
Purposive sampling (want info-rich participants who can inform our understanding)
Snowball sampling
Types of purposive sampling
Extreme case: getting participants who are unusual or represent extremes (ex. best and worst profs)
Maximum variation: Getting diverse perspectives, location-wise, racial, etc.
Snowball
Do we want a big sample size?
The point is not to generalize to a broad range of people, it’s to get an in-depth understanding of the topic
DEPTH not breadth
Saturation and sample size will depend on
Scope of study
Nature of topic
Quality of data
# interviews/participant
Study design
Guidelines for sample sizes based on strategy of inquiry
JUST GUIDELINES
Narrative: 1 - 2 participants
* Phenomenology: 5 – 8 participants
* Grounded theory: 30 – 50 participants
* Ethnography: typically a single group
* Case study: a single case OR 4-5 cases (collective case study)
How do we gain access to participants?
Usually through relationships, qual research is relational
Known sponsor approach is when we know a person with a solid relationship with the interest group
What is a theory?
A general explanation of an event, process, action, or phenomenon
Key defining features of constructivism:
Understanding of the world
Multiple participant meanings: Meanings of experience are subjective + socially constructed
* Social and historical construction: Meanings are formed through interactions and historical/cultural norms
* Theory generation: Inductive development of a theory or pattern of meaning
Purpose statement for qualitative
Focus is on central phenomenon
Begin with words that identify what the study is doing (describe/discover/understand/explore)
Identify the central phenomenon
Identify PW
Identify participants targeted
Mention strategy of inquiry
Identify where the research took place, any other important descriptors
