Week 3 Exam Flashcards
Qualitative vs Quantitative
Strictly speaking quantitative and qualitative refer to data not the research type
Using these terms people often mean:
Qualitative:
1) exploratory, theory-building research type
2) Research strategies - case studies, interviews, focus groups
3) Data analysis method - coding
4) Conceptual models, propositions as results
5) Subjective results
Quantitative:
1) Theory testing, decision science as research type.
2) Research strategies include: surveys, experiments, mathematical modelling
3) Data analysis method - econometrics, statistics (e.g., regression)
4) Hypothesis tests, effect size, optimal choice as results
5) Objective results
Research method (strategy) combination:
Research strategies can be combined:
Qualitative and Qualitative - interviews with archival studies
Quantitative and Quantitative - a survey with modeling
Qualitative and Quantitative - interviews to help formulate the hypothesis and problem, and experimentation test)
Mixed-methods require a lot of time and skills, but:
1) One method used first can help in designing the following strategy better
2) Using multiple methods can enrich the interpretation of findings
3) Better methodological quality (“triangulation”) and generalizability
Research strategies associated with Qualitative research:
- Ethnography – Immersive observation of people in their natural setting to understand culture and behavior.
- Case Studies – In-depth study of one or a few cases (e.g., companies or events) using multiple data sources.
- Focus Groups – Group discussions (6–10 people) to explore shared views and differences on a topic.
- Interviews – One-on-one conversations to gain deep individual insights (structured, semi-structured, or unstructured).
- Archival Research – Analysis of existing qualitative documents (e.g., reports, emails) to study past events or practices.
In certain fields (marketing), there are preferred strategies (focus groups) that do not work in other fields as well (supply chain)
Interviews:
1) Provides opinions not “accurate” information(informant bias - social values, norms, desirability)
2) Thus, is less useful for sensitive topics, as respondents might refuse to give accurate info
3) Depends on interviewee’s memory, thus, not suitable for researching things that have happened long-ago (recolletion bias)
4) Are subject to interviewer’s interpretation (researcher bias - values and opinions, professional skills)
5) Consume a lot of time
6) However, still widely used as are easier to obtain (than for example documents)
So better used in cases when other options not available.
Interview types:
Unstructured:
1) Essentially free-flowing chat on topic
2) Follows the lead of interviewee
3) Questions asked as they arise
4) All interviews are differnet
5) Used when you have no knowledge or theory in the topic
6) Used when building theory
Semi-structured:
1) Predefined questions and topics (interview guide)
2) Questions informed on theory
3) Order of questions can be changed during the interview
4) Additional questions can be asked
5) Used when needed some insights with some structure in place
6) For building or elaborating theory
Structured:
1) A fixed list of short, simple questions
2) Possibly many closed questions
3) Questions derived from theory and practice
4) Easy analysis and comparison
5) Used when needed to compare opinions
6) testing, building and modifying theory
Types of questions in SEMI-STRUCTURED interviews:
1) Open questions - main type to be used
Answer yes/no is not possible, and only one question inside. Open question starts with a question word (what, why)
2) Closed questions - minimized use.
No details, or long asnwers. Can be used to verify facts (Do I get your permission to take notes?)
3) Leading questions - must be avoided.
Lead the interviewee to the answer you are expecting. Very undesirable due to researcher bias (You must have felt terrible about this?!”)
Examples:
Do you think changing the customer order return process was the right thing to do?
* CLOSED QUESTION
Tell me about your hotel stay experience and what was the best part about it?
* OPEN QUESTION BUT 2 QUESTIONS IN 1
Failure of this project must have been a real disappointment for you, wasn’t it?
* LEADING QUESTION
Interview guide (for semi-structured):
1) Start with a topic in mind
2) Think about potential respondents (for different interviewees a different guide)
3) Divide topic into subtopics
4) Develop questions for each topic (adjust questions to respondents, think of biases, think about timing of questions)
5) Review logic and flow (ice-breaker, from boarder to narrow, easy questions at start)
6) After a couple of interviews review the guide
Potential interviewees and how many:
1) Knowledgable informants - people with knowledge in the topic.
Try to capture different perspectives (stakeholders) - Questions in interviews may have to be adjusted for different people
2) Make a preliminary list of interviewees (can evolve over time with understanding):
Alternatively ask interviewees for suggested people
Combination of approaches desirable (asking and making list
3) Always the more the better, but there are time constraints:
The more complex topic the more interviewees
Ideally reach “saturation”, when no new insights are being introduced
Asking the same question to multiple interviewees ensures triangulation (quality).
Case study:
Investigates a contemporary or recent phenomena within its real life context, especially if the boundaries between context and phenomena are not clear.
Suitable for:
1) exploratory research
2) Theory building (elaboration)
3) Theory testing - as uses both quantitative and qualitative data
- Case studies are suitable for many research objectives and questions “What?”, “How?”, “Why?”…
- They are less suitable for answering “How many?…” or “How often?…” questions, as it is harder to quantify the relationships between variables using case studies
- They can help identifying causal mechanisms, but not quantify or prove them
Advantages and disadvantages of case studies:
Advantages:
1. Deep-dive into real life context
2. Possible to study a new phenomena even if there is no theory
3. Possible to uncover issues that are not as easily identifiable through other strageties
4. Findings are usually well received by practitioners
5. Very versatile
Disadvantages:
1. Discretion of the researcher about which data to include (researcher bias risk)
2. Time- and resource-intensive
3. Lack of statistical generalizability
4 . Hard to study past phenomena due to recollection bias and lack of info
5. Difficult to write about concisely but convincingly
Typology of case studies (single-case studies):
Single-case studies:
1) Chosen when study is about a very unique (extreme) phenomena, or when there is very good data access
2) Deep dive into the studied case, interesting and rich insights
3) Limited generalizability (not sufficient to prove something is prove elsewhere, May be sufficient to prove the opposite – pointing at gaps and omissions in existing theories)
Typology of case studies (multi-case studies):
1) Allows to replicate and enrich data and findings
2) more convincing - higher generalizability
3) Additional complexity (compared to single case) lies in the seleciton of cases (similarity, differences, etc)
4) Key question - how many? The more the better, but:
There is a balance between depth and number (time constraints)
And between the case availability and comparability.
Characterisitcs of both single and multi case studies:
Cross-sectional - time dimension is not important: you look at data and study at one specific point in time
VS
Longitudinal - time dimension is important: you study concept over time; so multiple measurements overtime happen
Real-time - you study something as it is happening
VS
Retrospective - you study something that is in the past.
Combination of real time and retrospective is possible.
Facts:
Common mistake: I am doing study now, so it is real-time –no!
Single cases more often are associated with longitudinal research, multiple - with cross-sectional
Longitudinal research is more time-taking, so hard to execute it in multiple cases
Multiple cases are more often retrospective
Unit of analysis in case studies:
What you need to look for (or sample) in your case studies to asnwer RQ.
In essence it looks at how you operationalize complex concepts/constructs
For example: trust - how to measure trust
How to sample cases:
Cases are not sampled randomly; You select cases that are likely to provide information about about the pehomenon of your interest: this is called theoretical sampling
For single-case studies you typically pick unique or extreme case or case with great access to data
For multiple cases different options are possible:
1) Select as similar as possible (opportunities for replication)
2) You select different cases (failures and successess)
3) Combine aspects of similarity and dissimilarity - there is no one ‘recipe’, but your cases should be similar on some dimensions and dissimilar on a few others
Common data sources in case studies:
Interviews:
1) Semi-structured, structured, unstructured
2) Knowledgable interviewer and informants
3) Biases involved, thus multiple stakeholders, departments, levels
Observations:
1) Participant and non-participant types
2) Traditionally notes, but also audio and video (time-sensitive)
3) If a participant, it is hard to preserve objectivity
4) Can complement interviews and help to interpret them (vice versa)
Archival documents:
1) organization policies, contracts, protocols, e-mails, presentations, reports, etc.
2) Very valuable for recreating the “story” (especially for past), much needed for studying facts
3) Useful for veryfing info, received in interviews; can help develop interview guide
4) Sometimes challenging to gain access
Examples of other sources:
1) Video, photo, audio
2) Questionnaires
3) Results of focus groups or workshops
4) Industry, governmental, organizaitonal reports - secondary data
Case studies require more than one data source:
1) Variety of data sources, rich the data for deeper investigation and convincing results
2) Different sources may be needed for different purposes (recreate the context, get different perspectives, etc)
3) Combining sources is also important for quality: this is called data “traingulation” (documents verify interview findings)
4) Most common: interviews + archival documents
Can also use quantitative data!
General quality indicators for research, that uses qualitative data:
1) Validity - Are you measuring the right thing?
Construct - is the operationalization of the construct “sound”
Internal - is there a strong link between collected data and developed theoretical ideas (does the data support the
findings/conclusions)
External - to which degree can findings be generalized beyond context
2) Reliability - Are you measuring it consistently?
Replication is very difficult to achieve in qualitative research due to often unique settings, events, cases.
How thoroughly the research is performed and documented is the ‘proxy’ of reliability in studies that use qualitative data (can’t achieve the same results, but replicate methods)
General threats to quality in Qualitative research:
Informant bias (related to values, opinions, memory, honesty, social desirability) - links to construct validity
1) If the topic is in general too sensitive or personal
2) If there is a strong personal interest in the topic
3) If you are asking about an event that happened a long time ago.
Researcher bias (related to values, opinion, and skills) - links to internal validity
1) If there is a strong interest in the topic (might want to prove what you think is right)
2) Lack of skills may lead to poor construct definition/operationalization, problems with data analysis, poor documentation
Idiosyncratic findings - links to external validity - refer to results or patterns in your data that are specific to your particular sample, context, or informants—but may not apply more broadly
1) If the cases/informants are unique in their own rights
2) Your findings are true for one sample but difficult to generalize
How to increase methodological quality (what can be done):
Construct validity:
1) Operationalization of constructs using theory/literature (data collection)
2) Multiple sources of data - triangulation (data collection)
3) Multiple informants - triangulation of informants (data collection)
4) Key informants review the draft of research (reporting)
5) Experts review the draft, and notes (reporting)
Internal validity:
1) Pattern matching or explanation building (data analysis)
2) Grounding study in different literature streams (research design and analysis)
3) More than one researcher involved
External validity:
1) Replication logic in multiple case studies (research design)
2) Rich data from multiple sources (data collection)
3) Detailed reporting of cases (reporting)
Reliability:
1) Research protocol (data collection)
2) Consistent interviewer behavior (data collection)
3) Consistent interview guides (data collection)
4) Database of case data (collection and analysis)
5) More than one researcher
6) Documenting the data analysis and coding (Data analysis)
7) Using software for increased objectivity of analysis (data analysis)
8) Detailed reporting of cases (reporting)
Challenges in qualitative data analysis:
1) No clearly defined standards or detailed rules
2) Choice of the right method is not always clear
3) Often learned by doing or from other researchers, no systematic training
4) Hard to transparently explain and present
5) Can be very time-taking and costly
6) Based on individual interpretation - biases
7) Risk of poor quality of analysis if researchers have no right skills
Logic (steps) of qualitative data analysis:
1) Collect
2) Organize and prepare for coding (transcribe)
3) Code
4) Analyze for insights (describe, compare, relate)
5) Report the insights - develop findings, supporting arguments, defend the arguments with data, extend results beyond the original research setting (link back to existing knowledge)
The steps are not linear: coding starts in parallel with data collection and may lead to changes in it; analysis and reporting may lead to the need to go back to them again
Approaches to qualitative analysis:
Deductive approach:
Previous theories and concepts inform the study, so the main themes and categories are known, and data must be organized in them (top-down)
Inductive:
No previous theory, categories and themes emerge from data (bottom-up)
Abductive - some parts of the data are informed by theory, but likely not all data fits into these categories, the novel parts are coded inductively (start with deduction, continue with induction).
