Modes of Data Collection Flashcards
Observational Research
- Very common
- Behavioral patterns
- Things you can see and hear
- Doesn’t apply when you’re focused on attitudes
- Used often in “Mystery shopping” research
- Researcher takes role of a customer, visits stores, behaves normally; visiting to see quality of the service; responsiveness of staff, compliance of rules, etc.
- Can’t rely on verbal reports of own behavior
- Observing directly without asking anything
- Measuring time, reactions; hotjar; user research
- Environmental students checking baskets for recycling
6 Types of Observation
- Participant vs. Non-participant
- Standardized vs. not standardized
- Covert (hidden) vs. Overt (visible)
- Direct vs. Indirect
- Natural vs. contrived
- Personal vs. mechanical
Participant vs. Non-participant Observation
Participant
- You as researcher engaged in observed situation; researcher is an actor
- “Going native”
- You might have more control or influence
- Risk of identifying with one group; you lose distant view of the situation with a bias; self-identification with the situation
- Ethical issue: often people/participants don’t know that they’re being observed
- Full and better understanding of issues because you’re part of it
- Might have to rely on your memory too much; risk of not being able to write things down or record properly
Non-participant
- You’re standing aside and not affecting the situation
- Might take a long time to get what you need
Standardized vs. Not Standardized Observation
Standardized
- Used when you don’t know features or characteristics of target groups
- Use a recording sheet to fill in the form
- Structured without scales or attitudes
- Yes or no questions
Not Standardized
- Used more for qualitative research
Covert (hidden) vs. Overt (visible) Observation
Covert (hidden)
- Might take part but still be hidden by your role (mystery shopping)
Overt (visible)
- If you’re recognized as a researcher
- School inspections, auditors
- Typically non-participant
- When people know they’re being watched, behavior might be altered
- Hawthorne effect
- Mayo - wanted to measure impact of lighting on workers
- Thought more brightness would have higher performance
- Add lights, performance increased
- Add lights again, performance increases again
- Decreased lights, performance increased again
- Workers responding to his observation and didn’t notice change in lighting
- HR: First time they’d ever seen a white-collar person at their workplace asking about conditions; identified more with company because management was interested
Direct vs. Indirect Observation
Direct
- Can see results or processes in real-time
- Might only be observing results
- High risk of misinterpretation (fire alarm); Extraneous circumstances
- Analysis of bin checking for environmental students
- Can be a disadvantage because you can’t redo things; no second chance to see things
Indirect
involves the analysis of textual material generated either indirectly from transcriptions of audio recordings of verbal behavior in natural settings (e.g., conversation, group discussions) or directly from narratives (e.g., letters of complaint, tweets, forum posts).
Natural vs. Contrived Observation
Natural
- Mystery shopping natural
- Good quality results because there’s no interference
Contrived
- Typically in a flat; invite people to event and induce some test
- One-way mirror, cameras
- People not in natural setting’ people own’t behave normally
Personal vs. Mechanical Observation
Personal
- Must be done on your own; relying on yourself
- Your mental record/memories will often be very different than a photograph
Mechanical
- Using gadgets, appliance or tools
- Not used in hidden or participant observation
- Maybe even a problem for mystery shopping
Observation Schedule
“a form prepared prior to data collection that delineates the behavior and situational features to be observed and recorded during observation. Observation schedules vary on a quantitative–qualitative continuum.”
An observation schedule can be a simple list of things to look for in a particular situation
Can be far more complex; a minute by minute count of events such as verbal interactions between subjects
Can be like an escape strategy
Unobtrusive Measures
Not asking for participation
- Complementary to direct elicitation techniques (like interviews and questionnaires)
- Adaptable in situations where eliciting is difficult and/or dangerous
- Methodological weaknesses of interviews and questionnaires:
- Respondents provide answers that are in line with self-image of researcher expectations Mayo
- We only get answers from those who are accessible and willing to respond
- Hawthorne effect
Primary vs. Secondary Data Analysis
Secondary Data Analysis
- Data gathered and recorded by someone else prior to and for a purpose other than the current project. A type of research in which data collected by others are analyzed.
Primary Data Analysis
- Original analysis of the data collected in a study
Meta-Analysis Advantages & Disadvantages
Meta-Analysis
- Quantitative procedure for summarizing or integrating the findings obtained from literature
- Uses the results of individual research projects on the same topic
Advantages
- Saves cost for data collection
- Saves time required for data collection
- Requires no access to subjects
- May provide info otherwise not accessible
Disadvantages
- Data collected may not be suitable for researcher’s purpose
- All necessary data may not be available
- Requires time to search for the dataset
- Original dataset may not be accurate
- Need time to sort out the data
- Inappropriate units of measurement
- Time period inappropriate (outdated)
Content Analysis
General usage of the term
- Refers to research techniques for analysis of the content of written, spoken, or pictorial communication (books, newspapers, journals, papers, television programs, interview transcripts, web pages, songs, speeches, postings, correspondence, statements, etc.
- Analysis can relate the occurrence of coded content with other factors (for example, features of the producer or effects on the receiver)
Specific usage of the term
- Refers to a quantitative technique that attempts to quantify the meaning of the content of communication
- Critical for answering the classic question: who says what, to whom, why, how, and with what effect?
Content Analysis Strengths vs. Weaknesses
Strengths
- Easy to undertake - no staff, no special equipment
- Easy to correct errors - go back and recode
- Allows for the study of dynamic processes - time
Weaknesses
- Limited to recorded communication - much is lost
- High chance of misinterpretation
- Limited in terms of claims you can make
Qualitative Research Designs
- In-depth Interviews
- Focus Groups
- Semi-structured Interviews
Semi-Structured Interviews
- let participants explain, add some open-ended why questions
- useful to prepare phrases in advance; easy to interpret
- yes/no questions; multiple choice
In-depth Interviews
- Used most commonly; universal technique
- Interview plan; typically 45 min-1hour
- Provide a lot of time and space for people to respond
- Listen 95% of the time
- Sufficient 5-10 interviews per issue; results will start to be the same; saturated, not gaining anything new
- Key advantage that the tool is flexible and versatile; can customize flow of interview and questions
- Each person might be engaged with 3-4 topics specifically; can cover the rest of the topics with other interviewees
- Time-consuming; you need to travel to respondants; rewrite or create transcript (dication? Skype?)
- Transcription tools good for English but not great for other languages; you still need to correct or edit
- Interviews are recorded, so you must get permission to record before it’s starting
- Must be careful about quotes and wording
- Anonymous quotes shouldn’t lead to identify someone; careful when interviewing an expert that uses specific phrases, vocabulary, or wording
Focus Groups
- Specific because not a one-on-one interview
- One researcher aka “moderator” and a group of people you’re interviewing at once
- Interactive; spontaneous discussion
- Might see group dynamics; how ideas are formed; how people cooperate
- Some issues it’s good to see group dynamics, some not (Ex: sexual behavior of male adolescents)
- 1.5-2 hours typically
- Need a skilled moderator; different personalities; create the mood of the room; people need to feel safe to present attitudes/ideas; people might disagree
- Moderator needs to solve conflicts, keep everyone on task, elicit responses in a polite yet assertive way
- Key advantage - you might get a lot of focused, rich data on issues; participant’s own words as they describe different topics; might last two weeks from initial commission
- Spend most of time with recruitment and analysis
- Recorded with VCR; typically transcription not done but sometimes required
- Key disadvantages - technique is sensitive pending skills of the moderator; needs to be skilled in topic and communication; bad moderating can lose a lot of information and lose opportunities
- Never do just 1 focus group; do at least 2 - 4; outcomes will likely be the same but there could always be a bias from moderator or composition of the group; can’t rely on one group’s data
- Want internal thoughts of people
Projective Techniques
- Indirect interviewing methods which enable respondents to project their ideas, beliefs, and feelings to a third-party or into some task situation
- Researcher sets up a situation for the respondents asking them to express their own views, or to complete/interpret some ambiguous situations presented to them
- Involve situations in which participants are placed in (projected into) simulated activities in the hopes that they will divulge things about themselves that they might not reveal under direct questioning
- Generally used by the researcher when language barrier, illiteracy among resondents, social barrier or psychologocal barrier create difficulty
- Might make it easier to express thoughts, attitudes, motications
9 Types of Projective Techniques
Examples:
- Word Association
- Sentence Completion Test
- Cartoon Test
- Collages
- Expressive techniques
- Role-playing - better for focus groups
- Third-person technique
- Brand party
- Obituary
Word Association
Projective Interview Technique
- Records the first thoughts that come to a respondent in response to some stimulus
- Respondents are presented with a list of words, one at a time and asked to respond with the first word they think of
- Used in marketing; results of a word association test with alternative brand-names for a new fruit-flavored sparkling water drink
Sentence Completion Test
INSTRUCTION: write in words to complete each of the sentences below.
- A person who shops at M&S is [BLANK]
- Mercedes is most liked by [BLANK]
- When I think of shopping in a department store, I [BLANK]
Cartoon Test
Draw or show a picture with speech bubbles, ask participants to complete the second bubble
A: Let’s see if we can pick up some housewares at M&S
B: ______
Collages
- A pattern (larger picture, story, etc) made by sticking pictures or materials on a surface
- Respondents assemble pictures that represent their thoughts and feelings
