Week 5 and 6 Flashcards
Research question development framework
Spider question framework
PEO Question Framework
Good survey questions
▪ Measures the concept it is trying to examine
▪ Doesn’t measure irrelevant concepts
▪ Means the same thing to all participants
▪ Example: In general, how would you rate your health?
▪ Excellent
▪ Very Good
▪ Good
▪ Fair
▪ Poor
General rules for writing survey question
Avoid technical terms and jargon
Avoid vague or imprecise terms
▪ Example: How important is it to you that your supervisor meets with your regularly?
Define things very specifically
Avoid complex sentences:
Provide reference frames:
▪ Example: How supported do you feel by your supervisor?
▪ How supported have you felt by your supervisor in the past week?
Make sure scales are ordinal
▪ Example: All the time, some of the time, most of the time, half of the time, none of the time
Avoid double barrelled questions – Questions need to measure ONE thing.
▪ Example: Does your supervisor provide you with written and verbal feedback?
Responses should anticipate all possibilities
▪ Include an other, please specify category
Make sure your responses are unique
▪ Example: 0-4 hours, 5-9 hours, 10-14 hours, 15-19 hours, 20+ hours
Avoid questions using leading, emotional or evocative language
▪ Example: Do you support the use of student evaluation forms for supervisors to ensure that
terrible supervisors are not allowed to supervise students ever again?
survey question types
survey question types
5 types
Tips for questions
▪ Make sure you have clear instructions at the start of your survey/questionnaire
▪ How will you administer the survey/questionnaire?
▪ Facebook/Twitter
▪ Email
▪ Hard copy in person
▪ Mail
▪ Phone
▪ If administering survey over the phone – limit the number of questions and the
response options
▪ Pilot your questions!
▪ Do not develop them on your own!
Interviews
Advantages
▪ In-depth
▪ Semi-structured
▪ Key Stakeholder
▪ Narrative
▪ Life History
▪ Dyad
▪ Think aloud
Why use interview?
To increase rapport with participants
▪ Face-to-face contact
Provides support to participants
▪ Direct – “No-one has ever talked to me about this before”
▪ Indirect – through provision of feed-back, referral…
Increases response rate
Allows immediate clarification of your interpretation
▪ Increases validity of your data
Interview guides- what is the purpose of?
▪ Useful in structured and semi-structured
interviews
▪ May provide suggestions of prompts
▪ Allow for flexibility
▪ You may need to follow-up information that is given, yet
that you have not considered previously
▪ Allows researcher to pursue new avenues
of enquiry, based upon the responses given
Interview questions
Use—- question
Don’t use —-
Interview tips
✓Build rapport, allocate time to chat and build rapport
✓LISTEN, express interest
✓Encourage participant to expand - give more details
e.g. Can you tell me more about that?
✓Let responses guide the direction - keep within topics of interest
✓Use participant’s language – reflect it back
✓80% of interview must be participant
✓Avoid excessive use of ‘why’
Stages of interview
▪ Stage 1. Arrival and introduction
▪ Establish rapport
▪ Stage 2. Introducing the research
▪ Scope of interview but emphasise the ‘openness of responses’
▪ Stage 3. Beginning the interview
▪ Provide some context
▪ Stage 4. During the interview
▪ Use semi structured approach to make sure you cover the areas needed
▪ Stage 5. Ending the interview
▪ Give some advice ‘almost at end’ and then end positively
▪ Stage 6. After the interview
▪ Thanks for participation and the value they have provided.
▪ Provide them ongoing contacts, in case there are concerns
focus groups and co-design workshops
Advantages
▪ Group interactions brings out different data (experiences/perspectives) to
individual interviews
▪ Exploratory – little known about participants or topic
▪ Testing ideas e.g. acceptability of a new program
▪ Evaluations
▪ Multi-method studies – triangulation
▪ Co-design
focus groups and co-design activities
-Q-sorting or Card Sorting:
Participants categorize or rank cards representing concepts, revealing patterns or priorities.
River of Life: Participants visualize life experiences along a river, sharing narratives on challenges and aspirations.
Listing, Rating, Ranking: Participants generate lists, rate, or rank items, prioritizing ideas or identifying consensus.
Sentence Completion: Participants complete prompts, expressing thoughts and experiences freely.
Collages: Participants create visual representations using images, words, or symbols to convey perspectives creatively.
focus group tips
All will have a perspective on this topic
▪ All ideas are valid, may be different from each other
▪ No judgement/ no personal attacks from anyone please
▪ Healthy discussion and different views welcome
▪ We want to hear from each of you
▪ Might be asked directly for a response
▪ May be asked not to answer next
▪ Please no talking over anyone else
Participant observation
Data management
“Research Data Management covers all of the decisions made during the research lifecycle to
handle research data, from the planning stage of your project up to the long-term preservation
of your data.” (University College London, Research Data Policy)
◦ Where is your data stored?
◦ How will it be analysed?
◦ Who will have access to the data?
◦ What are the requirements for sharing data?
◦ What quality checks will you conduct?
Research data
“Data are facts, observations or experiences on which an argument or theory is constructed or
tested. Data may be numerical, descriptive, aural or visual. Data may be raw, abstracted or
analysed, experimental or observational. Data include but are not limited to: laboratory
notebooks; field notebooks; primary research data (including research data in hardcopy or in
computer readable form); questionnaires; audiotapes; videotapes; models; photographs; films;
test responses. Research collections may include slides; artefacts; specimens; samples.”
(University College London, Research Data Policy)
Data entry and coding
Store
Closed ended Q
Database to store the data e.g. Excel or REDCapTM – secure server, password protection
Data storage for returned surveys (locked cabinet)
Values to be entered for close-ended questions e.g. 1, 2, 3 vs A, B, C
Set up data dictionary
Data coding
Examples of data coding:
◦ Assigning a number to a group of objects
→ use an unique number for categories
0 = No, 1 = Yes
0 = female, 1 = male
0 = <30 years; 1 = 30 – 39 years; 2 = >40 years
Data management For Qualitative data
Privacy
Remove identifiable information from file names and documents.
Have a spreadsheet with identifiable information that is password protected and saved in a
different location
Data Verification
Sources of error how to manage
Sources of error
◦ Data entry mistakes
◦ Data omissions
◦ Data errors
Mitigate errors
◦ Internal system checks e.g. permissible values
◦ Data integrity checks e.g. audit data (Data Integrity is the accuracy, completeness, and consistency of data throughout its lifecycle)
◦ Third party verification e.g. compared to admissions data
Coding of missing data
Don’t know, Not assessed, Not applicable, Unknown, Refused….
◦ Are they the same?
◦ Do you apply the same numeric value?
E.g. Common codes:
◦ 98 = don’t know, 99 = refused
◦ Missing responses: 999 or ‘.’ or ‘blank’
Other things to consider for data coding
Missing data
Reverse coding
Reporting frequencies
◦ Do you include or exclude people with missing data?
* What to do with additional responses ‘other’
* Reverse coding - One common validation technique for survey items is to rephrase a “positive” item in a “negative” way.
Approaches to qualitative data analysis
Approach to analysis depends on
1.Research Question
2.Theoretical
Content analysis involves systematically analyzing text to identify patterns and themes.
Narrative analysis interprets stories to understand customer feelings and behaviors.
The thematic analysis identifies patterns and themes in data.
Grounded theory analysis generates hypotheses from data.
Deductive: Applying existing concepts or ideas to the data.
Semantic: Coding based on the explicit content of the data, focusing on factual codes.
Realist: Identifying reality evident in the data.
Inductive: Deriving codes directly from the data.
Latent: Coding based on underlying concepts and assumptions within the data.
Constructionist: Analyzing how reality is constructed in the data.
Considering the influence of YOU
In qualitative data analysis
YOU are the data analysis tool
Qualitative data analysis is subjective and inductive
How YOU see the world will influence how you COLLECT your data and how you
INTERPRET your data
Acknowledge these influences
◦ Education
◦ Gender
◦ Religious affiliation
◦ Social class
◦ Biases, prejudice
◦ Preconceptions
◦ Ethnicit
Qual data analysis (Rice and Ezzy, 1999)
Begins…
◦ Part of the research design
◦ Part of the literature review
◦ Part of theory formation
◦ Part of data collection
◦ Part of data ordering, filing and reading
◦ Part of the writing
Glossary of terms
What is code
What’s category
What is theme
Code – A label, a name that describes the meaning of the text, usually one or two words
Category – Grouping codes that a related to each other through content or context. Organising codes
into a category. A category answers questions about who, what, when or where? Category names are
factual and short
Theme – Expresses underlying meaning across categories. Interpretative. Answers the questions why,
how, in what way or by what means. Poetic and metaphoric language can be used. Descriptive and uses
verbs, adverbs and adjectives.
Inductive versus Deductive analysis
Qualitative Analysis
Spectrum
Content analysis involves systematically analyzing text to identify patterns and themes.
Narrative analysis interprets stories to understand customer feelings and behaviors.
The thematic analysis identifies patterns and themes in data.
Grounded theory analysis generates hypotheses from data.
Identify the unit of analysis
A tool to scrutinize your data: The tool or method used for data analysis.
Meanings: Individual interpretations or understandings.
Processes: Activities or workflows.
Practices: Observable behaviors or routines.
Encounters: Interactions or exchanges.
Narrative structures: Structural elements of stories.
Organizations: Specific institutions or entities.
Lifestyles: Patterns or choices in living.
Qualitative coding
Different method 6
Deductive – existing concepts or ideas
Semantic – explicit content of the data (factual codes)
Realist – reality evident in the data
Inductive – derived from the data
Latent – concepts and assumptions underpinning the data
Constructionist – how reality is created in the data
Coding guide
◦ Can be developed prior to analysis based on literature review
◦ Evolves through the process of coding
◦ Codes can be FACTUAL (content analysis)
◦ or INTERPRETIVE (thematic analysis/grounded theory)
Step 1: deconstruction/Fragmentation
Assign a code word or phrase that
accurately describes the meaning
of the text segment
Line-by-line coding is done first in
theoretical research (GT,
Framework) / segmentation
(applied thematic analysis)
More general coding involving
larger segments of text is
adequate for practical research
Breaks down data and reconceptualises it
Comparison between events, actions, and interactions
Apply conceptual labels
Group into categories
Develop initial relationships
Coding
Can u used inductive and deductive coding in one study?
Can use deductive and inductive coding in one project
◦ Study title: ‘care-seeking, self-management and the drivers, costs and benefits of
complementary and alternative medicine (CAM) used by people with diabetes’ Manderson, L and
Canaway, R Camelot Study
Deductive codes ensure key areas are analysed
◦ Drivers or motivators of CAM use
◦ Costs of CAM use
◦ Benefits of CAM use
Inductive coding then further explores issues raised by participants
◦ e.g. Faith and spiritualit
What can be coded?
Clustering codes
After open coding text, make a list of all code words
Cluster similar codes and look for redundant codes
Objective:
◦ reduce the long list of codes to a smaller, more manageable number (e.g. 25 or 30)
Things to look for
data analysis
Key phrases
◦ These are things that make some sort of (not necessarily describable) sense
Topics that occur and recur? (repetition)
◦ People often circle through the same ideas in their talk
Local or commonly-used terms used in an unfamiliar way
◦
‘NVivo’ coding
◦ e.g. “women’s troubles,” “bad blood,” “difficulties”
Use of metaphor
◦ What do they represent?
◦ e.g. ‘Rock solid’ marriages are imagined to last, ‘cooking with gas’
Step 2 – RECONSTRUCTION
Specify more rigorous codes
Put data back together – new ways of making connections between a category and sub-
categories
Interconnecting the data
Coding Process (Charmaz, 1991)
- Begin by exploring the general research question
- Gather data, and code for respondents’ meanings, feelings and actions
- Look for processes and relationships between specific events and general processes
- Coding leads to new categories
- Collect more data on the developing categories
- Go back re-read earlier data for the new categories, formulate new questions
- Constantly compare individuals, different events, and categories
- Write memos throughout re; categories, processes, ideas
- Move towards memos that are more conceptual and codes that are more abstract
Coding process (Creswell 2002)
Thematic
Steps in reflexive thematic analysis – Braun
and Clarke
- Familiarisation with data
- Coding – apply labels to the data
- Generating initial themes – identify patterns from the coding
- Reviewing themes – check themes against coding – does it tell the story of the data? Does it
answer the research questions?
◦ “themes are defined as pattern of shared meaning underpinned by a central concept or idea” - Defining and naming themes – the story of the theme
- Writing up
