research final Flashcards
two types of asking questions
- interviews (telephone or face to face)
- self administered questionnaires (online or hard copies)
standardization
Everything done the same way each time
– In both the asking of questions and the recording of answers
Interview schedule:
a formal list of questions that the interviewer must follow in detail
– The questions must be asked in exactly the same way each time
– Must also be asked in the order given – the same order each time
Intra-interviewer variability:
Interviewer is not consistent in asking questions or recording answers
– Can happen during 1 interview, or across many
Inter-interviewer variability:
multiple interviewers are
inconsistent compared to each other
Sources of Error in Structured Surveys
- Poorly worded questions
- Interviewer error in asking a question
- Misunderstanding on the part of the interviewee
- Interviewee lapses in memory
- Interviewer error in recording information
- Mistakes in entering the data into a computer file and
- Biases caused by the innate characteristics of the interviewer and interviewees
two types of non response
– Item non-response: Respondents won’t (or can’t) answer specific questions
– Survey non-response: Potential respondents refuse to participate at all
CATI/CAPI
- CATI: Computer assisted telephone interviewing
- CAPI: Computer assisted personal interviewing– Reduces errors from inconsistent wording or question
order
– Capture and process data at the same time (since it goes right into the computer)
– Reduces item non-response due to interviewer error
Acquiescence:
the respondent is just trying to
please the researcher, be cooperative
Social Desirability:
Respondents give what they believe is the politically/socially correct answer
pros of phone interviews
– Cheaper
– Quicker to administer
– Easier to supervise
– Can also reduce bias from the characteristics of the interviewers or interviewees
cons of phone interviews
– Only people who can be contacted by phone during normal hours can be in the study
* If directory is used, non-listed and cell phone users left out * Hard for people with impaired hearing
– Most cannot exceed 20–25 minutes
– Sensitive issues may be more difficult to discuss in
this context
– Cannot see if interviewee understands or is upset
– The interviewer cannot use visual aids (like show cards)
advantages of questionnaires over interviews
Cheaper and more convenient to administer
– N.B. text says also faster – but depends on context * No interviewer effects
* Less social desirability bias
* Respondents are more likely to (honestly) answer sensitive questions
disadvantages of questionnaires
Greater risk of missing data (Higher survey and item non-response) and respondent error
– Why?
* Researcher cannot explain the question
* Cannot probe
* Cannot keep participants engaged
– So must be kept short and simple
* Very limited number and type of questions
– Limit misunderstandings and respondent fatigue
* Simplequestionsonly
– Limit open-ended questions and filtersCan be read as a whole before completion
– questions are then not really independent of each other
– Can potentially answer them in any order * Order effects may occur
* Not appropriate for certain respondents – Limited literacy
– Limited language proficiency
* Designated respondent may not have completed questionnaire!
– We have no way to know!
ways of administering a questionnaire
- through mail
- in person
- online
Feminist critique
Like much social science generally:
– these methods are ultimately exploitative
* involve asymmetrical power relationship between the researcher and the respondent
– established within a social system of knowledge that is also made of unequal systems of power
* Only some kinds of people get to ask questions, frame how they should be answered, or what constitutes evidence, whose viewpoints should be considered etc.
* Yet some opposition has declined in recent decadesWhy decline in opposition?
– increased attention given to the rights of research respondents
* privacy rights, the right to end the interview at any time, etc.
* Rights to help formulate and have access to research
– Changes to inclusivity in the scientific/knowledge production establishment
* Who is asking the questions, framing the discussion is changing over time
– Thesemethodsconsistentlyshowempiricalsupportforfeminist theoretical statements
* Like documenting the details of various forms of domination
– Someoppositionremains
* Change is slow and also unequal
* Some of the inequalities are actually inherent to the methods, and are about ultimate authority and ‘expertise’
– so while we can limit the difficulties, they will always there
Types of questions
- Personal factual questions
– Most common independent variables
* Ex. income, age, occupation, personal behaviour, etc.
– Problems: not many - memory, honesty 2. Factual questions about others
– Ex. spouse’s income, child’s activities, sister’s personal relationships etc.
– Problems: limited knowledge, bias, memory 3. Factual questions about an entity or event
– Ex. accounts of a political demonstration or riot
* How many people, what characteristics, what response etc.
– Problems: Witnesses are not researchers * Not specifically looking for those details
* Sameproblemsasabove4/5. Questions about attitudes/beliefs
* Very common variables
– Both cause and/or effect
* Heavy use of use of Likert scales - Questions about knowledge
* Ex. How many census metropolitan areas exist in Canada?
* less common is sociology
* Can lead to high item non-response
General rules for designing questions
- Focus on the research question(s)
- Make sure the question is going to tell you what you actually want to know
- Put yourself in the position of the respondent
Specific Rules
- Avoid questions that ask two questions
- Avoid leading questions
- Avoid questions that include negatives
- Ensure symmetry between a closed question and its answers
- Ensure that the answers provided for a closed question are balanced
- people have memory problems (don’t ask how long its been since something took place)
-question order should be uniform amount participants - avoid long questions
- ## avoid general questions
Advantages of Closed Questions
Easy to process answers
* Standardization of Q’s and A’s – Higher reliability
* Reduced bias in recording answers – No chance to ‘interpret’ responses
* Fixed responses help clarify what the question means
* Easier and quicker to complete– Mutually exclusive:
* No overlap between categories– Exhaustive:
* Everyone can answer the question
Pilot Studies
Small scale testing of your research instruments
– 1. Used to test whether individual items or the instrument as a whole operates well
* how long does it take?
* Is it well received?
* Does it have good flow?
– 2. Provide interviewers with experience in administering the instrument
– 3. Can identify questions that are embarrassing, uninteresting, difficult to
understand, ambiguous, etc.
– 4. Can be used to ensure that there is variability in the answers given
* Ex. Develop a scale attempting to measure ‘happiness’ (0-20) but everyone in pilot study scores 19 or 20.
* Either your target population is ‘very happy’ or your scale is problematic. (either way, a problem)
– 5. Can be useful in developing closed questions
Semi-structured
– Have a simple, changeable interview schedule – Usually called an interview guide in this context
– Have little or no interview schedule
Unstructured
– Usually used with personal documents
– Builds the story of one person, or one family over the course of their lives
– Usually retrospective
– Ex. Oscar Lewis, 1961, the Sanchez family in Mexico city (from the culture of poverty example)
Life history
what is a focus group?
- In-depth group interviews
– 6-10 participants usually considered optimum
– Usually meet on one occasion
– usually recorded
– work with a specific issue, and an interview guide - Discussion controlled by a skilled moderator
– success heavily dependent on his/her skills
– make sure everyone participates
– exploit group dynamics to obtain conflicting view points - collects a lot of information quickly
– Much of it not relevant
– But potential for new ideas, rare, interesting, or socially created ideas - potential problems:
– dominant/shy personalities
2 types of structured observation
- Natural Observation
– No manipulation of situation
– Just watch and record human behaviours in a ‘natural’ setting - Field Experiment
– Manipulation of an independent variable and observe differences between groups or over time
Reactive effects
- People change their behaviour when they know they are being studied/tested
Guinea Pig effect (Hawthorne effect):
– 1. Role selection
– 2. Trying to help the researcher
– 3. Researcher presence as a change agent
When is covert observation okay?
- public and non sensitive
-harmless and no threat to privacy
Humphrey’s (1970) Tearoom Trade
– Watched men have sex in a public bathroom by pretending to be a ‘watch queen’
– Took the license plate numbers of the men
– Used his contacts to track them to their homes
– Confronted them at home and interviewed them about their (very secret) activities
Closed settings
– privateorrestrictedsettings:
Overt observation
– the people being studied know they are being observed by a researcher
Covert observation
– the people being studied do not know they are being observed by a
researcher
Types of notes for ethnographic data?
Mental notes
Jotted notes
Direct observation notes
analytic memos
personal notes
– Written and symbolic material
* Have ‘text’ that can be ‘read’
* Provide ethical unobtrusive measures
* Content Analysis most common research type
– Third of the three direct ways we can know the world
* Were not created for the purposes of social research
Documents
– Collected for the express purpose of research, but not yours
* By another researcher/organization for their own research
* By an official body (ex. government) to be made available to researchers
Secondary data
– Aggregate information like crime rates, unemployment rates
– Collected by official body for their own purposes, but useful to social researchers
Official Statistics
pros of secondary analysis
– Often very high quality data
– Cost and Time
* Obviously much cheaper and less time consuming
* More time for analysis
* Esp with regards to: – longitudinal data
– subgroup analysis
– cross national research
– Replication of past research and Reanalysis of existing data
limitations of secondary analysis
– Potential absence of key variables – No control over data quality
– Data is often very complex
* Difficult to ‘learn’
