Research Design and Research Question Flashcards
When Are Surveys Likely to Be a Wise Design Choice?
- The data are best obtained directly from the respondents.
- Your data can be obtained by brief answers to structured questions.
- You can expect respondents to give you reliable information.
- You know how you will use the answers.
- You can expect an adequate response rate.
What is the effect of social desirability bias on surveys?
It is often assumed that social desirability bias reduces the validity of much survey research, meaning that respondents may answer questions to reflect what they think is socially appropriate rather than what they really believe or do
What should the response rate be to ensure generalizability?
if your response rate is less than 50%, you can have little confidence in the generalizability of your answers
NB: Purposive sample for explorative survey- lower response rate acceptable
What are the three Modes of Administering Your Survey?
The three modes of administering a survey are face-to-face, telephone, and self- administered (paper or electronically)
What are the three basic approaches to time in survey research? and when should they be used?
if you want to study change over time, you need to measure your variables at more than one time.
The three basic approaches to time in survey research are: panel studies, cohort studies, and cross-sectional studies
In a panel study, you survey the same group of respondents two or more times.
- use when you want to record changes in respondents and when you have access to a group of respondents over time
In cohort studies, you take samples from the same population two or more times.
- use if you want to study change over time in a large population
What Question Formats Can You Use in a Survey Design?
structured questions
unstructured questions
likert scale
What are the similarities of Interviews and Surveys
you ask people what you want to know and record their answers.
The basic problems are also the same: you need to make sure to ask the right questions, to ask them in the right way, and to make special efforts to understand the answers.
What are the diffferences of Interviews and Surveys
usually ask different kinds of questions
Surveys are often shaped by a desire to generalize to a broader population. Interviews are more aimed at understanding.
Several differences distinguish the sampling and coding practices:
o Survey researchers usually ask questions of a bigger sample
o Interviewers usually select interviewees through purposive or judgment sampling, while respondents in surveys are usually some sort of representative sample of a broader population.
o survey questions and methods are usually better at obtaining answers to comparatively direct, low-inference questions.
o Interview questions and methods are more suited to pursuing complicated matters in depth
o the standardization of survey questions helps improve the reliability of answers, while the in-depth probing of interview questions enhances the validity of the answers.
What are the different interview approaches?
At one end of the interview spectrum is explanatory/confirmatory research such as theory testing and identifying causal mechanisms. At the other end of the interview spectrum is descriptive/exploratory research, such as the life story or life narrative approach. The goal of this kind of research is “to learn as much as possible about how one person views his or her own development over time and across the life cycle.
A third option, situated roughly between theory testing and pure description, is the grounded theory approach.
When is a focus group interview a great option?
Focus group interviews may be especially appropriate when the topic of your research deals with interaction in groups.
Focus groups is good at addressing questions that are targeted.
when your research problem requires that you talk to more people than you can contact for individual interviews, a focus group interview is relevant.
because of the larger numbers typically involved in a focus group study and the greater focus on a narrower range of issues, they seem to produce results that are more generalizable than individual interviews
The most relevant situation when focus groups seem the ideal method occurs when your topic involves group dynamics or social relations.
When would you chose a formal interview setting?
If you want consistency, and through consistency to eliminate site characteristics as a variable, you will need to be more formal.
Describe Randomized control trials
Random selection and assignment of participants/cases into experimental groups, controlled treatment, so the differences in outcomes can be only due to treatment
What’s Wrong with Gold‑Standard Thinking?
The inflexibility of gold-standard thinking has two main facets.
o First is the assumption that evidence gathered using RCTs will always be superior, regardless of the topic or setting, to evidence gathered in any other way.
o Second is the belief that research problems for which RCTs are impossible cannot be studied scientifically.
o Both assumptions are fundamentally wrong.
When Is an RCT a Good Option?
When you want to find a causal link between two variables.
What is the Heisenberg effect?
those research occasions in which the very act of measurement or observation directly alters the phenomenon under investigation.
what are some advantages and disadvantages of experiments?
see picture in word
what is the history effect?
Events that happen in the environment that change the conditions of a study, affecting its outcome.
What are the different types of blinding experiments?
They should be used whenever possible and the more the better, since each level of blinding eliminates a possible source of bias.
In single-blind experiments, participants do not know which treatment they are receiving.
In double-blind experiments, the researchers do not know either.
In triple-blind experiments, the data analysts are likewise ignorant.
What is a within-subjects design?
It is a before-and-after study, also called a repeated-measures design.
You measure, treat, and measure again.
What is a between-subjects experiments ?
Between-subjects experiments make comparisons between groups of participants— those in the control and experimental groups.
Here we make random assignments of participants (symbolized by R) to control and experimental groups.
By comparing the final observation between the two groups, the researchers can be much more confident about the effects of X
What is a Crossed or a Nested experiment Design?
In a crossed design, each participant receives each level of treatment; the scores compared are before-and-after scores and comparisons are within-subject.
o disadvantage is the practice effect
In a nested design, each participant receives only one level of treatment; the scores compared are between groups of participants.
o Disadvantage: need more participants and is more complicated to analyze.
When Should You use a Laboratory Experiment and When for a Field Experiment?
Laboratory settings are appropriate when controlling for extraneous variables is important, and when the consistency of the laboratory environment can be maintained for the duration of your study.
- However, most social science researchers are interested in how well a treatment works in the real world, not just in the artificial environment of a laboratory.
Studies conducted in natural settings have the advantage of not requiring researchers to generalize from the artificially world of the laboratory to the real world.
o But the messy field environment in the real world can cause big problems for researchers.
- When the authenticity of the field environment is more important than controlling for extra variables, a field experiment is a logical choice.
When Should You Use Quasi‑Experiments?
You use quasi-experiments when you are unable to assign individuals to control and experimental groups, but you can select groups at random to receive different treatments—or you can assign treatments at random to groups of recruits or volunteers.
When Should You Use Natural Experiments?
Natural experiment is often used loosely to apply to the study of any naturally occurring differences among groups.
Here the data are generated in a more-or-less natural process or by an event that can approximate an experimental intervention.
You use natural experiments when you can “find” experimental conditions and collect the data generated by natural or social forces.
When to Combine Research Designs?
see word page 11
what is meant by triangulation?
Triangulation is the use of several means to examine the same phenomenon
it will broaden your understanding of the phenomenon of interest, whether by confirming a finding or by contradicting it—or by discovering new variables.
- You can build triangulation into your combined design in several ways: by using data sources, multiple methods, or multiple investigators.
what are the strengths/weaknesses of qual and quant respectively?
see word page 12
what is OBSERVATION DESIGN?
Observing processes (events, phenomena) as they unfold without interfering
- Takes long time
Use it when you are interested in little known processes, usually individual or social dynamics in natural settings
- Usually exploratory- good for developing new theory and finding/refining constructs
- Thick, contextualized, detailed description
- Possibility to uncover very complex, causal, interacting relationships
what is ARCHIVAL DESIGN?
Analyzing existing (secondary) data- can be both text and numbers
Use when you are interested in ”objective” information, cannot get information from respondents
Good for large scale research, where surveys and interviews would be too difficult/expensive, and Interested in historical processes (development over time)
