Chapter 7: Quantitative Study Designs: Nonexperimental Flashcards
Research Details and Design
What is a quasi-experimental design?
Designs that are not truly experimental (often do not have random assignment or comparison groups). Without the random assignment, the third criteria of causal inference may not be met, the researcher cannot be certain the DV was not caused by some factor other than the independent variable.
What are the main types of quasi-experimental design?
1) Nonequivalent comparison group design
2) Time series study design
3) One group posttest only (no random assignment and no baseline assessment)
4) One group pre/post design (no comparison group)
What is nonequivalent comparison group design?
Nonequivalent comparison group design is just like the pretest-posttest control group design except without random participant assignment.
Ex: you want to study the effect of providing extra recess to elementary students during the school day to increase physical activity. It would be difficult to randomly assign students in a class to get an extra recess break without providing it to others in the same class.
What is the time series study design?
In the time series study design, there are multiple pretest and multiple posttest. This is useful for understanding the process of how behaviors or factors change over time, or how interventions or policies influence these behaviors or factors.
What are cross-sectional study designs?
Cross-sectional study designs are widely used in exploratory or descriptive research. The researcher examines the phenomenon by collecting and analyzing data from one point in time or gets a cross sectional view of the phenomenon.
Assesses prevalence; not longitudinal because it’s not the same people
Pro: cheaper and don’t need to worry about attrition
Con: can’t study change, it’s a one time snap shot.
Surveys assessing a topic at one pint in time are cross -sectional.
What are some types of longitudinal studies?
Longitudinal studies have a time order for data collection. By collecting data at different time points, researchers are able to determine whether the change in the IV precedes the change in the DV.
Looking for curvlinear change and want 3 or more data points.
Trend Studies
Cohort Studies
Panel Studies
What are trend studies?
Trend studies are repeated cross-sectional studies designed to capture data at two more more time points from different samples of the same population. Trend explore population changes over time.
Looks at prevalence over time
What are cohort studies?
Studies for which data are collected at two or more time points from individuals or groups with a common starting point. Cohort explore individual or group changes over time.
What are panel studies?
Panel studies involve data collected at two or more points in time from the same individuals but may not have the a common starting point. Panel studies focus on individual changes.
Participants don’t have a common feature and nothing connects the group but chance
What is a one-shot case study?
A one-shot case study is a design in which measurement of one group takes place after an intervention. There is no covariation, baseline measurement or comparison group. Useful to explore perceptions, behaviors or practices after a natural disaster or some sort of unexpected occurrence for which no baseline data was collected prior to the event.
What is the one-group pretest-posttest?
There is a baseline measurement before and after the intervention and the research can explore covariation. The design has low internal validity mainly due to the threats of history, maturation, and testing.
What is the posttest only with nonequivalent groups design?
Similar to the one-shot case study design but adds a comparison group. No pretest, intervention for one group and posttest for both groups. Low internal validity because of threats of selection bias. Without random assignment, there is no way of knowing whether the groups are comparable.
What is a single case study design?
A single case study design is less common in public health than other fields such as psych. An individual is assessed on the dependent variable multiple times prior to and after the intervention. The multiple data points show trends and sustainability of intervention outcomes.
What is external validity?
External validity is the extent to which a study’s findings are applicable or generalizable to other populations and settings.
Perfect external validity is unlikely as there are several factors that challenge it.
What are the two main types of external validity?
Population External Validity
Ecological External Validity
What is Population External Validity?
Population external validity refers to how well the research on a sample can be generalized to other samples or the whole population
What is Ecological External Validity
Ecological External Validity pertains to how well the results can be generalized to other settings or contexts.
What are some of threats to external validity?
Testing effect
The potential interaction among “treatments” o intervention components, this interaction may contribute to the outcome
Selection bias as a threat to generalizability
The way in which concepts are defined and tested in a study may reduce the likelihood of generalizability.
What should you include when reporting study results?
Recruitment procedures Participation and attrition rates sample representativeness setting implementation consistency importance of outcomes to a variety of audiences sustainability and long-term effects.
What are the pro’s of survey’s?
Surveys are generally the only source of publically available secondary data analysis options
Flexible with methods of assessment (in person, phone, web, mail, etc)
Wide range of types of questions (Can assess current and past behaviors as well a plans for future behavior)
Can be very cost effective (technology vs staff; secondary analysis)
Can be used when experimental manipulation is not possible/desirable
If using probability sampling, can generalize to large groups of people
The potential for anonymity can increase responding to sensitive questions
What are some of the con’s of surveys?
Unlike experiments, there is no manipulation of a factor or opportunity for an intervention (mostly observation)
Primarily rely on self report
Recruiting can be difficlt (not legal to robocall cel phones)
If using probability sampling, analysis can be more challenging because of need to adjust analytic approach
May be difficult to establish rapport that can be critical for the assessment of sensitive topics.
What are some potential sources of error ?
Coverage error - are you getting the right people?
Sampling error - a measure of variability in response.. big is bad
Non-response error - one of the most challenging, you are trying to compare your group to the group you couldn’t find or enroll
Measurement error - bad questions, bad reporting of the info, bad assessment technique
Selection bias - groups are not comparable (mostly clinical trials)
What are some types of missing data ?
MAR - missing at random ; “Ignorable missingness”
Missing variables unrelated to predictor but may be related to outcome
MCAR -missing completely at random
Missing variable unrelated to both predictor and outcome
MNAR - missing, not at random; “This is a problem”
Missingness related to study variables
This questions external validity
Questions causing people not want to participate in intervention.
How do we collect data out of the various study designs?
Each study design has certain data collection features that may be unique to it : Biochemical Verification Genetic Testing Lab Test Physical Example
There are certain types of assessments that can be easily used across study designs
1. Interviews (structured, unstructured, semistructured)
Face to face, telephone Computer Assisted
2.Questionnaires (Self-interviewing)
Internet, CASI, ACasi (low literacy groups) IVR, MAIL, Paper and Pencil
What different types of questions can be used to collect data?
Open ended questions Partially open ended question Close ended questions Likert Scales (5 is often better than 7) Visual analog scales
Keep tough questions in the middle, you don’t want to start or end the meeting with tough questions
