Chapter 7: Quantitative Study Designs: Nonexperimental

Question 1

What is a quasi-experimental design?

Answer 1

Designs that are not truly experimental (often do not have random assignment or comparison groups). Without the random assignment, the third criteria of causal inference may not be met, the researcher cannot be certain the DV was not caused by some factor other than the independent variable.

Question 2

What are the main types of quasi-experimental design?

Answer 2

1) Nonequivalent comparison group design
2) Time series study design
3) One group posttest only (no random assignment and no baseline assessment)
4) One group pre/post design (no comparison group)

Question 3

What is nonequivalent comparison group design?

Answer 3

Nonequivalent comparison group design is just like the pretest-posttest control group design except without random participant assignment.

Ex: you want to study the effect of providing extra recess to elementary students during the school day to increase physical activity. It would be difficult to randomly assign students in a class to get an extra recess break without providing it to others in the same class.

Question 4

What is the time series study design?

Answer 4

In the time series study design, there are multiple pretest and multiple posttest. This is useful for understanding the process of how behaviors or factors change over time, or how interventions or policies influence these behaviors or factors.

Question 5

What are cross-sectional study designs?

Answer 5

Cross-sectional study designs are widely used in exploratory or descriptive research. The researcher examines the phenomenon by collecting and analyzing data from one point in time or gets a cross sectional view of the phenomenon.

Assesses prevalence; not longitudinal because it’s not the same people

Pro: cheaper and don’t need to worry about attrition
Con: can’t study change, it’s a one time snap shot.

Surveys assessing a topic at one pint in time are cross -sectional.

Question 6

What are some types of longitudinal studies?

Answer 6

Longitudinal studies have a time order for data collection. By collecting data at different time points, researchers are able to determine whether the change in the IV precedes the change in the DV.

Looking for curvlinear change and want 3 or more data points.

Trend Studies
Cohort Studies
Panel Studies

Question 7

What are trend studies?

Answer 7

Trend studies are repeated cross-sectional studies designed to capture data at two more more time points from different samples of the same population. Trend explore population changes over time.

Looks at prevalence over time

Question 8

What are cohort studies?

Answer 8

Studies for which data are collected at two or more time points from individuals or groups with a common starting point. Cohort explore individual or group changes over time.

Question 9

What are panel studies?

Answer 9

Panel studies involve data collected at two or more points in time from the same individuals but may not have the a common starting point. Panel studies focus on individual changes.

Participants don’t have a common feature and nothing connects the group but chance

Question 10

What is a one-shot case study?

Answer 10

A one-shot case study is a design in which measurement of one group takes place after an intervention. There is no covariation, baseline measurement or comparison group. Useful to explore perceptions, behaviors or practices after a natural disaster or some sort of unexpected occurrence for which no baseline data was collected prior to the event.

Question 11

What is the one-group pretest-posttest?

Answer 11

There is a baseline measurement before and after the intervention and the research can explore covariation. The design has low internal validity mainly due to the threats of history, maturation, and testing.

Question 12

What is the posttest only with nonequivalent groups design?

Answer 12

Similar to the one-shot case study design but adds a comparison group. No pretest, intervention for one group and posttest for both groups. Low internal validity because of threats of selection bias. Without random assignment, there is no way of knowing whether the groups are comparable.

Question 13

What is a single case study design?

Answer 13

A single case study design is less common in public health than other fields such as psych. An individual is assessed on the dependent variable multiple times prior to and after the intervention. The multiple data points show trends and sustainability of intervention outcomes.

Question 14

What is external validity?

Answer 14

External validity is the extent to which a study’s findings are applicable or generalizable to other populations and settings.

Perfect external validity is unlikely as there are several factors that challenge it.

Question 15

What are the two main types of external validity?

Answer 15

Population External Validity

Ecological External Validity

Question 16

What is Population External Validity?

Answer 16

Population external validity refers to how well the research on a sample can be generalized to other samples or the whole population

Question 17

What is Ecological External Validity

Answer 17

Ecological External Validity pertains to how well the results can be generalized to other settings or contexts.

Question 18

What are some of threats to external validity?

Answer 18

Testing effect

The potential interaction among “treatments” o intervention components, this interaction may contribute to the outcome

Selection bias as a threat to generalizability

The way in which concepts are defined and tested in a study may reduce the likelihood of generalizability.

Question 19

What should you include when reporting study results?

Answer 19

Recruitment procedures
Participation and attrition rates
sample representativeness
setting
implementation consistency
importance of outcomes to a variety of audiences
sustainability and long-term effects.

Question 20

What are the pro’s of survey’s?

Answer 20

Surveys are generally the only source of publically available secondary data analysis options

Flexible with methods of assessment (in person, phone, web, mail, etc)

Wide range of types of questions (Can assess current and past behaviors as well a plans for future behavior)

Can be very cost effective (technology vs staff; secondary analysis)

Can be used when experimental manipulation is not possible/desirable

If using probability sampling, can generalize to large groups of people

The potential for anonymity can increase responding to sensitive questions

Question 21

What are some of the con’s of surveys?

Answer 21

Unlike experiments, there is no manipulation of a factor or opportunity for an intervention (mostly observation)

Primarily rely on self report

Recruiting can be difficlt (not legal to robocall cel phones)

If using probability sampling, analysis can be more challenging because of need to adjust analytic approach

May be difficult to establish rapport that can be critical for the assessment of sensitive topics.

Question 22

What are some potential sources of error ?

Answer 22

Coverage error - are you getting the right people?

Sampling error - a measure of variability in response.. big is bad

Non-response error - one of the most challenging, you are trying to compare your group to the group you couldn’t find or enroll

Measurement error - bad questions, bad reporting of the info, bad assessment technique

Selection bias - groups are not comparable (mostly clinical trials)

Question 23

What are some types of missing data ?

Answer 23

MAR - missing at random ; “Ignorable missingness”
Missing variables unrelated to predictor but may be related to outcome

MCAR -missing completely at random
Missing variable unrelated to both predictor and outcome

MNAR - missing, not at random; “This is a problem”
Missingness related to study variables
This questions external validity
Questions causing people not want to participate in intervention.

Question 24

How do we collect data out of the various study designs?

Answer 24

Each study design has certain data collection features that may be unique to it :
Biochemical Verification
Genetic Testing
Lab Test
Physical Example

There are certain types of assessments that can be easily used across study designs
1. Interviews (structured, unstructured, semistructured)
Face to face, telephone Computer Assisted

2.Questionnaires (Self-interviewing)
Internet, CASI, ACasi (low literacy groups) IVR, MAIL, Paper and Pencil

Question 25

What different types of questions can be used to collect data?

Answer 25

Open ended questions
Partially open ended question
Close ended questions 
Likert Scales (5 is often better than 7)
Visual analog scales

Keep tough questions in the middle, you don’t want to start or end the meeting with tough questions