Lecture 21: Non-Experimental and Quasi-Experimental Designs Flashcards
non- and quasi-experimental designs
- In both non- and quasi-experimental designs, there is no manipulation (by the experimenter) of the IV
- Researchers compare pre-existing groups on a DV of interest
- Groups are defined by time or participant characteristics
- Both non-and quasi-designs produce groups of scores to be compared for significant differences
characteristics of non- and quasi-experimental designs
- No random assignment
- We cannot determine causality
- Groups can differ on other factors: internal validity issues
research design terms used in quasi- and non-experimental designs
- Quasi-independent variable (IV): the variable used to differentiate the groups of participants or the groups of scores being compared
- Dependent variable (DV): the variable that is measured
similarities between non- and quasi-experimental designs
- No manipulation of the IV
- No random assignment
differences between non- and quasi-experimental designs
Quasi-experimental uses controls for confounds to keep internal validity high, but non-experimental doesn’t
IV in non- and quasi-experimental designs
- Typically, the IV is a participant variable (age, marital status) or a time variable (pre- or post-event)
- The researcher usually cannot control the assignment of participants to groups because the groups already exist
3 general categories of non- and quasi-experimental research
- between-groups design
- within-groups design
- developmental research design
types of between-groups designs
- Differential design: non-experimental
- Post-test only NEG design: non-experimental
- Pre-test/post-test NEG design: quasi-experimental
types of within-groups designs
- One group pretest-posttest design: non-experimental
- Time series design: quasi-experimental
types of developmental research designs
- Cross-sectional designs: between-subjects 2 or more groups
- Longitudinal designs: within-subjects 1 group, measured 2 or more times
- Cross-sectional and longitudinal: between- and within-subjects 2+ groups, measured 2+ times
non-equivalent group (NEG) between-groups design
- Pre-existing groups are used
- Groups are differentiated by one specific factor
- The purpose is to show that this factor is responsible for differences between the scores of the groups
- Called “non-equivalent groups” because the experimenter cannot control the group’s membership (or individual differences)
assignment bias in non-equivalent group (NEG) between-groups design
- These designs raise the problem of assignment bias
- Groups may differ on a variable other than the IV and this can be a possible confound
- Precludes cause-and-effect explanations
- Because of this lack of control, there is a threat to internal validity
differential designs (non-experimental)
- Also called an ex-post facto (after the fact) design
- Simplest of the between-group (NEG) designs
- Individual differences are the primary interest: form groups based on these differences
- Simply compare these pre-existing groups to find a difference in the variable of interest
- No manipulation of control of assignment of participants to groups; just comparing them on pre-existing characteristics
example of a differential design
SES, stress, and cognitive function study found that there were no differences in cognitive functioning between high and low SES groups but there were early stress differences in the younger groups
Post-test-only NEG designs
- Interested in examining the effectiveness of a treatment or intervention (applied research)
- No manipulation or control over who is exposed to the treatment or intervention
- Compare those who received treatment vs. those who did not, based on pre-existing groups
- No control over the assignment of participants to groups; one group is measured after tx, other group is measured at the same time (but received no treatment)
- Commonly used when tx is given to a cluster of individuals and compared to another cluster
assignment bias in post-test-only NEG designs
This method does not address the problem of assignment bias
pre-test/post-test NEG designs
- With this design, we try to add a bit of control by including a pre-test measurement for both groups
- Much stronger version than the NECG
- Limits threats to internal validity
- Can compare groups before treatment and look at differences pre- and post-treatment
how can you reduce threats to internal validity in pre-test post-test NEG designs
- Determine how similar the two groups are, before administering treatment by comparing pre-treatment measures
- If similar at pre-test, there is evidence that Ps in one group is not substantially different from those in another group
- Although this does not make the groups equivalent, it does reduce the impact of assignment bias on the results
- With this technique, you are controlling for one specific variable
- With randomization in a true experimental design, you can target multiple variables
what threats are pre-test post-test NEG designs vulnerable to?
the same threats to internal validity as experimental designs (history, instrumentation, testing effects, maturation, and statistical regression)
how should we interpret the following findings in a pre-test post-test design?: participants are similar at pre-test but there is a post-treatment difference between groups
we can be somewhat confident that it is due to the treatment
advantage of pre-test post-test NEG designs
limit time-related threats
one-group non-experimental pretest-posttest designs
- Studies in which a series of observations are made over time in just one group of participants
- There is no attempt to control threats to internal validity
- Two observations are made for each participant, one before and one after treatment
goal of one-group non-experimental pretest-posttest designs
evaluate an intervention/treatment by comparing observations made before with those made after
assignment bias in one-group non-experimental pretest-posttest designs
- We are not comparing groups (no control group) but rather individual scores before and after treatment
- The problem of assignment bias is eliminated
downsides of one-group non-experimental pretest-posttest design
- You cannot counterbalance the order of treatment as you can in experimental designs
- You can’t be sure that a change post-treatment is due to the treatment and not some other factor
threats to one-group non-experimental pretest-posttest designs
Threatened by time-related factors (history, instrumentation, testing effects, maturation, regression)
time-series designs
- Attempt to add some control by making a series of observations for each participant
- Some observations before and some after treatment
- Similar to single-case designs
Value of repeated pretest and posttest observations within participants:
- Allows researchers to see existing trends in data
- Identify practice, fatigue, maturation, instrumentation, and regression, since they should be measurable before treatment
- Allows researchers to observe post-treatment trends
- Can examine manipulated treatments
- Can look at non-manipulated events (data from archives) and societal changes
how many data points are needed in a time-series design?
- You need at least 3 data points before the treatment and 3 after
- Multiple observations allow you to see if the change is due to your treatment or not and examine post-treatment trends to see if the effect of X fades
threats to internal validity in time-series designs
- An external event can be a threat to internal validity in time series designs but only if it occurs at the same time as the treatment
- If treatment and outside events occur at different times, one can separate the effects in the time series using statistical techniques
cross-sectional developmental research designs
- A between-groups design that uses different groups for each age or age bracket
- The outcome variable is measured for each group at one point in time and the groups are compared
- When a study evaluates differences related to age, the design is typically called cross-sectional
example of a cross-sectional developmental research design
differences in ease of using technology by age group
advantages of cross-sectional developmental research designs
- Time-efficient (one measurement time-point)
- No long-term cooperation required
disadvantages of cross-sectional developmental research designs
- Cohort or generation effects (changes due to era or period)
- Individual changes are not assessed
longitudinal developmental research designs
- A within-groups design that measures a variable in the same group of individuals over time
- Participants have usually grown up in similar circumstances
- One group’s pretest-posttest measures of health are collected
- The treatment is age
advantages of longitudinal developmental research designs
- No cohort or generation effects
- Individual changes are assessed
disadvantages of longitudinal developmental research designs
- Time-consuming and costly
- Participant attrition can cause unbalanced measures
- Potential for practice effects over time (repeated tasks)
cross-sectional longitudinal designs
- Also known as mixed developmental designs
- Compare results that were obtained from separate samples (cross-sectional) that were obtained several times (longitudinal)
- Allows for the examination of the development of behaviours due to mechanisms other than individual aging
- Can draw comparisons within groups (across time) and between groups (same time point)
example of a cross-sectional longitudinal design
influence of sport on physical development
Major distinction of non- and quasi-experimental designs from experimental designs
no random assignment of participants to conditions
Major distinction of non-experimental from quasi-experimental designs
no attempt at control comparison in non-experimental designs
