Lecture 21: Non-Experimental and Quasi-Experimental Designs Flashcards

1
Q

non- and quasi-experimental designs

A
  • In both non- and quasi-experimental designs, there is no manipulation (by the experimenter) of the IV
  • Researchers compare pre-existing groups on a DV of interest
  • Groups are defined by time or participant characteristics
  • Both non-and quasi-designs produce groups of scores to be compared for significant differences
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

characteristics of non- and quasi-experimental designs

A
  • No random assignment
  • We cannot determine causality
  • Groups can differ on other factors: internal validity issues
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

research design terms used in quasi- and non-experimental designs

A
  • Quasi-independent variable (IV): the variable used to differentiate the groups of participants or the groups of scores being compared
  • Dependent variable (DV): the variable that is measured
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

similarities between non- and quasi-experimental designs

A
  • No manipulation of the IV
  • No random assignment
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

differences between non- and quasi-experimental designs

A

Quasi-experimental uses controls for confounds to keep internal validity high, but non-experimental doesn’t

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

IV in non- and quasi-experimental designs

A
  • Typically, the IV is a participant variable (age, marital status) or a time variable (pre- or post-event)
  • The researcher usually cannot control the assignment of participants to groups because the groups already exist
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

3 general categories of non- and quasi-experimental research

A
  • between-groups design
  • within-groups design
  • developmental research design
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

types of between-groups designs

A
  • Differential design: non-experimental
  • Post-test only NEG design: non-experimental
  • Pre-test/post-test NEG design: quasi-experimental
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

types of within-groups designs

A
  • One group pretest-posttest design: non-experimental
  • Time series design: quasi-experimental
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

types of developmental research designs

A
  • Cross-sectional designs: between-subjects 2 or more groups
  • Longitudinal designs: within-subjects 1 group, measured 2 or more times
  • Cross-sectional and longitudinal: between- and within-subjects 2+ groups, measured 2+ times
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

non-equivalent group (NEG) between-groups design

A
  • Pre-existing groups are used
  • Groups are differentiated by one specific factor
  • The purpose is to show that this factor is responsible for differences between the scores of the groups
  • Called “non-equivalent groups” because the experimenter cannot control the group’s membership (or individual differences)
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

assignment bias in non-equivalent group (NEG) between-groups design

A
  • These designs raise the problem of assignment bias
  • Groups may differ on a variable other than the IV and this can be a possible confound
  • Precludes cause-and-effect explanations
  • Because of this lack of control, there is a threat to internal validity
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

differential designs (non-experimental)

A
  • Also called an ex-post facto (after the fact) design
  • Simplest of the between-group (NEG) designs
  • Individual differences are the primary interest: form groups based on these differences
  • Simply compare these pre-existing groups to find a difference in the variable of interest
  • No manipulation of control of assignment of participants to groups; just comparing them on pre-existing characteristics
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

example of a differential design

A

SES, stress, and cognitive function study found that there were no differences in cognitive functioning between high and low SES groups but there were early stress differences in the younger groups

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

Post-test-only NEG designs

A
  • Interested in examining the effectiveness of a treatment or intervention (applied research)
  • No manipulation or control over who is exposed to the treatment or intervention
  • Compare those who received treatment vs. those who did not, based on pre-existing groups
  • No control over the assignment of participants to groups; one group is measured after tx, other group is measured at the same time (but received no treatment)
  • Commonly used when tx is given to a cluster of individuals and compared to another cluster
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

assignment bias in post-test-only NEG designs

A

This method does not address the problem of assignment bias

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

pre-test/post-test NEG designs

A
  • With this design, we try to add a bit of control by including a pre-test measurement for both groups
  • Much stronger version than the NECG
  • Limits threats to internal validity
  • Can compare groups before treatment and look at differences pre- and post-treatment
18
Q

how can you reduce threats to internal validity in pre-test post-test NEG designs

A
  • Determine how similar the two groups are, before administering treatment by comparing pre-treatment measures
  • If similar at pre-test, there is evidence that Ps in one group is not substantially different from those in another group
  • Although this does not make the groups equivalent, it does reduce the impact of assignment bias on the results
  • With this technique, you are controlling for one specific variable
  • With randomization in a true experimental design, you can target multiple variables
19
Q

what threats are pre-test post-test NEG designs vulnerable to?

A

the same threats to internal validity as experimental designs (history, instrumentation, testing effects, maturation, and statistical regression)

20
Q

how should we interpret the following findings in a pre-test post-test design?: participants are similar at pre-test but there is a post-treatment difference between groups

A

we can be somewhat confident that it is due to the treatment

21
Q

advantage of pre-test post-test NEG designs

A

limit time-related threats

22
Q

one-group non-experimental pretest-posttest designs

A
  • Studies in which a series of observations are made over time in just one group of participants
  • There is no attempt to control threats to internal validity
  • Two observations are made for each participant, one before and one after treatment
23
Q

goal of one-group non-experimental pretest-posttest designs

A

evaluate an intervention/treatment by comparing observations made before with those made after

24
Q

assignment bias in one-group non-experimental pretest-posttest designs

A
  • We are not comparing groups (no control group) but rather individual scores before and after treatment
  • The problem of assignment bias is eliminated
25
Q

downsides of one-group non-experimental pretest-posttest design

A
  • You cannot counterbalance the order of treatment as you can in experimental designs
  • You can’t be sure that a change post-treatment is due to the treatment and not some other factor
26
Q

threats to one-group non-experimental pretest-posttest designs

A

Threatened by time-related factors (history, instrumentation, testing effects, maturation, regression)

27
Q

time-series designs

A
  • Attempt to add some control by making a series of observations for each participant
  • Some observations before and some after treatment
  • Similar to single-case designs
28
Q

Value of repeated pretest and posttest observations within participants:

A
  • Allows researchers to see existing trends in data
  • Identify practice, fatigue, maturation, instrumentation, and regression, since they should be measurable before treatment
  • Allows researchers to observe post-treatment trends
  • Can examine manipulated treatments
  • Can look at non-manipulated events (data from archives) and societal changes
29
Q

how many data points are needed in a time-series design?

A
  • You need at least 3 data points before the treatment and 3 after
  • Multiple observations allow you to see if the change is due to your treatment or not and examine post-treatment trends to see if the effect of X fades
30
Q

threats to internal validity in time-series designs

A
  • An external event can be a threat to internal validity in time series designs but only if it occurs at the same time as the treatment
  • If treatment and outside events occur at different times, one can separate the effects in the time series using statistical techniques
31
Q

cross-sectional developmental research designs

A
  • A between-groups design that uses different groups for each age or age bracket
  • The outcome variable is measured for each group at one point in time and the groups are compared
  • When a study evaluates differences related to age, the design is typically called cross-sectional
32
Q

example of a cross-sectional developmental research design

A

differences in ease of using technology by age group

33
Q

advantages of cross-sectional developmental research designs

A
  • Time-efficient (one measurement time-point)
  • No long-term cooperation required
34
Q

disadvantages of cross-sectional developmental research designs

A
  • Cohort or generation effects (changes due to era or period)
  • Individual changes are not assessed
35
Q

longitudinal developmental research designs

A
  • A within-groups design that measures a variable in the same group of individuals over time
  • Participants have usually grown up in similar circumstances
  • One group’s pretest-posttest measures of health are collected
  • The treatment is age
36
Q

advantages of longitudinal developmental research designs

A
  • No cohort or generation effects
  • Individual changes are assessed
37
Q

disadvantages of longitudinal developmental research designs

A
  • Time-consuming and costly
  • Participant attrition can cause unbalanced measures
  • Potential for practice effects over time (repeated tasks)
38
Q

cross-sectional longitudinal designs

A
  • Also known as mixed developmental designs
  • Compare results that were obtained from separate samples (cross-sectional) that were obtained several times (longitudinal)
  • Allows for the examination of the development of behaviours due to mechanisms other than individual aging
  • Can draw comparisons within groups (across time) and between groups (same time point)
39
Q

example of a cross-sectional longitudinal design

A

influence of sport on physical development

40
Q

Major distinction of non- and quasi-experimental designs from experimental designs

A

no random assignment of participants to conditions

41
Q

Major distinction of non-experimental from quasi-experimental designs

A

no attempt at control comparison in non-experimental designs