Lecture 7 Flashcards
1
Q
____ is the key to controlling the outcomes from experimental research.
A
design
2
Q
6 general issues in experimental design:
A
- internal validity
- external validity
- frame of reference
- longitudinality
- frequency
- nested factors
3
Q
Internal validity is …
A
the basic minimum without which any study is not interpretable
4
Q
Internal validity is the extent to which the results of a study can:
A
- be attributed to the design of the study
- eliminate any rival hypothesis or events influencing the study
5
Q
To gain internal validity, the researcher attempts to control and/or eliminate possible _____ variables.
A
extraneous
6
Q
Internal validity is particularly important in ______ studies as well as other studies that establish _____-_____ or _____ _____.
A
- experimental
- causal-effect
- causal relationships
7
Q
When we think the independent variable produces a change in the dependent variable, what do we have to make sure of?
A
- must rule out the possibility of other factors producing the observed effect
- must consider the appropriateness of the methods (participants, experimental design, measurements)
8
Q
Main threat in internal validity:
A
unmeasured process accounting for the observed results
9
Q
Secondary threat in internal validity:
A
responses do not correctly reflect underlying dimensions
10
Q
How to minimize threats to internal validity:
A
- random assignment of participants into treatment and control groups
- ID confounding factors
- multiple methods approach provides evidence leading to similar results
11
Q
3 main categories to threats to internal validity:
A
- single group threats
- multiple group threats
- social threats
12
Q
Single-group threats:
A
a threat to internal validity that occurs in a study that uses only a single program or treatment group and no comparison or control
13
Q
Multiple-group threats:
A
an internal validity threat that occurs in studies that use multiple groups
14
Q
Social threats:
A
an internal validity threat that arise because social research is conducted in real-world human contexts where people will react to not only what affects them, but also to what is happening to others around them
15
Q
Types of single-group threats:
A
- history
- maturation
- testing
- instrumentation
- mortality
- regression
16
Q
History (single-group threat):
A
- events occurring during he experiment that are not part of the treatment
- eg. children watching Sesame Street and math score
17
Q
Maturation (single-group threat):
A
- process within the participants that operate as a result of time passing
- eg. children mature and gain muscle strength
18
Q
Testing (single-group threat):
A
the effects of one test on subsequent administrations of the same test
19
Q
Instrumentation (single-group threat):
A
changes in instrument calibration, including lack of agreement within and between observers
20
Q
Mortality:
A
loss of participants from comparison group for nonrandom reasons
21
Q
Regression:
A
the fact that groups selected based on extreme scores are not as extreme on subsequent testing
22
Q
Multiple group threats to validity is ____ ____:
A
- selection bias
- any factor other than the program that leads to posttest differences between groups
23
Q
Multiple-group threats types:
A
- selection-history
- selection-maturation
- selection-testing
- selection-instrumentation
- selection-mortality
- selection-regression
24
Q
Selection-history:
A
A threat to internal validity that results from any other event that occurs between pretest and posttest that the groups experience differently
25
Selection-maturation:
A threat to internal validity that arises from any differential rates of normal growth between pretest and posttest for the groups.
26
Selection-testing:
Selection-testing: A threat to internal validity that occurs when a differential effect of taking the pretest exists between groups on the posttest
27
Selection-instrumentation:
A threat to internal validity that results from differential changes in the test used for each group from pretest to posttest
28
Selection-mortality:
A threat to internal validity that arises when there is a
| differential nonrandom dropout between pretest and posttest
29
Selection-regression:
A threat to internal validity that occurs when there are
| different rates of regression to the mean in the two groups
30
Social interaction threats occur because...
- the various groups participating in the
study and the researchers carrying out the study are aware of each other’s existence and role in the study
- many of these people come in contact with each other
- try to isolate
31
Why is it hard to randomly assign and isolate to get rid of social interaction threats?
it introduces external validity (generalizability) concerns
32
4 types of social interaction threats:
- diffusion or imitation of treatment
- compensatory rivalry
- resentful demoralization
- compensatory equalization of treatment
33
Diffusion or imitation of treatment:
Occurs because a
| comparison group learns about the program either directly or indirectly from program group participants.
34
Compensatory rivalry:
Occurs when one group knows the program another group is getting and, because of that, develops a competitive attitude with the other group
35
Resentful demoralization:
Occurs when the comparison group knows what the program group is getting and becomes
discouraged or angry and gives up
36
Compensatory equalization of treatment:
Occurs when the
control group is given a program or treatment designed to make up for or “compensate” for the treatment the program group gets
37
Many threats to internal validity are controlled by making the participants in the experimental and control groups as _____ as possible.
alike
38
4 methods for controlling internal validity threats:
- randomization (random assignment
- placebo
- blind setup
- double-blind set up
39
Randomization (random assignment):
- Controls for history up to the point of the experiment but not during the experiment
- Controls for maturation because passage of time is equivalent in all groups
40
Placebo:
a control group receives a false treatment while the
| experimental group receives the real treatment
41
Blind setup:
the participant does not know whether he/she is receiving the experimental or control treatment
42
Double-blind setup:
neither the participant nor the experimenter knows which treatment the participant is receiving
43
External validity threats arise when...
experimenters draw
| incorrect inferences from the sample data to other individuals, settings, and past/future situations
44
Why do external validity threats arise?
- characteristics of individuals selected
- uniqueness of the setting
- timing of the experiment
45
3 major threats to external validity:
- people
- places
- times
46
How to increase external validity:
- Drawing on a representative probability sample and avoiding obtrusive measures
- Develop a model for the population from which one has a sample selected
- Replicate the sampling procedure and see how it compares to original sample taken
47
Experimental research involves comparisons of ____ performance with some _____ of performance.
- observed
| - expectations
48
2 types of comparisons:
- relative
| - absolute
49
Relative comparisons:
what would have occurred without the intervention or the difference between 2 of more interventions?
50
Absolute comparisons:
what is the degree to which the interventions reaches some particular desired outcomes?
51
_____ comparisons are harder to implement.
relative
52
____ comparisons require more advanced planning because..
- absolute
| - a consensus or other evidence has to be used to compare
53
Longitudinality:
the extent to which measures are repeated and extended across time
54
Challenge with longitudinality:
mortality (ie. dropping out) over the course of time
55
One way to overcome mortality is...
overlapping panel designs with different cohorts
56
Frequency:
the number of repetitions within a time period
57
Higher frequency leads to...
higher costs
58
Nested factors:
effects which are restricted to a single level of a factor are said to be nested within a factor
59
In nested factors, selection of _____, _____, or _____ are not selected ______ from each other.
- locations
- individuals
- organizations
- independently
60
In nested factors, there may be an interaction effect between ...
treatment within the embedded groups
61
____ ____ ___ may be the most powerful means of generating new knowledge as their confidence in the ____ ____ _____ relationship.
- true experimental design
| - cause and efffect
62
True experimental design is typically conducted in a _____ setting.
- laboratory
| - controlled environment
63
True experimental design has at least one _____ group (also termed _____ or ____ group).
- experimental
- treatment
- intervention
64
True experimental design also has at least one _____ group.
control
65
In true experimental design, membership is typically defined using _____ _____.
random assignment
66
Any study that is designed with .... is considered a true experiment.
- random assignment of participants
- a control group
- manipulation of the independent variable
67
A true experimental study design allows researchers to ____ data and test their ______ with a minimum _____ or ______ from _____ variables.
- collect
- hypothesis
- influence
- contamination
- extraneous
68
In true experimental study, ____ validity is high but ____ validity is hard to claim. Why?
- internal
- external
- tightly controlled settings will have a harder time generalizing to a larger setting
69
R =
random assignmnet
70
O =
observation of DV
71
X =
treatment by IV
72
A1, B1, A2, B2 =
notation for multiple IV, moderator, or control variables in factorial design
73
Quasi-experimental studies are those that are ...
"sort of " experimental in design
74
The big distinguishing feature between a true experiment and a quasi-experimental design is that...
there is no randomization to groups in a quasi-experimental research design
75
Quasi-experimental studies are low on ____ validity because...
- internal
| - groups are not likely to be equivalent at the outset of the study
76
Quasi-experimental studies are often higher on _____ validity than true experiments because...
- external
| - people are studied in a natural environment, making generalizability more feasible
77
The basic formula for a quasi-experimental study is:
- people are studied in real-word settings
- an independent variable is introduced or manipulated
- there is a dependent variable (the effect) that is measured.
78
Xa, Xb, Xc =
levels of the IV
79
C =
groups assigned according to a cutoff score
80
Repeated measures design is a specific type of ____ ______ design where there is ...
- quasi-experimental
| - only an experimental group with multiple levels of the independent variable
81
In repeated measures design, _____ are often ____ _____ control group.
- participants
| - their own
82
Strength of repeated measures design is that it is ...
economical in both time and effort of the researcher since the volunteer participants are involved several times with the study
83
In pre-experimental design, researchers examine .... and provide an _____ during the study.
- one group of individuals
| - intervention
84
In pre-experimental design, there is either an _____ of a control group for ______ or a ____ _____ _____ group is identified during the study or after the intervention.
- absence
- comparison
- non-equivalent comparison
85
Pre-experimental research design pose challenges to both _____ and _____ validity due to ...
- internal
- external
- not being able to control for many of the threats
86
Non-experimental research design has:
- no randomization to groups
- no groups to randomize to
- no manipulation of the IV
87
Non-experimental research design relies on researchers' ______ and ______ to arrive at conclusions.
- interpretation
| - observations
88
Non-experimental research design is used to report ______ and ______ among variables of interest.
- correlations
| - associations
89
Non-experimental research design generally looks at _____ research problems such as...
- descriptive
- cross section (participants assessed at one point in time)
- longitudinal (following participants over time)
