Week 5 Flashcards
What is experimental research?
Designed to control for confounding variables
based on logic, changes are systematically introduced, “cause & effect”
Extraneous variables
any factor NOT related to the purpose of the study
may affect dependent variables
In experimental research, the control group may receive what?
standard treatment, no treatment/”wait and see”, placebo treatment
What is the gold standard of experimental research?
RCT
three essential components of true experimental designs
- independent variable “manipulated” by the experimenter
- Include a control or comparison group
- participants are randomly assigned to groups
What is random assignment?
** not the same as random sampling **
each participant has EQUAL chance of being assigned to any group
helps control for extraneous variables or prognostic indicators
should result in homogeneous groups at baseline (not IV)
Random Assignment Strategies
Simple random assignment
Block random assignment
Stratified random assignment
Cluster random assignment
What is simple random assignment
coin flip
every person has an equal chance to be in either group
can have UNEVEN groups
what is block random assignment
envelopes
divided equally into “blocks”
assures groups will be evenly distributed by group or treatment sequence
what is stratified random assignment
used when certain characteristics (attributes) may be confounding
what is cluster random assignment
ex: 3 different hospitals
clusters are formed, and each cluster is randomly assigned a treatment
-all members of a cluster get the same treatment
Concealed Allocation
the researchers do NOT know group assignment
types of Experimental Design Validity
statistical conclusion validity
internal validity
construct validity
external validity
statistical conclusion validity
“is there a relationship between the independent and dependent variables?”
Internal Validity
“is there evidence of a causal relationship between independent and dependent variables?”
Construct Validity
“to what constructs can results be generalized?”
external validity
can the results be generalized to other persons, settings, or times?
assumption of causality requires three components:
- Temporal precedence
- Covariation of cause and effect
- no plausible alternative explanation
temporal precedence definition
cause precedes effect
covariation of cause and effect definition
outcome ONLY occurs in the presence of the intervention
No plausible alternative explanation definition
consider confounding variables
Internal Validity Internal Threats
History, Maturation, Attrition, Testing, Instrumentation, Regression to the mean, Selection, and Social Interaction
Internal Threat: History
an unrelated event influences outcomes - did something else happen between measurements?
ex: caffeine
Internal Threat: Maturation
Passage of time affects DV
Internal Threat: Attrition
“experimental mortality”
-patients drop out
diet to restrictive?
diet not working?
Internal Threat: Testing
Collection data changes the response
Internal Threat: Instrumentation
a measuring instrument (or measurer) changes over times
Internal Threat: Regression to the mean
extreme scores become less extreme over time
Internal Threat: Selection
Problematic when participants self-select groups
addressed with random assignment
Internal Threat: Social Interaction
- diffusion/imitation
- compensatory equalization
- compensatory rivalry
- demoralization
Internal validity social threats
Diffusion/imitation, compensatory equalization, compensatory rivalry, demoralization
Social threats: Diffusion/imitation
participants in control group changed behavior to match experimental group
Social threats: Compensatory equalization
Researchers treat participants in control group differently to make up the difference
-researcher feels bad for a group
Social threats: compensatory rivalry
participants in control group look at a LOT of puppy photos to increase happiness
Social threats: demoralization
participants in the control group “give up”
Ruling Out threats to internal validity
random assignment and blinding
types of blinding
single = patient doesn’t know
double = patient + PT OR patient + assessor
triple = extremely rare
Threats to Construct validity
operational definitions, comprehensive measurements, time frame, multiple treatment interactions
Threats to external validity
- influence of selection
- influence of settings
- influence of history
influence of selection definition
age, gender, diagnosis
Influence of settings definition
RCT vs Pragmatic clinical trials
Influence of history
older studies still have value!
with Noncompliance participants may:
- refuse the assigned treatment after allocation
- cross over to another group
- be compliant with assigned treatments
- withdraw from the study
Missing Data
- needs to be accounted for throughout trial
- concerning if > 20%
- a problem if related to treatment
- drop-outs? switch-overs?
Per Protocol Analysis
analyze only those who completed study
Intention-to-Treat (ITT) Analysis
-analyze in the group that they were assigned to
-imputation: estimating missing data values
Researchers can minimize bias using
randomization, concealed allocation, blinding, and ITT
Internal threats are
INSIDE the study (methodology)
External threats are
OUTSIDE the study (generalizing)
True Experimental design is
- IV “manipulated” by researcher
- at least two comparison groups
- subjects randomly assigned
Quasi-experimental designs
- may lack randomization
- may lack comparison group
- may lack both
types of experimental designs
- between-subjects design
- within-subjects
- factorial
Between-subjects design
assigned to independent groups
Within-subjects design
participants act as their own controls
Factorial design
how we describe designs that have 2+ IV
- two-way design, 3x3 factorial design
One-way Designs
only 1 IV
AKA single-factor designs
Pretest-posttest control group design
- used to compare 2 or more groups formed by random assignment
Pretest-Posttest control group design
- both groups are measured before and after treatment
- cause and effect
- internal validity is strong
Posttest-only Control Group Design
- similar to pretest-posttest control group, but no pretest is administered to either group
- used when pretest is impractical, contraindicated, or potentially reactive
- internal validity is strong
what should we assume about the posttest-only control group design
assume groups are equivalent due to random assignment
best with large group of subjects
Factorial designs for independent groups
Two-way factorial design
Randomized block design
Two-way factorial design
- incorporates two or more independent variables
- randomly assigned to various combinations of the levels of the variables
Randomized block design
- 1 IV is not randomized
- attribute variable
- blocked
Benefits of factorial designs for ind groups
gives you information beyond what a single-factor (one-way) design would
Challenges of factorial designs for ind groups
- Larger sample
- results of analysis can be difficult to interpret
Naming Factorial Designs definition
- described according to the number of IVs
- number of levels within each factor
Designs for Repeated Measures
same people in each level of the IV
- within-subject design
One-way (aka single-factor) repeated measures design
- subjects are exposed to all levels of the IV
- there is no control group
Effects of Repeated Measures
- practice effects
- carryover effects
- order effects
practice effects definition
the more you do something, the more efficient you become
carryover effects definition
the effects of the first rx still present when second rx is introduced
Order effects definition
use of Latin Square as a possible solution of order effects
Crossover designs
- participants are randomized to a treatment sequence
- used to control for order effects
- considerations for washout period
Two-way (multi-factor) repeated measures design
all IVs are “repeated” or “within-subject”
Experimental designs can be _____
between-subject, or within-subject, or mixed
a one-way pretest-posttest control group design is the _______
most basic structure of a RCT
A week before the end of the study, all employees are told that there will be layoffs. The participants are stressed on the date of the post-test, and performance may suffer.
What type of internal threat?
History
What type of internal threat?
Most participants are new to the job at the time of the pre-test. A month later, their productivity has improved as a result of time spent working in the position.
maturation
What type of internal threat?
In the pre-test, productivity was measured for 15 minutes, while the post-test was over 30 minutes long.
Instrumentation
What type of internal threat?
Participants showed higher productivity at the end of the study because the same test was administered. Due to familiarity, or awareness of the study’s purpose, many participants achieved high results.
Testing
What type of internal threat?
Low-scorers were placed in Group A, while high-scorers were placed in Group B. Because there are already systematic differences between the groups at the baseline, any improvements in group scores may be due to reasons other than the treatment.
Selection bias
What type of internal threat?
Because participants are placed into groups based on their initial scores, it’s hard to say whether the outcomes would be due to the treatment or statistical norms.
Regression to the mean
What type of internal threat?
Groups B and C may resent Group A because of the access to a phone during class. As such, they could be demoralized and perform poorly.
Social interaction
What type of internal threat?
20% of participants provided unusable data. Almost all of them were from Group C. As a result, it’s hard to compare the two treatment groups to a control group.
Attrition bias
