Quant quiz 2 Flashcards
Quasi – Experimental Designs
Used when control features of experimental
design cannot be achieved
*Independent variable cannot be manipulated, or
*Random assignment to groups cannot be achieved
*Internal validity may be affected
Quasi Example- One-Group Posttest-Only Design
- Lacks a control/comparison group
- Lack baseline data
- Difficulty making inferences about the effect of the IV on the
DV (low internal validity)
Quasi Example- One-Group pretest-posttest design
- Now there is baseline comparison data, but no control/comparison group
- Threats to the internal validity of these types of studies:
- History
- Maturation
- Testing
- Instrument Decay
- Regression towards the mean
Threats- history
Refers to any event that occurs between the 1st and 2nd
measurement (not a part of the experiment)
- Any confounding event that occurs at the same time as the
experimental manipulation - EXAMPLE:
- change in political leader
- terrorist attack
- natural disaster
- Global pandemic!
Threats- maturation
People change over time
- Short periods of time:
- Boredom
- Fatigue
- Hunger
- Wiser
Long periods of time:
* Increased coordination
* Increased analytical skills
* Change of priorities/core values
* Life events
Threats- testing
Testing may be acting as an
intervention on its own
- A pre-test may sensitize participant in
unanticipated ways and their
performance on the post-test may be
due to the pre-test, not to the
treatment, - Or, more likely, and interaction of the
pre-test and treatment
Threats- Instrument decay
- The characteristics of the
measurement change over time - Ex: observation or self-
monitoring - The rater changes their rating
behavior over time due to any
number of reasons - Scale needs calibration
Threats- Regression towards the mean
- When participants score really high or really low on a
measure, they are likely to score closer towards the mean
when retested at a different time - Statistically, outliers tend to be less extreme when tested again
Nonequivalent control group
design
- Separate control group, but the participants in the two conditions are not equivalent
- i.e., they were not randomly assigned so we cannot guarantee equivalence…
- Selection Difference/ bias
- The groups are not a result of random assignment but are from existing natural groups
- Thus, not matter what, are vulnerable to selection bias
- Example:
- Treatment group are people who volunteered for treatment and the control group are people who just meet criteria
- There might be group differences between the groups of volunteers
Nonequivalent Control Post-test
Group Design
*Participants are not randomly assigned to groups
*No pre-test
Nonequivalent Control Group
Pretest-Posttest Design
*Still not randomly assigned groups, BUT the model
is improved with a pretest
*Not randomly assigned but we can test for some
equivalence using the pretest
Propensity score matching of
nonequivalent treatment and control
groups
*Nonequivalent control group problem: groups can
differ in important ways
*Improve with:
* Score matching
* matches one participant from the experimental group with
one person from the control group on potential confounding
* E.g., Age
* Propensity score
* scores on multiple variables are combined to produce a
propensity score and then participants are matched using that
number
* E.g., Age, level of education, ethnicity
Why control/comparison groups,
equivalent or non-equivalent, are so
important
Can see if these threats to
internal validity affect the
control group AND the
treatment group!!
Single Case Experimental
Designs
Single Subjects
- Made popular from operant conditioning research (B.F. Skinner)
- Now used in ABA (applied behavior analysis)
- Does an experimental manipulation have an effect on a single research participant?
- You can use this with individual therapy clients to help foster insight into their own patterns. Have them do their own single case experiments to made behavioral changes (stop a bad habit)
Single Case Experimental
Designs
*Baseline (A):
*Observed behavior before manipulation
*Treatment (B):
*Introduce manipulation/treatment
*Many measurement timepoints
*Change in behavior from A -> B presumes a
treatment effect
Reversal (or Withdrawal) Design
Baseline (A)
Treatment (B)
Baseline (A)
*Minimizes
alternative
explanations when
we see direct change
in behavior
Additional Reversal Designs
ABAB
Multiple baseline design
Observe change under multiple circumstances
*Introduce manipulation at different points of time
*Determine if manipulation is the cause of change
*Use when:
*Change might be long-lasting, even with withdrawal of treatment (B)
*Unethical to reverse treatment
Multiple baseline designs- across subjects
the behavior of several
subjects is measured over
time. For each subject, the
manipulation/intervention
is introduced as different
points in time
Multiple baseline designs- across behaviors for a single subject
ncrease ADLs by
introducing one by one
Multiple baseline designs- Across Situations
Same bx, same subject,
different settings
* Home, work, school
Take away: A manipulation is always introduced at a different time, with the expectation
that a change in behavior in each situation will occur only after the manipulation
Developmental Designs
Studies the ways individuals change as a function of
age
– Cross-Sectional method:
Persons of different ages are measured at the
same point in time
*Easier and more convenient way to collect
developmental data
*Problem: generational differences
– Longitudinal method
Same group of people are observed at different times as they age
Cohort Effect
Effect of group of people born at the same
time, exposed to the same events, and
influenced by the same demographic trends
*i.e., generational effect
* Economic and political conditions
* Music and arts
* Educational systems, and child-rearing practices
*Differences in cross-sectional study may arise
due to cohort effects
Longitudinal method
Longitudinal method
*Expensive
*Takes longer duration
*Can attribute change to
development
*Additional variables can
be assessed at a later
time
*Cohort effect vs. history
Cross-Sectional method
Relatively cheap
*Comparisons can be
obtained quickly
*Inferring differences to
developmental change is
challenging
*One time measurement
*Cohort effects
Developmental Designs- Sequential method:
*Combination of longitudinal and cross-sectional
methods
*Ex: total time of actual observation is 10 years,
not 20, but age is 55-75 years old
Quantitative Approaches in
Observational Designs
Focuses on specific behaviors that can be
easily quantified
*Uses larger samples
*Assigns numerical values to responses
*Conclusions are based upon statistical
analysis of data
To compare: Qualitative Approaches
in Observational Designs
Focuses on behavior in natural settings
*Small groups and limited setting
*Data are non-numerical and expressed in
language and/or images
*Conclusions based on interpretations drawn
by the investigator
Naturalistic Observation- natural setting
*Goals
- Describe setting, events, and persons
- Accurate description, objective interpretation, no prior hypotheses
Aggression outside of bars in a cityHandwashing in public restrooms
Naturalistic Observation- data
- detailed notes
*Maybe video or audio recordings
*Might interview “informants” for additional
observations
*Or might be already occurring documents
*Primarily qualitative
*Generate hypotheses to help explain the
data
*May supplement with quantitative (like
demographics)
Naturalistic
Observation- issues
Does researcher become active participant or observe from outside?
* Participation –open and part of group* Able to remain objective?
* Concealment – going under cover* Reduces reactivity (Hawthorne effect)…
* Ex. Hiding under bed: “Egocentricity” in Adult Conversation
ethics and expectation of privacy
Naturalistic Observation
*Limitations
Cannot be used to study all issues
*Best for investigating complex social settings
*Understand settings
*Develop theories based on observations
*Less useful when studying well-defined hypotheses under precisely specific conditions
*Hard to do!
*1300 nights, 118 bars, 74 male-female paired observers
Systematic Observation
Careful observation of specific behaviors in a particular setting
*Each behavior needs to have a good operational definition
*Usually prior hypotheses
*Coding systems
- Video coding
- Tallies
- Time samples
Systematic Observation- isses, considerations
Methodological issues
*Equipment
*Paper and pencil or audio/video
*Reactivity
*Participants behave differently because they are being observed
*Reliability
*2+ raters of same behavior
SamplingDepends on what you are observing…
*In general, time samples spread throughout day are more accurate than 1 longer
Observation: Case Studies
*Provides a detailed description of an individual
*Valuable in informing us of conditions that are rare, unusual, or noteworthy
*New disorders/medical conditions
usually start with a case study
*Phineous Gage
*Ken Horne: 1980 San Francisco resident is reported to the CDC with Kaposi’s sarcoma
Observational:
Archival Research
*Involves using previously compiled
information to answer research questions
* Public records
* Medical records
* Survey archives from other studies
* Written and mass communication records
* Social media?
IRB considerations
