Research Design Flashcards
True experimental design
at least one IV Is manipulated and subjects are randomly assigned
Quasi-experimental design
at least one IV is manipulated, but there is non-random assignment of subjects, typically because subjects are already in pre-existing groups
Observational, passive, or non-experimental
no intervention or manipulation
Between groups design
only compares groups that are independent
Within subjects design
groups contrasted are correlated or related (e.g. matched or repeatedly measured)
Counterbalancing
Because of possible carryover effects
Latin square is most sophisticated form of counterbalancing
Mixed design
involve groups that are both independent or correlated
e.g. between and within subjects design
Single subjects designs (five types)
one or very few subjects are studied intensively and each subject is measured many times
AB, ABAB, multiple baseline, simultaneous treatment, changing criterion
Autocorrelation
effect of measuring the same person repeatedly, which results in highly correlated data
significant problem associated with any single subject design
AB Design
single subjects design baseline condition (A) followed by treatment condition (B) most significant problem is the threat of history - hard to determine whether it was intervention that caused change or some other event
ABAB Design
single subjects design
baseline (A), treatment (B), baseline (A), treatment (B)
protects against threat of history
two problems: failure of DV to return to baseline and issue of ethics regarding removing effective treatment
multiple baseline design
single subjects design
treatment applied sequentially or consecutively across subjects, situations, or behaviors
problems: more time consuming and expensive
simultaneous (alternating) treatment design
single subjects design
involves two or more interventions implemented concurrently during the treatment phase
treatments are balanced and varied across time of day
allows for comparison of the relative effectiveness of two or more interventions for an individual
changing criterion design
single subject design
attempt made to change behavior in increments to match a changing criterion
time sampling
type of behavioral measurement that is useful when a behavior is not discrete, and thus has no distinct beginning and end
two types: momentary time sampling and whole-interval sampling
momentary time sampling
observer recording whether target behavior is present or absent at the moment the time interval ends
whole-interval sample
aka interval sampling or interval recording
involves scoring target behavior positively only if it is exhibited for a full duration of the time interval
event recording
involves tallying the number of times the target behavior occurred
analogue research
evaluates treatment under conditions that only resemble or approximate clinical situations
typically study less severe problems
treatments tend to be highly standardized
e.g. participants are college students and volunteers and therapists are graduate students
problem - limited generalizability
clinical trials
outcome investigations conducted in clinical settings
cross-sequential research
aka cohort-sequential research
takes several cross sections (e.g. of age groups) and follows them over briefer periods of time (e.g. 5-10 years instead of lifespan)
simple random sampling
every member of population has an equal chance of being randomly selected
stratified random sampling
population first divided into strata (e.g. age levels, income levels, ethnic groups), and then random sample of equal size from each stratum is selected
proportional sampling
individuals randomly selected in proportion to their representation in the general population
systematic sampling
involves selecting every kth element after a random start
cluster sampling
involves identifying naturally occurring groups of subjects (clusters) and randomly selecting certain clusters (e.g. classes or departments at a university); once cluster has been selected, all the subjects within the cluster are usually surveyed
internal validity
degree of validity of statements made about whether X causes Y
threats to internal validity (8)
history, maturation, testing or test practice, instrumentation, statistical regression, selection bias, attrition or experimental mortality, diffusion
threat to internal validity - history
specific incidents that intervene between measuring points, either in or outside of the experimental situation
threat to internal validity - maturation
factors that affect the subjects’ performance because of the passing of time (fatigue, maturing)
threat to internal validity - testing or test practice
occurs when familiarity with testing affects scores on repeated testing
Solomon Four-Group Design
controls for testing threat to internal validity
divides subjects into four groups
Group 1: pre-test, intervention, post-test
Group 2: pre- and post-test
Group 3: intervention and post-test
Group 4: post-test only
threat to internal validity - instrumentation
refers to changes in observers of calibration of equipment
threat to internal validity - selection bias
caused by non-random assignment
threat to internal validity - attrition or experimental mortality
when there is a differential loss of subjects from the groups
threat to internal validity - diffusion
occurs when the no-treatment group actually gets some of the treatment
construct validity
refers to factors other than the desired specifics of the intervention that result in differences
these factors often lumped under threats to external validity
threats to construct validity (4)
attention and contact with clients
experimenter expectancies
demand characteristics
John Henry effect
threat to construct validity - experimenter expectancies
aka Rosenthal effect
refers to cues or clues transmitted to subjects by the experimenter
eg. smiling inadvertently nodding encouragingly only to those receiving treatment
threats to construct validity - demand characteristics
factors in the procedures that suggest how subject should behave
e.g. subjects given medication told it might cause side effects, those with placebo told there won’t be side effects
to reduce demand characteristics, subjects should be blind to treatment condition
threats to construct validity - John Henry Effect
aka compensatory rivalry
occurs when persons in a control group try harder than usual in spirit of competition with experimental group
JH was legendary railroad steel driver who swung his hammer in competition with steam drill and died from overexertion
to control for JOhn HEnry effect, experimental and control groups should not know each other
external validity
generalizability
threats to external validity (3)
sample characteristics
stimulus characteristics
contextual characteristics
threats to external validity- sample characteristics
refers to differences between sample and population
threats to external validity - stimulus characteristics
such as artificial research arrangements
features of the study with which the intervention is associated
threats to external validity - contextual characteristics
refer to conditions in which the intervention is imbedded
reactivity - when subjects behave in a certain way just because they are participating in research and being observed
Hawthorne effect - frequently described example of reactivity
threats to statistical conclusion validity (4)
low power
unreliability of measures
variability in procedures
subject heterogeneity
threats to statistical conclusion validity - low power
low power is the diminished ability to find significant results
contributing factors - sample size, inadequate interventions (e.g. tx only lasting two sessions)
threats to statistical conclusion validity - unreliability of measures
even if intervention is effective, significant differences may not be found if the outcome measure sured is unreliable
threats to statistical conclusion validity - variability in procedures
refers to inconsistency in treatment procedures, which can obscure significant treatment findings
especially a concern in psychotherapy outcome research
threats to statistical conclusion validity - subject heterogeneity
makes it more difficult to find significant differences between groups
Face validity
What test appears superficially to measure
Experimentwise Error Rate
Probability of making a Type I error
idiographic
term used to describe single subject approaches
nomothetic
term used to describe group approaches
normative data
data that can be compared both within and across subjects
ipsative data
results from forced choice format
can only describe relative strengths or interests wihin a subject and cannot be used for comparisons across subjects
