Research Methods Flashcards
Scales of Measurement
Nominal or categorical-identification
Ordinal-ordered
Interval-ordered with equal intervals
Ratio-equal intervals and absolute zero
Single Subject Design
Inferences are made by comparing effects of different conditions on the same person
Key characteristics: ongoing assessment, baseline assessment, stability
Types: AB Design, ABAB/reversal designs, multiple baseline, changing criterion
Advantages: Do not require many participants or assigning participants to groups/cheaper and easier to implement; allow for small scale application of an intervention to refine the intervention before large-scale implementation; receive ongoing feedback about an intervention; shows potent effects
Disadvantages: generalizability; ethics of removing the treatment; repeated measurement of effects; no inferential stats, visual inspection
AB Design
One baseline phase and one intervention phase
Disadvantages: cannot have certainty that the intervention caused behavior change; internal validity threats
ABAB/Reversal Designs
Alternating baseline and intervention phases, removal of intervention at second and subsequent baselines
Advantages: Can provide more support of the intervention working if performance improves during intervention phase, reverts to baseline conditions when intervention is removed and improves again
Disadvantages: ethical concerns with removing intervention; cannot implement in applied settings
Multiple Baseline Designs
Introduce intervention to different baselines (people, behaviors, settings) at different points in time
No set number of baselines
Advantages: don’t need to remove intervention; internal threats to validity controlled (confident that intervention produced behavior change)
Disadvantages: shouldn’t be used with interventions that would produce changes in multiple domains without being implemented in those domains
Changing Criterion Design
Baseline phase, then intervention phase
During intervention phase, there are multiple sub-phases, criterion is set for performance and once the criterion is met, it changes to a slightly more stringent criterion
Advantages: well suited for terminal responses that are arrived at or approximated gradually; good for applied settings; strengthened by making bidirectional changes
Disadvantages: not good for behavior that changes rapidly-change could be due to other factors; less persuasive to making threats to validity implausible
Group Designs
Having more than one group included in the study
Includes Pretest-Posttest Control Group Design, Posttest Only Control Group Design, Soloman Four Group Design, Factorial Designs, Multiple Treatment Designs
Pretest-Posttest Control Group Design
One or more groups receives experimental manipulation and other group receives no manipulation
Collect data prior to intervention/manipulation and after intervention/manipulation
Experimental design –> random assignment
Advantages: controls threats to internal validity; use of pretest: increases statistical power, allows for matching of subjects, allows for evaluation of variables across time, can evaluate attrition
Disadvantages: possibility of attrition/differential attrition; pretest sensitization, ethics of not giving an intervention
Posttest Only Control Group Design
Minimum of two groups, no pretest is given
Experimental design
Advantages: controls threats to internal validity; no pretest sensitization; more participants may be available; reduces cost of assessment materials
Disadvantages: can’t assess if there were group differences prior to intervention; threats to internal validity (history, maturation)
Soloman Four Group Design
Experimental design
Four groups required (1: Assessment-Intervention-Assessment; 2: Assessment-Assessment; 3: Intervention-Assessment; 4: Assessment)
Advantages: Controls for threats to internal validity; evaluate effects of testing; replication of intervention
Disadvantages: requires lot of participants; high cost/resource intensive
Factorial Designs
Experimental design
Simultaneous investigation of multiple IVs
Advantages: assess effects of separate variables in the same experiment as well as combined effect; fewer subjects than in multiple experiments
Disadvantages: number of groups quickly grows as more variables are added, number of groups may be prohibitive in terms of costs and participants needed; can be difficult to describe complex relationships
Multiple Treatment Designs
Each intervention is delivered to each participant at different time (groups are based on the sequence that participants receive the intervention)
Crossover design-at some point during the study, all participants switch to the other condition
Multiple treatment counterbalanced design-each intervention is delivered in each possible placement (e.g., first, second, third) to a different group
Advantages: examine differential effects of treatment; alleviates ethical concerns with removing or not giving treatment to participants
Disadvantages: need to consider order and carryover effects; cannot be used with conflicting treatments; consider ceiling and floor effects
Effectiveness Designs
Evaluates whether an intervention works under typical clinical conditions
Considerations: assessing fidelity; less study team involvement
Advantages: larger, more representative samples; higher generalizability
Disadvantages: lack of randomization–> clients can choose treatment; competency of clinicians is not certain; may be multiple treatments happening (one treatment carryovers into other groups); cost effectiveness
Efficacy Designs
Evaluates if the intervention works in a controlled research context
Randomized controlled trials are gold standard
Advantages: random assignment; strict treatment protocol and specific inclusion/exclusion criteria; high internal validity; fully powered
Disadvantages: recruiting enough participants; differential attrition, follow-up assessments difficult to interpret; fixed number of sessions/dosing considerations; investigator allegiance
Hybrid Efficacy-Effectiveness Designs
Includes features of both efficacy and effectiveness trials
Effectiveness-Implementation Hybrid Designs
Type 1: test clinical intervention while gathering information on its delivery during an effectiveness trial and/or its potential for implementation in a real-world situation
Type 2: simultaneously testing a clinical intervention and an implementation intervention strategy
Type 3: testing an implementation strategy while observing/gathering data on the clinical interventions and outcomes
Feasibility Trials
Assesses intervention’s suitability for everyday use
Useful to support an efficacy trial
Can be done by tracking recruitment, retention/drop-out, engagement with intervention, satisfaction/acceptability
Early in the process
Pilot Designs
Preliminary test to evaluate aspects of the procedure to see if it works, is feasible, and has the effect (e.g., manipulation check)
Advantages: can receive feedback from participants to strengthen procedures; can problem solve before full experiment
Longitudinal Designs
Collect data on multiple occasions starting with the present and going into the future for comparisons
Advantages: change over time; controls for history effects; demonstrates development; more powerful than cross-sectional
Disadvantages: resource intensive-cost, time, large sample size; attrition; cohort effects not examined/selection effects; maturation; testing effects
Cross-Sectional Designs
Collect data at one point in time; make comparisons across different types of respondents or participants
Advantages: no attrition; cost efficient; can include more variables; useful for generating theory
Disadvantages: cannot determine cause and effect; snapshot timing (may not be representative); cohort effects; history effects
Cross-Sequential Designs
Combination of cross-sectional and longitudinal designs
Advantages: cohort effects can be examined; examine change over time for different groups; reduces internal validity concerns
Disadvantages: resource intensive; attrition
Naturalistic Designs
Done in real-world settings
Advantages: lowers threats to external validity; good for behaviors that are low frequency
Disadvantages: threats to internal validity; more difficult to design; ethical concerns with confidentiality; cannot determine cause and effect
Laboratory Designs
Conducted in a lab setting with high control
Advantages: high control = threats to internal validity reduced; control for confounding variables
Disadvantages: less ecological validity; difficult for low frequency behaviors
Quasi-Experimental Designs
Cannot randomly assign participants
Includes: Non-randomized control group pretest-posttest/posttest only; One group pretest/posttest design; Interrupted time series design; equivalent time series design; nonequivalent group designs/non-randomized control group design
Advantages: more feasible; ecological validity
Disadvantages: confounding variables; internal threats to validity
Non-Randomized Control Group Pretest-Posttest/Posttest Only
At least two groups, not randomly assigned
Collect either pretest and posttest data or just posttest data
Advantages: can test subjects that are naturally embedded in a particular group
Disadvantages: may be differences in groups pre intervention
One Group Pretest/Posttest Design
Measure DV -> Intervention -> Measure DV
No control/comparison group
Disadvantages: threats to internal validity; pretest sensitization
Nonequivalent Group Designs/Nonrandomized Control Group Design
Participants naturally embedded in larger groups
No random assignment
Involves comparison group
Can be pretest-posttest or posttest only
Interrupted Time Series Design
Multiple measures pretest and posttest
Can have control/comparison group which helps control internal validity threats
Advantages: reduce maturation effects and regression to the mean; assess persistence of change; does not require access to large number of participants
Disadvantages: cannot rule out history or selection effects
Equivalent Time Series Design
Used with outcomes that are expected to return to baseline without intervention
Repeatedly readminister and take away the intervention
Advantages: useful when measuring effects of treatment over a long period of time; does not require access to large number of participants
Disadvantages: ability to remove treatment/residual effects; practice effects, reactivity, sensitization
