Research Design Flashcards
Independent Variables (IV)
The groups that are being compared in a research study
Ex-Contrasting 3 txs for depression, type of tx is one IV, gender might be another
2 Main types of IVs
Manipulated IVs-directly manipulated by the researcher
Non Manipulated IVs-cannot be manipulated b/c they are pre-existing, like gender
Dependent Variables (DV)
Outcome measures in a study
Ex- a depression measure
Can be nominal, ordinal, interval or ratio
Types of Research
True Experimental
Quasi-Experimental
Observational, Passive, or Non-experimental
True Experimental Research
At least one IV is manipulated & subjects are randomly assigned
Quasi-experimental
At least one IV is manipulated, but non-random assignment of subjects
Ex- 2 school programs tested at 2 different schools
Observational, passive, non experimental research
No intervention or manipulation, but still looking at group differences
Ex- comparing rates of smoking between males & females
Types of Group Designs in Research
Between Groups
Within Subjects
Mixed Design
Between Groups Design
Compares groups that are independent
Ex- comparing reading levels of 2 different 2nd grade classrooms
Within Subjects Design-3 types
Groups contrasted are related
- Repeated measures
- subjects matched prior to assignment to groups
- subjects have an inherent relationship (twins)
With repeated measures design, important to account for ____ effects by use of _____
carryover; counterbalancing
Latin Square
most sophisticated method for counterbalancing
Mixed Design Groups
Involves groups that are both independent & correlated
Ex- patients assigned to 2 different treatment groups (independent) but also complete before & after measures (repeated measures, correlated)
Idiographic
Single subject approach
Nomothetic
Group approach
Autocorrelation
Effect of measuring the same person repeatedly, resulting in highly correlated data
Main problem of any single subject design
AB Design
Single Subject Design
Baseline condition (A) followed by treatment cond (B)
Ex: Weight measured at baseline then during/after prescribed diet
Major problem w/AB design
History- is the change due to the treatment being researched or some other event
ABAB Design
Single Subject Design
Baseline (A), Treatment (B), baseline (A), treatment (B)
Protects against effects of history
Major problems w/ABAB Design
- DV (which is measured daily) is unlikely to return to baseline after 1st round of tx
- Ethical issue of removing an effective tx
Types of Single Subject Designs
AB ABAB Multiple Baseline Simultaneous Tx Design Changing Criterion Design
Multiple Baseline Design-3 Types
Across Subjects
Across Situations
Across Behaviors
Ex of Multiple Baseline Across Subjects Design
3 children w/ADHD complete baseline measures daily for a week
1 child then given ADHD tx and measured daily for one week
Other 2 children measured daily without tx for one week
(other 2 children gradually phased into tx)
Ex of Multiple Baseline Across Situations
Behavioral intervention for tantrums at home, at school, and in the park
Ex of Multiple Baseline Across Behaviors
Intervention for 3 different bxs of one subject, intervention applied consecutively to the diff bx
Ex- autistic child target head banging, then self biting, then rocking
Simultaneous (Alternating) Tx Design
2 or more interventions implemented concurrently thru research phase, but balanced & varied across time of day
Ex- compare effectiveness of M&Ms vs praise to target head banging- alternate during day and balanced across days
Changing Criterion Design
Change bx in increments to match a changing criterion
Ex- reduce coffee consumption over time by setting gradual goals
Methods for Behavioral Measurement
Time Sampling
Event Recording
Time Sampling
Used when the bx being measured is not discrete (no distinct beginning & end)
Breaks a period of interest into smaller chunks for measurement
2 types of Time Sampling
Momentary-record if target bx is present or absent at end of time interval
Whole Interval- record target bx positively only if it is present the whole duration of the time interval
Event Recording
Tally the number of times a target bx occurs
Useful when target bx is discrete and occurs relatively infrequently
Ex-counting number of times a student is late to class in a semester
Analogue Research
Evaluates tx under conditions that resemble or approximate clinical situations
Ex- college students volunteer as participants for a highly standardized group therapy treatment
Limited generalizability of findings due to tight experimental control
Clinical Trials
Outcome investigations in real world clinical settings
Cross Sectional Research
Looks at diffs across sections, or ages, by sampling age categories of interest at one point in time.
Ex: study internet usage at diff age brackets (20 year old, 40 year olds, 60 year olds)
Cohort effects
Homogeneity of particular age range or experience
Ex- 20 yr olds may use internet more often than 60 yr olds just because they have grown up with computers
*common problem w/Cross Sectional Research
Longitudinal Research
Follow group of subjects over many years
Cross Sequential Research
Takes several cross sections, or age groups, and follows them over briefer intervals of time than longitudinal research
Ex- study health habits of different age brackets over 5 year time frame
Most important issue in subject selection is ___ ____
representative sampling
Major Sampling Strategies
Simple Random Stratified Random Proportional Sampling Systematic Cluster
Simple Random Sampling
Every member of population has equal chance of random selection
Ex- random selection of all registered voters in a state
stratified Random Sampling
Population divided into strata (age, income, ethnicity) and random sample within each stratum
Proportional Sampling
Random selection in proportion to representation in the general population
Ex- study mirrors racial demographics of the US
Systematic Sampling
Ex- select every 10th person out of 1000 to achieve 100 sample size
Cluster Sampling
Selection of naturally occurring groups of subjects
Ex-Randomly select 10 schools in a district, then test achievement of all 8th graders within those schools
Internal Validity is threatened when:
factors other than the IV may have caused the change in the DV
Common Threats to Internal Validity
History Maturation Testing or Test Practice Instrumentation Statistical Regression Selection Bias Attrition or Experimental Mortality Diffusion
History
Specific incidents that intervene between measuring points, making it difficult to know if change is due to intervention or the incident
Best control for this is a control group
Maturation
Factors affecting subjects’ performance simply due to passing of time, making it difficult to know if change is due to intervention or the natural maturation
Best control for this is a control group
Testing or Test Practice
Occurs w/repeated testing, where cannot determine if change is due to prior experience w/the test vs. the intervention
Best control for this is Solomon Four Group design
Solomon 4 Group Design
Subjects divided into 4 groups
- Pre test, intervention, post test
- Pre test, post test (no intervention)
- Intervention, post test
- Post test, no intervention
Instrumentation
Changes in observers or calibration equipment over time, threatens internal validity
Control group controls for this
Statistical Regression/ Regression to the Mean
Extreme scores naturally become less extreme (closer to the mean) on retesting without any intervention
Control group helps control for this
Selection Bias
Non random assignment of subjects
Makes it difficult to determine if effects of a treatment are related to the treatment or the type of subjects who volunteer to participate
Attrition/Experimental Mortality
Differential loss of subjects from groups
Ex- less depressed ind drop out of control group, more depressed drop out of treatment group
Change in scores may be due to pattern of attrition rather than intervention itself
Assess for attrition problems by comparing on variables through t tests
Diffusion
No treatment group actually gets some of the treatment, clouding treatment effects
Ex- CBT therapy compared to non specific therapy, cognitive strategies may be discussed in non specific tx
Threats to Construct validity refers to:
factors other than the desired specifics of our intervention that result in differences
Ex- CBT (IV) shown to decrease level of depression (DV), still difficult to say was if the change in automatic thoughts vs other strategies (behavioral activation) contributing to the change
Threats to Construct validity include:
Attention & Contact w/clients
Experimenter Expectancies
Demand Characteristics
John Henry Effect
Rosenthal Effect
Experimenter expectancies impact the research outcomes and are communicated through cues from the experimenter to the subject, thus impacting their responses
Ex-therapist nods while subjects on experimental medication describe reduction in sxs, resulting in subjects reporting more improvement
Helps for experimenters to be blind to tx conditions
Demand Characteristics
Factors in study design that suggest how the subject should behave
Ex- subjects told about possible side effects will report more side effects
Helps for subjects to be blind to tx condition
John Henry Effect
Compensatory Rivalry-JH was steel driver who drove himself to death to outperform a steam drill
Ppl in control group try harder than usual in spirit of competition with experimental group
Experimental & control groups should not know about each other
External Validity
Generalizability of results to the larger population
Internal Validity
When group differences are found, ability to conclude that change in outcome (DV) is the result of intervention (IV)
Threats to External Validity
Sample Characteristics-diffs between sample & population
Stimulus Characteristics-features of study with which intervention is associated (ex- testing memory in lab rather than natural setting)
Contextual Characteristics-level of reactivity (extent to which subjects behave in a certain way b/c they are being observed (Hawthorne Effect)
Threats to Statistical Validity
Low Power-diminished ability to find sig results (small sample size, inadequate interventions)
Unreliability of Measures
Variability in Procedures-especially of concern in psychotherapy outcome research
Subject Heterogeneity
The greater the internal validity, or experimental control, the lower the ____ validity, or ability to ___findings
external; generalize
Ipsative data
results from a forced-choice format. It can only describe relative strengths or interests within a subject and cannot be used for comparisons across subjects
Solomon 4 group design
true experimental design to evaluate effects of pre-testing when some groups are pretested and others aren’t
