Validity in Experimental Design Flashcards
Research Validy
- Extent to which the conclusion of research are belivele and useful
Types of research validy
- internal validity
- construct validity
- external validity
- statistical conclusion validity
- internal validity
is there evidence of a causal relationship between independent and dependent variables
Construct validity
- to what constructs can results be generalized
- external validity
can the results be generalized to other persons, setting or times
- statistical conclusion validity
- is there a relationship between the indep. and depend. variables
Internal validity: what needs to be in place (assumption of causality requires)
- temporal precedence: causes precedes effect; outcomes change must be observed only after intervention
- co-variation of cause and effect: outcome only occurs in presence of intervention
- no plausible alternative explanation: confounding variables can offer competing explanations for relationship between IV and DV
Threats to internal validity
- events or changes that may affect relationship between IV and DV
- controls that lessen effect of many threats to internal validity
1. random assignment to 2 or more groups (control group)
2. comparing group outcomes before and after intervention
History: threat to internal validity and strategies to minimize the effects
- events unrelated to treatment of interest occur during the study (may change the DV)
- outside the clients and study
- strategies to minimize effects of history:
1. planning
2. randomly selecting control groups
3. description of unavoidable historical events
Maturation: threats to internal validity
- changes within participant caused by passage of time
(independent of eternal events that may change DV) - maturation of children, condition can worsen or improve over time
- strategies to minimize the effects of maturation
1. control group - observe effects of maturation alone
and compare improvement in experimental group with control group to observe the treatment effects
2. multiple baseline measures of participants before implementing treatment (results after intervention interpreted in light of baseline) - look in light of the baseline
Subject attrition: threats to internal validity
- participants lost from different study groups at different rates or for different reasons
- plan: make adherence to experimental routine as easy as possible
- collect data about lost participants and reasons for loss of participants
- should address differential group attrition in statistical analysis
Testing: Threats to internal validity
- repeated testing could lead to changes in DV
- strategies to minimize effects of repeating testing
1. randomly selected groups
2. conduct familiarize sessions to minimize learning during study
Instrumentation: threats to internal validity
- changes in measuring tools are responsible to observed change in DV
- calibration
- human instruments: observation clinical measures
1. identification of gait deviations, functional levels, abnormal muscle tone
2. training increases reliability
3. document reliability
Statistical regression to the mean: threats to internal validity
- extreme scores become less extreme over time
- related to degree of measurement error in DV
- if someone gets an extreme score, overtime those scores tend to be less extreme - gets closer to the average
- minimize effect: control group - outliers affect all groups consistently
Assignment: threats to internal validity
groups are different from one another at baseline on DV of interest
- usually a threat to validity when:
1. participants not randomly assigned to groups
2. non experimental: group assignment not manipulated by investigator
- designs that do no randomly assign subjects to groups are quasi-experimental
- strategies:
1. randomly assign subjects to groups
2. when not possible, use statistical methods to equalize groups