Validity in Experimental Design Flashcards

1
Q

Research Validy

A
  • Extent to which the conclusion of research are belivele and useful
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

Types of research validy

A
  • internal validity
  • construct validity
  • external validity
  • statistical conclusion validity
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q
  • internal validity
A

is there evidence of a causal relationship between independent and dependent variables

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

Construct validity

A
  • to what constructs can results be generalized
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q
  • external validity
A

can the results be generalized to other persons, setting or times

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q
  • statistical conclusion validity
A
  • is there a relationship between the indep. and depend. variables
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

Internal validity: what needs to be in place (assumption of causality requires)

A
  1. temporal precedence: causes precedes effect; outcomes change must be observed only after intervention
  2. co-variation of cause and effect: outcome only occurs in presence of intervention
  3. no plausible alternative explanation: confounding variables can offer competing explanations for relationship between IV and DV
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

Threats to internal validity

A
  • events or changes that may affect relationship between IV and DV
  • controls that lessen effect of many threats to internal validity
    1. random assignment to 2 or more groups (control group)
    2. comparing group outcomes before and after intervention
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

History: threat to internal validity and strategies to minimize the effects

A
  • events unrelated to treatment of interest occur during the study (may change the DV)
  • outside the clients and study
  • strategies to minimize effects of history:
    1. planning
    2. randomly selecting control groups
    3. description of unavoidable historical events
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

Maturation: threats to internal validity

A
  • changes within participant caused by passage of time
    (independent of eternal events that may change DV)
  • maturation of children, condition can worsen or improve over time
  • strategies to minimize the effects of maturation
    1. control group - observe effects of maturation alone
    and compare improvement in experimental group with control group to observe the treatment effects
    2. multiple baseline measures of participants before implementing treatment (results after intervention interpreted in light of baseline)
  • look in light of the baseline
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

Subject attrition: threats to internal validity

A
  • participants lost from different study groups at different rates or for different reasons
  • plan: make adherence to experimental routine as easy as possible
  • collect data about lost participants and reasons for loss of participants
  • should address differential group attrition in statistical analysis
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

Testing: Threats to internal validity

A
  • repeated testing could lead to changes in DV
  • strategies to minimize effects of repeating testing
    1. randomly selected groups
    2. conduct familiarize sessions to minimize learning during study
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

Instrumentation: threats to internal validity

A
  • changes in measuring tools are responsible to observed change in DV
  • calibration
  • human instruments: observation clinical measures
    1. identification of gait deviations, functional levels, abnormal muscle tone
    2. training increases reliability
    3. document reliability
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

Statistical regression to the mean: threats to internal validity

A
  • extreme scores become less extreme over time
  • related to degree of measurement error in DV
  • if someone gets an extreme score, overtime those scores tend to be less extreme - gets closer to the average
  • minimize effect: control group - outliers affect all groups consistently
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

Assignment: threats to internal validity

A

groups are different from one another at baseline on DV of interest
- usually a threat to validity when:
1. participants not randomly assigned to groups
2. non experimental: group assignment not manipulated by investigator
- designs that do no randomly assign subjects to groups are quasi-experimental
- strategies:
1. randomly assign subjects to groups
2. when not possible, use statistical methods to equalize groups

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

Diffusion or imitation of treatments: threats to internal validity

A
  • subjects in control group aware of intervention intended for another group (may try to change behaviors)
  • control strategy: minimize contact between participants in different groups
  • instruct participants in importance of adhering program they are assigned
17
Q

Compensatory equalization of treatments: threats to internal validity

A
  • researcher with preconceived notions about which treatment is more effective gives excess attention to subjects receiving other treatment
  • when a researcher has a preconceived idea about a procedure
  • Control strategies:
    1. avoid topics about which researcher is biased
    2. design studies with researcher masking
    3. use independent clinician to record provision of treatment
18
Q

Compensatory rivalry or resentful demoralization: threats to internal validity

A
  • members of one group react to perception that they are receiving a less desirable treatment than other groups
  • see either:
    1. compensatory rivalry: one groups work hard to “show up” other group or change behavior
    2. resentful demoralization: why should I bother anyway
  • control strategies:
    1. control information given to participants
    2. blinding to group membership
    3. have positive attitude toward all groups
19
Q

Construct validity

A
  • concerned with how variables are conceptualized: related to how variables are operationally defined and potential subject or experimental bias
  • concerns meaning attached to concepts within study
20
Q
  • factors that affect construct validity:
A
  1. operational definitions: results of study can only be interpreted within contexts of the operational definitions used
  2. comprehensive measurements: full analysis of most constructs should use multiple forms of measurement
  3. subgroup differences: when some levels of IV interact differently with DV
  4. time frame: duration of intervention, intervention onset relative to pt. diagnosis, and follow up duration
  5. multiple treatment interaction: when treatments are combine cannot generalize findings to treatments
  6. experimental bias: introduced by subject or experimenter expectations
  7. subjects try to: fulfill researchers expectations, present themselves in best way possible
    - responses do not represent natural behavior
    - strategy: subject blinding, researcher blinding (to group)
21
Q

External validity

A
  • concerns whom in what settings, and at what times research results can be generalized
  • influenced by design of study and research consumer who hopes to use the study findings
  • design: thoughtfully consider population to whom the results of the study can be applied
  • concerns:
    1. selection and sampling for the study
    2. setting
    3. ecological validity (ex: insurance coverage- can it be carried out in real world)
    4. influence of history: difficult to generate results to different times