Chapter 9 Flashcards
What is the most rigorous form of scientific investigation for testing hypotheses
Experiment
What is the purpose of an experiment
to support a cause and effect relationship between a particular action or condition (independent variable) and and observed response (dependent variable)
What is an extraneous variable
any factor that is not directly related to the purpose of the study, but that may affect the dependent variable
What is it called when extraneous variables are not controlled for and contaminate the independent variable in such a way that their separate effects are obscured
Confounding influence
What are the three essential characteristics a study must have in order to be considered a true experiment
- The independent variable must be manipulated by the experimenter
- The subjects must be randomly assigned to groups
- A control or comparison group must be incorporated within the design
What are the two types of independent variables
Active and Attribute
What is an active independent variable
It is one that is manipulated by the experimenter so that the subjects are assigned to levels of the independent variable
What is an attribute independent variable
It is a variable that the researcher is not able to assign; Must observe within natural groupings according to inherent characteristics. ie: Age, gender, occupation
What does random assignment mean
each subject has an equal chance of being assigned to any group
What is the most effective design strategy for ruling out extraneous effects
The use of a control group against which the experimental group is compared
When is it justifiable to design studies with comparative treatments as controls
When previous research has clearly established their effectiveness against a true control group, or when it is considered unreasonable to leave patients untreated
What type of analysis eliminates subjects who did not get or complete their assigned treatment, and include only those subjects who sufficiently complied with the trials protocol
On- Protocol or On-treatment analysis
How might bias effect the results of an on-protocol analysis
It will favor the treatment, as those who succeed at treatment are the most likely to stick with it
What is Intention to Treat Analysis
When data are analyzed according to the original random assignments, regardless of the treatment subjects actually received. Analyze the data in the way we INTENDED to treat the subjects
Do most researchers use On-Protocol or Intention to Treat Analysis
Most researchers analyze data using both just to be safe
What is the Noncompleter equals failure approach for handling missing data
When the outcome is dichotomous, dropping out is considered failure. Patients who refused their assigned intervention, who received additional, or who were missing final measurements are also considered failures.
What is the most conservative approach for handling missing data
Noncompleter equals failure
What is the Last Observation Carried Forward approach for handling missing data
the subjects last data point before dropping out is carried forward and used as the outcome score
What is a double blind study
Neither the subjects nor the investigator are aware of the identity of the treatment groups until after data are collected
What kind of bias does blinding guard against
Observation bias
When would a single blind study be appropriate
When it is not possible or feasible to blind the subjects from which group they are a part of; only the measurement team is blinded
What is the simplest way to control for specific subject traits that may interfere with the dependent variable
Choosing subjects who are homogeneous
What is one of the major drawbacks of homogeneous subjects in an experiment
Research findings can be generalized only to the type of subjects who participated in the study
To control for extraneous effects, you can systematically manipulate attribute variables and build them into the experimental design as an independent variable. When this is done, what is that new independent variable referred to as
Blocking variable
What would you be controlling for if you use a matching procedure to guarantee an equivalent group of males and females within different age ranges in the experimental and control groups
Extraneous variables
When the levels of the independent variable are assigned to different groups, the variable is considered what
an independent factor
When all levels of the independent variable are experienced by all subjects, the independent variable is considered what
a repeated factor or repeated measure
What type of design ensures the highest possible degree of equivalence across treatment conditions because the subjects are matched with themselves
a repeated measures designs
What statistical technique is used to equate groups on extraneous variables; making covariates artificially equivalent across groups
Analysis of Covariance (ANCOVA)
What is a covariate
an identified extraneous variable; can be controlled for using ANCOVA
What are the 5 design strategies for controlling intersubject differences
- Selection of homogeneous subjects
- Blocking
- Matching
- Using subjects as their own controls
- Analysis of covariance
What is statistical conclusion validity
concerns the appropriate use of statistical procedures; if inappropriate, may lead to invalid conclusions about the relationship between independent and dependent variables
What are the threats to statistical conclusion design validity
- Low Statistical Power
- Violated assumptions of statistical tests
- Error rate
- Reliability
- Variance
What is internal validity
focuses on the cause and effect relationship; Given a statistical relationship between the independent variable and dependent variable, is there evidence that one causes the other
What three components are required for the assumption of causality
- Temporal Precedence
- Covariation of cause and effect
- No plausible alternative explanations
What types of threats to internal validity exist
- Single group threats
- Multiple group threats
- Social threats
What are the six types of single group threats to internal validity
- History
- Maturation
- Attrition
- Testing
- Instrumentation
- Regression toward the mean
What are the four types of social threats to internal validity
- Diffusion or imitation of treatment
- Compensatory equalization of treatment
- Compensatory Rivalry
- Resentful demoralization
What are the five threats to construct validity of causes and effects
- Operational Definitions of independent and dependent variables
- Time frame within operational definitions
- Multiple treatment interactions
- Experimental bias
- Hawthorne effect
What are the three threats to external validity
- Interaction of treatment and selection
- Interaction of treatment and setting
- Interaction of treatment and history
What is temporal precedence
documenting that the cause precedes the effect; change in outcome must be observed only after a treatment is applied
What is covariation of cause and effect
Showing that the outcome only occurs in the presence of the intervention or that the degree of outcome is related to the magnitude of the intervention
What is meant when we say there is no plausible alternative explanations to a result in an experiment
Helps us explain a cause and effect relationship and increase internal validity. States that there are no competing explanations for the result, such as confounding variables which may have caused the result.
What is meant by single group threats to internal validity
threats that may affect the relationship between the independent and dependent variables when only one group of subjects is tested
Explain the single group threat: History
refers to the confounding effect of specific events, other than the experimental treatment, that occur after the introduction of the independent variable or between a pretest and posttest. ie: if studying exercise plan on knee extensor strength, subjects participation in additional exercise would be a history threat
Explain the single group threat: Maturation
concerns processes that occur simply as a function of time and are independent of external events. May cause subjects to respond differently on a second measurement because they have grown older, stronger, healthier, more experienced, etc since the first measurement
Explain the single group threat: Attrition
When a study drops out of a study before it is completed. Concerning when it results in a differential loss of subjects, dropouts that occur for specific reasons related to the experimental situation
Explain the single group threat: Testing
concerns the potential effect of pretesting or repeated testing on the dependent variable. The mere act of collecting data could change the response that is being measured. Increased skill may occur because of familiarity with the measurements.
What are reactive measurements
Tests that have the potential to change the response they are measuring. Occur when testing process stimulates change rather than measuring passive behavior. Someone being taped might change their behavior because they know they are being watched
Explain the single group threat: Instrumentation
concerned with the reliability of the measurement. ie: Changes that occur because of calibration of measuring tool, measuring therapist becomes better at measurement over the course of the research study
Explain the single group threat: Regression toward the mean
Even in the absence of intervention, extreme scores on the pretest tend to regress toward the mean on the posttest
How are multiple group threats similar to single group threats
Explain similar threats to validity, but concern the relationship between independent and dependent variables when more than one group is being tested
Explain the multiple group threat: Selection interaction
refers to factors other than the experimental intervention that can influence posttest differences between groups. When groups are not comparable, the single group threats to internal validity may affect the groups differently
Explain the multiple group threat: Selection- History Effects
result when experimental groups have different experiences between pretest and posttest, especially important in multicenter studies
Explain the multiple group threat: Selection - Maturation
Occur when the groups experience maturational changes at different rates, ie: if studying motor development in kids ages 3 and 10, we can expect a different rate of change that could confound treatment affects
Explain the multiple group threat: Selection- Testing
Occurs when the pretest affects each group differently
Explain the multiple group threat: Selection - Instrumentation
occurs when the test is not consistent across groups, often due to variances in reliability
Explain the multiple group threat: Selection- regression
of concern if the groups are specifically divided based on higher and lower pretest scores
Explain what is meant by a social threat to internal validity
refer to the pressures that can occur in research situations that may lead to differences between groups
Explain the social threat: Diffusion or imitation of treatments
Because the nature of many interventions makes blinding impractical, control subjects are often aware of the interventions intended for another group, and may attempt to change their behaviors accordingly
Explain the social threat: Compensatory equalization of treatments
When an experimental treatment is considered a desirable service or condition, those who work with and care for the subjects might try to even out experiences by providing compensatory services to the control group
Explain the social threat: Compensatory Rivalry
When one groups assigned treatment is perceived as more desirable than the others, subjects receiving the less desirable treatment may try to compensate by working extra hard to achieve similar results
Explain the social threat: Resentful Demoralization
When one groups assigned treatment is perceived as more desirable than the others, subjects receiving less desirable treatment may be demoralized and resentful, Their reaction may be to respond at lower levels of performance
What are Constructs
abstract behaviors that cannot be directly observed, but that can be inferred from other relevant observable variables
What is Construct validity of causes and effects
Concerns the theoretical concepts of the intervention and response variables, and whether these have been developed sufficiently
Explain how the operational definition related to an experiment can affect construct validity
When studies incorporate only one type of measurement or only one age group is measured, we can generalize outcomes only to events similar to measurements or age groups tested
Describe how construct validity might be affected by multiple treatment interaction
If multiple treatments are given, it is difficult to generalize effects of treatment. Limited by the possibility of carryover or combined effects; cannot generalize to situation where only one treatment is given
Describe how construct validity might be affected by order effects
can result when treatments or measurements are consistently given in the same order, creating possible influences on subsequent responses; cannot generalize where only one treatment is given or given in different order
Describe how construct validity may be affected by length of follow up
If a study addresses the response of subjects over time, cannot generalize result outside of that time frame. ie: basing long term decisions on short term data
Describe how experimental bias can affect construct validity
Concern biases that are introduced into a study by expectations either of the subject or the experimenter. Subject might try their best/ try harder because they want to meet the expectations of the experimenter, this no longer represents natural behavior
What is the Hawthorne effect? How can it affect Construct Validity
It is the tendency of persons who are singled out for special attention to perform better merely because they are being observed. No longer natural behavior, cannot generalize
Describe how experimenter effects may affect construct validity
Experimenters may have certain expectations that influence how subjects respond, may react more positively to subjects in the experimental group or give less attention to the control group
What is external validity
refers to the extent to which the results of a study can be generalized beyond the study sample; can the results be generalized to persons, settings, and times from those in the experimental situation?
Describe how interaction of treatment and selection can affect internal validity
If subjects are sampled according to specific characteristics, those characteristics define the target population and results cannot be generalized to those who do not have these characteristics
Describe how interaction of treatment and setting can affect external validity
If we find a causal relationship between an exercise program and functional improvement in a rehabilitation hospital, can we generalize these findings to a nursing home or home care; Have to replicate these results in different settings.
Describe how interaction of treatment and history can affect external validity
Concerns the ability to generalize results to different periods of time in the past of future.
What is the CONSORT statement
Guidelines for reporting that help authors and readers determine why a study was undertaken and how it was conducted and analyzed. It is composed of a checklist of 22 items pertaining to the content of the Title, Abstract, Introduction, methods, results, and discussion sections of an article.
