Module 5 Flashcards
What is scientific rigor?
Internal precision of the exact-ness of the research study
What is the difference between a nondirectional and a directional hypothesis?
Nondirectional hypotheses state the existence of a relationship between variables, a directional hypotheses states the existence, direction, and magnitude of a relationship between variables
The statistical hypothesis is also known as the ______ .
Null
What is a null hypothesis?
A hypothesis that states that there is no relationship between variables. The goal of a research study is to accept or reject the null
Extraneous variables are also known as what type of variable?
Confounding
What is research validity?
The approximate truth of an inference
What are the four major types of validity?
- Internal
- External
- Statistical conculsion
- Construct
How can internal validity be described?
The degree to which it can be inferred that a study’s independent (predictor) variable actually caused the dependent (outcome) variable and that it was not a result of an extraneous (confounding) variable(s)
Which validity improves our ability to infer cause-and-effect relationships and improves control of extraneous factors?
Internal validity
What are examples of threats to internal validity?(6)
- History
- Maturation
- Mortality
- Selection bias
- Testing
- Instrumentation
Which type of internal validity threat can be described as the occurrence of external events that take place concurrently with the independent variable and that can affect the outcome variable?
History
Ex- 9/11, election
Which type of threat to internal validity refers to processes occurring within participants during the course of the study as a result of the passage of time rather than as a result of the independent variable?
Maturation
Which type of threat to internal validity arises when participants drop out, pass away, etc during the course of a study?
Mortality (also known as Attrition)
What does the threat to internal validity, selection bias, refer to?
Also known as Selection. Refers to biases resulting from preexisting differences between groups. When participants are not randomized to groups there is a great risk for this
What does the threat to internal validity, testing, refer to?
Testing refers to the effects of taking a pretest on people’s performance on a posttest. The initial test may sensitize participants
What does the threat to internal validity, instrumentation, refer to?
This refers to the bias that comes from changes in measuring instruments or methods of measurement between two points of data collection
Ex-poorly calibrated thermometer
What is external validity?
Generalizability of findings to a target population, a key component of EBP.
Whether inferences about observed relationships will hold over variations in persons, setting, time, or measures of the outcomes.
What are 3 threats to external validity?
- Selection effects
- Reactive effects
- Measurement effects
Which threat to external validity can be due to researchers not having access to an adequate number of individuals who meet study criteria (access to target population)?
Selection effects
What do the reactive and measurement effects refer to as a potential threat to external validity?
Participants may react or act differently to being measured/assessed, to novel exposure to intervention or questionnaire, and therefore the data collection will be in some way flawed/skewed.
Internal and external validity share a ________ relationship. Why?
Inverse
To increase internal validity researchers exert control on study participants to reduce extraneous influences which increases threats to external validity (less representative condition or sample). If researchers relax their sample requirements to be more representative of target population that creates a threat to internal validity
What is statistical conclusion validity?
The degree to which inferences about relationships from a statistical analysis of the data are correct
What is a Type I error?
A false positive
Rejecting a null hypothesis that is, in fact, true
What is a Type II error?
A false negative
Accepting a null hypothesis that is, in fact, false
Threats to statistical conclusion validity include: (3)
- Low statistical power
- Excessive homogeneity
- Lack of treatment fidelity
When does statistical power tend to be low?
When samples sizes are small
When does the threat to statistical conclusion validity, excessive homogeneity, occur?
When study participants are too alike. This happens when great control is exerted to increase internal validity and creates a situation where the sample is not representative of the target population (dec. external validity)
What does the threat to statistical conclusion validity, lack of treatment fidelity, refer to?
Variance in the way an intervention is used or applied. This non-standardized method of intervention application threatens statistical conclusion validity
What is construct validity?
The degree to which evidence about study particulars supports inferences about the higher order constructs they are intended to represent; in measurement, the degree to which a measure truly captures the focal construct
(*how accurately what is meant to be measured is being measured)
Threats to construct validity include:(4)
- Poor construct definition
- Researcher expectancies
- Novelty effects
- Mono-method bias
What are novelty effects?
Participants and research agents altering their behavior in reaction to a new treatment
What is mono-method bias, a threat to construct validity?
Researchers assessing a key category in the research design using only one method. This decreases scientific rigor and threatens construct validity
What is the most effective method for controlling for extraneous variables and individual characteristics?
Randomization
What are 6 ways of controlling participant characteristics- characteristics that could compete with the independent variable as the cause of the outcome?
- Randomization
- Crossover
- Homogeneity
- Stratification/Blocking
- Matching
- Statistical Control
How is homogeneity used to control participant characteristics?
Used when randomization and crossover aren’t feasible. Using a sample that is homogeneous with respect to confounding variables (ex- only men used in sample)
How is stratification/blocking used to control participants characteristics?
This refers to including the confounders into the research design through stratifications. (ex-randomized block design–> if gender was the confounding variable, men and women are assigned separately to treatment groups).
How is matching used to control participants characteristics?
This refers to using information about people’s characteristics to create comparable groups (ex-matching a participant from the intervention group with one of the same age and gender in the control group)
What is an example of a method used to control participant characteristics using statistical control?
ANCOVA- does this by statistically removing the effect of confounding variables on the outcome
What is the Hawthorne effect?
The alteration of behavior by the subjects of a study due to their awareness of being observed.
