Internal/External Validity

Question 1

list the 5 subcategories of external validity

Answer 1

1) population validity (generalizability of a study’s results to other people in the pop from which subjects were drawn)

2) ecological validity: generalizability of results to other settings/environments, esp. real-world settings (e.g., clinical practice)

3) temporal validity: generalizability of results across time

4) treatment validity: generalizability of results to variations of the treatment (independent variable)

5) outcome validity: generalizability of results to different but related dependent variables

Question 2

list 7 threats to internal validity

Answer 2

1) history
2) maturation
3) differential selection
4) statistical regression
5) testing
6) instrumentation
7) differential attribution

Question 3

best way to control history threats to internal validity when its due to events that occur outside of the study’s context

Answer 3

include more than 1 group & randomly assign participants to the different groups (participants in all groups should be affected by history to the same extent)

Question 4

this threat to internal validity refers to the physical, cognitive, & emotional changes that occur within subjects during the course of the study and are due to the passage of time & affect the study’s results

Answer 4

maturation

Question 4

best way to control maturation threats to internal validity

Answer 4

include more than 1 group in the study & randomly assign participants to the different groups (participants in all groups should experience similar maturational effects & any difference between groups at the end of the study will not be due to maturation)

Question 5

this type of threat to internal validity occurs when groups differ at the beginning of the study due to the way they were assigned to groups & this difference affects the study’s results

Answer 5

differential selection

**Differential selection is a misnomer because it actually refers to differential assignment of subjects to treatment groups. **

Question 6

best way to control differential selection threats to internal validity

Answer 6

randomly assign participants to groups so the groups are similar at the start of the study

Question 7

best way to control statistical regression threat to internal validity

Answer 7

not including extreme scorers or have more than 1 group & ensure the groups are equivalent in terms of extreme scoreres at beginning of study

Question 7

this type of threat to internal validity refers to participants selected for inclusion in the study because of their extreme scores on a pretest and occurs because many characteristics are not entirely stable over time & many measuring instruments are not perfectly reliable

Answer 7

statistical regression threat

Question 8

this type of threat to internal validity occurs when taking a pretest and affects how participants respond to the posttest

Answer 8

testing

Question 9

best way to control testing threats to internal validity

Answer 9

not administering a pretest or using the Solomon four-group design

**The Solomon four-group design is used to identify the effects of pretesting on a study’s internal and external validity. When using this design, the study includes four groups that allow the researcher to evaluate (a) the effects of pretesting on the independent variable by comparing two groups that are both exposed to the independent variable, with only one group taking the pretest and (b) the effects of pretesting on the dependent variable by comparing two groups that are not exposed to the independent variable, with one group taking the pre- and posttests and the other taking the posttest only. **

Question 10

this type of threat to internal validity occurs when participants drop out of group for different reasons than participants in other groups do and, as a result, the composition of the group is altered in a way that affects the study’s results

Answer 10

differential attrition

Question 11

best way to control differential attrition threats to internal validity

Answer 11

attrition is difficult to control because researchers often don’t have the information needed to determine how participants who drop out of a study differ from those who remain

Question 12

this type of threat to internal validity occurs when the instrument used to measure the dependent variable changes over time

Answer 12

instrumentation

(e.g., raters may become more accurate at rating participants over the course of the study)

Question 13

best way to control instrumentation threats to internal validity

Answer 13

ensure instruments don’t change over time; if not possible, consider the potential effects when interpreting the study’s results

Question 14

list the 4 major threats to external validity

Answer 14

1) Reactivity
2) Multiple Treatment Interaction
3) Selection - Treatment Interaction
4) Pretest - Treatment Interaction

Question 15

this type of threat to external validity occurs when participants respond differently to the independent variable during a study than they normally would respons.

Answer 15

reactivity

Question 16

list & describe 2 factors contributing to reactivity threats to external validity

Answer 16

1) demand characteristics: cues that inform participants of what behavior is expected of them

2) experimenter expectancy: the experimenter acts in ways that bias the results, including:
2a) actions that take the form of demand characteristics & directly affect participants (e.g., saying “good” when a participant gives the expected or desired response)
2b) actions that don’t directly affect participants (e.g., recording the responses of participants inaccurately in a way that supports the purpose of the study)

Question 17

best way to control reactivity threats to external validity

Answer 17

• use unobtrusive measures
• use deception
• use the single- or double-blind technique

Question 18

difference between the single- vs. double-blind techniques

Answer 18

single-blind: participants don’t know which group they’re participating in (e.g., treatment or control)

double-blind: neither participants or researchers know which group participants are in

Question 19

best way to control multiple treatment interference threats to external validity

Answer 19

use counterbalancing (*e.g., having different groups of participants receive the different levels of the independent variable in a different order)

Latin Square is a type of counterbalanced design in which each level of the independent variable occurs equally often in each ordinal position

Question 20

this type of threat to external validity occurs when taking a pretest affects how participants respond to the independent variable

(e.g., answering questions about a controversial issue in a pretest makes subjects pay more attention to information about that issues when it’s addressed in a lecture or discussion during the study)

Answer 20

Pretest - Treatment Interaction

Question 21

in the Solomon four-group design the study includes 4 groups that allow the researcher to evaluate….

Answer 21

a) effects of pretesting on the independent variable by comparing 2 groups that are both exposed to the independent variable with only 1 group taking the pretest

b) effects of pretesting on the dependent variable by comparing 2 groups that are not exposed to the independent variable with only 1 group taking the pre- and posttests & the other takking the posttest only