Experiments (Exam #2 social research methods) Flashcards
Key features of a “true” experiment
- Manipulated independent variable
(treatment); measured dependent variable (control) - Pre-testing and Post-testing
- Two groups – an experimental and
comparison group - Random assignment of subjects to groups
internal validity (causal validity)
The extent to which the results in a study can be attributed to treatments rather than to flaws in research design
external validity
The extent to which results can be
generalized beyond the particular context of
the experiment
random assignment vs random sampling
Random assignment: A procedure by which each experimental subject is placed in a group randomly.
Random sampling: A method of sampling that relies on a random, or chance, selection method so that every element of the sampling frame has a known probability of being selected
random sampling = selection
random assignment = assignment
threats to internal validity
- History effects / External effects
- Endogenous change (maturation, regression, and testing)
- Testing effects
- Instrumentation
- Statistical regression
- Selection bias (Differential attrition / mortality)
- Contamination
- Staff expectations (Double-blind experiments)
- Hawthorne effect
Identify the threat to internal validity: Ferrer-Wreder et al. conducted a study to evaluate school-based intervention programs geared towards urban at-risk minority youth. As part of the design, a pretest and posttest were administered. What is one of the sources of internal validity that may be compromised through the participants’ exposure to a pretest if there is no comparison group?
Identify the threat to internal validity: In a study of the effectiveness of public, Catholic, and non-religious private schools within the Chilean voucher system, the
researcher attempts to evaluate the success of institutions by using student-level data. The researcher compares students from each school but suppose that he does not control for individual ability or motivation. What threat to internal validity does this lack of control create?
Identify the threat to internal validity: In a national evaluation of the Gang Resistance Education and Training (GREAT) Program, a cross-sectional design was utilized. Although random assignment was not used, the evaluation had a low rate of differential attrition. Specifically what
type of internal validity is differential attrition?
Identify the threat to internal validity: The research design used in Ebensen and Osgood’s study of GREAT was a posttest only comparison between participants who were in GREAT the previous year, as well as non-participants of the program. Students were not randomly assigned to either treatment or comparison groups. There were no pretest measures. Participants in GREAT interacted with non-participants during the study. What threat to internal validity could this interaction create?
Identify the threat to internal validity: In 1997, Farrington studied methods of evaluating community crime prevention in the UK. He used community as his unit of
analysis and compared communities through pretest and posttest measures, looking specifically at variables of crime
delinquency, substance abuse, and adolescent problem behavior. Suppose that between the pretest and posttest, a large
company goes out of business. This causes a surge in unemployment and shortly thereafter, crime rises, as well. The
impact of this loss of jobs is not felt in other communities. This example illustrates what type of interval validity?
Identify the threat to internal validity: In a 1996 study of the effectiveness of behavioral modification programs for adolescents, the researchers evaluated the behavior of the subjects using a pre/posttest design without any
control group. The researchers were anticipating cyclical behavior patterns, so they decided to use scores of points
averaging over a period of four weeks. Specifically, what potential threat to internal validity were the researchers
attempting to avoid?
Identify the threat to internal validity: Using the previous study of the effectiveness of behavioral modification programs for adolescents, let’s imagine that the
subjects were aware of the ongoing evaluations by staff. If this awareness results in the adolescents’ modification of behavior,what threat to internal validity does this create?
Identify the threat to internal validity: In the same study of hospital-based behavior modification programs, the researchers employed various staff members, over a span of four weeks, to assign points to score participants on the basis of their behavior. If the staff members were informed of the participants expected behavior scores at the start of the program, what threat to internal validity is created?
In Smalltown, USA, they have recently instituted a voluntary energy conservation program. An evaluation researcher has
been hired by the town to estimate energy savings due to this program based on the volunteer participants and non-participants. However, the researcher realizes that an effective evaluation cannot be conducted because of this threat to internal validity?
Identify the threat to internal validity: In McEwan’s study of the effectiveness of public, Catholic, and non-religious private schools within the Chilean voucher system,
the researcher could not isolate the groups so that they were prevented from interacting with one another during the course of the evaluation. What threat to internal validity did this create?
