Experiments Flashcards
Experiments & Experimental Design
scientific investigation involving manipulation of one or
more independent variables
Nonexperimental research
no manipulation of variables or random assignment
Types of Faulty Design
“One Shot Case Study” (Experimental)
Pre-test/Post-test
Simulated Before/After
“One Shot Case Study” (Experimental)
X intervention/treatment is implemented, Y
outcome is measured (still weak: no control, difficult to attribute Y to X)
Pre-test/Post-test
Same as “One Shot” but Y is measured before/after for group receiving treatment (still weak: no control, pre/post change can still be the result of a host of factors)
Issues with Pre-test/Post-test
Reactive measures: post-manipulation measures could result from pretest sensitization (reactive measures): controversial attitude and memory measures, for example, are affected)
History: greater the time between pre/post testing the greater chance extraneous variables are affecting post measures Specific to situation)
Maturation: things that occur between pre/post test that aren’t specific to the situation (e.g., changes in age that influence memory and attitudes)
Regression to the Mean: Low pretest scorers tend to be higher on post-test and High pre-test scorers tend to be lower on post-test. (Can lead investigator to falsely
conclude the experimental variable had an effect when in reality the scores were high/low due to chance)
Simulated Before/After
Same as pre-test/post-test except pre-test is done through a comparison group that is chosen to be as similar as possible to the experimental group
(control is weak because we don’t know if the groups were actually similar premanipulation)
Criteria of Research Design
Does the design answer the research questions? Does the design adequately test the hypotheses?
Control of Extraneous Independent Variables
Generalizability
Internal & External Validity
Types of Good Research Designs with Experiments
Experimental Group-Control Group: Randomized
Experimental Group-Group: Matched Participants
Control Groups
Matching by Equating Participants
Frequency Distribution Matching Method
Matching by Holding Variables Constant
Matching by Incorporating Nuisance Variable
Participant as Own Control
Pretesting & Difference Scores
Simulated Before-After, Randomized
Non Experimental Research
“Non-experimental research is systematic empirical inquiry in which the scientist does not have
direct control of independent variables because their manifestations have already occurred or
because they are inherently not manipulable”
Limitations of Non Experimental Research Interpretation
Danger of erroneous interpretations because of the plausibility of so many other explanations
Hypotheses (if-then predictions) can’t be made as soundly in nonexperimental research, but still better than having none and then creating hypotheses post-hoc after conducting every possible analysis (by chance alone some will be significant)
Value of Nonexperimental Research & Key Points
We need non-experimental methods because experiments are not always possible, and the things that aren’t conducive to being tested in an experiment are often important
Conditional statements can and should be explored using both experimental and nonexperimental
methods
Replication does not always mean repetition, empirical implications of theory can be tested in different situations experimental and nonexperimental
Laboratory Experiment
“ a research study in which the variance of all, or nearly all, of the possible influential independent variables not pertinent to the immediate problem of the investigation is kept at a minimum. This is accomplished by isolating the research in a physical situation apart from the routine of ordinary living , and by manipulating one or more independent variables under rigorously specified, operationalized, and controlled conditions”
Strengths of Laboratory Experiment
Relatively most complete control out of all experimental options: situation control, random assignment, and IV manipulation
High degree of specificity in operational definition of variables as compared to the field
More precision than other alternatives, driven by precision of experimental procedure and measuring instruments (lessons error variance) - controlled manipulation and environment where “contaminating conditions” are eliminated
Weaknesses of Laboratory Experiment
Lack of strong IV’s, effects detected by experimental manipulations are usually small (one reason for preoccupation with laboratory precision and refined statistics - otherwise effects won’t be detected)
Artificiality of research situation - might not hold in field, and low external validity (not as generalizable)
Field Experiment
“ a research study conducted in a realistic situation in which one or more independent variables are manipulated under conditions as carefully controlled as the situation will permit”
Strength of Field Experiment
If research situation can be “kept tight” then a field experiment is powerful
IVs usually have stronger effect than in lab experiments
Conducive to studying complex social and psychological influences, & processes in real-life situations
Weakness of Field Experiment
When IVs are manipulated and randomization is used the “criterion of control is satisfied”, but the experimenter is always worried that IV’s are contaminated by uncontrolled environmental variables
Manipulation and randomization is not always possible/realistic (e.g., giving one group of students a treatment that other students don’t receive)
Difficult to achieve precision and accuracy in the field as compared to the lab (IVs - extraneous variables, DVs sometimes measures are not sensitive enough to pick up variation caused by IV)
Field Study
“nonexperimental scientific inquiries aimed at discovering the relations and interactions among sociological, psychological and educational variables in real social structures.”
Types of Field Study
Exploratory & Hypothesis Testing
Qualitative Research
“social and behavioral research based on unobtrusive field observations that can be analyzed without using numbers or statistics”
Quasi-experimental designs
the manipulation of the IV makes the design experimental, but the lack of randomization makes it “quasi”.
Quasi-field
field experiments where its is hard to define what the field is - e.g., a study of a random scattered population of people across a country (such as through MTurk) rather than a study of an organization (organizational members)
Common misconceptions about field experiments
Eden, D. (2017)
Anti-experimental Bias: an inflexible belief that field experiments are inappropriate or too hard
Treatability: it’s true that not all variables are easy to alter by experimental treatment, but they can be studied experimentally as they naturally occur.
Sensitive variables that experimenters often want to avoid could be better for field research
Random Assignment vs. Random Sampling: random assignment supports internal validity while random sampling supports external validity. Random sampling does not meet randomization criteria for an experiment.
Quasi-experimental and nonexperimental studies are not experiments
Too many degrees of freedom: in a randomized experiment df should be based on number of groups not larger n within those groups
Field samples are hard to get - not true!
Overcoming Deterrents to Field Experimentation
Refrain from jargon
Explain randomization to lay managers
Capitalize upon management indifference
Use randomization as a fair way to allocate treatment
Invert the treatment (stress vs. coping)
Piggyback on naturally occurring events
Transform delicate data (e.g., private data of companies)
5 key benefits of Quasi-experiments
Strengthen causal inferences when random assignment and controlled manipulation are
not possible or ethical
Build better theories of time and temporal progression
Minimize ethical dilemmas
Collaborate constructively with practitioners. Quasi-experimental studies resolve 3 barriers to researcher-practitioner collaboration
Use context to explain conflicting findings
