Experiments

Question 1

Experiments & Experimental Design

Answer 1

scientific investigation involving manipulation of one or

more independent variables

Question 2

Nonexperimental research

Answer 2

no manipulation of variables or random assignment

Question 3

Types of Faulty Design

Answer 3

“One Shot Case Study” (Experimental)
Pre-test/Post-test
Simulated Before/After

Question 4

“One Shot Case Study” (Experimental)

Answer 4

X intervention/treatment is implemented, Y

outcome is measured (still weak: no control, difficult to attribute Y to X)

Question 5

Pre-test/Post-test

Answer 5

Same as “One Shot” but Y is measured before/after for group receiving treatment (still weak: no control, pre/post change can still be the result of a host of factors)

Question 6

Issues with Pre-test/Post-test

Answer 6

Reactive measures: post-manipulation measures could result from pretest sensitization (reactive measures): controversial attitude and memory measures, for example, are affected)

History: greater the time between pre/post testing the greater chance extraneous variables are affecting post measures Specific to situation)

Maturation: things that occur between pre/post test that aren’t specific to the situation (e.g., changes in age that influence memory and attitudes)

Regression to the Mean: Low pretest scorers tend to be higher on post-test and High pre-test scorers tend to be lower on post-test. (Can lead investigator to falsely
conclude the experimental variable had an effect when in reality the scores were high/low due to chance)

Question 7

Simulated Before/After

Answer 7

Same as pre-test/post-test except pre-test is done through a comparison group that is chosen to be as similar as possible to the experimental group
(control is weak because we don’t know if the groups were actually similar premanipulation)

Question 8

Criteria of Research Design

Answer 8

Does the design answer the research questions? Does the design adequately test the hypotheses?

Control of Extraneous Independent Variables

Generalizability

Internal & External Validity

Question 9

Types of Good Research Designs with Experiments

Answer 9

Experimental Group-Control Group: Randomized

Experimental Group-Group: Matched Participants

Control Groups

Matching by Equating Participants

Frequency Distribution Matching Method

Matching by Holding Variables Constant

Matching by Incorporating Nuisance Variable

Participant as Own Control

Pretesting & Difference Scores

Simulated Before-After, Randomized

Question 10

Non Experimental Research

Answer 10

“Non-experimental research is systematic empirical inquiry in which the scientist does not have
direct control of independent variables because their manifestations have already occurred or
because they are inherently not manipulable”

Question 11

Limitations of Non Experimental Research Interpretation

Answer 11

Danger of erroneous interpretations because of the plausibility of so many other explanations

Hypotheses (if-then predictions) can’t be made as soundly in nonexperimental research, but still better than having none and then creating hypotheses post-hoc after conducting every possible analysis (by chance alone some will be significant)

Question 12

Value of Nonexperimental Research & Key Points

Answer 12

We need non-experimental methods because experiments are not always possible, and the things that aren’t conducive to being tested in an experiment are often important

Conditional statements can and should be explored using both experimental and nonexperimental
methods

Replication does not always mean repetition, empirical implications of theory can be tested in different situations experimental and nonexperimental

Question 13

Laboratory Experiment

Answer 13

“ a research study in which the variance of all, or nearly all, of the possible influential independent variables not pertinent to the immediate problem of the investigation is kept at a minimum. This is accomplished by isolating the research in a physical situation apart from the routine of ordinary living , and by manipulating one or more independent variables under rigorously specified, operationalized, and controlled conditions”

Question 14

Strengths of Laboratory Experiment

Answer 14

Relatively most complete control out of all experimental options: situation control, random assignment, and IV manipulation

High degree of specificity in operational definition of variables as compared to the field

More precision than other alternatives, driven by precision of experimental procedure and measuring instruments (lessons error variance) - controlled manipulation and environment where “contaminating conditions” are eliminated

Question 15

Weaknesses of Laboratory Experiment

Answer 15

Lack of strong IV’s, effects detected by experimental manipulations are usually small (one reason for preoccupation with laboratory precision and refined statistics - otherwise effects won’t be detected)

Artificiality of research situation - might not hold in field, and low external validity (not as generalizable)

Question 16

Field Experiment

Answer 16

“ a research study conducted in a realistic situation in which one or more independent variables are manipulated under conditions as carefully controlled as the situation will permit”

Question 17

Strength of Field Experiment

Answer 17

If research situation can be “kept tight” then a field experiment is powerful

IVs usually have stronger effect than in lab experiments

Conducive to studying complex social and psychological influences, & processes in real-life situations

Question 18

Weakness of Field Experiment

Answer 18

When IVs are manipulated and randomization is used the “criterion of control is satisfied”, but the experimenter is always worried that IV’s are contaminated by uncontrolled environmental variables

Manipulation and randomization is not always possible/realistic (e.g., giving one group of students a treatment that other students don’t receive)

Difficult to achieve precision and accuracy in the field as compared to the lab (IVs - extraneous variables, DVs sometimes measures are not sensitive enough to pick up variation caused by IV)

Question 19

Field Study

Answer 19

“nonexperimental scientific inquiries aimed at discovering the relations and interactions among sociological, psychological and educational variables in real social structures.”

Question 20

Types of Field Study

Answer 20

Exploratory & Hypothesis Testing

Question 21

Qualitative Research

Answer 21

“social and behavioral research based on unobtrusive field observations that can be analyzed without using numbers or statistics”

Question 22

Quasi-experimental designs

Answer 22

the manipulation of the IV makes the design experimental, but the lack of randomization makes it “quasi”.

Question 23

Quasi-field

Answer 23

field experiments where its is hard to define what the field is - e.g., a study of a random scattered population of people across a country (such as through MTurk) rather than a study of an organization (organizational members)

Question 24

Common misconceptions about field experiments

Answer 24

Eden, D. (2017)

Anti-experimental Bias: an inflexible belief that field experiments are inappropriate or too hard

Treatability: it’s true that not all variables are easy to alter by experimental treatment, but they can be studied experimentally as they naturally occur.
Sensitive variables that experimenters often want to avoid could be better for field research

Random Assignment vs. Random Sampling: random assignment supports internal validity while random sampling supports external validity. Random sampling does not meet randomization criteria for an experiment.

Quasi-experimental and nonexperimental studies are not experiments

Too many degrees of freedom: in a randomized experiment df should be based on number of groups not larger n within those groups

Field samples are hard to get - not true!

Question 25

Overcoming Deterrents to Field Experimentation

Answer 25

Refrain from jargon

Explain randomization to lay managers

Capitalize upon management indifference

Use randomization as a fair way to allocate treatment

Invert the treatment (stress vs. coping)

Piggyback on naturally occurring events

Transform delicate data (e.g., private data of companies)

Question 26

5 key benefits of Quasi-experiments

Answer 26

Strengthen causal inferences when random assignment and controlled manipulation are
not possible or ethical

Build better theories of time and temporal progression

Minimize ethical dilemmas

Collaborate constructively with practitioners. Quasi-experimental studies resolve 3 barriers to researcher-practitioner collaboration

Use context to explain conflicting findings