Experimental Research Flashcards
True or false? Experimental research is the best technique to test causal relationships.
True; best if Classical
What must you consider when selecting a research question for experimental research?
- Must confront ethical and practical limitations of intervening in human affairs
- Researcher must be able to manipulate conditions (intrusive!)
- It is NOT an experiment unless YOU are manipulating something
- Best for micro-level concerns; narrow-scope with few variables that can be controlled
- Hard to address questions that require looking at conditions over time or entire society
- Researcher cannot examine numerous variables at once
- May limit one’s ability to generalize to larger settings due to artificial control
Is it an experiment if you take a group of people that smoke and a group of people that don’t smoke and see who develops lung disease?
No; you need to be actively manipulating the situation!
What is random assignment?
Dividing subjects into groups using a random process, so the experimenter can treat the groups as systematically equivalent
What is the point of random assignment? Why is it random and unbiased?
- Facilitates comparison in experiments by creating similar groups that differ only in treatment
- Random means a case has an equal chance of ending up in each group
- Unbiased because a researcher’s desire to confirm a hypothesis does not enter into the selection process
What is matching and its challenges?
- An alternative to random assignment
- Matching the characteristics of cases in each group to get equivalent groups
- Challenges ⟶ what are the relevant characteristics to match on, can one locate exact matches
What is random sampling?
Researcher selects smaller subset of cases from a larger pool
What is science of the sophomore?
A term that refers to the potentially limited external validity of studies based on undergraduate samples
What are the 7 components of the classical experiment process?
- Random Assignment
- Treatment/Independent Variable (X)
- What the researcher manipulates - Dependent Variable (O)
- Physical conditions, social behaviours, attitudes, feelings, or beliefs of subjects that change in response to a treatment - Pretest
- Measurement of dependent variable prior to treatment - Post-test
- Measurement of dependent variable after treatment - Experimental Group
- Group that receives treatment - Control Group
- Group that does not receive treatment
What is deception and how does it work?
- When an experimenter lies to subjects about the true nature of an experiment or creates a false impression through their actions/setting
- Allows the researcher to control subjects’ definition of the situation
- By focusing participants’ attention on a false topic, the researcher induces the unaware subjects to “act naturally”
What are confederates?
People who pretend to be other subjects or bystanders but who actually work for the researcher and deliberately mislead subjects
When is deception acceptable and how should it be done?
- Only acceptable if research cannot be achieved otherwise
- Type and amount should not go beyond what is minimally necessary
- Participants should be debriefed
What are the 3 main types of experimental design?
- Classical
- Pre-Experimental
- You have control over independent variable (eg. intervention), but lacking aspects of full classical design (NO random assignment) - Quasi-Experimental
- Less control over independent variable (real life), but still stronger than PED for establishing causation
- Stronger than pre-experimental designs for establishing causal relationships
What are the 3 types of pre-experimental designs?
NO RANDOM ASSIGNMENT
- One-Shot Case Study (One Group Post-Test Only)
- One group gets treatment, post-test
- Eg. students writing an exam - One Group Pretest Post-test Design
- One group gets treatment, pretest, post-test
- Eg. fitness regime for weight loss - Static Group Comparison (Post-Test Only Non-Equivalent Group)
- Two groups (one gets treatment), post-test
What are the 6 types of quasi-experimental design?
- Two-Group Post-Test-Only
- Random assignment, two groups (one gets treatment), post-test - Interrupted Time Series
- One group (no random assignment), dependent variable is measured at many time points (to get a baseline), and treatment occurs in the middle - Equivalent Time Series
- One group (no random assignment), several repeated pre-tests, post-tests, and treatments over a period of time
- Building credibility by making sure effect is due to intervention - Latin Square
- Random assignment, multiple groups, treatments in different sequences, group outcomes compared at the end
- See whether order of treatment has an effect
- Eg. taking courses in different semesters - Solomon Four-Group
- Random assignment, 2 control groups and 2 experimental groups, only 1 experimental group and 1 control group get pre-test
- Addresses the issue of pretests affecting dependent or treatment variable - Factorial Design
- Treatment is combination of several independent variables simultaneously
- Interaction effect: the effect of 2 independent variables that opereate together to create a “boost”
What is design notation?
- The name of the symbol system used to discuss parts of an experiment
- O = observation of dependent variable
- O1 = pretest
- O2 = post-test
- X = treatment/independent variable
- R = random assignment
What is the threat to external validity?
Reactivity: subjects are aware thay are in an experiment and being studied
Explain the internal and external validity of lab/field/natural experiments. If applicable, talk about how they are addressed.
- Lab experiments have greater internal validity but lower external validity as they are aware they are in an experiment
- Field experiments are the reverse; more generalizable, less controlled
- Solution ⟶ deception, no informed consent, or natural experiments
- QED where researchers examine impact of change in social system by comparing an outcome before and after change is implemented
What are the 7 threats to internal validity?
Anything that can affect the dependent variable OTHER than the independent variable!
- Selection bias
- Groups in an experiment are not equivalent at the beginning
- Solution ⟶ random assignment - History effects
- Something that is unplanned and outside the control of the experiment occurs and affect the dependent variable during an experiment
- Solution ⟶ equivalent time series, acknowledge that study has limitations - Maturation
- Due to natural processes of growth or boredom that occurs to subjects (eg. teens) during the experiment
- Solution ⟶ pre-tests, control groups - Testing effect
- When the pretest itself affects the dependent variable
- Solution ⟶ Solomon Four Group Design - Mortality / Attrition
- Subjects failing to participate through the entire experiment
- The reason subjects left may have an impact on the outcome
- Solution ⟶ let participants know it’s OK to leave anytime, ask for feedback to understand why - Contamination/Diffusion of treatment
- Treatment “spills over” from experimental group, and control group subjects modify their behaviour because they learn of the treatment
- Solution ⟶ isolate groups, have subjects promise not to reveal anything to others who will become subjects - Experimenter expectancy
- Researcher indirectly communicates desired findings to subjects
- Solution ⟶ double-blind experiment (neither the subjects or experimenter knows specifics of treatment)
- Placebo: a false treatment that appears to be real
What is the 2 threats to internal reliability?
- Statistical Regression
- A problem of extreme values or a tendency for random errors to move group results toward the average
- If many research participants score very high or low on a variable, random chance alone will produce a change between the pretest and post-test
- Solution ⟶ change population, use statistical tools - Instrumentation
- The measure of dependent variable changes during experiment
- Solution ⟶ maintenance/calibration of equipment
Discuss cross-sectional and longitudinal studies.
- Cross-sectional: single point in time (descriptive, cheap)
- Longitudinal: multiple points in time (explanatory)
- Time series ⟶ different people looked at different times (eg. time it takes to get home from work in 2005 vs. 2019)
- Panel study ⟶ same people across multiple time points
- Cohort study ⟶ category of people who share similar life experience over time
