Lecture 5 - Experimental Designs

Question 1

The classical experiment

Answer 1

• IV and DV
• Pretesting & Post-testing
  Pretesting: Measure DV before IV exposure
  Post-testing: Measure DV after IV exposure
• Experimental & Control Group (+ Random Assignment)
  Experimental: Exposed to the IV
  Control: Not exposed the IV

Question 2

Random Assignment (key component to classical experiments)

Answer 2

Assign participants to groups randomly

• produces statistically equivalent groups

Question 3

Criteria for causality

Answer 3

• Empirical association
• Temporal order
• Non-spuriousness
  (confounding variable)

Random Assignment helps to reduce spuriousness

Question 4

Threats to Internal Validity

Answer 4

• History
• External events that occur during the experiment that confound results
• Maturation
• Biological or psychological changes in participants during the study (not due to IV/DV)
• Testing
• process of testing and retesting may influence participants’ behavior (bias or foreknowledge)
• Instrumentation
• Changes in measurement tools or process during study
  Ex. Coach uses measuring tape in the study, then changes to electronic jump mat. Problem is the jump mat records slightly higher jump heights
• Causal time order
• Inability to distinguish order of the IV/DV (confounds results)
  Ex. study tests aggression due to gaming, but participants could just be aggressive to begin with
• Statistical Regression
• Tendency for extreme scores to shift towards the average during subsequent testing
  Ex. study tests speed of slowest athletes, but after one month of training, their results improve during the post-test
• Selection Bias
• Process of participant selection (may lead to differences in sample and population)
• Mortality (Attrition)
• Participants drop out of/ do not complete the study
• Diffusion of Treatment
• The experimental group may “contaminate” the control group when communicating
  Ex. study gives new physio treatment to experimental group, but standard treatment to control group. The control group finds out about the new treatment and uses it
• Compensatory Treatment
• Control group is deprived of something valuable and researchers try to make up for it consciously or subconsciously
  Ex. The experimental group receives new strength training, the control group receives regular training. Coach feels bad for control group and gives them extra conditioning workouts to “make it fair”
• Compensatory Rivalry
• Control group may attempt to “keep up” with the treatment group in their results
  Ex. Soccer players in control group feel overlooked and train harder to improve their results
• Demoralization
• Feelings of discouragement in the control group may result in participants giving up
  Ex. soccer players in control group feel demoralized because they are not receiving the new training program and give up or put in less effort

Question 5

Minimizing threats to Internal Validity

Answer 5

• Theory of Plausibility: Not all threats will be relevant
• Careful and controlled design and implementation
  Ex. well-defined, keeping groups separate, balance time between pre and posttest, offer incentives to complete study, observe for demoralization or rivalry

Question 6

Threats to construct validity

Answer 6

• Generalizing test results to causal processes in real world
• Complete measures or not?
• Common threat from insufficiently defined constructs
  (concepts or variables not clearly defined, leading to inaccurate or inconsistent measurement)

Question 7

Threats to External Validity

Answer 7

Can results of the study be replicated in other circumstances?

• High Internal Validity = Low External Validity
• High External Validity =Low Internal Validity

Prioritize Internal or External Validity depending on the goal of the study

Question 8

Threats to Statistical Conclusion Validity

Answer 8

Threatened when findings are based on a small sample
* Smaller samples only able to detect extreme differences

Question 9

Issues with Experimental Designs

Answer 9

Ethical considerations:
* Can’t deny treatment
* Can’t assign certain variables (victimization)

Practical considerations:
* Costly
* Intrusive
* Can’t manipulate some variables (i.e. race)

Question 10

Building Blocks of Experiments

Answer 10

• # of treatment and control groups
• # of variations of the IV
• # of pretest and posttest measures
• Participant selection and assignment procedures

Question 11

Two-Group Post-test only

Answer 11

• Has random assignment
• 2 groups - No Pre-test

Question 12

Classical Experiment

Answer 12

• Has random assignment
• Controls most validity threats
• 2 groups: Pre-test - Treatment - Post-test

Question 13

Solomon Four-group Design

Answer 13

• Has random assignment
• 4 Groups: 2 experimental & 2 control
• 1 experimental & 1 control receive pretest, other does not
• has the advantages of both post-test only and classical experiments

Question 14

Factorial Design

Answer 14

• Random Assignment
• Examines 2 or more IVs (factors)
• At least 4 groups:
  1. No treatment
  2. Treatment 1 only
  3. Treatment 2 only
  4. Treatment 1 & 2
• Allows for examination of multiple treatments and interaction of these treatments
• Greater flexibility and efficiency