Exam 3 Flashcards
Quasi-Experiments
Independent variable is manipulated despite no random assignment, groups aren’t equivalent
- Practical/ethical reasons
- Must carefully monitor threats to internal validity
- Could yield meaningful results even w/ no causation established
Pre-Experimental Designs
Data collected in a way that offers multiple explanations for data
Types of Pre-Experimental Designs
- One Shot Study
- Pretest/Posttest (one group)
- Nonequivalent control group
One-Shot Study
One group of participants is tested
- Disadvantage: How do we know if there was any change?
- Example: count drug-related deaths after a rehab program is implemented
Nonequivalent Control Group
An experimental and nonequivalent control group are tested once
- Disadvantage: may already be significant differences between groups
- Example: count drug-related deaths in this and another city after implementing rehab program
Pretest/Posttest
One group of participants is tested at pretest and posttest
- Disadvantage: Something else could have happened between the two testings
- Example: compare drug-related deaths before and after a rehab program is implemented
Types of Quasi-Experiments
- Pretest-Posttest Nonequivalent Control Group
- Time-Series Design
- Multiple Time-Series Design
Pretest-Posttest Nonequivalent Control Group
Two comparable but not randomly assigned groups of participants are tested before and after treatment
- Allows you to measure relative change between two groups
Time-Series Designs
One group is tested several times before and several times after treatment
- If data show a consistent trend, the likelihood that results are due to a confounding variable is reduced
Multiple Time-Series Design
Two nonequivalent groups are tested several times before and several times after treatment
- Can observe change in experimental group before and after manipulation
- Can measure relative change between both groups when independent variable manipulation is introduced
- Essentially a combo of time-series design and nonequivalent control groups design
Threats to Internal Validity of Quasi-Experimental Designs
- When multiple measures are made
- Statistical Regression
- Subject Attrition (Mortality)
- Selection Bias
- Interactions of selection with other threats to internal validity
When Multiple Measures are Made (Threats to Internal Validity in Quasi-Experiments)
Can lead to history, maturation, testing, and instrumentation effects
- Can be controlled by assessing/accounting for issues
Statistical Regression (Threats to Internal Validity in Quasi-Experiments)
Extreme scores are likely to move toward the mean upon retesting
- Can be avoided by not choosing participants on the basis of extreme scores
Subject Attrition/Subject Mortality (Threats to Internal Validity in Quasi-Experiments)
Loss of participants’ data either due to participants withdrawing from study OR because a decision was made to drop their data due to criteria
- Threat when dropout rate is very high or when dropout rate varies between groups
- Bigger problem with nonequivalent groups
Selection Bias (Threats to Internal Validity in Quasi-Experiments)
Differences between the comparison groups within a study
- Sometimes no better alternative
- Example: if someone had sleep apnea it would be hard to access enough patients to assign some to a no-treatment group, so recruit family/friends
Interactions of Selection with Other Threats to Internal Validity (Threats to Internal Validity in Quasi-Experiments)
Occurs when extraneous variables effect one group, but not the other
- Selection can also interact with extraneous variables like maturation, instrumentation, and regression towards the mean
Factorial Designs
The effect of a dependent variable of two or more independent variables is assessed simultaneously
- Can save on time and participants
Factor (Factorial Design)
Another name for independent or subject variables in the design
- Equal to number of main effects
- Marginal means
Levels of Factors (Factorial Design)
Different levels of each factor (would be conditions in non-factorial design)
Unique Condition (Factorial Design)
Unique combination of factors
- equal to levels of all factors multiplied
Types of Factorial Designs
- Within-Groups Design
- Between-Groups Design
- Mixed Design
Higher Order Design
3 or more factors
Within-Groups Design (Factorial Design)
All participants experience all levels/combos of all factors
Advantages: saves time, lower error variance, less participants
Disadvantages: carryover effects, can’t test subject variable
Between-Groups Design (Factorial Design)
Each combination of factors is experienced by a different group of participants
Advantages: no carryover effects (scores shouldn’t change from multiple testing)
Disadvantages: more participants/time, higher error variance
Mixed Design (Factorial Design)
Some factors are within-groups and some are between-groups
- Chosen for practical reasons
How to Analyze Factorial Designs
Use some sort of ANOVA (Analysis of Variance)
- Dependent variable: y-axis
- Independent variable: one on x-axis, others are separate lines
Generally
- Factor w/ most levels OR continuous factor goes on x-axis
- Done as line graphs even if both factors are categorical
How to Ensure a Well-Designed Factorial Design
Avoid possible confounds
- Pilot Study: mini version of the experiment with a few participants
Correlation Coefficients
Requires at least 2 scores from each participant to calculate
- Absolute difference from zero = strength
- Sign = direction
Regression
Using correlation coefficients to predict scores
Multiple Correlation
Measures relationships between multiple measures and one particular measure
- 0.00-1.00
- only looks at strength, not direction
Multiple Regression
Use multiple correlation coefficient to predict a particular measure
- Provides info about degree to which each initial measure contributes to prediction of measure of interest
Observational Research
Investigations involving no manipulation of an independent variable
- Use operational definition
- Requires forethought and planning
Naturalistic Observation
Unobtrusively observe behavior/do nothing to interfere with natural behaviors
- Participants may not realize they are being observed
- Often broad focus, observe/record many different types of behavior but may have specific focus
Issues:
- Does researcher’s presence affect behavior?/How would the researcher know if it did?
Participant Observation
Becoming an active participant in the situation being studied
- Passive observation yields limited/biased info and a lack of context
Disguised Participant Observation
Other participants are unaware researcher is observing them
Undisguised Participant Observation
Other participants know researcher is observing their behavior
- Often used in anthropology
Field Experiments
A controlled study that occurs in a natural setting with random assignment
- Independent variable manipulated
- Dependent variable measured
- Determines causation
- Potential issues w/ informed consent
- Greater external validity than lab experiments, but some issues with internal valdiity
Habituation
Spend enough time among participants that they resume normal behavior
Desensitization
Gradually move closer to participants until one can be near/along them
Laboratory Experiments
Experiments done in a controlled lab setting with controlled variables
Advantages of Observational Research
Offers a better understanding of natural enviroment interactions
- Starting point for research on a new topic (helps avoid jumping into costly lab study)
- Can be used after lab research to view phenomena in a natural setting
Disadvantages of Observational Research
Most are correlational, therefore NO CAUSATION
- Confounds such as: Hawthorne Effect, Reactivity, Observer Bias, etc.
Reactivity
The actual change in a participants behavior when they know they’re being studied
Minimizing Reactivity
- Desensitize
- Habituate
- No guarantee measurement isn’t reactive, only unobtrusive measures can guarantee no reactivity
Reactive Measures
Measured behavior that changes because a participant is aware they are being studied
Interobserver Reliability
The degree to which a measurement procedure yields consistent results when used by different observers
Hawthorne Effect
Effect of observer on participant behaviors
- Reference to study at Hawthorne Plant by Western Electric Company
Video/Audio Recording (Observational Research)
Advantages: Less Conspicuous, Permanent Record
Disadvantages: Less Flexible/Complete, still susceptible to Observer Bias
Video Recording (Observational Research)
Advantages: May be less intrusive, Gives visual information
Disadvantages: Only records what is in front of it. May miss larger context
Audio Recording (Observational Research)
Advantages: Easiest to conceal/keep from influencing behavior
Disadvantages: No video
Observer/Experimenter Bias
Occurs when researcher has an expectation about the study results and unconsciously perceives and records differently
Demand Characteristics
Participants change their behavior based on what they think the study is looking for
How to Calculate Agreement Between Observers
(Number of Agreements/Number of Opportunities to Agree) * 100
Static vs. Action Checklist
Record things that will not change during the observation VS. record presence/absence of specific behaviors/characteristics over time
Sampling Techniques
How you choose a sample:
- Pick the group your observe
- Pick when you observe them
- Pick where you observe them
Can be used to generalize observations/conclusions to a larger population, or can be used to focus on a specific group
Biased Sample
A sample that is unrepresentative of the population
Behavior Sampling
A researcher observes subsets of participant behavior at different times and/or in different situations
Time Sampling
Record behavior at particular time intervals
- Systematic
- Random
Event Sampling
Random/Systematic sampling of events that include the behavior of interest
- When behaviors don’t necessarily occur on a continuous basis
- Systematic
- Random
Situation Sampling
Observations made in different circumstances/settings
- Increases external validity
- Doesn’t rely on a single sample to be representative
