Chapter 11 Flashcards
What are some things done to preserve internal validity?
- control groups
- standardized, controlled, environment
- many participants
- double-blind study design
Threats to internal validity
Design Confounds
There is a problem with how the experiment is designed
Threats to internal validity
Selection Effects
Levels of IV differ because the participants in each level are different
Threats to internal validity
Order Effects
- within groups design
- differences may be due to IV or the order that info is presented
- Includes fatigue and practice effects
Threats to internal validity
What are maturation threats
- A change in behavior that emerges spontaneously as a result of time, ex. spontaneous remission of depression
Threats to internal validity
How can you prevent maturation threats?
- include a comparison/control group
- Ex. Campers that do not have low sugar diet, or women enrolled in a different type of therapy
Threats to internal validity
What are history Threats
- Threat occurs because something specific has happened between the pretest and posttest that influences most participants in the same way
- Ex. weather changing
Threats to internal validity
How can you prevent history threats?
With a good comparison/control group
Threats to internal validity
What is regression threat?
- Due to regression to the mean: when a group average is extreme at time 1, it is often less extreme when the behavior is measured again at time 2. Due to beginner’s luck or choking
- Extreme scores are rare. You should expect that extreme measures on time one will regress to the mean. People who are chosen because they are extreme also regress to the mean at time 2.
Threats to internal validity
How to prevent regression threat?
Have a comparison group
Threats to internal validity
What is attrition threat
- Attrition: when participations drop out of the study before the end (common when pre and post test occur on different days)
- Attrition threat: when only certain kinds of participants drop out (the dropout isn’t random)
- Often in Clinical/medical studies. (Participants that live farther away from lab/clinic/hospital are less likely to come back, and are also more likely to live in food/health care deserts)
Threats to internal validity
How to prevent threats to attrition?
When people drop out, remove their scores from the entire data set
What is the difference between attrition and history threats
Attrition only affects some participants, history affects all
Threats to internal validity
What is testing threat?
- Specific kind of order effect
- Change in participant as a result of taking a test more than once.
- Participatns can get better at taking the test, or bored, or fatigued
Threats to internal validity
How to prevent test threats?
- Get rid of the pretest all together
- Use different forms for the 2 tests
- Have a comparison group
Threats to internal validity
Instrumentation threats
- Occurs when a measuring instrument changes over time
- Coders may change how they judge behaviors over time, or when a researcher uses a different form of measurement for pre/post test
Threats to internal validity
How to prevent instrumentation threats to internal validity?
- post test only design
- make sure pre and posttest measures are equivalent
- Retrain coders throughout the experiment
Threats to internal validity
Observer bias and demand characteristics
- Observer bias: When researchers’ expectations influence their interpretation of the results. Threatens internal and construct validity.
- Demand characteristics: Participants guess the purpose of the study and change their behavior
Threats to internal validity
How to control for observer bias and demand characteristics?
- Double-bind study design
- Masked design- the participants know and raters/coders do not
Threats to internal validity
Placebo effect
- When participant receives a treatment and measurably improves. But only bc of participant’s beliefs.
- Pain is very susceptible to placebo effects.
- Using a placebo equates expectancy
Threats to internal validity
How to control for placebo effect?
- Double blind placebo control study
- Neither the people treating nor the partcipants know which group they are in
- The type of placebo matters
Null effects overview
- When difference between levels of the IV are not big enough
- Sometimes null effects are important
Null Effects
Weak manipulations
- When IV is not altered enough to produce change in DV
- Ex. If I want to test does money=happiness. How much money is necessary to get the effect? $10 vs $10,000
Null effects
Insensitive measures
The change is too small to be detected
Null effects
Ceiling and floor effects
- Ceiling: all the scores are clustered together at the high end. (could be test is too easy)
- Floor: All scores clustered at low end
- Makes it hard to see real effects
What does it mean if an experiment gives a null result?
- The IV had no effect OR there is an effect but the study didn’t detect it
- There might not be enough between-group difference, or too much within group variability
What is a manipulation check?
- Detect weak manipulations, ceilings, and floors
- Looks at how varibles were operationally defined to confirm IV was successfully manipulated.
- Definition: Separate dependent variable that makes sure the manipulation worked.
- Ex. in an anxiety study, after telling people they will recieve electric shocks, ask about anxiety directly
Within group variability
- Too much variability within a group creates error variance (noise), making it hard to see true differences
What is measurement error?
- a human or instrument inflates/deflates a true score
- Ex. A person’s height is recorded wrong because of the angle of the observer
directly related to within group variablility
How to control for measurement error?
- Use a reliable and precise tool
- Measure more instances
Other causes to within-group variability
- individual differences: consider money and mood, some people are naturally more cheerful than others
How to control for within group variability
- Change the design (from independent groups to within groups)
- Add more participants
What is situation noise and how is it controlled
- Situation noise: external distractions, lots going on in the environment. Things of this sort can influence behavior
- Control: Often why researchers try and control the environment, helps to control extraneous variables
Reporting Null Effects
- They can be disappointing and somtimes you don’t want to report
- But it is important to accept what the data tells us
- Null effects are informative: vaccines do not cause autism, a safer treatmnet is just as effetive as an aggressive treatment, an education program is not more effective at teaching reading, etc.
Easter egg
8 kids throw basketball in pool
