Lecture 5 Flashcards
In a non-experimental study do we have an IV or DV?
in a non-experimental study we don’t have IV or DV because we are observing both
What are the 3 steps for establishing causality?
- Correlation, or relationship, between IV and DV
(this means if we have a 2 conditions study, that means that those conditions have different levels of the DV because the IV is changing it.) - Temporal precedence
- Causes must precede effects § IV precedes DV
(we manipulate the variable first and then observe the effects of it after) - Eliminate alternative explanations, or confounds § (similar to third variables in non-experimental designs)
- IV is the only variable impacting DV
(third variables is for the non-experimental method and we use the word confound for experimental designs)
What is a confound?
Confound: a variable that researcher manipulates that is not directly related to the hypothesis
- Think ‘accidental IV’
(its not something they wanted to manipulate but it is changing and systematically)
What are the 2 rules of confounds?
- Confounds must covary/correlate with IV (the confound will usually be present at one level of the IV and not at the other level of the IV)
- Confounds must cause change in DV (Ths is because if it doesnt impact the DV it cant be responsible for the change)
Provide 2 confound examples with a hypothsesis, method, and IV and DV
Hypothesis: “Listening to relaxing music improves test performance”
Method: Randomly assign participants to either wear headphones playing relaxing music or a control condition
IV: Relaxing music vs. control
DV: Test performance
n=100
- Relaxing music (headphones)
- high test performance
- no music
- Low test performance
the presence of the headphones is a confound. There is now uncertainty about which of these causal flows is correct.
There’s another possible confound. The expectation of the participants. If you are doing a study, the researchers give you headphones and you are listening to music that makes sense, but if nothing is playing you might be confused which could also impact your performance
What threatens internal validity? Example?
What is helpful to internal validity?
Foe:
- Confound: Manipulated variable that is unrelated to the hypothesis
- Example: Experiment testing relaxing music and test performance § Intended IV: music vs. no music
- Accidental IVs:
- headphones vs. no headphones
- ambient noise vs. no ambient noise
(have we correctly identified the causal variable? if we are not confident that it is the music and no t the headphones impacting the DV, we have low internal validity. )
Foe:
Confound: Participant variable – preexisting individual differences
Intended IV: relaxing music vs. no music
Accidental IV: Participant music preferences
Friend:
- Experimental Control: Only IV changes/varies
- Everyone wears headphones
- Everyone hears no ambient noise
(if you replicate the same effect with a variety of different confounds, then we can be more confident that the IV is impacting DV and not any one of the compounds. )
Friends:
- Random assignment: Eliminate participant variable confound
- Assigning conditions usually equates participant variables
- Matched pairs design: Pretest known or suspected confound
- “Do you LIKE or HATE listening to relaxing music?”
- Pair up, or match, participants
(you pair 2 people with the same preferences and you give them each different conditons. Therefore, you have for sure controlled that pethis could not possibly be a confound )
What is the question related to internal validity?
is the cause valid?
How could you change the method in the music listening study to avoid the confound of headphones?
Method: Headphone for all, participants choose whether to listen to music or not
When is the failure of random assignment higher?
the failure of random assignment in a normal experimental study is higher in smaller sample groups so you should replicae to ensure it hasnt failed.
Why is it risky to let people pick their own conditions? Give example from music experiment relaxing music and no music
the confound is present because you allowed people to select their own conditions, if you like relaxing music, then you will select that condition and vice versa. Also, your music preference also probably indicates other things about you, and you could have other factors that are common to music likers that makes them score higher.
Provide a confound example for an experiment testing if caffeine increases athletic performance.
200 high-performance track runners completed 4 different races and were timed for each. The runners were either assigned to the 100mg caffeine condition or the control condition.
During the first 2 races, each of the runners were timed to make sure there were no differences between the caffeine group and control group. Before the last 2 races, the caffeine group was given a small pill containing 100mg of caffeine.
Runners in the caffeine group ran 3.85 seconds faster on average than did runners in the control group.
could be a placebo affect because we gave one group a pill that is percieved to make a difference and the other group didn’t recieve a pill. This could lead to a placebo. By being given a pill you might think its going to do something which creates pre-existing ideas on how you will perform which could impact it. If they saw the other people get pills, the control group might think they are going to lose.
Provide a confound example testing the question Does the “SuperSmarty” program enhance reading comprehension?
50 elementary school students enrolled in the study. First, students all took an exam to measure their baseline scores on reading comprehension. Next students completed 3 weeks of the “SuperSmarty Program!”
§ After completing the program, students took another exam designed to measure reading comprehension. Results showed that students scored 10 points higher after they completed the “SuperSmarty Program!”
the name of the program creates expectations. There’s also no control group so they are all learning which already makes it more likely that they will score higher.
How do you control participant variables?
- Randomly assign Ss to ONE condition: Between-subjects design
- Pair Ss based on pretest: Matched pairs design (this isn’t necessarily only 2 peopel it is just the amount of conditions you have)
- Ss experience ALL conditions: Within-subjects design (on some days the runners would take the caffeine pill. On other days they would take the placebo control pill)
What are the 3 possible types of between subjects designs?
- Independent groups
- Between-groups
- Posttest-only
What are the 2 possib le types of the matched pairs design?
- Dependent samples
- Pretest-Posttest
What are the 2 possib le types of the within subjects design?
Repeated-measures
Within-groups
How does an independent groups between subjects design work?
- Each group is randomly assigned (not actually randomly assigned)
(Did I randomly assign groups? How might the groups vary?) - Participants experience only one level of the manipulation
Compare results from each group:
how does a matched pairs pre-test posttest design work? Example? What variables do you measure in the pretest?
- Measure additional variable in a pretest that we wish to control with certainty
- Example: How fast do you recognize colors without word labels? (could take out the word part of it and just present colours. Then we will look at perceptual processing speed (basically how fast they react))
- Perceptual Processing Speed § Experience matched pairs!
- Measure variable that you believe a priori will be important to control for (usually something that could be very related to the dependent variable.)
Example: How fast do people recognize colors without word labels? - Pair up those with similar speed; assign to different groups
- Also known as “pretest-posttest design”
- Pretest = color naming task without word labels
- Posttest = color naming task with labels
What does a priori mean?
a priori = before the study
What are 3 reasons to use a matched pairs design?
- Study can only recruit small sample
- Ensure successful random assignment on key participant variable - fMRI/EEG: pretest necessary so that participants understand task
- Ensure high-quality scanning data - High study mortality expected
- Ensure Ss dropouts don’t create unequal conditions - Example: Low-income parents and children
What is the challenge of a matched groups design
Challenge: Pretest may change psychological experience of IV
- Practicing color task with nonsense letters may change behavior (likely help performance in post-test)
(pre-test might change the psychological experience of the independent variable.)
(if you give peopel a pretest it is more likely that they will develop strategies on how to take the test and ‘do well’)
What is the solution to the problems with a matched group design?
Solomon Four-group Design
M+
- P +
- P -
M-
- P+
- P-
could expose one group of the match condition a pretest, one group of the match condition no pretest and same for mismatch
you want to look at the difference between the pretest and no pretest to see if it has a meanigful difference
What is order effects. Provide examples. Is there always a solution?
- Evaluate White or Black candidate first: Does it matter? § Probably!
- Should participants…
- Evaluate White candidate then Black candidate?
- Evaluate Black candidate then White candidate?
- Psychological processes will differ based on what manipulation
came first! - Sometimes, there is no obvious solution L
Compare repeated measures to matched groups design
Repeated-measures is MUCH more powerful
- 16 Ss for repeated-measures vs. 450+ for between-subjects
- Matched pairs somewhere between repeated-measures and between-subjects
(by powerful he means it will be easier to detect a true effect on the DV vs IV. If we use matched we need 16 participants to observe the relationship to have confidence, for the between subjects we need 450 + observed effects to be confident)
- Random assignment of Ss unnecessary for repeated-measures § Participant variables automatically, and completely controlled
- “Everyone serves as their own control” (or own ‘pair’)
- Between-subjects need 30-50 Ss for successful random assignment
(the smallest study that we could ever run would be 30-50 but there is no minimum bounds for a repeated measures study. You could have 4 participants only) - Repeated-measures subject to order effects, a type of confound § Example: Practice effects
- Participants develop strategies for conditions
(They can start developing strategies after the first block. Ideally, we would have some people sstart with the matching and others starting with the mismatchin and observe if people are getting better no matter what. )
( order cna be a confound)
What are the challenges with repeated measures?
Increased risk of manipulation awareness: Participant knows what is being manipulated
- Demand effect: Ss behaves in way they think experimenter wants § Reactance effect: Ss behaves opposite from what experimenter
wants
- Order effects: Performance may depend on order of conditions, even without manipulation awareness
- Practice effects: Performance improves on later tasks
- Fatigue effect: Performance may decline or inattention may
develop later in experiment
- This is a change in psychological experience!
- Contrast effect: Participants contrast their response against previous manipulation
what is a carryover effect?
Carryover effect: Ss become aware of the manipulation
WATCH LECTURE RECORDING FOR THIS
Summarize the challenge of repeated measures. What is the solution? Whats the problem with the solution?
Challenge: Order effects
- Psychological experience is confounded with the order of
manipulations
- Example: Evaluating Black vs. White resume first will impact evaluations of second resume
- Solution: Counterbalanced design
1. List all possible orders of IV
2. Ss complete one of these orders
But what if?!?
- Also manipulating sex of applicants and also add Asian applicants
- 6 different experimental treatments, or conditions
- 720 orders!!
What is another name for the latin square design?
parital counterbalancing
What is the latin square? What are the advantages and disadvantages?
- Each treatment appears once in each ordinal position
- Example: Asian female applicant appears in position 1,2,3,4,5,6 2. Each treatment appears both before and after unique
treatments
- Example: Asian female appears after condition 2, 3, 4, 5, 6
- Advantage: From 720 conditions back to 6
- Disadvantage: Only tests a subset of all possible orders
