Midterm 2 - Chpt. 9 Flashcards
How do we design an experiment to maximize the chances that we’ll be able to conclude that the IV caused changes in the DV?
Detect an effect if it’s there? (2 methods)
Experimental control:
- Ensuring only the IV changes across conditions
- Avoiding alternative explanations: experimenter expectancy effects, demand characteristics
Optimizing our operational definitions of the IV and the DV
- Avoiding ceiling and floor effects
- Having an appropriate pairing of strong IV and sensitive DV
Experimenter expectancy effects
A cognitive bias that occurs when a researcher knows what condition participants are in and therefore intentionally or unintentionally manipulates an experiment, thus impacting the findings. (EX: Clever Hans)
Experimenter treated Ps/data differently because they knew the expected outcome of the study
Experimenter Expectancy Effects are a threat to…
Internal validity
Solutions to Experimenter Expectancy Effects
1) a “double blind” study
2) automated instructions
3) all conditions run simultaneously
Demand Characteristics
- “A demand characteristic is any feature of an experiment that might inform participants of the purpose of the study.” - Cozby
- Threat to internal validity
- THUS, Participants might try to help/hurt you
- Be the “good participant”
- Mess up the results
₋- Socially desirability
When is deception needed?
To control demand characteristics
Straightforward Manipulation
Simple and easy - present p’s with something for the IV that will influence the DV
Staged Manipulation
1: Try to elicit a state (ex - frustration with a hard task)
2: Put participants into a situation where they actually feel part of it
Deception:
often, we deceive subjects about the real purpose of a study
- “Unrelated tasks” (filler tasks)
- Use of confederates or cover story
- Attempt to avoid demand characteristics
- NOT minimal risk (more on this later!)
- Participants can get suspicious
(Critical to debrief!!)
Pros of Straightforward Deception:
- PRO: Simple
- PRO: Cheap
- PRO: Very common
- PRO: Easy to pull off
Pros of Staged Deception:
- PRO: high experimental realism
- PRO: can often avoid demand characteristics
Cons of Straightforward Manipulation:
- CON: sometimes quite
artificial…might not really
simulate the “real world”
experience (low mundane
realism) - CON: demand characteristics
Cons of Staged Deception:
- CON: Tricky to pull off
- Ps Suspicion
- Hard to replicate?
- CON: Lots can go wrong
- May be costly
- Ethical?
Measuring Variables - types of DV in experimental designs:
○ DV in experimental designs
1. Self-report
2. Behaviours & Responses
3. Physiological
Ceiling Effect
Ps are all at maximum performance on the DV, so can’t see effect of IV
EX: question is too easy
Floor effect
Ps are all at minimum performance on DV, so can’t see effect of IV
EX: question is too hard
A strong IV…
Can cause big differences in levels of manipulation
A weak DV…
Causes little differences in levels of manipulation
Tough DVs are…
□ Resistant to change (takes a lot for people to switch their opinions)
Sensitive DVs…
Change easily
Studying an unknown topic - When would you use one over the other?
strong IV, sensitive DV
Finding a large effect - When would you use one over the other?
weaker IV, strong DV
READINGS
After stating hypothesis, deciding research designs, the next step is to…
operationalize variables and create an experiment for your participants
Turning hypothesis into a set of operations:
specific instructions, events, and stimuli to be presented to participants
2 Options for Manipulating the IV in Experiments:
- Straightforward manipulation
- Staged manipulation
Straightforward manipulation
operationalize IV using instructions and stimulus presentations
Stimuli may be presented verbally, in written form, or via video
EASIER TO INTERPRET RESULTS
Straightforward Manipulation - Whenever tasks involved in a study mimic experiences and conditions present in everyday life, the study is said to have…
high mundane realism
Staged Manipulation
sometimes, it’s necessary to create a series of events that occur during the experiment to manipulate the IV successfully
Can involve actors, or take the form of a cover story - ALL INVOLVE DECEPTION
Why would staged manipulation be used? (2)
Researcher may be trying to create a certain psychological state in participants, such as frustration, anger, or a temporary lowering of self-esteem
- SM can be used to stimulate situations that occur in the real world
- EX: an actor engaging in conversation with a driver
Staged Manipulations - Confederates
aka accomplice
Can be in different ways
Typically introduced as another participant in the experiment
May be used to create a particular social situation or administer the IV
When a study engages and involves participants in this deep way, the study is said to have experimental realized
Different from mundane, doesn’t resemble real-world experiences, but can still engage participants in a meaningful way
Allow for a great deal of subtle interpersonal communication that is hard to put into words
3 Additional considerations when manipulating the IV:
- Strength of Manipulation
- Cost of manipulation
- Manipulation checks
Strength of Manipulation
When operationalizing, researchers must consider manipulation strength
In general, it’s a good idea to make the manipulation as strong as possible
Making the levels of IV maximally different, while keeping everything else between the two groups the same
However, the strongest manipulation possible can sometimes be ethically problematic
◊ EX: 10 drinks instead of 2 in an alcohol condition
Cost of manipulation:
- Some receive grants
- Straightforward manipulations are often much less costly than complex, staged, experimental manipulations
Manipulation checks - WHY?
Often a good idea to include a manipulation check to directly measure whether the manipulation of the IV was successful, inducing the intended psychological state among participants
Why are manipulation checks difficult to implement?
Can be difficult to decide when to administer, as it might distract participants or inform them about the purpose of the experiment
◊ If the effects of the IV are expected to last long enough, manipulation checks can be placed after measuring the DV
2 Advantages of Manipulation Checks
If used in an early pilot study and reveals that your manipulation is not effective, you can change the procedures before running the actual experiment
- AKA, you can inc. the strength of your manipulation for the IV
The manipulation check is advantageous if the results show no effect of the IV on the DV]
- To conclude that your IV doesn’t truly affect your DV, you first need to rule out the possibility that you failed to manipulate the IV at all
- You know at least that the results were not due to a problem with the manipulation; reason must lie elsewhere
3 options for measuring variables:
- Self-report
- Behavioural
- Physiological
Self-report measures:
can be used to measure many different aspects of human thought and behaviour, including:
- explicit attitudes, judgements about someone’s personality characteristics
- intended behaviours
- emotional states
- confidence in one’s judgements
Using a published scale is preferred
Behavioural Measures
direct observation of behaviours
Record when behaviour happens, repeats within a period (rate), reaction time to a certain stimulus, duration
Physiological measures:
recording of a response of the body
Most popular: galvanic skin response (GSR): electrical conductance of the skin, which changes when sweating occurs
Often used to measure general emotional arousal, anxiety, or stress
◊ EMG measures muscle tension
◊ ECG measures heartbeat regularity and rate
◊ EEG measures the electrical activity of brain cells
} As learning occurs, or during different phases of sleep
◊ MRI captures images of brain structures
} fMRI measures blood flow to the brain
NOTE: the same variable can be measured using more than one category of measurement
3 Additional Considerations When Measuring Variables
- Sensitivity
- Multiple Measures
- Cost of measures
Sensitivity:
important when using behavioural measures of performance
Different ways of measuring can be of different difficulties
Ceiling effect
- Leads to issues interpreting results
- Floor effect
To diagnose a potential ceiling/floor effect, look for average values close to the minimum or maximum possible score
Multiple Measures:
Because any variable can be measured using various operationalizations, researchers sometimes include multiple measures of the same variable
◊ Producing various DVs; can let us know if the same IV can affect some measures of a DV, but not others
- Important to include all studies in the research report
◊ Unethical to only report the ones that worked
How should you sequence multiple. measures?
Worried about order-effects
SOLUTION:
- Present the most important measures first, less important later
- Overall impact of study is minimized
- Counterbalance the order of presenting the measures, or rely on complete randomization of order
Setting the stage:
Consider getting external feedback
Then, plan the experience from the participant’s viewpoint, AKA setting the stage
- No rules other than ensuring the study’s setting is plausible to the participants
- Prep informed consent form
2 Advanced considerations for ensuring control:
- Eliminating as many alt. explanations as possible
- Controlling participant expectations:
- Controlling for experimenter expectations:
Controlling participant expectations - Demand Characteristics
any feature of a study that might inform participants of the study’s purpose and consequently affect their behaviour
Reason why researchers don't want to expose hypothesis - Participants who were informed had ended up acting in ways that confirm it
How to control demand characteristics - 3 methods:
Use deception to mislead from the actual purpose
- Can use filler items/tasks
Asses whether demand characteristics are a problem by asking participants what they thought research was about
- If they don’t have an accurate view of the purpose of the study, they are unlikely to react to demand characteristics
Can be eliminated when people are not aware that an experiment is taking place or that their behaviour is being observed
Controlling participant expectations - Placebo Effects
Not knowing whether improvements were caused by what participants expect to feel after taking a drug
How to control placebo effects - 2 methods:
Placebo group can be added: participants receive a fake, ineffective version of the manipulation (EX: useless drug)
- Can occur in any context
- Sometimes used for ethical implications
Participant expectations can also be the primary focus
- Balanced placebo designs
Controlling for experimenter expectations - Experimenter bias/experimenter expectancy effects::
Experimenter might unintentionally treat participants differently depending on what condition they’re in - can alter participant behaviour
- Words emphasized, smiling more
® Experimenters record participant’s behaviours, with subtle differences emerging in how the experimenter interprets and records behaviours for people in different conditions
Controlling for experimenter expectations - 2 potential sources:
Experimenter might unintentionally treat participants differently depending on what condition they’re in - can alter participant behaviour
Words emphasized, smiling more
Experimenters record participant’s behaviours, with subtle differences emerging in how the experimenter interprets and records behaviours for people in different conditions
Controlling for experimenter expectations - 3 potential solutions:
Experimental procedures:
- Run everyone’s conditions simultaneously, so that experimenter’s behaviour is exactly the same for all participants
- Only under certain circumstances, EX: printed materials - Single-blind study: participants are unaware of which condition they are in
- Double-blind: neither the participant nor the experimenter knows the participant’s condition
- Requires 2 experimenters: one who administers IV, another who takes over and administers DV, without knowledge of the IV
Before collecting data, researchers must seek approval from…
their institutions research ethics board
- Who will examine procedure, materials, and informed consent form
- Deception must be justified
2 decisions that must be made by institutional ethics boards before applying for ethics approval:
- Participant selection process
- Debriefing procedures
5 Requirements for Selecting research participants:
- Method must be justified by the REB or Animal Care Committee, and have intention for generalizing research from your sample to a population
- Samples may be drawn from any population using probability sampling
- Important, as it informs whether we can generalize results to a population - Benefits/burdens must be equally distributed
- Special procedures required for sensitive populations: Indigenous, people with mental illnesses, children, people living in poverty/institutions
- Inc. sample size = inc. in likelihood that you will find an effect
Funnelled debriefing
- Begin broadly, then narrow on a key deception
- Can prove useful in diagnosing whether patients were behaving as they normally would, or if they were suspicious of the manipulated IV
2 issues to keep in mind when collecting data:
Whether to complete a pilot study
The commitments that researchers have to the participants
Pilot studies:
Trial run, with a small # of p’s, to test out the procedures
- Used when procedures are elaborate or costly
Reveals if p’s understand, whether experimental setting seems plausible, whether any questions are confusing, and so on
- Esp. important when using a staged manipulation of the IV, to ensure a believable scenario
Can help experimenters rehearse and standardize procedures
Think Aloud Protocol - Pilot Studies
ive, verbal comments during the study
4 Researcher commitments:
Contracts with participants
P’s value punctuality, an important obligation of researchers
Keeping promises (ex: sending results)
Maintaining trust
Analyzing/interpreting results:
- Analyze data after being collected
- Stats allow researchers to examine and interpret the pattern of data obtained in the study
- Relation? IV on DV?
- Depending on results, researchers might choose to conduct follow-up studies
Communicating research to others
Professional conferences:
- CPA, APA, APS hold presentations for research
- Journalists also attend
Journal articles:
Editor sends a submitted paper for peer review: two or more scientists read the paper and provide an evaluation
Can recommend acceptance of rejection
