Week 12 - Carrying out experiments Flashcards
Recap of experimental design
- Used to establish cause + effect
- The IV is manipulated while the other variables are controlled for
- Researcher wants to understand how varying the levels of the IV affect the DV
Recap of quasi-experiment
- Researcher cannot randomly assign ppts to conditions
- The non-randomisation of ppts to conditions means we cannot claim that manipulation of the IV causes the DV to change
-> instead we talk about a relationship or link between the IV + DV
How do we interpret the results from an experiment?
In the ideal experiment:
- Changes in DV are due to manipulating the IV
However:
- Change in DV could also reflect:
-Biases -> systematic error
-> might conclude there was a significant effect when there wasn’t actually one
-> particularly problematic
-> sources of systematic error can affect the study’s validity
e.g. consistently starting the timer at different times for ppts
-Random error -> unsystematic error
-> factors that influence individual’s scores or individual’s performance
-> tend to be less detrimental because we typically average across several ppts’ performance when we make conclusions
-> affects the study’s reliability - the consistency in our results
e.g. fluctuating weather conditions
What is observer/experimenter bias?
- Where researcher’s subjective opinion influences data collection
-> Subjective variables most vulnerable e.g. instances where researchers are required to rate ppts’ behaviour - Woods and colleagues (1998) found a baby was rated as being morree troubled when individuals were told the baby’s mother took cocaine during pregnancy
-> an expectation influenced their subjective ratings of the baby’s behaviour - Rosenthal and Fode (1963) found that telling students in advance about their test-rats maze solving ability influenced the outcome of the task
-> “Maze bright” rats were found to perform better than “maze dull” rats (despite rats being randomly allocated to a group)
How can we tackle observer/experimenter bias?
- Limit the researcher’s role in data collection
-> if we use subjective measures, could we use more objective ones?
-> could also use computer to collect data rather than researcher or written instructions rather than verbal - Single-blind technique
-> either the ppt or the researcher don’t know what condition thee latter is in
Double-blind technique
- Neither the researcher nor the ppt know which condition the latter is in
Why should we keep test conditions constant?
- Should keep testing environment (e.g. situational variables and procedural variables) exactly the same for each ppt
- Typically researchers tend to carry out experiments in labs
Why do we need to follow a standardised procedure?
- Each testing session should follow a standardised set of procedures
i.e. each ppt is treated in a standard way
-> this will help reduce any potential sources of error in your work
-> particularly important when multiple researchers are involved in the data collection process
Why should we take multiple measurements?
- Ppts themselves may behave inconsistently (ppt error)
- Therefore, its a good idea to take a large sample of their behaviour as opposed to a limited number of measurements
-> measure their behaviour across several different experimental trials rather than taking one measurement for each of the conditions
Trials usually presented in experimental blocks
- Experimental blocks consist of a number of experimental trials
-> experimental blocks allow ppts to take a short break before the next experimental block
- Another reason why we take multiple measurements is to overcome variation in stimuli not related to the experiment manipulation itself
e.g. if you want to look at how attractiveness affected sociability rating of faces
-> would need to include a selection of photos for both attractive and less attractive conditions
-> averaging across faces would overcome variation in stimuli
What are demand characteristics?
- Ppt may exhibit demand characteristics not only in respect to what they think the researcher expects to find but also could be a reaction to the testing environment itself (experimental setup)
- Orne + Scheibe (1964) found that asking ppts to sign a release form as well as giving them a panic button to press during a sensory deprivation experiment caused them to act in a more extreme manner than controls
-> findings suggest that variations in the experimental setup influenced how ppts responsed to sensory deprivation
-> experimental setup can prime ppts behaviour
How can we prevent demand characteristics?
- Deception
-> may conceal study’s hypothesis so they don’t know what we expect to find
-> may conceal what measurements are of interest so ppt won’t be able to guess what we are actually measuring
If using deception, need to debrief ppts at earlier opportunity + give them the opportunity to withdraw their data after finding this out
- Use measures that are hard to control e.g. reaction time, physiological responses
- Blind techniques
What is internal validity?
- Whether we can determine or confidently say that there is a causal relationship between the two variables we are studying
-> can we confidently say that the changes observed in the DV are due to manipulating the IV?
-> Highly controlled experiments have high internal validity
-> if carrying out between subjects design, also important we randomly assign ppts to experimental conditions
-> systematic sources of error need to be eliminated otherwise this would affect the validity of the experiment
What is external validity?
- Refers to whether findings can be applied to situations beyond the study itself (generalisability)
High external validity means findings can be applied to a wide range of people, times and across different settings
Ecological validity -> refers to the extent to which the experiment reflects the real world
-> would we get the same results if the experiment was carried out in a natural setting compared to the lab
- Carrying out highly controlled experiments limits external validity
Why do we base our conclusions on groups of ppts?
- Individual differences could affect results if basing our inferences on the performance of single ppts
- Testing multiple individuals is like performing the experiment over and over again
Why do we need to randomly allocate ppts to conditions?
- If using a between subjects design, need to randomly allocate ppts to groups in order to establish cause + effect
-> to ensure any individual differences wash out
Random allocation -> the allocation of previous ppts should not influence the allocation of future ppts
-> every ppt has an equal chance of being assigned to each condition
- One way of randomly allocating ppt to groups is to use an online random number generator
What is effect size?
- The number of ppts needed depends on what is being studied
- How big of effect the IV has on the DV
i.e. the magnitude of the difference between groups
-> effect size
-> the bigger the difference, the bigger the effect size
Cohen (1988) provided guidelines on how to interpret effect sizes
- Typically, effect sizes are classified as small, medium and large
- Less ppts are required if your IV is likely to have a large effect on ppts
What is statistical power?
- Refers to the study’s capacity to correctly reject the null hypothesis
- Powerful experiments will be able to detect small effects, but experiments lacking power will have difficulty detecting even larger effects
-> if experiment doesn’t have enough statistical power, it won’t show that causal relationship we want to identify even if the IV is thought to have a large effect on the DV - Increasing the number of ppts, increases the power of the experiment
