Experimental Designs & Hypothesis Testing Flashcards
Main research methods
- Experimental research
- Correlational and descriptive research
Experimental research
- Manipulating a variable (IV) and measuring the outcome (DV)
- e.g. showing people violent pictures and measuring heart rate
- Advantages:
- Control over variables
- Can identify cause and effect relationships
- Disadvantages
- Sometimes unethical or impossible to manipulate a predictor e.g. illness
- Lack of ecological and external validity
Validity
- Is the study/measure accurately measuring what we think it is?
- Ecological validity
- External validity
Ecological validity
Does the experimental situation represent well enough the ‘real world’ situation we are interested in?
External validity
Do the findings from your sample generalise to other samples, the population, other times, other situations?
Non-experimental research: correlational and descriptive studies
- Cross-sectional designs
- Observing and measuring what happens in the world
- e.g. observing mother-child interactions or looking if personaliity factors are associated with certain behaviours
- Advantages:
- Ecological validity, less interference with real world events
- Can be used in situations where experiments would be unethical
- Disadvantages:
- Less control over variables - unmeasured variables may also be influencing data
- Can’t infer causality
Hypothesis testing
- A good hypothesis is both testable and falsifiable
- Step 1: formulate null hypothesis and alternative hypothesis
- Step 2: test to see if we can reject the null hypothesis
- If we can then we can accept the alternative/experimental hypothesis
- If the null hypothesis were true, what is the probability that data as extreme as our observed data would have occurred?
- The lower this probability is, the less likely that our findings are chance findings
- We reject the null hypothesis if the probability of us getting data as extreme as we have by chance is less than 5% p<.05
- Therefore it is said to reflect a real or statistically significant result
Inductive reasoning
Generalise from specific observations (bottom up)
observation → pattern → hypothesis → theory
Deductive reasoning
Making predictions about specific cases based on known facts. From general to specific (top down)
theory → hypothesis → observation → support for theory
Hume
- Probelm of induction
- Inductive reasoning generalises from specific observations but confirmatory observations are not evidence
- e.g. Will the sun rise tomorrow? Inductive reasoning would be the sun came up on Saturday, Sunday etc so therefore it will come up tomorrow
- Solution: deductive reasoning
- State a theory that the sun will rise everyday and if the sun does not rise, the theory would be falsified and needs to be replaced by another one
- Until the theory is falsified, there is no need to reject it
Karl Popper
- Falsifiability
- “One can sum up all this by saying that the criterion of the scientific status of a theory is its falsifiability, or refutability or testability”
Two-tailed test
Tests a non-directional hypothesis
e.g. driving ability (DV) is affected by the amount of alcohol consumed (IV)
One-tailed test
Tests directional hypothesis
e.g. Increased alcohol consumption (IV) decreases driving ability (DV)
Type I error
- Incorrectly rejecting the null hypothesis (false positive) i.e. believe our experimental manipulation has been successful but in fact it hasn’t
- E.g. find a difference between group scores on a variable when there isn’t really one
- Probability of Type I error is alpha .05 → 5% chance of this occurring
Type II error
- Incorrectly accepting the null hypothesis (false negative)
- I.e. we believe that our manipulation has had no effect on the dependent variable but in fact it has
- E.g. don’t find a difference between group scores on a variable when there actually is one
- Can be due to our method eg. because we have too few participants to reliably detect the effect
- Could be that our test is not powerful enough to detect the effect
Types of experimental designs
- Within-subject(s) designs (repeated measures)
- Between-groups designs (independent measures/between-subjects)
- Mixed designs - a mixture of both
Within-subject designs (repeated measures)
- Same participants: each participants takes part in each condition
- Conditions are compared to see if IV manipulation has had an effect on DV e.g. memory score (DV) before drug vs after drugs
- Advantages:
- Time and effort: fewer participants needed
- Participants are matched between conditions
- Same participants = less random variation between conditions
- Less random variation = more senstive to effect of manipulations (IV)
- Disadvantages
- Cannot use when a condition has an irreversible effect e.g. learning French via two methods
- Practice/fatigue/carryover effects
- Participants get better or worse/bored
- Therefore we need to counterbalance order of conditions across participants
Between-groups design (independent measures)
- Different participants
- Separate groups of participants for each condition
- Each participant tested once only
- Groups are compared to see if IV manipulations has had an effect on DV e.g. memory score (DV) drug vs no drug (control) group
- Advantages:
- No practice/fatigue/carryover effects: each participant just does experiment once
- Good when difficult to use repeated measures e.g. comparing methods to teach something - can only learn it once
- Sometimes just makes sense to use between-groups designs e.g. comparing depression levels of pensioners and adolescents
- Disadvantages:
- Expense, time and effort: need more participants, less data per participant
- Differences between groups: need to make sure randomly allocated to groups
- Within-group variation: manipulation has different effect within each group
Sources of bias
- Responder bias
- Demand characteristics/effects: participants believe they know what the experiment is about and manipulate their own responses to confirm or disconfirm the hypothesis
- Solution: always disguise the real hypothesis
- Mere exposure and placebo effects: familiarity alone can change attitudes to something; believing one has received treatment can lead to changes in health in clinical studies
- Solution: always compare effects to control groups, participants ‘blind’ to condition
- Demand characteristics/effects: participants believe they know what the experiment is about and manipulate their own responses to confirm or disconfirm the hypothesis
- Experimenter bias
- Experimenter effects: experiment knows hypothesis and (unconsciously) influences manipulation
- Solution: use ‘double-blind’ studies where possible - neither experimenter nor the participant knows which conditions they are in
- Experimenter effects: experiment knows hypothesis and (unconsciously) influences manipulation
Parsimonious explanation/theory
A theory that is as clear, concise and commonsensical as possible
Evidence-based
A theory/practice/idea should be based upon high quality evidence
Representativeness of sample
How typical the characteristics of a sample are of the population it is representing
Variable conceptualisation
Specific definition of a variable that a researcher is using
Subjectivity of measures
The extent to which a measure can be interpreted in different ways by different researchers or participants
Internal validity
The extent to which we can say that our manipulation of the IV, and only the manipulation of the IV, caused an observed effect
Generalisability of results
How well results generalise based on design factors such as validity and representativeness of sample etc.
