Experimental Designs & Hypothesis Testing

Question 1

Main research methods

Answer 1

• Experimental research
• Correlational and descriptive research

Question 2

Experimental research

Answer 2

• Manipulating a variable (IV) and measuring the outcome (DV)
  
  • e.g. showing people violent pictures and measuring heart rate
• Advantages:
  
  • Control over variables
  • Can identify cause and effect relationships
• Disadvantages
  
  • Sometimes unethical or impossible to manipulate a predictor e.g. illness
  • Lack of ecological and external validity

Question 3

Validity

Answer 3

• Is the study/measure accurately measuring what we think it is?
• Ecological validity
• External validity

Question 4

Ecological validity

Answer 4

Does the experimental situation represent well enough the ‘real world’ situation we are interested in?

Question 5

External validity

Answer 5

Do the findings from your sample generalise to other samples, the population, other times, other situations?

Question 6

Non-experimental research: correlational and descriptive studies

Answer 6

• Cross-sectional designs
• Observing and measuring what happens in the world
• e.g. observing mother-child interactions or looking if personaliity factors are associated with certain behaviours
• Advantages:
  
  • Ecological validity, less interference with real world events
  • Can be used in situations where experiments would be unethical
• Disadvantages:
  
  • Less control over variables - unmeasured variables may also be influencing data
  • Can’t infer causality

Question 7

Hypothesis testing

Answer 7

• A good hypothesis is both testable and falsifiable
• Step 1: formulate null hypothesis and alternative hypothesis
• Step 2: test to see if we can reject the null hypothesis
  
  • If we can then we can accept the alternative/experimental hypothesis
  • If the null hypothesis were true, what is the probability that data as extreme as our observed data would have occurred?
    
    • The lower this probability is, the less likely that our findings are chance findings
    • We reject the null hypothesis if the probability of us getting data as extreme as we have by chance is less than 5% p<.05
    • Therefore it is said to reflect a real or statistically significant result

Question 8

Inductive reasoning

Answer 8

Generalise from specific observations (bottom up)
observation → pattern → hypothesis → theory

Question 9

Deductive reasoning

Answer 9

Making predictions about specific cases based on known facts. From general to specific (top down)

theory → hypothesis → observation → support for theory

Question 10

Hume

Answer 10

• Probelm of induction
• Inductive reasoning generalises from specific observations but confirmatory observations are not evidence
  
  • e.g. Will the sun rise tomorrow? Inductive reasoning would be the sun came up on Saturday, Sunday etc so therefore it will come up tomorrow
• Solution: deductive reasoning
  
  • State a theory that the sun will rise everyday and if the sun does not rise, the theory would be falsified and needs to be replaced by another one
  • Until the theory is falsified, there is no need to reject it

Question 11

Karl Popper

Answer 11

• Falsifiability
• “One can sum up all this by saying that the criterion of the scientific status of a theory is its falsifiability, or refutability or testability”

Question 12

Two-tailed test

Answer 12

Tests a non-directional hypothesis

e.g. driving ability (DV) is affected by the amount of alcohol consumed (IV)

Question 13

One-tailed test

Answer 13

Tests directional hypothesis

e.g. Increased alcohol consumption (IV) decreases driving ability (DV)

Question 14

Type I error

Answer 14

• Incorrectly rejecting the null hypothesis (false positive) i.e. believe our experimental manipulation has been successful but in fact it hasn’t
• E.g. find a difference between group scores on a variable when there isn’t really one
• Probability of Type I error is alpha .05 → 5% chance of this occurring

Question 15

Type II error

Answer 15

• Incorrectly accepting the null hypothesis (false negative)
• I.e. we believe that our manipulation has had no effect on the dependent variable but in fact it has
• E.g. don’t find a difference between group scores on a variable when there actually is one
• Can be due to our method eg. because we have too few participants to reliably detect the effect
• Could be that our test is not powerful enough to detect the effect

Question 16

Types of experimental designs

Answer 16

• Within-subject(s) designs (repeated measures)
• Between-groups designs (independent measures/between-subjects)
• Mixed designs - a mixture of both

Question 17

Within-subject designs (repeated measures)

Answer 17

• Same participants: each participants takes part in each condition
• Conditions are compared to see if IV manipulation has had an effect on DV e.g. memory score (DV) before drug vs after drugs
• Advantages:
  
  • Time and effort: fewer participants needed
  • Participants are matched between conditions
    
    • Same participants = less random variation between conditions
    • Less random variation = more senstive to effect of manipulations (IV)
• Disadvantages
  
  • Cannot use when a condition has an irreversible effect e.g. learning French via two methods
  • Practice/fatigue/carryover effects
    
    • Participants get better or worse/bored
    • Therefore we need to counterbalance order of conditions across participants

Question 18

Between-groups design (independent measures)

Answer 18

• Different participants
• Separate groups of participants for each condition
• Each participant tested once only
• Groups are compared to see if IV manipulations has had an effect on DV e.g. memory score (DV) drug vs no drug (control) group
• Advantages:
  
  • No practice/fatigue/carryover effects: each participant just does experiment once
  • Good when difficult to use repeated measures e.g. comparing methods to teach something - can only learn it once
  • Sometimes just makes sense to use between-groups designs e.g. comparing depression levels of pensioners and adolescents
• Disadvantages:
  
  • Expense, time and effort: need more participants, less data per participant
  • Differences between groups: need to make sure randomly allocated to groups
  • Within-group variation: manipulation has different effect within each group

Question 19

Sources of bias

Answer 19

• Responder bias
  
  • Demand characteristics/effects: participants believe they know what the experiment is about and manipulate their own responses to confirm or disconfirm the hypothesis
    
    • Solution: always disguise the real hypothesis
  • Mere exposure and placebo effects: familiarity alone can change attitudes to something; believing one has received treatment can lead to changes in health in clinical studies
    
    • Solution: always compare effects to control groups, participants ‘blind’ to condition
• Experimenter bias
  
  • Experimenter effects: experiment knows hypothesis and (unconsciously) influences manipulation
    
    • Solution: use ‘double-blind’ studies where possible - neither experimenter nor the participant knows which conditions they are in

Question 20

Parsimonious explanation/theory

Answer 20

A theory that is as clear, concise and commonsensical as possible

Question 21

Evidence-based

Answer 21

A theory/practice/idea should be based upon high quality evidence

Question 22

Representativeness of sample

Answer 22

How typical the characteristics of a sample are of the population it is representing

Question 23

Variable conceptualisation

Answer 23

Specific definition of a variable that a researcher is using

Question 24

Subjectivity of measures

Answer 24

The extent to which a measure can be interpreted in different ways by different researchers or participants

Question 25

Internal validity

Answer 25

The extent to which we can say that our manipulation of the IV, and only the manipulation of the IV, caused an observed effect

Question 26

Generalisability of results

Answer 26

How well results generalise based on design factors such as validity and representativeness of sample etc.