Uncertainty Management

Question 1

What are the two types of uncertainty?

Answer 1

• methodological

- statistical

Question 2

What is methodological uncertainty?

Answer 2

• related to confidence that our research design and procedures allow us to answer the research question
• usually managed by trying to reduce it
• in doing so have to consider internal and external sources of uncertainty

Question 3

What are sources of methodological uncertainty?

Answer 3

• internal uncertainty is produced by lack of control and is reduced by experimental control
• external uncertainty is produced by sampling error or naive empiricism and is reduced by appropriate sampling or theory development
• integrity uncertainty is produced by questionable research practices and scientific fraud and is reduced by open science practices, including preregistration

Question 4

Internal vs External uncertainty?

Answer 4

• methods for managing uncertainty are inter-related and not all can be achieved in single study
• appropriately designed experiments address the problem but may leave residual external uncertainty about ability to generalise findings to real world
• due to this most attempt to tackle same problem using range of different methodological strategies

Question 5

What are methods to reduce methodological uncertainty?

Answer 5

• choosing appropriate method
• eliminating confounds
• controlling for extraneous variables
• controlling threats to internal validity
• using representative samples

Question 6

What are the two types of statistical uncertainty?

Answer 6

• descriptive

- inferential

Question 7

Definition of descriptive uncertainty?

Answer 7

• not all Ps provide exactly same score, variability in figures
• larger standard deviation in leaders groups, more variation amongst leaders than followers
• sources: inter-individual differences (variation in responses between individuals), intra-individual differences (variation in responses within individuals), measurement error (variation in responses due to inconsistent measurement)

Question 8

Definition of inferential uncertainty?

Answer 8

• results may have happened by chance without experimental effect
• want to be able to make inferences about populations on basis of sample data
• affected by: signal (sample behaviour), noise (random variance), sample size
• all contribute to p-value
• overlap in distribution of scores, any test has to take overlap into account
• higher the p-value the more uncertainty there is about effect being genuine
• goal is to usually measure it, not reduce it

Question 9

How should uncertainty be managed?

Answer 9

• strategic decisions about the management of uncertainty always involves trade-offs and compromises
• being more certain about some things increases the uncertainty in others

Question 10

Should psychological research strive to minimise methodological and statistical uncertainty?

Answer 10

• it shouldn’t
• there’s many forms of uncertainty that are relevant to psychological research
• argued that progress in research is often achieved by creating uncertainty rather than reducing it