Stats and Research Methods

Question 1

What is the scientific method according to Karl Popper?

Answer 1

Science is being wrong on purpose

—> problem —> attempted solution —> elimination —>

Question 2

What is the scientific method according to David Spiegelhalter?

Answer 2

Science is being methodical

—> problem—> plan—>data—> analysis—> conclusion—>

Question 3

How are odds presented?

Answer 3

in fractions
e.g. chance ofgetting tails is 1/2

Question 4

How is probability presented?

Answer 4

in a percentage
e.g. probability of getting tails is 50%

Question 5

What are p-values?

Answer 5

P-values are the probability of getting our result if there’s no effect in the whole population

If the p-value is less than .05 then it is ‘significant’

Question 6

What are the problems with P-values?

Answer 6

• significance tells us nothing about importance
• p<.05 is arbitrary and encourages all or nothing thinking

Question 7

What do inferential statistics tell you?

Answer 7

tell you how likely it is that there’s no effect in reality

Question 8

What is an experiment?

Answer 8

• an experiment is defined by its use of randomisation
• this way we can be reasonably sure any differences between groups are because of our manipulation
• experiments are the gold standard in science because they let you infer causation

Question 9

Why is it hard to infer causation without an experiment?

Answer 9

The 3rd variable problem

• in non-experimental designs you look at events or groups that you did not create and try and infer their role in producing an outcome

Question 10

What is the difference between disinformation and misinformation?

Answer 10

disinformation = lying
misinformation = being wrong and telling people

Question 11

What is deductive reasoning?

Answer 11

• general to specific
• if the initial premises are true, the conclusion must be true
• logical arguments of this sort are intrinsically easy to disprove
• this trait is called “falsifiability”

Question 12

What is inductive reasoning?

Answer 12

• specific to general
• it is possible for initial premises to be absolutely true, but the conclusion false with this form of argument
• widely (and necessarily) used in science
• inductive arguments cannot be falsified so they are intrinsically less robust

Question 13

What are the pros and cons of deduction?

Answer 13

• major breakthroughs come from falsifying previously accepted theories
• rigorous
• but … rejects descriptive research that doesn’t try to falsify theories

Question 14

What are the pros and cons of induction?

Answer 14

• precise description of breakthroughs requires exploratory work where we try to generalise
• inferential statistics are inductive by nature
• but… without clear failure criteria science dissolves

Question 15

What does meta-analysis calculate?

Answer 15

meta-analysis calculates a composite effect by assigning more weight to powerful studies
- it treats studies with higher sample sizes as more informative, they are given a bigger weighting

Question 16

What is the ‘File Drawer problem’?

Answer 16

When “unwanted” results are less likely to be published, resulting in them ending up in someone’s “file drawer” rather than in a searchable form online

This leads to the literature being biased towards certain types of results

Question 17

Why might results be “unwanted”?

Answer 17

• non-significant p-values
• effects which contradict the “received wisdom”

Question 18

What are 4 things that can lead to biased evidence, making ‘evidence based dentistry’ not as simple as it seems?

Answer 18

• study publication bias
• outcome reporting bias
• spin
• citation bias

Question 19

What is study publication bias?

Answer 19

The file drawer problem

Less of the negative studies will be published than the positive ones

Question 20

What is outcome reporting bias?

Answer 20

When negative studies that are published focus more in the positives than the null results - trying to make it sound more positive

Question 21

What is spin?

Answer 21

When outcomes/results of studies are spun to make them sound better, not giving a fully accurate appraisal of what happened in detail

Question 22

What is citation bias?

Answer 22

Out of the published papers, more of the positive ones are cited in other literature compared to negative ones, making it seem like there is more evidence for the positive outcome

Question 23

What does a funnel plot show?

Answer 23

• if our results are unbiased, we expect equal numbers of studies on either side of the composite effect size
• we expect the large studies to be less biased so the graph should be broadly symmetrical at the top

Question 24

Are small or big studies more likely to be biased? Why?

Answer 24

Small studies more likely to be biased than the big studies, because they can disappear without a trace but big studies with lots of funding more likely to be published

Question 25

What phenomenon looks like publication bias, but isn’t?

Answer 25

the “small study effect”

Question 26

What is the “small study effect”?

Answer 26

When results appear to be becoming less significant, but it’s because the studies are getting bigger and bigger, and including participants that won’t show as much of an effect

e.g. in a small drug study the participants are those that are very ill, but as the study expands it starts to include people that may just be at risk of the disease

Question 27

What does positive predictive value show?

Answer 27

how many of the people that test positive that actually have the disease

Question 28

What does negative predictive value show?

Answer 28

how many of the people that test negative that are actually negative

Question 29

How do you calculate false positive rate?

Answer 29

false positive / false positive + true positive

Question 30

How do you calculate false negative rate?

Answer 30

false negative / false negative + true negative

Question 31

What are the 2 types of errors encountered in science?

Answer 31

Type 1 error - false positive
Type 2 error - false negative

Question 32

When does a type 1 error occur?

Answer 32

False positive
- when we think there is an effect when it reality there isn’t

(p-values (alpha level) are connected to the probability of making a type 1 error)

Question 33

When does a type 2 error occur?

Answer 33

False negative
- When we think there isn’t an effect when there really is

(connected to beta level, experiments are usually designed so that beta < 20%)

Question 34

What is statistical power?

Answer 34

the probability that you will find an effect where that effect actually exists
-> 1 - beta

Question 35

What does statistical power depend on?

Answer 35

• the effect size sought
• the p-value used as a criterion (typically .05)
• the type of statistical test that will be employed
• the sample size

Question 36

What is statistical power needed to calculate?

Answer 36

needed to calculate required sample sizes