Hypotheses

Question 1

contrast type 1 error vs type 2 error

Answer 1

• type 1 error = false positive: this is worse if:
  
  • the cost of a false alarm is high
  • treatment or followup testing is invasive, expensive or otherwise difficult
  • there is no effective treatment/delayed treatment is still effective
• type 2 error = false negatve–this is worse if:
  
  • the cost of a false alarm is low
  • treatment or followup testing is easy and readily available
  • the condition is treatment responsive only in the early stages

Question 2

describe a posteriori tests

Answer 2

• a posteriori tests are done after the data were collected, often for something else
  
  • example: researchers collect data on sleep quality but find nothing publishable: so they check for an association with lunar phase, find it’s significant at p=0.05 and publish the paper

Question 3

describe how to spot multiple-comparisons artifact

Answer 3

• suspect multiple-comparisons artifact if:
  
  • many comparisons were done, but p was still 0.05
  • a large sample was collected, but results are presented only for a subgroup
  • clinical plausibility: comparisons were done without a clear reason to suspect an association
  • research plausibility: association is not commonly studied (out of right field)
  • time lag: one or more of the variables presented can be collected long after the study is completed (e.g. lunar phase)