false positives

Question 1

what is a type 1 error

Answer 1

• False positives
• alpha inflation
• multiple stats tests & comparisons
• reject the null when it is true
• there isn’t a difference it was just by chance
• statistically significant effect that we will obtain a p value p<.05

Question 2

what are statistical errors

Answer 2

They are what happens when we either accept or reject the null hypothesis and whether that is right to do so. How the either accepting or rejecting can impact our conclusions and results

Question 3

what are type 2 errors

Answer 3

• accept the null when there is actually a relationship
• false negative

Question 4

what is the importance of type 1 errors

Answer 4

• drawing an incorrect conclusion about something
• if there is a 5% chance of something being false some data is going to be incorrect

Question 5

what is publication bias

Answer 5

• statistically significant results are more likely to be published
• the alt hypothesis is the one that we typically find more interesting
• if you are unable to find evidence for the hypothesis we less of the paper and less likely to be accepted
• file drawer problem
• if a researcher fails to find a particular results after a replication they’re less likely to publish

Question 6

what are researchers degrees of freedom

Answer 6

• have the ability to analyse their data many time in many ways
• increase the chances that they are going to find a significant result
• increase chance of reporting a result that isn’t true

Question 7

why is there a lack of transparency

Answer 7

• may not be giving us all of the relevant info about their study design
• not explained it or how they collected and analysed data

Question 8

what is human error

Answer 8

• humans make mistakes
• findings can’t always be replicated due to an error down the line that wasn’t spotted
• mistaken inaccuracy in the data somewhere

Question 9

what is transparency

Answer 9

• often trying to figure out what hasn’t been said
• what was relevant but now has been omitted
• don’t have all the info we need to analyse data so can we actually trust the results

Question 10

what is the likelihood of getting a false positive

Answer 10

• ā = 1 - (1 - α per comparison)m
• alpha = .05
• m = the no. of comparison
• conducting multiple comparisons without adjustment is a problem (can lead to)

Question 11

what is the importance of having multiple comparisons

Answer 11

• if a researcher collects lots of data it’s data mining
• even if your first analysis didn’t show what you wanted another analysis most likely can
• HARKing
• more trustworthy studies are based on previously established evidence

Question 12

what are the guidelines for authors

Answer 12

• must decided the termination of data collection before data is collected
• 20 observations per cell or cost-of-data-collection justification
• list all variables
• report all experimental conditions even if failed
• must show data with removed data

Question 13

what are the guidelines for reviewers

Answer 13

• ensure authors follow all requirements
• tolerant of imperfections in results
• data shouldn’t hinge on arbitrary and analytical decisions
• if justification or analysis aren’t compelling authors can be instructed to do an exact replication