NHST

Question 1

what is the p value used in null hypothesis significance testing

Answer 1

The first point

• something used to make decisions, typically.
• a criterion (.05) allows us to say whether something is significant or not
• if its larger than criterion then we typically retain the null
• if above .05 then we typically reject the null and say something is significant

A second point

under the assumption the null hypothesis is true, how likely are you to observe the data you observed?

p value tells you well, assuming nothing is going on and assuming that all the assumptions of ANOVA are held (we have homgeneity etc), the probability we would have observed this result is = whatever the p value is.

if the p value is .03 , tells us the probability of having observed this pattern of results across these conditions, under the assumption the null hypothesis is true, is .03. 3 out of 100 times we would expect to see a result like this just by chance.

if we have such a low probability - we reject the assumption that nothing is going on, I reject the null. this is what we’re saying when we say something is significant, we are rejecting the null

Question 2

what does the p value tell us in the end?

Answer 2

not much. what we’re interested in is the probability that this treatment will actually improve symptoms?

that’s what we want to know we don’t just want to know the probability of this happening by chance

so p-value is a probability, its something you compute that helps guide decisions about rejecting or retaining the null hypothesis

but there is a disconnect between what we would like to know from a given study and what the p-value tells us

Question 3

Wha is the posterior probability computed using Bayes formula?

Answer 3

the probability (like the p value) that we would like to have when we do an experiment and calculate our statistics rather than the p value. Tells us how likely the hypothesis of this model is correct.

very different to the p value

need the pvalue to calculate the posteiror probability, also need other elements (prior probability - have a prior for the null a prior for the alternative, also have the ptobbailty of having observed the data under the assumption the alternative was true, and the. pvalue - the probabiltiy of having observed the data under the assumption the null was true.)

Question 4

If the posterior probability is better than the p value why not use that

Answer 4

Because there are many instances where the its not feasible to calculate the posterior probability. Not always possible to compute it. Some of the reasons whya re given in the report (I think this refers to cohens paper – earth is round)

Sometimes you might just not have a prior probability for your alternative, or for the null

Question 5

What is p(D|HO) ?

Answer 5

Another notation for the p value. The probability of the data under the assumption the null hypothesis is true

Question 6

What is p(HO|D)

Answer 6

The probability of the null hypothesis given the data

Question 7

What is NHST

Answer 7

Comes back to the issue we calculate p values, we have a criterion that we use to reject or retain the null hypothesis. As well as the whole philosophy behind its actual value.

Question 8

What is Bayes theorem?

Answer 8

Tells us how to combine prior probabilities and likelihood (specific data observations) to obtain a posterior probability.

Question 9

What things can be used to improve statistical practice beyond the use of p values?

Answer 9

• replicating studies
• confidence intervals
• effect sizes
• figures - data visualisations of any sort - plots/graphs/tables
• use models - maybe you have a quantitive model where you have a prediction, theny ou compare your data to the predicted model. sometimes a model could be something as simple as i have a clear hypothesis of the direction of the difference or the magnitude of the difference