Hypothesis testing Flashcards
Familiarise with significance tests etc
What are the three central features of significance testing?
1) Assume a normally distributed range of error around an event
2) Assess likelihood of a particular variant of this event accounting for the normal range of error
3) Decide that the likelihood of that event under these circumstances is far too low and therefore must be significant
How would we go about calculating the difference between two means to determine the presence/absence of a significant effect?
We would calculate a TEST STATISTIC i.e. a z-score
This is standardised so we are able to know its exact distribution under the null hypothesis, and through the distribution curve we are also able to calculate the probability of that test statistic given the null hypothesis is true - probability is area under the curve and this is found using reference tables
RECAP: How is the Test Statistic calculated?
Need to calculate SE first
Then z = (sample mean - population mean (i.e. mean under null hypothesis))/SE
We then look the resulting value up in the tables and we get the probability of that test statistic assuming null hypothesis to be true
What would we say if a sample mean falls below the lower boundary of a 95% interval?
It is in the CRITICAL REGION under the null hypothesis and the sample is SIGNIFICANTLY DIFFERENT from the population
What is a uni-directional hypothesis?
What are the advantages and disadvantages of this type of hypothesis?
Predict difference and state direction (One-tailed test)
ADVANTAGE - Value of z-score doesn’t have to be as large to reject null hypothesis (remember we are interested in the most extreme 5% of scores and in a one-tailed test this is all at one end)
DISADVANTAGE - if direction of difference is opposite to what was predicted, null hypothesis cannot be rejected
What is a bi-directional hypothesis?
What are the advantages and disadvantages of this type of hypothesis?
Predict difference but do not state direction (Two-tailed test)
ADVANTAGE - We can reject the null if an effect is significant in either direction
DISADVANTAGE - The z-score has to be a larger value in order to reject the null as the extreme 5% is now halved and is 2.5% either end of the distribution so to be beyond the critical point the value for z is going to need to be higher
What is a Type 1 error?
Finding an effect which isn’t actually there i.e. rejecting null hypothesis even when it is true
What is a Type 2 error?
Missing an effect that actually is there i.e. accepting null hypothesis when it is actually false
Why is effect size important?
The relative size of an effect is more useful than its absolute size
What are the 3 main measures of effect size that can be used?
1) Cohen’s d
2) Pearson’s correlation coefficient, r
3) Odds ratio
When would we want to calculate Cohen’s d and how can we use it?
Comparing 2 means - the d value is the difference in the means of 2 groups divided by the average of their standard deviations (so if we see d=1, we know that the 2 means differed by 1 SD)
Small effect = 0.2
Medium effect = 0.5
Large effect = 0.8
When would we use Pearson’s correlation coefficient and why is it useful?
Covers whole strength of relationship range, from none at all (0) to perfect (1)
Tells us how large a relationship between two variables actually is, INDEPENDENT OF SAMPLE SIZE Small effect = 0.1 (i.e. effect measured explains 1% of total variance)
Medium = 0.3 (i.e. effect measured explains 9% of variance, r squared)
Large = 0.5 (i.e. effect measured explains 25% of total variance
Why would the odds ratio be used?
Measures how many times bigger the odds of one outcome is for one value of an IV compared to another
Essentially measures association between exposure and outcome
To convert from odds ratio to effect size, simply divide odds ratio by 1.81
Define what a z-score actually is
The number of standard deviations a score is from the mean
What is meant by “Sampling Distribution”? What do such distributions look like?
Distribution of means of samples rather than individual values
“The collection of means of all possible random samples, size N, from a population, where the mean of the distribution is estimated as the sample mean”
Approaches a normal distribution as sample size increases
