Long Answer Practice Questions Flashcards
Consider the formula for the independent samples t-test. Discuss what properties of a dataset contribute to the size of an obtained t-test statistic. For each property, discuss how that property influences the size of the t-value and why it conceptually makes sense that the property should have the impact it does on judging the plausibility of the null hypothesis.
There are 3 properties of a dataset that contribute to the size of an obtained t-test statistic: sample size, effect size, and standard error.
As sample size increases, so does precision. Power will also increase and the probability of making a type II error will decrease. It makes sense that as sample size increases, the absolute value of the t-test statistic will increase because the sample error becomes small enough that we are able to detect the most minuscule deviations from the null.
As effect size increases (and the means become farther apart), t-value also increases. This makes sense because it is apparent that the independent variable is having an effect, hinting that we can discredit the null hypothesis.
Standard error measures how much difference is expected by chance and is inversely related to sample size. It intuitively makes sense that as sample size increases, standard error will decrease, and t-score will increase, because the means become less spread out and therefore it becomes more likely that any of the given means is an accurate representation of the true sample mean.
Consider the formula for the ANOVA F-ratio. Discuss what sources of variance contribute to the size of an obtained F-ratio. For each source, discuss how that source influences the size of the F-ratio and why it conceptually makes sense that the source should have the impact it does on judging the plausibility of the null hypothesis.
Both systematic and unsystematic variance contribute to the size of an obtained F-ratio. More specifically, treatment effects, individual differences, and experimental error. The larger the variance associated with the treatment effect, the larger the F-ratio. Smaller variances associated with unsystematic variances (individual differences and experimental error) also yield a larger F-ratio.
This intuitively makes sense when judging the plausibility of the null hypothesis because when the F-ratio is small, and unsystematic variance is large, the difference that exists is due to chance. However, when the F-ratio is large, and systematic variance is large, the difference that exists is likely due to treatment effect.
Define alpha, beta, and power. Then describe the relationship that exists between all three.
Alpha is typically set to 5% this is the chance of type I error we are willing to accept (type I error is concluding a difference exists when in fact no difference does exist).
Beta is the likelihood of committing a type II error (type II error is concluding a difference does not exist when in fact a difference does exist).
Power + beta = 1, so the higher the power the lower the likelihood of committing a type II error. Researchers typically aim for a power of .80 (80%), meaning that researchers are willing to accept a .20 (20%) chance of committing a type II error. Power is the likelihood of a test finding an existing effect.
Why does a paired samples t-test have more power than an independent samples t-test?
When subjects are measured repeatedly over a period of time, unsystematic variation that may exist within two independent samples (i.e., individual differences) is eliminated. This makes it easier for researchers to identify systematic variation caused by differences in the independent variable.
Consider the formulas for Cohen’s d. Discuss what properties of a dataset contribute to the size of an obtained Cohen’s d. For each property, discuss how that property influences the size of Cohen’s d and why conceptually it makes sense that the property should have the impact that it does.
The mean difference and standard deviation are two properties that influence the effect size.
It makes sense that when the means are farther apart from each other, that the effect size would increase.
With regard to standard deviation, the smaller the standard deviation the larger the effect. This makes sense because a smaller standard deviation is more reliable as the data clusters more closely around the mean.
List the factors that impact power and describe how exactly these factors have the impact that they do.
Alpha, sample size, and effect size all influence power.
Alpha positively correlates with power. As alpha decreases so does power, less likely that we will commit the type I error and find a difference.
A smaller sample size is less representative of the population and thus power decreases and vice versa.
The larger the effect size the higher the power because this means that the independent variable has a greater effect or impact.
Explain how sampling error influences small and large samples with regard to NHST.
Sampling error occurs when the sample means are slightly different even though the population has no difference.
In small samples, NHST will only assess a difference as unlikely (not due to chance) if it is substantial since modest differences often occur in small samples with high sampling error.
In large samples, NHST will assess a difference as unlikely even if it is only moderate since sampling error is low in this situation.
