Weeks 9 and 10 Flashcards
The explanation for the solutions for some questions in this section refer to the diagram above. In this diagram, the sampling distribution associated with a hypothesized distribution is shown at the top left and the sampling distribution for the true population is shown at bottom right
Power is the probability of rejecting a hypothesis when it is false.
a) true
b) false
1st line graph shows a bell curv with the alpha in the + side of the curve
2nd line graph shows a bell curve with the b/power on the = side, the majority of which is above the line (1-B)
a) True
Explanation … This is the definition of the power of a test. The more power you have, the better able you are to reject a false hypothesis.
If you have 20% power in an experimental design, this mean that you will be unable to reject the null hypothesis even when it is false.
a) true
b) false
b) False
Explanation … Suppose that you have a treatment which is effective and has a Cohen’s d value that is truly nonzero. The null hypothesis the d = 0 is therefore false. If a design has 20% power under these circumstances, it means that if you gather 100 datasets and conduct 100 t-tests, you will successfully reject the null hypothesis 20 times but would be unsuccessful in rejecting it the other 80 times.
The output of an a priori power calculation is the number of observations per sample needed for you to achieve a specified power.
a) true
b) false
a) True
Explanation … In a post hoc power calculation you specify a number of properties of an experimental design in order to find its power. “Post hoc” is Latin for “after the fact” and this term is used because, frequently, the properties you specify are the properties of a study you have just undertaken. In that case the power you get as the output is the statistical power of a t-test you have just conducted. In contrast, in an a priori power calculation you specify the power you want to attain in an experiment you have not yet undertaken. “A priori” is Latin for “what is before” and refers to the idea that this calculation is usually performed before you undertake a study. The output an a priori power calculation tells you how many participants you need to recruit to reach a desired power and this is something you often need to figure out before you actually begin taking data in a study.
The only time you can make a Type II error is when you fail to reject a hypothesis.
a) true
b) false
a) True
Explanation … In the diagram at the top of this section, the vertical dashed line indicates the criterion for rejecting or failing to reject hypotheses. If a sample mean is to the right of the line, reject the hypothesis, if it is to the left of the line, fail to reject. The area associated with Type II errors is indicated in the diagram by B. Since this area is to the left of the vertical line, you can see that Type II errors happen only when you fail to reject a hypothesis.
Decreasing the allowed rate of Type I errors increases the power of a test.
a) true
b) false
b) false
Explanation … In the diagram, the sampling distribution at the top left belongs to the hypothesized population. Decreasing the allowed rate of Type I errors for this sampling distribution is the same as decreasing alpha, and this corresponds to moving the criterion for rejection (the dashed vertical line in the diagram) to the right. But this has consequences in the true sampling distribution shown at the bottom right. You can see that moving the dashed line to the right will increase the rate of Type II errors (B) and decrease 1-B, the power. There is therefore a tug-of-war between Type I and Type II errors. It is thus possible to decrease the rate of Type I errors but this also decreases power.
When are you susceptible to criticism that you do not have enough power in a t-test?
a) when the t-value is large
b) when you accept the null hypothesis
c) when you use a 1-tailed test
d) when the value of your test-statistic is in the rejection region of its distribution
b) when you accept the null hypothesis
Explanation … If you accept the null hypothesis then there is always the possibility that you have done so in error. In other words you have committed a type II error. Power addresses this issue, it tells you the probability (beta) that you have committed a Type II error and a low power is associated with a high probability of such an error. If you have already succeeded in rejecting the null hypothesis then you cannot have committed a Type II error and you do not need to worry about power. Answers a) to c) are all situations which lead to rejection of the null hypothesis and so, in these situations, there are no concerns about having adequate power in the test.
7) When would you have the best chance of demonstrating that there really is a difference due to a treatment?
a) when the sampling distribution of the mean is narrow
b) when the sampling distribution of the mean is wide
c) when the hypothesized mean is close to the true mean
d) when the sample size is small
a) when the sampling distribution of the mean is narrow
Explanation … In this question all the answers except the correct one are associated with a small power. But the power of a test is really your chance of demonstrating that a treatment is truly effective because it is the probability of rejecting the null hypothesis when it deserved to be rejected. Thus the correct answer is “When the sampling distribution of the mean is narrow” … the only one associated with a large power.
8) For a given N and Cohens d, an independent-samples t-test is has more statistical power than a 1-sample t-test.
a) true
b) false
b) false
Explanation … A 1-sample t-test can be up to twice as powerful as a 2-sample t-test. This is because in a 2-sample t-test you devote all the observations in one sample defining the characteristics of the control population whereas in a 1-sample t-test you don’t need to do this and can use those observations to better define the treatment population. An example where you would know the characteristics of the control population ahead of time would be standardized T-scores (such as Hostility T-scores) where, based on much prior research, it is known ahead of time that the population of psychologically normal adults should score with a mean of 50 and a standard deviation of 10.
The power of a matched-pairs t-test depends on the size of the correlation coefficient between 2 samples.
a) true
b) false
a) True
Explanation … The value of the noncentrality parameter a matched-pairs t-test can be written
ds = d (square root of n/(2(1-r))).
This says that, for a given effect size between 2 samples, the closer the correlation the 2 samples is to a value of 1, the larger the ds-value will be. But this, in turn, means that the power is high. Thus power depends of the correlation between the samples
Usually, a paired-samples t-test is best for analyzing a repeated-measures design.
a) true
b) false
Explanation … If 2 samples are correlated, a paired-samples t-test is more powerful than an independent-samples t-test and is therefore preferred. A repeated-measures design often produces such a correlation because individuals who tend to score lower than others on the first measurement of some variable tend to score lower than others on the second measurement too. This is the same as saying that there are stable individual differences between the participants in the design and this is something the paired-samples analysis can take advantage of.
3) A paired-samples t-test is really a single-sample t-test carried out on pair differences.
a) true
b) false
a) True
Explanation … A paired-samples takes 2 samples and uses the pairing to create a single sample of pair differences. The actual test is then carried out on this new single sample of difference scores. So although it starts out as a 2-sample test, a matched-pairs test ends up as a single-sample test carried out on pair-differences.
4) There are n-1 degrees of freedom in a paired-samples t-test.
a) true
b) false
a) True
Explanation … Since a paired-samples t-test is really a single-sample test carried out on pair differences, and since there are n pair differences in a design with N observations, the degrees of freedom in the test is df = n - 1.
5) Cohen’s effect size for a paired samples design is
d = Md/Sd.
a) true
b) false
b) False
Explanation … Cohen’s effect size for a paired-samples design is . We use spooled in the denominator because this is fundamentally a 2-sample design and an effect size should keep track of just the effect of the treatment on the dependent variable and not arbitrary choices by a researcher on whether to use a large or small sample size, a 1-or 2 tail test, or an independent-samples t-test or a paired-samples t-test in the analysis of the data.
Suppose that in a paired-samples design the value of the mean of pair differences is found to be MD = 2 and the standard deviation of pair differences is found to be sD = 4. What is the t-value for a paired samples t-test if there are 9 participants in the design?
a) 0.5
b) 1
c) 1.5
d) 2
c) 1.5
Explanation … There are 2 main ways to calculate a t-value for a paired samples t-test. One is via a knowledge of the properties of the difference scores and the other is to rely on properties (such as spooled) of the original samples and the correlation between them. In this question you are given the properties of the difference scores so you should use the formula t=M_D/(s_D⁄√n) . Plugging values from the question into this formula gives t=M_D/(s_D⁄√n) =2/(4⁄√9)=6/4=1.5
7) Suppose that in a paired-samples design the difference between the means of the 2 samples is M1 – M2 = 4 and the pooled standard deviation for those samples is spooled = 2, and the samples are correlated with correlation coefficient r = 0.5. What is the t-value in a paired samples t-test if there are 16 participants in the design?
a) 8
b) 10
c) 16
d) 20
a) 8
Explanation … There are 2 main ways to calculate a t-value for a paired samples t-test. The appropriate formula when you know the properties of the 2 samples (rather than the properties of the difference scores) is t=d√(n/2(1-r) ) . This formula involve Cohen’s effect size but that is easy to calculate from the values given in the question … d= (M_1-M_2)/s_pooled =4/2= 2 . Plugging this result and the other values from the question into the formula for the t-value gives values from the question into this formula gives t=d√(n/2(1-r) ) = 2√(16/2(1-0.5) )= 2√(16/1)=2×4=8
