lecture 8 - within subjects t-test Flashcards
when do we use a within subjects t-test?
within-subjects tests of difference for two condition experiments with interval data
reporting the result
eg t(5) = 2.825, p=0.037
t = type of statistic
5 = degrees of freedom
2.825 = value of the statistic
p = the significance level
conceptualisation
- The fact that there is a mean difference of 5 between the conditions suggests that condition A yields higher scores, but maybe this is just noise variability?
- The idea is to show that it is unlikely that the true (“population”) mean difference is, in fact, zero.
- Specifically, if the true mean difference were zero (the null hypothesis), would the chance of our result occurring be smaller than say 5%?
We set a null hypothesis (e.g. “nothing is happening”) and use that to work out how likely the actual results are if that null hypothesis were true.
graphically
the null hypothesis (Ho) is that the true mean difference is zero if so the histogram of mean differences from many experiments would look like a normal distribution
the key thing is to precisely define this distribution -
1 - its mean is zero
2 - we assume the measured standard deviation is a good estimate
formal specification a within subjects t-test by hand
t = D-bar (mean difference)/ Sm (standard error of the differences)
how to find sample variability - the standard deviation of the differences
s= √∑ (D - D-bar)^2 / N - 1
d is used to for ‘differences”
N = no of data values
variability of the sample mean - the standard error of the difference
Sm = s / √N
then plug these values into the first equation
two variations of the t-test
Independent t-test: This test is used when you want to compare two means that come from conditions consisting of different entities (this is sometimes called the independent-measures or independent-means t-test).
Paired-samples t-test: This test, also known as the dependent t-test, is used when you want to compare two means that come from conditions consisting of the same or related entities
rationale for the t-test
Two samples of data are collected and the sample means calculated. These means might differ by either a little or a lot.
If the samples come from the same population, then we expect their means to be roughly equal (see Section 2.7). Although it is possible for their means to differ because of sampling variation, we would expect large differences between sample means to occur very infrequently. Under the null hypothesis we assume that the experimental manipulation has no effect on the participant’s behaviour: therefore, we expect means from two random samples to be very similar.
We compare the difference between the sample means that we collected to the difference between the sample means that we would expect to obtain (in the long run) if there were no effect (i.e., if the null hypothesis were true). We use the standard error (see Section 2.7) as a gauge of the variability between sample means. If the standard error is small, then we expect most samples to have very similar means. When the standard error is large, large differences in sample means are more likely. If the difference between the samples we have collected is larger than we would expect based on the standard error then one of two things has happened
There is no effect but sample means from our population fluctuate a lot and we happen to have collected two samples that produce very different means.
The two samples come from different populations, which is why they have different means, and this difference is, therefore, indicative of a genuine difference between the samples. In other words, the null hypothesis is unlikely.
The larger the observed difference between the sample means (relative to the standard error), the more likely it is that the second explanation is correct: that is, that the two sample means differ because of the different testing conditions imposed on each sample.
the t-statistic can be expressed as
t = observed difference between sample means - expected difference between population means (if null hypothesis is true) / estimate of the standard error of the difference between two sample means
the paired samples t-test equation explained
t= D-bar -µD / σ d-bar = d-bar/ σd-bar
This equation compares the mean difference between our samples (D-bar) to the difference that we would expect to find between population means (µD), relative to the standard error of the differences (σD-bar). If the null hypothesis is true, then we expect no difference between the population means and µD = 0 and it drops out of the equation
standard error of the means
The standard deviation of this sampling distribution is called the standard error of differences. Like any standard error (refresh your memory of Section 2.7 if you need to), a small standard error suggests that the difference between means of most pairs of samples will be very close to the population mean (in this case 0 if the null is true) and that substantial differences are very rare. A large standard error tells us that the difference between means of most pairs of samples can be quite variable: although the difference between means of most pairs of samples will still be centred around 0 substantial differences from zero are more common (than when the standard error is small). As such, the standard error is a good indicator of the size of the difference between sample means that we can expect from sampling variation. In other words, it’s a good baseline for what could reasonably happen if the conditions under which scores are collected are stable.
conditions under which scores are collected are not stable
In experiments, we systematically manipulate the conditions under which scores are collected. For example, to test whether looking like a human affects trust of robots, participants might have two interactions with a robot: in one the robot is concealed under clothes and realistic flesh, whereas in the other their natural titanium exoskeleton is visible. Each person’s trust score in the first interaction could be different from the second; the question is whether this difference is the product of how the robot looked, or just what you’d get if you test the same person twice. The standard error helps us to gauge this by giving us a scale of likely variability between samples. If the standard error is small then we know that even a modest difference between scores in the two conditions would be unlikely from two random samples. If the standard error is large then a modest difference between scores is plausible from two random samples. As such, the standard error of differences provides a scale of measurement for how plausible it is that an observed difference between sample means could be the product of taking two random samples from the same population.
how to calculate standard error of differences σD-bar
D-bar/ sD/√N
The standard error of differences is estimated from the standard deviation of differences within the sample (sD) divided by the square root of the sample size (N).
Therefore, t is a signal-to-noise ratio or the systematic variance compared to the unsystematic variance.
the independent t-test equation explained
when we want to compare scores that are independent
we compute the differences between the two sample means (X̄1 -X̄2) and not between individual pairs of scores. The difference between sample means is compared to the difference we would expect to get between the means of the two populations from which the samples come (µ1 − µ2)
t = (X̄1 -X̄2) - (µ1 − µ2)/ estimate of the standard error
If the null hypothesis is true then the samples have been drawn from populations that have the same mean. Therefore, under the null hypothesis µ1 = µ2, which means that µ1 − µ2 = 0, and so µ1 − µ2 drops out of the equation leaving us with
t = X̄1 -X̄2/ estimate of the standard error
standard error for the sampling distribution
It is, therefore, straightforward to estimate the standard error for the sampling distribution of each population by using the standard deviation (s) and size (N) for each sample
SE of sampling distribution of population 1 = s1/√N1
These values don’t tell us about the standard error for the sampling distribution of differences between means, though. To estimate that, we need to first convert these standard errors to variances by squaring them
variance of sampling distribution of population1
(s1/√n1)^2 = s1^2/n1
the variance of the sampling distribution of differences between two sample means will be equal to the sum of the variances of the two populations from which the samples were taken. This law means that we can estimate the variance of the sampling distribution of differences by adding together the variances of the sampling distributions of the two populations
variance of sampling distribution of differences = s1^2/n1 + s2^2/n2
We convert this variance back to a standard error by taking the square root
Se of sampling distribution of differences = √ s1^2/n1 + s2^2/n2
If we pop this equation for the standard error of differences into original t
equation
t = X̄1 -X̄2/ √ s1^2/n1 + s2^2/n2
this equation is true only when the sample sizes are equal, which in naturalistic studies may not be possible. To compare two groups that contain different numbers of participants we use a pooled variance estimate instead, which takes account of the difference in sample size by weighting the variance of each sample by a function of the size of sample on which it’s based
sp^2 = (n1-1) s1^2 +(n2 -1 s2^2 / n1 + n2 - 2
This weighting makes sense because (as we saw in Chapter 1) large samples more closely approximate the population than small ones; therefore, they should carry more weight. In fact, rather than weighting by the sample size, we weight by the sample size minus 1 (the degrees of freedom
The pooled variance estimate in the last equation is a weighted average: each variance is multiplied (weighted) by its degrees of freedom, and then we divide by the sum of weights (or sum of the two degrees of freedom) to get an average. The resulting weighted average variance is plopped into the equation for t
t = X̄1 -X̄2/ √ sp^2/n1 + sp^2/n2
One thing that might be apparent from equation (10.16) is that you don’t actually need any raw data to compute t; you just need the group means, standard deviations and sample sizes
assumptions of the t-test
Both the independent t-test and the paired-samples t-test are parametric tests and as such are prone to the sources of bias
For the paired-samples t-test the assumption of normality relates to the sampling distribution of the differences between scores, not the scores themselves
There are variants on these tests that overcome all of the potential problems
