Topic 5: Two-Sample Hypothesis Testing Flashcards
Key features of Between subject designs (3)
- Each participant participates in one and only one group
- Comparisions made between the groups
- Random assignment used to assign participants to groups (if true experiment)
Two group designs do not always have to…… for example …..
- compare an experimental group to a control
- fpr example, it can compare two different amounts of an IV (comparision of length of prision centre such as 2 years and 4 years) or compare an “apple vs orange” such as how good people do depending on AM/PM lecture
Ex-Post-Factor (2)
- IV not manipulated (already occured)
- Does not allow cause and effect claims
One sample T test vs Independent sample t-test
- 1 sample: you find the sample distribution and standarize it
- Independent sample: look at the difference between the sample means and compare that diff to 0. *Standaridize the difference
How do we know if variances are equal?
Not equal if one variance is more than 3x the other
Three necessary conditions to use a t-test for independent samples:
- The samples must be independent
- Each population should have a normal distribution. However, this assumption is frequently violated with little harm as long as n>25
- Homogeneity of variance: Both groups must be sampled from populations with similar varince, if not do not pool variance, use adjusted df
Two-Sample t-test for the difference between Means steps:
- State the claim mathematically. Identify the Null and alternative hypotheses
- Specify the level of significance (identify alpha)
- Identify the degrees of freedom and sketch the sampling distribution (df= n1+n2 -2 or df= smaller of n1-1 or n2-1)
- Describe the critical values (t-table)
- Determine the rejection region(s)
- Find the standaridized test statistic with formula below
- Make a desicion to reject or fail to reject null
- interpret the desicion in the context of the original clai
Power (2)
- Probability of rejecting a false H0
probability that you will find difference thats really there - 1-beta
Beta
- The probability of making a type 2 error
1-B
Chance of finding an effect that exist
Influences on power+the disadvantages of each ways (5):
- Increase alpha value increases the ability to find an effect: raising alpha levbel increases our probability of a type 1 error
- One tail test increase power: not appropriate for a non-directional hypothesis, goes against convention as many do 2 tail test without specifying
- Decrease population variance (σ) which is impacted by population SD and sample size: Results are harder to generalize
- Increase sample size increases power as the sampling distribution for means are skinnier: may be hard/expensive to get alot of ppl
- Make your manipulation strong aka effect size: may not be ethical
Effect size + formula (2)
- Magnitude of true difference between null and alternative hypotheses (u1-u2)
- d= l (u1-u2)/σ l
Large effect size means that
your null and HA population dont overlap very much
Delta
Value used in referring to power tables that combines effect size and sample size
variance and SD relationship
SD is the value when variance is sqaure rooted
SD^2= Variance
Standard deviance symbol
σ
Variance symbol
σ2
What does this mean: Estimated power for delta= 1.5, alpha=0.05 and is roughly 0.32?
If the study were to run repeatedly, 32% of the time, the result would be statistically significant
Steps to find out what sample size we need to give us the power to detect a difference (3):
- Start with anticipated effect size (d)
- Determine delta required for desired level of power
- Calaculate n required for that value of delta
Importance of power when evaluating study results
- When a result is statistically significant: Effect size can tell us whether the result is practically significant
- When a result is not statistically significant: May not be that there is no difference, but simply that the study lacked the power to find it
Why does larger sample make distribution narrower when talking about impact of sample size on power
Since the standard deviation of the sampling distribution decreases with increasing n, the curve has a narrower, taller graph as more probability is squeezed toward the middle.
Matched pair (2)
- When participants are measured and equated on some variable and then assigned to each group based on that
- can be experimental or quasi-experimental (non-randomization)
Natural pairs (4)
What it is+ wjat kind of experiment+ ex+ advantage/disadvantage
- When participants are matched based on some biological or social relationship
- Typically quasi-experimental
- Ex: using siblings: may differ in personality but similar genectics and childhood experience.
- The primary advantage of the natural pairs design is that it uses a natural characteristic of the participants to reduce sources of error. The primary limitation of this design is often the availability of participants. The researcher must locate suitable pairs of participants (such as identical twins) and must obtain consent from both participants
Repeated Measures
Where the same participants are exposed to and measured in both conditions
Matched pair vs Natural pair
In a natural pairs design, scores were paired for some natural reason. In a matched pairs design, scores are
paired because the experimenter decides to match them on some variable.
Dependent (Paired) Samples design
- When we compare either the same participants or participants who have some type of predetermined relationship.
- Subjects in one group do provide information about subjects in other groups. The groups contain either the same set of subjects or different subjects that the analysts have paired meaningfully.
Within-subjects designs ex (3):
- matched pairs
- Natural pairs
- repeated measure
Dependent (paired): advantages (3)
- Greater control over the equality of groups
- more statistical power to find an effect: variability due to individual differeces decreased/nonexistent
- can often have smaller sample sizes
Which method to use if you dont have alot of people or you have alot of variability in a population?
Dependent (paired)
đ
the mean of the diffferences between each data pair in a paired t-test
d
The difference between each data pair:
d= x1-x2
The SD of the differences between the each data pair
Assumptions of the Paired t-test
- The samples must be dependent (paired)
- Both populations must be normally distributed
What are we comparing for a paired t-test?
We are comparing our average difference to a null hypothesis that the average difference between out two groups is 0.
t-test for the difference between paired means steps (9)
- State the claim mathematucally. Identify the null and alternative hypotheses
- Specify the level of signifcance
- Identify the degrees of freedom and sketch the sampling distribution [df= (number of pairs: n) -1]
- Determine the critical values (Use t-distribution)
- Determine the rejection regions(s)
- Calculate d bar and Sd
- Find the standardied test statistic of the t value with formula.
- Make a desicion to reject or fail to reject the null hypothesis. (depends if t-observed is in the rejection region or not)
- Interpret the desicion in the context of the original claim
We will use t-tests exclusively for tests of ——- and the z-test for tests of —–.
- tests of either independent or dependent group means
- proportions
t-tests can be used for comparing means of —— level variables
- interval and ratio
statistically significant
Is the difference large enough that we can reject the null hypothesis of no difference?
Z-tests can be used to compare proportions of —– level variables.
nominal and ordinal
When to use t and z test, 2 components:
The t-test is used when the population variance is unknown, or the sample size is small (n < 30). At the same time, the z-test is applied when the population variance (σ2) is known and the sample size is large (n > 30).
As the number of samples and/or the size of the samples increase, the standard error becomes….
smaller and smaller indicating less variation around the population mean.
The smaller the standard error, the more — the estimate of the population mean.
precise
The central limit theorem states that
the sampling distribution of the mean will always be normally distributed, as long as the sample size is large enough. Regardless of whether the population has a normal, Poisson, binomial, or any other distribution, the sampling distribution of the mean will be normal.
The probability associated with a difference between two means can be determined once …..
the difference is standardized as a t-statistic. A calculated t-value with an associated probability in the “rejection region” in either of the two tails of the distribution would provide evidence that there is a difference between the two means.
If we expect there will be a difference between the two means but are not predicting that one will be larger than the other, this would constitute a —
two-tailed test
As you learned previously, the t-distribution is —- whose shapes depend on —-
- a family of distributions
- their degrees of freedom (df)
degrees of freedom
refer to the number of values in a calculated statistic that are free to vary (do not have a fixed value).
Explain why its df-1 for degrees of freedom
If you are given the first three numbers in a distribution of four numbers as 2, 4, and 5, and then must identify the fourth number so that all four numbers will add to 20, the fourth number must be 9. The first three numbers can vary, but once they are determined, the last number is no longer free to vary. So we say that this distribution has n – 1 degrees of freedom, or df = 3.
Both methods, Z and T-test assume a ——-, but the z-tests are most useful when the —-
- normal distribution of the data
- standard deviation is known.
When do we use a pooled estimate of the standard error
For independent sample means, if one variance is no more than twice the other, they are considered approximately equal, and a pooled estimate of the standard error can be used.
sample variances
Using a pooled estimate when the variances are “too different” has been shown to lead to
unreliable results with the t-test (i.e., low p-values)
Figuring Degrees of Freedom – Approximately Equal Variances:
The p-value —- as the df decrease
increases
The variance that determine if we use a T test or Z test is the
variance for the population providing the samples
How to use the p-value method approach for independent sample means
- calculate the p-value of t and compare that to the probability limit set by alpha
- Ex: the p-value associated with a t-statistic of 10.00 and df = 48 is P < .00001. Since the P-value is less than the alpha limit of .05, the null is rejected.
The value for d tells us
the mean difference between the two groups is equivalent to x.xx standard deviations
