Week 4: T-Tests and ANOVA

Question 1

What is the purpose of a t-test?

Answer 1

To determine if there is a significant difference between the means of two groups or across time for one group

Question 2

What is ANOVA used for?

Answer 2

To compare means across more than two groups to determine if at least one group differs significantly

Question 3

What is the command to compare the sample mean of a variable to a specific population value?

Answer 3

<ttest varname==x>

Question 4

When do you use a one-sample t-test?

Answer 4

When comparing the sample mean to a known value from the general population

Question 5

What is the independent t-test used for?

Answer 5

To compare means between two independent groups

Question 6

What are the assumptions of the independent t-test?

Answer 6

Normally distributed data, random sampling, homogeneity of variance, and a continuous dependent variable

Question 7

When do you use a paired t-test?

Answer 7

When comparing means from the same group at two time points or under two conditions

Question 8

What is H0 in a paired t-test?

Answer 8

The mean difference between the paired observations is zero

Question 9

What does ANOVA test do?

Answer 9

Whether there are significant differences in means among three or more groups on one continuous variable.

Question 10

What is a post-hoc test in ANOVA?

Answer 10

A follow-up test (e.g., Tukey’s HSD) to identify which groups differ after a significant ANOVA result.
Command: <pwmean dependent_variable, over(independent_variable) mcompare(tukey) effects>

Question 11

What is the non-parametric equivalent of the independent t-test?

Answer 11

Mann-Whitney U test (also called Mann-Whitney-Wilcoxon test or Wilcoxon rank-sum test)

Question 12

When is the Kruskal-Wallis test used?

Answer 12

As a non-parametric alternative to one-way ANOVA when data do not meet normality assumptions

Question 13

How do you test for normality?

Answer 13

Use <swilk> for Shapiro-Wilk test and <sktest> for skewness and kurtosis test</sktest></swilk>

Question 14

How do you conduct an independent t-test?

Answer 14

<ttest dependent_variable, by(independent variable)>

Question 15

What command checks homogeneity of variance?

Answer 15

<sdtest variable, by(group)> or <robvar variable, by(group)>
Note: We need to assess if there is enough evidence to reject the H0 of equal variances. The H1 of interest is that the variances in the two groups are not equal (Ha: ratio ! = 1 - if the variances were equal, their ratio would be equal to 1. If p > 0.05, there is evidence to suggest the variances are equal)

Question 16

How do you perform a one-way ANOVA?

Answer 16

<oneway dependent_variable independent _variable, tabulate>

Question 17

What is the H0 for comparing means between groups?

Answer 17

There is no significant difference in the means between groups

Question 18

Is a hypothesis comparing two groups typically one- or two-tailed?

Answer 18

Two-tailed, unless there is a specific directional expectation

Question 19

How do you interpret a p-value in hypothesis testing?

Answer 19

A p-value < 0.05 typically indicates significant evidence against H0

Question 20

What does a significant Levene’s test indicate?

Answer 20

Variances between groups are not equal

Question 21

What is the alternative if normality assumptions are violated?

Answer 21

Use non-parametric tests like Mann-Whitney U or Kruskal-Wallis

Question 22

How do you interpret the 95% CI of a mean executive function score of 17.3 among older English people?

Answer 22

Command: <ci>
Output: 95% CI 17.14, 17.50
We can be 95% certain that the true population mean of executive function among older English people is between 17.14 and 17.50.</ci>

Question 23

What is included in the output for a one-sample t-test?

Answer 23

• H0
• Student t-statistic
• Df
• Three alternative hypothesis: The mean is lower than μ; The mean is NOT equal to μ; The mean is greater than μ
• P-values for each alternative hypothesis

Question 24

How do you a one-sample t-test for a specific sub-group?

Answer 24

Example: Executive function among older English people (aged 60+) among those who report no vigorous physical activities
1. Find the mean of executive function among adults aged 60+ <mean>=60 (mean = 16.31)
2. Use the mean in a one-sample t-test to assess whether the mean executive function of those reporting no vigorous physical activities differs significantly from the whole ELSA sample
3. <ttest cfex=16.31 if dhager>=60 & heacta==4></mean>

Question 25

How do you interpret results from the Shapiro-Wilk test?

Answer 25

H0 = the variable is normally distributed
If p < 0.05, there is enough evidence against H0 (and the variable is not normally distributed).

Question 26

What is the problem when using formal normality tests on small and large sample sizes?

Answer 26

Normality tests have little power to provide against H0 - small samples often pass normality tests. For large sample sizes, significant results would be derived even in the case of a small deviation from normality, although this small deviation wouldn’t affect the results of a parametric test

Question 27

What tools can be used to assess normality?

Answer 27

A combination of visual inspection, assessment using skewness and kurtosis, and formal normality tests

Question 28

What do we do when we think data are showing substantial departures from normality?

Answer 28

Transform the data (e.g., by taking logarithms) or select a non-parametric method. The non-parametric alternatives have less power and require the two distributions to be more or less the same

Question 29

What’s the difference between <robvar> and <sdtest>?</sdtest></robvar>

Answer 29

<robvar> is a more robust when dealing with skewed distributions. The output will show information for three versions of the test: centred at the mean (W0), centred at the median (W50), and centred using a 10% trimmed mean (W10)
</robvar>

Question 30

How do you formally interpret results from an independent samples t-test?

Answer 30

Example: An independent samples t-test was conducted to compare the executive function for men and women. Descriptive statistics showed that men (mean = 17.53, sd 4.28) had slightly higher executive function than women (mean = 17.14, sd 4.29). When the t-test was performed, we found some evidence to suggest that the executive function of older people differ by gender [t(df)=2.096, p = 0.0362), with older men having on average higher executive function than women by 0.39 (95% CI 0.02, 0.075).

Question 31

What different ANOVAS can be used?

Answer 31

Two-way ANOVA includes two conditions. MANOVA includes more than one dependent variable. There are different types of ANOVA for longitudinal versus cross-sectional analysis.

Question 32

How do you test for equality of variance in one-way ANOVA?

Answer 32

Bartlett’s test for equal variances is produced as part of the output. If the test is significant, the variances are not equal across the groups.
If the difference in variances is not large and if your sample sizes are equal, you could still report results from ANOVA. Otherwise, we use the Welch or Brown-Forsythe test statistic as they are more robust (or use the non-parametric alternative, Kruskal-Wallis test)

Question 33

What code produces pairwise comparisons using the Welch test?

Answer 33

<ttest dependent_variable if inlist(independent_variable, 1, 2), by(independent_variable) welch>
<ttest dependent_variable if inlist(independent_variable, 1, 3), by(independent_variable) welch>
<ttest dependent_variable if inlist(independent_variable, 2, 3), by(independent_variable) welch>

Question 34

What does the Mann-Whitney U test and Kruskal-Wallis test do?

Answer 34

Mann-Whitney U test puts everything in terms of rank rather than in terms of new values. At its basic level, the test ranks everything, sums the ranks, and produces a statistic which tells you whether the two populations likely came from the same underlying population.
The Kruskal-Wallis test uses summed rank scores to determine the results. Informally, it used to test if the distributions have the same median

Question 35

How do you compute a Mann-Whitney U test?

Answer 35

<ranksum dependent_variable, by(independent_variable)>
In the output, below the table there is a variance adjustment to account for tied ranks. The values we are interested in are z and Prob > |z|

Question 36

How do you compute a Kruskal-Wallis test?

Answer 36

<kwallis dependent_variable, by(independent_variable)>
You should use the “ties” value when two or more observations can have the same score.

Question 37

How do you compute a paired t-test?

Answer 37

<ttest var1==var2>
To assess by a category, the command is <bysort cat_var: ttest var1==var2>

Question 38

Does a paired t-test assume homogeneity of variance?

Answer 38

No - the variances of the two variables separately are irrelevant.

Question 39

When would use a non-parametric alternative to paired t-test?

Answer 39

When two variables are not interval and you assume the difference is ordinal. The non-parametric version is the Wilcoxon signed rank sum test

Question 40

How do you compute a Wilcoxon signed rank sum test?

Answer 40

<signrank>
</signrank>

Question 41

How do you formally report one-way ANOVA results?

Answer 41

• A one-way ANOVA was conducted to compare the executive function scores by five different groups of self-rated health. Descriptive statistics show that those in excellent health report higher scores (mean=19.4, sd=4.06), compared to those who reported their health as poor (mean=15.6, sd=4.29). When we tested these differences using ANOVA, we found some strong evidence to suggest differences in mean executive function scores by self-rated health [F(4,1081)=21.54, p < 0.001].
• Post-hoc comparisons using Tukey’s HSD indicated that mean executive function of those in excellent health was significantly higher than respondents who reported their health as very good or less, with an average higher score of 1.19 compared to respondents in “very good health” and of 3.79 compared to those in “poor health”
• Overall, there was a significant difference in mean executive function by self-rated health such that older people with better health had higher scores than those in poorer health.