Session 2a

Question 1

Hypothesis

Answer 1

Empirically testable statement derived from our theories.Hypotheses are assumptions made about population parameters (not sample statistics).

Question 2

Hypothesis testing

Answer 2

Traditionally, experimental research engages in a procedure for hypothesis testing: Null hypothesis significance testing (NHST). NHST is still widely used. More recent approaches focus on effect sizes and formation of confidence intervals

Question 3

Hypothesis testing step 1

Answer 3

• Step 1: Set Up a hypothesis (Usually a prediction that there is an effect of certain variable(s) in the population)
• Step 2: Choose α (significance level: decide the area consisting of extreme scores which are unlikely to occur if the null hypothesis is true)
• Step 3: examine your data and compute the appropriate test statistic
• Step 4: make the decision whether to reject or not reject the null hypothesis

Question 4

Step 1 (hypothesis)

Answer 4

• Null Hypothesis (H0): This is what we test statistically. Null is true: no effect.
• Alternative Hypothesis (H1): research/experimental hypothesis. Non-directional H1: Some effect. Directional H1: Specifies the direction of the difference.

Question 5

Step 2 (criterion)

Answer 5

α is the proportion of times we are willing to accidentally reject H0, even if H0 is true. The cutoff value of the test statistic for α is called the critical value.

Question 6

Step 4 (decision)

Answer 6

Compare the calculated value of your test statistic to the critical value for α. If your value is greater than or equal to the critical value, reject H0. Otherwise, retain H0. A decision to reject H0 implies support for H1.

Question 7

Step 4 alternative (p-value)

Answer 7

Alternatively, look at the significance level (p-value) for the test statistic value. The p-value represents the proportion of data sets that would yield a result as extreme or more extreme than the observed result if H0 is true. Such values often given by SPSS, R, or other statistical software.. If p ≤ α (e.g., p ≤ .05), reject H0. Otherwise, retain H0. It is a Yes-No decision.

Question 8

Significance

Answer 8

If H0 is rejected, you may conclude that there is a statistically significant effect in the population.

Question 9

A “statistically significant” effect does not indicate that. . .

Answer 9

• We have a precise estimate of the effect: it may be that an effect is “significant”, but there is some error around our estimate. The effect in the population may be smaller or larger than our estimate. The amount of error is represented in the standard error for the estimate.
• The effect is important or meaningful:
  Suppose we find that seeing a spider increases anxiety by 2 points. Is 2 points a meaningful increase? The anxiety gain may be “significant” if it was observed from many people, but it does mean that the effect is practically meaningful.

Question 10

Confidence interval

Answer 10

A Confidence Interval gives us information about the precision of our estimates.
Example: a 95% confidence interval (CI) may indicate that true increase in anxiety in the population is between 0.5 and 3.5 points. If we repeated our experiment many times, 95% of the time a 95% CI will contain the true effect. As sample size increases, our estimate becomes more precise (and our CI intervals become more narrow). As α decreases, our CI intervals become larger or wider.

Question 11

Effect sizes

Answer 11

A standardized measure of the magnitude (importance) of a treatment effect. Commonly used measures of effect size:
- Pearson’s correlation coefficient or correlation ratio squared
- Cohen’s d
- Omega or omega squared
- Eta (η) squared
It is strongly encourage to always report effect size measures.

Question 12

Errors in hypothesis testing

Answer 12

The procedures we follow in hypothesis testing does not guarantee that our decision will be correct. Two types of errors:
- Type I: Reject H0 when it is true (False Positive). α = probability of committing Type I error
- Type II: Retain H0 when it is false (False Negative). β = probability of committing Type II error
- 1 − β (power) = probability of correctly rejecting a false H0

Question 13

Controlling errors in hypothesis testing

Answer 13

Complete control of α and β is not possible because of the relationship between α and β. Higher values of α means lower values of β (i.e., a less conservative test), which means the power to reject H0 is higher.
We usually select α a priori. The most common choice is α = 0.05, which means that if H0 is true, we stand only a 5% chance of rejecting it (and thus making a Type I
error).

Question 14

z-test for a single mean

Answer 14

• Purpose: Based on the sample mean (X) we test whether the population mean (µ) is equal to some hypothesized value
• Prior Requirements/Assumptions: The variable, X, in the population is normally distributed. The sample must be a simple random sample of the population (independence of observations). The population standard deviation, σ, must be known.
• Computing a z-test for a single mean is like calculating a Z-score for your sample mean.

Question 15

Limitations of z-test

Answer 15

• Knowing the true value of the population standard deviation (σ) is unrealistic, except in cases in which the entire population is known
• Alternative? The t-test

Question 16

t-test for a single mean

Answer 16

• Purpose: Based on the sample mean (X) we test whether the population mean (µ) is equal to some hypothesized value. We can also use it to compare means of 2 samples.
• Prior Requirements/Assumptions: The variable, X, in the population is normally distributed + The sample must be a simple random sample of the population (independence of observations)
• A t-statistic is obtained by replacing the population standard deviation with the sample standard deviation. Due to this replacement, the t-statistic does not follow the standard normal distribution. Instead, it follows the t-distribution.

Question 17

The t-distribution

Answer 17

The t distribution wasdiscovered by William S. Gosset in 1908. Gosset was a statistician employed by the Guinness brewing company and he couldn’t publish under his
own name. He instead used a pen name: “Student”. The distribution is thus also called the Student’s t- distribution. It varies in shape according to the degrees of freedom. The t distribution approaches a normal distribution as df (or sample size) becomes large. The distributions are quite close for df > 30. Note: t becomes closer to z as N increases.