PSY201: Chapter 8 - Hypothesis Testing

Question 1

Hypothesis Testing

Answer 1

general goal of hypothesis test is to rule out chance (sampling error) as plausible explanation for the results from research study
allows us to use sample data to draw inferences about pop of interest
technique to help determine whether treatment has an effect on the individuals in pop
can never know 100% if we are right, but we can assign statistical probability to likelihood that our conclusion is correct

Question 2

Hypothesis Testing

Answer 2

used to evaluate the results from a research study in which

1. A sample is selected from the pop
2. The treatment is administered to sample
3. After treatment, individuals in sample are measured

Question 3

Hypothesis Testing

Answer 3

if indiv in sample are noticeably different from indiv in original pop, have evidence treatment has an effect
also possible that diff between the sample + pop is sampling error

Question 4

Hypothesis Testing

Answer 4

purpose to decide betw 2 explanations:

1. Any diff betw sample + pop can be explained by SE (does not appear to be a treatment effect)

Question 5

Hypothesis Testing

Answer 5

2. diff betw sample + pop too large to be explained by sampling error (appears to be a treatment effect)

Question 6

Logic of Hypothesis Testing

Answer 6

1. State a hypothesis about a population.
2. Use hypothesis to predict characteristics that a sample should have
3. Obtain random sample from the pop

Question 7

Logic of Hypothesis Testing

Answer 7

4. Compare obtained sample data with prediction in 2nd step to see if we have support for our hypothesis/if hypothesis is wrong

Question 8

Step 1

Answer 8

State hypotheses
Ho: treatment (IV) has no effect on DV for pop
null hypothesis: pop mean after treatment is same as it was before treatment

Question 9

Step 1

Answer 9

H = hypothesis, 0 subscript = zero-effect
scientific alternative hypothesis
H1: effect of treatment on DV
there is a change, difference/relationship for general pop

Question 10

Step 1

Answer 10

frame our significance testing in terms of Ho: easier to prove that something is false than to prove that it’s true

Question 11

Null Hypothesis

Answer 11

Acts as starting point
Accepted as true in absence of other info
Provides comparison for observed outcomes
Both hypotheses written in conventional way, either in words/symbols
research paper hypothesis in everyday language not symbol

Question 12

Some differences between the null and research hypothesis

Answer 12

Ho: no difference, refers to pop, indirectly tested, implied

Research H: is a diff, refers to sample, directly tested, explicit

Question 13

Null Hypothesis

Answer 13

use sample data to determine likelihood of Ho being correct

determining what sample means would be consistent with Ho + what sample means would be inconsistent with it

Question 14

Null Hypothesis

Answer 14

look at distribution of sample means to determine which means near enough to pop + which too far from μ for Ho to hold

Question 15

Step 2

Answer 15

¨ To determine what is a high- vs. low-probability sample, we select a specific probability value,
¨ This value is called the level of significance or the alpha level, α-level, for the hypothesis test.

Question 16

α level

Answer 16

establishes a criterion, or “cut-off”, for making a decision about the null hypothesis. The alpha level also determines the risk of a Type I error.

Question 17

Step 2

Answer 17

We usually set α = .05 (5%), .01 (1%), or .001 (0.1%).
¤ E.g., α = .05 separates the most unlikely 5% of the sample means from the most likely 95% of the sample means.
¤ Where would α = .05 be in terms of a z-value? ¤ z = +/- 1.96 (unit table for p=.025 in the tail)

Question 18

Step 3

Answer 18

¨ So, we have determined the hypotheses and set the α-level.
¨ Our next step is to select a sample and administer the treatment to them.

Question 19

Step 3

Answer 19

Then, we determine the appropriate descriptive statistics – the sample mean, and compute the appropriate test statistic (in this chapter a z-score)
¨ Once we have the test statistic, we can compare the sample mean with the null hypothesis.

Question 20

Step 3

Answer 20

Compute the test statistic. The test statistic (in this chapter a z-
score) forms a ratio comparing the obtained difference between the sample mean and the hypothesized population mean versus the amount of difference we would expect without any treatment effect (the standard error)
z= M-μ/σM

Question 21

Step 3

Answer 21

Locate the critical region. The critical region consists of outcomes that are very unlikely to occur if the null hypothesis is true. That is, the critical region is defined by sample means that are almost impossible to obtain if thetreatmenthasnoeffect. Thephrase“almost impossible” means that these samples have a probability (p) that is less than the alpha level.

Question 22

Step 4

Answer 22

Alargevalueforthez-score(e.g.,z ≥ +/- 1.96 for α=.05) means that the obtained mean difference is more than can be explained by chance.
¨ Ifoursamplemeanisinthecritical region,
¤ the extremely unlikely sample mean values,

Question 23

Step 4

Answer 23

Then our null hypothesis is unlikely to be true.
¨ So, we reject the null hypothesis.
¤ i.e., conclude treatment did have an effect.

Question 24

Step 4

Answer 24

Ifthemeandifferenceisrelativelysmall,thentheteststatistic will have a low value.
¨ In this case, we fail to reject the null hypothesis.
¨ We only either reject or fail to accept the null; we are always
testing the null hypothesis.

Question 25

Step 4

Answer 25

¨ Wedonotsaythatweacceptthealternatehypothesis,because there is always the risk that we might be wrong.
¤ We are inferring from the sample to the population, and we are assuming that the sample is representative of the populations.
¤ We can say that the results “support” the alternative hypothesis, though.

Question 26

Step 4

Answer 26

A large value for the test statistic shows that the obtained mean difference is more than would be expected if there is no treatment effect. If it is large enough to be in the critical region, we conclude that the difference is significant or that the treatment has a significant effect. In this case we reject the null hypothesis. If the mean difference is relatively small, then the test statistic will have a low value. In this case, we conclude that the evidence from the sample is not sufficient, and the decision is fail to reject the null hypothesis.

Question 27

The Logic of Hypothesis Testing

Answer 27

You gather enough evidence to demonstrate that the treatment works.
n Here, we reject the null hypothesis.
q Your evidence is not convincing that the treatment
works.
n Here, we fail to reject the null hypothesis.

Question 28

Errors in Hypothesis Testing

Answer 28

¨ Just because the sample mean (after treatment) is different from the original population mean does not necessarily indicate that the treatment has caused a change.
¨ E.g., Imagine that you are trying to determine if extraneous noise while taking a test leads to anxiety,

Question 29

Errors in Hypothesis Testing

Answer 29

¨ So you test your group of students right before exams.
¨ Perhaps if you find that noise à anxiety, it could only hold because students are already exhausted and stressed out so anything will increase their anxiety.

Question 30

Errors in Hypothesis Testing

Answer 30

n Because the hypothesis test relies on sample data, and because sample data are not completely reliable, there is always the risk that misleading data will cause the hypothesis test to reach a wrong conclusion.
n Two types of error are possible.

Question 31

Type I Errors

Answer 31

n A Type I error occurs when the sample data appear to show a treatment effect when, in fact, there is none.
n In this case the researcher will reject the null hypothesis and falsely conclude that the treatment has an effect.

Question 32

Type I Errors

Answer 32

n Type I errors are caused by unusual, unrepresentative samples. Just by chance the researcher selects an extreme sample with the
result that the sample falls in the critical region even though the treatment has no effect.
n The hypothesis test is structured so that Type I errors are very unlikely; specifically, the probability of a Type I error is equal to the alpha level.

Question 33

Type I Errors

Answer 33

¨ The hypothesis test is structured so that Type I errors are very unlikely.
¨ What is the probability of a Type I error? ¨ The alpha level.

Question 34

Type I Errors

Answer 34

¤ E.g., 5% or less chance of saying the treatment had an effect when it was really only a chance finding.
¤ i.e., You are saying you will reject the null hypothesis if there is less than a 5% chance that you could have obtained your results if the null hypothesis were true.

Question 35

Type II Errors

Answer 35

sample does not appear to have been affected by the treatment when, in fact, the treatment does have an effect.
n In this case, the researcher will fail to reject the null hypothesis and falsely conclude that the treatment does not have an effect.

Question 36

Type II Errors

Answer 36

Type II errors are commonly the result of a very small treatment effect. Although the treatment does have an effect, it is not large enough to show up in the research study.

Question 37

Type II Errors

Answer 37

¤ when the sample does not appear to have been affected
by the treatment, but the treatment has had an effect.
¤ In this case, the researcher will fail to reject the null hypothesis and falsely conclude that the treatment does not have an effect.

Question 38

Type II Errors

Answer 38

Type II errors are commonly the result of a very small treatment effect;
¤ Although the treatment has an effect, it is not large enough to show up in the research study.
¨ Or setting the alpha level really high (e.g., 0.001).

Question 39

Type II Errors

Answer 39

n Although it is generally felt that making a Type II error is better than making a Type I error,
n It can have implications for practical applications of research
q e.g.,wemightnotuseahelpfulprocedurewhenit really works.

Question 40

Uncertainty and Errors in Hypothesis Testing

Answer 40

Hypothesis testing rests on a few assumptions:
q Wehaveusedrandomsamplingtoobtainour sample.
q Thevaluesmustconsistofindependent
observations

Question 41

Uncertainty and Errors in Hypothesis Testing

Answer 41

n There is no consistent, predictable relationship between observations; they are independent of each other.
n Random sampling helps us to have independent observations..
q Thedistributionofsamplemeansmustbe normal.

Question 42

Directional Tests

Answer 42

n Whenaresearchstudypredictsaspecificdirectionfor the treatment effect (increase or decrease), it is possible to incorporate the directional prediction into the hypothesis test.
n The result is called a directional test or a one-tailed test. A directional test includes the directional prediction in the statement of the hypotheses and in the location of the critical region.

Question 43

Directional Tests

Answer 43

For example, if the original population has a mean of μ = 80 and the treatment is predicted to increase the scores, then the null hypothesis would state that after treatment:
H0: μ < 80 (there is no increase)
n In this case, the entire critical region would be located in the right-hand tail of the distribution because large values for M would demonstrate that there is an increase and would tend to reject the null hypothesis.

Question 44

Directional (One-Tailed) Hypothesis Tests

Answer 44

we must:
q Incorporatethedirectionalpredictionintothe statement of the hypothesis.
n Start with H1 and then figure out H0.
q Havetheentirecriticalregioninonetailofthe distribution.

Question 45

Directional (One-Tailed) Hypothesis Tests

Answer 45

¨ The one-tailed test allows you to reject the null hypothesis when the difference between the sample and population is quite small (in the right direction);
¨ The two-tailed test requires relatively large mean differences (but direction does not matter).

Question 46

Directional (One-Tailed) Hypothesis Tests

Answer 46

¨ As such, two-tailed tests are more rigorous and typical because they require more evidence to reject Ho.
¨ You need a strong a priori one-tailed hypothesis to justify using a one-tailed hypothesis test.

Question 47

Measuring Effect Size

Answer 47

¨ Statistical significance doesn’t tell us about the absolute size of the treatment effect.
¨ It just tells us the relative size of the treatment effect compared to the standard error.
¨ If standard error is very small, the treatment effect can be very small and still be large enough to be significant.

Question 48

Measuring Effect Size

Answer 48

As you know, one way to minimize the standard error is to obtain a very large sample, n.

Question 49

A. Effect Size

Answer 49

¨ Because a significant effect does not necessarily mean a large/substantial treatment effect, it is recommended that the hypothesis test be accompanied by a measure of the effect size,
¤ Which provides a measurement of the absolute magnitude of a treatment effect, independent of the sample size.

Question 50

A. Effect Size

Answer 50

¨ Effect size indicates the extent to which 2 groups do not overlap; i.e., how much they’re separated due to the experimental procedure.
¨ Cohen’s d is a standardized measure of effect size;
¨ Much like a z-score, Cohen’s d measures the mean difference in terms
of the standard deviation.
Cohen’sd = μtreatment −μnotreatment/σ

Question 51

A. Effect Size

Answer 51

The population mean after treatment is unknown, so we estimate it using the treated sample mean:
estimatedCohen’sd = Mtreatment −μnotreatment/σ
¨ It tells you how much the treatment changed the mean in terms of standard deviation units.
¨ As you can see from the equation, Cohen’s d is not influenced by sample size.

Question 52

Measuring Effect Size

Answer 52

n Becauseasignificanteffectdoesnotnecessarilymeana large effect, it is recommended that the hypothesis test be accompanied by a measure of the effect size.
n We use Cohen=s d as a standardized measure of effect size.
n Much like a z-score, Cohen’s d measures the size of the mean difference in terms of the standard deviation.

Question 53

Effect Size

Answer 53

Cohen’s guidelines as to what are small/ large effects:
q Smalleffectisaboutd=0.2;thegroupshave
quite a lot of overlap;
n i.e., the mean difference is around 0.2 standard deviations.
q Mediumeffect,d=0.5
q Alargeeffect,d=0.8,indicatestheoverlapis considerably less.