Week 2: Hypothesis Testing and Its Implications

Question 1

What question are we focusing on the framework this lecture?

Answer 1

Meets assumption of parametric tests?

Question 2

Answering the question: Meets assumption of parametric tests will determine whether our continous data can be tested with

Answer 2

with parametric or non-parametric tests

Question 3

A normal distribution is a distribution with the same general shape which is a

Answer 3

bell shape

Question 4

A normal distribution curve is symmetric around

Answer 4

the mean μ

Question 5

A normal distribution is defined by two parameters - (2)

Answer 5

the mean (μ) and the standard deviation (σ).

Question 6

Many statistical tests (parametric) cannot be used if the data is not

Answer 6

normally distributed

Question 7

What does this diagram show? - (2)

Answer 7

μ = 0 is peak of distribution

Block areas under the curve and gives us insight to way data is distributed and certain scores occuring if they belong to normally distribution e.g., 34.1% of values lie one SD below mean

Question 8

A z score in standard normal distribution will reflect the number of

Answer 8

SD above or below the mean of a particular score is

Question 9

How to calculate a z score?

Answer 9

Take a value of participant (e.g., 56 years old) and take away mean of distribution (e.g., mean age of class is 23) divided by SD (class like 2)

Question 10

If a person scored a 70 on a test with a mean of 50 and a standard deviation of 10

Converting the test scores to z scores, an X of 70 would be…

What the result means…. - (2)

Answer 10

a z score of 2 means the original score was 2 standard deviations above the mean

Question 11

We can convert our z scores to

Answer 11

pecentiles

Question 12

Example: What is the percentile rank of a person receving a score of 90 on the test? - (3)

Mean - 80
SD = 5

Answer 12

First calculating z score: graph shows that most people scored below 90. Since 90 is 2 standard deviations above the mean z = (90 - 80)/5 = 2

Z score to pecentile can be looked at table that z score of 2 is equivalent to the 97.7th percentle:

The proportion of people scoring below 90 is thus .977 and proportion of people scoring above 90 is 2.3% (1-0.977)

Question 13

We can not always measure the whole… for a study

Answer 13

population

Question 14

What is the sample mean?

Answer 14

an unbiased estimate of the population mean.

Question 15

Example of sample vs population - (3)

Answer 15

You want to study political attitudes in young people.

Your population is the 300,000 undergraduate students in the Netherlands.

Because it’s not practical to collect data from all of them, you use a sample of 300 undergraduate volunteers from three Dutch universities – this is the group who will complete your online survey.

Question 16

How can we know how that our sample mean estimate is representative of the population mean?

Answer 16

Via computing standard error of mean - smaller SEM the better

Question 17

What does this diagram shows you? - (2)

Answer 17

If you take several samples from same population,

each sample has its own mean and some sample means will be different or same as population mean- error - known as SEM

Question 18

What is sample variation and example - (2)

Answer 18

samples will vary because they contain different members of the population;

a sample that by chance includes some very
good lecturers will have a higher average (higher rating of all lectures) than a sample that, by chance, includes some awful lecturers.

Question 19

Standard deviation is used as a measure of how

Answer 19

representative the mean was of the observed data.

Question 20

Small standard deviations represented a scenario in which most data points were

Answer 20

most data points were close to the mean

Question 21

Large standard deviation represented a situation in which data points were

Answer 21

widely spread
from the mean.

Question 22

How to calculate the standard error of mean?

Answer 22

computed by dividing the standard deviation of the sample by the the square root of the number in the sample

Question 23

The larger the sample the smaller the - (2)

Answer 23

standard error of the mean

more confident we can be that the sample mean is representative of the population.

Question 24

The central limit therom proposes that

Answer 24

as samples get large (usually defined as greater than 30), the sampling distribution has a normal distribution with a mean equal to the population mean, SD = SEM

Question 25

The standard deviation of sample means is known as the

Answer 25

SEM (standard error of the mean)

Question 26

A different approach to assess accuracy of sample mean as estimate of - population mean, aside from SE, is to - (2)

Answer 26

calculate boundaries and range of values within which we believe the true value of the population mean value will fall.

Such boundaries are called confidence intervals.

Question 27

Confidence intervals are created by

Answer 27

samples

Question 28

A 95% confidence intervals is consructed such that

Answer 28

these intervals (created by samples) will contain the population mean

Question 29

95% Confidence interval for 100 samples (CI constructed for each) would mean

Answer 29

95 of these samples, the confidence intervals we constructed would contain the true value of the mean in the population.

Question 30

In fact, for a specific confidence interval, the probability that it contains the population value is either - (2)

Answer 30

0 (it does not contain it) or 1 (it does contain it).

You have no way of knowing which it is.

Question 31

Diagram shows- (4)

Answer 31

• Dots show the means for each sample
• Lines sticking out representing Ci for the sample means
• If there was a vertical line down it represents population mean
• If confidence intervals don’t overlap then it shows significant difference between the sample means

Question 32

if our sample means were normally distributed with a mean of 0 and a
standard error of 1, then the limits of our confidence interval

Answer 32

would be –1.96 and +1.96 -

Question 33

95% of z scores fall between

Answer 33

-1.96 and 1.96

Question 34

Confidence intervals can be constructed for any estimated parameter, not just

Answer 34

μ - mean

Question 35

. If the mean represents the true mean well, then the confidence interval of that mean should be

Answer 35

small

Question 36

if the confidence interval is very
wide then the sample mean could be

Answer 36

very different from the true mean, indicating that it
is a bad representation of the population

Question 37

Remember that the standard error of the mean gets smaller with the number of observations and thus our confidence interval also gets

Answer 37

smaller - make sense as more we measure more certain sample mean close to population mean

Question 38

Confidence intervals can be constructed for any estimated parameter, not just

Answer 38

mean , μ

Question 39

Calculating Confidence intervals (for observations) - rearraning z formula

Answer 39

Know most scores remain at z = 1.96 (upper bound) and z = -1.96 (lower bound)

LB = (-1.96* SD of sample) + mean sample
UB = (+1.96* SD of sample) + mean sample

Question 40

Calculating Confidence Intervals for sample means - rearranging in z formula

Answer 40

LB = Mean - (1.96 * SEM)
UB = Mean + (1.96 * SEM)

Question 41

The standard deviation of SAT verbal scores in a school system is known to be 100. A researcher wishes to estimate the mean SAT score and compute a 95% confidence interval from a random sample of 10 scores.

The 10 scores are: 320, 380, 400, 420, 500, 520, 600, 660, 720, and 780.

Calculate CI

Answer 41

* M - 530
* N = 10
* SEM = 100/ square root of 10 = 31.62
* Value of z for 95% CI is number of SD one must go from mean (in both directions) to contain 0.95 of the scores
* Value of 1.96 was found in z-table
* Since each tail is to contain 0.025 of the scores, you find the values of z for which is 1-0.025 = 0.975 of the socres below
* 95% of z scores lie between -1.96 and +1.96
* Lower limit = 530 - (1.96) (31.62) = 468.02
* Upper limit = 530 + (1.96)(31.62) = 591.98

Question 42

Null hypothesis is that there is

Answer 42

no effect of the predictor variable on the outcome variable

Question 43

The alternate hypothesis is that there is an effect of

Answer 43

the predictor variable on the outcome variable

Question 44

Null hypothesis signifiance testing computes the probability of the null hypothesis being true which si referred as the

Answer 44

p-value

Question 45

To test the fit of statistical models to test our hypotheses, we calculate

Answer 45

getting that model (Data) if the Null hypothesis H0 were true (Statistical significance)

Question 46

What if the proability p- value was small?

Answer 46

we conclude the model fits the data well (explains a lot of the variance) and we gain confidence in the alternative hypothesis H1

Question 47

Steps in Hypothesis testing (6)

Answer 47

1. specify the null hypothesis H0 and the alternative hypothesis H1
2. select a significance level. Typically the 0.05 or the 0.01 level.
3. calculate a statistic analogous to the parameter specified by the null hypothesis. (e.g. if null defined by parameter μ1- μ2 (diff between two means) then the statistic is M1-M2 (difference between sample means))
4. calculate the probability value of obtaining a statistic (statistic computed from the data) as different or more different from the parameter specified in the null hypothesis (often 0 or based on past evid and mean stay same)
5. probability value computed in Step 4 is compared with the significance level chosen in Step 2.
6. If the outcome is statistically significant, then the null hypothesis is rejected in favor of the alternative hypothesis.

Question 48

Think of test statistic capturing

Answer 48

signal/noise

Question 49

Hypo

A testStatistic for which the frequency of particular values is known (t, F, chi-square) and thus we can calculate the

Answer 49

probability of obtaining a certain value or p value.

Question 50

To test whether the model fits the data or whether our hypothesis is a good explanation of the data, we compare

Answer 50

systematic variation against unsystematic

Question 51

If the probability (p-value) less than or equal to the significance level, then

Answer 51

the null hypothesis is rejected; When the null hypothesis is rejected, the outcome is said to be “statistically significant”

Question 52

If the proabilibty (p-value) is greater than the signifiance leve, the

Answer 52

null hypothesis is not rejected.

Question 53

We accept the results as true (accept our alternative hypothesis) if there is either

Answer 53

%, 1% (p<0.05 OR p<0.01) or less probability of a test statistics happening by chance.

Question 54

P-value less than 0.05 means there is a low probability of obtaining at least as

Answer 54

extreme results given that H0 is true

Question 55

What is a type 1 error in terms of variance? - (2)

Answer 55

think the variance accounted for by the model is larger than the one unaccounted for by the model (i.e. there is a statistically significant effect but in reality there isn’t)

Question 56

Type 1 is a false

Answer 56

positive

Question 57

What is type II error in temrs of variance?

Answer 57

think there was too much variance unaccounted for by the model (i.e. there is no statistically significant effect but in reality there is)

Question 58

Type II error is false

Answer 58

negative

Question 59

Example of Type I and Type II error

Answer 59

Question 60

Type I and Type II errors are mistakes we can make when testing the

Answer 60

fit of the model

Question 61

Type 1 errors when we believe there is a geniue effect in

Answer 61

population, when in fact there isn’t.

Question 62

Acceptable level of type I error is usually

Answer 62

a-level of usually 0.05

Question 63

Type II error occurs when we believe there is no effect in the

Answer 63

population when, in reality, there is.

Question 64

Acceptable level of Type II error is probability/-p-value is

Answer 64

β-level (often 0.2)

Question 65

An effect size is a standardised measure of

Answer 65

the size of the an effect

Question 66

Properities of effect size (3)

Answer 66

Standardized = comparable across studies

Not (as) reliant on the sample size

Allows people to objectively evaluate the size of observed effect.

Question 67

Effect Size Measures

r = 0.1, d = 0.2 (small effect):

Answer 67

the effect explains 1% of the total variance.

Question 68

Effect size measures

r = 0.3, d = 0.5 (medium effect) means

Answer 68

the effect accounts for 9% of the total variance.

Question 69

Effect size measures

r = 0.5, d = 0.8 (large effect)

Answer 69

effect accounts for 25% of the variance

Question 70

Beware of the ‘canned’ effect sizes (e.g., r = 0.5, d = 0.8 and rest) since the size of

Answer 70

effect should be placed within the research context.

Question 71

We should aim to achieve a power of

Answer 71

.8, or an 80% chance of detecting
an effect if one genuinely exists.

Question 72

When we fail to reject the null hypothesis, it is either that there truly are no difference to be found,

OR

Answer 72

it may be because we do not have enough statistical power

Question 73

Power is the probability of

Answer 73

correctly rejecting a false H0 OR the ability of the test to find an effect assuming there is one in the population,

Question 74

Power is calculated by

Answer 74

1 - β OR probability of making Type II error

Question 75

To increase statistical power of study you can increase

Answer 75

your sample sizee

Question 76

Factors affecting the power of the test: (4):

Answer 76

1. Probability of a type 1 error or a-level [level at which we decide effect is sig - p-value) –> bigger [more lenient] alpha then more power)
2. True alternate hypothesis H1 [effect size] (degree of overlap, less means more power) - if you find large effect in lit then better chance of detecting something
3. The sampel size [N]) –> bigger the sample, less the noise and more power
4. The particular tests to be employed - parametric tests greater power to detect sig effect since more sensitive

Question 77

How to calculate the number of pps they need for reasonable chance of correctly rejecting null hypothesis?

Answer 77

Sample size calculation at a desired level of power (usually power set to 0.8 in formula)

Question 78

Tests of normality (2)

Answer 78

• Kolmogorov-Smirnov test
• Shapiro-Wilks test

Question 79

If distribution of data looks normally distributed but test saying not normally distributed

Answer 79

Just do the parametric tests

Question 80

Plot your data because this informs you on what decisions you want to make

Answer 80

with respect to normality –> normality tests have limitations

Question 81

With power, we can do 2 things - (2)

Answer 81

• Calculate power of test
• Calculate sample size necessary to detect an decent effect size and achieve a certain level of power based on past research

Question 82

Diagram of Type I error, Type II error, power - (4) and making correct decisions

Answer 82

Type 1 error p = alpha
Type II error p = beta
Accepting null hypothesis which is correct - p = 1- alpha
Accepting alternate hypo which is correct - p = 1 - beta

Question 83

If there is a less degree of overlap in h0 and h1 then

Answer 83

bigger difference means higher power and and correctly reject the null hypothesis than distributions that overlap more

Question 84

If distribution between h0 and h1 are narrower then

Answer 84

This means that the overlap in distributions is smaller and the power is therefore greater, but this time because of a smaller standard error of our estimate of the means.

Question 85

Most people want to assess how many participants they need to test to have a reasonable chance of correctly rejecting the null hypothesis (the Power). This formula shows - (2)

Answer 85

us how.

We usually set the power to 0.8.

Question 86

What is z scores? - (2)

Answer 86

A measure of variability:

The number of standard deviations from the population mean or a particular data point is
Z-scores are a standardised measure, hence they ignore measurement units

Question 87

Why should we care about z scores? - (2)

Answer 87

Z-scores allow researchers to calculate the probability of a score occurring within a standard normal distribution

Enables us to compare two scores that are from different samples (which may have different means and standard deviations)

Question 88

Diagram of finding percentile of Trish

Trish takes a test and gets 25

Mean of the class is 20

SD = 4

25-20/4 = 1.25

Z-score = 1.25

Answer 88

Let’s say Trish takes a test and scores 25 and the mean is 20 You may calculate the z-score to be 1.25 you would use a z-score table to see what percentile they would be in (marked in red) so to read the table you would go down to the value 1.2 and you would go across to 0.05 which totals to 1.25 and you can see about 89.4% of other students performed worse.

Question 89

Diagram of z score and percentile

Josh takes a different test and gets 1150
Mean of the class is 1000
SD = 150

1150 – 1000/150 = 1.0
Z score = 1.0

Who performed better Trish or Josh?

Trish had z score of 1.25

Answer 89

We would use our table and look down the column to a z-score of 1 and across to the 0.00 column (in purple) and we can see 84.1% of students performed worse than Josh so Trish performed better than Josh.

Question 90

Diagram of z scores and normal distribution - (3)

Answer 90

68% of scores are within 1 SD of the mean,

95% are within 2 SDs and

99.7% are within 3 SDs.

Question 91

Whats standard error?

Answer 91

: by taking into account the variability and size of our sample we can estimate how far away from the real population mean our mean is!

Question 92

If we took infinite samples from the population, 95% of the time the population mean will lie within the

Answer 92

the 95% confidence interval range

Question 93

Due to the frequency of normal distribution we can get a number for the lower and upper limits of the

Answer 93

95% confidence interval.

Question 94

How to calculate standard error?

Answer 94

Question 95

What does narrow CI represent?

Answer 95

high statistical power

Question 96

Wide CIs represent?

Answer 96

low statistical power

Question 97

Power bring the probability of catching a real effect (as opposed to

Answer 97

missing a real effect – Type II error)

Question 98

Null hypothesis in terms of same and fifferent population is that

Answer 98

No actual difference exists in the real world, all data comes from the same population

Question 99

Alternative (experimental) hypothesis in terms of same or different population

Answer 99

There is an actual difference and we found it!

Question 100

We collect evidence to

Answer 100

reject null hypothesis

Question 101

We can never say the null hypothesis is

Answer 101

FALSE (or TRUE).

Question 102

TheP valueor calculated probability is the estimated probability of us

Answer 102

us finding an effect when the null hypothesis (H0) is true.

Question 103

p = probability of observing a test statistic at least as a big as the one we have if the

Answer 103

H0 is true

Question 104

Hence, a significant p value (p <.05) tells us that there is a less than 5% chance of getting a test statistic that is

Answer 104

larger than the one we have found if there were no effect in the population (e.g. the null hypothesis were true)

Question 105

The p-value gets smaller as the test statistic calculated from your data gets

Answer 105

gets further away from the range of test statistics predicted by the null hypothesis.

Question 106

Statistical signifiance does not equal importance - (2)

Answer 106

p = .049, p = .050 are essentially the same thing- the former is ‘statistically significant’.

Importance is dependent upon the experimental design/aims: e.g., A statistically significant weight increase of 0.1Kg between two adults experimental groups may be less important than the same increase between two groups of babies.

Question 107

Children can learn a second language faster before the age of 7’. Is this statement:

A. One-tailed

B. A non scientific

C. Two-tailed

D. Null hypothesos

Answer 107

A as one-tailed is directional and two tailed is non-direcitonal

Question 108

Which of the following is true about a 95% confidence interval of the mean:

A. 95 out of 100 CIs wll contain population mean

B. 95 out of 100 sample means will fall within the limits of the confidence interval.

C. 95% of population means will fall within the limits of the confidence interval.

D. There is a 0.05 probability that the population mean falls within the limits of the confidence interval.

Answer 108

A as If we’d collected 100 samples, calculated the mean and then calculated a confidence interval for that mean, then for 95 of these samples the confidence intervals we constructed would contain the true value of the mean in the population

Question 109

What does a significant test statistic tell us?

A. That the test statistic is larger than we would expect if there were no effect in the population.

B. There is an important effect.

C. The null hypothesis is false.

D. All of the above.

Answer 109

A and just because test statistic is sig does not mean its important effect

Question 110

Of what is p the probability?

(Hint: NHST relies on fitting a ‘model’ to the data and then evaluating the probability of this ‘model’ given the assumption that no effect exists.)

A.p is the probability of observing a test statistic at least as big as the one we have if there were no effect in the population (i.e., the null hypothesis were true).

B. p is the probability that the results are due to chance, the probability that the null hypothesis (H0) is true.

C. p is the probability that the results are not due to chance, the probability that the null hypothesis (H0) is false

D. p is the probability that the results would be replicated if the experiment was conducted a second time.

Answer 110

A

Question 111

A Type I error occurs when:

(Hint: When we use test statistics to tell us about the true state of the world, we’re trying to see whether there is an effect in our population.)

A. We conclude that there is an effect in the population when in fact there is not.

B. We conclude that there is not an effect in the population when in fact there is.

C. We conclude that the test statistic is significant when in fact it is not.

D. The data we have typed into SPSS is different from the data collected.

Answer 111

A as If we use the conventional criterion then the probability of this error is .05 (or 5%) when there is no effect in the population

Question 112

True or false?

a. Power is the ability of a test to detect an effect given that an effect of a certain size exists in a population.

Answer 112

TRUE

Question 113

True or False?

We can use power to determine how large a sample is required to detect an effect of a certain size.

Answer 113

TRUE

Question 114

True or False?

c. Power is linked to the probability of making a Type II error.

Answer 114

TRUE

Question 115

True or False?

d. The power of a test is the probability that a given test is reliable and valid.

Answer 115

FALSE

Question 116

What is the relationship between sample size and the standard error of the mean?

(Hint: The law of large numbers applies here: the larger the sample is, the better it will reflect that particular population.)

A. The standard error decreases as the sample size increases.

B. The standard error decreases as the sample size decreases.

C. The standard error is unaffected by the sample size.

D. The standard error increases as the sample size increases.

Answer 116

A The standard error (which is the standard deviation of the distribution of sample means), defined as σ_Χ ̅ =σ/√N, decreases as the sample size (N) increases and vice versa

Question 117

What is the null hypothesis for the following question: Is there a relationship between heart rate and the number of cups of coffee drunk within the last 4 hours?

A. There will be no relationship between heart rate and the number of cups of coffee drunk within the last 4 hours.

B. People who drink more coffee will have significantly higher heart rates.

C. People who drink more cups of coffee will have significantly lower heart rates.

D. There will be a significant relationship between the number of cups of coffee drunk within the last 4 hours and heart rate

Answer 117

A The null hypothesis is the opposite of the alternative hypothesis and so usually states that an effect is absent

Question 118

A Type II error occurs when :

(Hint: This would occur when we obtain a small test statistic (perhaps because there is a lot of natural variation between our samples.)

A. We conclude that there is not an effect in the population when in fact there is.

B. We conclude that there is an effect in the population when in fact there is not.

C. We conclude that the test statistic is significant when in fact it is not.

D. The data we have typed into SPSS is different from the data collected.

Answer 118

A A Type II error would occur when we obtain a small test statistic (perhaps because there is a lot of natural variation between our samples)

Question 119

In general, as the sample size (N) increases:

A. The confidence interval gets narrower.

B. The confidence interval gets wider.

C. The confidence interval is unaffected.

D. The confidence interval becomes less accurate

Answer 119

A

Question 120

Which of the following best describes the relationship between sample size and significance testing?

(Hint: Remember that test statistics are basically a signal-to-noise ratio, so given that large samples have less ‘noise’ they make it easier to find the ‘signal’.)

A. In large samples even small effects can be deemed ‘significant’.

B. In small samples only small effects will be deemed ‘significant’.

C. Large effects tend to be significant only in small samples.

D. Large effects tend to be significant only in large samples.

Answer 120

A

Question 121

The assumption of homogeneity of variance is met when:

A. The variances in different groups are approximately equal.

B. The variances in different groups are significantly different.

C. The variance across groups is proportional to the means of those groups.

D. The variance is the same as the interquartile range.

Answer 121

A - To make sure our estimates of the parameters that define our model and significance tests are accurate we have to assume homoscedasticity (also known as homogeneity of variance)

Question 122

Next, the lecturer was interested in seeing whether males and females reacted differently to the different teaching methods.

Produce a clustered bar graph showing the mean scores of teaching method for males and females.

(HINT: place TeachingMethod on the X axis, Exam Score on the Y axis, and Gender in the ‘Cluster on X’ box. Include 95% confidence intervals in the graph).

Which of the following is the most accurate interpretation of the data?

A.Females performed better than males both the reward and indifferent conditions. Regarding the confidence intervals, there was a large degree of overlap between males and females in all conditions of the teaching method.

B.Males performed better than females in the reward condition, and females performed better than males in the indifferent condition. Regarding the confidence intervals, there was no overlap between males and females across any of the conditions of teaching method.

C.Males performed better than females in all conditions. Regarding the confidence intervals, there was a small degree of overlap between males and females for the reward and indifferent conditions, and a large degree of overlap between males and females for the punish condition.

D.Males performed better than females in the reward condition, and females performed better than males in the indifferent condition. Regarding the confidence intervals, there was a small degree of overlap between males and females for the reward and indifferent conditions, and a large degree of overlap between males and females for the punish condition.

Answer 122

D

Question 123

Produce a line graph showing the change in mean anxiety scores over the three time points.

NOTE: this is a repeated measures (or within subjects) design, ALL participants took part in the same condition.

Which of the following is the correct interpretation of the data?

A.Mean anxiety increased across the three time points.

BMean anxiety scores were reduced across the three time points, and there was a slight acceleration in this reduction between the middle and end of the course.

CMean anxiety scores were reduced across the three time points, though this reduction slowed down between the middle and end of the course.

DMean anxiety scores did not change across the three time points.

Answer 123

B