Chapter 7 - Hypothesis Testing

Question 1

Alternative hypothesis (H1)

Answer 1

Corresponds to the research hypothesis. It usually takes the form: there is something happening, there is a difference or an effect, there is a relationship.

Question 2

Experiment

Answer 2

A study in which the researcher controls (or manipulates or changes) the conditions experimental units experience.
A study in which the researchers determine which units (people) receive is an example of an experiment.

Question 3

Experimental units

Answer 3

The individual “things” data is being recorded about in an experiment, e.g., people, cars, tomato plants, etc.

Question 4

Explanatory variable

Answer 4

A variable used to attempt to predict or explain a response.

Question 5

Factor

Answer 5

Categorical variable; defines group membership.

Question 6

Hypothesis test

Answer 6

The use of data to assess the strength of evidence against the null hypothesis, Hº, and hence got the alternative hypothesis, H1.
A test of whether a group difference seen in the data could be generated by a relevant chance mechanism - like the luck of the random allocation or the luck of the random sampling draw.

Question 7

Null hypothesis (Hº)

Answer 7

The hypothesis which is tested in a hypothesis test.
Usually takes a skeptical point of view: the researcher’s hunch is nonsense, there is nothing new or interesting happening, there is no effect.

Question 8

Practically significant

Answer 8

A difference is practically significant if it is big enough to have a real-world impact.
Relates to the size of the difference (consider the confidence interval limits)

Question 9

P-Value

Answer 9

The conditional probability of observing a test statistic as extreme as that observed or even more so, if the null hypothesis, Hº, were true.
Measures the strength of evidence against the null hypothesis. The smaller the P-value, the stronger the evidence against the null hypothesis, Hº.
The P-value comes from the tail proportions.

Question 10

Random allocation

Answer 10

The random process by which experimental units are allocated to treatments. The treatments should be allocated to units in such a way that each treatment is equally likely to be applied to each unit.

Question 11

Randomisation test

Answer 11

A hypothesis test that uses a re-randomisation distribution to estimate the P-value.

Question 12

Randomised experiment

Answer 12

An experiment where conditions are changed purposefully and a random process is used to decide who (or what entities) will be subject to what conditions.

Question 13

Response variable

Answer 13

The outcome variable on which comparisons are made.

Question 14

Statistical inference

Answer 14

Going beyond the data at hand, either by generating the observed results to a larger group or population (sample-to-population inference) or by drawing a more profound conclusion about the type of relationship between two variables such as, in an experiment, the explanatory variable causes a change in the response (experiment-to-causation inference).
Confidence intervals and hypothesis tests are two common statistical inference techniques.

Question 15

Statistical significance

Answer 15

Relates to the size of the P-value: A difference is statistically significant if it produces a small P-value, commonly less than 5%.

Question 16

Statistically significant

Answer 16

An observed difference is so large that it cannot reasonably be attributed to chance alone.
An observed difference is unlikely to have happened by chance alone.

Question 17

Test statistic

Answer 17

The summary statistic that we use to evaluate the hypotheses in a statistical test.
The larger the test statistic, the greater the evidence we have against the null hypothesis.

Question 18

Treatment

Answer 18

Something researchers administer to experimental units

Question 19

t-statistic

Answer 19

The distance between the estimate and the hypothesised value in terms of standard errors. (Shorthand tº)

Question 20

t-test

Answer 20

A hypothesis test that uses a Student’s t-distribution to estimate the P-value. Based on mathematical theory obtained under idealised assumptions (normal distribution).

Question 21

Units

Answer 21

The individual ‘things’ data is being recorded about.