Chapter 2

Question 1

α-level

Answer 1

the probability of making a Type I error (usually this value is .05).

Question 2

Alternative hypothesis

Answer 2

the prediction that there will be an effect (i.e., that your experimental manipulation will have some effect or that certain variables will relate to each other).

Question 3

β-level

Answer 3

the probability of making a Type II error (Cohen, 1992, suggests a maximum value of .2).

Question 4

Bonferroni correction

Answer 4

a correction applied to the α-level to control the overall Type I error rate when multiple significance tests are carried out. Each test conducted should use a criterion of significance of the α-level (normally .05) divided by the number of tests conducted. This is a simple but effective correction, but tends to be too strict when lots of tests are performed.

Question 5

Central limit theorem

Answer 5

this theorem states that when samples are large (above about 30) the sampling distribution will take the shape of a normal distribution regardless of the shape of the population from which the sample was drawn. For small samples the t-distribution better approximates the shape of the sampling distribution. We also know from this theorem that the standard deviation of the sampling distribution (i.e., the standard error of the sample mean) will be equal to the standard deviation of the sample(s) divided by the square root of the sample size (N).

Question 6

Cohen’s d

Answer 6

An effect size that expressed the difference between two means in standard deviation units. In general it can be estimated using the formula above.

Question 7

Confidence interval

Answer 7

for a given statistic calculated for a sample of observations (e.g., the mean), the confidence interval is a range of values around that statistic that are believed to contain, with a certain probability (e.g., 95%), the true value of that statistic (i.e., the population value).

Question 8

Degrees of freedom

Answer 8

an impossible thing to define in a few pages, let alone a few lines. Essentially it is the number of ‘entities’ that are free to vary when estimating some kind of statistical parameter. In a more practical sense, it has a bearing on significance tests for many commonly used test statistics (such as the F-ratio, t-test, chi-square statistic) and determines the exact form of the probability distribution for these test statistics. The explanation involving soccer players in Chapter 2 is far more interesting…

Question 9

Deviance

Answer 9

the difference between the observed value of a variable and the value of that variable predicted by a statistical model.

Question 10

Effect size

Answer 10

an objective and (usually) standardized measure of the magnitude of an observed effect. Measures include Cohen’s d, Glass’s g and Pearson’s correlations coefficient, r.

Question 11

Experimental hypothesis

Answer 11

synonym for alternative hypothesis.

Question 12

Experimentwise error rate

Answer 12

the probability of making a Type I error in an experiment involving one or more statistical comparisons when the null hypothesis is true in each case.

Question 13

Familywise error rate

Answer 13

the probability of making a Type I error in any family of tests when the null hypothesis is true in each case. The ‘family of tests’ can be loosely defined as a set of tests conducted on the same data set and addressing the same empirical question.

Question 14

Fit

Answer 14

how sexually attractive you find a statistical test. Alternatively, it’s the degree to which a statistical model is an accurate representation of some observed data. (Incidentally, it’s just plain wrong to find statistical tests sexually attractive.)

Question 15

Linear model

Answer 15

a model that is based upon a straight line.

Question 16

Meta-analysis

Answer 16

this is a statistical procedure for assimilating research findings. It is based on the simple idea that we can take effect sizes from individual studies that research the same question, quantify the observed effect in a standard way (using effect sizes) and then combine these effects to get a more accurate idea of the true effect in the population.

Question 17

Method of least squares

Answer 17

a method of estimating parameters (such as the mean, or a regression coefficient) that is based on minimizing the sum of squared errors. The parameter estimate will be the value, out of all of those possible, that has the smallest sum of squared errors.

Question 18

Null hypothesis

Answer 18

the reverse of the experimental hypothesis, it says that your prediction is wrong and the predicted effect doesn’t exist.

Question 19

One-tailed test

Answer 19

a test of a directional hypothesis. For example, the hypothesis ‘the longer I write this glossary, the more I want to place my editor’s genitals in a starved crocodile’s mouth’ requires a one-tailed test because I’ve stated the direction of the relationship (see also two-tailed test).

Question 20

Parameter

Answer 20

a very difficult thing to describe. When you fit a statistical model to your data, that model will consist of variables and parameters: variables are measured constructs that vary across entities in the sample, whereas parameters describe the relations between those variables in the population. In other words, they are constants believed to represent some fundamental truth about the measured variables. We use sample data to estimate the likely value of parameters because we don’t have direct access to the population. Of course it’s not quite as simple as that.

Question 21

Population

Answer 21

in statistical terms this usually refers to the collection of units (be they people, plankton, plants, cities, suicidal authors, etc.) to which we want to generalize a set of findings or a statistical model.

Question 22

Power

Answer 22

the ability of a test to detect an effect of a particular size (a value of .8 is a good level to aim for).

Question 23

Sample

Answer 23

a smaller (but hopefully representative) collection of units from a population used to determine truths about that population (e.g., how a given population behaves in certain conditions).

Question 24

Sampling distribution

Answer 24

the probability distribution of a statistic. We can think of this as follows: if we take a sample from a population and calculate some statistic (e.g., the mean), the value of this statistic will depend somewhat on the sample we took. As such the statistic will vary slightly from sample to sample. If, hypothetically, we took lots and lots of samples from the population and calculated the statistic of interest we could create a frequency distribution of the values we got. The resulting distribution is what the sampling distribution represents: the distribution of possible values of a given statistic that we could expect to get from a given population.

Question 25

Sampling variation

Answer 25

the extent to which a statistic (the mean, median, t, F, etc.) varies in samples taken from the same population.

Question 26

Standard error

Answer 26

the standard deviation of the sampling distribution of a statistic. For a given statistic (e.g., the mean) it tells us how much variability there is in this statistic across samples from the same population. Large values, therefore, indicate that a statistic from a given sample may not be an accurate reflection of the population from which the sample came.

Question 27

Standard error of the mean (SE)

Answer 27

the standard error associated with the mean. Did you really need a glossary entry to work that out?

Question 28

Test statistic

Answer 28

a statistic for which we know how frequently different values occur. The observed value of such a statistic is typically used to test hypotheses.

Question 29

Two-tailed test

Answer 29

a test of a non-directional hypothesis. For example, the hypothesis ‘writing this glossary has some effect on what I want to do with my editor’s genitals’ requires a two-tailed test because it doesn’t suggest the direction of the relationship. See also One-tailed test.

Question 30

Type I error

Answer 30

occurs when we believe that there is a genuine effect in our population, when in fact there isn’t.

Question 31

Type II error

Answer 31

occurs when we believe that there is no effect in the population, when in fact there is.