Statistical tests Flashcards
K-S test or Kolmogorov-Smirnov test
Nonparametric test for the equality of continuous, one-dimensional probability distributions for one sample or two samples.
The null distribution of this statistic is calculated under the null hypothesis that the samples are drawn from the same distribution (in the two-sample case) or that the sample is drawn from the reference distribution (in the one-sample case)
The Kolmogorov–Smirnov test can be modified to serve as a goodness of fit test. In the special case of testing for normality of the distribution, samples are standardized and compared with a standard normal distribution. This is equivalent to setting the mean and variance of the reference distribution equal to the sample estimates, and it is known that using these to define the specific reference distribution changes the null distribution of the test statistic: see below. Various studies have found that, even in this corrected form, the test is less powerful for testing normality than the Shapiro–Wilk test or Anderson–Darling test.
Mann-Whitney U test
Also known as Mann–Whitney–Wilcoxon (MWW) , Wilcoxon rank-sum test or Wilcoxon-Mann-Whitney test
This is a non-parametric statistical hypothesis test for assessing whether one of two samples of independent observations tends to have larger values than the other. It is one of the most well-known non-parametric significance tests.
A very general formulation is to assume that:
(1) All the observations from both groups are independent of each other,
(2) The responses are ordinal (i.e. one can at least say, of any two observations, which is the greater),
(3) Under the null hypothesis the distributions of both groups are equal, so that the probability of an observation from one population (X) exceeding an observation from the second population (Y) equals the probability of an observation from Y exceeding an observation from X, that is, there is a symmetry between populations with respect to probability of random drawing of a larger observation.
(4) Under the alternative hypothesis the probability of an observation from one population (X) exceeding an observation from the second population (Y) (after exclusion of ties) is not equal to 0.5. The alternative may also be stated in terms of a one-sided test, for example: P(X > Y) + 0.5 P(X = Y) > 0.5.
Additional facts:
-related to kendall’s tau: equivalent if one variable is binary.
Non-parametric tests
Non-parametric (or distribution-free) inferential statistical methods are mathematical procedures for statistical hypothesis testing which, unlike parametric statistics, make no assumptions about the probability distributions of the variables being assessed.
Examples include:
Kolmogorov–Smirnov test
Mann–Whitney U or Wilcoxon rank sum test
Siegel–Tukey test
sign test
Wilcoxon signed-rank test
Anderson–Darling test
Kuiper’s test
Logrank Test
McNemar’s test
median test
Pitman’s permutation test
Wald–Wolfowitz runs test
Parametric tests
Parametric statistics is a branch of statistics that assumes that the data has come from a type of probability distribution and makes inferences about the parameters of the distribution.
Parametric methods make more assumptions than non-parametric methods. If those extra assumptions are correct, parametric methods can produce more accurate and precise estimates. They are said to have more statistical power.
Examples of parametric tests:
t-tests
What test(s) to use when you have two samples of data independently collected and you want to compare whether one has values greater than the other?
(1) Mann-Whitne U test (also called Mann–Whitney–Wilcoxon (MWW) , Wilcoxon rank-sum test or Wilcoxon-Mann-Whitney test)
- More robust to outliers
- Good if data is ordinal, but not interval scaled
- In case of normality, 95% efficient compared to t-test
- non parametric, usually a better choice, except perhaps in small sample sizes, power might be important
(2) independent samples Student’s t-test
- Assumes normality (parametric)
Wilcoxon signed-rank test
Non-parametric statistical test to compare two related samples, matched samples, or repeated measurements on a single sample to determine if their population mean ranks differ – a paired difference test.
Assumes:
(1) Data are paired and come from the same population.
(2) Each pair is chosen randomly and independent.
(3) The data are measured on an interval scale (ordinal is not sufficient because we take differences), but need not be normal.
Related methods: sign test, t-test (paired student’s t-test, t-test for matched pairs, t-test for dependent samples), paired z-test
Paired difference tests
(what it is, methods, and popular uses)
In statistics, a paired difference test is a type of location test that is used when comparing two sets of measurements to assess whether their population means differ. A paired difference test uses additional information about the sample that is not present in an ordinary unpaired testing situation, either to increase the statistical power, or to reduce the effects of confounders.
Methods include:
- t-test (when stdev. is not known)
- paired Z-test (when stdev. is known)
- Wilcoxon signed-rank test (non-normal distributions, assumes symmetric distribution?)
- sign test (non-parametric, does not assume symmetry?, less powerful)
Popular uses include:
- before and after a treatment – “repeated measures” tests (increases power)
- reduce confounding by introducing artificial pairs that match on some level…
Sign test
Non parametric test to test the hypothesis that there is “no difference in medians” between the continuous distributions of two random variables X and Y, in the situation when we can draw paired samples from X and Y.
Because it is non-parametric it has very general applicability but may lack the statistical power of other tests such as the paired-samples t-test or the Wilcoxon signed-rank test.
Method:
Let p = Pr(X > Y), and then test the null hypothesis H0: p = 0.50. In other words, the null hypothesis states that given a random pair of measurements (xi, yi), then xi and yi are equally likely to be larger than the other.
Then let W be the number of pairs for which yi − xi > 0. Assuming that H0 is true, then W follows a binomial distribution W ~ b(m, 0.5). The “W” is for Frank Wilcoxon who developed the test, then later, the more powerful Wilcoxon signed-rank test.
Median test
*Mostly thought to be obsolete due to low power
Instead use: Wilcoxon–Mann–Whitney U two-sample test
It is a nonparametric test that tests the null hypothesis that the medians of the populations from which two samples are drawn are identical.
Difference betwen this and Mann-Whitney U
The relevant difference between the two tests is that the median test only considers the position of each observation relative to the overall median, whereas the Wilcoxon–Mann–Whitney test takes the ranks of each observation into account. Thus the latter test is usually the more powerful of the two.
Methods to compare means
See http://en.wikipedia.org/wiki/Comparing_means
Kuiper’s test
Kuiper’s test is used in statistics to test that whether a given distribution, or family of distributions, is contradicted by evidence from a sample of data.
Properties: invariant to cyclic transformations, and as sensitive in tails as near median
Uses: cyclic variations by time of year or day of wee/time of day, in general any circular probability distributions
Related to: Kolmogorov–Smirnov test, Anderson–Darling test
More
Kuiper’s test[1] is closely related to the more well-known Kolmogorov–Smirnov test (or K-S test as it is often called). As with the K-S test, the discrepancy statistics D+ and D− represent the absolute sizes of the most positive and most negative differences between the two cumulative distribution functions that are being compared. The trick with Kuiper’s test is to use the quantity D+ + D− as the test statistic. This small change makes Kuiper’s test as sensitive in the tails as at the median and also makes it invariant under cyclic transformations of the independent variable. The Anderson–Darling test is another test that provides equal sensitivity at the tails as the median, but it does not provide the cyclic invariance.
This invariance under cyclic transformations makes Kuiper’s test invaluable when testing for cyclic variations by time of year or day of the week or time of day, and more generally for testing the fit of, and differences between, circular probability distributions.
Jarque–Bera test
The Jarque–Bera test is a goodness-of-fit test of whether sample data have the skewness and kurtosis matching a normal distribution
Cramér–von Mises criterion
Cramér–von Mises criterion is a criterion used for judging the goodness of fit of a cumulative distribution function compared to a given empirical distribution function , or for comparing two empirical distributions. It is also used as a part of other algorithms, such as minimum distance estimation.
In one-sample applications is the theoretical distribution and is the empirically observed distribution. Alternatively the two distributions can both be empirically estimated ones; this is called the two-sample case.
Related alternative tests: Kolmogorov–Smirnov test, Watson test (almost same)
Siegel–Tukey test
The Siegel–Tukey test is a non-parametric test which may be applied to data measured at least on an ordinal scale. It tests for differences in scale between two groups.
The test is used to determine if one of two groups of data tends to have more widely dispersed values than the other. In other words, the test determines whether one of the two groups tends to move, sometimes to the right, sometimes to the left, but away from the center (of the ordinal scale).
1960
More:
The principle is based on the following idea:
Suppose there are two groups A and B with n observations for the first group and m observations for the second (so there are N = n + m total observations). If all N observations are arranged in ascending order, it can be expected that the values of the two groups will be mixed or sorted randomly, if there are no differences between the two groups (following the null hypothesis H0). This would mean that among the ranks of extreme (high and low) scores, there would be similar values from Group A and Group B.
If, say, Group A were more inclined to extreme values (the alternative hypothesis H1), then there will be a higher proportion of observations from group A with low or high values, and a reduced proportion of values at the center.
Hypothesis H0: σ2A = σ2B & MeA = MeB (where σ2 and Me are the variance and the median, respectively)
Hypothesis H1: σ2A > σ2B
Statistical hypothesis tests
Statistical hypothesis tests answer the question Assuming that the null hypothesis is true, what is the probability of observing a value for the test statistic that is at least as extreme as the value that was actually observed?.[2] That probability is known as the P-value.
Statistical hypothesis testing is a key technique of frequentist statistical inference. The Bayesian approach to hypothesis testing is to base decisions on the posterior probability.
Wald–Wolfowitz runs test
The runs test (also called Wald–Wolfowitz test) is a non-parametric statistical test that checks a randomness hypothesis for a two-valued data sequence. More precisely, it can be used to test the hypothesis that the elements of the sequence are mutually independent.
“run” of a sequence is a maximal non-empty segment of the sequence consisting of adjacent equal elements. For example, the sequence “++++−−−+++−−++++++−−−−” consists of six runs, three of which consist of +’s and the others of −’s. The run test is based on the null hypothesis that the two elements + and - are independently drawn from the same distribution.
Under the null hypothesis, the number of runs in a sequence of length N is a random variable whose conditional distribution given the observation of N+ positive values and N− negative values (N = N+ + N−) is approximately normal.
The mean and variance do not depend on the “fairness” of the process generating the elements of the sequence, that is, that +’s and −’s have equal probabilities, but only on the assumption that the elements are independent and identically distributed. If the number of runs is significantly higher or lower than expected, the hypothesis of statistical independence of the elements may be rejected.
Runs tests can be used to test:
the randomness of a distribution, by taking the data in the given order and marking with + the data greater than the median, and with – the data less than the median; (Numbers equalling the median are omitted.)
whether a function fits well to a data set, by marking the data exceeding the function value with + and the other data with −. For this use, the runs test, which takes into account the signs but not the distances, is complementary to the chi square test, which takes into account the distances but not the signs.
The Kolmogorov–Smirnov test is more powerful, if it can be applied.
Kendall’s W
Kendall’s W (Kendall’s coefficient of concordance) is a non-parametric statistic. It is a normalization of the statistic of the Friedman test, and can be used for assessing agreement among raters. Kendall’s W ranges from 0 (no agreement) to 1 (complete agreement).
More:
uppose, for instance, that a number of people have been asked to rank a list of political concerns, from most important to least important. Kendall’s W can be calculated from these data. If the test statistic W is 1, then all the survey respondents have been unanimous, and each respondent has assigned the same order to the list of concerns. If W is 0, then there is no overall trend of agreement among the respondents, and their responses may be regarded as essentially random. Intermediate values of W indicate a greater or lesser degree of unanimity among the various responses.
While tests using the standard Pearson correlation coefficient assume normally distributed values and compare two sequences of outcomes at a time, Kendall’s W makes no assumptions regarding the nature of the probability distribution and can handle any number of distinct outcomes.
W is linearly related to the mean value of the Spearman’s rank correlation coefficients between all pairs of the rankings over which it is calculated.
Friedman test
The Friedman test is a non-parametric statistical test used to detect differences in treatments across multiple test attempts. The procedure involves ranking each row (or block) together, then considering the values of ranks by columns. Applicable to complete block designs, it is thus a special case of the Durbin test.
Classic examples of use are:
n wine judges each rate k different wines. Are any wines ranked consistently higher or lower than the others?
n wines are each rated by k different judges. Are the judges’ ratings consistent with each other?
n welders each use k welding torches, and the ensuing welds were rated on quality. Do any of the torches produce consistently better or worse welds?
The Friedman test is used for one-way repeated measures analysis of variance by ranks. In its use of ranks it is similar to the Kruskal-Wallis one-way analysis of variance by ranks.
When using this kind of design for a binary response, one instead uses the Cochran’s Q test.
Durbin test
In the analysis of designed experiments, the Friedman test is the most common non-parametric test for complete block designs. The Durbin test is a nonparametric test for balanced incomplete designs that reduces to the Friedman test in the case of a complete block design.
More:
In a randomized block design, k treatments are applied to b blocks.For some experiments, it may not be realistic to run all treatments in all blocks, so one may need to run an incomplete block design. In this case, it is strongly recommended to run a balanced incomplete design. A balanced incomplete block design has the following properties:
Every block contains k experimental units.
Every treatment appears in r blocks.
Every treatment appears with every other treatment an equal number of times.
The Durbin test is based on the following assumptions:
The b blocks are mutually independent. That means the results within one block do not affect the results within other blocks.
The data can be meaningfully ranked (i.e., the data have at least an ordinal scale).
Cochran’s Q test is applied for the special case of a binary response variable (i.e., one that can have only one of two possible outcomes)
Cochran’s Q test
In statistics, in the analysis of two-way randomized block designs where the response variable can take only two possible outcomes (coded as 0 and 1), Cochran’s Q test is a non-parametric statistical test to verify if k treatments have identical effects.[1][2] It is named for William Gemmell Cochran. Cochran’s Q test should not be confused with Cochran’s C test, which is a variance outlier test.
Cochran’s Q test assumes that there are k > 2 experimental treatments and that the observations are arranged in b blocks
Cochran’s Q test is
H0: The treatments are equally effective.
Ha: There is a difference in effectiveness among treatments.
Cohen’s kappa
Cohen’s kappa coeffici
ent is a statistical measure of inter-rater agreement or inter-annotator agreement[1] for qualitative (categorical) items. It is generally thought to be a more robust measure than simple percent agreement calculation since κ takes into account the agreement occurring by chance. Some researchers[2][citation needed] have expressed concern over κ’s tendency to take the observed categories’ frequencies as givens, which can have the effect of underestimating agreement for a category that is also commonly used; for this reason, κ is considered an overly conservative measure of agreement.
Others[3][citation needed] contest the assertion that kappa “takes into account” chance agreement. To do this effectively would require an explicit model of how chance affects rater decisions. The so-called chance adjustment of kappa statistics supposes that, when not completely certain, raters simply guess—a very unrealistic scenario.
Cohen’s kappa measures the agreement between two raters who each classify N items into C mutually exclusive categories.
The equation for κ is:
where Pr(a) is the relative observed agreement among raters, and Pr(e) is the hypothetical probability of chance agreement, using the observed data to calculate the probabilities of each observer randomly saying each category. If the raters are in complete agreement then κ = 1. If there is no agreement among the raters other than what would be expected by chance (as defined by Pr(e)), κ = 0.
Kappa assumes its theoretical maximum value of 1 only when both observers distribute codes the same, that is, when corresponding row and column sums are identical. Anything less is less than perfect agreement. till, the maximum value kappa could achieve given unequal distributions helps interpret the value of kappa actually obtained. The equation for κ maximum is: (See http://en.wikipedia.org/wiki/Cohen%27s_kappa)
Cochran’s Q test
In statistics, in the analysis of two-way randomized block designs where the response variable can take only two possible outcomes (coded as 0 and 1), Cochran’s Q test is a non-parametric statistical test to verify if k treatments have identical effects. Cochran’s Q test should not be confused with Cochran’s C test, which is a variance outlier test.
Cochran’s Q test is
H0: The treatments are equally effective.
Ha: There is a difference in effectiveness among treatments.
The Cochran’s Q test statistic is
k is the number of treatments
X• j is the column total for the jth treatment
b is the number of blocks
Xi • is the row total for the ith block
N is the grand total
Cochran’s Q test is based on the following assumptions:
A large sample approximation; in particular, it assumes that b is “large”.
The blocks were randomly selected from the population of all possible blocks.
The outcomes of the treatments can be coded as binary responses (i.e., a “0” or “1”) in a way that is common to all treatments within each block.
Related/alternatives:
When using this kind of design for a response that is not binary but rather ordinal or continuous, one instead uses the Friedman test or Durbin tests.
The case where there are exactly two treatments is equivalent to McNemar’s test, which is itself equivalent to a two-tailed sign test.
Chow Test
The Chow test is a statistical and econometric test of whether the coefficients in two linear regressions on different data sets are equal. The Chow test was invented by economist Gregory Chow. In econometrics, the Chow test is most commonly used in time series analysis to test for the presence of a structural break. In program evaluation, the Chow test is often used to determine whether the independent variables have different impacts on different subgroups of the population.
The null hypothesis of the Chow test asserts that a1=a2, b1=b2, and c1=c2, and there is the assumption that the model errors are independent and identically distributed from a normal distribution with unknown variance.
Likelihood-ratio test
In statistics, a likelihood ratio test is a statistical test used to compare the fit of two models, one of which (the null model) is a special case of the other (the alternative model). The test is based on the likelihood ratio, which expresses how many times more likely the data are under one model than the other. This likelihood ratio, or equivalently its logarithm, can then be used to compute a p-value, or compared to a critical value to decide whether to reject the null model in favour of the alternative model. When the logarithm of the likelihood ratio is used, the statistic is known as a log-likelihood ratio statistic, and the probability distribution of this test statistic, assuming that the null model is true, can be approximated using Wilks’ theorem.
In the case of distinguishing between two models, each of which has no unknown parameters, use of the likelihood ratio test can be justified by the Neyman–Pearson lemma, which demonstrates that such a test has the highest power among all competitors.
The likelihood ratio, often denoted by (the capital Greek letter lambda), is the ratio of the likelihood function varying the parameters over two different sets in the numerator and denominator. A likelihood-ratio test is a statistical test for making a decision between two hypotheses based on the value of this ratio.
It is central to the Neyman–Pearson approach to statistical hypothesis testing, and, like statistical hypothesis testing generally, is both widely used and much criticized.
More:http://en.wikipedia.org/wiki/Likelihood_ratio_test
A statistical model is often a parametrized family of probability density functions or probability mass functions f(x|theta) . A simple-vs-simple hypotheses test has completely specified models under both the null and alternative hypotheses, which for convenience are written in terms of fixed values of a notional parameter theta:
H0: theta=theta_0
H1:theta = theta_1
likelihood ratio:
if Lambda>c, do not reject H0, if Lambda
G-test
In statistics, G-tests are likelihood-ratio or maximum likelihood statistical significance tests that are increasingly being used in situations where chi-squared tests were previously recommended.[citation needed]
The general formula for G is
where Oi is the observed frequency in a cell, E is the expected frequency on the null hypothesis, and the sum is taken over all cells, and where ln denotes the natural logarithm (log to the base e) and the sum is taken over all non-empty cells.
G-tests are coming into increasing use, particularly since they were recommended at least since the 1981 edition of the popular statistics textbook by Sokal and Rohlf.
Related
Given the null hypothesis that the observed frequencies result from random sampling from a distribution with the given expected frequencies, the distribution of G is approximately a chi-squared distribution, with the same number of degrees of freedom as in the corresponding chi-squared test.
For very small samples the multinomial test for goodness of fit, and Fisher’s exact test for contingency tables, or even Bayesian hypothesis selection are preferable to the G-test
The commonly used chi-squared tests for goodness of fit to a distribution and for independence in contingency tables are in fact approximations of the log-likelihood ratio on which the G-tests are based.
The G-test quantity is proportional to the Kullback–Leibler divergence of the empirical distribution from the theoretical distribution.
For analysis of contingency tables the value of G can also be expressed in terms of mutual information.
An application of the G-test is known as the McDonald–Kreitman test in statistical genetics. Dunning[6] introduced the test to the computational linguistics community where it is now widely used.
Deviance
In statistics, deviance is a quality of fit statistic for a model that is often used for statistical hypothesis testing.
The deviance for a model M0, based on a dataset y, is defined as
Here hat-theta_0 denotes the fitted values of the parameters in the model M0, while hat-theta_s denotes the fitted parameters for the “full model” (or “saturated model”): both sets of fitted values are implicitly functions of the observations y. Here the full model is a model with a parameter for every observation so that the data are fitted exactly. This expression is simply −2 times the log-likelihood ratio of the reduced model compared to the full model. The deviance is used to compare two models - in particular in the case of generalized linear models where it has a similar role to residual variance from ANOVA in linear models (RSS).
Suppose in the framework of the GLM, we have two nested models, M1 and M2. In particular, suppose that M1 contains the parameters in M2, and k additional parameters. Then, under the null hypothesis that M2 is the true model, the difference between the deviances for the two models follows an approximate chi-squared distribution with k-degrees of freedom.
Welch’s t test
In statistics, Welch’s t test is an adaptation of Student’s t-test intended for use with two samples having possibly unequal variances.[1] As such, it is an approximate solution to the Behrens–Fisher problem
Once t and have been computed, these statistics can be used with the t-distribution to test the null hypothesis that the two population means are equal (using a two-tailed test), or the null hypothesis that one of the population means is greater than or equal to the other (using a one-tailed test). In particular, the test will yield a p-value which might or might not give evidence sufficient to reject the null hypothesis.
Welch’s t-test defines the statistic t by the following formula where Xi, s_i, and N_i are the th sample mean, sample variance and sample size
Unlike in Student’s t-test, the denominator is not based on a pooled variance estimate.
(note the degrees of freedom associated with this variance estimate is approximated using the Welch-Satterthwaite equation)
Z-test
A Z-test is any statistical test for which the distribution of the test statistic under the null hypothesis can be approximated by a normal distribution. Because of the central limit theorem, many test statistics are approximately normally distributed for large samples. For each significance level, the Z-test has a single critical value (for example, 1.96 for 5% two tailed) which makes it more convenient than the Student’s t-test which has separate critical values for each sample size. Therefore, many statistical tests can be conveniently performed as approximate Z-tests if the sample size is large or the population variance known. If the population variance is unknown (and therefore has to be estimated from the sample itself) and the sample size is not large (n < 30), the Student t-test may be more appropriate.
If T is a statistic that is approximately normally distributed under the null hypothesis, the next step in performing a Z-test is to estimate the expected value θ of T under the null hypothesis, and then obtain an estimate s of the standard deviation of T. We then calculate the standard score Z = (T − θ) / s, from which one-tailed and two-tailed p-values can be calculated as Φ(−|Z|) and 2Φ(−|Z|), respectively, where Φ is the standard normal cumulative distribution function
Requirements and assumptions:
For the Z-test to be applicable, certain conditions must be met.
Nuisance parameters should be known, or estimated with high accuracy (an example of a nuisance parameter would be the standard deviation in a one-sample location test). Z-tests focus on a single parameter, and treat all other unknown parameters as being fixed at their true values. In practice, due to Slutsky’s theorem, “plugging in” consistent estimates of nuisance parameters can be justified. However if the sample size is not large enough for these estimates to be reasonably accurate, the Z-test may not perform well.
The test statistic should follow a normal distribution. Generally, one appeals to the central limit theorem to justify assuming that a test statistic varies normally. There is a great deal of statistical research on the question of when a test statistic varies approximately normally. If the variation of the test statistic is strongly non-normal, a Z-test should not be used.
If estimates of nuisance parameters are plugged in as discussed above, it is important to use estimates appropriate for the way the data were sampled. In the special case of Z-tests for the one or two sample location problem, the usual sample standard deviation is only appropriate if the data were collected as an independent sample.
In some situations, it is possible to devise a test that properly accounts for the variation in plug-in estimates of nuisance parameters. In the case of one and two sample location problems, a t-test does this.
Examples: (location tests)
The term Z-test is often used to refer specifically to the one-sample location test comparing the mean of a set of measurements to a given constant. If the observed data X1, …, Xn are (i) uncorrelated, (ii) have a common mean μ, and (iii) have a common variance σ2, then the sample average X has mean μ and variance σ2 / n. If our null hypothesis is that the mean value of the population is a given number μ0, we can use X −μ0 as a test-statistic, rejecting the null hypothesis if X −μ0 is large.
To calculate the standardized statistic Z = (X − μ0) / s, we need to either know or have an approximate value for σ2, from which we can calculate s2 = σ2 / n. In some applications, σ2 is known, but this is uncommon. If the sample size is moderate or large, we can substitute the sample variance for σ2, giving a plug-in test. The resulting test will not be an exact Z-test since the uncertainty in the sample variance is not accounted for — however, it will be a good approximation unless the sample size is small. A t-test can be used to account for the uncertainty in the sample variance when the sample size is small and the data are exactly normal. There is no universal constant at which the sample size is generally considered large enough to justify use of the plug-in test. Typical rules of thumb range from 20 to 50 samples. For larger sample sizes, the t-test procedure gives almost identical p-values as the Z-test procedure.
Other location tests that can be performed as Z-tests are the two-sample location test and the paired difference test.
Examples: (Z-tests other than location tests)
Location tests are the most familiar t-tests. Another class of Z-tests arises in maximum likelihood estimation of the parameters in a parametric statistical model. Maximum likelihood estimates are approximately normal under certain conditions, and their asymptotic variance can be calculated in terms of the Fisher information. The maximum likelihood estimate divided by its standard error can be used as a test statistic for the null hypothesis that the population value of the parameter equals zero. More generally, if hat_θ is the maximum likelihood estimate of a parameter θ, and θ_0 is the value of θ under the null hypothesis,
can be used as a Z-test statistic.
When using a Z-test for maximum likelihood estimates, it is important to be aware that the normal approximation may be poor if the sample size is not sufficiently large. Although there is no simple, universal rule stating how large the sample size must be to use a Z-test, simulation can give a good idea as to whether a Z-test is appropriate in a given situation.
Z-tests are employed whenever it can be argued that a test statistic follows a normal distribution under the null hypothesis of interest. Many non-parametric test statistics, such as U statistics, are approximately normal for large enough sample sizes, and hence are often performed as Z-tests.
Example Usage:
Suppose that in a particular geographic region, the mean and standard deviation of scores on a reading test are 100 points, and 12 points, respectively. Our interest is in the scores of 55 students in a particular school who received a mean score of 96. We can ask whether this mean score is significantly lower than the regional mean — that is, are the students in this school comparable to a simple random sample of 55 students from the region as a whole, or are their scores surprisingly low?
We begin by calculating the standard error of the mean : 1.62
Next we calculate the z-score, which is the distance from the sample mean to the population mean in units of the standard error: -2.47
n this example, we treat the population mean and variance as known, which would be appropriate either if all students in the region were tested, or if a large random sample were used to estimate the population mean and variance with minimal estimation error.
The classroom mean score is 96, which is −2.47 standard error units from the population mean of 100. Looking up the z-score in a table of the standard normal distribution, we find that the probability of observing a standard normal value below -2.47 is approximately 0.5 - 0.4932 = 0.0068. This is the one-sided p-value for the null hypothesis that the 55 students are comparable to a simple random sample from the population of all test-takers. The two-sided p-value is approximately 0.014 (twice the one-sided p-value).
Another way of stating things is that with probability 1 − 0.014 = 0.986, a simple random sample of 55 students would have a mean test score within 4 units of the population mean. We could also say that with 98.6% confidence we reject the null hypothesis that the 55 test takers are comparable to a simple random sample from the population of test-takers.
The Z-test tells us that the 55 students of interest have an unusually low mean test score compared to most simple random samples of similar size from the population of test-takers. A deficiency of this analysis is that it does not consider whether the effect size of 4 points is meaningful. If instead of a classroom, we considered a subregion containing 900 students whose mean score was 99, nearly the same z-score and p-value would be observed. This shows that if the sample size is large enough, very small differences from the null value can be highly statistically significant. See statistical hypothesis testing for further discussion of this issue.
