Part 6. Hypothesis Testing

Question 1

Hypothesis testing

Answer 1

The part of statistical inference, the process of making judgments about a larger group (a population) based on a smaller group of observations (sample).

Definition:

We test to see whether a sample statistic is likely to come from a population with the hypothesized value of the population parameter.

Question 2

Statistical inference

Answer 2

1. Estimation

2. Hypothesis testing

Question 3

Hypothesis

Answer 3

A statement about one or more populations that we test using sample statistics.

Question 4

The process of hypothesis testing

Answer 4

1. State the hypothesis.
2. Identify the appropriate test statistic.
3. Specify the level of significance.
4. State the decision rule.
5. Collect data and calculate the test statistics.
6. Make a decision.

Question 5

Null hypothesis (H0)

Answer 5

A statement concerning a population parameter or parameters considered to be true, unless the sample we use to conduct the hypothesis test gives convincing evidence the null hypothesis is false.

The statement we want to reject, in favour of alternative hypothesis, Ha.

Question 6

Level of significance

Answer 6

The probability of Type I error in testing hypothesis denoted by alpha, a.

i.e. probability of incorrectly rejecting the true H0.

Question 7

Confidence level

Answer 7

The complement of level of significance; 1-a.

i.e. 5% probability of rejecting a true H0, corresponds to 95% confidence level.

Question 8

Type 1 and 2 error dilemma

Answer 8

• Both errors involve a trade off
• If we decrease the probability of Type 1 error by specifying a smaller significance level (1% instead of 5%), we increase the probability of making a Type 2 error as will reject H0 less frequently.
• To reduce the probability of both types of errors simultaneously is to increase sample size, n.

Question 9

Power of test

Answer 9

The probability of correctly rejecting the null, the probability of rejecting the null when it is false.

The complement of Type 2 error.

Question 10

Critical values

Answer 10

The action of comparing calculated test statistic with specified value or values.

Question 11

Statistically significant

Answer 11

Finding result of calculated value of test statistic is more extreme than critical value/values, we reject H0.

Question 12

Collecting data considerations:

Answer 12

1. Ensure that the sampling procedure does not include biases, such as sample selection or time bias.
2. We need to cleanse the data, checking inaccuracies and other measurement errors in data.

Once assured sample is unbiased and accurate, the sample info is used to calculate the appropriate test statistic.

Question 13

Make a decision

Answer 13

1. Statistical decision:
   - consider a test of mean risk premium, comparing population mean with zero using bounds.
2. Economic decision:

• Considers statistical decision, but all pertinent economic issues, i.e. reject H0 that risk premium is zero for greater than zero. This is economically meaningful that investor commit funds to US equities.
• Non-statistical considerations such as investors tolerance for risk and financial position.

3. Statistical significance, not economic?

• The smaller the standard error of mean, the larger value of t-stat and greater chance H0 rejected all else equal.
• Standard error decreases as sample size, n, increases, so that for large samples we can reject H0 for small departures from it.
• There is a statistically positive mean return, results may not be economically significant when accounting for transaction costs, taxes, and risk.

Question 14

P-Value

Answer 14

The area in the probability distribution outside the calculated test statistic, for a 2-sided test this is the area outside +- the calculated test statistic.

For one sided test, this is the area outside the calculated test statistic on the appropriate side of the probability distribution.

The p value is the smallest level of significance at which the null hypothesis can be rejected.

Question 15

P-value

Answer 15

The smallest level of significance at which null hypothesis can be rejected.

The area in the probability distribution outside the calculated test statistic for a two-sided test, the area outside +- the calculated test statistic.

For one sided test, this is the area outside the calculated test statistic on the appropriate side of probability distribution.

Question 16

Why are p-values not completely uniform?

Answer 16

• The p-values for true H0 are generally uniformly distributed between 0% and 100%, as under H0 there is a 5% chance of p-values < 5%, and 10% chance < 10%.
• If we took 1000 samples of 50, taking more samples/ larger samples means more uniform distribution of p-values.
• p-values for false H0, there is no uniform distribution, but a peak around 0% and little elsewhere.
• There is a difference in p-values for 2 false hypothesised means of 6.5% and 7%, showing further the false hypothesis is away from truth, the greater the power of test and better ability to detect false hypothesis.

Question 17

False discovery rate (FDR)

Answer 17

The expected portion of false positives, where this result the H0 is rejected even though H0 is true.

Question 18

Multiple testing problem

Answer 18

• Drawing a 1000 samples of 50 observations each, there are samples in which we reject the true H0 of population mean 6%.
• If we draw enough samples with level of significance of 0.05, the approx. 0.05 of time you reject H0 even if its true.
• If you run 100 tests, and use 5% level of significance, you will get 5 false positives on average.

Question 19

Paired observations

Answer 19

Observations that are dependent as they have something in common.

e.g. a dividend policy of companies before and after a change in the tax law affecting taxation of dividends.

Pairs of observations for same companies, dependent samples as pairs of samples before and after tax law change.

Question 20

Parametric test

Answer 20

Any test/procedure with either characteristic of:

1. Concerned with parameters
2. Validity depends on a definite set of assumptions

Question 21

Non-parametric test

Answer 21

A test that is not concerned with a parameter or test that makes minimal assumptions about the population from which the sample comes.

Question 22

Use of non-parametric procedures

Answer 22

1. when data we use do not meet distributional assumptions
2. when there are outliers
3. when the data are given in ranks or use an ordinal scale
4. when the hypotheses we are addressing do not concern a parameter.

Question 23

Non-parametric characteristics:

Answer 23

1. We must refer to specialised statistical tables to determine the rejection points of the test statistic, at least for small samples.
2. Although the underlying distribution of population may be normal, there may be extreme values or outliers that influence parametric statistics, but not non-parametric stats.
   e. g. we may want to use nonparametric test of median, in case of outliers we may test the mean.
3. A sample in which observations are ranked, we use nonparametric tests as parametric tests require stronger measurement scale than ranks.
4. When our question does not concern a parameter, e.g. if the question concerns whether sample is random or not, we use ‘runs test’.
   e. g. runs test used

Question 24

Significance test of correlation coefficient

Answer 24

Allows us to assess whether the relationship between 2 random variables is the result of chance.

Question 25

Correlation coefficient

Answer 25

The number between -1 and +1, where -1 denotes perfect negative or inverse relation between variables, +1 denotes a perfect positive relationship, 0 is an absent relationship.

Question 26

What is the parametric pairwise correlation coefficient often referred to as:

Answer 26

Pearson correlation, bivariate correlation, or simply the correlation.

Question 27

What does magnitude of r needed to reject H0 decrease as sample size n increases?

Answer 27

1. As n increases, no. of df increases, and absolute value of CV of t-stat decreases.
2. The absolute value of numerator increases with larger n, resulting in larger magnitude of calculated t-stat.
   e. g. n = 12, r=0.35, t-stat = 1.182, not different from zero at 0.05 level (ta/2 = +-2.228).

n=32, r = 0.35 yields t-stat 2.046 sign level 0.05 (ta/2 = +-2.042)

Question 28

Spearman rank correlation coefficient

Answer 28

The equivalent to the usual correlation coefficient but is calculated on the ranks of the 2 variables with their respective samples.