Chapter 2. Test for Significance Flashcards
the process by which we
determine the probability that there
is a significant difference between
two samples
Significance Testing (Hypothesis testing)
Indeterminate error is sufficient to explain any
difference in the values
being compared.
- no significant difference hypothesis
Null Hypothesis
The difference between the
values is too great to be explained by random error
and, therefore, must be real.
- have a significant difference
Alternative Hypothesis
A null hypothesis is retained whenever the evidence is insufficient to prove it is incorrect. Because of the way in which significance tests are conducted, it is impossible to prove that a null hypothesis is true.
the confidence level for retaining the null hypothesis (95%), or the probability that the null hypothesis will be incorrectly rejected
Significance Level
Retain Null = statistically the same
Reject Null = significant difference
If there is greater than a 5% chance of a result as extreme as the sample result when the null hypothesis is true, then the null hypothesis is retained. This does not necessarily mean that the researcher accepts the null hypothesis as true—only that there is not currently enough evidence to conclude that it is true.
Rejecting the null hypothesis means that a relationship does exist between a set of variables and the effect is statistically significant (p > 0.05)
The null hypothesis cannot be proven, although the hypothesis test begins with an assumption that the hypothesis is true, and the final result indicates the failure of the rejection of the null hypothesis. Thus, it is always advisable to state ‘fail to reject the null hypothesis’ instead of ‘accept the null hypothesis.
Two-tailed test
- comparison
- key words: significant difference, same effect, generate the same…
One-tailed test
- key words: greater than, lower than, more effective, less effective…
Type I Error
- The risk of falsely rejecting
the null hypothesis (𝛼) - risk is always equivalent to α
- false positive
Example:
Ho: he likes you back
Truth: he likes you back
Decision: falsely rejected Ho (missed your chance)
Type II Error
- The risk of falsely retaining
the null hypothesis (β). - (β) - depends on sample size and variance
- false negative
Example:
Ho: he likes you back
Truth: he doesn’t like you back
Decision: accepted Ho (wrongly invited him)
Minimizing a type 1 error by
decreasing 𝛼, for example,
increases the likelihood of a type 2 error
F-Test
a test designed to indicate
whether there is a significant
difference between two
methods based on their
standard deviations.
Always write the formula Ysa!!!
F-Test
- higher value is always on the numerator
F is defined in terms of the
variances of the two methods, where the variance is the
square of the standard deviation
If 𝑭𝒄𝒂𝒍𝒄 > 𝑭𝒕𝒂𝒃𝒍𝒆, then the
variances being compared are
Significantly different
If 𝑭𝒄𝒂𝒍𝒄 < 𝑭𝒕𝒂𝒃𝒍𝒆, then the
variances being compared are
Statistically the same
The purpose of significance testing is
provide evidence to reject the null hypothesis
Null hypothesis
Something we believe/expect to be true; no difference hypothesis
have no significant difference
Alternative hypothesis
have significant difference
comparison is made between two
sets of replicate measurements
made by two different means
t-test
If 𝒕𝒄𝒂𝒍𝒄 > 𝒕𝒕𝒂𝒃𝒍𝒆, then the data
being compared have
Significant Difference
If 𝒕𝒄𝒂𝒍𝒄 > 𝒕𝒕𝒂𝒃𝒍𝒆, then the data
being compared have
significant difference
Several ways and several situations in which t-test can be
used:
▸ t- test when a reference value is known
▸ Comparison of the means of two methods
▸ Paired t-Test
▸ t-test to compare different samples
➢ used to obtain an improved estimate of the precision of a
method and for calculating the precision of the two sets of data
➢ provides a more reliable estimate of the precision of a method
than is obtained from a single set
➢ if random error is assumed to be the same for each set, then
the data of the different sets can be pooled
Pooled Standard deviation
side = denominator
top = numerator
(0.05, numerator, denominator)
➢ two methods are used to make
single measurements on
several different samples.
➢ No measurement has been
duplicated
Paired t-Test
▸ Analysis of Variance
▸ allows comparisons among more than
two population means
▸ use a single test to determine whether
there is or is not a difference among
the population means rather than
pairwise comparisons as is done with
the t test.
ANOVA
𝐻0: 𝜇1 = 𝜇2 = 𝜇3 = ⋯ = 𝜇𝑖
𝐻𝐴 : at least two of the 𝜇𝑖’s
are different
Applications
- Is there a difference in the results of five analysts determining calcium
by a volumetric method? - Will four different solvent compositions have differing influences on the
yield of a chemical synthesis? - Are the results of manganese determinations by three different
analytical methods different? - Is there any difference in the fluorescence of a complex ion at six
different values of pH?
The basic principle of ANOVA is to
compare the variations between the
factor levels (groups) to that within
factor levels
When H0 is TRUE, the variation between the group MEANS IS CLOSE to
the variation WITHIN groups.
When H0 is FALSE, the variation between
the group mean is LARGE compared to the variation within groups.
Grand mean can be calculated as the weighted average of the individual
group means,
Single Factor ANOVA
A statistical t value is calculated
and compared with a tabulated
value for the given number of
tests at the desired confidence
level
t-test when a reference value is known
use this when comparing
two methods, that is, when
analyzing a sample by two
different methods
comparison of the means of two methods
▸ use this when comparing two
methods with individual
differences (𝐷𝑖)
▸ average difference is calculated and the individual
deviations of each from are
used to compute a standard
deviation, sd
paired t-test
To calculate the variance ratio needed in the F test, it is necessary to
obtain several other quantities– the sums of squares:.
- The sum of the squares due to the factor SSF
- The sum of the squares due to error SSE
- The total sum of the squares SST is obtained as the sum of SSF and SSE: SST = SSF + SSE
Assumptions:
1. equal variance (variances of the I populations are assumed to be
identical-tested through Hartley test)
- the largest s should not be much more than twice the smallest
s for equal variances to be assumed.
- each of the I populations is assumed to follow a Gaussian distribution
If 𝑭𝒄𝒂𝒍𝒄 > 𝑭𝒕𝒂𝒃𝒍𝒆, then the
variances being compared are
SIGNIFICANTLY DIFFERENT
If 𝑭𝒄𝒂𝒍𝒄 < 𝑭𝒕𝒂𝒃𝒍𝒆, then the
variances being compared are
STATISTICALLY THE SAME
a statistical measure (expressed as a number) that describes the size and
direction of a relationship between two or more variables.
correlation
A correlation between variables does not automatically
mean that the change in one variable is the cause of the
change in the values of the other variable.
indicates that one event is the result of the occurrence of
the other event(i.e. there is a causal relationship between
the two events)
This is also referred to as cause and effect.
causation
study examples on ppt and solve chapter 7 problems
