Exam 2 Flashcards
Practices that lead to misleading graphs
- truncated graphs
2. improper scaling
What is a truncated graph? what precaution should be taken with them?
A graph where the vertical axis does not start at 0, that causes bars to be out of proportion. The illustrator should include a special symbol to signify that graph truncation
Where does improper scaling occur the most?
pictograms
Guidelines for constructing effective graphs
- Title and axes labels
- Start vertical axis at 0 if possible
- Use caution with figures and pictograms
- If variables differ greatly, consider another graph or plotting relative sizes
- Use simplicity and clarity
Parts of a graph analysis
- purpose of graph
- are results observational or experimentally obtained
- what variable is measured and is it quantitative or categorical
- what type of data display?
- Can SOCS be used to describe the data if it’s numerical
- Is data displayed correctly and is the graph misleading?
explanatory variable
variable that is manipulated/experimented with
response variable
variable that measures the outcome of interest
lurking variable
unobserved variable that influences the association between explanatory and response variables and is associated with both of those variables
Designed experiment
An experiment where researchers impose treatments and controls. These can help establish causation
Observational study
A study where researchers observe characteristics and take measurements, these can only reveal association or correlation
Advantages of experiments
- Reduces chance of lurking variables affecting results
- Effect of an explanatory variable on a response variable is more accurately determined, it is easier to adjust for lurking variables
- best method for determining causality
sampling frame
a list of all members of a population
sampling design
method used to obtain a sample
random sampling
employs a random device to select a sample, each member of a population has an equal chance of being selected for the sample
Simple random sample
(SRS) each possible sample of a given size has the same chance of being selected, can be done with or without replacement.
What is the difference when SRS is performed with replacement vs. without replacement?
With replacement: a member of a population can be chosen more than once
Without replacement: a member of the population can only be selected once
Margin of error
Gives a range of plausible values for the population parameter, helps you determine how accurate results are, denoted by E, represents precision at a confidence level, half the width/length of a confidence interval
How to find the range of plausible values using a margin of error
Add and subtract the margin of error from the middle value
Approximate margin of error formula
1/(n)^(1/2)
Potential sources of bias in surveys(just a list of the types not definitions)
- sampling bias
- nonresponse bias
- response bias
Sampling bias
Bias that occurs in surveying when the sampling method does tends to obtain non-representative samples, including under coverage and overcoverage
Undercoverage
occurs when sampling frame does not represent parts of a population, some portion(s0 of the population are not sampled or get smaller representation than it has int he population
Overcoverage
Occurs when members that are not in the population of interest are included in the sample
Nonresponse bias
Bias that occurs in surveying when sampled subjects can’t be reached or refuse to participate, including when those who respond do not respond to certain questions resulting in missing data.
Response bias
Bias that occurs in surveying when the wording of a question is confusing, the question is asked in a misleading way, or subjects lie because they think their response is socially unacceptable
LIST of poor ways to sample
- Convenience sample
- Volunteer sample
- Large, non-representative sample
Convenience sampling
a poor method of sampling, includes individuals who are easy to sample and therefore, may not represent the whole population
Volunteer sample
a poor method of sampling, most common type of convenience sample, difficult to define sampling frame, may not represent the population because people who volunteer tend to have stronger opinions about the issue
Large non-representative sample
a poor method of sampling, sample size doesn’t matter if it’s not representative of the population
Questions to be asked when assessing the validity of surveys
- How was the sample selected?
- Sample size?
- Nonresponse rates?
- How are the questions worded-how many, confusing, misleading, controversial?
- Who sponsored the study?
treatment group
group that receives the treatment or experimental condition
placebo
a “fake” treatment that looks just like the treatment being tested, ensures that treatments appear the same to the subjects so that control subjects don’t know they are in the control group
placebo effect
subjects treated with a subject sometimes improve
single blind
subjects don’t know which groups they’re in
double blind
subjects and data collectors don’t know which group the subjects are in
Perks of randomization
eliminates bias, balances the groups on variables that may affect the groups, both known and unknown by researchers
statistically significant
when differences in an experiment are larger than the differences that result from randomization alone
Four principles of good experimental design
- control
- randomization
- replication
- Blocking(optional)
experimental units
people in the study
Things that can go wrong in an experiment
- making generalizations out of convenience
- sample isn’t representative
- no volunteers
- carefully evaluate displays
Systematic sampling characteristics
- Less expensive
- order of a list can not be associated in any way with the responses sought
- beware of confounding variables
When is cluster random sampling preferred?
when a reliable sampling frame is not available or when the cost of an SRS is too high
Cluster random sampling sampling
- Split the population into representative, heterogenous groups called clusters
- Use random sampling to select several clusters
- Perform a census of each selected cluster
Stratified random sampling
- stratify the population into homogenous groups
- SRS is used to choose members from each strata
- Combine the groups from each strata to form your sample
Multistage sampling
sampling schemes combining several methods
Types of observational studies
Retrospective observational studies: Look into the fast
Case-control study: a type of retrospective study, often used in medical research. Subjects who have a response outcome are referred to as cases and subjects who have the other response outcomes are referred to as cases
Prospective observational study: looks into future, aka cohort studies
Cross-sectional: sample survey of a cross section of a population in current time
Experimental design diagrams
enables a quick comparison of results, can use only number of groups for the explanatory variable
Purpose of matching and blocking
There are two ways researchers can balance the effects of potential lurking variables
Matching
used in observational studies, attempts to achieve the balance that randomization achieves, subjects are paired due to similarites not being studied, includes case control studies
Matched-pairs
used in experiments, subjects paired with themselves, each treatment is observed for each subject, pre test/ post test/ cross-over designs
Blocking
used in experiments, groups similar experimental units together, randomized, reduce potential bias, treatments are usually randomly assigned within a block
What does a statistic describe?
a sample
Statistical inference
uses sample data to draw conclusions about a population, involves probability calculations on a sampling distribution of a statistic, requires random sampling or randomization
point estimate
single number, representing our best guess for the parameter, for any particular parameter, there are several possible point estimates depending on the sample selected
Interval estimate
a range of plausible values for the parameter, consists of a point estimate and margin of error
Properties of point estimates
- unbiased
- small standard deviation
- likely precision
- high confidence level
standard error,
abbreviated SE, use of a statistic to compute the standard deivation of the sampling distribution, different for means and proportions
Z-scores for 0.90, 0.95 and 0.99 confidence levels
- 90: 1.645
- 95: 1.96
- 99: 2.576
Steps for constructing a confidence interval for one population proportion
- Check assumptions
- Calculate confidence interval
- Interpret confidence interval
Confidence interval assumptions
- Data is obtained by randomization
2. Large enough sample sizeL at least 15 successes and failures
What determines length of a confidence interval?
the precision of the estimate(wider=less precise)
Relationship between confidence interval and precision
indirect
when do we use a t-distribution?
when we estimate the population standard deviation with the sample standard deviation
Total area of z and t distributions
1 for both
Curve shape of z and t distributions
extends indefinitely in both directions and approaches the horizontal axis asymptotically for both
distribution of z and t curves
z is normally distributed and t is not normally distributed
Mean and standard deviation of z and t distributions
Mean=0 for both. standard deviation=1 for z and is greater than 1 for t
What causes variation in values on z and t curves?
Z: caused solely by variation of sample means
t: variation of sample means and sample standard deviations
How does sample size effect distribution in z and t curves?
Z: same distribution regardless of size
t: different distribution for each sample size, identified by degrees of freedom or n-1
One mean t-interval assumptions
- Data obtained randomly
2. Normal population or large enough sample size
What is useful about t-intervals in terms of assumptions?
it is robust to moderate violations of the normality assumption
What may cause a t-interval to not work even though it is robust?
outliers since sample mean and sample standard deviation are both susceptible to outliers
one mean t-interval procedure
- Check assumptions
- Calculate confidence interval
- Interpret CI in context
How is an outlier handled in good statistical procedure
Inferential procedures are run with and without outliers
What affect margin of error?
Margin of error is affected directly by standard error and indirectly by sample size, because sample size affects standard error
sample size factors
- Margin of error
- Desired precision
- Confidence level
- Variability in the data
- Cost of obtaining a sample
Sample size formula and how to use it when determining sample size for p
n=P(1-P)Z^2 / E^2
Use the probability you have. If you don’t have one, use 0.50 because it will give you the largest possible sample size so you will definitely have a large enough sample
relationship between confidence interval and sample size
direct when precision is held constant
Sample size formula for u
n=(standard deviation)^2(z)^2 / E^2
limitation of t-procedures for one mean
Valid for any n, use caution for small sample sizes, does not work with outliers or highly skewed data
limitation of CI for one proportion
requires at least 15 successes and failures, if sample size is too small, P will not be normal
1-proportion plus-4 z-interval
CI formula is still valid if we use it after adding 2 to the original number of successes and failures, adding 4 total to the sample size. This moves the sample proportion towards 1/2, used with confidence levels over 90% and sample sizes over 10
Things not to say about confidence intervals
- Suggest that a parameter varies
- Making claims about a sample proportion
- asserting that the population proportion cannot be outside your interval
- Overgeneralizing results
CI flaws
- Not all confidence intervals will capture the true parameter value
- Whole interval is not treated equally
- Margin of error is not small enough to be useful
- assumptions are violated
Bootstrap
Allow you to construct confidence intervals when: it is difficult to find the SE, and when the CI interval doesn’t work well, it is a simulation method where the population is viewed as many, many copies of the original sample(data distribution)
Bootstrap method
- Resample with replacement the original sample to produce a bootstrap sample, compute point estimate of parameter
- Resample a very large number of sets of n observations from the original data distribution(at least 10,000) with point estimates for each of the samples
- Create a distributions of the point estimates to produce a bootstrap distribution
Methods of using a bootstrap method to estimate a CI
- Standard error method
2. Percentile method
Standard error method
estimate SE by using standard deviation of bootstrap distribution
Percentile method
use a percentile of the middle area of the distribution to create a confidence interval(usually 95%)
Significance tests
aka hypothesis test, uses sample data to decide between two competing claims about a population characteristic, uses probability to determine the plausability of a parameter, consider evidence based on sample data
Two possible conclusions of a significance test
Reject Ho or fail to reject Ho
Null hypothesis
Ho, always specifies a single value for the parameter, it is a claim about a population parameter that is initially assumed to be true
Alternative hypothesis
Ha, depends on purpose of hypothesis test, number appearing in alternative hypothesis is identical to number appearing in null hypothesis,
Two-Tailed test
Test to determine whether a population proportion is difference from a specified value
left tailed test
test to determine whether a population proportion is less than a specified value
right tailed test
test to determine whether a population proportion is greater than specified value
Assumptions/conditions of CLT
- representative sample
- independent sample values
- sample size if sufficiently large
- We have sampled less than 10% of the population
steps of a significance test
- Assumptions
- Define parameter and hypotheses before gathering or looking at data
- Calculate a test statistic
- Obtain a p-value
- State a conclusion in context
steps of a significance test
- check assumptions
- hypotheses/significance level
- compute test stat
- Find the p-value
- State the conclusions: report test stat and p-value, interpret results in context
Other names for a significance test for a one population proportion
one-proportion z-test, one-sample z-test for a population proportion
Assumptions for a one population proportion significance test
- Variable is categorical
- data is obtained randomly
- sample size is sufficiently large
P-value definition
probability of obtaining values that are more extreme than the observed test statistic value if the null hypothesis is true
other names for a significance test for one population mean
one-mean t-test, one-sample t-test for a population mean
Assumptions for a significance test for one population mean
- Variable is quantitative
- data is obtained randomly
- Population distribution is approximately normal according to CLT
What does a p-value that is greater or less than the significance level indicate?
P-value less than significance level: unsual result, reject null hypothesis
P-value greater than significance level: sample data is not unsual, fail to reject null
Relationships between conclusions of two-sided significance tests and CI’s
they are consistent
What must be true of a two-sided test says you can reflect the hypothesis that = 0?
then 0 is not in the corresponding confidence interval
What must be true of the p-value is >0.05 in a two-sided test?
a 95% confidence interval will contain the H0 value
What must be true of the p-value is <0.05 in a two-sided test?
a 95% confidence interval will not contain the H0 value
In what situation will a confidence interval definitely not contain H0?
i the p-value is less than the significance level in a two-sided test, a (1-a)x100 CI does not contain the H0 value
with what samples will a one-mean t-test work best?
large samples
What can cause a one-mean t-test to not work?
Skewed distributions and outliers
relationship between sample size and p-value
indirect
What causes decisions in significance tests to have uncertainty?
sampling variability
4 possible results in the decision of a significance test?
- do not reject true H0
- Reject true H0
- Do not reject false H0
- Reject false H0
Type 1 error
error when a true H0 is rejected
Type 2 error
error when a false H0 is not rejected
What is the only sure way to prevent Type 1 and 2 errors?
a census
What is the biggest issue with type 1 and 2 errors?
you have no way of knowing you committed one before consequences are experienced
What does the alpha symbol stand for? what rule comes with it?
a=the probability of committing a type 1 error, the significance level of a hypothesis test
the significance level must be set before running the test
What does the Beta symbol stand for? What are some of its characteristics?
B=probability of committing a type 2 error
this value is usually not known, has an inverse relationship with alpha
What determines the reasoning behind setting a significance level?
trying to balance the risk of committing a type 1 or type 2 error, which consequences are more serious is taken into account
Is a CI or significance test more informative? why or why not?
A CI is more informative because because it displays the entire set of plausible values, while a significance test just tells you whether a specific value for H0 is plausible
What must you always do after a null hypothesis is rejected?
use a CI to estimate the value of the parameter, then compare the hypothesized value to the CI values to determine how far the parameter is from the hypothesized value
What if the parameter value is very close to the hypothesized value of significance test?
the result may not be practically significant
Two proportion z-tes5/z-interval assumptions
- Variables are categorical
- Independent random samples from both groups
- 10 successes and failures
What if 0 is in the CI for proportion intervals?
it is plausible that P1=P2
What if 0 is not int he CI for proportion intervals?
If all values in the CI are positive, P1>P2. If all values in the CI are negative, P1
What does the magnitude of a CI tell you? What do small magnitudes indicate?
how large the true difference is, small magnitude equals a small difference in practical terms
Tests for comparing two proportions with a categorical response
- Two-proportion z-test
2. Two-proportion z-interval
Tests for comparing two means with a quantitative response
- Independent samples t-test
2. independent samples t-interval
independent samples t-test/t-interval assumptions
- Quantitative response variable for both groups
- independent random samples
- Normal or large enough samples, no outliers
Pooled t-test assumptions
- Quantitative response variable
- independent random samples
- normal population or large enough sample
- equal standard deviations
Pooled t-interval assumptions
- quantitative response variable for two groups
- independent random samples
- normal or large enough distribution
- standard deviations are equal
What purpose do BOTH pooled and non-pooled t-tests share?
both tests are used to compare the means of 2 populations based on independent random samples
What is the purpose of using a pooled t-test? what is its drawback?
increases statistical power, decrease chance of type 2 error, increases chance of type 1 error when used when standards deviations are not equal
simple random paired sample
each possible paired sample is equally likely to be selected
Purpose of using dependent samples
used when members of 2 populations have natural pairings, remove extraneous sources of variation, most likely to detect difference between population means when such difference exist
Paired t-test/t-interval+assumptions
d=difference
mean=u1-u2 difference between two population means
Assumptions
- Simple random paired sample
- Normal distribution of differences or large enough sample for what is believed to be the shape of the distribution of the differences
assumptions for just a confidence interval
- Sample is randomly obtained
2. Large enough sample: At least 15 successes and 15 failures
F distribution
infinitely many F distributions identified by its number of degrees freedom, has a numerator and denominator degree of freedom, total area under curve=1, curve is right skewed
What does ANOVA stand for?
Analysis of variance
How does ANOVA work?
ANOVA compares means of a variable for populations from a classification by a categorical explanatory variable(factor) and level(possible values of a factor)
What variation does ANOVA compare?
It compares variation between samples to variation within samples
What information can you get from comparing “between: and “within” variation?
If “between” > “within” , not all population means are equal
total sum of squares
measures total variation, can be partitioned into between(SSTR) and within (SSE) samples. SST=SSTR+SSE
Assumptions for one way ANOVA
- data is randomly obtained
- independent samples
- normal or large enough samples(robust)
- Equal population standard deviations
- Largest/smallest<2
- robust provided sample sizes are equal
How to find F, df1 and df2 in ANOVA analysis
F=MSTR/MSE
df1=g-1
df2=N-g
post hoc analyses
Analyses conducted after the initial analysis to see which means actually differ, compare each mean as a pair using their confidence intervals
Why is random sampling important?
it ensures samples are independent
What sample size is large enough to ensure normality?
30
how to determine which confidence interval to use for a one sided significance test?
1-2(significance level)
How do we know which confidence interval to use for a two sided test?
1-significance level=CI
Difference between lurking and confounding variables
lurking variables are not accounted for in the study, confounding variables are
