Week 1 Flashcards
3 Types of bias
- Selection bias (includes coverage issues) - sample doesn’t adequately represent the target pop.
- Response bias - respondents do not provide truthful answers
- Under-coverage - sampling frame leaves out units that should’ve been included
- Non-response bias - when ppl of interest can’t be reached or refuse to participate
Survey vs Experiment - 8 differences
(from W10 slides also)
Survey
- RANDOM SAMPLE from finite population; selected individuals asked to respond to questions
- for OBSERVATIONAL STUDY/factual description, estimates of population characteristics
- no intervention
**reference to BIAS in observational studies arising from CONFOUNDING
4. Samples frequently large
5. Often employed in social and behavioural sciences
6. Seldom require equipment
7. Main problem often determining validity
8. Less costly
Experiment
- RANDOMLY assign INTERVENTIONS (treatment/control group) & compare outcome
- to study CAUSAL EFFECTS of interventions, scientific research
3. Samples often quite small
4. Often associated w/ natural and physical sciences
5. Usually require equipment
6. Main problem often analysing causality
7. Costlier
8. Data usually come from changes in outcomes, as influenced by the independent variables
Random sampling vs Quota sampling
Random sampling
- each pop. element has a KNOWN PROBABILITY of being selected in the sample (in advance)
*not necessarily equal prob.
Quota sampling
- well represent the characteristics of pop.
- NON-RANDOM sampling, but can be a better method if it’s representative
Survey errors: Sampling error + Non-sampling error
Explain the difference between sampling and non-sampling error.
[4m, 2012, 2011]
Sampling error - RANDOM VARIATION DUE TO sampling scheme
Non-sampling error - due to FAILURES OF the sampling scheme, eg. non-response bias.
Sampling error
- arises from differences between characteristics of sample & frame
- occurs in sample surveys (but not censuses)
- quantified through sampling BIAS & VARIABILITY
Non-sampling error
- in surveys and censuses
- can lead to bias
Includes:
1. Selection bias (eg. judgment, convenience, coverage problems)
2. Measurement error
eg. interviewer effect
3. Nonresponse {bias}
4. Processing error
eg. data entry, coding
Sources of error - Bias (systematic error)
- Differential nonresponse
- nonrespondents & respondents are DIFFERENT on the variable of interest - Measurement error
eg. poorly written Qs, lies, mistakes, coding errors
+ nonrepresentative sample
Sources of error - Precision (random error)
Larger sample size, lower variability and higher precision
Why is the non-probability sampling scheme generally a bad way to sample?
- Non-random
- Likely would lead to selection bias & a non-representative sample, due to way of choosing
eg. convenience, volunteer, snowball sampling (+ also quota)
Measurement error
Difference between the response given & the true value
Quota sampling
+ an advantage and 5 disadvantages
+ how this method relates to stratified sampling [5m, 2017, 2015, 2019]
Quota sampling is a method for selecting survey participants that is a NON-PROBABILISTIC version of stratified sampling.
- Sampling units divided into GROUPS, and a PREDETERMINED number (quota) of units from each group is chosen SUBJECTIVELY.
Advantage: quota help reduce bias relative to purely subjective sampling by controlling for composition of sample
Disadvantages:
1. sampling within quota can still lead to bias, due to a SUBJECTIVE element relative to probability sampling
2. Interviewer discretion (really a subtle form of accessibility sampling)
3. NON-RESPONDENTS are ignored
4. Choice of groups is arbitrary and may be unsuitable
5. Non-probability sampling, so no way to assess accuracy.
Explain briefly by using two examples how measurement error may lead to bias in surveys. [5m, 2018]
(examples from the lectures)
eg 1. The amount of alcohol use in surveys is lower than official sales
- leads to downward bias
eg 2. Drug use prevalence from surveys appears lower than police confiscations indicate is the case.
Other eg.s can be leading to upward bias
Selection bias
- Part of the target population is not in the sampled population.
- Selection probabilities are different from the ones decided by the researcher. (e.g. undercoverage leads to selection bias
4 Factors that reduce and control Sampling Error
- Employ PROBABILITY sample designs - elements in the sampling have known & non-zero prob. of being selected (to avoid selection bias)
- The sample is REPRESENTATIVE of key population characteristics that are important for the survey
- Large sample size
- Elements are drawn INDEPENDENTLY or in clusters
Example of differential nonresponse (for non-sampling error Q)
- Imagine girls study on average 10 hours per week, and boys 20 hours.
- Response rate for an entire class: female = 70%, male = 30%.
Therefore, underestimating the true value of no. of hours studied
Pro & con of probability sampling
Pro: Can remove biasing effect of sampling
Con: still face possible bias from NON-SAMPLING ERROR like nonresponse and measurement error
Simple random sampling
[2m, 2012, 2011]
- Sampling without replacement
- Every possible sample has an equal probability of being selected
