Errors Flashcards
What is sampling error?
Sampling error is also known as sampling variance.
This is the error incurred through using a sample survey to produce estimates for the population. It is the difference between an estimate from a sample survey and the true value for the population. This is because samples don’t always reflect a population’s true characteristics because of random variation in sample composition, even if the sample is drawn using the same approach.
Larger samples have less chance of producing results that are uncharacteristic of the population as a whole.
What is non-sampling error?
Error not caused by taking a sample. This can therefore affect both sample surveys and census surveys and will lead to error in the survey findings.
Non-sampling error consists of variable errors (which cancel each other out) and bias/systematic error which consists of selection bias, non-response bias and response bias.
What is bias error?
A type of non-sampling error. Shift the responses in one direction or the other meaning that they shift the survey results in one direction. Also known as systematic error. Not to be confused with bias/unbiased sample which is about the sample being representative of the population (although they do link in that they both affect accuracy of findings).
3 types of bias error are:
Selection bias - the tendency to exclude a certain type of unit from the sample.
Non-response bias - if selected units in the sample do not respond.
Response bias - error through inaccurate answers being given. This could be caused by misleading or ambiguous questions. Poorly designed questionnaires, respondent bias and interviewer bias.
What is variable error?
These are non-sampling errors where the responses are just as likely to be too big as they are too small.
For example, ONS asks businesses to round their turnover values to the nearest thousand pound. Some round up and some round down, so the errors balance each other out.
What is interviewer bias?
A type of response bias which is a non-sampling bias error. It is the error introduced into the survey through the interviewer misleading or prompting a respondent into an answer. This can also occur because of the respondent being affected by the mannerisms, appearance, gender, accent and opinions of the interviewer.
What is non-response bias?
A type of non-sampling bias error which occurs when a unit in the sample does not respond to the survey. You can attempt to replace the unit with a ‘similar’ one. Keep in mind that non-probability sampling can also have non-response bias as even though you can keep asking more people to participate, the ‘refusers’ are not represented.
What is survey error (could be called total error as is any kind of error)?
This is any error that can creep into the survey that may lead to an eventual loss in the accuracy of the survey estimates and the extent to which the findings represent the target population. The total error in a survey estimate is the difference between the estimate derived from the data collected and the true value for the population. The total error can be divided into two main types: systematic (non-sampling) and random error (sampling error).
What is section bias?
The tendency to exclude a certain type of unit from the sample. This is a non-sampling bias/systematic error
It can arise from inadequate sampling frames making the survey results unrepresentative of the target population and the survey estimates may be biased.
It can be reduced by regularly up-dating the sampling frame and selecting well-designed probability sampling.
What is systematic error?
This is another word for bias error and is a type of non-sampling error that shifts the responses in one direction so shifts survey estimates in one direction. For example, through mistakenly collecting data in inches when the estimate is for cm.
This includes selection bias, non-response bias and response bias.
What is respondent bias?
This is when the respondent gives inaccurate answers because they are protecting their privacy and may give what they deem to be socially acceptable answers.
What is response bias?
This is when the respondent does not give accurate answers. This could happen because of poor questionnaire design, misleading questions, respondent bias and interviewer bias.
Can a larger sample help with error?
Not necessarily.
If there is a lot of variance in the population then a larger sample would help to capture this.
However, a smaller, representative sample with less potential for non-sampling error is usually more cost effective than a large sample which will require more effort to make it effective. If you don’t get the right information from the right people then it doesn’t really matter how much information you get from how many people.
You also need a high-quality data collection methods - if your questionnaire is crap then a huge sample will still lead to inaccurate findings.
How can you try to avoid sample bias?
- Define the population of interest prior to drawing the sample
- Attempt to maximise population coverage by selection a sample that fairly represents the entire population
- Obtaining responses from as much of the selected sample as possible by lowering non-response
What’s the difference between sampling error and sample bias?
Sampling error is caused by natural/ random variations in different samples drawn in the exact same way as samples don’t always reflect a population’s true characteristics. This can be seen as a natural phenomenon.
Sample bias refers to the possibility that members of a sample differ from the larger population in a systematic way through things like excluding members of the population through data collection methods such as online surveys excluding those who do not have Internet and non-response. This is therefore caused by researcher error.
What are accuracy are precision?
Accuracy is a measure of how close the estimate is to the true value of the population. If there are various non-sampling errors then this will cause a decrease in accuracy. An unrepresentative sample would also decrease accuracy. Accuracy is 100% if a perfect census is carried out.
Results from a survey are precise if similar results are obtained with repeated measurement. If you carried out the survey multiple times on the same population and the results were close together then you have evidence of precision. Precision is determined but how much natural variation there happens to be in the population and the extent to which this is represented in your samples. By chance alone, different samples will return different results. A larger sample will help with precision because of decreasing variability but the accuracy will need to be good also to ensure that it the right thing being measured amongst the right units.
Precision links to the confidence that you have the your findings will be applicable to the population every time you carry out the study on a new sample.
