Chapter 7- Sampling- estimating the frequency of behaviors and beliefs Flashcards
External validity
Can the results of a study be generalized to some larger population of interest? Good external validity means that the sample is representative of the population
How is external validity calculated for frequency claims?
To interrogate external validity for frequency claims, we ask whether people who responded to a survey (the sample) can adequately represent the larger population or can be generalized to another setting.
Population
The entire set of people or products in which you are interested.
Sample
A smaller set taken from that population
Census
A census would include surveying every individual in the population- this usually isn’t necessary in research
Population of interest
Researchers have to specify which population they want to generalize- this is the population of interest. Researchers are rarely looking at the world’s entire population. For example, if a sample of people rated a style of shoes on how well they fit, we might be interested in generalizing to the population of people who have worn those shoes
Biased sample
Some members of the population of interest have a much higher probability than other members of being included in the sample.
Unbiased sample
All members of the population have an equal chance of being included in the sample. Only unbiased samples allow us to make inferences about the population of interest
When is a sample considered biased?
In a consumer survey or an online opinion poll, a biased sample would contain too many unusual people. Ex- the students who rate a professor on a website might tend to be the most angry or disgruntled and not represent the group of students. A researcher’s sample might include only one kind of people when the population of interest has more variety. Ex- a sample of only men when the population of interest contains men and women. Sometimes, the population of interest might actually be one specific group, like men. In this case, the researcher has to make sure that the same of men represents the general population of men
Ways a sample might be biased (2)
- Sampling only those who are easy to contact
2. Sampling only those who volunteer
Which validity does a biased sample threaten?
External
Convenience sampling
Using a sample of people who are easy to contact and readily available to participate. College professors conducting studies might tend to use college students, but these samples might not be representative of populations that are less educated, older, or younger. Researchers might also end up with a convenience sample if they are unable to contact a certain subset of people. They might not be able to study those who live far away or those who don’t show up to a study appointment
Self-selection
When a sample is known to contain only people who volunteer to participate. When Internet users choose to rate something, like an Amazon product or a professor on RMP, they are self-selecting when doing so- people who take the time to rate things might have stronger opinions or might be more willing to share ideas with others. We can rule out this bias when participants are invited via random selection
Probability sampling/random sampling
Every member of the population of interest has an equal and known chance of being selected for the sample, regardless of if they are convenient or motivated to volunteer.
Nonprobability sampling
Involves nonrandom sampling and results in a biased sample
Probability sampling techniques (5)
- Simple random sample
- Oversample
- Stratified random sample
- Systematic sample
- Cluster sample or multistage sample
Nonprobability sampling techniques (4)
- Convenience sample
- Quota sample
- Purposive sample
- Snowball sample
Simple random sampling
Some examples- the names of every member of a population are written on a plastic ball, and a machine spits out the number of balls necessary for a sample. A number is assigned to each individual in a population and then certain ones are selected using a table of random numbers- researchers use software to generate random numbers
Systematic sampling
Using a computer or a random number table, the researcher starts by selecting 2 random numbers (4 and 7, for example). If the population of interest is a roomful of students, the researcher would start with the fourth person in the room, counting off every seventh person until the sample is the desired size. The Mehl study is an example of this, since researchers sampled conversations every 12.5 minutes
Why are simple random sampling and systematic sampling not frequently used?
Simple random sampling and systematic sampling can be difficult and time consuming. It can be almost impossible to find and enumerate every member of the population of interest, so the next 3 are variants of the basic technique.
Cluster sampling
An option when people are already divided into arbitrary groups. Clusters of participants within a population of interest are randomly selected, and then all individuals in each selected cluster are used. Example- studying all public high schools (clusters) in a state. A researcher could randomly select 100 of those schools and then select students from each school
Multistage sampling
Two random samples are selected, a random sample of clusters and then a random sample of people within those clusters. High school example- the researcher would select 100 schools randomly, and then select a random sample of students from each of the 100 schools.
Stratified random sampling
Stratified random sampling- the researcher purposefully selects particular demographic categories (strata) and then randomly selects individuals within each of the categories, proportionate to their assumed membership in the population. Example- researchers want to make sure their sample of 1,000 Canadians includes people of South Asian descent in the same proportion as the Canadian population (4%). In a sample of 1,000, they would select 40 people of South Asian descent. All members of the sample would be selected randomly, including the South Asian individuals.
How is stratified random sampling different from cluster sampling? (2)
- Strata are meaningful categories (like ethnic or religious groups), while clusters are more arbitrary.
- The final sample size of the strata reflect their proportion in the population, while clusters aren’t selected with such proportions in mind.
Oversampling
The researcher intentionally over represents one or more groups. With the Canadian example, researchers might want to include more than 40 South Asian people to get a more precise estimate. However, the increased proportion of this group is still selected randomly. The final survey results will be adjusted so the oversampled group is weighted to their actual proportion in the population
Weighting
Researchers can adjust the data so categories of people that were underrepresented in the sample can be better represented.
Random
In research, random has a precise meaning- occurring without any order or pattern
Random sampling
Researchers create a sample using some random method, like drawing names from a hat, so that each member of the population has an equal chance of being in the sample. Random sampling enhances external validity
Random assignment
Only used in experimental designs. Individuals are randomly assigned to an experimental or control group. Enhances internal validity by ensuring that the experimental and control group have the same types of people in them, which controls for alternative explanations of outcomes.
When would researchers use an unrepresentative sample?
In cases where external validity is not vital to a study’s goals, researchers might be content with a nonprobability sampling technique. Convenience sampling is most common but not the only option
Purposive sampling
When researchers only want to study a particular group, they can recruit these participants in a nonrandom way. Limiting a sample to one group doesn’t necessarily make it purposive. Recruiting smokers by posting flyers at a tobacco store would make recruitment non random and therefore uses purposive sampling- this is not representative of all the smokers in the area
Snowball sampling
When participants are asked to recommend a few acquaintances for the study until the sample is large enough. This sampling is unrepresentative because people are recruited via social networks, which are not random
Quota sampling
The researcher identifies subsets of the population of interest and then sets a target number for each category in the sample. Next, the researcher samples from the population of interest non-randomly until the quotas are filled. Participants are generally selected through convenience or purposive sampling
Which validity is a priority for a frequency claim?
External validity is a priority- it is very important in order for frequencies for a specific population to be accurate
What is the best way to interrogate a frequency claim?
You usually can’t directly check the accuracy of a frequency claim because it’s usually impossible to survey every member of a certain population. The best way to interrogate a frequency claim is to examine the method the researchers used. As long as it was a probability sampling technique, you can be more confident in the external validity of the result.
If a frequency claim uses a nonprobability sample, what should you consider?
Researchers might not have the funds to obtain random samples for their studies and their priorities lie elsewhere. If a frequency claim is not based on a probability sample, you need to consider whether the reason for the sample’s bias is relevant to the claim.
How does a larger sample size affect external validity?
Larger samples are not more externally valid than smaller samples- how the sample was selected is more important. Large samples will sometimes be necessary in order to locate enough instances of a phenomenon, however.
What sample size is usually adequate?
When researchers do use random sampling, 1000-2000 people are usually all that they usually need, even for large populations
How does sample size affect the margin of error?
As a polling sample gets larger, its margin of error becomes more precise. However, the external validity of the poll comes from its sampling method, not the sample size
