Study Guide for Examination

Question 1

Survey research is a powerful tool that researchers use to …

Answer 1

gather data

Question 2

“Survey is a systematic method for (…) from (…) for the purposes of constructing quantitative descriptors of the attributes of the larger population of which the entities are members.”

Answer 2

1. gathering information (data) and 2. (a sample of)
2. (individuals, households or other)
3. the larger population

Question 3

What are some charac/teristics of a survey

Answer 3

Information is gathered primarily by asking people questions

Information is collected either by having interviewer ask questions and record answers or by having people read or hear questions and record their own answers

Information is collected from only a subset of the population to be described—a sample—rather than from all members

Question 4

What is a population?

Answer 4

The entire group of individuals about which we want information

Question 5

What is a sample?

Answer 5

The part of the population from which we collect information and is used to draw conclusions about the whole population

Question 6

The census is used to…

Answer 6

count and map population, for purpose of planning and forecasting, taxation and representation.

Question 7

Who is the father of modern surveys?

Answer 7

Charles Booth

Origins of modern survey research can be traced to Booth.
Conducted survey from 1889 to 1903 to learn
Reasons why people were poor in London
Very detailed descriptions of live in London based on interviews and observations
Poverty maps

Question 8

What is the difference between polls and surveys?

Answer 8

No clear distinction between the two terms…essentially do the same thing, but conducted by different actors…

Polls are used/conducted by the private sector to gather opinion.
Survey are used/conducted by the government and by researchers in academia who emphasize the scientific or scholarly character of their work.

Question 9

Why is it to hard to measure sensitive things such as discrimination? What can you do to reduce the bias of measuring something sensitive?

Answer 9

Respondents are reluctant to talk about personal things. The best way to reduce bias is personal interviewers are the best way to ask sensitive questions, because inter‑viewers have an opportunity to build rapport and to establish the kind of trust that is needed for respondents to report potentially sensitive information.

Question 10

When to do a survey?

Answer 10

Do you know enough about the topic?

You must understand topic to know what specific, structured questions to ask people.
When knowledge of the topic is rudimentary, you might want to do a focus group or qualitative interviewing.

Question 11

What is reliability?

Answer 11

Reliability (whether it works everytime)

Question 12

What is validity?

Answer 12

whether it measures what it should measure

Question 13

What are variables?

Answer 13

is a measure of quantity (Is the characteristic of the individual to be measured or observed). In survey research we distinguish between 4 types

Question 14

What is a descriptive variable?

Answer 14

those which are just to be reported on, with no conclusions drawn about causality or inference.

Question 15

What is a dependent variable?

Answer 15

caused or influenced by others

Question 16

What is a independent variable?

Answer 16

doing the causing or influencing

Question 17

What are extraneous variables?

Answer 17

provide an alternative causal explanation, casting doubt on the one we are advancing.

Question 18

What do we mean by “Total Survey Design” ? What are the three components?

Answer 18

In high-quality, modern surveys, focus on all 3 components

1. Sampling
2. Question Design
3. Data Collection

Question 19

What are the characteristics of a good research question?

Answer 19

Answerable using surveys
Descriptive or causal in nature
E.g. How interested are young people in politics? (Descriptive)
Does interest in politics influence voting behaviors? (Causal)

Question 20

Difference between systematic bias and random error?

Answer 20

Random errors usually result from the experimenter’s inability to take the same measurement in exactly the same way to get exact the same number. Systematic errors, by contrast, are reproducible inaccuracies that are consistently in the same direction.

Question 21

Sampling error

Answer 21

A sampling error is a statistical error that occurs when an analyst does not select a sample that represents the entire population of data and the results found in the sample do not represent the results that would be obtained from the entire population.

Question 22

What is a non response error?

Answer 22

Nonresponse error in surveys arises from the inability to obtain a useful response to all survey items from the entire sample. A critical concern is when that nonresponse leads to biased estimates. … The NCVS, like most federal household surveys, is voluntary and not required by law.

Question 23

How to reduce non-response error?

Answer 23

Better questionnaires, better modes, better training of interviewers help!

Question 24

What is coverage error?

Answer 24

Occurs when target population does not have a convenient sampling frame.

Question 25

What is measurement error?

Answer 25

occur when one cannot adequately capture the construct in question
Misunderstanding the question
Not having information to answer
Distorting the answers to look good

Question 26

What is response bias?

Answer 26

is the tendency of a person to answer questions on a survey untruthfully or misleadingly. For example, they may feel pressure to give answers that are socially acceptable

Question 27

What are the modes?

Answer 27

In-person
Telephone
Mail
Web
Phone apps
combinations of these: mixed mode

Question 28

What is a sample frame?

Answer 28

is a set of people that has a chance to be selected, given the sampling approach that is chosen. A sample can only be representative of the population included in the sample frame.

Question 29

What are the two different types of sampling methods?

Answer 29

Probability and non-probability

Question 30

What is simple random sampling?

Answer 30

Each element in the population has an equal chance of selection

Acquire a sample frame
Number all members of the population from 1 to n
Calculate your sample size (not covered in PAF 3105)
Using a random number generator, select the number of cases corresponding to your sample size

Question 31

Systematic random sampling

Answer 31

Every nth member from sample frame is selected

E.g. Customers at the Duane Reade on 14th st

Determine number of entries in list: 25
Sample size: 5
Divide latter/former: 5/25 (i.e. 1/5). Select every 5th person
Choose random start from 1 to 5
Take every 5th member

Question 32

Stratified Sampling

Answer 32

Subsamples are drawn from different strata

Ensures that proportions in each strata of sample are identical to the population
Nearly always done with National samples in the US to get representation of different States

Example
List all Baruch Undergraduate students
Arrange list by class: freshman, sophomore, junior, senior
Use Simple Random Sampling to select your sample

Question 33

What is sampling distribution?

Answer 33

A sampling distribution is a probability distribution of a statistic that is obtained by drawing a large number of samples from a specific population. Researchers use sampling distributions in order to simplify the process of statistical inference.

Question 34

What is sampling distribution?

Answer 34

A sampling distribution is a probability distribution of a statistic that is obtained by drawing a large number of samples from a specific population. Researchers use sampling distributions in order to simplify the process of statistical inference.

Question 35

What is a confidence interval?

Answer 35

range of values that is likely to contain the actual population proportion

Question 36

What is snowball sampling?

Answer 36

Contact initial respondents who then provide information on other respondents who have similar characteristics

E.g. Survey of illegal drug users. Drug users usually know others who use drugs. After identifying a first drug user, ask for contacts of others

Question 37

What is purposive sampling?

Answer 37

Individual selects the sample based on his or her judgment about some appropriate characteristics required of the sample member.

E.g. We recruit a set of drug users to survey since a random sample of drug users in the general population is very low.

Question 38

What is Quota Sampling?

Answer 38

Individual selects the sample based on his or her judgment about some appropriate characteristics required of the sample member.

E.g. We recruit a set of drug users to survey since a random sample of drug users in the general population is very low.

Question 39

What are some examples of hard to reach populations?

Answer 39

Commercial Sex Workers
Migrants
Undocumented immigrants
HIV positive and other disease positive
disabled
drug addicts
gang members

Question 40

What sampling method is often used for hard to reach populations

Answer 40

Non-probability methods are more typical.

Snowball sampling for example would be useful for explorative, qualitative and descriptive work.

Question 41

What are unstructured interviews?

Answer 41

Unstructured Interview Unstructured interviews do not use any set questions, instead, the interviewer asks open-ended questions based on a specific research topic, and will try to let the interview flow like a natural conversation. … They are sometimes called informal interviews.

Question 42

What are semi-structured interviews?

Answer 42

The semi-structured interview is a qualitative data collection strategy in which the researcher asks informants a series of predetermined but open-ended questions. … Researchers who use semi-structured interviewing develop a written interview guide in advance.

Question 43

What are some examples of probe questions

Answer 43

Say more about that
Explain that some more
Can you give me more detail?

Question 44

How should the interview for hard to reach populations be conducted?

Answer 44

snowball sampling, respondent-driven sampling (RDS)

Question 45

What is Respondent Driven Sampling?

Answer 45

Contact initial participants (“seeds”) who refer others to the study.

Question 46

Potential advantages of personal interviewing

Answer 46

most effective way of enlisting cooperation
build rapport and confidence are possible
probing for adequate answers

Question 47

Potential disadvantages of personal interviewing

Answer 47

It is likely to be more costly than the alternatives. • A trained staff of interviewers that is geographically near the sample is needed. • The total data collection period is likely to be longer than telephone procedures. • Some samples (those in high‑rise buildings or high‑crime areas, elites, employees, students) may be more accessible by some other mode.

Question 48

Potential advantages of telephone interviewing

Answer 48

• Lower unit costs than personal interviews. • Random‑digit dialing (RDD) sampling of general populations. • Better access to certain populations, especially compared to personal interviews. • Shorter data collection periods.

Question 49

Potential disadvantages of telephone studies:

Answer 49

Sampling limitations, especially the complexities of sampling those without landline tele‑phones or inability to contact those on list if correct phone number cannot be found. • Nonresponse is increasing concern and is usually considerably higher than personal interviews. • Questionnaire or measurement constraints, including limits on response alternatives, use of visual aids, and interviewer observations. • Possibly less appropriate for personal or sensitive questions.

Question 50

Potential advantages of self‑administered (versus interviewer‑ administered) data collections:

Answer 50

• Ease of presenting questions requiring visual aids (in contrast to telephone interviews). • Asking questions with long or complex response categories. • Asking batteries of similar questions. • The fact that the respondent does not have to share answers with an interviewer makes col‑lection of sensitive data likely more valid.

Question 51

Potential disadvantages of self‑administration:

Answer 51

Especially careful questionnaire design is needed. • Open questions usually are not useful. • Adequate reading and writing skills by respondents are needed. • The interviewer is not present to exercise quality control with respect to answering all ques‑tions, meeting question objectives, or the quality of answers provided. • Difficult to control who answers the questions for address‑based samples.

Question 52

Advantages of group administration:

Answer 52

Generally high cooperation rates. • The chance to explain the study and answer questions about questionnaire (in contrast to mail surveys). • Generally low cost.

Question 53

Advantages of mail procedures:

Answer 53

• Relatively low cost. • Can be accomplished with minimal staff and facilities. • Provides access to widely dispersed samples and samples that for other reasons are difficult to reach by telephone or in person. • Respondents have time to give thoughtful answers, to look up records, or to consult with others. • Response rates are often quite high, depending on the population and the topic.

Question 54

Disadvantages of mail procedures:

Answer 54

• Response rates can be quite low (depending on group to be studied and the topic). • Various disadvantages of not having interviewer involved in data collections. • Need for accurate mailing addresses for sample.

Question 55

Potential advantages of internet surveys:

Answer 55

• Low unit cost of data collection. • Potential high speed of returns. • All the advantages of a self‑administered instrument. • All the advantages of a computer‑assisted instrument. • Like mail surveys, providing time for thoughtful answers, checking records, or consulting with others.

Question 56

Potential disadvantages of internet surveys:

Answer 56

• Limited to samples of Internet users. • Need for comprehensive address lists. • Challenges of enlisting cooperation (depending on sampled groups and topic). • Various disadvantages of not having interviewer involved in data collection.

Question 57

What are open ended questions?

Answer 57

Respondents must create their own answers and state them in their own words:
Benefit: allows them to explain fully their views, & usually requires an interviewer (good? Bad? Depends?), fewer questions can be asked

Question 58

What are Closed Ended questions?

Answer 58

Respondents are asked to state which response most closely represents their answers to question.
Benefit: Provides more uniformity of responses & are more easily processed than open-ended questions, can answer many questions

Question 59

What are the three core principles of ethical research involving human subjects?

Answer 59

respect for persons, beneficence and justice.

Question 60

What is non-response bias?

Answer 60

bias occurs when non-responders from a sample differ in a meaningful way to responders (or early responders). This bias is common in descriptive, analytic and experimental research and it has been demonstrated to be a serious concern in survey studies.

Question 61

How to reduce non response bias?

Answer 61

Design your survey carefully; use well-trained staff and proven techniques.
Develop a relationship with respondents. …
Send reminders to respond.
Offer incentives to respond.
Keep surveys short.

Question 62

How to calculate the response rate?

Answer 62

The response rate can be calculated by dividing the number of completed survey responses by the number of people who viewed or started the survey. To convert this to a percentage, multiple your final number by 100.

Question 63

What is socially desirable bias?

Answer 63

is a type of response bias that is the tendency of survey respondents to answer questions in a manner that will be viewed favorably by others. It can take the form of over-reporting “good behavior” or under-reporting “bad”, or undesirable behavior

Question 64

What should be in a cover letter?

Answer 64

Brief and generic description of research (satisfy respondents curiosity)
Promise confidentially (maybe anonymity)
Tell respondents their opinion is important
If giving remuneration to respondents, discuss it here
Moral appeal to honesty (science is harmed if people lie)
Give respondents your contact information in case they need to clarify something

Question 65

How can you evaluate questions to make sure that they are reliable and valid?

Answer 65

Questions measure underlying concepts as intended (valid)

Questions perform the same way repeatedly (reliable)

Question 66

What are focus groups?

Answer 66

Focus groups are a research method used to gather feedback and opinions from customers. Each person in the group is encouraged to participate in a discussion .

Question 67

What is cognitive testing?

Answer 67

purpose of cognitive testing is to investigate how well questions perform when asked of survey respondents, that is, if respondents understand the question correctly and if they can provide accurate answers

Question 68

What is a traditional field test?

Answer 68

a test carried out in the environment in which a product or device is to be used.