Chapter 9: Non-experimental_survey

Question 1

For what purposes are surveys used?

Answer 1

• to determine how people feel about a particular issue, such as gun control or the performance of the president of the United States
• attempt to find out the effect of some event on people’s behavior.
• surveys provide an opportunity to examine correlations among the participants’ responses and to look for possible patterns of cause and effect, such as showing that bullying in childhood is related to internalizing problems later in life

Question 2

characteristics of survey

Answer 2

• First, surveys generally involve sampling.
• Second, surveys involve the self-reporting of behaviours, feelings, thoughts, and preferences, either in oral or written form.
• Third, all respondents answer the same set of predetermined questions to allow for comparison of responses.

Question 3

a sample

Answer 3

(i.e., a subset of individuals)

Question 4

population

Answer 4

all individuals of interest to the researcher

Question 5

sampling frame

Answer 5

a specific list of the individuals in the population

Question 6

Each individual who falls within the sampling frame is called

Answer 6

an element

Question 7

andom sample:

Answer 7

a sample in which every member of the population has an equal and independent chance of being selected

Question 8

haphazard sample:

Answer 8

population subgroup for whose selection the researcher uses hit-or-miss methods

Question 9

purposive sample:

Answer 9

a nonrandom sample that is chosen for some characteristic that it possesses

Question 10

convenience sample:

Answer 10

a nonrandom sample that is chosen for practical reasons

Question 11

A sample is representative of the population when the sample has the same — as the population

Answer 11

distribution of characteristics

Question 12

When some segments of the population are over-represented in the sample, — results

Answer 12

selection bias

Question 13

Face-to-Face survey advantages

Answer 13

• interviewers can establish rapport with the people being interviewed.
• Interviewers can direct the attention of the respondents to the material and motivate them to answer the questions carefully
• guarantee the order in which questions are administered, thus making sure that people answer the survey in the order intended.
• interviewers may be able to notice when respondents seem to misunderstand a question and explain its meaning and can probe for more complete answers when a respondent gives a brief answer or one that does not respond to the question.

Question 14

Face to face disadvantage

Answer 14

• The presence of the interviewer creates a social situation that may result in biased responses
• Respondents say what they think interviewers want to hear
• much more expensive
• safety of the interviewers

Question 15

Written response (mail)

Advantage

Answer 15

• very efficient use of time and money and can have a very high response rate if attendance by group members is high
• Low cost
• respondents can complete the questionnaire at their leisure, and they have greater anonymity in their responses, reducing interviewer bias

Question 16

Mail Survey Disadvantage

Answer 16

• The main problem with written questionnaires is response rate
• often less than 50%
• low response rates may invalidate the results because of differences between individuals who respond and those who don’t
• There is no possibility of clarifying questions that might be misunderstood. (illiterate or have vision problems)
• Illiterate participants are frequently embarrassed to admit their problem and sometimes respond to questions they do not understand, compromising the reliability and validity of the data.
• impossible to determine how seriously the respondent took the survey
• i.e. given to a child in the family to fill out

Question 17

internet survey

Advantage

Answer 17

• social desirability may be reduced
• all the questions were asked in order, and none were skipped.
• The computer can check for invalid responses and prompt the interviewer to recheck implausible answers, such as the presence of 20 children in the home
• control the sequencing and branching of questions so that, for example, people who do not drive will not need to be asked
• 24 hours a day, making a high level of participation possible without the costs of photocopying or postage
• some indications that Internet surveys have higher predictive validity than data collected by telephone without sacrificing measurement reliability

Question 18

Internet survey Disadvantage

Answer 18

• Illiterate or uncooperative participants will pro- vide meaningless data
• True measures of response rates are difficult
• typically rather low
• might not be honest about their ages or genders
• random sample of respondents may also be difficult to obtain because people participating in a Web-based survey are self-selected
• security of responding is vital. Further, storage of survey information obtained electronically must be ensured

Question 19

telephone survey

Advantage

Answer 19

• low cost
• involve less effort than some other methods of administration
• rapidly, without having to wait for interviewers to travel to many locations or for respondents to mail back their completed surveys
• possibility of using a computer-assisted interview/the interviewer reads the questions from a computer screen
• can be conducted from a central location where the interviewers can be supervised
• no reliable evidence that the data collected from cell phone–based studies are poorer in quality than landline- based surveys

Question 20

telephone survey Disadvantage

Answer 20

• Some states prohibit using auto-dialers
• since a person with a cell phone can be physically located almost anywhere, sampling appropriately from within a spe- cific location, such as everyone who may live within a particular town, is extremely difficult.
• introduce the possibility of interviewer bias
• must also be relatively short to get participants to finish the survey
• harder to establish rapport or to judge the degree of seriousness

Question 21

What are the relative advantages of using open-ended questions in a survey?

Answer 21

• Using open-ended questions makes it more likely that the questionnaire will discover something not anticipated by its designers
• more useful for smaller and preliminary studies.
• Coding a small number of open-ended surveys may be manageable, whereas hundreds would not
• trying out a preliminary version of a survey with open-ended questions can determine the range of likely answers, permitting you to standardize the alternatives into a closed-ended format

Question 22

What are the relative advantages and disadvantages of using closed-ended questions in a survey?

Answer 22

easier to code and analyze, and there are fewer off-the-wall responses

• alternatives are presented to the respondents, so they do not have to think as hard
• do not need to be as articulate to formulate their answers

Question 23

What are the relative disadvantages of using open-ended questions in a survey?

Answer 23

• harder to code, however, because the answers are in narrative form makes data analysis a messy job and makes it likely that you will have to break a cardinal rule of research by not deciding in advance how you are going to analyze your data
• issues being studied may be too complex to reduce to a small set of alternatives, or the respondent may not agree with any of them, resulting in simplistic answers

Question 24

What are the relative disadvantages of using closed-ended questions in a survey?

Answer 24

• issues being studied may be too complex to reduce to a small set of alternatives, or the respondent may not agree with any of them, resulting in simplistic answers
• tend to put words into the mouths of respondents, suggesting alternatives that respondents might never come up with them- selves
• errors can creep into the closed-ended questionnaire

Question 25

criteria for a useful questionnaire item

Answer 25

• Must be reliable and valid
• Because it is reactive, i.e. They know they are being examined, they may not tell the truth - social desirability effect
• May have hidden agendas so have to design accordingly

Question 26

Six steps in developing questionnaire and consider type of questions, the clarity and conciseness of questions, and the ordering of questions

Answer 26

• 1) decide what information is required;
• (2) decide what type of questionnaire will be used;
• (3) draft and then revise the questionnaire;
• (4) pretest the questionnaire;
• (5) edit the questionnaire; and
• (6) specify the procedures for its use.
• be written in language familiar to all respondents.
• be clear and specific. Define the terms and the context clearly. Make alternatives clear, exhaustive, and mutually exclusive in the case of closed-ended questions.
• avoid any leading, loaded, or double-barreled questions, which would result in biases in responses (e.g., socially desirable responses).
• be as concise as possible.
• all conditional information be presented prior to the key idea

Question 27

Describe how the ordering of questions in a survey may influence the quality of data

Answer 27

• Mail
  
  • ask first those questions that are of greatest interest to the respondents so as to capture their attention, and to leave the questions about demographic data at the end
• Personal or telephone
  
  • establish rapport with the respondent. Such interviews often begin with questions that are easy to answer, such as demographic questions
• The generally accepted method is to begin with the most general question and then move on to more specific questions within a general topic. Often, branching items are used to direct respondents to skip inappropriate questions. The use of branching items reduces the demand on the respondent’s time.

Question 28

the way that a question is asked is very important and can easily become a source of —

Answer 28

bias

Do you believe in killing unborn babies?

The second one’s newsletter asks, Should women be forced to bear unwanted children?

carefully constructing the survey questions, such that the frame of evaluation is clear, helps to avoid bias

Question 29

acquiescence:

Answer 29

the tendency to agree with a statement on a questionnaire, regardless of its content

Question 30

qualities of aqcuiesence

Answer 30

Agree/Disagree or True/False because they are easy to administer, these formats are highly susceptible to bias caused by acquiescence

Participants exhibiting acquiescence will have a tendency to agree to any statement on the inventory, regardless of its con- tent. The effect of this bias is that participants will agree with both a state- ment and its opposite, so, for example, will respond yes to both I like fish and I don’t like fish.

It is estimated that the effect accounts for approximately 10% of responses

Question 31

response rate:

Answer 31

in survey research, the percentage of individuals in the sample who return the completed survey

Question 32

increasing return rate for mail survey =

Answer 32

• The response rate for mail surveys is highest when the questionnaire has a “personal touch,”
• the effort required from the respondent is minimal,
• the topic of the survey is of interest to the respondent,
• and the respondent identifies with the sponsoring organization or the researcher conducting the survey
• Researchers can increase the return rate by administering the survey in groups if attendance by group members is high

Question 33

two limitations of survey research

Answer 33

• Social desirability effect
• Hard to ascertain patterns with open ended questions though these are the ones that would be more nuanced

Question 34

Briefly describe the ethical issues in conducting survey research. What steps should researchers take to ensure that ethical principles are followed?

Answer 34

• anonymity and confidentiality - take measures to ensure both
• Informed consent is another important ethical guideline to be followed. A researcher must let respondents know that their participation is voluntary, that they can choose not to answer any of the questions in the survey, and that they can terminate their participation at any time, without any penalty

Question 35

mutually exclusive questions:

Answer 35

categories defined so that membership in one rules out membership in another

Question 36

exhaustive questions:

Answer 36

categories defined so that all possible cases will fall into one of them

Question 37

verification key:

Answer 37

a collection of items on a questionnaire designed to detect dishonest answers

Question 38

branching items:

Answer 38

a set of questions that enable the respondent to move through a survey in different ways, depending upon the responses