non - experimental research - survey methods Flashcards

Question

adding demographic information

Answer 1

its the basic data that identifies the characteristics of survey respondent. these data can include age, gender, socioeconomic status, marital status and so on. sometimes the empirical question will determine the type of demographic information needed. including demographic information enables the survey researcher to group the results by demographic categories. in general its good to put questions about demographic information at the end of a survey. if you start the survey with them ptps might become bored and not attend to the key items as well as you would like. also only include only demographic categories that are important for the empirical question that interests you. the more demographic information you include the longer the survey and the greater the risk that respondents will tune out. respondents might become irritated; some requests for demographic information (e.g., income) can be perceived as invasions of privacy, even when respondents are assured about confidentiality (Patten, 1998).

Answer 2

A major problem in survey construction concerns the wording of the items. Although it is impossible for the survey writer to ensure that all respondents interpret each question or statement the same way, some guidelines can help in the construction of a good survey. The most important one is to conduct a pilot study (Chapter 3) to test the instrument on several groups of friends, colleagues, and even people you don’t like. You will be surprised at how often you think you have a perfect item and then three friends interpret it three different ways. One tip is to define any terms you think could be interpreted in more than one way. Avoid these three specific problems: linguistic ambiguity, asking two questions in one item, and asking leading questions. first questions can be ambiguous. second survey writers sometimes include too much in an item resulting in one that actually asks for two response at once. this is sometimes referred to as a double-barrelled question. third is what layers call a leading question - one that is structured so that it is likely to produce an answer desired by the asker. its a type of survey bias and is used frequently in world of business and politics. always opt for simplicity over complexity use complete sentences avoid negatively phrased questions, negative statements are more difficult to process than positive ones. use balance items not those favouring one position or another. avoid most abbreviations avoid slang and colloquial expressions avoid jargon Another important consideration in creating a survey is to avoid creating any carry‐over effects (see Chapter 6) across items on a survey that may bias survey responses. Carry‐over effects within surveys occur when earlier items may influence responses on later items.

Answer 3

All data collection for survey data begins with some form of written survey that is delivered in‐person in the form of an interview, through the mail, on the phone, or online. In‐person surveys may be done in the form of an interview, which we will discuss later. Of course, all surveys begin with well‐ written survey items that can effectively engage the participant. Each mode of delivery of a survey (i.e., mail, on the phone, and online) have their own unique considerations

Answer 4

Kinsey report - most famous sex survey used interview survey The interview format for surveying individuals about attitudes, opinions, beliefs, and the like has the advantages of being comprehensive and highly detailed.3 Even though the interviewer typically asks a standard set of questions, the skilled interviewer is able to elicit considerable information through follow‐up questions or probes. Having an interviewer present also reduces the problem of unclear questions; the interviewer can clarify information on the spot. Sampling is sometimes a problem because, in many cases, sizable segments of the population may not be included if they refuse to be interviewed, cannot be located, or live in an area the interviewer would prefer to avoid. Interviews can occur in a group format; Besides sampling issues, other major problems with the interview approach are cost, logistics, and interviewer bias. Interviewers must be hired and trained, travel expenses can be substantial, and interviews might be restricted to a fairly small geographic area because of the logistical problems of sending interviewers long distances. And despite training, there is always the possibility that interviewer bias can affect the responses given in the face‐to‐face setting. The careful researcher using interviews will develop a training program to standardize the interview process as much as possible. Certain types of interviewers may be trained for specific purposes

Answer 5

Written surveys sent through the mail may have problems with how many people actually return a completed survey (as you know, because you’ve probably thrown a few away). Return rates of 85% or higher are considered excellent (and rare), 70–85% very good, and 60–70% more or less acceptable (Mangione, 1998). Anything below 60% makes researchers nervous about whether the data are representative. Another problem with rate of return occurs when people who return surveys differ in some important way from those who don’t return them, a problem called nonresponse bias (Rogelberg & Luong, 1998). When this happens, drawing a conclusion about a population is risky at best. The profile of the typical non‐responder is an older, unmarried male without a lot of education (Mangione, 1998); nonresponse also occurs when people have some attribute that makes the survey irrelevant for them (e.g., vegetarians surveyed about meat preferences). The best chance for a decent return rate for a mailed written survey is when (a) the survey is brief and easy to fill out; (b) the form starts with relatively interesting questions and leaves the boring items (i.e., demographic information) until the end; (c) before the survey is sent out, participants are notified that a survey is on the way and that their help will be greatly appreciated; (d) nonresponse triggers follow‐up reminders, a second mailing of the survey, and perhaps even a phone request to fill out the form; (e) the entire package is highly professional in appearance, with a cover letter signed by a real person instead of a machine; (f) return postage is included; and (g) the recipient has no reason to think the survey is merely the first step in a sales pitch (Fowler, 1993; Rogelberg & Luong, 1998). Return rates can also be improved by including a small gift or token amount of money. For example, Dillman et al., (2009) reported a study that increased response rate from 52% to 64% by adding a token gift of a dollar in the survey mailing. One important problem that exists with all forms of survey research is a social desirability bias. Sometimes, people respond to a survey question in a way that reflects not how they truly feel or what they truly believe, but how they think they should respond—that is, they attempt to create a positive picture of themselves, one that is socially desirable. Ensuring survey participants of anonymity can help reduce the social desirability bias, but the problem is persistent and it is hard to gauge its extent. The social psychology literature has a long history of research showing the attitudes people express on some issue do not always match their behavior. Thus, the results of survey research must be interpreted with this response bias in mind, and conclusions drawn from surveys can be strengthened to the extent that other research provides converging results.

Answer 6

according to Dillman et al 2009 phone surveying had its peak popularity in 1980s - virtually every phone had a telephone, households could be selected randomly through a procedure called random-digit dialing and people were generally open to being surveyed. And then two developments created serious difficulties for the phone surveying business: telemarketing and cell phones. Telemarketing produced a level of annoyance sufficient to lead to the creation of national do‐not‐call lists, and the marketing strategy also had the effect of creating high levels of suspicion and distrust in the general public, especially when telemarketers begin the call by pretending they are conducting a survey, when in fact they are selling a product (a ploy known in marketing research as sugging— Selling Under the Guise of a survey).5 As for cell phones, they have changed the dynamic of phone surveying. When the only phones in the house were landlines, surveyors would call and the unit of measurement typically would be the “household.” With cell phones, however, the unit is the individual, thereby changing the nature of the population. Nonetheless, phone surveying does have positive aspects. Unlike a mailed written survey, there is more human contact when one is speaking over the phone than when one is reading a paper survey. The method combines the efficiency of a mailed survey with the personal contact of an interview. To increase the chances of people responding, one technique used by legitimate phone surveyors is to precede the call with a brief letter, post card, or e‐mail alerting the respondent that a phone call is on the way (“and please cooperate, we could really use your help; and we promise it will be brief”). This is an example of the mixed‐mode approach (mixing mail and phone) advocated by Dillman et al., (2009).

Answer 7

One of the most common forms of Internet data collection involves online surveys, which is accomplished in several ways. First, online surveys can be sent as Internet url’s via e‐mail to a selected sample of individuals. E‐mail lists can be purchased or, following the ethically dubious lead of spammers, obtained by using “search spiders” that search the Internet for posted e‐mails and accumulate e‐mail addresses. A second form of online survey is one that can be posted on a listserv or social media site, collecting data from those who choose to respond. A third procedure is to follow a probability sampling procedure with incentives for participating in online surveys. For example, some companies create national samples by randomly sampling addresses rather than phone numbers. They then recruit subjects by regular mail to participate in multiple online surveys by providing the incentive of free Internet access (e.g., Wirth & Bodenhausen, 2009). Several technologies have emerged that allow survey developers to use software to create online surveys and even experiments. For example, SurveyMonkey and Qualtrics are two popular types of software for developing online surveys. Amazon’s Mechanical Turk (MTurk) is a platform that allows researchers to place their surveys and program experiments that can be conducted online. Further, MTurk participants can be paid for their participation for very small amounts of money (e.g., a dollar or two per participant). The main advantage of online surveying is that a large amount of data can be collected in a relatively short time for minimal cost. There are costs for researcher time and surveying software, but usually no postage, phone, or employee costs. And with the Internet open 24 hours a day, online surveying can be completed in less time than other forms of surveys. Problems exist, however. Although Internet use is widespread, the sample tends to be biased; for instance, responders are unlikely to be representative of all income and education levels. With e‐mail surveys, the researcher runs the risk of having the survey appear to be just another piece of spam or, worse, a potential entryway for a virus. By one estimate (Anderson & Kanuka, 2003), 85% of e‐mail users delete messages without reading them, at least some of the time. So, at the least, the e‐mail survey must have a subject line that will catch the reader’s attention. With web‐ based surveys, the problem (which you probably have already recognized) is that the sample is bound to be self‐selected, resulting in bias. Furthermore, it could be that a teenager out there has little else to do but respond to your survey several hundred times a day for a week or two. Despite the difficulties, however, online surveying occupies a large niche in the 21st‐century survey business. As with phone surveying, savvy researchers can use a mixed mode approach—sending a letter in the mail that appeals to the reader to complete the survey, perhaps including in the letter a website address with a password to enter the survey. Or, as mentioned earlier, survey companies sometimes provide incentives for participation.

Answer 8

in survey research is some cases the APA does not require informed consent - in anonymous questionnaires. but it is customary for researcehrs to include consent language in cover letter that precedes the survey or in the opening screens of an online survey. A second point about survey research and ethics is that decisions affecting people’s lives are sometimes made with the help of survey data, and if the surveys are flawed and biased, or poorly constructed, people can be hurt or, at the least, have their time wasted. Although professional psychologists operating within the APA’s ethics code are unlikely to use surveys inappropriately, abuses nonetheless can occur. The problem is recognized by the judicial system, which has established a set of standards for the use of survey data in courts of law (Morgan, 1990). The standards amount to this: If you are going to collect and use survey data, behave like a professional psychologist—that is, be careful about sampling, survey construction, and data analysis