Chapter 14- Survey Design Flashcards
When might we use surveys?
Surveys serve many purposes and can be used to find out about many psychological constructs and help to test the associations among them. Surveys are often also implemented by organizations and governments for various purposes. Some typical uses might be:
Assessing political beliefs – this might be especially helpful leading up to
elections, for example.
Assessing attitudes toward current or impactful issues
* bullying in schools and workplaces
* environmental beliefs and behaviours
* equity, diversity, inclusion, and accessibility at work and school
census
the government collects data from residents on particular issues
open-ended survey questions
“what aspects of your job do you find to be the most
stressful?” Respondents then share their answer, for example, by typing it in an online survey or speaking their response to an interviewer. Often, we know what typical work stressors are, and so we can use established scales to measure these stressors, but sometimes we may want to see if what people say matches up with the scales we typically use. It is possible that we may be missing something! Right now, during this pandemic, for example, what people find to be most stressful may not be the same compared to pre-pandemic times. We may want to allow for a variety of responses to learn about new, or potentially more salient, stressors.
restricted survey questions
(similar to multiple choice). We provide a variety of options for people to choose from, typically in addition to an “other” option to make sure we don’t miss anything! See the example below:
What University do you attend?
a. Saint Mary’s University
b. Dalhousie University
c. St. FX
d. Mount Saint Vincent
e. Acadia
f. Cape Breton University
g. Other: ________________
likert-type scale
In Psychology, we often use these Likert-type scale ratings, often with anchors ranging from strongly disagree to strongly agree. Participants are asked to read a statement and then indicate the extent to which they agree or disagree with it. Depending on what is being measured, these anchors may shift. For example, if we are interested in the frequency of exposure to some behaviour (e.g., incivility), we may have anchors that range from “never” to “always”, for example.
how many points on a likert scale
5-7 is the most common
mail surveys
people are mailed a survey and then are asked to return the completed survey to the researcher. Often a pre-paid envelope is included so that the participants do not bear any costs. Many people prefer completing surveys this way, and using mail surveys may reach participants who may not have internet access (e.g., those in more rural areas) which can be a benefit. Of course, response rates are not always great (10-20% is pretty typical) and printing and mailing costs can be high. There is also a potential issue with non-response bias. This reflects the idea that those who return surveys are not representative of the entire group who were sent them, which means we may be left with a biased sample. The same problem holds true of internet surveys. We may, for example, only get people responding who are very passionate about the issue, which may lead us to miss out on those who have less extreme or more neutral opinions (so we are missing
a part of our normal curve in many instances!)
telephone surveys
conducting surveys over the phone can be very time-consuming for the researcher and can sometimes be difficult for participants as they don’t have a visual depiction of questions and response options. Phone surveys can be good, however, as they would allow more in-depth responses possibly, relative to other survey administration methods. In addition, the participants have someone in the moment to whom they can ask questions. Interviewer bias (a form of experimenter bias) can be an issue here though as the interviewer may give off subtle, or not so subtle, cues as to the expected answers!
in-person surveys/interviews/focus groups
Having in-person surveys or focus groups (a session with a few participants at a time) can be really good for delving deeper into an issue and having discussions about a topic. These methods, of course, require a greater time commitment on the part of participants and they are likely not anonymous in this case. Depending on the topic area, having discussions in front of others may also limit the responses given. In person designs are typically limited to cases where you don’t want a very large sample size. As you can imagine, the time commitment from the researcher here is also quite high.
web-based/ internet surveys
Administering surveys over the internet is
becoming very common and has vast benefits, especially in terms of reaching people all over the world. Interestingly, there are now multiple “panel companies” that exist that supply researchers with potentially willing participants to take their online surveys. The panel companies typically are responsible for compensating individual participants and so this is a large benefit to researchers (of course, the researchers pay the panel companies!). These participants are almost always anonymous to the
researchers, which can also be a very good thing!
problems with online surveys
it is less clear (relative to in-person surveys) who is taking the survey and what they are doing at the
time of completion. Are participants paying attention and reading items carefully?
- Often with panel respondents, researchers may build in “attention checks” to flag those who do not appear to be reading the survey items (this might involve asking participants to choose a particular response option for an item, for example “strongly disagree”).
- The time it takes participants to complete a survey is typically logged, which means that researchers may also have a sense of who is “speeding” through a survey and not necessarily providing thoughtful responses.
