L5 Survey Research Flashcards

1
Q

What is survey research?

A

Research, based on a questionnaire to which respondents record their answers typically with closely defined alternatives

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

What is the ICS?

A

Index of Consumer Sentiment:

focused on past and current personal and general economic conditions

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

What is CBS?

A

Centraal Bureau Statistiek:

Focused, among others, on demographic, social and economic statistics

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

When to use surveys?

A
  • When you want to say something about a population, but you cannot measure the whole population (very often)
  • When you are interested in quantitative descriptors
  • personal measures (archival is mostly only objective and not subjective)
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

What are the survey research design decisions?

A
  1. Operationalization of concepts (variables)
  2. Decide on survey mode
  3. Appearance of the questionnaire
  4. Data collection
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

What is operationalization

A
  • Reduction of abstract concepts

* to render them measurable in a tangible way

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

What are off-the-shelf measures?

A

usen before and proven succesful by prior researchers

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

What are open-ended questions?

A

Allows respondents to answer a question in any way they choose

(+)No limit to possible responses, sometimes participants prefer own words
(-) More difficult to respond and analyze

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

What are closed-ended questions?

A

Asks the respondents to make choices
among a set of alternatives

(+) Help respondents to make quick decisions
(-) Alternatives need to be mutually exclusive and collective exhaustive

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

Single-item measures

A

When concrete singular object/attribute:

  • maritial status
  • profession
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

Multi-item measures

A

When measuring more abstract questions
In all other cases
• Use ‘off-the-shelf’ scales
• Develop your own scale

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

Mixed-mode designs

A

a combination of different survey models, leads to a trade-off costs and errors (e.g. web survey + mail to seniors)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

Advantages and disadvantages of Off-the-shelf scales

A
(+)
• Known and “good” validity
and reliability
• Comparability of results
• Low cost

(-)
• Not tailored to your exact
research need
• Require translation if in different language (source of error)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

Double barreled questions

A

question that lends itself to different possible responses to its subparts (2 subjects in one)

“How is the taste and appearance of your pancakes?”

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

ambiguous questions

A

the respondent may not be sure what the question exactly means

“Are you happy?”

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

Leading questions

A

lead the respondent to give the responses that the researcher would like them to give

“A true American’s favorite colors are red, white and blue. Are these your favorite colors?”

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

Loaded question

A

question phrased in an emotionally charged manner

“have you stopped cheating on tests?”
“do you still beat your wife”

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

Double negatives

A

unclear what people mean when answering yes/no

“Do you oppose not allowing the board to pass article 10 of the ballot?”

19
Q

what are comparative scales (ranking scales)?

A
  • Paired comparison
  • Rank ordering
  • Constant sum
20
Q

Paired comparison

A

Now you will see ten pairs of shampoo brands. For each pair, please indicate which one of the two brands you would prefer for personal use

21
Q

Rank ordering

A

Rank these brands in order of preference for personal use.

  1. Cola ___
  2. Ice Tea ___
  3. Fanta ___
22
Q

Constant sum

A

Divide 100 points among the following brands, indicating your preference for personal use:

  1. Cola (points)
  2. Ice Tea (points)
  3. Fanta (points)
23
Q

What are non-comparative scales (rating scales)

A
  • Continuous rating scale
  • Likert scale
  • Semantic differentials
24
Q

Continuous rating scale

A
  • “How would you rate De Bijenkorf as a department store?”

* 0 ———————————- 100

25
Q

Likert scale

A
  • Disagree/Agree (with statements)

- 5- or 7-point

26
Q

Semantic differentials

A

“What do you think of the Apple iPhone?”
• Good / bad
• Powerful / weak
• Modern / old-fashioned

27
Q

mutually exclusive

A

Only one answer applies

28
Q

collectively exhaustive

A

the answer possibilities cover the entire realm of possible answers

29
Q

Survey modes

A

How the data are collected

30
Q

Factors that play a role when deciding on survey mode

A
  1. Measurement
  2. Representation
  3. Economics
31
Q

Appearance of the questionnaire

A
  1. A good introduction
  2. Organizing the questions & giving instructions and guidance
  3. Personal data (do not scare them off)
  4. Concluding the questionnaire
    - also include confidentially
32
Q

Goals of pretesting

A
  • Catch errors/unclarities
  • Discover sensitive topics
  • Check response categories
  • Optimize length

How?
• Pick 5-10 people from target group
• Let them “think out loud” + observe
• Improve survey

33
Q

Response rate

A

of people that participated in the survey divided by # of people sampled

is generally low
• Internal surveys: 30-40%
• External surveys: 10-15%

34
Q

How to increase your response rate?

A

Maximize rewards of participation
• Showappreciation
• Use interesting/friendly questionnaires
• Offer tangible rewards

Minimize costs of participation
• Minimize time and effort required
• Minimize chance of feeling threatened by questions

Maximize trust
• Ensureanonymity/confidentiality
• Open lines of communication with participant
• Identify research with well-known, legitimate organization
also provide contact info

35
Q

What is validity?

A
  • Does an instrument measure what it was intended to measure?
  • Do we measure what we are supposed to measure?
36
Q

What is reliability?

A

Are the data accurate (free from measurement error) and consistent (from one occasion to another)?

37
Q

Measurement validity

A
  • Provide precedence (other studies same measure)
  • Provide sound logic to support that considerable conceptual overlap exists between measurement/proxy and construct
  • Be aware: single-item measures for abstract constructs = low validity
38
Q

Proxy

A

proxy= indirect measure of desired structure.

commonly used when direct measures are unavailable

39
Q

social desirability bias

A

Respondents may not always be willing to communicate their true response in case of sensitive issues

  • drugs use
  • finances
40
Q

How to minimize social desirability bias

A
  • “Everybody-does-it”
  • “Assume-the-behavior”
  • “Authorities-recommend-it”
  • “Reasons-for-doing-it”

in all other cases, avoid leading and loaded questions

41
Q

Survey mode effects

A

Participants are more likely to give socially desirable responses on the phone or face-to-face

42
Q

How to measure multi-item?

A

Cronbach’s alpha

43
Q

What does Cronbach’s alpha do

A
  • Cronbach’s alpha measures to what extent a set of items are inter-related
  • High inter-relatedness = high reliability
  • Cronbach’s alpha is between 0 and 1
  • Values > .7 are considered acceptable