survey design

Question 1

Problems with Surveys

Answer 1

Superficial measures
Causality
Self-reported data

Question 2

Important thing to consider with Surveys

Answer 2

Who is my population
How do I capture information
What is my DV, IV, control variables

Question 3

Nominal variables

Answer 3

“Counts”, “mode”

Question 4

Ordinal variables

Answer 4

The order of values is known
“Counts”
Mode
Median

Question 5

Interval variables

Answer 5

The order of values is known
"Counts"
Mode
Median
Mean
Can quantify the difference between each value
Can add or subtract value

Question 6

Probability sampling methods

Answer 6

Simple random sampling
Systematic
Stratified
Multi-stage cluster

Question 7

Non-probability sampling methods

Answer 7

Availability samples

Purposive sampling

Question 8

What about no-response?

Answer 8

No reponse is a response

Question 9

Good design in survey

Answer 9

Explain purpose
Establish credibility
Drop down menus
Don't overcrowd the questions
Include time it takes to fill in the survey
Send reminders (2)

Question 10

Cognitive process of answering a questions

Answer 10

What is the researcher “really” asking?

Each new answer should create new information

Question 11

Surveys and coding

Answer 11

The harder the coding the more expensive and time-consuming is the questions

Question 12

“Dont know” in surveys

Answer 12

Unlike the possibility to answer other:”…typed in response…” no opinion option allows respondents basically skipping uncomfortable or difficult questions

No opinion option doesn’t improve reliability

The no opinion is usually chosen for rather random various reasons

Question 13

Avoid in surveys

Answer 13

Leading questions
Are you against giving too much power to trade unions?

Referencing authority
“Researchers have found out that…”
“According to the specialist…”
“Doctors say…”

Ambiguous questions
“Do you feel generally supported by the management?”

Vague terms
Long time
Fairly
In general
Kind of

Too specific terms
“Do you display high organizational citizenship behaviour at work?”

Question 14

Satisficing

Answer 14

Conditions that foster satisficing

Task difficulty
Respondent’s ability
Respondent’s motivation

Solution:
Clear questions, simple terms

Question 15

Acquiescence

Answer 15

tendency to endorse any assertion made in a question, regardless of its content, perhaps due to human tendency to appear agreeable or due to human cognitive biases

I love dancing / I hate dancing- should have completely opposite responses, but they don’t, about 22% of respondents across studies would agree with both!

Strong satisficing- more likely to answer yes/true without thinking at all even to factual questions

Confirmatory bias and weak satisficing

The tendency for acquiescence is correlated with other satisficing strategies, higher in people with lower cognitive abilities, higher with fatigue and encouragement to guess

What to do about it?
Avoid this type of questions- not always possible

Change to forced answer question giving people agreement/neutral and negative option

Question 16

Common Method Variance Bias (CMV bias)

Answer 16

The covariance between two constructs/measures might be caused by utilizing the same method to collect data on both constructs rather than true relationships that we want to measure

Common problem- self-reports

Major sources:
Same rater
Response style
Proximity of items on the survey
Item wording
Item context