Survey Design Flashcards
What 6 components do we need to consider when designing surveys?
1) Instructions (to standardise and increase reliability)
2) Order of the questions
3) Demographics (characteristics of our sample)
4) Open vs Closed questions
5) Writing/formulating the questions
6) Avoiding response bias
What are the criteria for good survey instruction?
- Standardised (the same instructions given to all Ps)
- Clear and direct
- Avoid jargon and complex/technical terms
How do we order the survey questions?
List 4 things we could do.
- Divide a survey into sections (e.g. by topic or question type)
- Ensure Ps are eligible (screening)
- Start with easy and engaging questions
- Use funnelling/branching questions if appropriate
What are survey demographics?
The characteristics of the sample that is participating in the survey
What could be included in the survey to obtain information on sample demographics? List 8 examples
- D.O.B./age
- Gender
- Ethnic and/or racial background
- Place of birth
- Religion
- Sexual orientation
- Relationship status
- Occupation or earnings
What 2 criteria must demographic survey questions follow?
1) Must be relevant questions
2) Ensure that response options are inclusive to all
What are the advantages of open questions?
List 3
- Provide more detail
- Rich data
- Don’t impose assumptions
What are the disadvantages of open questions?
List 2
- Longer and more difficult to complete
- Difficult to analyse responses (often subjective)
When should we use open questions?
Only when they are justified
When are open questions most useful?
For descriptive, exploratory work
What are the advantages of closed questions?
List 3
- Quick to complete
- Easy to analyse
(objective) - Standardised responses
What are the disadvantages of closed questions?
List 2
- Can impose assumptions
- Oversimplify complex issues
What 3 things must we consider when asking closed questions?
- Ensure questions are unambiguous
- Provide clear response options
- Carefully consider the style of response options
When are closed questions most useful?
For explanatory or predictive work
When formulating survey questions, what 5 things should we avoid?
1) Double-barrelled questions
2) Double-negative questions
3) Ambiguous questions
4) Leading questions
5) Questions with jargon
What are double-barrelled questions?
Composed of two separate issues or topics but allows just one response
Composed of two separate issues or topics but allows just one response
This is known as…?
Double-barrelled questions
Why should we avoid double-barrelled questions?
Because they are confusing and it is impossible to determine the true intention of respondents from data
What is the problem with this question?
Is your personal style influenced by magazines and TV?
Double-barrelled question
What is the problem with this question?
How satisfied are you with your pay and work environment?
Double-barrelled question
Why should we not ask double-negative questions?
Double negatives can occur as a result of negative response options
What is wrong with this question?
It is not a good idea to not submit work on time
Double-negative question
Why should we not ask questions with negotiations? List 2 reasons
1) Introduce complexity and can be missed
2) Can result in non-response or misunderstanding
Instead of negotiations, how should we present questions?
Present questions as a positive statement instead of negative statements
What is wrong with this question?
Statistics should not be a compulsory unit of study
Negotiations
What are ambiguous questions?
Questions that are vague, or over-generalised and are open to interpretation
What is wrong with this question?
Where do you like to shop?
Ambiguous question
What is wrong with this question?
Is your personal style influenced by magazines?
Ambiguous question
What is wrong with this question?
Do you believe that the media have a dangerous influence on women’s perceptions of themselves?
Question with emotive language
What is wrong with this question?
Experts suggest that exposure to violent video games can result in behavioural problems in children. Do you agree?
Leading
How do we improve this question?
Do you believe that the media have a dangerous influence on women’s perceptions of themselves?
Do you believe that the media have an influence on women’s perceptions of themselves?
How do we improve this question?
Experts suggest that exposure to violent video games can result in behavioural problems in children. Do you agree?
To what extent do you believe that exposure to violent video games can result in behavioural problems in children?
What is a question with jargon?
When the question contains technical terms that Ps won’t be familiar with
What is wrong with this question?
The use of cloning and burning in fashion photographs is morally wrong
Leading
Why should we avoid asking questions leaning towards social desirability? List 2 reasons
1) Ps may respond in a positively biased way
2) Ps may identify with a lie or social desirability scale
What is wrong with this question?
How much is each statement True or False for you?
No matter who I’m talking to, I’m always a good listener
Question leaning towards social desirability (response bias)
What are the 2 types of survey response bias?
1) Response acquiescence
2) Social desirability effects
What is a response acquiescence?
The tendency to agree rather than disagree
What type of survey induces response acquiescence?
Surveys which include both positively and negatively worded questions
What is wrong with this survey?
It was easy to find what I wanted on the website
I had no problems navigating the website
Response acquiescence
It includes both positively and negatively worded questions
Match the statement below to a problem.
My mood is influenced by sport
a. Jargon
b. Leading
c. Double-barrelled
d. Double-negative
e. Ambiguous
e. Ambiguous
Match the statement below to a problem.
The height and weight of a person impacts our first impressions of them
a. Jargon
b. Leading
c. Double-barrelled
d. Double-negative
e. Ambiguous
c. Double-barrelled
Match the statement below to a problem.
It is not unlike me to have a glass of wine with dinner
a. Jargon
b. Leading
c. Double-barrelled
d. Double-negative
e. Ambiguous
d. Double-negative
Match the statement below to a problem.
An implementation mode which decouples performance indicators from organisational objectives seems to further implementation success
a. Jargon
b. Leading
c. Double-barrelled
d. Double-negative
e. Ambiguous
a. Jargon
Match the statement below to a problem.
Do you agree with the widely held view that capital punishment is morally wrong?
a. Jargon
b. Leading
c. Double-barrelled
d. Double-negative
e. Ambiguous
b. Leading
Which of the following is a benefit of using open questions?
a. Provide rich data
b. Quick to complete
c. Easy to analyse
d. Standardised responses
a. Provide rich data
Which is a drawback of closed questions?
a. Can oversimplify complex issues
b. Difficult to complete
c. Subjective responses are dificult to analyse
d. Tend to be longer than open questions
a. Can oversimplify complex issues
