Surveys

Question 1

How would you identify a population of interest?

Answer 1

1. Identify units of analysis e.g. individuals, hospitals
2. Define the characteristics that need to apply to them e.g. geography, age, other demographic variables (gender) or a range of measures that could reflect personal background

Question 1

What is probability sampling?

Answer 1

Each person in the survey population has an equal chance of being selected for inclusion in the sample
- more desirable way of achieving a representative sample
- we can calculate the probability of our sample results being representative of the population as a whole

Simple random sampling is the most straightforward approach to obtaining a representative sample

Question 2

What is non-probability sampling?

Answer 2

Each unit in target population does not have an equal chance of being included
- form sample using other considerations, such as convenience or a particular characteristic

Question 3

What are some issues in simple random sampling?

Answer 3

Not all respondents may reply to invitation within the same time frame.
? move on and invite other people –> introduces bias in inviting people who are more available
Instead we should make repeated attempts to contact the originally selected person to make sure that our random sampling approach properly works

Question 4

What is cluster sampling?

Answer 4

When not able to create a sample from the full list of our population the population can be divided into clusters e.g. based on geography or organisation
Randomly select some of the clusters and collect data from everyone within those clusters.
Can still be difficult
Should still reflect the larger population

Question 5

What is stratified sampling?

Answer 5

Stratification allows to define subpopulations or strata then draw a sample within each
Useful when whole-population approach to random sampling mat result in biased samples
It ensures sample size for each relevant group is completely reached
Subgroups not to be left to randomisation but controlled by the researcher

Question 6

What is convenience sampling?

Answer 6

Choosing participants based on convenience, e.g. location
Little generalisability or representativeness
- certain distinct characteristics
Websites may ask users on that site to fill in a survey
- suitable idea for a company that wants to find out something about the actual users of the website

Rarely suitable for proper survey research

Question 7

What is purposive sampling?

Answer 7

A more targeted approach
The researcher would have a clear target group or population in mind but would recruit respondents without the use of probability of quota-based approach
? no explicit sampling frame or the group is fairly narrow
May be chosen because researchers do not generalise beyond the sample of individuals surveyed

Can be meaningful when used in conjunction with qualitative interviews of the individuals surveyed to obtain additional information from the,

Question 8

What is quota sampling?

Answer 8

Generating a sample that mirrors target population
- proactively recruit respondents that match the characteristics of the population
e.g. in any sample of the adult population of the country it would contains both men and women and it would also have to cover the different age groups of the populationTry to design the structure of the sample according to the proportions in the population

Question 9

What is snowball sampling?

Answer 9

Word of mouth
A sample grows continuously with help of the research participants

Question 10

What are different collection methods of survey-style data?

Answer 10

Face-to-face
Online
Telephone
Interviews
Written

Question 11

What is the gold standard of survey research?

Answer 11

Face-to-face

Question 12

Discuss face-to-face data gathering

Answer 12

Gold standard in survey research
Require a substantial amount of time
Requires people to be in the same place
For surveys spanning large populations, interviewers are typically conducted by teams of researchers
Used more in qualitative research

Strengths:
- Able to build rapport
- Analyse verbal and non-verbal
- Permits more direct engagement with respondents
- Allows for the administration of multiple instruments in the presentation of the survey permitting a great deal of question types and observations to be explored (e.g. videos, leaflets)
- Can be utilised well with certain probability based sampling techniques
- Allows rephrasing of questions if people don’t understand

Limitations:
- Geography
- Time
- Less costly

Question 13

Discuss S+L of telephone survey gathering

Answer 13

Strengths:
- Less costly than F2F
- Less time consuming
- No travel
- Shorter
- Don’t require to be in certain location
- Bigger sample of participants

Limitations
- Can respond or analyse non-verbal
- Lack of rapport
- Require telephone connection

Question 14

Discuss S+L of direct mail survey gathering

Answer 14

Strengths:
- People can complete in their own time
- People can think about it more as not on the spot
- Can include other information with the survey
- Can send in different languages

Limitations:
- Could be ignored
- If not paid for the return service people may not want to participant
- Not up to date address
- Not completed within deadline
- Require people to be literate and speak the language the questionnaire is printed in
- Must have an address

Question 15

Discuss S+L of internet based survey gathering

Answer 15

Strengths:
- Much cheaper
- Quick
- Gives time to respond
- ? save and come back at another time
- Images, video and audio content can be incorporated into questionnaires
- Offers substantial flexibility in term of options for different ways of asking and presenting questions

Limitations:
- Participants must have internet access
- Participants must be literate
- Potentially have to start again if internet error

JISC is the online survey website

Question 16

Discuss combining methods

Answer 16

Can be a substantial advantage to combine multiple data collection methods
- reduce error
- reduce bias –> giving more than one option on how participants could get in contact, e.g. if they don’t have internet access
- increased response rates

Two ways to combine methods
- Sequentially –> utilisation of one method (commonly to recruit people) and a second method to administer the survey
- Concurrently –> option to choose between different methods of responding

Mode of interviewing could vary the persons response

Question 17

What should be considered with non-response bias?

Answer 17

Imbalances regarding which groups of the target population may or may not be covered

May not want to take part because of
- trust
- interest
- incentive
- time
- disclosing sensitive information

Could be a concern
- those who respond are different to those to don’t, e.g. a certain type of group of people don’t respond

Can help by
- trying to reduce user bias
- offering more than one method of survey
- shorter length of survey
- is there a question that puts people off wanting to answer? (unit non-response)

Question 18

What makes a good survey question?

Answer 18

Clear and answerable
Something that people are willing to answer
In a language that resonates with people
Intuitive to get a genuine, authentic answer
It needs to be understood the same way by everyone answering it
Unambiguous
Write the question thinking about the overall aim of goal

Avoid
- Leading questions
- Questions that are too complex or difficult
- Future based questions

Question 19

What are double-barrelled questions?

Answer 19

Asking two questions at the same time
- impossible to answer if it is a closed question
- people usually only answer one section of the question

Better to ask two separate questions

Question 20

How can you ensure survey quality?

Answer 20

Get someone to answer the survey before sending it out to test the quality of the survey and then analyse response

Look at existing surveys in similar fields and compare although it is rare that the questions you are asking have been asked before

Undertake some form of cognitive testing

Cognitive interviewing for new questions –> piloting –> interviewer preparation –> translation, when conducting surveys in multiple countries in more than one language

Question 21

What is cognitive interviewing?

Answer 21

When new questions are developed it is difficult to know how the respondent will understand, interpret and subsequently respond to the question.

Showing people the questions and asking them how they would understand the question and answer it –> get them to feedback to you

Questions are ideally in the same form as the survey
Usually a small number of respondents selected
May not be in the required population - its just for question feedback

Question 22

How would you pilot a survey?

Answer 22

Test to see if the survey actually works when administered in the field
Main goal is to investigate how the actual administration works in the field
Sample selected for the sample should already approximate the one that will ultimately be selected, following the same methods of sampling and data collection

Can then look at responses
Was there a question not being answered?
Does anything need to be changed or altered

Insights about the process from those involved in carrying out the survey

Question 23

What preparations and briefings should be considered for interviews?

Answer 23

Will there be a script?
Will additional information be provided?
How many interviewers?
How will you make the first impression?
Is training required?

Question 24

What is data cleaning?

Answer 24

Identifies when respondents provide careless answers to questions and do not engage with the questions at all
Did the respondent actually read the questions or just randomly select answers?
This can be done by noting response patterns that are unlikely e.g. selecting the same answer
Also the time that respondents take to answer the questions is commonly recorded

Errors can be caused by very trivial factors but they can also be more systematic and require analytical techniques to identify them
Look at the distribution of values for each variable to identify highly unlikely or extreme results that may require investigation