Accurate Data

Question 1

For which procedures is the surveyor likely to know not only whether but also precisely when a check is happening during the survey?

Answer 1

Spot checks and accompaniments

Since both spot checks and accompaniments involve observing surveys as they are being implemented, the surveyor is aware whether and when precisely the check is happening.

Question 2

High-Frequency Checks

Answer 2

• Happen in the office with the full or partial dataset
– Paper data: can be done only after data collection and data entry is completed
– Digital data: can be done during data collection (possibly daily)
• Checking for validity of the data, possible enumerator effects, possible problems with the data-collection instrument itself
– Summary statistics, distributions, checks for outliers, logical checks
• Implemented using data analysis software such as Stata, R, SAS, etc. or by digital data collection tool

Question 3

When planning for back-checks, we take only a subsample of…

Answer 3

Respondents to be back-checked
Questions to be asked in the back-check questionnaire

Back-checking is only done on a subset of respondents. The back-check questionnaire does not include all questions in the original survey. It only includes a subset of questions that fall under specific categories as mentioned in the previous video. However, all surveyors should be back-checked.

Question 4

Validation of responses - purpose?
Percentage of “don’t know” and “refusal” responses - purpose?
Completion by treatment status - purpose?

Answer 4

Validation of responses - Logic check
“Dont know/refusal” - enumerator check
Completion by treatment status - project check

Question 5

What common things are checked with paper surveys?

Answer 5

For paper surveys, editors or scrutinizers can be used to check skip patterns and validity of responses, or most of the ouptut for logical constraints (amidst other things)

Question 6

What sort of checks can only be done reliably with digital devices - and not for paper surveys?

Answer 6

Speed limits for questions and modules

Question 7

What is the largest potential risk of conducting spot checks and accompaniments

Answer 7

They can lead to or exacerbate social desirability bias

The ultimate concern, of course, is that an extra observer might bias the data in some way, maybe by increasing the social desirability effect whereby respondents are uncomfortable giving unpopular or embarrassing responses

Question 8

Which characteristics do back-checks and audio audits share?

Answer 8

Assigned to all enumerators
Usually conducted with a subset of questions
Enumerators are unaware of which questions are audited

Question 9

What goals can be accomplished by reviewing audio audits in the office?

Answer 9

We can review the way questions were asked to see if protocols are followed

Review respondents’ answers to check for accuracy

Question 10

Spot checks

Answer 10

• Happen in the field, when enumerators are collecting data

– Unannounced visits to observe a survey
– Done by supervisors or field managers (higher-level
managers may get involved if required)

Question 11

Accompaniments

Answer 11

Happen in the field, when enumerators are collecting data

– Visit to observe a survey that the surveyor knows about
– Done by supervisors or field managers (higher-level
managers may get involved if required)

Question 12

Back-checks

Answer 12

Audits; partly in the field, partly in the office

– Different enumerator (or supervisor) revisits sub-sample of already-interviewed respondents
– Administers audit questionnaire

Question 13

Audio audits

Answer 13

Audits; partly in the field, partly in the office

– Original surveyor is simultaneously administering survey and being audited through recording device
– Digital data: Software is programmed to randomly or fully audio record as survey is being administered
• In the office, audit data is reconciled with surveyor-collected data

Question 14

Logic checks

Answer 14

– Check that all interviews were completed
– Check that there are no duplicate observations
– Check that all surveys have consent
– Check that certain critical variables have no missing values
– Check that follow-up record IDs match original
– Check skip patterns and constraints
– Check that no variable has all missing values
– Check specify-other vars for items that can be included
– Check that date values fall within survey range
– Check outliers for numeric fields

Question 15

Enumerator checks

Answer 15

– Check the percentage of “don’t know” and “refusal”
responses by enumerator
– Check the distribution of responses for key questions by enumerator
– Check the number of surveys per day by enumerator
– Check the average interview duration by enumerator
– Check the duration of consent by enumerator
– Check the duration of other modules by enumerator (e.g., anthropometrics, games, etc.)

Question 16

Project checks

Answer 16

– Overall survey progress relative to planned sample
– Summaries of key research variables
– Two-way summaries of survey variables by
demographic/geographic characteristics
– Attrition rates by type and treatment status
– Comparisons of variables with known distributions
– Maps/GIS: all observations where they’re meant to be?

Question 17

What should automatically be captured when setting up your digital survey instrument?

Answer 17

the survey start and end date and time, version number, and information about the device used. Also capture GPS locations.