Collecting MEAL data

Question 1

DATA QUALITY

Answer 1

The data you collect will never be free of bias. Thus, you need to determine, with the help of your stakeholders, what quality and quantity of data is “good enough” for your decision-making, learning and accountability needs. As you begin to think about collecting MEAL data, it is useful to consider data quality standards.

Question 2

Data Quality Standard:
VALIDITY

Answer 2

Data are valid when they accurately represent what you intend to measure. In other words, the data you collect helps you measure the indicators you have chosen.

Question 3

Data Quality Standard:
RELIABILITY

Answer 3

Data are reliable when the collection methods used are stable and consistent. Reliable data are collected by using tools such as questionnaires that can be implemented in the same way multiple times. Consider this factor when you are designing your discussion guides and questionnaires for focus groups and interviews.

Question 4

Data Quality Standard:
PRECISION

Answer 4

Data are precise when they have a level of detail that gives you an accurate picture of what is happening and enables you to make good decisions. When designing your data collection tools, make sure any subgroups you have identified are incorporated into the design. Accordingly, precise data are collected using appropriate sampling methods.

Question 5

Data Quality Standard:
INTEGRITY

Answer 5

Data have integrity when they are accurate. Data should be free of the kinds of errors that occur, consciously or unconsciously, when people collect and manage data.

Question 6

Data Quality Standard:
TIMELINESS

Answer 6

Timely data should be available when you need it for learning that informs decisions and for communication purposes. Data are not useful to you when they arrive too late to inform these processes. This factor plays a significant role in your planning for data collection, which is the reason for the column in the PMP on timing. Design your data collection efforts to coincide with when you need to make decisions, and report to stakeholders. Timeliness should also be factored into the design and implementation of your tools.

Question 7

Data Collection Tools Outline
1. INTRODUCTION

Answer 7

The introduction to your tool gives you the chance to explain the project and the data collection process to the respondent. This overview should explain:

● Why information is being collected
● How participants were identified
● How the data will be collected
● How much time the data collection will take
● How the data will be used
● Who will have access to the data

Of particular importance in the introduction is an explanation of the ethical principles that guide your data collection efforts.

Question 8

Data Collection Tools Outline
QUESTIONS

Answer 8

After the introduction, your tool lists the questions to be asked of the respondent that are designed to gather the data you need to meet your information requirements. The specific design of questions is dependent on the type of tool you are using.
● Ensure that the language you use in your questions is simple, clear and free of jargon.
● Organize questions using a clear, orderly sequence.
● Make sure that your data collection tool includes fields to record important data analysis and management information (date, location, participant identification or pseudonym, etc.)

Question 9

Data Collection Tool Outline
CONCLUSION

Answer 9

All tools should close by offering the respondent a chance to ask questions and provide feedback on the experience. Always thank participants for their time and reiterate how the data will be used and when respondents might be able to hear the results of the data collection effort.

Question 10

Quantitative data collection tools
QUESTIONNAIRE

Answer 10

A questionnaire is structured set of questions designed to elicit specific information from respondents.

Question 11

Questionnaire
CLOSED-ENDED QUESTIONS

Answer 11

They are questions that provide a predefined list of answer options. This makes it easier for responses to be coded numerically allowing for statistical analysis.

Question 12

Questionnaire
GUIDELINES FOR DESIGNING

Answer 12

● Questionnaires include “skip logic,” which allows respondents to skip a question based on their answer to a previous question.
● Questions include the option to answer “I don’t know,” as appropriate.
● Questions include all appropriate responses. These responses should be exhaustive, should be very different from each other and shouldn’t overlap.
● In many cases, it is not feasible to include every possible category of response, in which case an “Other” category, with a space for the respondent to fill in a more specific response, is a good solution.

Question 13

Questionnaire
DELIVERY METHODS

Answer 13

1. Personal Interview
2. Self-administered

Question 14

Questionnaire Delivery Methods
ADVANTAGES

Answer 14

1. Personal Interview
   ● Respondents don’t need to be literate
   ● Facilitators can motivate and support respondents
   ● There is a high rate of cooperation and a low rate of refusal
2. Self-administered
   ● Easy and cheap to distribute
   ● Access to a broader population in a larger geographic area

Question 15

Questionnaire Delivery Methods
DISADVANTAGES

Answer 15

1. Personal Interview
   ● Activities are time-consuming and expensive
   ● Facilitators can influence respondents’ interpretation of questions (and their responses)
   ● Data entry can be difficult if responses are not collected using digital devices
2. Self-Administered
   ● Requires respondent literacy
   ● Data input can be cumbersome if responses are not collected using digital devices
   ● Potentially low response rates

Question 16

Questionnaire Delivery Methods
REQUIREMENTS

Answer 16

1. Personal Interview
   ● Space and privacy for interviews
   ● Budget for travel
   ● Trained facilitators
2. Self-administered
   ● Logistics for distributing and collecting questionnaires
   ● Budget for distribution and collection of questionnaires

Question 17

QUALITATIVE DATA COLLECTION TOOLS

Answer 17

1. Semi-structured Interview
2. Focus Group Discussion

Question 18

Qualitative Data Collection Tools
SEMI-STRUCTURED INTERVIEW

Answer 18

A guided discussion between an interviewer and a single respondent designed to explore and understand the rich depth and context of the respondent’s perspectives, opinions and ideas.

Question 19

Qualitative Data Collection Tools
FOCUS GROUP DISCUSSION

Answer 19

A guided discussion between respondents in a group. It is a qualitative data collection tool designed to explore and understand the rich depth and context of a group’s perspectives, opinions and ideas. As well as an experienced facilitator, they require a notetaker.

Question 20

Focus Group Discussions
PARTICIPANTS

Answer 20

For focus group discussions, it is crucial to recruit the right participants. Typically, a focus group includes 8 to 12 participants. Once you have narrowed down the topics and questions, you’ll have a better understanding of who should participate in the discussion. Choose participants who can speak directly to the perspectives or experiences that you are interested in knowing about. When participants speak about personal perspectives and experiences, there is an increased likelihood of lively discussion, which leads to richer information and more reliable data. Also, identify focus group participants with a shared characteristic or experience so the discussion doesn’t become an unfocused brainstorm.

Question 21

Qualitative Data Collection Tools
QUESTIONS

Answer 21

1. Open-ended questions:
   A. Content-mapping questions
   B. Content-mining questions

Question 22

Qualitative Data Collection Tools
OPEN-ENDED QUESTIONS

Answer 22

They are those that allow someone to give a free-form response in their own words.

Question 23

Opend-ended questions
CONTENT-MAPPING QUESTIONS

Answer 23

They are also known as opening questions. These are intended to initiate the exploration of a topic by raising and broadly exploring an issue.

Question 24

Opend-ended questions
CONTENT-MINING QUESTIONS

Answer 24

They are also known as probing questions. These are follow-up questions that elicit more detail or explanation about a response to a content-mapping question.

Unlike content-mapping questions, content-mining questions are unscripted and free-form. Facilitators must have the skills and flexibility to adapt the flow of the conversation and ask the right content-mining questions. Content-mining questions enable the facilitator to explore a topic more deeply and investigate unanticipated topics.

Question 25

Data Collection Tools
SAMPLE

Answer 25

A sample is a subset of the population or community that you choose to study that will help you understand the population or community as a whole

Question 26

Data Collection Tools
TYPES OF SAMPLING

Answer 26

1. Random sampling
2. Purposive sampling

Question 27

Sampling
RANDOM SAMPLING

Answer 27

Random sampling is a probability sample that includes respondents selected from a list of the entire population of interest so that each respondent has an equal chance of being selected.

Random sampling is used when you plan to use quantitative methods and analysis.

Random samples are created using mathematical calculations to identify how many people will participate in your data gathering efforts. These calculations are developed based on how strong you need your analysis results to be and how varied the population is.

Question 28

Sampling
SAMPLING BIAS

Answer 28

Sampling bias occurs when some members of the population are more or less likely to be selected for participation in your data gathering efforts than others.

When your sample is biased, you are not taking into consideration all the available perspectives, ideas and opinions. This means that your data will not be as valid (accurate) and cannot be easily generalized to the population you want to address.

Question 29

Sampling Bias
TYPES OF BIAS

Answer 29

Pay close attention to two specific types of bias that can be especially problematic:

1. Convenience sampling bias
2. Voluntary response bias

Question 30

Sampling Bias
CONVENIENCE SAMPLING BIAS

Answer 30

Convenience sampling bias occurs when data are collected from respondents who are easy to reach, or who are easy to work with. Data that suffer from convenience sampling bias could run the risk of over-representing people located closer to main roads, or groups that are fluent in the predominant language.

Question 31

Sampling Bias
VOLUNTARY RESPONSE BIAS

Answer 31

Voluntary response bias occurs when data are collected disproportionately from self-selected volunteers. Data that suffer from voluntary response bias, could run the risk of under-representing people with busy schedules or people who travel frequently, and over-representing people with strong opinions or specific agendas related to the project.

Question 32

Sampling
GENERALIZATION

Answer 32

Generalization is possible when data gathered from a sample accurately represent the general population from which the sample was drawn.

Question 33

Random Sampling
STEPS TO IDENTIFY A RANDOM SAMPLE

Answer 33

1. Define population and sampling unit
2. Choose a method to calculate random sample
3. Determine your sample size
4. Select your sample units

Question 34

Steps to identify a Random Sample:
1. Define population and sample unit

POPULATION

Answer 34

Population is set of similar people, items or events that is of interest for some question or experiment.

Question 35

Steps to identify a Random Sample:
1. Define population and sample unit

SAMPLE UNIT

Answer 35

Sampling unit is the individual person, category of people, or object from whom/which the measurement (observation) is taken.

Question 36

Steps to identify a Random Sample:
2. Choose a method to calculate random sample

RANDOM SAMPLING METHODS

Answer 36

1. Simple random sample
2. Systematic sample
3. Cluster sampling

Question 37

Steps to identify a Random Sample:
2. Choose a method to calculate random sample

Method: SIMPLE RANDOM SAMPLE

Answer 37

Every unit in your population has an equal chance of being selected.

Question 38

Steps to identify a Random Sample:
2. Choose a method to calculate random sample

Method: SYSTEMATIC SAMPLE

Answer 38

A process of listing and numbering all potential subjects and then selecting every 10th person, for example, until you have reached your sample size.

Question 39

Steps to identify a Random Sample:
2. Choose a method to calculate random sample

Method: CLUSTER SAMPLING

Answer 39

The population is divided into naturally occurring clusters such as geographical areas, schools or places of employment. All the clusters are listed, and a sample of clusters is randomly selected.
In some cases, all subjects in the cluster are included in the data collection. In other cases, teams will conduct a two-stage cluster sampling process in which participants are chosen from within the cluster and serve as a sample group for the cluster.

Question 40

Steps to identify a Random Sample:
2. Choose a method to calculate random sample

STRATIFIED SAMPLE

Answer 40

Stratified sample A type of sampling method in which the population is divided into separate subgroups, called strata. Then, a probability sample is drawn from each subgroup, which allows for the statistical comparison of results within the sample.

Question 41

Steps to identify a Random Sample:
3. Determine your sample size

MARGIN OF ERROR

Answer 41

Margin of error expresses the maximum expected difference between the true population and the sample estimate. To be meaningful, the margin of error should be qualified by a probability statement (often expressed in the form of a confidence level).

Question 42

Steps to identify a Random Sample:
3. Determine your sample size

CONFIDENCE LEVEL

Answer 42

Confidence level refers to the percentage of all possible samples that can be expected to include the true population parameter.

Question 43

Steps to identify a Random Sample:
3. Determine your sample size

MARGIN OF ERROR
X
CONFIDENCE LEVEL

Answer 43

Decisions about margin of error and confidence level should be determined by MEAL experts on your team based on your information needs, the context you work in, and the resources available for MEAL. However, as a general guideline, the confidence level will increase (and the margin of error will decrease) as you increase your sample size.

Question 44

Steps to identify a Random Sample:
4. Select your sample units

SAMPLE FRAME

Answer 44

Sample frame A specific list of units (men, women, households, individuals, children, adolescents, etc.) that you will use to generate your sample. Examples could be a census list or a list of employed teachers, a registration log or a list of project participants.

Question 45

Sampling
PURPOSIVE SAMPLING

Answer 45

Purposive (selective) sampling is a non-probability sample where sampling units that are investigated are based on the judgement of the researcher. Sampling units are selected based on characteristics of a population and the objective of the study.

Question 46

Sampling
USING PURPOSIVE SAMPLING

Answer 46

Purposive sampling is used primarily when you want to collect qualitative data. In this kind of sampling, your sample units are deliberately, rather than randomly, selected to reflect important features of groups within the sampled population.

Question 47

Purposive Sampling
STEPS TO IDENTIFY A PURPOSIVE SAMPLE

Answer 47

1. Identify the type of purposive sampling you desire
2. Determine your sample size

Question 48

Steps to identify a purposive sample
1. Identify the type of purposive sample you desire

PURPOSIVE SAMPLING METHODS

Answer 48

1. Best and worst case sampling
2. Typical case sampling
3. Critical case sampling
4. Quota sampling
5. Snowball or chain sampling

Question 49

Steps to identify a purposive sample
1. Identify the type of purposive sample you desire

Method: BEST AND WORST CASE SAMPLING

Answer 49

Compares communities or individuals who are considered best and worst cases based on certain characteristics. (i.e. most vulnerable and least vulnerable).

Question 50

Steps to identify a purposive sample
1. Identify the type of purposive sample you desire

Method: TYPICAL CASE SAMPLING

Answer 50

Provides an understanding of the general scenario by choosing those communities or individuals who are considered average.

Question 51

Steps to identify a purposive sample
1. Identify the type of purposive sample you desire

Method: CRITICAL CASE SAMPLING

Answer 51

Collects information from communities or individuals who are important for understanding a particular context or situation.

Question 52

Steps to identify a purposive sample
1. Identify the type of purposive sample you desire

Method: QUOTA SAMPLING

Answer 52

Attempts to collect information from participants with characteristics of interest according to estimates of their proportion in the population.

Question 53

Steps to identify a purposive sample
1. Identify the type of purposive sample you desire

Method: SNOWBALL OR CHAIN SAMPLING

Answer 53

Collects information from participants in stages, starting with respondents known to the evaluators or partners and then asking those respondents for recommendations of who else to speak to. The advantage of this method is that it helps you identify sources of information previously unknown to you.

Question 54

Steps to identify a purposive sample
2. Determine your sample size

NUMBER OF FOCUS GROUPS DISCUSSIONS

Answer 54

Experience tells us that 80 percent of themes about an area of interest are identified by two to three focus group discussions. Furthermore, 90 percent of themes are identified by holding six to nine focus group discussions.

Question 55

Steps to identify a purposive sample
2. Determine your sample size

RELEVANT FACTORS

Answer 55

● If the data analysis plan in your PMP requires that you compare subgroups, you will require a larger sample, and the size increases exponentially the more subgroups you have. For example, if you want to look at both large- and small-sized families, you will need to hold two to three focus groups for each of these subgroups.

● Budget constraints and resource limitations can influence your sample size decisions. You may need to limit the number of subgroups you compare (and associated data collection events) if you lack the resources to implement data collection events.

Question 56

Using Data Collection Tools
STEPS FOR USING DATA COLLECTION TOOLS

Answer 56

1. Translate your data collection tools
2. Train data collectors and test your tools
3. Revise and finalize your tools
4. Plan for implementation and data management

Question 57

Using Data Collection Tools
Steps for using data collection tools

1. TRANSLATE YOUR DATA COLLECTION TOOLS

Answer 57

Is your project working in a region that uses multiple languages? If so, then your tool will need to be translated so that it is not biased toward those who speak the initial language of your tool.

Question 58

Using Data Collection Tools
Steps for using data collection tools

2. TRAIN DATA COLLECTORS AND TEST YOUR TOOLS

Answer 58

Written instructions accompanying your collection tool are essential. Often, additional training is also needed, for both new collectors and as a refresher for those who are skilled. Training should include the following:
● An explanation of the basic ethical principles of good data collection.
● An explanation of the purpose of the tool. Make sure everyone using the tool understands the purpose of each question and how the answers received will feed into analysis and use.
● Instruction that emphasizes the skills needed to use the tool. Data collectors need skills to collect high-quality data. The skills needed for quantitative and qualitative data collection are often different. For example, when collecting quantitative data, enumerators need training to know the order of questions to ask and how to ask them without leading respondents. When collecting qualitative data, interviewers need to be able to elicit information from respondents while making them feel comfortable, and must create a trusting relationship with respondents while remaining neutral in attitude and appearance.
● The opportunity to physically test the tool with potential respondents.

Question 59

Using Data Collection Tools
Steps for using data collection tools

2. TRAIN DATA COLLECTORS AND TEST YOUR TOOLS: PURPOSES

Answer 59

Training data collectors serves two purposes: building the skills of your data collectors and ensuring that your tool works as it should. You must always test your tool, a process that can be built directly into your training.

Question 60

Using Data Collection Tools
Steps for using data collection tools

3. REVISE AND FINALIZE YOUR TOOLS

Answer 60

After you have tested your tool, any revisions can be incorporated into your final document.

Question 61

Using Data Collection Tools
Steps for using data collection tools

4. PLAN FOR IMPLEMENTATION AND DATA MANAGEMENT

Answer 61

● Allow enough time for each data collection event.
● Choose a venue for interviews and focus group discussions that provides privacy and an appropriate level of comfort.
● Identify how you intend to manage the data you collect.
○ Who will be responsible for entering the data into the selected databases
○ Who will be responsible for conducting data quality checks and when.
○ How you will protect and store questionnaires once they have been completed.
○ How you will protect the privacy of respondents and who will be responsible for this function

Question 62

Managing Data

DATA MANAGEMENT

Answer 62

Data management is the process of managing data through the phases of its life. Complete data management includes four primary components: entry, cleaning, storage and security, and retention and disposal.

Question 63

Data Management
PRIMARY COMPONENTS

Answer 63

1. Entry
2. Cleaning
3. Storage and security
4. Retention and disposal

Question 64

Primary Components of Data Management
1. DATA ENTRY

Answer 64

The term “data entry” means putting the data you have collected into a form you can use by entering it into an electronic database. Effectively using a database improves your ability to:
● Access, manage and share data
● Improve data security and protection
● Integrate data more effectively
● Manage data quality
● Facilitate timely decision-making

Question 65

1. Data Entry
   STEPS

Answer 65

1. Create a data entry protocol
2. If necessary, identify your requirements for entering data

Question 66

1. Data Entry
   DATA ENTRY PROTOCOL

Answer 66

Inconsistent data entry procedures and data entry errors can compromise your data, analysis and MEAL findings. To reduce this risk, create a standard data entry protocol that includes guidance on:
● The data entry process, outlining the rules and instructions for entering data into the database.
● The timing of data entry to ensure that data are available to meet reporting requirements and decision-making needs.

Question 67

1. Data Entry
   REQUIREMENT FOR THOSE ENTERING DATA

Answer 67

Most data entry is now conducted electronically, often using digital devices to collect information that is then uploaded automatically into the project MEAL database. However, in some environments, there may still be a need to input data by hand. Any data entry protocols you create should clearly indicate whether those inputting data require previous experience or training. If appropriate, identify a supervisor who is ultimately responsible for quality management of the data entry process.