Collecting MEAL data Flashcards
DATA QUALITY
The data you collect will never be free of bias. Thus, you need to determine, with the help of your stakeholders, what quality and quantity of data is “good enough” for your decision-making, learning and accountability needs. As you begin to think about collecting MEAL data, it is useful to consider data quality standards.
Data Quality Standard:
VALIDITY
Data are valid when they accurately represent what you intend to measure. In other words, the data you collect helps you measure the indicators you have chosen.
Data Quality Standard:
RELIABILITY
Data are reliable when the collection methods used are stable and consistent. Reliable data are collected by using tools such as questionnaires that can be implemented in the same way multiple times. Consider this factor when you are designing your discussion guides and questionnaires for focus groups and interviews.
Data Quality Standard:
PRECISION
Data are precise when they have a level of detail that gives you an accurate picture of what is happening and enables you to make good decisions. When designing your data collection tools, make sure any subgroups you have identified are incorporated into the design. Accordingly, precise data are collected using appropriate sampling methods.
Data Quality Standard:
INTEGRITY
Data have integrity when they are accurate. Data should be free of the kinds of errors that occur, consciously or unconsciously, when people collect and manage data.
Data Quality Standard:
TIMELINESS
Timely data should be available when you need it for learning that informs decisions and for communication purposes. Data are not useful to you when they arrive too late to inform these processes. This factor plays a significant role in your planning for data collection, which is the reason for the column in the PMP on timing. Design your data collection efforts to coincide with when you need to make decisions, and report to stakeholders. Timeliness should also be factored into the design and implementation of your tools.
Data Collection Tools Outline
1. INTRODUCTION
The introduction to your tool gives you the chance to explain the project and the data collection process to the respondent. This overview should explain:
● Why information is being collected
● How participants were identified
● How the data will be collected
● How much time the data collection will take
● How the data will be used
● Who will have access to the data
Of particular importance in the introduction is an explanation of the ethical principles that guide your data collection efforts.
Data Collection Tools Outline
QUESTIONS
After the introduction, your tool lists the questions to be asked of the respondent that are designed to gather the data you need to meet your information requirements. The specific design of questions is dependent on the type of tool you are using.
● Ensure that the language you use in your questions is simple, clear and free of jargon.
● Organize questions using a clear, orderly sequence.
● Make sure that your data collection tool includes fields to record important data analysis and management information (date, location, participant identification or pseudonym, etc.)
Data Collection Tool Outline
CONCLUSION
All tools should close by offering the respondent a chance to ask questions and provide feedback on the experience. Always thank participants for their time and reiterate how the data will be used and when respondents might be able to hear the results of the data collection effort.
Quantitative data collection tools
QUESTIONNAIRE
A questionnaire is structured set of questions designed to elicit specific information from respondents.
Questionnaire
CLOSED-ENDED QUESTIONS
They are questions that provide a predefined list of answer options. This makes it easier for responses to be coded numerically allowing for statistical analysis.
Questionnaire
GUIDELINES FOR DESIGNING
● Questionnaires include “skip logic,” which allows respondents to skip a question based on their answer to a previous question.
● Questions include the option to answer “I don’t know,” as appropriate.
● Questions include all appropriate responses. These responses should be exhaustive, should be very different from each other and shouldn’t overlap.
● In many cases, it is not feasible to include every possible category of response, in which case an “Other” category, with a space for the respondent to fill in a more specific response, is a good solution.
Questionnaire
DELIVERY METHODS
- Personal Interview
- Self-administered
Questionnaire Delivery Methods
ADVANTAGES
- Personal Interview
● Respondents don’t need to be literate
● Facilitators can motivate and support respondents
● There is a high rate of cooperation and a low rate of refusal - Self-administered
● Easy and cheap to distribute
● Access to a broader population in a larger geographic area
Questionnaire Delivery Methods
DISADVANTAGES
- Personal Interview
● Activities are time-consuming and expensive
● Facilitators can influence respondents’ interpretation of questions (and their responses)
● Data entry can be difficult if responses are not collected using digital devices - Self-Administered
● Requires respondent literacy
● Data input can be cumbersome if responses are not collected using digital devices
● Potentially low response rates
Questionnaire Delivery Methods
REQUIREMENTS
- Personal Interview
● Space and privacy for interviews
● Budget for travel
● Trained facilitators - Self-administered
● Logistics for distributing and collecting questionnaires
● Budget for distribution and collection of questionnaires
QUALITATIVE DATA COLLECTION TOOLS
- Semi-structured Interview
- Focus Group Discussion
Qualitative Data Collection Tools
SEMI-STRUCTURED INTERVIEW
A guided discussion between an interviewer and a single respondent designed to explore and understand the rich depth and context of the respondent’s perspectives, opinions and ideas.
Qualitative Data Collection Tools
FOCUS GROUP DISCUSSION
A guided discussion between respondents in a group. It is a qualitative data collection tool designed to explore and understand the rich depth and context of a group’s perspectives, opinions and ideas. As well as an experienced facilitator, they require a notetaker.
Focus Group Discussions
PARTICIPANTS
For focus group discussions, it is crucial to recruit the right participants. Typically, a focus group includes 8 to 12 participants. Once you have narrowed down the topics and questions, you’ll have a better understanding of who should participate in the discussion. Choose participants who can speak directly to the perspectives or experiences that you are interested in knowing about. When participants speak about personal perspectives and experiences, there is an increased likelihood of lively discussion, which leads to richer information and more reliable data. Also, identify focus group participants with a shared characteristic or experience so the discussion doesn’t become an unfocused brainstorm.
Qualitative Data Collection Tools
QUESTIONS
- Open-ended questions:
A. Content-mapping questions
B. Content-mining questions
Qualitative Data Collection Tools
OPEN-ENDED QUESTIONS
They are those that allow someone to give a free-form response in their own words.
Opend-ended questions
CONTENT-MAPPING QUESTIONS
They are also known as opening questions. These are intended to initiate the exploration of a topic by raising and broadly exploring an issue.
Opend-ended questions
CONTENT-MINING QUESTIONS
They are also known as probing questions. These are follow-up questions that elicit more detail or explanation about a response to a content-mapping question.
Unlike content-mapping questions, content-mining questions are unscripted and free-form. Facilitators must have the skills and flexibility to adapt the flow of the conversation and ask the right content-mining questions. Content-mining questions enable the facilitator to explore a topic more deeply and investigate unanticipated topics.
Data Collection Tools
SAMPLE
A sample is a subset of the population or community that you choose to study that will help you understand the population or community as a whole
Data Collection Tools
TYPES OF SAMPLING
- Random sampling
- Purposive sampling
Sampling
RANDOM SAMPLING
Random sampling is a probability sample that includes respondents selected from a list of the entire population of interest so that each respondent has an equal chance of being selected.
Random sampling is used when you plan to use quantitative methods and analysis.
Random samples are created using mathematical calculations to identify how many people will participate in your data gathering efforts. These calculations are developed based on how strong you need your analysis results to be and how varied the population is.
Sampling
SAMPLING BIAS
Sampling bias occurs when some members of the population are more or less likely to be selected for participation in your data gathering efforts than others.
When your sample is biased, you are not taking into consideration all the available perspectives, ideas and opinions. This means that your data will not be as valid (accurate) and cannot be easily generalized to the population you want to address.
Sampling Bias
TYPES OF BIAS
Pay close attention to two specific types of bias that can be especially problematic:
- Convenience sampling bias
- Voluntary response bias
Sampling Bias
CONVENIENCE SAMPLING BIAS
Convenience sampling bias occurs when data are collected from respondents who are easy to reach, or who are easy to work with. Data that suffer from convenience sampling bias could run the risk of over-representing people located closer to main roads, or groups that are fluent in the predominant language.
Sampling Bias
VOLUNTARY RESPONSE BIAS
Voluntary response bias occurs when data are collected disproportionately from self-selected volunteers. Data that suffer from voluntary response bias, could run the risk of under-representing people with busy schedules or people who travel frequently, and over-representing people with strong opinions or specific agendas related to the project.
Sampling
GENERALIZATION
Generalization is possible when data gathered from a sample accurately represent the general population from which the sample was drawn.
Random Sampling
STEPS TO IDENTIFY A RANDOM SAMPLE
- Define population and sampling unit
- Choose a method to calculate random sample
- Determine your sample size
- Select your sample units
Steps to identify a Random Sample:
1. Define population and sample unit
POPULATION
Population is set of similar people, items or events that is of interest for some question or experiment.
Steps to identify a Random Sample:
1. Define population and sample unit
SAMPLE UNIT
Sampling unit is the individual person, category of people, or object from whom/which the measurement (observation) is taken.
Steps to identify a Random Sample:
2. Choose a method to calculate random sample
RANDOM SAMPLING METHODS
- Simple random sample
- Systematic sample
- Cluster sampling
Steps to identify a Random Sample:
2. Choose a method to calculate random sample
Method: SIMPLE RANDOM SAMPLE
Every unit in your population has an equal chance of being selected.
Steps to identify a Random Sample:
2. Choose a method to calculate random sample
Method: SYSTEMATIC SAMPLE
A process of listing and numbering all potential subjects and then selecting every 10th person, for example, until you have reached your sample size.
Steps to identify a Random Sample:
2. Choose a method to calculate random sample
Method: CLUSTER SAMPLING
The population is divided into naturally occurring clusters such as geographical areas, schools or places of employment. All the clusters are listed, and a sample of clusters is randomly selected.
In some cases, all subjects in the cluster are included in the data collection. In other cases, teams will conduct a two-stage cluster sampling process in which participants are chosen from within the cluster and serve as a sample group for the cluster.
Steps to identify a Random Sample:
2. Choose a method to calculate random sample
STRATIFIED SAMPLE
Stratified sample A type of sampling method in which the population is divided into separate subgroups, called strata. Then, a probability sample is drawn from each subgroup, which allows for the statistical comparison of results within the sample.
Steps to identify a Random Sample:
3. Determine your sample size
MARGIN OF ERROR
Margin of error expresses the maximum expected difference between the true population and the sample estimate. To be meaningful, the margin of error should be qualified by a probability statement (often expressed in the form of a confidence level).
Steps to identify a Random Sample:
3. Determine your sample size
CONFIDENCE LEVEL
Confidence level refers to the percentage of all possible samples that can be expected to include the true population parameter.
Steps to identify a Random Sample:
3. Determine your sample size
MARGIN OF ERROR
X
CONFIDENCE LEVEL
Decisions about margin of error and confidence level should be determined by MEAL experts on your team based on your information needs, the context you work in, and the resources available for MEAL. However, as a general guideline, the confidence level will increase (and the margin of error will decrease) as you increase your sample size.
Steps to identify a Random Sample:
4. Select your sample units
SAMPLE FRAME
Sample frame A specific list of units (men, women, households, individuals, children, adolescents, etc.) that you will use to generate your sample. Examples could be a census list or a list of employed teachers, a registration log or a list of project participants.
Sampling
PURPOSIVE SAMPLING
Purposive (selective) sampling is a non-probability sample where sampling units that are investigated are based on the judgement of the researcher. Sampling units are selected based on characteristics of a population and the objective of the study.
Sampling
USING PURPOSIVE SAMPLING
Purposive sampling is used primarily when you want to collect qualitative data. In this kind of sampling, your sample units are deliberately, rather than randomly, selected to reflect important features of groups within the sampled population.
Purposive Sampling
STEPS TO IDENTIFY A PURPOSIVE SAMPLE
- Identify the type of purposive sampling you desire
- Determine your sample size
Steps to identify a purposive sample
1. Identify the type of purposive sample you desire
PURPOSIVE SAMPLING METHODS
- Best and worst case sampling
- Typical case sampling
- Critical case sampling
- Quota sampling
- Snowball or chain sampling
Steps to identify a purposive sample
1. Identify the type of purposive sample you desire
Method: BEST AND WORST CASE SAMPLING
Compares communities or individuals who are considered best and worst cases based on certain characteristics. (i.e. most vulnerable and least vulnerable).
Steps to identify a purposive sample
1. Identify the type of purposive sample you desire
Method: TYPICAL CASE SAMPLING
Provides an understanding of the general scenario by choosing those communities or individuals who are considered average.
Steps to identify a purposive sample
1. Identify the type of purposive sample you desire
Method: CRITICAL CASE SAMPLING
Collects information from communities or individuals who are important for understanding a particular context or situation.
Steps to identify a purposive sample
1. Identify the type of purposive sample you desire
Method: QUOTA SAMPLING
Attempts to collect information from participants with characteristics of interest according to estimates of their proportion in the population.
Steps to identify a purposive sample
1. Identify the type of purposive sample you desire
Method: SNOWBALL OR CHAIN SAMPLING
Collects information from participants in stages, starting with respondents known to the evaluators or partners and then asking those respondents for recommendations of who else to speak to. The advantage of this method is that it helps you identify sources of information previously unknown to you.
Steps to identify a purposive sample
2. Determine your sample size
NUMBER OF FOCUS GROUPS DISCUSSIONS
Experience tells us that 80 percent of themes about an area of interest are identified by two to three focus group discussions. Furthermore, 90 percent of themes are identified by holding six to nine focus group discussions.
Steps to identify a purposive sample
2. Determine your sample size
RELEVANT FACTORS
● If the data analysis plan in your PMP requires that you compare subgroups, you will require a larger sample, and the size increases exponentially the more subgroups you have. For example, if you want to look at both large- and small-sized families, you will need to hold two to three focus groups for each of these subgroups.
● Budget constraints and resource limitations can influence your sample size decisions. You may need to limit the number of subgroups you compare (and associated data collection events) if you lack the resources to implement data collection events.
Using Data Collection Tools
STEPS FOR USING DATA COLLECTION TOOLS
- Translate your data collection tools
- Train data collectors and test your tools
- Revise and finalize your tools
- Plan for implementation and data management
Using Data Collection Tools
Steps for using data collection tools
- TRANSLATE YOUR DATA COLLECTION TOOLS
Is your project working in a region that uses multiple languages? If so, then your tool will need to be translated so that it is not biased toward those who speak the initial language of your tool.
Using Data Collection Tools
Steps for using data collection tools
- TRAIN DATA COLLECTORS AND TEST YOUR TOOLS
Written instructions accompanying your collection tool are essential. Often, additional training is also needed, for both new collectors and as a refresher for those who are skilled. Training should include the following:
● An explanation of the basic ethical principles of good data collection.
● An explanation of the purpose of the tool. Make sure everyone using the tool understands the purpose of each question and how the answers received will feed into analysis and use.
● Instruction that emphasizes the skills needed to use the tool. Data collectors need skills to collect high-quality data. The skills needed for quantitative and qualitative data collection are often different. For example, when collecting quantitative data, enumerators need training to know the order of questions to ask and how to ask them without leading respondents. When collecting qualitative data, interviewers need to be able to elicit information from respondents while making them feel comfortable, and must create a trusting relationship with respondents while remaining neutral in attitude and appearance.
● The opportunity to physically test the tool with potential respondents.
Using Data Collection Tools
Steps for using data collection tools
- TRAIN DATA COLLECTORS AND TEST YOUR TOOLS: PURPOSES
Training data collectors serves two purposes: building the skills of your data collectors and ensuring that your tool works as it should. You must always test your tool, a process that can be built directly into your training.
Using Data Collection Tools
Steps for using data collection tools
- REVISE AND FINALIZE YOUR TOOLS
After you have tested your tool, any revisions can be incorporated into your final document.
Using Data Collection Tools
Steps for using data collection tools
- PLAN FOR IMPLEMENTATION AND DATA MANAGEMENT
● Allow enough time for each data collection event.
● Choose a venue for interviews and focus group discussions that provides privacy and an appropriate level of comfort.
● Identify how you intend to manage the data you collect.
○ Who will be responsible for entering the data into the selected databases
○ Who will be responsible for conducting data quality checks and when.
○ How you will protect and store questionnaires once they have been completed.
○ How you will protect the privacy of respondents and who will be responsible for this function
Managing Data
DATA MANAGEMENT
Data management is the process of managing data through the phases of its life. Complete data management includes four primary components: entry, cleaning, storage and security, and retention and disposal.
Data Management
PRIMARY COMPONENTS
- Entry
- Cleaning
- Storage and security
- Retention and disposal
Primary Components of Data Management
1. DATA ENTRY
The term “data entry” means putting the data you have collected into a form you can use by entering it into an electronic database. Effectively using a database improves your ability to:
● Access, manage and share data
● Improve data security and protection
● Integrate data more effectively
● Manage data quality
● Facilitate timely decision-making
- Data Entry
STEPS
- Create a data entry protocol
- If necessary, identify your requirements for entering data
- Data Entry
DATA ENTRY PROTOCOL
Inconsistent data entry procedures and data entry errors can compromise your data, analysis and MEAL findings. To reduce this risk, create a standard data entry protocol that includes guidance on:
● The data entry process, outlining the rules and instructions for entering data into the database.
● The timing of data entry to ensure that data are available to meet reporting requirements and decision-making needs.
- Data Entry
REQUIREMENT FOR THOSE ENTERING DATA
Most data entry is now conducted electronically, often using digital devices to collect information that is then uploaded automatically into the project MEAL database. However, in some environments, there may still be a need to input data by hand. Any data entry protocols you create should clearly indicate whether those inputting data require previous experience or training. If appropriate, identify a supervisor who is ultimately responsible for quality management of the data entry process.
