Download StudyGeneral

Study Overview

Title:
The Millennium Villages Project: A protocol for the final evaluation - Malawi
Study is 3ie funded:
No
Study ID:
RIDIE-STUDY-ID-552fd6b95a320
Initial Registration Date:
04/16/2015
Last Update Date:
04/28/2015
Study Status:
Ongoing
Location(s):
Malawi
Abstract:
The Millennium Villages Project is a ten-year integrated rural development project implemented in ten sub-Saharan African sites. The final evaluation of the MVP will include an adequacy assessment, impact evaluation, cost assessment, process evaluation, and description of systems design and tools for the ten sites. Taken together, this evaluation will assess the Millennium Village Project’s model for achieving the Millennium Development Goals in rural sub-Saharan Africa. For the impact evaluation, one very important stage is the selection of control villages using a combination of matching and random selection. This process and its timing differs somewhat across the ten countries, requiring each country to be registered separately. This is the registration for Malawi. After the selection of comparison villages, survey data will be collected and regression models fit to estimate causal effects on a variety of poverty, education, and health outcomes.
Registration Citation:
Categories:
Agriculture and Rural Development
Education
Energy
Health, Nutrition, and Population
Multisector
Transportation
Water and Sanitation
Additional Keywords:
Secondary ID Number(s):

Principal Investigator(s)

Name of First PI:
Jeffrey D. Sachs
Affiliation:
Columbia University, The Earth Institute
Name of Second PI:
Affiliation:

Research Partner

Location:
Intervention

Intervention Overview

Intervention:
The Millennium Villages Project (MVP) was initiated in 2005 to implement the UN Millennium Project’s recommended interventions across multiple sectors in rural Africa. The MVP was piloted in Sauri, Kenya and Koraro, Ethiopia in 2005, and expanded in 2006 to include fourteen sites across ten countries covering roughly half-a-million inhabitants in total. The project is a village-cluster-level intervention to facilitate rural populations to achieve the MDGs and move whole communities toward self-sustaining economic growth. MVP interventions target extremely poor, rural communities and individuals within them, with a multi-sector approach addressing food production, nutrition, education, health services, roads, energy, communications, water supply and sanitation, enterprise diversification, environmental management, and business development. The MVP delivers diverse intervention packages, varying by site (country) to address differing community needs, resources, and priorities. In Malawi, MVP technologies and techniques include improved seed and fertilizer, improved cookstoves, insecticide-treated bednets, antiretroviral drugs, and community deworming.
Theory of Change:
Multiple Treatment Arms Evaluated?
No

Implementing Agency

Type of Organization:
Other

Intervention Timing

Intervention or Program Started at time of Registration?
Yes
Start Date:
07/01/2006
End Date:
07/31/2016
Evaluation Method

Evaluation Method Overview

Primary (or First) Evaluation Method:
Other (specify)
Other (not Listed) Method:
We will use pre-treatment variables to match treatment to comparison villages, then regression on treatment and pre-treatment variables in a multi-level Bayesian model fit to 2015 survey data.
Additional Evaluation Method (If Any):
Other (not Listed) Method:

Method Details

Details of Evaluation Approach:
We used Demographic Health Surveys & geographic (not MVP baseline) data to find comparison areas that were similar to MVs in 2005. Algorithms selected 5 grid cells of size equal to the Malawi MV, minimizing distance in means & variances of pre-treatment variables between treatment & control. Villages in matched grid cells were listed & pop estimates retrieved from village leaders. For village size comparability, we restrict selection to villages within the MV pop range (480-1120 people per village as determined by the MVP 2014-15 demographic census). We randomly select one village per grid cell (attached code to be public after data collection). If no villages lie in the MV population range, we will select the village closest in size to the range; if the selected village is found to be outside the grid cell or its leadership refuses participation, we will select another village from the original list (in the same order, except the excluded village) using the original seed. We will collect 2015 survey data and regress outcomes on treatment & pre-treatment variables in multi-level Bayesian models.
Change History for Details of Evaluation Approach
Changed On Previous Value
04/28/2015 We used Demographic Health Surveys & geographic data (not MVP baseline data) to find comparison areas that were similar to MVs in 2005. Algorithms selected 5 grid cells of size equal to the Malawi MV, minimizing distance in means & variances of pre-treatment variables between treatment & control. Villages in matched grid cells were listed & population estimates retrieved from village leaders. For comparability of village size in treatment & control, we restrict selection to villages within the MV population range (480-1120 people per village as determined by the MVP 2014 demographic census). We will set a seed & randomly select one village per grid cell (see below & attached code). If no villages lie in the MV population range, the village closest in size to the range will be selected; if the selected village is found to be outside the grid cell or if its leadership refuses participation, another village will be selected from the original village list (in the same order, except the excluded village) using the original seed. We will collect 2015 survey data and regress outcomes on treatment & pre-treatment variables in multi-level Bayesian models.
Outcomes (Endpoints):
The primary outcomes of interest, for both the adequacy assessment and impact evaluation, are a subset of Millennium Development Goals (MDG) indicators and proxies, including indicators of poverty alleviation, agriculture, education, gender equality, health, environmental sustainability, and infrastructure. What we refer to as “outcomes” are a mixture of output, outcome, and impact indicators. The complete list of outcomes is listed in the appendix of the attached protocol, including indicators such as the proportion of population below 1.25 USD per day, the poverty gap ratio, underweight, wasting, stunting, adjusted net attendance ratio in primary schools, gender parity in primary schools, infant, under-5, and maternal mortality rates, ANC coverage, skilled birth attendance, bed-net usage, HIV testing among pregnant women, malaria testing, access to improved drinking water and sanitation, etc.
Unit of Analysis:
Data will be analyzed at village and unit (household and individual) levels. See attached protocol for more details.
Hypotheses:
We prefer not to frame our evaluation in terms of hypothesis testing, and instead will report estimates and intervals of uncertainty for treatment effects on health and economic development indicators. However, these intervals can be used to test the hypotheses that the MVP had no effect on each of these indicators. Our composite measure (mentioned above) can be used as an overall hypothesis test, should multiple comparisons be a concern.
Unit of Intervention or Assignment:
Interventions were implemented at a village-cluster level. See attached protocol for more details.
Number of Clusters in Sample:
1 village cluster in the Malawi MV (5 villages), 1 village in each of 5 comparison areas, total 700 HH in Malawi.
Number of Individuals in Sample:
In Malawi, total number of expected household observations collected in the treatment areas are approximately 350, and in the comparison areas, approximately 350 as well.
Size of Treatment, Control, or Comparison Subsamples:
In Malawi, total number of expected household observations collected in the treatment areas are approximately 350, and in the comparison areas, approximately 350 as well.
Data

Outcomes Data

Description:
A HH survey will collect demography, assets, income, consumption, expenditure, education, and malaria bed net usage data. Sex-specific adult (15-49 years old) surveys will capture individual health data, a birth history, and child health. A food frequency questionnaire will be administered to adults. Anemia and malaria will be tested in children and adults (6-59 mos, 5-14 years old, 15-49 years old). Weight, height, length, and mid-upper-arm-circumference will be measured in children 6-59 mos.
Data Already Collected?
No
Data Previously Used?
Data Access:
Data Obtained by the Study Researchers?
Data Approval Process:
Approval Status:

Treatment Assignment Data

Participation or Assignment Information:
Yes
Description:
Data Obtained by the Study Researchers?
Data Previously Used?
Data Access:
Data Obtained by the Study Researchers?
Data Approval Process:
Approval Status:

Data Analysis

Data Analysis Status:

Study Materials

Upload Study Materials:

Registration Category

Registration Category:
Prospective, Category 1: Data for measuring impacts have not been collected
Completion

Completion Overview

Intervention Completion Date:
Data Collection Completion Date:
Unit of Analysis:
Clusters in Final Sample:
Total Observations in Final Sample:
Size of Treatment, Control, or Comparison Subsamples:

Findings

Preliminary Report:
Preliminary Report URL:
Summary of Findings:
Paper:
Paper Summary:
Paper Citation:

Data Availability

Data Availability (Primary Data):
Date of Data Availability:
Data URL or Contact:
Access procedure:

Other Materials

Survey:
Survey Instrument Links or Contact:
Program Files:
Program Files Links or Contact:
External Link:
External Link Description:
Description of Changes:

Study Stopped

Date:
Reason: