Download Study

General

Study Overview

Title:

Impact Evaluation of World Vision' ACCESO program: A Study on Food Security and Water Quality in Guatemala

Study is 3ie funded:

No

Study ID:

RIDIE-STUDY-ID-662bede28b076

Initial Registration Date:

04/26/2024

Last Update Date:

04/03/2024

Study Status:

Ongoing

Location(s):

Guatemala

Abstract:

In this study evaluates the effectiveness of the The Assistance for Change and Economic Growth for Food Security (Asistencia para el Cambio y Crecimiento Económico por la Seguridad Alimentaria, ACCESO) program in Guatemala, which aims to improve food security and water quality for vulnerable households. The program provides cash transfers to help families purchase food and distributes water filters to improve the safety of drinking water. The research will assess how these interventions impact household food security, particularly in reducing moderate to severe hunger, and the quality of drinking water, specifically in reducing fecal contamination. The findings will inform future efforts to enhance food security and access to clean water for communities in Guatemala.

Registration Citation:

Categories:

Agriculture and Rural Development
Health, Nutrition, and Population
Social Protection
Other

Additional Keywords:

Food security, water quality

Secondary ID Number(s):

Principal Investigator(s)

Name of First PI:

Miles Kirby

Affiliation:

World Vision

Name of Second PI:

Affiliation:

Study Sponsor

Name:

The Humanitarian Assistance Evidence Cycle (HAEC)

Study Sponsor Location:

United States

Research Partner

Name of Partner Institution:

World Vision

Type of Organization:

NGO-international

Location:

United States

Intervention

Intervention Overview

Intervention:

The ACCESO program in Guatemala aims to address food insecurity and improve drinking water quality among vulnerable households. WV proposes an impact evaluation of the following programmatic activities: the implementation of Multipurpose Cash Assistance (MPCA) as a mechanism for food assistance and access to other products to cover basic needs according to the context in order to understand if MPCA will increase household food security and allow households to restore their livelihoods. The distribution of household water filters (“Ecofiltro”) to provide access to clean water that is free of E. coli. Access to safe drinking water is essential if households experience a self-reported improvement in their health compared to those without water filters. Improved drinking water quality could reduce risk of diarrheal illness, which contributes to nutritional deficits and can impact children and adults. Reduced illness and associated care costs could also provide more financial security.

The participants of the ACCESO program come from primarily agrarian communities and are selected based on vulnerability criteria including food insecurity, households with children under the age of 5 years, households with disabled family members, households that are led by women, and households with elderly family members.

Theory of Change:

The theory of change for the ACCESO program in Guatemala revolves around two primary interventions: direct cash transfers and the distribution of water filters. For the cash transfer component, the theory posits that providing financial resources directly to vulnerable households will enable them to purchase sufficient and nutritious food, thereby improving household food security. The impact evaluation will focus on the mechanism of how these cash transfers lead to reduced instances of moderate to severe hunger, as measured by the Household Hunger Score (HHS). This includes examining whether the cash transfers enable households to access a more diverse and balanced diet, leading to improved nutritional status and reduced food insecurity.

The second component involves the distribution of water filters to improve drinking water quality. The theory suggests that by providing households with these filters, the program will reduce fecal contamination in drinking water, thus lowering the risk of waterborne diseases. The impact evaluation will investigate whether the filters are effectively removing contaminants and providing households with safe and clean drinking water.

In summary, the theory of change for the ACCESO program focuses on two main mechanisms: 1) how direct cash transfers improve household food security and reduce hunger, and 2) how the distribution and use of water filters enhance drinking water quality, leading to better health outcomes. The impact evaluation will delve into these mechanisms to understand the program's effectiveness in achieving its objectives.

Multiple Treatment Arms Evaluated?

Yes

Implementing Agency

Name of Organization:

World Vision

Type of Organization:

NGO (International)

Program Funder

Name of Organization:

USAID’s Bureau for Humanitarian Assistance

Type of Organization:

Foreign or Multilateral Aid Agency

Intervention Timing

Intervention or Program Started at time of Registration?

Yes

Start Date:

07/07/2023

End Date:

Evaluation Method

Evaluation Method Overview

Primary (or First) Evaluation Method:

Randomized control trial

Other (not Listed) Method:

Additional Evaluation Method (If Any):

Matching

Other (not Listed) Method:

Method Details

Details of Evaluation Approach:

The methodological approach for the evaluation of the ACCESO program involves a cluster randomized controlled trial (cRCT) design. This design randomly assigns clusters of households, such as villages, to either the intervention group receiving the ACCESO program's cash transfers and water filters, or the control group receiving no intervention. The impact evaluation focuses on two primary research questions: the impact of the cash transfer component on food security and the impact of the water filter component on fecal contamination of household drinking water. The primary analysis will use an intention-to-treat (ITT) approach at the household level, with generalized estimating equations to account for clustering. Sensitivity analyses will explore variations in cluster size, adjust for strata, and also a secondary analysis would be conducted for treatment-on-the-treated (TOT). Propensity score matching may also be utilized as a robustness check to establish whether the treatment and control groups are statistically equivalent groups based on criteria not related to the intervention. Cost-effectiveness analysis (CEA) will assess the effectiveness of the interventions in improving household food security and water quality, respectively, relative to the costs incurred. The analysis will consider both short-term and longer-term impacts to capture the sustained effects of the interventions.

Outcomes (Endpoints):

The key outcomes of interest are as follows:

Primary Outcomes:

• Reported moderate or severe hunger in September of 2023 (according to Household Hunger Score, with modified extended recall period to specific month of September, asked in December 2023-January 2024)
• Fecal contamination of household drinking water in May-June 2024 (1 or more E. coli colony forming units/100mL).

Secondary outcomes measured in first and second survey rounds (December 2023-January 2024 and estimated May-June 2024) :

• Food Consumption Score (FCS) (last 7 days, binary "acceptable" or not, as well as continuous).
• Reduced Coping Strategies Index (rCSI) (last 7 days, continuous).
• Household Hunger Score in September 2023 (binary, severe hunger or not).
• Household Hunger Score in previous 30 days (binary, moderate/severe hunger or not; severe hunger or not).
• Above FCS, rCSI, and HHS indicators among most vulnerable 30% of houses.
• Reported respiratory illness among children <5yrs of age within the previous 7 days, and reported care-seeking.
• Reported diarrhea among children <5yrs of age within the previous 7 days, and reported care-seeking.
• Household drinking water quality: E. coli >0 colony forming units/100mL.
• Household drinking water quality: High risk (E. coli >10 colony forming units/100mL).
• Household drinking water quality: Very high risk (E. coli >100 colony forming units/100mL).
• Presence of soap and water at handwashing location.
• Reported usual treatment of household drinking water.

Unit of Analysis:

Household

Hypotheses:

The research questions that this study will answer are:

• Primary Research Question 1: What is the impact of the cash transfer component of the intervention on food security, and specifically moderate/severe hunger according to the Household Hunger Score (HHS).
• Primary Research Question 2: What is the impact of the water filter component of the intervention on fecal contamination of household drinking water (as measured by E. coli).
• Secondary Research Question 1 (Cost effectiveness): What is the cost-effectiveness of the cash transfer component of the intervention on moderate/severe household hunger in September 2023 (according to the HHS).
• Secondary Research Question 2 (Cost effectiveness): What is the cost-effectiveness of the water filter component of the intervention on safe household drinking water (free of detectable E. coli) 1 month and 6 months after filter receipt. 

Unit of Intervention or Assignment:

Number of Clusters in Sample:

46

Number of Individuals in Sample:

1142 households

Size of Treatment, Control, or Comparison Subsamples:

There are 412 intervention households (17 clusters) and 730 control houses (29 clusters)

Supplementary Files

Analysis Plan:

Other Documents:

Data

Outcomes Data

Description:

The data set used to measure outcomes in the ACCESO program evaluation will include household-level information collected through surveys. These surveys will capture data on food security using the Household Hunger Score (HHS) as well as fecal contamination of household drinking water measured through quantitative E. coli colony forming units per 100mL of water.

Data Already Collected?

No

Data Previously Used?

Data Access:

Data Obtained by the Study Researchers?

Data Approval Process:

Approval Status:

Treatment Assignment Data

Participation or Assignment Information:

Yes

Description:

Data Obtained by the Study Researchers?

Data Previously Used?

Data Access:

Data Obtained by the Study Researchers?

Data Approval Process:

Approval Status:

Data Analysis

Data Analysis Status:

Study Materials

Upload Study Materials:

Registration Category

Registration Category:

Prospective, Category 1: Data for measuring impacts have not been collected

Completion

Completion Overview

Intervention Completion Date:

Data Collection Completion Date:

Unit of Analysis:

Clusters in Final Sample:

Total Observations in Final Sample:

Size of Treatment, Control, or Comparison Subsamples:

Findings

Preliminary Report:

Preliminary Report URL:

Summary of Findings:

Paper:

Paper Summary:

Paper Citation:

Data Availability

Data Availability (Primary Data):

Date of Data Availability:

Data URL or Contact:

Access procedure:

Other Materials

Survey:

Survey Instrument Links or Contact:

Program Files:

Program Files Links or Contact:

External Link:

External Link Description:

Description of Changes:

Study Stopped

Date:

Reason: