Download Study

General

Study Overview

Title:

WASH UP! India Impact Evaluation

Study is 3ie funded:

No

Study ID:

RIDIE-STUDY-ID-662bed52ae1e7

Initial Registration Date:

04/26/2024

Last Update Date:

04/18/2024

Study Status:

Completed

Location(s):

India

Abstract:

This study is designed to examine the effectiveness of the WASH UP! curriculum and a water, sanitation, and hygiene (WASH) infrastructure operation & maintenance (O&M) intervention in improving water, sanitation, and hygiene knowledge and behaviors among first and fourth-grade students in public primary schools in Uttar Pradesh, India. Utilizing both survey and observational data, this study seeks to determine the effectiveness of delivering WASH programming in schools, which could be a more cost-effective method than household-level programs. The practice of handwashing with soap is critical for children’s health, significantly reducing the occurrence of diarrhea and respiratory infections. This study is the first managed experiment of this scale to evaluate the individual and combined impacts of educational and infrastructural interventions on promoting healthy WASH behaviors in schools, an essential setting for establishing lasting healthy habits. The motivation for this study stems from the persistent challenge in many resource-constrained contexts, including Indian primary schools, where there is a notable deficit in both maintenance of WASH facilities and the practice of healthy WASH behaviors among students. Previous studies have yielded mixed results, failing to identify interventions that are both effective and sustainable in the long-term. Literature reviews by Poague et al. (2022) and Pu et al. (2022) highlight the global struggle to maintain adequate WASH facilities in schools and the minimal impact of resource infusions on sustainable WASH practices. This study seeks to build on these findings with a robust research design, incorporating a 4-arm cRCT to assess the individual and combined effects of educational and infrastructure interventions, setting it apart in scale and potential impact from previous research. 

Registration Citation:

Categories:

Water and Sanitation

Additional Keywords:

education, operation and maintenance

Secondary ID Number(s):

Principal Investigator(s)

Name of First PI:

Jenna Davis

Affiliation:

Stanford University

Name of Second PI:

Gary Darmstadt

Affiliation:

Stanford University

Study Sponsor

Name:

World Vision

Study Sponsor Location:

United States

Research Partner

Name of Partner Institution:

Oxford Policy Management - India

Type of Organization:

Research institute/University

Location:

India

Intervention

Intervention Overview

Intervention:

This study evaluates the individual and combined impacts of two interventions. First, WASH UP! is a play-based curriculum designed by Sesame Workshops, and implemented by World Vision - India. World Vision - India trains teachers to deliver the curriculum to their students. The curriculum includes messaging about healthy WASH behaviors such as wearing shoes to the bathroom, washing hands at critical times, using the school toilet, and sharing cleaning responsibilities between boys and girls. The curriculum also teaches students about safe water source identification and about what germs are. The objective of this intervention is to increase student WASH knowledge and improve key WASH behaviors (handwashing, toilet use) that have previously been linked to improved health outcomes, especially for young children. Students are the primary expected beneficiaries of this intervention. 

The second intervention being evaluated is a newly-designed operation and maintenance (O&M) intervention which aims to improve the usability of school WASH infrastructure through enhanced cleaning, improved information flows on the status of infrastructure, and meaningful accountability mechanisms. Schools received a part-time school cleaner contracted through a private third-party agency. This cleaner was responsible for cleaning and reporting on the status of school toilets, drinking water sources, and handwashing stations. Schools were supplied the necessary cleaning supplies by the third-party agency, but schools were responsible for supplying soap for student handwashing. The objective of this intervention was to give students an enabling environment for practicing good WASH behaviors. Students are the primary expected beneficiaries of this intervention.

Private Intervention Details:

Theory of Change:

For the WASH UP! curriculum, key aspects of the theory of change that will be examined during this impact evaluation include the following outputs: Play-based curriculum draws sustained attention from students --> Primary school students know when it is important to wash hands --> Primary school students wash hands with soap and water at critical times. We will also examine a key assumption, which is "Schools have handwashing facilities and supplies available to students."

For the O&M intervention, key aspects of the theory of change that will be examined during this impact evaluation include the following outputs: Trained cleaners maintain school WASH facilities consistently and effectively --> School WASH facilities are (almost) always functional and clean --> Students use school WASH facilities correctly & consistently.

We have selected causal pathways for each intervention which have overlapping assumptions and lead to the same two primary behavioral outcomes -- student handwashing and student toilet use. We will also evaluate student knowledge, infrastructure cleaniliness and functionality, and consumables availability to characterize any potentially synergistic effects of WASH education and infrastructure maintenance. 

Multiple Treatment Arms Evaluated?

Yes

Implementing Agency

Name of Organization:

World Vision - India

Type of Organization:

NGO (local)/Community Based Organization/Other civil society organization

Program Funder

Name of Organization:

World Vision

Type of Organization:

NGO (International)

Intervention Timing

Intervention or Program Started at time of Registration?

Yes

Start Date:

10/31/2022

End Date:

12/29/2023

Evaluation Method

Evaluation Method Overview

Primary (or First) Evaluation Method:

Randomized control trial

Other (not Listed) Method:

Additional Evaluation Method (If Any):

Other (not Listed) Method:

Method Details

Details of Evaluation Approach:

Research Design

The study employed a 4-arm cluster randomized controlled trial (cRCT), which included a control group, a WASH UP! only group, an O&M only group, and a combined WASH UP! + O&M group. The timeline of the study encompassed three measurement periods: baseline data collection in Fall 2022 before the interventions were implemented, a midline assessment in Spring 2023 during intervention delivery, and an endline measurement in Fall 2023 following the completion of the educational program. This design allows for a comprehensive evaluation of the interventions' individual and combined effects on WASH behaviors.

Sampling Frame

The study's eligible population for sampling is first and fourth graders, along with their teachers, from 199 government primary schools in one district of Uttar Pradesh, India. The sample comprises two distinct groups: students in their respective grades, tracked over the course of one year, and teachers  engaged with the WASH UP! curriculum, primarily 1st and 4th grade teachers. The study ensures gender balance in the sample, with equal numbers of boy and girl students selected for interviewing. The sample encompasses 20 clusters across 5 administrative blocks, with each block containing 4 clusters assigned to different study arms. The expected sample size is up to 8,000 students, approximately 400 teachers, and 105 cleaning personnel. The sample is representative of the district, although the condition of WASH infrastructure in sampled schools may be above average due to prior upgrades performed by our implementing partners, which is a consideration for the study's generalizability. An additional subset of 11 government-recommended schools may present unanticipated differences, which will be examined for potential systematic variance.

Private Details of Evaluation Approach:

Outcomes (Endpoints):

The primary variables of interest in this evaluation are listed below under three broader outcome categories: Student knowledge, enabling school environment, and student behavior. We prioritized including indicators we hypothesized may be impacted by both the WASH UP! curriculum and the O&M intervention to test the individual and combined impacts of each intervention. Additional details on indicator definitions and statitiscal power are provided in the attached pre-analysis plan.

 

• Student knowledge

  • Critical times to wash hands

  • Accurate and relevant information about germs

• Enabling school environment

  • Schools’ relevant WASH infrastructure is functional and necessary supplies are available to students for handwashing

  • Schools’ relevant WASH infrastructure is present, functional, and clean and schools have necessary supplies available to students for toilet use

• Student behavior

  • Student handwashing with soap at critical times

  • Student use of school toilets

Unit of Analysis:

Student knowledge outcomes and student peer-reported behavior outcomes are measured at the student level, whereas observational data on student behavior outcomes was observed at the school level.

Hypotheses:

The main research question this study seeks to answer is: 

What is the impact of the (i) WASH UP! curriculum, (ii) an infrastructure maintenance intervention, and (iii) both on (a) WASH behavior-related knowledge, (b) self-reported WASH behaviors, (c) and observed student hand washing among first- and fourth-grade students? (d) How do these impacts vary by student gender and grade? 

We hypothesize the WASH UP! curriculum will have a positive impact on both student WASH knowledge and behavior. We also hypthesize the infrastructure maintenance (O&M) intervention will have a positive impact on student WASH behaviors. However, the novelty of this study lies in our hypothesis that the WASH UP! and O&M interventions will have synergistic, positive impacts on student knowledge and behavior when the interventions are delivered in tandem. 

Unit of Intervention or Assignment:

Schools were assigned to geographic clusters within administrative blocks. Clusters were assigned to treatment arms.

Number of Clusters in Sample:

The study contains 20 clusters of schools. Four clusters were formed in each of five administrative blocks included in the study sample.

Number of Individuals in Sample:

Our study planned to recruit up to 8,000 students, 400 teachers, and 105 cleaners for surveying and observation.

Size of Treatment, Control, or Comparison Subsamples:

Five clusters were assigned to each treatment arm. Thus, each study arm (control, WASH UP!, O&M, WASH UP! + O&M) contains approximately 50 schools.

Supplementary Files

Analysis Plan:

20240314_WASH UP! India PAP_FULL TEXT.pdf

Other Documents:

Data

Outcomes Data

Description:

This study collected data in the form of a student survey, teacher survey, and school cleaner survey. Student and teacher surveys were collected at baseline, midline, and endline, whereas the school cleaner survey was collected at endline only. Observational data was also collected on the status of school infrastructure and student handwashing behavior. For this impact evaluation, we will primarily utilize data from the student survey, infrastructure observations, and behavior observations.

Data Already Collected?

Yes

Data Previously Used?

No

Data Access:

Restricted -- Access requires a formal approval process

Data Obtained by the Study Researchers?

Data Approval Process:

Researchers wishing to access this data should contact Jenna Davis (jennadavis@stanford.edu) or Gracie Hornsby (gmhornsb@stanford.edu) to request access.

Approval Status:

Yes-obtained approval and have received the data

Treatment Assignment Data

Participation or Assignment Information:

Yes

Description:

Data Obtained by the Study Researchers?

Data Previously Used?

Data Access:

Data Obtained by the Study Researchers?

Data Approval Process:

Approval Status:

Data Analysis

Data Analysis Status:

No

Study Materials

Upload Study Materials:

Registration Category

Registration Category:

Prospective, Category 3: Data for measuring impacts have been obtained/collected by the research team but analysis for this evaluation has not started

Completion

Completion Overview

Intervention Completion Date:

Data Collection Completion Date:

Unit of Analysis:

Clusters in Final Sample:

Total Observations in Final Sample:

Size of Treatment, Control, or Comparison Subsamples:

Findings

Preliminary Report:

Preliminary Report URL:

Summary of Findings:

Paper:

Paper Summary:

Paper Citation:

Data Availability

Data Availability (Primary Data):

Date of Data Availability:

Data URL or Contact:

Access procedure:

Other Materials

Survey:

Survey Instrument Links or Contact:

Program Files:

Program Files Links or Contact:

External Link:

External Link Description:

Description of Changes:

Study Stopped

Date:

Reason: