Download Study

General

Study Overview

Title:

Randomized Evaluation of the Educate Girls program in Bhilwara District, Rajasthan

Study is 3ie funded:

No

Study ID:

RIDIE-STUDY-ID-56042ebbd220d

Initial Registration Date:

09/24/2015

Last Update Date:

09/24/2015

Study Status:

Ongoing

Location(s):

India

Abstract:

Educational outcomes for girls in rural Rajasthan, India are critically low: 44% of women are literate and only 1% graduate from high school. Educate Girls (EG) is an education NGO that takes a multifaceted approach to addressing low female educational outcomes by leveraging existing government and community infrastructure. In order to estimate the causal impact of the EG program on student learning outcomes, we will conduct a three-year (May 2015 – April 2018) cluster randomized controlled trial across 332 schools in a new district in which EG is expanding its program in 2015. This evaluation will play a key role in the world’s first Development Impact Bond (DIB): Our estimates of learning gains attributable to EG and changes in enrollment rates in treatment villages will determine how much the outcome payer (the Children’s Investment Fund Foundation) will pay EG for their service provision. The UBS Optimus Foundation is providing up-front working capital to EG, and Instiglio is providing project management on the DIB.

Registration Citation:

McManus, J. and Shah, N.B., 2015. Randomized Evaluation of the Educate Girls program in Bhilwara District, Rajasthan. Registry for International Development for Impact Evaluations (RIDIE). Available at: 10.23846/ridie070

Categories:

Education

Additional Keywords:

Secondary ID Number(s):

Principal Investigator(s)

Name of First PI:

Jeffery McManus

Affiliation:

IDinsight

Name of Second PI:

Neil Buddy Shah

Affiliation:

IDinsight

Study Sponsor

Name:

The Children's Investment Fund Foundation

Study Sponsor Location:

United Kingdom

Research Partner

Name of Partner Institution:

Type of Organization:

Location:

Intervention

Intervention Overview

Intervention:

EG is an eight-year-old nonprofit organization that focuses on enrollment and retention of girls in primary government schools, and learning outcomes for boys and girls in primary government school. EG is currently present in six critical gender gap districts in Rajasthan, and operates in 4,400 villages and 8,000 schools. EG’s program in schools centers on their use of the Creative Learning and Teaching (CLT) curriculum. The CLT curriculum is based on a set of interactive tools, developed in collaboration with Pratham, that teach basic reading, writing, and math skills through activities. It is designed to be integrated into the state curriculum for students in grades 3-5 in government schools. EG’s “Team Balika” volunteers will deliver the CLT materials and train teachers on how to use them in treatment schools, who will then incorporate the materials into their weekly lessons. Team Balika volunteers will also provide classroom support to trained teachers on a regular basis. For more information on the intervention, please see http://www.educategirls.in/What-We-Do.aspx#intervention-innovation.

Theory of Change:

Multiple Treatment Arms Evaluated?

No

Implementing Agency

Name of Organization:

Educate Girls

Type of Organization:

NGO (local)/Community Based Organization/Other civil society organization

Program Funder

Name of Organization:

The Children's Investment Fund Foundation

Type of Organization:

Foreign or Multilateral Aid Agency

Intervention Timing

Intervention or Program Started at time of Registration?

No

Start Date:

10/01/2015

End Date:

02/28/2018

Evaluation Method

Evaluation Method Overview

Primary (or First) Evaluation Method:

Randomized control trial

Other (not Listed) Method:

Additional Evaluation Method (If Any):

Other (not Listed) Method:

Method Details

Details of Evaluation Approach:

In order to estimate the causal impact of the EG program on learning outcomes, we will conduct a three-year RCT across 332 schools in Bhilwara District. Our estimator of interest is the difference between treatment and control schools in aggregate learning gains for students in grades 3-5, as measured by repeated assessments using the Annual Status of Education Report instrument (ASER). We are currently conducting the baseline assessment for a random sample of 50% of students in treatment and control schools, and we will conduct endline assessments for the same group of students at the end of each school year. We will separately estimate the effects of the EG program across grades, but for the purposes of pricing outcomes in the DIB we will not differentiate between gains in the first vs. second vs. third year. A secondary outcome is the change in enrollment of out-of-school girls (OOSGs) in treatment villages. Due to budgetary and logistical constraints, we will measure enrollment before and after the evaluation only in treatment villages. Since there may be other factors that explain changes in enrollment, we cannot attribute changes in enrollment solely to the EG program.

Outcomes (Endpoints):

The primary outcome of the evaluation is learning gains for students in grades 3-5 in government primary schools. Learning gains are defined as the change in the student’s score on the assessment tool (see the Measurement section below) between baseline and endline. A secondary outcome is enrollment of out-of-school girls ages 7-14 in the treatment group. Please see the pre-analysis plan for further explanation and examples of how these outcomes will be measured.

Unit of Analysis:

The student is the main unit of analysis for learning outcomes, and the out-of-school girl is the main unit of analysis for enrollment outcomes.

Hypotheses:

Our hypothesis is that the EG program will increase the enrollment rate of out-of-school girls and will also increase learning gains for boys and girls in treatment schools. For the purposes of pricing outcomes for the DIB, the Working Group has estimated expected performance for each outcome based on EG’s historical performance: EG is expected to enroll approximately 80% of girls off of the out-of-school girls lists and induce learning gains of 0.75 additional steps on the ASER assessment for the average student in treatment schools.

Unit of Intervention or Assignment:

The unit of assignment is the village. All eligible schools within a village will receive the same treatment status.

Number of Clusters in Sample:

280 villages. The evaluation will take place in 332 schools across these villages, or 1 to 4 schools per village.

Number of Individuals in Sample:

Approximately 8,000 students (50% of anticipated students in treatment and control schools). We will also randomly sample 20% of OOSGs who EG enrolls each year to confirm their enrollment status.

Size of Treatment, Control, or Comparison Subsamples:

There are 140 villages in the treatment group and 140 villages in the control group. We will assess approximately 4,000 students in each group (i.e. 50% of students in grades 3-5 in eligible schools).

Supplementary Files

Analysis Plan:

EG DIB Pre-Analysis Plan_11 Sept 2015.pdf

Data

Outcomes Data

Description:

Our internal team of enumerators will collect data within schools for this evaluation. To measure enrollment outcomes, we will randomly sample newly-enrolled girls at the end of each school year. To measure learning outcomes, we are currently assessing students before the intervention starts, and will assess them again at the end of each school year. We will also collect data on student demographics to cross-check with administrative data, but this data will not be used in impact calculations.

Data Already Collected?

No

Data Previously Used?

Data Access:

Data Obtained by the Study Researchers?

Data Approval Process:

Approval Status:

Treatment Assignment Data

Participation or Assignment Information:

Yes

Description:

Data Obtained by the Study Researchers?

Data Previously Used?

Data Access:

Data Obtained by the Study Researchers?

Data Approval Process:

Approval Status:

Data Analysis

Data Analysis Status:

Study Materials

Upload Study Materials:

Baseline student survey: Baseline Survey_Student_English.pdf
Student assessment: Explanation of ASER tool and example: http://www.asercentre.org/p/141.html

Registration Category

Registration Category:

Prospective, Category 1: Data for measuring impacts have not been collected

Completion

Completion Overview

Intervention Completion Date:

Data Collection Completion Date:

Unit of Analysis:

Clusters in Final Sample:

Total Observations in Final Sample:

Size of Treatment, Control, or Comparison Subsamples:

Findings

Preliminary Report:

Preliminary Report URL:

Summary of Findings:

Paper:

Paper Summary:

Paper Citation:

Data Availability

Data Availability (Primary Data):

Date of Data Availability:

Data URL or Contact:

Access procedure:

Other Materials

Survey:

Survey Instrument Links or Contact:

Program Files:

Program Files Links or Contact:

External Link:

External Link Description:

Description of Changes:

Study Stopped

Date:

Reason: