Download StudyGeneral

Study Overview

Title:
Randomized Evaluation of the Educate Girls program in Bhilwara District, Rajasthan
Study ID:
RIDIE-STUDY-ID-56042ebbd220d
Initial Registration Date:
09/24/2015
Last Update Date:
09/24/2015
Study Status:
Ongoing
Location(s):
India
Abstract:
Educational outcomes for girls in rural Rajasthan, India are critically low: 44% of women are literate and only 1% graduate from high school. Educate Girls (EG) is an education NGO that takes a multifaceted approach to addressing low female educational outcomes by leveraging existing government and community infrastructure. In order to estimate the causal impact of the EG program on student learning outcomes, we will conduct a three-year (May 2015 – April 2018) cluster randomized controlled trial across 332 schools in a new district in which EG is expanding its program in 2015. This evaluation will play a key role in the world’s first Development Impact Bond (DIB): Our estimates of learning gains attributable to EG and changes in enrollment rates in treatment villages will determine how much the outcome payer (the Children’s Investment Fund Foundation) will pay EG for their service provision. The UBS Optimus Foundation is providing up-front working capital to EG, and Instiglio is providing project management on the DIB.
Categories:
Education
Additional Keywords:
Secondary ID Number(s):

Principal Investigator(s)

Name of First PI:
Jeffery McManus
Affiliation:
IDinsight
Name of Second PI:
Neil Buddy Shah
Affiliation:
IDinsight

Study Sponsor

Name:
The Children's Investment Fund Foundation
Study Sponsor Location:
United Kingdom
Funding Proposal:

Research Partner

Name of Partner Institution:
Type of Organization:
Website:
Location:
Intervention

Intervention Overview

Intervention:
EG is an eight-year-old nonprofit organization that focuses on enrollment and retention of girls in primary government schools, and learning outcomes for boys and girls in primary government school. EG is currently present in six critical gender gap districts in Rajasthan, and operates in 4,400 villages and 8,000 schools. EG’s program in schools centers on their use of the Creative Learning and Teaching (CLT) curriculum. The CLT curriculum is based on a set of interactive tools, developed in collaboration with Pratham, that teach basic reading, writing, and math skills through activities. It is designed to be integrated into the state curriculum for students in grades 3-5 in government schools. EG’s “Team Balika” volunteers will deliver the CLT materials and train teachers on how to use them in treatment schools, who will then incorporate the materials into their weekly lessons. Team Balika volunteers will also provide classroom support to trained teachers on a regular basis. For more information on the intervention, please see http://www.educategirls.in/What-We-Do.aspx#intervention-innovation.
Theory of Change:
Multiple Treatment Arms Evaluated?
No

Implementing Agency

Name of Organization:
Educate Girls
Type of Organization:
NGO (local)/Community Based Organization/Other civil society organization

Program Funder

Name of Organization:
The Children's Investment Fund Foundation
Type of Organization:
Foreign or Multilateral Aid Agency

Intervention Timing

Intervention or Program Started at time of Registration?
No
Start Date:
10/01/2015
End Date:
02/28/2018
Evaluation Method

Evaluation Method Overview

Primary (or First) Evaluation Method:
Randomized control trial
Other (not Listed) Method:
Additional Evaluation Method (If Any):
Other (not Listed) Method:

Method Details

Details of Evaluation Approach:
In order to estimate the causal impact of the EG program on learning outcomes, we will conduct a three-year RCT across 332 schools in Bhilwara District. Our estimator of interest is the difference between treatment and control schools in aggregate learning gains for students in grades 3-5, as measured by repeated assessments using the Annual Status of Education Report instrument (ASER). We are currently conducting the baseline assessment for a random sample of 50% of students in treatment and control schools, and we will conduct endline assessments for the same group of students at the end of each school year. We will separately estimate the effects of the EG program across grades, but for the purposes of pricing outcomes in the DIB we will not differentiate between gains in the first vs. second vs. third year. A secondary outcome is the change in enrollment of out-of-school girls (OOSGs) in treatment villages. Due to budgetary and logistical constraints, we will measure enrollment before and after the evaluation only in treatment villages. Since there may be other factors that explain changes in enrollment, we cannot attribute changes in enrollment solely to the EG program.
Outcomes (Endpoints):
The primary outcome of the evaluation is learning gains for students in grades 3-5 in government primary schools. Learning gains are defined as the change in the student’s score on the assessment tool (see the Measurement section below) between baseline and endline. A secondary outcome is enrollment of out-of-school girls ages 7-14 in the treatment group. Please see the pre-analysis plan for further explanation and examples of how these outcomes will be measured.
Measurement:
In order to quantify learning gains, we will use the Annual Status of Education Report (ASER) assessment tool. This tool consists of three sections (Hindi, Math, English), which each measure four basic competencies. We have added one extra level to the Hindi section (advanced story fluency) in order to reduce the likelihood of ‘top censoring’, or underestimating learning gains for students who obtain the highest possible score. Trained IDinsight enumerators will administer this tool and grade student performance based on the highest competency level within each section that the student can successfully complete. Enrollment will be measured as the percentage of girls enrolled off of a list of eligible OOSGs in treatment villages. This list has already been prepared by EG. We validated this list in August 2015 by randomly sampling 140 girls on the list and 259 girls off of the list and cross-checking their eligibility status; the discrepancy rate between our sample and EG's list fell below the pre-specified threshold of 10%.
Unit of Analysis:
The student is the main unit of analysis for learning outcomes, and the out-of-school girl is the main unit of analysis for enrollment outcomes.
Hypotheses:
Our hypothesis is that the EG program will increase the enrollment rate of out-of-school girls and will also increase learning gains for boys and girls in treatment schools. For the purposes of pricing outcomes for the DIB, the Working Group has estimated expected performance for each outcome based on EG’s historical performance: EG is expected to enroll approximately 80% of girls off of the out-of-school girls lists and induce learning gains of 0.75 additional steps on the ASER assessment for the average student in treatment schools.
Unit of Intervention or Assignment:
The unit of assignment is the village. All eligible schools within a village will receive the same treatment status.
Number of Clusters in Sample:
280 villages. The evaluation will take place in 332 schools across these villages, or 1 to 4 schools per village.
Number of Individuals in Sample:
Approximately 8,000 students (50% of anticipated students in treatment and control schools). We will also randomly sample 20% of OOSGs who EG enrolls each year to confirm their enrollment status.
Size of Treatment, Control, or Comparison Subsamples:
There are 140 villages in the treatment group and 140 villages in the control group. We will assess approximately 4,000 students in each group (i.e. 50% of students in grades 3-5 in eligible schools).

Supplementary Files

Analysis Plan:
EG DIB Pre-Analysis Plan_11 Sept 2015.pdf
Data

Outcomes Data

Description:
Our internal team of enumerators will collect data within schools for this evaluation. To measure enrollment outcomes, we will randomly sample newly-enrolled girls at the end of each school year. To measure learning outcomes, we are currently assessing students before the intervention starts, and will assess them again at the end of each school year. We will also collect data on student demographics to cross-check with administrative data, but this data will not be used in impact calculations.
Data Already Collected?
No
Survey Name:
Data Previously Used?
Data Access:
Data Obtained by the Study Researchers?
Data Approval Process:
Approval Status:

Treatment Assignment Data

Participation or Assignment Information:
Yes
Description:
Data Obtained by the Study Researchers?
Data Previously Used?
Data Access:
Data Obtained by the Study Researchers?
Data Approval Process:
Approval Status:

Data Analysis

Data Analysis Status:

Study Materials

Upload Study Materials:
Baseline student survey: Baseline Survey_Student_English.pdf
Student assessment: Explanation of ASER tool and example: http://www.asercentre.org/p/141.html

Registration Category

Registration Category:
Prospective, Category 1: Data for measuring impacts have not been collected
Completion

Completion Overview

Intervention Completion Date:
Data Collection Completion Date:
Unit of Analysis:
Clusters in Final Sample:
Total Observations in Final Sample:
Size of Treatment, Control, or Comparison Subsamples:

Findings

Preliminary Report:
Preliminary Report URL:
Summary of Findings:
Paper:
Paper Summary:
Paper Citation:

Data Availability

Data Availability (Primary Data):
Date of Data Availability:
Data URL or Contact:
Access procedure:

Other Materials

Survey:
Survey Instrument Links or Contact:
Program Files:
Program Files Links or Contact:
External Link:
External Link Description:
Description of Changes:

Study Stopped

Date:
Reason: