Download Study

General

Study Overview

Title:

Mixed-Methods Cluster-Randomized Controlled Trial of Impact Network’s eSchool 360 Model in Rural Zambia

Study is 3ie funded:

No

Study ID:

RIDIE-STUDY-ID-5c90b72783220

Initial Registration Date:

03/19/2019

Last Update Date:

03/03/2019

Study Status:

Ongoing

Location(s):

Zambia

Abstract:

Impact Network’s eSchool 360 program is a multifaceted program comprising an e-learning technology component, ongoing teacher training and professional development, and community ownership among students in community schools in rural Zambia. The e-learning component includes electricity via solar power, projectors, and tablets loaded with materials in the local language that are structured around a curriculum approved by the Zambian government. The eSchool 360 program represents a promising approach to improving educational outcomes in rural Zambia by incorporating three potentially high-impact components that could create important synergies. Each component could, on its own, have positive impacts on student outcomes by engaging students, teachers, and parents. Combining these elements into a single program may be particularly impactful by working to align all three actors towards improving educational outcomes. AIR will design and implement a two-stage mixed-methods evaluation to measure the effect of Impact Network’s eSchool 360 program on learning outcomes among students in community schools in rural Zambia as well as the cost-effectiveness of any measured impacts.

Registration Citation:

de Hoop, T. and Brudevold-Newman, A., 2019. Mixed-Methods Cluster-Randomized Controlled Trial of Impact Network’s eSchool 360 Model in Rural Zambia. Registry for International Development for Impact Evaluations (RIDIE). Available at: 10.23846/ridie164

Categories:

Education
Information and Communications Technology

Additional Keywords:

Reading; Math; Literacy; Zambia; Randomized Controlled Trial; RCT; Learning;

Secondary ID Number(s):

Principal Investigator(s)

Name of First PI:

Thomas de Hoop

Affiliation:

AIR

Name of Second PI:

Andrew Brudevold-Newman

Affiliation:

AIR

Study Sponsor

Name:

American Institutes for Research

Study Sponsor Location:

United States

Research Partner

Name of Partner Institution:

Palm Associates

Type of Organization:

Research institute/University

Location:

Zambia

Intervention

Intervention Overview

Intervention:

The eSchool 360 model incorporates three potentially high-impact interventions that could offer important complementarities: e-learning technology, ongoing teacher training and professional development, and community ownership. The e-learning component includes electricity via solar power (provided by Impact Network), and projectors and tablets (provided by Impact Network’s partner, iSchool) for the community schools loaded with materials in the local language that are structured around a curriculum approved by the Zambian government. Impact Network supplements the technology by providing teacher training and professional development to community schoolteachers and creating community ownership. Locally hired teachers receive weekly training focused on using the technology and enhancing their pedagogical skills. Impact Network introduced the program in 34 schools that had a dedicated physical structure across three districts in Zambia’s Eastern Province: Petauke, Sinda, and Katete. Impact Network also selected schools that were largely informal, and decided to implement the program only in schools that had more community teachers than government teachers.

Theory of Change:

The underlying theory of change suggests that teacher professional development and the infrastructure improvements in the community schools may lead to improvements in the knowledge and practices of untrained teachers and increases in the demand for education. These improvements could change the quality of education, school attendance and enrollment; thereby resulting in higher preliteracy rates, better early grade reading and early grade math outcomes as well as decreases in the age-at-enrollment. In addition, the program could potentially improve students’ and parents’ aspirations by changing expectations about their future in the domains of education, labor market, and family outcomes. First, greater expectations on the returns to education could lead to higher prospects about labor market outcomes for students and parents. Second, aspirations in education and labor market domains could lead to increases in expectations about students’ marriage prospects and their age-at-marriage. The validity of this theory of change depends on several assumptions: teachers would need to comply with the e-school programming; community schools would need to have sufficient capacity to implement the program; and locally selected teachers would need to have the right incentives to provide quality education. Moreover, the language of instruction also needs to be consistent with the student population’s needs. The effects of the program may also vary with several individual-level, household-level, and community-level moderators; including, gender, age, and socioeconomic household-level characteristics. The program may be less effective in improving school attendance and enrollment for students who live further away from the Impact Network schools. Furthermore, the program impacts may be moderated by baseline pre-literacy, reading, and math outcomes, as well as the mother and father’s education levels. We will test each of these potential heterogeneities in the impact evaluation.

Multiple Treatment Arms Evaluated?

No

Implementing Agency

Name of Organization:

Impact Network

Type of Organization:

NGO (International)

Program Funder

Name of Organization:

American Institutes for Research

Type of Organization:

Research Institution/University

Intervention Timing

Intervention or Program Started at time of Registration?

Yes

Start Date:

01/01/2018

End Date:

Evaluation Method

Evaluation Method Overview

Primary (or First) Evaluation Method:

Randomized control trial

Other (not Listed) Method:

Additional Evaluation Method (If Any):

Instrumental variables

Other (not Listed) Method:

Method Details

Details of Evaluation Approach:

To determine the impact of the model, we randomly assigned the program among schools that satisfied Impact Network’s geographic, infrastructure, and organizational structure eligibility criteria across three districts in Zambia’s Eastern Province: Petauke, Sinda, and Katete. Of the 64 eligible schools, 30 schools were randomly assigned to receive the eSchool 360 model while the remaining 34 schools did not receive the model. This study will estimate the impact of the eSchool 360 model on children eligible to enroll in first grade and who live near the 64 schools. To identify the sample of children, we conducted a census in the areas surrounding the sample schools to identify all households with children eligible to enroll in first grade in January 2018; that is, children ages 6 years or older in January 2018 and who did not attend first grade in the prior school year. We identified all households with children eligible to enroll in first grade within a diameter of 1.5 kilometers of the schools. We then randomly sampled 30 children in each of the 64 school catchment areas. To increase statistical power, we used stratified random sampling by age.

Outcomes (Endpoints):

The primary achievement indicators for numeracy, preliteracy, and literacy come from the Early Grade Reading Assessment, Early Grade Math Assessment, and Zambian Achievement Test, which have already been adapted and used in the same region as this study. The primary cognitive skills outcomes are aggregate scores on the early grade reading assessment (EGRA), early grade math assessment (EGMA), and Zambian achievement tests, and the secondary outcomes are the EGRA, EGMA, and Zambian achievement test subtasks as well as measures of oral reading fluency. To mitigate concerns about floor effects it will still be important to add additional outcome measures. We will therefore also estimate impacts on oral reading fluency. We have created a short oral vocabulary subtask to enable these impact estimates. During the follow-up surveys, we will also collect enrollment and attendance data during the household survey. Outcome indicators on parental and community perceptions of school, teachers, and their children’s education will also be collected through this household survey. We will collect specific data on parents’ and children’s educational, labor market, and marriage aspirations.

Unit of Analysis:

Student and household

Hypotheses:

Primary: The eSchool 360 program will result in improvements in numeracy and literacy skills. 2. The eSchool 360 program will result in increased attendance and enrollment and reductions in the age at primary school enrolment. The eSchool 360 program will lead to an improved perception of school and education quality among students, teachers, and parents. 4. The eSchool 360 program will improve parental and children’s aspirations. Secondary: The eSchool 360 program will result in larger improvements in numeracy and literacy skills for students who enroll in school earlier. The effects of the eSchool 360 programs are moderated by gender, age, and region.

Unit of Intervention or Assignment:

Schools

Number of Clusters in Sample:

64

Number of Individuals in Sample:

1920

Size of Treatment, Control, or Comparison Subsamples:

900 students in treatment (30 household from each school; 1 eligible child per household) and 1,020 students in control (30 households from each area; 1 eligible child per household)

Supplementary Files

Analysis Plan:

ImpactNetwork_InceptionReport_Submission_new.pdf

Other Documents:

: Baseline_Report_10April-Submission.pdf

Data

Outcomes Data

Description:

The main data set will comprise of baseline survey and two rounds of postintervention data collection: a midline survey and an endline survey for each of the 30 children in each of the 64 school catchment areas.

Data Already Collected?

No

Data Previously Used?

Data Access:

Data Obtained by the Study Researchers?

Data Approval Process:

Approval Status:

Treatment Assignment Data

Participation or Assignment Information:

Yes

Description:

Data Obtained by the Study Researchers?

Data Previously Used?

Data Access:

Data Obtained by the Study Researchers?

Data Approval Process:

Approval Status:

Data Analysis

Data Analysis Status:

Study Materials

Upload Study Materials:

Registration Category

Registration Category:

Prospective, Category 1: Data for measuring impacts have not been collected

Completion

Completion Overview

Intervention Completion Date:

Data Collection Completion Date:

Unit of Analysis:

Clusters in Final Sample:

Total Observations in Final Sample:

Size of Treatment, Control, or Comparison Subsamples:

Findings

Preliminary Report:

Preliminary Report URL:

Summary of Findings:

Paper:

Paper Summary:

Paper Citation:

Data Availability

Data Availability (Primary Data):

Date of Data Availability:

Data URL or Contact:

Access procedure:

Other Materials

Survey:

Survey Instrument Links or Contact:

Program Files:

Program Files Links or Contact:

External Link:

External Link Description:

Description of Changes:

Study Stopped

Date:

Reason: