Download Study

General

Study Overview

Title:

Evaluating the Impact of a Maternal & Child Health Intervention in Sindh, Pakistan

Study is 3ie funded:

No

Study ID:

RIDIE-STUDY-ID-62380b025cb64

Initial Registration Date:

03/21/2022

Last Update Date:

01/15/2024

Study Status:

Ongoing

Change History for Status

Changed On	Previous Value
01/15/2024	In Development

Location(s):

Pakistan

Abstract:

CCTs are a now standard tool used by governments trying to break the intergenerational transmission of poverty.  The increased frequency/severity of climate shocks raises the question of whether and how cash transfer programs need to be redesigned to maintain their effectiveness in the face of climate shocks. We ask this question in the context of a CCT in the Sindh province of Pakistan that was intended to address high rates of stunting by improving the use of pre- and post-natal care by poor women and their infants.  We argue that early life health indicators are important for intergenerational transmission of poverty. Between our baseline and first endline surveys, our study area was seriously, but unevenly hit by the flooding that began in August 2022.We know that early childhood development (first 1000 days), including linear growth and brain development, is notoriously sensitive to shocks.  Particularly concerning is the evidence that shock-induced losses that occur at an early age may be irreversible. Against this backdrop, we ask how effective are CCTs in the presence of shocks and if climate shocks erode the impacts of the transfer program in the short-term and especially in the longer-term? We will assess heterogeneity in these effects, if any, along the dimensions of distance to a health center, opportunity cost of the trip to the hospital, and household norms. To answer these questions, we will combine geographic regression discontinuity design (GRD) with a natural experiment as the identification strategy. For the GRD, we conducted a census in an area of around 28 km around the border of the treated and untreated regions and selected 4014 pregnant women for the evaluation sample. The running variable will be defined as a function of the household location vis-a-vis the border separating treatment and control areas. For the natural experiment, we will interact the CCT treatment status with flood intensity and severity.

Change History for Abstract

Changed On

Previous Value

01/15/2024

Conditional cash transfers (CCTs) have been identified as a popular policy instrument to increase the utilization of health care, particularly among women and children. The question, however, remains if a CCT is enough to alter individual behavior in certain contexts where the recommended activities are not only constrained by financial resources, but also by persistent norms, traditional practices, and intra-household tensions in preferences.  Then, the impact evaluation of a program which ties cash receipts to a fixed number of health visits for the expectant mother and the newborn should go beyond the estimation of the extensive and intensive margins of take-up and explore the differential trends in treatment effects. These dimensions of heterogeneity help us identify cohorts where the CCT is successful and where it is not, leaving us with finer policy recommendations. To that end, we evaluate a Maternal Health CCT program in Pakistan, and answer two main questions: Does the CCT increase use of pre- and post-natal care and institutional deliveries, with downstream effects on child health and growth? How does this effect differ based on distance to the health center, opportunity cost of the trip to the hospital, and household norms? To answer these questions, we will use a geographic regression discontinuity design (GRD) as the identification strategy. To implement this methodology, we will conduct a census listing in an area of 20 to 25 km around the border of the treated and untreated regions and randomly select up to 4000 pregnant women for the evaluation sample. The running or forcing variable will be defined as a function of the latitude and longitude coordinates as well as the distance from the border separating the treatment and control areas.

Registration Citation:

Categories:

Health, Nutrition, and Population
Social Protection

Additional Keywords:

Conditional Cash Transfers, Climate Shocks, Bargaining Power, Gender

Change History for Additional Keywords

Changed On	Previous Value
01/15/2024	Conditional Cash Transfers, Bargaining Power, Gender

Secondary ID Number(s):

Principal Investigator(s)

Name of First PI:

Syeda Warda Riaz

Affiliation:

University of California - Davis

Name of Second PI:

Michael R Carter

Affiliation:

University of California - Davis

Study Sponsor

Name:

University of California - Davis

Study Sponsor Location:

United States

Research Partner

Name of Partner Institution:

Lahore University of Management Sciences

Type of Organization:

Research institute/University

Location:

Pakistan

Intervention

Intervention Overview

Intervention:

The program under evaluation is a conditional cash transfer aimed at improving maternal and child health. To enroll in the program, a woman must be pregnant at the time of registration, be at least 18 years of age with valid identity documentation, and be a resident of the treatment region. The CCT provides a pregnant woman with regular transfers at various trigger points during a pregnancy and post-delivery. There is no poverty threshold for obtaining the cash transfer but the treatment must be availed at the selected public health centers. These health centers pass an initial screening to ensure that they are adequately resourced for maternal and child health. The condition to avail services at public health centers will also most likely prevent the well-off and rich to enroll for the CCT. While there is no individual poverty score requirement to partake in the CCT, the district where the program is introduced was chosen on account of its second to last rank in multidimensional poverty.

Soon after the baseline survey and the introduction of the CCT program, both the treatment and control groups/regions were hit by the severe floods that occurred in summer 2022 in Pakistan. Because of the floods, the research agenda has been expanded to look at the joint effects of the CCT and flooding. Just the effects of the CCT will absorb the flood’s impact as well, which we now wish to disentangle. We will utilize the variation in flooding as a natural experiment and combine it with the cash transfer treatment status to assess the efficacy of CCTs in the face of weather shocks.

Change History for Intervention

Changed On

Previous Value

01/15/2024

The program under evaluation is a conditional cash transfer aimed at improving maternal and child health. To enroll in the program, a woman must be pregnant at the time of registration, be at least 18 years of age with valid identity documentation, and be a resident of the treatment region. The CCT provides a pregnant woman with regular transfers at various trigger points during a pregnancy and post-delivery. There is no poverty threshold for obtaining the cash transfer but the treatment must be availed at the selected public health centers. These health centers pass an initial screening to ensure that they are adequately resourced for maternal and child health. The condition to avail services at public health centers will also most likely prevent the well-off and rich to enroll for the CCT. While there is no individual poverty score requirement to partake in the CCT, the district where the program is introduced was chosen on account of its second to last rank in multidimensional poverty.

Theory of Change:

The CCT for the pregnant woman and her newborn can affect the outcomes of interest in several ways. The first pathway is the income effect where the money is expected to alleviate any binding liquidity constraints of the recipient. The second channel is the substitution effect where the conditionality of the transfer incentivizes certain behaviors directly by raising the opportunity cost of not undertaking them. In addition, the CCT can create distributional effects as it is targeted to the woman which in turn can alter the intra-household resource allocation – for example, the woman may start to enjoy higher agency in matters pertaining to her own healthcare which translates into a higher take-up of formal maternal health seeking behavior. 

While the above channels continue to be the focus of the analysis, we hypothesize that the heavy flooding exerts an opposite force on the income, substitution, and distributional effects that a conditional cash transfer is expected to create in normal times. Climate shocks are a negative income shock, increase the opportunity cost of resources spent on health care/ program compliance, and have also been associated with a more stressful situation for women.  In addition, they have a direct negative impact on child health outcomes. Flooding can then undermine the average treatment effect of the CCT making it important to disentangle and look at the effects of the two separately.

Change History for Theory of Change

Changed On

Previous Value

01/15/2024

The CCT for the pregnant woman and her newborn can affect the outcomes of interest in several ways. The first pathway is the income effect where the money is expected to alleviate any binding liquidity constraints of the recipient. The second channel is the substitution effect where the conditionality of the transfer incentivizes certain behaviors directly by raising the opportunity cost of not undertaking them. While no fee is charged for the obstetric care services at the public health facilities, there are other costs incurred by individuals in accessing formal care and the transfer amount can help compensate for them through these two. In particular, we are interested in the transportation cost which can be modeled as a function of distance and road terrain, and the opportunity cost of visiting a health facility which depends on the next best use of one's time and will vary based on the woman's employment and/or housework burdens. It will also include the time cost for the individual accompanying the woman to the hospital. The opportunity cost also in part depends on the distance to the hospital: the longer it takes to get there, the greater the time cost. In addition, the CCT can create distributional effects as it is targeted to the woman which in turn can alter the intra-household resource allocation – for example, the woman may start to enjoy higher agency in matters pertaining to her own healthcare which translates into a higher take-up of formal maternal health seeking behavior. We hypothesize that this channel is ambiguous as the cash transfer targeted to the woman may not necessarily increase her bargaining power through an improvement in her threat point utility outside of marriage in a context like ours where separation and divorce are considered a taboo for women. We aim to test for this by estimating how the treatment effect varies by the baseline bargaining power of the woman.

Multiple Treatment Arms Evaluated?

No

Intervention Timing

Intervention or Program Started at time of Registration?

No

Start Date:

04/25/2022

Change History for Start Date

Changed On	Previous Value
01/15/2024	04/30/2022

End Date:

Evaluation Method

Evaluation Method Overview

Primary (or First) Evaluation Method:

Regression discontinuity

Other (not Listed) Method:

Additional Evaluation Method (If Any):

Natural experiment

Other (not Listed) Method:

Method Details

Details of Evaluation Approach:

This evaluation is based on a discontinuity design. The provincial government of Sindh decided to roll the project out initially in the Umerkot district.  The neighboring Mirpukhas district is also receiving the program, but only after the evaluation is complete (The program was launched in Mirpurkhas a month after the completion of the first follow-up survey).  Households that straddle the Umerkot-Mirpukhas border are arbitrarily included or excluded from the program based on the arbitrary political delineation of their households. This staggered rollout by district creates a sharp geographic discontinuity and we will use pregnant women who live in the border regions of the currently untreated district of Mirpurkhas as our control for the CCT. Women who live in Umerkot, but on the border with Mirpurkhas will be our treatment group. The feasibility of this design was confirmed by using household level census data which did not reveal any "jumps" in the pre-treatment variables (placebo outcomes) at the cutoff point of the running variable. The same exercise will be repeated using the baseline data of this study and in the event that there are any "jumps" suggesting an imbalance between treatment and control groups, the RDD will be combined with regression adjustment and matching techniques to account for it. Under statistically conservative assumptions, the study is powered at 80% to detect an increase of 11% in the number of antenatal visits relative to the control mean of 2.93, and a decrease of 17% in severe stunting relative to the control mean of 0.34.

Soon after the completion of the baseline survey, both the treatment and control regions were hit by the severe floods that occurred in summer 2022 in Pakistan. We will utilize the variation in flooding as a natural experiment and combine it with the cash transfer treatment to assess the efficacy of CCTs in the face of climate shocks.

Change History for Details of Evaluation Approach

Changed On

Previous Value

01/15/2024

This evaluation is based on a discontinuity design. The provincial government of Sindh decided to roll the project out initially in the Umerkot district.  The neighboring Mirpukhas district is also receiving the program, but only after the evaluation is complete.  Households that straddle the Umerkot-Mirpukhas border are arbitrarily included or excluded from the program based on the arbitrary political delineation of their households. This staggered rollout by district creates a sharp geographic discontinuity and we will use pregnant women who live in the border regions of the currently untreated district of Mirpurkhas as our control for the CCT. Women who live in Umerkot, but on the border with Mirpurkhas will be our treatment group. The feasibility of this design was confirmed by using household level census data which did not reveal any "jumps" in the pre-treatment variables (placebo outcomes) at the cutoff point of the running variable. The same exercise will be repeated using the baseline data of this study and in the event that there are any "jumps" suggesting an imbalance between treatment and control groups, the RDD will be combined with regression adjustment and matching techniques to account for it. Under statistically conservative assumptions, the study is powered at 80% to detect an increase of 11% in the number of antenatal visits relative to the control mean of 2.93, and a decrease of 17% in severe stunting relative to the control mean of 0.34.

Outcomes (Endpoints):

Primary: These outcomes include extensive and intensive take-up rates of pre- and post-natal care as well as the place of delivery. The consequent impact on health will be studied by looking at fertility/contraception use after delivery, morbidity incidence among children, and child anthropometric measures, including height for age, weight for age, and middle upper arm circumference. 

Secondary: Additionally, we will also analyze the targeting of this program as a secondary outcome as the CCT will follow universality with no poverty threshold to enroll in the program. The idea is that by mandating medical treatment at the public hospitals, the program implicitly excludes the well-off who are more likely to use private providers and we want to test how effective this implicit targeting is by calculating inclusion and exclusion errors. We will also determine if the cash transfer incentivizes individuals to substitute private with public care.

Other outcomes include food security indicators and measures of women empowerment.

Change History for Outcomes (Endpoints)

Changed On

Previous Value

01/15/2024

The primary outcomes are extensive and intensive take-up rates of pre- and post-natal care as well as the place of delivery. The consequent impact on health will be studied by looking at fertility/contraception use after delivery, morbidity incidence among children, and child anthropometric measures such as stunting rates.  Additionally, we will also analyze the targeting of this program as a secondary outcome as the CCT will follow universality with no poverty threshold to enroll in the program. The idea is that by mandating medical treatment at the public hospitals, the program implicitly excludes the well-off who are more likely to use private providers and we want to test how effective this implicit targeting is by calculating inclusion and exclusion errors. We will also determine if the cash transfer incentivizes individuals to substitute private with public care.

Unit of Analysis:

The unit of analysis is the pregnant woman and her child after the child is born.

Hypotheses:

In addition to studying the average treatment effect, we propose to study 4 impact modifiers:

Distance to hospital: It is suspected that the cash transfer amount may not be enough to compensate for the relatively high costs that will be experienced by women who live further from the health centers. This analysis can inform a threshold of distance beyond which the CCT will become ineffective. One can also imagine that the CCT may not have any effect on the women who live really close to the health center and would have gone anyways.

Opportunity cost: In addition to transportation cost, a visit to the health facility will come at the expense of missed work for the woman and for someone who accompanies her. The work can be either paid, unpaid, or domestic chores. This could mean that there is a dimension of seasonality to seeking health care or heterogenous behaviors based on a woman’s burden of work, both inside and outside of home.  

Norms & attitudes: Decisions about maternal care are reportedly heavily influenced by spouses and mothers-in-law, with expectant mothers often enjoying little autonomy over their own medical treatments. We will thus measure the importance that spouses, mothers-in-law and pregnant women themselves place on pre- and post-natal care, hypothesizing that the cash transfers will be less effective when traditional norms about pre- and post-natal care are strongly held by these powerful others.

Flood: The CCT treatment was disrupted by heavy flooding in the monsoon season of 2022. The floods, by diminishing a household's resource base and aggravating disease burden, can undo the benefits of a CCT by either making compliance difficult and/or negatively affecting mother-child health. On the contrary, it may act as relief that households can fall back on during the floods.

Change History for Hypotheses

Changed On

Previous Value

01/15/2024

In addition to studying the average impacts of the program on the outcomes listed above, we also propose to study 3 impact modifiers that we hypothesize will diminish the program's effectiveness.  Knowing if these modifiers matter will be crucial to improved policy design.  The 3 impact modifiers we will study are: Distance to the health center: It is suspected that the cash transfer amount may not be enough to compensate for the relatively high costs that will be experienced by women who live further from the health centers. This analysis can inform a threshold of distance beyond which the CCT will become ineffective. One can also imagine that the CCT may not have any effect on the women who live really close to the health center and would have gone anyways. Opportunity cost of seeking healthcare: In addition to the transportation cost, a visit to the health facility will come at the expense of missed work for the woman and for someone who accompanies her. The work can be either paid, unpaid, or domestic chores. This could mean that there is a dimension of seasonality to seeking health care or heterogenous behaviors based on a woman’s burden of work, both inside and outside of home.   Limited say of the woman and negative attitudes towards formal care: Decisions about maternal care are reportedly heavily influenced by spouses and mothers-in-law, with expectant mothers often enjoying little autonomy over their own medical treatments. We will thus measure the importance that spouses, mothers-in-law and pregnant women themselves place on pre- and post-natal care, hypothesizing that the cash transfers will be less effective when traditional norms about pre- and post-natal care are strongly held by these powerful others. We further hypothesize that the program will be less effective when the woman is exposed to intimate partner violence.

Unit of Intervention or Assignment:

Under the geographic discontinuity design, assignment to treatment will be based on the woman's residence relative to the district border.

Number of Clusters in Sample:

While the treatment is not clustered, a random sample of roughly 4000 pregnant women will be drawn from a census listing to be carried out in 150 revenue villages around 20-25 km of the border.

Number of Individuals in Sample:

The planned sample size for this study is approximately 4000 pregnant women to be divided equally between the treatment and control groups.

Size of Treatment, Control, or Comparison Subsamples:

There are only two arms: a pure treatment and a pure control. The sample of pregnant women will be split evenly between the two.

Supplementary Files

Analysis Plan:

Econometric Specification_Final.docx

Data

Outcomes Data

Description:

A household level survey will be fielded in the treatment and control regions to collect information from pregnant women. The survey will have modules on household composition and characteristics, economic activity of the woman, her spouse and in-laws, decision-making, intimate partner violence, attitudes to health care and care seeking behavior in the previous and current pregnancy. In addition, child health outcomes and food security indicators were collected in the first follow-up survey.

Change History for Description

Changed On	Previous Value
01/15/2024	A household level survey will be fielded in the treatment and control regions to collect information from pregnant women. The survey will have modules on household composition and characteristics, economic activity of the woman, her spouse and in-laws, decision-making, intimate partner violence, attitudes to health care and care seeking behavior in the previous and current pregnancy present.

Data Already Collected?

No

Data Previously Used?

Data Access:

Data Obtained by the Study Researchers?

Data Approval Process:

Approval Status:

Treatment Assignment Data

Participation or Assignment Information:

Yes

Description:

Data Obtained by the Study Researchers?

Data Previously Used?

Data Access:

Data Obtained by the Study Researchers?

Data Approval Process:

Approval Status:

Data Analysis

Data Analysis Status:

Study Materials

Upload Study Materials:

Baseline Questionnaire: Feb272022_Evaluating the impact of Mother and Child CCT.pdf
Midline Survey Questionnaire: Round 2 Evaluating the impact of Mother and Child CCT V2.pdf

Change History for Upload Study Materials

Changed On	Previous Value
01/15/2024	Description: Baseline Questionnaire Filename: Feb272022_Evaluating the impact of Mother and Child CCT.pdf Link: http://

Registration Category

Registration Category:

Prospective, Category 1: Data for measuring impacts have not been collected

Completion

Completion Overview

Intervention Completion Date:

Data Collection Completion Date:

Unit of Analysis:

Clusters in Final Sample:

Total Observations in Final Sample:

Size of Treatment, Control, or Comparison Subsamples:

Findings

Preliminary Report:

Preliminary Report URL:

Summary of Findings:

Paper:

Paper Summary:

Paper Citation:

Data Availability

Data Availability (Primary Data):

Date of Data Availability:

Data URL or Contact:

Access procedure:

Other Materials

Survey:

Survey Instrument Links or Contact:

Program Files:

Program Files Links or Contact:

External Link:

External Link Description:

Description of Changes:

Study Stopped

Date:

Reason: