Download Study

General

Study Overview

Title:

Impact Evaluation of the Support to the Maternal Mortality Reduction Project (SMMRP) in Tanzania

Study is 3ie funded:

No

Study ID:

RIDIE-STUDY-ID-662be9bbc4efd

Initial Registration Date:

04/26/2024

Last Update Date:

08/08/2023

Study Status:

Ongoing

Location(s):

Tanzania

Abstract:

The Independent Development Evaluation function (IDEV) at the African Development Bank (AfDB) recently launched the Impact Evaluation (IE) of the Support to the Maternal Mortality Reduction Project (SMMRP) in Tanzania. The objective of the evaluation is to inform the implementation of the Bank’s Strategy for Quality Health Infrastructure in Africa (SQHIA, 2022-2030) approved in 2022, which aims at achieving a high-quality health system in Regional Member Countries (RMC) by increasing access to quality health services for Africans by 2030 with a focus on equity, quality care, efficiency, and the resilience of health systems. Specifically, the lessons learned from the SMMRP would be useful in improving the design and implementation of similar health projects in RMCs under the SQHIA. Approved by the AfDB in 2006 and implemented between 2007 and 2015, the SMMRP financed the construction, rehabilitation, and equipping of health infrastructure (health centres, obstetric theatres, maternity wards, staff houses, and training institutes) to reduce maternal and newborn mortality in remote and underserved areas. The project also trained health workers on maternal and child healthcare. The total project cost was UA 44.44 million and was financed by the African Development Fund (ADF), with the project covering Tabora, Mara, and Mtwara on the Mainland and Unguja and Pemba on the Island of Zanzibar. 

Registration Citation:

Categories:

Health, Nutrition, and Population

Additional Keywords:

Secondary ID Number(s):

Principal Investigator(s)

Name of First PI:

Dana Kassem

Affiliation:

Independent Consultant

Name of Second PI:

Giulia Zane

Affiliation:

Study Sponsor

Name:

African Development Bank

Study Sponsor Location:

Cote Divoire

Research Partner

Name of Partner Institution:

Eastern Africa Statistical Training Centre

Type of Organization:

NGO (local) or other civil society organization

Location:

Tanzania

Intervention

Intervention Overview

Intervention:

The SMMRP aimed to accelerate the reduction of maternal and newborn deaths in Mara, Mtwara, Tabora and Zanzibar. While the project’s medium-term outcome was to reduce maternal and neonatal morbidity and mortality, its longer-term outcome was to improve the health and well-being of Tanzanians. The project comprised the following components: i) ?Strengthened Delivery of Maternal Health Services, ii) Strengthened Delivery of Health Care Services, and iii)  Management and Coordination. The SMMRP financed two types of activities. First, it rehabilitated, constructed, and equipped health infrastructure (dispensaries, 2nd line dispensaries, health centres, and district hospitals), including Maternal and Child Health (MCH) Units at dispensaries and obstetric theatres at health centres and some selected district hospitals. The project also constructed new OBYS theatres in selected health facilities, and constructed, rehabilitated, and equipped training institutes in the Mainland (Tabora and Mtwara). Similar activities were undertaken at the ?College of Health Sciences (CHS) in Zanzibar. Other activities include the rehabilitation of the Project Management Unit (PMU) Office in Zanzibar, the procurement, and installation of biomedical equipment, furniture, and radio for all SMMRP-supported health facilities and the procurement of ambulances for district hospitals in the Mainland. Additionally, all new and existing facilities that do not have appropriate water and sanitation facilities were provided with machine-dug boreholes, VIP latrines and placenta pits. For Zanzibar, this comprised constructing incinerators for 3 Primary Health Centres (PHC) for the proper disposal of medical waste in Pemba. 

The second activity financed by the SMMRP was the in-service training of health workers through workshops to update their knowledge and skills in the provision of services on maternal and newborn health care. 

Theory of Change:

The project's direct outcomes can be broadly summarised as increased access to higher quality healthcare, while the expected intermediate outcomes are related to an increase in the utilisation of the healthcare services provided. Finally, the project’s expected impacts are related to improvements in general well-being, reduction in maternal mortality, and reductions in infant mortality and morbidity. 

The expected benefits that accrued to project beneficiaries from the SMMRP are conditional on the following assumptions: i) targeted project beneficiaries use MCH services provided by SMMRP-supported health facilities, ii) project contractors meet all the performance requirements for all SMMRP components, including avoiding implementation delays, iii) health workers in AfDB-supported health facilities are well-trained technically competent to provide quality MCH services for beneficiaries, iv) AfDB-supported health facilities provide adequate and reliable MCH services in project areas, v) targeted beneficiaries are aware and knowledgeable about MCH services provided in AfDB-supported health facilities, vi) training of health workers in the Mainland and Zanzibar would increase intake of students to enable deployment of full-time, qualified staff, vii) provision of staff-housing at SMMRP-supported facilities ensures that communities have access to a health provider at all times, especially during emergencies such as complicated deliveries. In addition, the provision of staff housing contributes to staff retention in remote and underserved areas supported by the project.

Multiple Treatment Arms Evaluated?

No

Implementing Agency

Name of Organization:

Ministry of Health

Type of Organization:

Public Sector, e.g. Government Agency or Ministry

Program Funder

Name of Organization:

African Development Bank

Type of Organization:

Public Sector, e.g. Government Agency or Ministry

Intervention Timing

Intervention or Program Started at time of Registration?

Yes

Start Date:

12/30/2006

End Date:

04/30/2015

Evaluation Method

Evaluation Method Overview

Primary (or First) Evaluation Method:

Matching

Other (not Listed) Method:

Additional Evaluation Method (If Any):

Other (not Listed) Method:

Method Details

Details of Evaluation Approach:

To answer the question of how investments in health infrastructure impact health and economic outcomes, the ideal experiment would be to randomize such investment at the regional level across the whole economy and restrict access to healthcare such that there are no spillovers. However, randomizing the location of infrastructure, especially health facilities, is often infeasible for logistical and ethical reasons. Therefore, we propose to evaluate this program using Propensity Score Matching (PSM), a quasi-experimental method that compares outcomes for program beneficiaries with those of a carefully selected control group relying on some identifying assumptions. 
In SMMRP, because baseline data is not available, we need to assume that, absent the program, outcomes for the treatment group would be identical to those of a control group and that any differences observed between the two groups are caused by the program. Therefore, the selection of a valid control group is critical for the impact evaluation. Since the program took place at the health facility level, the first step is to pair treated facilities with facilities that did not participate in the program but were similar before the program took place. To do so, we used administrative data on characteristics of the health facilities (type of facility and opening year) as well as data from population and demographic census to ensure that local socio-demographic characteristics in the areas served by the facilities were similar before the program took place. 

We will implement a PSM approach to ensure that households in the treatment group are as comparable as possible to households in the control group by matching on similar observable characteristics on a set of variables that should not be affected by the program such as age, education level, assets availability, and housing quality.

Outcomes (Endpoints):

Summary of indicators to be measured:

1. Access to health services: are health services available to people in the study area?  To what extent did the project increase access to services for the treated communities? Were there spillovers to the control communities?

2. Quality of health services: did the program increase the perceived quality of the health services provided? Are patients more likely to trust the treated healthcare facilities than those that did not benefit from the program? Did the program increase the availability of highly trained staff at the facilities? Are healthcare professionals more satisfied with their work?

3. Utilisation of services: did the program increase utilisation of healthcare facilities for antenatal care, delivery, and postnatal care? Did it decrease the number of home births? Did it increase children's vaccination rates and the use of recommended treatments such as deworming? What barriers remain in the utilisation of healthcare services in the study area? How did the program affect the capacity of healthcare facilities?

4. Knowledge: increased utilisation of services should result in higher knowledge on some health-related topics such as contraception, disease prevention, and good practices for taking care of infants and children. We will measure whether women in treated communities are more likely to answer knowledge questions correctly.

5. Health outcomes: did the program improve children's health and nutrition? We will look at the effect of the program on the likelihood of children being sick and on their anthropometric measures. We will also use administrative and/or secondary data to estimate the effect of the program on the mortality rate of mothers and children under 5. 

Unit of Analysis:

Household

Hypotheses:

Our hypothesis is that the program improved health outcomes for the beneficiaries. 

Unit of Intervention or Assignment:

Number of Clusters in Sample:

120 health facility catchment areas

Number of Individuals in Sample:

7000

Size of Treatment, Control, or Comparison Subsamples:

3500 household living in around 60 treatment facilities, 3500 in control living around 60 control facilities.

Data

Outcomes Data

Description:

The data set that will be used to measure outcomes is a household survey conducted by the evaluation team.

Data Already Collected?

No

Data Previously Used?

Data Access:

Data Obtained by the Study Researchers?

Data Approval Process:

Approval Status:

Treatment Assignment Data

Participation or Assignment Information:

Yes

Description:

Data Obtained by the Study Researchers?

Data Previously Used?

Data Access:

Data Obtained by the Study Researchers?

Data Approval Process:

Approval Status:

Data Analysis

Data Analysis Status:

Study Materials

Upload Study Materials:

: ENGLISH household_data.html.pdf

Registration Category

Registration Category:

Prospective, Category 1: Data for measuring impacts have not been collected

Completion

Completion Overview

Intervention Completion Date:

Data Collection Completion Date:

Unit of Analysis:

Clusters in Final Sample:

Total Observations in Final Sample:

Size of Treatment, Control, or Comparison Subsamples:

Findings

Preliminary Report:

Preliminary Report URL:

Summary of Findings:

Paper:

Paper Summary:

Paper Citation:

Data Availability

Data Availability (Primary Data):

Date of Data Availability:

Data URL or Contact:

Access procedure:

Other Materials

Survey:

Survey Instrument Links or Contact:

Program Files:

Program Files Links or Contact:

External Link:

External Link Description:

Description of Changes:

Study Stopped

Date:

Reason: