Download StudyGeneral

Study Overview

Title:
Measuring the Impact of a Health Delivery Intervention in the Ahafo, Ashanti, Bono East, Central Eastern, Northern, North East, Upper East, and Volta Regions of Ghana
Study is 3ie funded:
No
Study ID:
RIDIE-STUDY-ID-5f84429e27881
Initial Registration Date:
10/12/2020
Last Update Date:
09/11/2020
Study Status:
Ongoing
Location(s):
Ghana
Abstract:

In 2019, Zipline, a medical product drone delivery service, expanded its operations from Rwanda into Ghana. It has since established four distribution centres to ultimately serve approximately 2,000 health facilities across the country, with the goal of improving access to medical products such as blood, antivenom, vaccines, and essential medicines. Since March 2020, Zipline has also been distributing Covid-19 related products, including PPE, test samples, and supportive treatments, and it is considering the potential for distribution of Covid-19 vaccines. Despite the growing popularity of the use of drone technology to deliver medical products, little evidence exists that quantifies its impact on medical stocks and quality of care. Zipline, LLamasoft, and IDinsight are partnering to study the impact of Zipline’s operations on health facilities.

The study will leverage Zipline’s non-universal roll-out in Ghana to estimate impacts using difference-in-differences with matching strategy. This involves sampling health facilities with certain observable characteristics within Zipline service zones and matching these facilities with similar health facilities outside the service zones on baseline characteristics and pre-intervention trends. To identify the impact of Zipline, we will compare the differences in pre-post outcomes between the two types of facilities using survey and administrative data. A discrete event simulation model of the Ghana public health supply chain will be used to estimate Zipline’s effect on national supply chain costs. 

Results from the study will contribute to a nascent but growing body of evidence in global public health on the effectiveness of on-demand product fulfillment, and further, specifically those involving drone delivery.

Registration Citation:
Categories:
Health, Nutrition, and Population
Additional Keywords:
Secondary ID Number(s):

Principal Investigator(s)

Name of First PI:
Dr Valentina Brailovskaya
Affiliation:
IDinsight
Name of Second PI:
Dr Alison Connor
Affiliation:
IDinsight

Study Sponsor

Name:
Bill & Melinda Gates Foundation
Study Sponsor Location:
United States

Research Partner

Name of Partner Institution:
LLamasoft
Type of Organization:
Private firm
Location:
United States
Intervention

Intervention Overview

Intervention:

Zipline is a drone delivery service that provides third-party logistics (3PL) services, holding customers’ products in warehouses to minimize the lead time between order and delivery for end recipients. In Ghana, Zipline has partnered with Ghana Health Service (GHS) to provide last mile logistics services and improve access to crucial health products in health facilities. Zipline is establishing four distribution centres (Omenako, Mampong, Walewale, and Sefwi Wiawso) to serve approximately 2,000 health facilities across Ghana.

Zipline provides emergency and non-emergency on-demand medical products (such as blood, emergency and routine vaccines, and emergency and essential medicines). This delivery system is intended to improve timely access to medical products, reduce instances and duration of stockouts in health facilities, and ultimately improve health outcomes. The process is initiated when a healthcare worker submits an order for a particular product via SMS or a phone call. The order is recorded at the closest distribution center, and the required medical products are packed into a drone. The automatically operated drone drops off the medical products in an allocated location nearby the facility. The medical products are picked up by the facility staff within about 30 minutes.

Theory of Change:

The overall theory of change operates as following:

Health workers and health facilities use Zipline to order health care products, which are supplied by the  Ghana Health Service. Zipline receives the order, then processes and confirms it. The package is then launched and dropped at the health facility. This results in a faster and more convenient delivery system for health facilities. The main outcome is the reduction of stockout instances and duration, and improved product availability and a more equitable medical supply system. This can lead to fewer patient referrals and less waiting time before treatment. Ultimately this feeds into the a final impact of a) better quality treatment and improved health outcomes, and b) less wastage of medical products and lower supply chain costs, leading to cost savings for the Ghanaian public health system. 

Multiple Treatment Arms Evaluated?
No

Implementing Agency

Name of Organization:
Zipline
Type of Organization:
Private for profit organization

Program Funder

Name of Organization:
Bill and Melinda Gates Foundation
Type of Organization:
Other

Intervention Timing

Intervention or Program Started at time of Registration?
Yes
Start Date:
04/01/2019
End Date:
Evaluation Method

Evaluation Method Overview

Primary (or First) Evaluation Method:
Other (specify)
Other (not Listed) Method:
Difference in differences on a matched sample of facilities
Additional Evaluation Method (If Any):
Other (not Listed) Method:

Method Details

Details of Evaluation Approach:

The research team conducted a two stage matching process.

The first stage of matching was completed before the baseline data collection using  District Health Information Management System (DHIMs). The dataset contained basic information about facilities and health records for selected outcomes. Researchers used 1:1 coarsen matching  to create a comparable treatment and control sample based on observable characteristics. The process created a sample of 452 facilities across the two groups, interviewed during baseline visits. The baseline sample consisted of district hospitals, healthcare centers, and Health Service Community-Based Health Planning and Services (CHPS). During the data collection, the following staff members were interviewed to document outcomes of interest: heads of facilities, staff members in charge of all dispensaries, health workers and patients. Patients were also interviewed. The final sample of baseline facilities consisted of  422 health facilities ( 238 and 184 within and outside of Zipline zones).

The second 1:1 coarsen matching is completed using information collected during the baseline and is used to further narrow down to the sample of the most comparable facilities. 

The impacts on outcomes of interest will be measured using data from  high frequency phone surveys and an endline survey. To quantify impacts of Zipline surveys we will either use difference-in-differences and compare trends in outcomes of interest between treatment and control groups or ANCOVA for quantifying the impacts on variables which were not collected during the baseline pretreatment period. We plan to use Conley estimation of standard errors to adjust for spatial correlation of the error term. 

Outcomes (Endpoints):

Zipline’s near-term effects on health facilities, by exploring differences between facilities served by Zipline and control facilities across outcome groups:

Primary outcomes:

  1. Product stocking - whether or not mandated medical products are stocked at the facility

  2. Product stockouts - instances and durations of stockout of mandated products

  3. Stocking behavior - the actions taken by heath facility workers to resolve stockouts

  4. Treatment behavior - the actions taken by health workers to treat patients for certain medical conditions

Secondary outcomes:

  1. Attitudes and perceptions - the views of health workers and local community members towards Zipline

  2. What effect do Zipline’s operations have on supply chain costs? Cost components to be considered include: transport costs; storage and physical infrastructure costs; expiry, waste, and loss of potency; inventory holding cost; and management, overhead, and labour.

  3. How is the supply chain and facility-level patient demand for services impacted by the COVID-19 pandemic? Are drone deliveries an impactful intervention during a public health emergency and after? 

Exploratory outcomes:

  1. Treatment seeking behaviour - patient decisions to visit particular health facilities to seek care, and the costs associated with these decisions

Unit of Analysis:
Health facility
Hypotheses:

H1: Does Zipline improve healthcare’s capacity to stock all mandated products?

H2: Does Zipline reduce instances and duration of medical product stockouts?

H3: Does Zipline improve healthcare worker’s ability to treat patients?

H4: Is there an indication that changes in healthcare systems lead to reduced supply chain costs?

H5: Is there an indication that changes in healthcare systems lead to improved patient outcomes?

H6: Is drone-delivery service impactful during emergencies (such as COVID19 pandemic?)

 

Unit of Intervention or Assignment:
Health facilities
Number of Clusters in Sample:
Zipline currently has 4 distribution centers (only 3 are in the study sample) and all healthcare facilities located within the flying can be served by Zipline.
Number of Individuals in Sample:
422 facilities for baseline. Final sample between 200 - 300 facilities.
Size of Treatment, Control, or Comparison Subsamples:
200 and 300 facilities in total

Supplementary Files

Analysis Plan:
Other Documents:
Data

Outcomes Data

Description:
Primary data will be collected on Health worker stocking, treatment behaviour, attitudes and perceptions of healthcare workers and patients through interviews with the heads of health facilities, staff members in charge of dispensaries, health workers and patients. Data collection activities include the following: In-person baseline survey conducted at health facilities: At selected treatment facilities At control facilities that share similar characteristics to tmt
Data Already Collected?
No
Data Previously Used?
Data Access:
Data Obtained by the Study Researchers?
Data Approval Process:
Approval Status:

Treatment Assignment Data

Participation or Assignment Information:
No
Description:
List of facilities obtained from Zipline.
Data Obtained by the Study Researchers?
Yes
Data Previously Used?
Not sure
Data Access:
Restricted -- Access requires a formal approval process
Data Obtained by the Study Researchers?
Data Approval Process:
Contact Zipline and IDinsight team.
Approval Status:
Yes-obtained approval and have received the data

Data Analysis

Data Analysis Status:

Study Materials

Upload Study Materials:

Registration Category

Registration Category:
Prospective, Category 1: Data for measuring impacts have not been collected
Completion

Completion Overview

Intervention Completion Date:
Data Collection Completion Date:
Unit of Analysis:
Clusters in Final Sample:
Total Observations in Final Sample:
Size of Treatment, Control, or Comparison Subsamples:

Findings

Preliminary Report:
Preliminary Report URL:
Summary of Findings:
Paper:
Paper Summary:
Paper Citation:

Data Availability

Data Availability (Primary Data):
Date of Data Availability:
Data URL or Contact:
Access procedure:

Other Materials

Survey:
Survey Instrument Links or Contact:
Program Files:
Program Files Links or Contact:
External Link:
External Link Description:
Description of Changes:

Study Stopped

Date:
Reason: