RIDIE
Home » Search » Testing the effectiveness of Quality-Based WhatsApp Messaging on Vaccine Hesitancy and Childhood Immunization in a National Maternal Messaging Program
Download StudyGeneral

Study Overview

Title:
Testing the effectiveness of Quality-Based WhatsApp Messaging on Vaccine Hesitancy and Childhood Immunization in a National Maternal Messaging Program
Study is 3ie funded:
No
Study ID:
RIDIE-STUDY-ID-62bb041beb158
Initial Registration Date:
06/28/2022
Last Update Date:
09/16/2022
Study Status:
Ongoing
Location(s):
South Africa
Abstract:

The case for mobile health (mHealth) has been gaining traction as a source to improve health outcomes of individuals. mHealth refers to the use of Information and Communication Technologies (ICT) to support health care. The ubiquity and penetration of mobile phones presents the opportunity to deliver health care services directly to citizens, with the greatest potential gains in under-resourced health ecosystems. Operating for 7 years and currently supporting 1.28 million mothers through their pregnancies and early childhood care, MomConnect represents one of the largest maternal health messaging platforms in the world. Implemented at the national level, MomConnect has been credited with being the first national-scale mHealth program of its kind and has won numerous international awards. 

As a WhatsApp-delivered service, MomConnect has been limited by WhatsApp’s historical terms of service. In its current form, MomConnect can only start a conversation with mothers once a week by alerting them of their upcoming antenatal care (ANC) appointment. Recent changes to WhatsApp’s terms of service have removed the restriction around push messages for certain programs, including MomConnect. Messages have historically been restricted to weekly conversation starters and carry only generic administrative reminder-style information. As a result, it is possible that many mothers are failing to engage with the content as much as they could.  If mothers were more engaged, it could greatly increase exposure and knowledge of reducing the total number of mothers exposed to critical maternal and infant health content. With the updated terms of service, there is an opportunity to test different ways of engaging mothers. 

This study proposes to evaluate, through a randomized controlled trial (RCT), the relative effect of four new behaviorally-informed program models on key knowledge of and adoption of healthy behaviors.

Registration Citation:
Categories:
Health, Nutrition, and Population
Additional Keywords:
Mobile Health
Secondary ID Number(s):
NCT05505760 (clinicaltrials.gov)

Principal Investigator(s)

Name of First PI:
Crystal Huang
Affiliation:
IDinsight
Name of Second PI:
Debbie Rogers
Affiliation:
Praekelt.org

Study Sponsor

Name:
Facebook
Study Sponsor Location:
United States

Research Partner

Name of Partner Institution:
Praekelt.org
Type of Organization:
NGO-international
Location:
South Africa
Intervention

Intervention Overview

Intervention:

This study targets moms who are between 15-30 weeks pregnant and register for MomConnect during their antenatal care appointments. Each of the messaging models is described below:

  • Appointment Reminder Model (ARM), WhatsApp (control) - Mothers receive weekly conversation starter messages reminding them about their upcoming clinic appointments, providing more comprehensive and relevant maternal health information only after mothers respond to the appointment reminder.

  • Relevant Content Model (RCM), WhatsApp: Mothers receive weekly conversation starter messages, which carry both clinic appointment reminders along with some health information relevant to their pregnancy/postpartum stage. In addition, a list of “frequently asked questions” relevant to the week of pregnancy the mother is in are provided so that mothers can engage further.

  • RCM, SMS: Mothers receive the same relevant content via SMS. Mothers can access the list of frequently asked questions relevant to their week of pregnancy via USSD.

  • Browsable Content Model (BCM), WhatsApp: Mothers receive weekly conversation starter messages, which carry clinic appointment reminders and maternal health information, as well as the option to browse a menu of relevant health information that mothers can explore themselves over WhatsApp.

  • RCM + BCM, WhatsApp: This is a combination of the RCM and BCM models, including appointment reminders, clinical information, a browsable menu and prompts to relevant stage-based topics.

Theory of Change:

The theory of change posits that if moms are better able to access relevant information, they will be more likely to adopt healthy behaviors that will benefit them and their infant.

As MomConnect’s content is approved by the National Department of Health (NDOH),  it avoids women needing to navigate complicated, contradictory, or unverified information available on the internet. Recent changes to WhatsApp’s terms of service have removed the restriction around push messages for certain programs, including MomConnect. Messages have historically been restricted to weekly conversation starters and carry only generic administrative reminder-style information. As a result, it is possible that many mothers are failing to engage with the content as much as they could.  If mothers were more engaged, it could greatly increase exposure and knowledge of reducing the total number of mothers exposed to critical maternal and infant health content. With the updated terms of service, there is an opportunity to test different ways of engaging mothers. 

In turn, if moms are given access to more reliable information, they will better be able to know and act according to recommended best practices, therefore improving maternal health outcomes.

Multiple Treatment Arms Evaluated?
Yes

Implementing Agency

Name of Organization:
Praekelt.org
Type of Organization:
NGO (International)

Program Funder

Name of Organization:
Facebook
Type of Organization:
Private for profit organization

Intervention Timing

Intervention or Program Started at time of Registration?
Yes
Start Date:
05/09/2022
End Date:
12/23/2022
Evaluation Method

Evaluation Method Overview

Primary (or First) Evaluation Method:
Randomized control trial
Other (not Listed) Method:
Additional Evaluation Method (If Any):
Other (not Listed) Method:

Method Details

Details of Evaluation Approach:

Randomization is performed at the level of the individual mom, where a “unique user” is defined by their phone number. Randomization is done automatically whenever a user opts into MomConnect. We will stratify treatment by Mom’s province, weeks pregnant, and age, with a total of 54 strata.

Outcomes (Endpoints):

Primary Outcomes:

  • Number of childhood immunizations 8 weeks after birth (out of 6)
  • Number of ANC visits during poregnancy
  • Knowledge and intention of healthy behaviors (summative index)
  • Adoption of healthy behaviors (summative index)

Secondary Outcomes:

  • Survey enrollment
  • Survey completion
  • Engagement
  • Rate of successful message delivery
  • Rate of read messages
  • Opt-out
  • Conversation starters sent
  • Clinic reminders sent
  • Keyword messages sent
  • Non-keyword messages sent
Change History for Outcomes (Endpoints)
Changed On Previous Value
09/16/2022

Primary Outcomes:

  • Knowledge and intention of healthy behaviors
  • Adoption of healthy behaviors

Secondary Outcomes:

  • Survey enrollment
  • Survey completion
  • Engagement
  • Rate of successful message delivery
  • Rate of read messages
  • Opt-out
  • Conversation starters sent
  • Clinic reminders sent
  • Keyword messages sent
  • Non-keyword messages sent
08/22/2022

Primary Outcomes:

  • Knowledge and intention of healthy behaviors
  • Adoption of healthy behaviors

Secondary Outcomes:

  • Survey enrollment rate
  • Survey completion rate
  • Engagement
  • Rate of successful message delivery
  • Rate of read messages
  • Opt-out rate
  • Conversation starters sent
  • Clinic reminders sent
  • Keyword messages sent
  • Non-keyword messages sent
Unit of Analysis:
Pregnant woman/mom (individual)
Hypotheses:

Primary:

  • Users in any/all treatment groups have better knowledge of maternal health than those in the control group.

  • Users in any/all treatment groups adopt healthy best practices more than those in the control group.

Unit of Intervention or Assignment:
Pregnant woman/mom (individual)
Number of Clusters in Sample:
NA
Number of Individuals in Sample:
6000
Size of Treatment, Control, or Comparison Subsamples:
1200

Supplementary Files

Analysis Plan:
20220916_MomConnect PAP.pdf
Change History for Analysis Plan
Changed On Previous Value
09/16/2022 20220822_MomConnect PAP.pdf
08/22/2022 MomConnect PAP.pdf
Other Documents:
Data

Outcomes Data

Description:
Primary data will be collected over a series of WhatsApp or USSD surveys in baseline and midline. At endline, IDinsight will conduct a series of mobile phone surveys with moms. Backend data will be collected by Praekelt.org to monitor moms’ levels of engagement.
Data Already Collected?
No
Data Previously Used?
Data Access:
Data Obtained by the Study Researchers?
Data Approval Process:
Approval Status:

Treatment Assignment Data

Participation or Assignment Information:
Yes
Description:
Data Obtained by the Study Researchers?
Data Previously Used?
Data Access:
Data Obtained by the Study Researchers?
Data Approval Process:
Approval Status:

Data Analysis

Data Analysis Status:

Study Materials

Upload Study Materials:
Baseline Survey Instrument: MomConnect_RIDIE_Baseline.pdf
Midline Survey Instrument: MomConnect_RIDIE_Midline.pdf
Endline Survey Instrument: MomConnect_RIDIE_Endline.pdf
Change History for Upload Study Materials
Changed On Previous Value
09/16/2022 Description: Baseline Survey Instrument
Filename: MomConnect_RIDIE_Baseline.pdf
Link: http://
Description: Midline Survey Instrument
Filename: MomConnect_RIDIE_Midline.pdf
Link: http://
Description: Endline Survey Instrument
Filename: MomConnect_RIDIE_Endline.pdf
Link: http://
Description: Pre-Analysis Plan
Filename: 20220822_MomConnect PAP.pdf
Link: http://
08/22/2022 Description: Baseline Survey Instrument
Filename: MomConnect_RIDIE_Baseline.pdf
Link: http://
Description: Midline Survey Instrument
Filename: MomConnect_RIDIE_Midline.pdf
Link: http://
Description: Endline Survey Instrument
Filename: MomConnect_RIDIE_Endline.pdf
Link: http://

Registration Category

Registration Category:
Prospective, Category 1: Data for measuring impacts have not been collected
Completion

Completion Overview

Intervention Completion Date:
Data Collection Completion Date:
Unit of Analysis:
Clusters in Final Sample:
Total Observations in Final Sample:
Size of Treatment, Control, or Comparison Subsamples:

Findings

Preliminary Report:
Preliminary Report URL:
Summary of Findings:
Paper:
Paper Summary:
Paper Citation:

Data Availability

Data Availability (Primary Data):
Date of Data Availability:
Data URL or Contact:
Access procedure:

Other Materials

Survey:
Survey Instrument Links or Contact:
Program Files:
Program Files Links or Contact:
External Link:
External Link Description:
Description of Changes:

Study Stopped

Date:
Reason: