Download Study

General

Study Overview

Title:

Evaluating the Impact of Behavioural Messages on Honest Reporting of COVID-19 Symptoms via the Mobile COVID-19 Connect Platform in South Africa

Study is 3ie funded:

No

Study ID:

RIDIE-STUDY-ID-62732db8236fe

Initial Registration Date:

05/04/2022

Last Update Date:

05/02/2022

Study Status:

Ongoing

Location(s):

South Africa

Abstract:

In response to rising COVID-19 cases, the South African National Department Of Health (NDOH), in partnership with Praekelt.org, launched the digital COVID-19 Connect platform to disseminate accurate and timely information on COVID-19 transmission, prevention, symptoms, and up-to-date statistics to the public. The COVID-19 Connect platform consists of a HealthAlert tool, which helps users assess their COVID-19 risk through a COVID-19 symptom checker and receive appropriate health behavior recommendations in return. In collaboration with Praekelt.org and the NDOH, IDinsight will evaluate the effectiveness of light-touch behavioral messaging nudges on new and existing HealthCheck users. The intervention tests whether messaging appealing to a new or existing user’s commitment to honesty can lead to more truthful responses on the HealthCheck symptom tracker survey. The intervention targets college students, faculty, and other employees of colleges in South Africa who regularly use HealthCheck as a requirement for the “Higher Health” initiative, which requires students to complete HealthCheck to be granted entry to public spaces on college campuses to reduce the spread of COVID-19. 

IDinsight and Praekelt.org are conducting a randomized controlled trial to assess the impact of three different types of messages on honest reporting of COVID-19 symptoms. Three groups of users will receive different messages (an honesty declaration at the beginning of HealthCheck, a message with a pro-social appeal, and a message that highlights the consequences of dishonesty). One group of users will receive the status quo messaging model. The study will compare the user behavior, symptoms reported, and usage of HealthCheck across these different groups. We hope this study can contribute to the literature on ways to optimize the efficacy of mobile health platforms on key health outcomes. 

Registration Citation:

Categories:

Health, Nutrition, and Population

Additional Keywords:

COVID-19, Mobile Health

Secondary ID Number(s):

Principal Investigator(s)

Name of First PI:

Dr. Crystal Haijing Huang

Affiliation:

IDinsight

Name of Second PI:

Debbie Rogers

Affiliation:

Praekelt.org

Study Sponsor

Name:

Global Innovation Fund

Study Sponsor Location:

United Kingdom

Research Partner

Name of Partner Institution:

Praekelt.org

Type of Organization:

NGO-international

Location:

South Africa

Intervention

Intervention Overview

Intervention:

This study targets college students, faculty, and other employees of colleges in South Africa who regularly use HealthCheck to be granted entry to public spaces on college campuses to reduce the spread of COVID-19. Each of the messaging models is described below:

Control (Status-quo) - The HealthCheck survey currently includes an honesty declaration at the end of the survey where the user is asked to confirm that the information they have provided is accurate and whether they consent to being contacted by the NDOH if necessary. It is possible that in its current state where both questions are asked as one, the user may muddle their response. The control arm will use this existing HealthCheck feature but separate out the two components (honesty declaration, consent to be contacted) into two distinct questions. 

Treatment 1 (Status-quo + beginning): This treatment arm will use the same honesty declaration used in the control arm. However, instead of placing this declaration at the end of the survey, it will place the declaration at the beginning of the survey. 

Treatment 2 (Pro-social appeal): This treatment arm will be similar to treatment 1 as it will also situate the honesty declaration at the beginning of the survey. However, instead of using a neutral framing, the declaration will have a “pro-social” framing, by appealing to the user’s motivation to protect the health of those around them (i.e. their campus community) by honestly reporting their symptoms. 

Treatment 3 (Salience of consequences): Similar to treatment arms 1 and 2, this treatment arm will also situate the honesty declaration at the beginning of the HealthCheck survey. However, the honesty declaration will include additional information emphasizing that the user may regret passing COVID to a vulnerable family member. 

Theory of Change:

The theory of change posits that specifically framed messages can improve individuals' honesty, which in turn, can mitigate the risk of spreading COVID-19 on university campuses, thereby improving societal health outcomes. 

Since HigherHealth users must repeatedly complete the COVID-19 risk self-assessment on a daily or near-daily basis and produce a low risk result to gain entry to college campuses, we hypothesize that over time responses are likely to become less truthful on average, with a higher proportion of “low risk” users relative to the true prevalence of low-risk symptoms in the population. One reason is that HealthCheck represents a barrier to entry and inconvenience: incentivized by freedom of movement within public spaces on campus, HigherHealth users may be likely to downplay symptoms to achieve a “low risk” result, particularly those who are on the margins of low and moderate risk. Another reason could simply be that over time, the task becomes more mechanical and less deliberate, so users are likely to fill out the same set of responses automatically each time, even when their symptoms may have changed, also leading to a greater proportion of “low risk” categorizations than is true.

Evidence from the literature reveals that messaging can increase honesty by increasing people’s attention to their own moral standards and/or outlining threats to dishonest behavior. Since the costs of dishonesty in the context of the COVID-19 pandemic are not just private (such as financial or reputational losses) but also societal (health), we posit that messages emphasizing honest behavior could have an added effect on improving honesty. If students with COVID-19 are more honest, they are more likely to honestly report "high risk" symptoms as opposed to falsifying their symptoms, which would decrease the spread of COVID-19 on college campuses.

Multiple Treatment Arms Evaluated?

Yes

Implementing Agency

Name of Organization:

Praekelt.org

Type of Organization:

NGO (International)

Program Funder

Name of Organization:

Global Innovation Fund

Type of Organization:

NGO (International)

Intervention Timing

Intervention or Program Started at time of Registration?

Yes

Start Date:

03/23/2022

End Date:

04/19/2022

Evaluation Method

Evaluation Method Overview

Primary (or First) Evaluation Method:

Randomized control trial

Other (not Listed) Method:

Additional Evaluation Method (If Any):

Other (not Listed) Method:

Method Details

Details of Evaluation Approach:

Randomization is performed at the level of the individual user, where a “unique user” is defined by a combination of their phone number and the channel they are using to access the COVID-19 Connect platform (USSD, WhatsApp). Randomization is done automatically whenever a user opts into HealthCheck - they will be assigned to one of the three treatment arms or the control arm and see the corresponding message. We stop the intervention once we have randomized 20,000 individuals (5,000 users per arm). We will stratify treatment by source (USSD, WhatsApp) and province (9 provinces), with a total of 18 strata.

Outcomes (Endpoints):

Metric	Unit of analysis	Type
User completed full set of HealthCheck symptom questions	HealthCheck	Binary
Number of days user avoided visiting campus in a week, where proxy for avoidance =1 if healthcheck is either moderate risk, high-risk, or incomplete	User-week	Continuous
HealthCheck resulted in a “medium” or ‘high” risk categorization compared to “low risk”	Completed HealthCheck	Binary
Number of symptoms reported	Completed HealthCheck	Continuous
Type of symptom reported	Completed HealthCheck	Binary
Time taken to complete HealthCheck (in minutes)	Completed HealthCheck	Continuous
User completed HealthCheck more than once per calendar day	User-day	Binary
User completed HealthCheck more than once per calendar day and moved from a higher risk category to a lower one (Moderate > Low, High > Moderate, High > Low)	User-day	Binary

Unit of Analysis:

Individual User

Hypotheses:

Primary:

• Users in any/all treatment groups are more likely to complete all questions on the HealthCheck symptoms tracker than those in the control group.

• Users in any/all treatment groups avoid campus more than those in the control group. 

Secondary:

• Users in any/all treatment groups submit more symptoms in which they are tagged as “high risk” than those in the control group. 

• Users in any/all treatment groups report more symptoms than those in the control group. 

• Users in any/all treatment groups spend more time on Health Check than those in the control group. 

• Users in any/all treatment groups fill out Health Check in a given day fewer times than those in the control group.

• Users in any/all treatment groups move from a riskier to lower risk state in a given day less than those in the control group.

Unit of Intervention or Assignment:

Individual users

Number of Clusters in Sample:

NA

Number of Individuals in Sample:

20,000

Size of Treatment, Control, or Comparison Subsamples:

5,000

Supplementary Files

Analysis Plan:

Honesty - RIDIE PAP Attachment.pdf

Other Documents:

Data

Outcomes Data

Description:

No primary data will be collected for this study. All outcomes of interest will be measured through back-end user-data collected on the COVID-19 Connect platform. Praekelt.org will share the anonymized data.

Data Already Collected?

Yes

Data Previously Used?

No

Data Access:

Restricted -- Access requires a formal approval process

Data Obtained by the Study Researchers?

Data Approval Process:

Praekelt.org and IDinsight have a data sharing agreement. This agreement necessitates that data collected by Praekelt.org is anonymized and stored on an encrypted cloud based server. If others want to access the data, they will need to create a similar data sharing agreement with Praekelt.org. Approval is required from the Praekelt.org team to grant access to data.

Approval Status:

Yes-obtained approval and have received the data

Treatment Assignment Data

Participation or Assignment Information:

Yes

Description:

Data Obtained by the Study Researchers?

Data Previously Used?

Data Access:

Data Obtained by the Study Researchers?

Data Approval Process:

Approval Status:

Data Analysis

Data Analysis Status:

No

Study Materials

Upload Study Materials:

Registration Category

Registration Category:

Prospective, Category 3: Data for measuring impacts have been obtained/collected by the research team but analysis for this evaluation has not started

Completion

Completion Overview

Intervention Completion Date:

Data Collection Completion Date:

Unit of Analysis:

Clusters in Final Sample:

Total Observations in Final Sample:

Size of Treatment, Control, or Comparison Subsamples:

Findings

Preliminary Report:

Preliminary Report URL:

Summary of Findings:

Paper:

Paper Summary:

Paper Citation:

Data Availability

Data Availability (Primary Data):

Date of Data Availability:

Data URL or Contact:

Access procedure:

Other Materials

Survey:

Survey Instrument Links or Contact:

Program Files:

Program Files Links or Contact:

External Link:

External Link Description:

Description of Changes:

Study Stopped

Date:

Reason: