Download StudyGeneral

Study Overview

Evaluating the Impact of Behavioural Messages on Honest Reporting of COVID-19 Symptoms via the Mobile COVID-19 Connect Platform in South Africa
Study is 3ie funded:
Study ID:
Initial Registration Date:
Last Update Date:
Study Status:
South Africa

In response to rising COVID-19 cases, the South African National Department Of Health (NDOH), in partnership with, launched the digital COVID-19 Connect platform to disseminate accurate and timely information on COVID-19 transmission, prevention, symptoms, and up-to-date statistics to the public. The COVID-19 Connect platform consists of a HealthAlert tool, which helps users assess their COVID-19 risk through a COVID-19 symptom checker and receive appropriate health behavior recommendations in return. In collaboration with and the NDOH, IDinsight will evaluate the effectiveness of light-touch behavioral messaging nudges on new and existing HealthCheck users. The intervention tests whether messaging appealing to a new or existing user’s commitment to honesty can lead to more truthful responses on the HealthCheck symptom tracker survey. The intervention targets college students, faculty, and other employees of colleges in South Africa who regularly use HealthCheck as a requirement for the “Higher Health” initiative, which requires students to complete HealthCheck to be granted entry to public spaces on college campuses to reduce the spread of COVID-19. 

IDinsight and are conducting a randomized controlled trial to assess the impact of three different types of messages on honest reporting of COVID-19 symptoms. Three groups of users will receive different messages (an honesty declaration at the beginning of HealthCheck, a message with a pro-social appeal, and a message that highlights the consequences of dishonesty). One group of users will receive the status quo messaging model. The study will compare the user behavior, symptoms reported, and usage of HealthCheck across these different groups. We hope this study can contribute to the literature on ways to optimize the efficacy of mobile health platforms on key health outcomes. 

Registration Citation:
Health, Nutrition, and Population
Additional Keywords:
COVID-19, Mobile Health
Secondary ID Number(s):

Principal Investigator(s)

Name of First PI:
Dr. Crystal Haijing Huang
Name of Second PI:
Debbie Rogers

Study Sponsor

Global Innovation Fund
Study Sponsor Location:
United Kingdom

Research Partner

Name of Partner Institution:
Type of Organization:
South Africa

Intervention Overview


This study targets college students, faculty, and other employees of colleges in South Africa who regularly use HealthCheck to be granted entry to public spaces on college campuses to reduce the spread of COVID-19. Each of the messaging models is described below:

Control (Status-quo) - The HealthCheck survey currently includes an honesty declaration at the end of the survey where the user is asked to confirm that the information they have provided is accurate and whether they consent to being contacted by the NDOH if necessary. It is possible that in its current state where both questions are asked as one, the user may muddle their response. The control arm will use this existing HealthCheck feature but separate out the two components (honesty declaration, consent to be contacted) into two distinct questions. 

Treatment 1 (Status-quo + beginning): This treatment arm will use the same honesty declaration used in the control arm. However, instead of placing this declaration at the end of the survey, it will place the declaration at the beginning of the survey. 

Treatment 2 (Pro-social appeal): This treatment arm will be similar to treatment 1 as it will also situate the honesty declaration at the beginning of the survey. However, instead of using a neutral framing, the declaration will have a “pro-social” framing, by appealing to the user’s motivation to protect the health of those around them (i.e. their campus community) by honestly reporting their symptoms. 

Treatment 3 (Salience of consequences): Similar to treatment arms 1 and 2, this treatment arm will also situate the honesty declaration at the beginning of the HealthCheck survey. However, the honesty declaration will include additional information emphasizing that the user may regret passing COVID to a vulnerable family member. 

Theory of Change:

The theory of change posits that specifically framed messages can improve individuals' honesty, which in turn, can mitigate the risk of spreading COVID-19 on university campuses, thereby improving societal health outcomes. 

Since HigherHealth users must repeatedly complete the COVID-19 risk self-assessment on a daily or near-daily basis and produce a low risk result to gain entry to college campuses, we hypothesize that over time responses are likely to become less truthful on average, with a higher proportion of “low risk” users relative to the true prevalence of low-risk symptoms in the population. One reason is that HealthCheck represents a barrier to entry and inconvenience: incentivized by freedom of movement within public spaces on campus, HigherHealth users may be likely to downplay symptoms to achieve a “low risk” result, particularly those who are on the margins of low and moderate risk. Another reason could simply be that over time, the task becomes more mechanical and less deliberate, so users are likely to fill out the same set of responses automatically each time, even when their symptoms may have changed, also leading to a greater proportion of “low risk” categorizations than is true.

Evidence from the literature reveals that messaging can increase honesty by increasing people’s attention to their own moral standards and/or outlining threats to dishonest behavior. Since the costs of dishonesty in the context of the COVID-19 pandemic are not just private (such as financial or reputational losses) but also societal (health), we posit that messages emphasizing honest behavior could have an added effect on improving honesty. If students with COVID-19 are more honest, they are more likely to honestly report "high risk" symptoms as opposed to falsifying their symptoms, which would decrease the spread of COVID-19 on college campuses.

Multiple Treatment Arms Evaluated?

Implementing Agency

Name of Organization:
Type of Organization:
NGO (International)

Program Funder

Name of Organization:
Global Innovation Fund
Type of Organization:
NGO (International)

Intervention Timing

Intervention or Program Started at time of Registration?
Start Date:
End Date:
Evaluation Method

Evaluation Method Overview

Primary (or First) Evaluation Method:
Randomized control trial
Other (not Listed) Method:
Additional Evaluation Method (If Any):
Other (not Listed) Method:

Method Details

Details of Evaluation Approach:

Randomization is performed at the level of the individual user, where a “unique user” is defined by a combination of their phone number and the channel they are using to access the COVID-19 Connect platform (USSD, WhatsApp). Randomization is done automatically whenever a user opts into HealthCheck - they will be assigned to one of the three treatment arms or the control arm and see the corresponding message. We stop the intervention once we have randomized 20,000 individuals (5,000 users per arm). We will stratify treatment by source (USSD, WhatsApp) and province (9 provinces), with a total of 18 strata.

Outcomes (Endpoints):


Unit of analysis


User completed full set of HealthCheck symptom questions  



Number of days user avoided visiting campus in a week, where proxy for avoidance =1 if healthcheck is either moderate risk, high-risk, or incomplete 



HealthCheck resulted in a “medium” or ‘high” risk categorization compared to “low risk”

Completed HealthCheck


Number of symptoms reported

Completed HealthCheck


Type of symptom reported

Completed HealthCheck


Time taken to complete HealthCheck (in minutes) 

Completed HealthCheck


User completed HealthCheck more than once per calendar day



User completed HealthCheck more than once per calendar day and moved from a higher risk category to a lower one (Moderate > Low, High > Moderate, High > Low) 



Unit of Analysis:
Individual User


  • Users in any/all treatment groups are more likely to complete all questions on the HealthCheck symptoms tracker than those in the control group.

  • Users in any/all treatment groups avoid campus more than those in the control group. 


  • Users in any/all treatment groups submit more symptoms in which they are tagged as “high risk” than those in the control group. 

  • Users in any/all treatment groups report more symptoms than those in the control group. 

  • Users in any/all treatment groups spend more time on Health Check than those in the control group. 

  • Users in any/all treatment groups fill out Health Check in a given day fewer times than those in the control group.

  • Users in any/all treatment groups move from a riskier to lower risk state in a given day less than those in the control group.

Unit of Intervention or Assignment:
Individual users
Number of Clusters in Sample:
Number of Individuals in Sample:
Size of Treatment, Control, or Comparison Subsamples:

Supplementary Files

Analysis Plan:
Honesty - RIDIE PAP Attachment.pdf
Other Documents:

Outcomes Data

No primary data will be collected for this study. All outcomes of interest will be measured through back-end user-data collected on the COVID-19 Connect platform. will share the anonymized data.
Data Already Collected?
Data Previously Used?
Data Access:
Restricted -- Access requires a formal approval process
Data Obtained by the Study Researchers?
Data Approval Process: and IDinsight have a data sharing agreement. This agreement necessitates that data collected by is anonymized and stored on an encrypted cloud based server. If others want to access the data, they will need to create a similar data sharing agreement with Approval is required from the team to grant access to data.
Approval Status:
Yes-obtained approval and have received the data

Treatment Assignment Data

Participation or Assignment Information:
Data Obtained by the Study Researchers?
Data Previously Used?
Data Access:
Data Obtained by the Study Researchers?
Data Approval Process:
Approval Status:

Data Analysis

Data Analysis Status:

Study Materials

Upload Study Materials:

Registration Category

Registration Category:
Prospective, Category 3: Data for measuring impacts have been obtained/collected by the research team but analysis for this evaluation has not started

Completion Overview

Intervention Completion Date:
Data Collection Completion Date:
Unit of Analysis:
Clusters in Final Sample:
Total Observations in Final Sample:
Size of Treatment, Control, or Comparison Subsamples:


Preliminary Report:
Preliminary Report URL:
Summary of Findings:
Paper Summary:
Paper Citation:

Data Availability

Data Availability (Primary Data):
Date of Data Availability:
Data URL or Contact:
Access procedure:

Other Materials

Survey Instrument Links or Contact:
Program Files:
Program Files Links or Contact:
External Link:
External Link Description:
Description of Changes:

Study Stopped