Download Study

General

Study Overview

Title:

Impact evaluation of the Collective Impact (CI) Initiative program

Study is 3ie funded:

No

Study ID:

RIDIE-STUDY-ID-61e93959dcd42

Initial Registration Date:

01/20/2022

Last Update Date:

12/09/2021

Study Status:

In Development

Location(s):

India

Abstract:

The H&M Foundation is funding a 3-year Collective Impact (CI) initiative (or Saamuhika Shakti) in Bengaluru, India to improve the lives of waste pickers in the city. Waste pickers have traditionally been neglected and interventions aimed at improving their living conditions have been few and far between. The CI initiative seeks to address this issue holistically through its novel multi-sectoral approach that targets waste pickers’ systemic exclusion and restores their right to life with dignity.

This is an impact evaluation (IE) of the CI initiative which uses a mixed methods approach, combining quantitative evaluation with embedded process and qualitative evaluation. For the quantitative evaluation, we will use the triple difference framework to compare changes in the average outcomes over time between the waste picker and non-waste picker households across the treatment and control localities. The IE would ascertain if the initiative worked to improve the lives of waste pickers and their families, factors that facilitated or inhibited the impact, and what might be the areas to focus when scaling up the programme or replicating it in another context. It would also try to identify if (and how) the programme impacted subgroups differently.

Additionally, process evaluation will be used to unpack the programme theory of change to understand ‘how’ and ‘why’ the CI initiative led (or did not lead) to a change in the outcomes of interest. We will do so by exploring aspects such as appropriateness, feasibility, acceptability, and adoption and how this impinges upon programme fidelity or how the programme is delivered and how participants interact with the programme. It will also yield insights on how and why participants’ acceptance and adoption of the programme may vary by gender, socio-economic status, thus answering ‘for whom’ the programme worked.

 

Registration Citation:

Categories:

Education
Health, Nutrition, and Population
Multisector
Social Protection
Water and Sanitation

Additional Keywords:

Gender, financial inclusion

Secondary ID Number(s):

Not applicable

Principal Investigator(s)

Name of First PI:

Stuti Tripathi

Affiliation:

International Initiative for Impact Evaluation (3ie)

Name of Second PI:

Chandan Jain

Affiliation:

International Initiative for Impact Evaluation (3ie)

Study Sponsor

Name:

H&M Foundation

Study Sponsor Location:

Sweden

Research Partner

Name of Partner Institution:

Not applicable

Type of Organization:

Location:

India

Intervention

Intervention Overview

Intervention:

The H&M Foundation is funding the Collective Impact (CII) initiative in Bengaluru, India to improve the lives of waste pickers in the city. Waste pickers have traditionally been neglected and interventions aimed at improving their living conditions have been few and far between. The CII initiative seeks to address this issue holistically through its novel multi-sectoral approach that targets waste pickers’ systemic exclusion and restores their right to life with dignity. Under this initiative various organizations are working together to take an integrated approach to improve the living conditions of waste pickers.

The CI Initiative will focus on the following 10 intervention areas:

1. Development of alternate livelihood options for waste pickers and their family members who seek to leave the profession
2. Collectivisation of waste pickers and their family members
3. Improved social security and linkages to government schemes
4. Financial literacy and inclusion
5. Access to health camps
6. Improved access to quality education for waste pickers’ children
7. Affordable access to drinking water and sanitation facilities for waste pickers and their families
8. Increased awareness on domestic violence
9. Increased awareness on substance abuse
10. Sector perceptions change program to improve professional pride amongst waste pickers

Gender is an overarching theme that runs across most interventions under the CI Initiative and some of these interventions target gender perceptions, issues, and roles more explicitly than others. Advocacy is another cross-cutting theme - most partners plan to work with city and local government authorities to improve waste picker acces

Theory of Change:

The main thematic areas that would be considered under the theory of change (ToC) are:

1. Skills
2. Financial literacy and inclusion
3. Social security
4. Health
5. Water, sanitation and hygiene
6. Substance abuse
7. Pre-school and school education
8. Domestic violence
9. WP dignity and perception
10. Advocacy

Through various interventions targetting the above listed thematic areas, the CI initiative aims to improve quality of life for waste pickers in Bengaluru; the 5 key outcomes in the ToC are: (i) increase in HH income, (ii) improved physical health of HHs, (iii) increase in enrolment, attendance, retention of students in schools and Anganwadis, (iv) increased awareness and reporting of domestic violence cases and (v) and greater social acceptance of waste pickers and their work are interconnected and reinforce each other, which together leads to the final outcome of improved lives of waste picker HHs. For each of the partner's intervention we have drafted an intervention level theory of change and these individual ToCs have been cumulated to produce a meta ToC to understand and reflect the way in which different interventions affect related outcomes and ultimately lead to improved well being for the waste picker households. A diagrammatic representation of meta and intervention level theory of change can be viewed by clicking on the following this link.

Multiple Treatment Arms Evaluated?

No

Implementing Agency

Name of Organization:

BBC Media Action, CARE India, Hasiru Dala, Sambhav by LabourNet, Save the Children India, WaterAid India, Social Alpha

Type of Organization:

NGO (local)/Community Based Organization/Other civil society organization

Program Funder

Name of Organization:

H&M Foundation

Type of Organization:

NGO (International)

Intervention Timing

Intervention or Program Started at time of Registration?

Yes

Start Date:

01/01/2021

End Date:

12/31/2023

Evaluation Method

Evaluation Method Overview

Primary (or First) Evaluation Method:

Difference in difference/fixed effects

Other (not Listed) Method:

Additional Evaluation Method (If Any):

Other (specify)

Other (not Listed) Method:

Process Evaluation

Method Details

Details of Evaluation Approach:

Given operational and program level constraints it is not feasible to randomly allocate interventions in the present context. Therefore, a design based on Difference-in-Difference (DID) is being proposed to evaluate the causal impact of CII on different household and individual level outcomes for waste picker households in Bengaluru. In the present context, the unit of intervention is a neighbourhood or locality in Bengaluru, which may be either formal or informal settlement of households (including waste picker households). Our proposed evaluation design will exploit this variation in the number of partners across localities. Given that CII aims to understand the impact of multiple partner interventions, we compare localities that have received multiple interventions (of at least 2 partners) with those that have received interventions of either one or no partners. The proposed evaluation design will allow us to estimate the effects of being exposed to multiple programs versus the interventions single or no partners. Even though DID would allow us to tease out any time varying changes that are common across the treatment and control group, it would not be able to control for any differential time varying changes, other than the program interventions, across the two groups. Given this, the localities that receive multiple interventions might be intrinsically different in observable and unobservable characteristics from those that receive one or no intervention. We should be able to control for some of the observable characteristics using information gathered using a locality level survey. To control for unobservable differences, we propose an evaluation design based on the triple difference framework wherein we will sample for non-waste picker households from the same locality as waste picker households. Note that the non-waste picker households are not eligible to receive benefits from the CII interventions.

 

Outcomes (Endpoints):

The list of final and intermediate outcomes can be accessed using this link.

 

Unit of Analysis:

Household and individual

Hypotheses:

Key question - Evaluate the effect of exposure to interventions of two or more partners versus interventions of single or no partners on the waste picker population in these 100 localities on a range of outcomes including -

• Participation in skill training programs, labour force participation and income
• Health and child’s education
• Access to social security benefits, loans and financial linkages 
• Participation in collectives (such as self-help groups, common interest groups)
• Gender attitudes and women decision making
• Access to WASH facilities
• Dignity and perceptions

Unit of Intervention or Assignment:

Locality and household

Number of Clusters in Sample:

100 Localities

Number of Individuals in Sample:

4,400 households

Size of Treatment, Control, or Comparison Subsamples:

The exact size of the treatment and control localities will be decided at the endline depending upon the number of implementation partners operating across localities.

Supplementary Files

Analysis Plan:

Other Documents:

Data

Outcomes Data

Description:

We will collect data on household and individual level outcomes using a household level survey. All households will be surveyed twice, once at baseline (November 2021 - January 2022) and then at Endline (this will start tentatively in September 2023). Broadly, the survey contains two modules, a) a Household Module and b) a Household spouse Module. The household module will be administered to the head of the household, whereas the other module will be administered to the spouse.

Data Already Collected?

No

Data Previously Used?

Data Access:

Data Obtained by the Study Researchers?

Data Approval Process:

Approval Status:

Treatment Assignment Data

Participation or Assignment Information:

Yes

Description:

Data Obtained by the Study Researchers?

Data Previously Used?

Data Access:

Data Obtained by the Study Researchers?

Data Approval Process:

Approval Status:

Data Analysis

Data Analysis Status:

Study Materials

Upload Study Materials:

Registration Category

Registration Category:

Prospective, Category 1: Data for measuring impacts have not been collected

Completion

Completion Overview

Intervention Completion Date:

Data Collection Completion Date:

Unit of Analysis:

Clusters in Final Sample:

Total Observations in Final Sample:

Size of Treatment, Control, or Comparison Subsamples:

Findings

Preliminary Report:

Preliminary Report URL:

Summary of Findings:

Paper:

Paper Summary:

Paper Citation:

Data Availability

Data Availability (Primary Data):

Date of Data Availability:

Data URL or Contact:

Access procedure:

Other Materials

Survey:

Survey Instrument Links or Contact:

Program Files:

Program Files Links or Contact:

External Link:

External Link Description:

Description of Changes:

Study Stopped

Date:

Reason: