Download Study

General

Study Overview

Title:

Impact evaluation of the Adapting to Markets and Climate Change Project in Nicaragua (NICADAPTA)

Study is 3ie funded:

No

Study ID:

RIDIE-STUDY-ID-5fad88636c17c

Initial Registration Date:

11/12/2020

Last Update Date:

12/09/2021

Study Status:

Completed

Change History for Status

Changed On	Previous Value
05/26/2021	In Development

Location(s):

Nicaragua

Abstract:

This study evaluates the causal impact the Adapting to Markets and Climate Change Project (NICADAPTA) in Nicaragua. The target population of NICADAPTA are coffee and cacao cooperatives and their members. The goal of the project is to improve the living conditions of coffee and cacao farmers by increasing their competitiveness and reducing their vulnerability to climate change.

Agriculture, employing one third of the population, contributes 15.4% to the Nicaragua’s GDP. Coffee constitutes a central part of national agricultural output and 20 to 25 % of export revenues. The projected increases in temperatures and changes in rainfall adversely affects the suitability of land for coffee and cacao production and thereby threatens the livelihoods of smallholder farmers.

To improve the productivity and resilience or coffee and cacao farmers, NICADAPTA funds investment plans of coffee and cacao cooperatives, covering trainings of farmers and cooperative leaders, equipment and improved agricultural inputs. Furthermore, through the strengthening of national institutions, improved genetic material and education material is developed and relevant weather information is generated and distributed. 

The evaluation strategy is a mixed-methods approach, relying on non-experimental ex-post impact evaluation methods and qualitative data. For the precise estimation of the project’s impact, we will compare the change in productivity and resilience members of about 200 participating and not participating cooperatives over five years. This covers of all cooperatives who benefited from NICADAPTA. We will use propensity score matching with farmers of all benefiting cooperatives as robustness check.

Change History for Abstract

Changed On

Previous Value

03/09/2021

This study evaluates the causal impact the Adapting to Markets and Climate Change Project (NICADAPTA) in Nicaragua. The target population of NICADAPTA are coffee and cacao cooperatives and their members. The goal of the project is to improve the living conditions of coffee and cacao farmers by increasing their competitiveness and reducing their vulnerability to climate change. Agriculture, employing one third of the population, contributes 15.4% to the Nicaragua’s GDP. Coffee constitutes a central part of national agricultural output and 20 to 25 % of export revenues. The projected increases in temperatures and changes in rainfall adversely affects the suitability of land for coffee and cacao production and thereby threatens the livelihoods of smallholder farmers. To improve the productivity and resilience or coffee and cacao farmers, NICADAPTA funds investment plans of coffee and cacao cooperatives, covering trainings of farmers and cooperative leaders, equipment and improved agricultural inputs. Furthermore, through the strengthening of national institutions, improved genetic material and education material is developed and relevant weather information is generated and distributed.  The evaluation strategy is a mixed-methods approach, relying on non-experimental ex-post impact evaluation methods and qualitative data. For the precise estimation of the project’s impact, we will compare the change in productivity and resilience members of about 112 participating and not participating cooperatives over five years. This covers 30% of all cooperatives who benefited from NICADAPTA. We will use propensity score matching with farmers of all benefiting cooperatives as robustness check.

Registration Citation:

Categories:

Agriculture and Rural Development
Environment and Disaster Management

Additional Keywords:

Secondary ID Number(s):

Principal Investigator(s)

Name of First PI:

Esther Heesemann

Affiliation:

Center for Evaluation and Development

Name of Second PI:

Alessandra Garbero

Affiliation:

IFAD

Study Sponsor

Name:

International Fund for Agricultural Development

Study Sponsor Location:

Italy

Research Partner

Name of Partner Institution:

Centro Agronómico Tropical de Investigación y Enseñanza - CATIE

Change History for Name of Partner Institution

Changed On	Previous Value
12/09/2021	Centro Agronómico Tropical de Investigación y Enseñanza - CATIE; Universidad Nacional Agraria UNA

Type of Organization:

Research institute/University

Location:

Nicaragua

Intervention

Intervention Overview

Intervention:

The impact evaluation will focus on the first component of NICADAPTA; namely Sustainable Development of Coffee and Cacao Productivity.  The strategy of this component is the financing of investment plans (Planes de Inversión, in Spanish, hereinafter referred to as PI), submitted by coffee and cacao producer groups (cooperatives, women and youth groups) or local/regional governments. NICADAPTA finances public and private goods for collective use. Subject of full funding are public goods such as small rural roads, as well as technical assistance. Private good for collective use, for example storage facilities, are financed through matching grants. While private good for private use are generally excluded from funding, an exception is made for investments related to climate change of this kind. The investment plan need to lay out how they will enhance to productivity of the cooperative and cooperative members, by adapting the agricultural practices to the challenges of climate change and improve market access. Most PIs focus on the promotion and management of cultivations and post-harvest processes, reforestation, disease management, cacao agroforestry, marketing and capacity building of the producers and organizations. Component 1 is complemented by strengthening of national institutions in the field of coffee and cacao agriculture and climate information (component 2), with the aim to increase the supply of improved, climate-change adapted seed varieties and the developed an early warning system for weather shocks and diseases.      

Private Intervention Details:

The funding opportunity was promoted through the MEFCAA Webpage, promotion material, producer group meetings, seminars and workshops and communication means. The entities applying for the grant are supported by technical service providers in their application. The applications are reviewed and rated by the project management units. Plans that reach 70% of the points or more in total and no less than 50% in each category qualify for funding. Throughout the years, only nine applications had been rejected.  By June 2020, 83 applications for productivity enhancing PIs have been accepted by the project. 18 of the 83 plans have been directed to women and youth who were members or associated to producer organizations, nine were directed to subnational governments (populated by Indigenous and Afro-descendant communities). Each productivity enhancing PI covers one or several organizations (cooperatives can come together and present one PI jointly), such that the total number of individual organizations in NICADAPTA as of June 2020 amounts to 149, plus three regional governments.

Theory of Change:

NICADAPTA aimed to improve the quality of life of rural families by increasing their income in a sustainable way from the added value in coffee and cacao production. To reach this goal, small producers shall be better integrated in the markets and better equipped against the consequences of climate change.  

Component 1 adds to the overall project objective by improving the competitiveness of producer organizations. It works through the equipment and capacity building of producer organizations and their members, as well as the local governments’ investment in infrastructure. Better equipped and trained producer organization can help to channel essential knowledge on new agricultural techniques and seed variation to farms, and thereby improve their adaptive capacity. Investments in the organizational infrastructure can improve the post-harvest production of coffee and cacao beans and yield to higher quality of the produce and less post-harvest loss. The improved roads will facilitate the transport of coffee and cacao from the farmers to the cooperatives.

Via Component 2, enhanced and improved services for agricultural technology, organization management, commercialization of products and climate information relevant for the production of coffee and cacao will be offered to the public. The improved supply of information and educational material, as well as genetic products shall give cooperatives and smallholder farmers the intellectual and material inputs to adapt their agricultural practices to the change in temperatures and precipitation. Specific whether information, relevant to the livelihoods of coffee and cacao farmers, will help the rural families to react in timely manner to adverse whether events and to be hence more resistant to shocks.

Multiple Treatment Arms Evaluated?

No

Implementing Agency

Name of Organization:

MEFCCA (Ministerio de la Economía Familiar, Comunitaria, Cooperativa y Asociativa)

Type of Organization:

Public Sector, e.g. Government Agency or Ministry

Program Funder

Name of Organization:

IFAD; C4ED

Type of Organization:

NGO (International)

Intervention Timing

Intervention or Program Started at time of Registration?

Yes

Start Date:

07/01/2014

End Date:

07/01/2020

Evaluation Method

Evaluation Method Overview

Primary (or First) Evaluation Method:

Matching

Change History for Primary (or First) Evaluation Method

Changed On	Previous Value
12/09/2021	Difference in difference/fixed effects

Other (not Listed) Method:

Additional Evaluation Method (If Any):

Difference in difference/fixed effects

Change History for Additional Evaluation Method (If Any)

Changed On	Previous Value
12/09/2021	Matching

Other (not Listed) Method:

Method Details

Details of Evaluation Approach:

The proposed research design follows a concurrent mixed-methods approach, building on both quantitative and qualitative primary data collection and analysis.

In light of the non-random allocation of funds to producer groups, we use the non-experimental ex-post inverse probability weighted regression adjusted regression approach (IPWRA). We check the robustness of our findings with inverse probability weighted Difference-in-Difference (DiD) estimations, which is possible thanks to the comprehensive baseline farmer survey in 2014, covering farmers of 56 of the NICADAPTA cooperatives and 100 non-NICADAPTA cooperatives.

The results of the qualitative study will be integrated with the findings from the quantitative data analysis to inform on the intangible and unintended impacts of the project, and hone in on how the project has affected women and youth groups, as well as indigenous and Afro-descendant communities. Additionally, qualitative interviews with stakeholders will be used to derive lessons learned about project design and implementation process.  

Change History for Details of Evaluation Approach

Changed On

Previous Value

12/09/2021

The proposed research design follows a concurrent mixed-methods approach, building on both quantitative and qualitative primary data collection and analysis. In light of non-random allocation of funds to producer groups, we considered only non-experimental designs for the impact evaluation. The impact assessment will be done using non-experimental, quantitative methods. Thanks to the comprehensive baseline farmer survey, covering farmers of 30% of the NICADAPTA participating cooperatives, we suggest a panel Difference-in-Difference (DiD) with matching design as principal impact evaluation method. As this excludes a large share of the beneficiary population, namely those cooperatives that were not covered by the baseline, we will complement our analysis with an ex-post propensity score matching design. With this strategy, we will combine the advantages for the internal validity from a more rigorous IA method in restricted sample (Panel DiD), with the advantages for external validity of a more assumption-heavy method (ex-post matching) in the full sample of NICADAPTA beneficiaries.   The results of the qualitative study will be integrated with the findings from the quantitative data analysis to inform on the intangible and unintended impacts of the project, and hone in on how the project has affected women and youth groups, as well as indigenous and Afro-descendant communities. Additionally, qualitative interviews with stakeholders will be used to derive lessons learned about project design and implementation process.

Private Details of Evaluation Approach:

Outcomes (Endpoints):

Final outcomes:

• Primary
  • Resilience to climate change
  • Income from agriculture
  • Coffee and cacao production
• Secondary
  • Household food security index

Intermediate outcomes

• Primary:
  • Knowledge on climate smart agricultural techniques
  • Access to climate information
  • Access to markets
• Secondary:
  • Application of climate smart agricultural techniques (Index of techniques applied, crop diversification, climate resilients seeds)

Change History for Outcomes (Endpoints)

Changed On

Previous Value

03/09/2021

Final outcomes: Primary Resilience to climate change Income from agriculture Coffee and cacao production Secondary Household food security index Intermediate outcomes Primary: Knowledge on climate smart agricultural techniques Access to climate information Access to markets Secondary: Application of climate smart agricultural techniques (Index of techniques applied, crop diversivication, climate resilients seeds)

Unit of Analysis:

Household

Hypotheses:

Intermediate outcomes

Primary hypotheses:

h1.1 Farmers, associated to cooperatives who obtained funding from NICADAPTA ("NICADAPTA farmers"), will be more knowledgable about climate-smart agricultural practises

H1.2 NICADAPTA farmers will be better informed about whether events related to agricultural production.

H1.3  NICADAPTA farmers will be better connect to markets

Secondary hyptheses:

H2 NICADAPTA farmers will apply more climate smart agricultural techniques on their land

Final outcomes:

Primary hypotheses:

H3.1 Households of NICADAPTA farmers will increase their production of coffee and/or cacao.

H3.2 Households of NICADAPTA farmers will have higher incomes from agricultural production.

H3.3 Households of NICADAPTA farmers will be more resilient to shocks, in particular to the consequences of climate change

Secondary hyptheses:

H4 Households of NICADAPTA farmers will be more food secure.

Unit of Intervention or Assignment:

Cooperative

Number of Clusters in Sample:

112 in the panel DID analysis, 200 in the matching analysis (max.) (Numbers might change after verification of the treatment status)

Change History for Number of Clusters in Sample

Changed On	Previous Value
03/09/2021	112 in the panel DID analysis, 249 in the matching analysis (max.) (Numbers might change after verification of the treatment status)

Number of Individuals in Sample:

1,120 in the panel DID analysis, 2,100 in the matching analysis (max.) (Numbers might change after verification of the treatment status)

Change History for Number of Individuals in Sample

Changed On	Previous Value
03/09/2021	1,120 in the panel DID analysis, 2,490 in the matching analysis (max.) (Numbers might change after verification of the treatment status)

Size of Treatment, Control, or Comparison Subsamples:

DID analysis: 56 treatment and 100 control cluster with on average 560 households each. Matching analysis: 100 treatment cluster and 100 control cluster with on average 1000 and 1000 households each

Change History for Size of Treatment, Control, or Comparison Subsamples

Changed On	Previous Value
03/09/2021	DID analysis: 56 treatment and 56 control cluster with on average 560 households each. Matching analysis: 149 treatment cluster and 100 control cluster with on average 1490 and 100 households each

Supplementary Files

Analysis Plan:

Other Documents:

Data

Outcomes Data

Description:

At baseline: farmer survey ( 1744 farmers from 156 cooperatives). At endline: farmer survey (max. 1,900 farmers of 190 cooperatives), cooperative survey (94 cooperatives)

Change History for Description

Changed On	Previous Value
05/26/2021	At baseline: farmer survey ( 1744 farmers from 156 cooperatives). At endline: farmer survey (max. 2100 farmers of 2000 cooperatives), cooperative survey (94 cooperatives)
03/09/2021	At baseline: farmer survey ( 1744 farmers from 156 cooperatives). At endline: farmer survey (max. 2490 farmers of 249 cooperatives), cooperative survey (249 cooperatives)

Data Already Collected?

No

Data Previously Used?

Data Access:

Data Obtained by the Study Researchers?

Data Approval Process:

Approval Status:

Treatment Assignment Data

Participation or Assignment Information:

No

Description:

We will use monitoring data from the implementing agency to define if a cooperative and its associated farmers have received the treatment.

Data Obtained by the Study Researchers?

Yes

Data Previously Used?

No

Data Access:

Restricted -- Access requires a formal approval process

Data Obtained by the Study Researchers?

Data Approval Process:

The data was obtained from the implementing agency.

Approval Status:

Yes-obtained approval and have received the data

Data Analysis

Data Analysis Status:

Study Materials

Upload Study Materials:

Registration Category

Registration Category:

Prospective, Category 1: Data for measuring impacts have not been collected

Completion

Completion Overview

Intervention Completion Date:

12/31/2020

Data Collection Completion Date:

05/02/2021

Unit of Analysis:

Households

Clusters in Final Sample:

196 Cooperatives for the matching analysis; 155 for the DID analysis

Total Observations in Final Sample:

1,901 for the matching analysis, 1,342 for the DID analysis

Size of Treatment, Control, or Comparison Subsamples:

910 obs in the control group and 991 in the treatment group for the matching analysis, 773 obs in the control group and 569 in the treatment group for the DID analysis

Findings

Preliminary Report:

Yes

Change History for Preliminary Report

Changed On	Previous Value
12/09/2021	No

Preliminary Report URL:

Summary of Findings:

Our results show that NICADAPTA was successful in improving the access to post-harvest and water infrastructure and increased the participation in agricultural trainings. Yet we do not find evidence for a change in agricultural techniques, higher yields or income from agriculture. Instead, we find weak evidence for a small decrease in the practice of using crop residuals as cover drops on the fields and planting shade trees. At the same time, there is a noticeable increase in agricultural inputs, driven by the spending on organic fertilizer. Market access has not improved of farmers of NICADAPTA cooperatives. Nevertheless, our findings show that farmers are slightly more able to recover from weather shocks, experience less food insecurity and have more productive assets.

Even though NICADAPTA interventions have not yet yielded tangible improvements to producers’ quality of life, the qualitative study results suggest that beneficiaries have experienced benefits in terms of increased motivation to improve their circumstances; there is anecdotal evidence as to new knowledge and techniques acquired to take better care of their coffee and cocoa plants; there is evidence of acquisition of infrastructure and other inputs to help them improve their production processes; and there are additional enhancements such as improved emotional health and motivation of women participants as a result of gender trainings and specific support delivered to women and youth PIs.

Change History for Summary of Findings

Changed On	Previous Value
12/09/2021	Not yet available

Paper:

No

Paper Summary:

Paper Citation:

Data Availability

Data Availability (Primary Data):

No--Data not expected to be available

Date of Data Availability:

Data URL or Contact:

Access procedure:

Other Materials

Survey:

No

Survey Instrument Links or Contact:

Program Files:

No

Program Files Links or Contact:

n/a

External Link:

External Link Description:

Description of Changes:

Study Stopped

Date:

Reason: