Download Study

General

Study Overview

Title:

The contribution of legal awareness raising toward access to justice and stability in Afghanistan

Study is 3ie funded:

No

Study ID:

RIDIE-STUDY-ID-53750faee20a1

Initial Registration Date:

05/15/2014

Last Update Date:

01/26/2015

Study Status:

Completed

Location(s):

Afghanistan

Abstract:

Over 30 years of war has left Afghanistan’s informal and formal justice institutions weakened, limiting access to equitable justice and effective dispute resolution. The Rule of Law Stabilization - Informal Component (RLS-I) addresses the primary objectives of (1) strengthening Traditional Dispute Resolution (TDR) mechanisms, including women’s roles in TDR as disputants, witnesses, and decision makers, (2) enhancing linkages between the formal and informal justice, and (3) facilitating the resolution of longstanding and destabilizing disputes. The RLS-I development hypothesis is that capacity-building of informal justice providers, combined with networking opportunities to share experience and build solidarity around improved TDR practices, strengthens stability through increased access to justice and citizen confidence in TDR mechanisms. This hypothesis is tested through a difference-in-differences (d-i-d) evaluation design that examines treatment and control groups both before and after the program.

Registration Citation:

Killian, D. and Agee, J., The contribution of legal awareness raising toward access to justice and stability in Afghanistan. Registry for International Development for Impact Evaluations (RIDIE). Available at: 10.23846/ridie029

Categories:

Education
Social Protection

Additional Keywords:

Access to justice, Stability

Secondary ID Number(s):

AID-306-C-12-00013 (USAID)

Principal Investigator(s)

Name of First PI:

Dan Killian

Affiliation:

Checchi and Company Consulting

Name of Second PI:

James Agee

Affiliation:

Checchi and Company Consulting

Study Sponsor

Name:

USAID

Study Sponsor Location:

Afghanistan

Research Partner

Name of Partner Institution:

Strategic Social

Type of Organization:

Private firm

Location:

Afghanistan

Intervention

Intervention Overview

Intervention:

RLS-I’s program instructs participants on a variety of legal topics relevant to dispute prevention and conflict resolution through six legal education workshops courses conducted over the course of 4-6 months, for a total instruction time of approximately 30 hours. The program reaches approximately 125 male and female participants per district, with a target of at least 75 male elders attending all six courses and 40 females attending four courses. This target group encompasses a natural group of 30-40 district-level dispute resolvers, as well as other elders who would help resolve village or family level disputes. The program also provides a series of discussion sessions on pressing issues, coordination meetings with state justice actors, and plenary networking meetings to share lessons learned and encourage group action. RLS-I's intervention culminates with a handover network meeting for each district cohort, allowing participants to reflect on their participation, pledge not to follow cultural practices identified as harmful to their communities, and commit to advancing RLS-I objectives independently.

Private Intervention Details:

Theory of Change:

Multiple Treatment Arms Evaluated?

Yes

Implementing Agency

Name of Organization:

Checchi and Company Consulting

Type of Organization:

Private for profit organization

Program Funder

Name of Organization:

USAID

Type of Organization:

Public Sector, e.g. Government Agency or Ministry

Intervention Timing

Intervention or Program Started at time of Registration?

Yes

Start Date:

10/01/2012

End Date:

01/13/2014

Evaluation Method

Evaluation Method Overview

Primary (or First) Evaluation Method:

Difference in difference/fixed effects

Other (not Listed) Method:

Additional Evaluation Method (If Any):

Other (not Listed) Method:

Method Details

Details of Evaluation Approach:

A cross-section of elders and disputants in both program and non-program districts are surveyed at program inception and again at conclusion. Impact is then defined as the difference in mean scores on various measures from baseline to endline, and between the program group and non-program group. To the extent feasible, the same respondents from baseline are sought out at endline, thus allowing both group comparisons through a repeated cross-section design, and individual comparisons through a panel data set made up of the same respondents.

Private Details of Evaluation Approach:

Assumptions underlying this theory of change include the following: a) workshop content effectively imparts knowledge, b) participants are willing and able to change their attitudes and practices that may conflict with Afghan statutory law and Shari’ah, c) participants will be able to use their new knowledge effectively in context, upon returning to their communities, d) participation will generate a critical mass of elders in a given community sufficient to effect change in adjudication reflective of Afghan statutory law, Shari’ah and human rights norms, e) improper influence and interference with informal dispute resolution by local power brokers will gradually lessen as a result of security and governance gains, f) threats from insurgent or anti-government actors fail to deter program participation, and g) the programming environment is stable enough to enable social change.

Outcomes (Endpoints):

- Percent of TDR decisions recorded / registered with government entity (identified in self-reported survey data as well as auditing decision book records and district government files) - Percentage of elders reporting positive change in adjudication practices compared to previous year (survey pre and post) - Knowledge gain from baseline to endline, and across program and non-program groups (Constitutional law, criminal law, family law, inheritance, property / deeds law) - Attitudinal change from baseline to endline, and across program and non-program groups (harmful social practices, role of women in TDR) - Disputant perception of external influence over their dispute adjudication - Gains in disputant perceptual indices on the process and outcome of their dispute adjudication (procedural justice, corruption, and justice of outcome)

Unit of Analysis:

Results are analyed at the level of elders (knowledge and attitude) and by individual TDR event (disputant perception of the adjudication practices of the elders who helped resolve their dispute)

Hypotheses:

The evaluation is organized around four key hypotheses: 1) The intervention leads to TDR decisions that better reflect Afghan law, Shari’ah, and human rights norms, 2) The intervention leads to TDR decisions and local adjudicators being perceived as more impartial, 3) The intervention will result in a decrease in the number of TDR decisions that negatively impact women and children, and 4) The intervention will result in an increased role for women in TDR processes as disputants, witnesses or decision-makers. In addition to the primary hypotheses, there are several secondary research questions of interest, such as the following: 1) What is the requisite exposure to RLS-I activities to effect behavior change? 2) What is the time frame governing any treatment effect, and for how long does it persist? 3) What is the requisite number of participants in a community to effect a change in dispute adjudication and outcomes in the community as a whole? 4) Do RLS-I activities for women provide an indirect means of affecting dispute prevention, adjudication, and outcomes? 5) Is the distinction between real and imposed adjudicators a meaningful one in the context of the intervention?

Unit of Intervention or Assignment:

Selection of program and non-program groups is first by district, and then by individuals within district

Number of Clusters in Sample:

The evaluation sampled 6 districts (3 treatment, 3 comparison), with each district sampling from approximately 35 villages. The average number of villages in a district is approximately 50.

Number of Individuals in Sample:

~ 500 elders and 600 disputants for each data collection wave, or 70 elders and 90 disputants per district . For mass outreach, ~ 1,800 respondents for each data collection wave (300 per district).

Size of Treatment, Control, or Comparison Subsamples:

Treatment and comparison sub-groups are approximately evenly balanced across the total sample size

Supplementary Files

Analysis Plan:

RLS-I Phase 3 Evaluation Inception Report (26 March 2013).pdf

Other Documents:

: FINAL RLS-I Phase 3 Baseline Evaluation Report.pdf

Data

Outcomes Data

Description:

Elder survey, disputant survey, citizen survey

Data Already Collected?

Yes

Data Previously Used?

No

Data Access:

Not restricted - access with no requirements or minimal requirements (e.g. web registration)

Data Obtained by the Study Researchers?

Yes

Data Approval Process:

Approval Status:

Treatment Assignment Data

Participation or Assignment Information:

Yes

Description:

Data Obtained by the Study Researchers?

Data Previously Used?

Data Access:

Data Obtained by the Study Researchers?

Data Approval Process:

Approval Status:

Data Analysis

Data Analysis Status:

Yes

Study Materials

Upload Study Materials:

: RLS-I elder interview (endline 5 Aug 2013).pdf
: RLS-I disputant case assessment (endline 5 Aug 2013).pdf
RLS-I Impact Evaluation Final Report: RLS-I Phase 3 FINAL Evaluation (Impact Assessment) Report_16 May 2014.pdf
RLS-I Phase 3 Evaluation Inception Report: RLS-I Phase 3 Evaluation Inception Report (26 March 2013).pdf
RLS-I Phase 3 Baseline Evaluation Report: FINAL RLS-I Phase 3 Baseline Evaluation Report.pdf
RLS-I Phase 2 Evaluation Report: RLS-I Phase 2 impact evaluation report (29 Aug 2012).pdf
RLS-I evaluation brief: RLS-I evaluation brief.pdf

Change History for Upload Study Materials

Changed On	Previous Value
01/26/2015	Description: Filename: RLS-I elder interview (endline 5 Aug 2013).pdf Link: http:// Description: Filename: RLS-I disputant case assessment (endline 5 Aug 2013).pdf Link: http:// Description: RLS-I Phase 3 Evaluation Inception Report Filename: RLS-I Phase 3 Evaluation Inception Report (26 March 2013).pdf Link: http:// Description: RLS-I Phase 3 Baseline Evaluation Report Filename: FINAL RLS-I Phase 3 Baseline Evaluation Report.pdf Link: http:// Description: RLS-I Phase 2 Evaluation Report Filename: RLS-I Phase 2 impact evaluation report (29 Aug 2012).pdf Link: http:// Description: RLS-I evaluation brief Filename: RLS-I evaluation brief.pdf Link: http://
01/26/2015	Description: Filename: RLS-I elder interview (endline 5 Aug 2013).pdf Link: http:// Description: Filename: RLS-I disputant case assessment (endline 5 Aug 2013).pdf Link: http:// Description: Final report - The contribution of legal awareness raising to access to justice and stability in Afghanistan Filename: RLS-I Phase 3 Final Evaluation Report (11 April 2014).pdf Link: http:// Description: RLS-I Phase 3 Evaluation Inception Report Filename: RLS-I Phase 3 Evaluation Inception Report (26 March 2013).pdf Link: http:// Description: RLS-I Phase 3 Baseline Evaluation Report Filename: FINAL RLS-I Phase 3 Baseline Evaluation Report.pdf Link: http:// Description: RLS-I Phase 2 Evaluation Report Filename: RLS-I Phase 2 impact evaluation report (29 Aug 2012).pdf Link: http:// Description: RLS-I evaluation brief Filename: RLS-I evaluation brief.pdf Link: http://
01/26/2015	Description: Filename: RLS-I elder interview (endline 5 Aug 2013).pdf Link: http:// Description: Filename: RLS-I disputant case assessment (endline 5 Aug 2013).pdf Link: http:// Description: Final report - The contribution of legal awareness raising to access to justice and stability in Afghanistan Filename: RLS-I Phase 3 Final Evaluation Report (11 April 2014).pdf Link: http:// Description: RLS-I Phase 3 Evaluation Inception Report Filename: RLS-I Phase 3 Evaluation Inception Report (26 March 2013).pdf Link: http:// Description: RLS-I Phase 3 Baseline Evaluation Report Filename: FINAL RLS-I Phase 3 Baseline Evaluation Report.pdf Link: http:// Description: RLS-I Phase 2 Evaluation Report Filename: RLS-I Phase 2 impact evaluation report (29 Aug 2012).pdf Link: http://
01/26/2015	Description: Filename: RLS-I elder interview (endline 5 Aug 2013).pdf Link: http:// Description: Filename: RLS-I disputant case assessment (endline 5 Aug 2013).pdf Link: http:// Description: Final report - The contribution of legal awareness raising to access to justice and stability in Afghanistan Filename: RLS-I Phase 3 Final Evaluation Report (11 April 2014).pdf Link: http:// Description: RLS-I Phase 3 Evaluation Inception Report Filename: RLS-I Phase 3 Evaluation Inception Report (26 March 2013).pdf Link: http:// Description: RLS-I Phase 3 Baseline Evaluation Report Filename: FINAL RLS-I Phase 3 Baseline Evaluation Report.pdf Link: http:// Description: RLS-I Phase 2 Evaluation Report Filename: RLS-I Phase 2 impact evaluation report (29 Aug 2012).pdf Link: http:// Description: RLS-I Evaluation Brief Filename: RLS-I evaluation brief.pdf Link: http://
06/12/2014	Description: Filename: RLS-I elder interview (endline 5 Aug 2013).pdf Link: http:// Description: Filename: RLS-I disputant case assessment (endline 5 Aug 2013).pdf Link: http:// Description: Final report - The contribution of legal awareness raising to access to justice and stability in Afghanistan Filename: RLS-I Phase 3 Final Evaluation Report (11 April 2014).pdf Link: http:// Description: RLS-I Phase 3 Evaluation Inception Report Filename: RLS-I Phase 3 Evaluation Inception Report (26 March 2013).pdf Link: http:// Description: RLS-I Phase 3 Baseline Evaluation Report Filename: FINAL RLS-I Phase 3 Baseline Evaluation Report.pdf Link: http:// Description: RLS-I Phase 2 Evaluation Report Filename: RLS-I Phase 2 impact evaluation report (29 Aug 2012).pdf Link: http://

Registration Category

Registration Category:

Non-Prospective, Category 4: Data for measuring impacts have been obtained/collected by the research team and analysis for this evaluation has started

Completion

Completion Overview

Intervention Completion Date:

03/13/2014

Data Collection Completion Date:

09/30/2013

Unit of Analysis:

Elder knowledge and attitude, disputant perception of their dispute adjudication process and outcome, household perception of various TDR measures

Clusters in Final Sample:

None

Total Observations in Final Sample:

850 elders, 996 disputants, 3820 citizens

Size of Treatment, Control, or Comparison Subsamples:

Elders: 425 treatment, 425 comparison Disputants: 472 treatment, 524 comparison Citizens: 1918 treatment, 1902 comparison

Findings

Preliminary Report:

No

Preliminary Report URL:

Summary of Findings:

- RLS-I elders gained an average of 12% in legal knowledge, with heterogeneous outcomes across different types of knowledge. Elders showed strong gains in practical and relevant knowledge in family and inheritance, but showed mild gains or even declines in knowledge such as constitutional rights they may not see in their communities or practiced by their district government. - Afghans seeking the mediation of RLS-I elders are 11% more satisfied with the procedural fairness and 8% more satisfied with overall justice of the outcome. - Citizens were 7% more likely to support alternatives to the practice of child marriages, and 4.6% more likely to affirm that giving away girls in marriage was not an effective solution to a dispute. -Households in RLS-I districts reported a 17% increase in women taking their disputes directly to the decision making body. Households were also 19% more likely to support the idea of women serving as dispute resolvers, and 24% more likely to support the idea of female dispute resolvers. - Female disputants report negative assessments of procedural fairness and justice of the outcome even as males report positive assessments on the same measures.

Paper:

No

Paper Summary:

Paper Citation:

Data Availability

Data Availability (Primary Data):

Yes--Available now

Date of Data Availability:

Data URL or Contact:

Dan Killian, dkillian@me.com

Access procedure:

Other Materials

Survey:

Yes

Survey Instrument Links or Contact:

See documents in Study Materials section of registry

Change History for Survey Instrument Links or Contact

Changed On	Previous Value
06/12/2014	See related program documents in registry

Program Files:

Yes

Program Files Links or Contact:

Dan Killian, dkillian@me.com

External Link:

External Link Description:

Description of Changes:

Study Stopped

Date:

Reason: