Download Study

General

Study Overview

Title:

Evaluating the Effectiveness of Sport as a Pedagogical Tool: A Year Two Impact Evaluation of the Monrovia Football Academy

Study is 3ie funded:

No

Study ID:

RIDIE-STUDY-ID-5b55455f20bcb

Initial Registration Date:

07/22/2018

Last Update Date:

07/20/2018

Study Status:

Ongoing

Location(s):

Liberia
United States

Abstract:

The purpose of our study is to evaluate the effectiveness of the Monrovia Football Academy (MFA) and their hybrid educational model that pairs football training with academic education. Tailored to Liberian students from Grades 3-6, the program aims to use sport as a means of incentivizing youths to apply themselves to their coursework. Administrators have made a concerted effort to construct a student body that is ethnically and religiously diverse, all while placing emphasis on having classrooms that are equally male and female. Outside of the academy, the program promises to use football as a means of tackling social issues ranging from sexism to xenophobia, positioning team sports as a conduit for fostering empathy. Utilizing a multidimensional approach, we aim to assess how MFA’s model and curriculum spur student growth by examining five key areas: academic ability, individual resilience, inter-ethnic tolerance, perceptions on gender equality, and non-cognitive skills. Building on the foundations of last year’s preliminary study, we aim to further examine evidence supporting the center’s influence on promoting positive socio-educational outcomes among youth.

Registration Citation:

Spitler, A. and Roessler, P., 2018. Evaluating the Effectiveness of Sport as a Pedagogical Tool: A Year Two Impact Evaluation of the Monrovia Football Academy. Registry for International Development for Impact Evaluations (RIDIE). Available at: 10.23846/ridie150

Categories:

Education

Additional Keywords:

sport for development, international development, pedagogy, capacity building, Liberia

Secondary ID Number(s):

N/A

Principal Investigator(s)

Name of First PI:

Aaron Spitler

Affiliation:

College of William & Mary

Name of Second PI:

Phil Roessler

Affiliation:

College of William & Mary

Study Sponsor

Name:

The Institute for the Theory and Practice of International Relations (ITPIR)

Study Sponsor Location:

United States

Research Partner

Name of Partner Institution:

Type of Organization:

Location:

Intervention

Intervention Overview

Intervention:

The Monrovia Football Academy (MFA) was founded in 2014 with the mission of empowering Liberia’s future leaders through a combination of academic and leadership training. With the primary goal of boosting the testing scores of Liberian youth (Grades 3-7), the program has expanded its offerings to tackle a variety of socio-educational challenges that many students face. In addition to promising a quality education and intensive professional football coaching, MFA also maintains that their program deconstructs gender barriers that allow female participants to gain a sense of empowerment. Moreover, their curriculum includes a “life skills” component that equips students with the attributes necessary to succeed in a demanding social environment. All things considered, MFA’s holistic approach to education outlines increased educational attainment, enhanced football prowess, and improved interpersonal attitudes as the chief benefits for those who enroll in its academy.

Theory of Change:

The crux of MFA's educational strategy is using sport as a means for spurring intellectual development, arguing that structured time for football, coupled with intensive training and access to quality facilities, can act as an "incentive" for children to commit to their classroom work. A 2011 piece published by the Education Innovation Laboratory at Harvard University supports this claim, arguing that these activity rewards can "improve achievement at relatively low costs" (Fryer and Allan, 2011). By positioning football, Liberia's most popular sport, as the goal for students, MFA catalyzes a competition in the classroom that should be evident in testing scores. Outside of academics, the MFA utilizes sport in order to tackle issues related to gender prejudice, a sensitive topic that is pervasive throughout the Liberian educational sector. The organization provides female students with the opportunity to play organized football, a reality many are inherently deprived of, and places them on the same pitch as their male peers. The group sees sport as a mechanism for not only enhancing female self-reliance, but also leveling the metaphorical playing field that has been off limits to many young women. Lastly, MFA cites how football, being an exemplar team sport, plays a role in bridging divides between ethnic rivals. Following war and a protracted rebuilding process, heightened feelings of suspicion between tribes has hampered the development of national pride. As one part of a larger solution to this friction, MFA offers sport as a mechanism to encourage the development of organic relationships between youths of different backgrounds while leveraging the group activity to catalyze conversations across divides. The larger goal remains that participation in sport will prompt emergent leaders to recognize the importance of amicably resolving disputes and call upon that knowledge when entering the public workforce.

Multiple Treatment Arms Evaluated?

No

Implementing Agency

Name of Organization:

Monrovia Football Academy (MFA)

Type of Organization:

NGO (International)

Intervention Timing

Intervention or Program Started at time of Registration?

Yes

Start Date:

03/01/2015

End Date:

Evaluation Method

Evaluation Method Overview

Primary (or First) Evaluation Method:

Regression discontinuity

Other (not Listed) Method:

Additional Evaluation Method (If Any):

Other (not Listed) Method:

Method Details

Details of Evaluation Approach:

As a caveat, it must be clear that admission into MFA (i.e. the "treatment") is an incredibly difficult process that sees multitudes of applicants every year. Acceptance is contingent upon written examinations, football skill, and a personal interview. The most competitive of potential students are invited to a week-long camp in which coaches and faculty assess applicants in-depth. Given how selection into MFA is not a randomized process, we utilize a regression discontinuity approach to mitigate the effects of attendance. In 2016, 23 students were admitted into MFA, while around 51 individuals with similar characteristics missed the cut due to academic, logistical, or behavioral reasons. In 2017, 22 students were accepted, while 48 students fall into the latter category. Students selected to join MFA form our treatment group, while those who almost join are included in the control group. Additionally, our evaluation compares admitted students to their "runner-up" peers as well as to students from Bravid Washington Harris Episcopal School (B.W. Harris), a notable local school that produces quality students.

Outcomes (Endpoints):

By looking to the academic assessment and additional exercises, we aim to test whether MFA's theoretical robust curriculum helps promote positive pro-social outcomes when placed into practice. Breaking things down by individual module, we expect that MFA children will perform better on the academic assessment and earn higher scores when compared to non-MFA students. This has been identified as a primary outcome by our research team, for it most directly relates to MFA's mission of advancing robust education. Moreover, we expect that MFA student's scores grew more over the past year compared to the control group. The second primary outcome we hope to "tease out" relates to gender equality, as we hypothesize that MFA students will describe males and females as equal more so than their non-MFA peers. This is of particular importance, as MFA highlights gender parity as a fundamental part of its operation. Finally, as a secondary outcome, we expect that MFA students will exhibit greater tolerance for peers of different tribes and view them as equal stakeholders in the process of rebuilding the country.

Unit of Analysis:

Individual Performance

Hypotheses:

The primary hypothesis is as follows: MFA students perform better in the classroom, with a particular emphasis on test scores, than their non-MFA peers. Our goal is to examine whether the academy's hybrid educational model enhances outcomes in the classroom, charting students' growth over an extended period of time. For the gender component, one of our secondary hypotheses speculates that MFA students will view males and females as equals more often than non-MFA youth. Shifting to the ethnicity component, we also argue that MFA students demonstrate more tolerant and accepting attitudes towards peers of differing tribal backgrounds than those who are not enrolled within the program. Finally, focusing on our self-efficacy and resilience modules, we posit that MFA students would be better equipped to access the resources they need to sustain a positive livelihood when compared to their non-MFA counterparts. In other words, MFA children will not only have the self-awareness necessary to properly understand their needs, but also have the wherewithal to act upon their instincts.

Unit of Intervention or Assignment:

Individuals (Treatment and Control Groups)

Number of Clusters in Sample:

Number of Individuals in Sample:

Between 170-185 participants

Size of Treatment, Control, or Comparison Subsamples:

45 individuals in the treatment group, 99 in our control group, 36 from secondary control (B.W. Harris) - all figures projected

Supplementary Files

Analysis Plan:

Data

Outcomes Data

Description:

We will utilize a variety of surveys to produce data that will shed light on the efficacy and impact of MFA's curriculum. There will be four components, each covering a respective issue, that mainly follow a classic Likert scale format. The four sections include Resilience, Self-Efficacy and Leadership, Gender Equality, and Ethnicity and Nationality. By looking at their responses and coding them based on their alignment with out hypotheses

Data Already Collected?

No

Data Previously Used?

Data Access:

Data Obtained by the Study Researchers?

Data Approval Process:

Approval Status:

Treatment Assignment Data

Participation or Assignment Information:

Yes

Description:

Data Obtained by the Study Researchers?

Data Previously Used?

Data Access:

Data Obtained by the Study Researchers?

Data Approval Process:

Approval Status:

Data Analysis

Data Analysis Status:

Study Materials

Upload Study Materials:

Final Surveys for Examination : Final Surveys for Examination .pdf

Registration Category

Registration Category:

Prospective, Category 1: Data for measuring impacts have not been collected

Completion

Completion Overview

Intervention Completion Date:

Data Collection Completion Date:

Unit of Analysis:

Clusters in Final Sample:

Total Observations in Final Sample:

Size of Treatment, Control, or Comparison Subsamples:

Findings

Preliminary Report:

Preliminary Report URL:

Summary of Findings:

Paper:

Paper Summary:

Paper Citation:

Data Availability

Data Availability (Primary Data):

Date of Data Availability:

Data URL or Contact:

Access procedure:

Other Materials

Survey:

Survey Instrument Links or Contact:

Program Files:

Program Files Links or Contact:

External Link:

External Link Description:

Description of Changes:

Study Stopped

Date:

Reason: