Download Study

General

Study Overview

Title:

Evaluating the impact of Southern New Hampshire University’s blended learning program on post-graduation employment and learning outcomes

Study is 3ie funded:

No

Study ID:

RIDIE-STUDY-ID-5c9848936449a

Initial Registration Date:

03/24/2019

Last Update Date:

03/20/2019

Study Status:

Ongoing

Location(s):

Rwanda

Abstract:

IDinsight is conducting a follow up study of an evaluation of the Southern New Hampshire (SNHU) / Kepler blended learning program in Kigali, Rwanda. IDinsight partnered with Kepler during the 2013-14 and 2014-15 academic years to conduct an evaluation of the program in Kigali. The 2013-2014 cohort study, which used a matching design, indicated that SNHU / Kepler students performed significantly better than the comparison students in English comprehension and computer literacy. Endline results from the 2014-15 academic years indicated that SNHU / Kepler students performed significantly better than comparison students in English language, computer literacy, cognitive skills and critical thinking. The follow-up study will measure post-graduation learning and labor market outcomes for the 2013-2014 and 2014-2015 SNHU / Kepler cohorts.

Registration Citation:

McManus, J., Chibwana, C., Bier, R. and Lokur, R., 2019. Evaluating the impact of Southern New Hampshire University’s blended learning program on post-graduation employment and learning outcomes . Registry for International Development for Impact Evaluations (RIDIE). Available at: 10.23846/ridie167

Categories:

Education

Additional Keywords:

Secondary ID Number(s):

Principal Investigator(s)

Name of First PI:

Jeff McManus & Chris Chibwana

Affiliation:

IDinsight

Name of Second PI:

Rebecca Bier & Radhika Lokur

Affiliation:

IDinsight

Research Partner

Name of Partner Institution:

Type of Organization:

Location:

Intervention

Intervention Overview

Intervention:

Kepler launched in Rwanda in 2013 and aspires to establish a network of universities across the continent providing affordable higher education. Kepler provides students with access to accredited American degrees by partnering with an innovative competency-based online degree program provided by Southern New Hampshire University (SNHU). This online work is paired with in-person coaching, support services and career preparation so graduates are able to meet the labor needs in their developing economies.

Theory of Change:

A blended learning program consisting of online content from SNHU, paired with in-person support through an in-country implementing partner (Kepler), can help students access a higher quality of education than would otherwise be possible. Students receive a curriculum tailored to application and core competencies, as well as a bachelor's degree from SNHU upon successful completion of the program. This system of tertiary education can have a positive impact on students' learning outcomes, specifically their (1) critical reasoning skills, (2) computer literacy, and (3) English comprehension skills. Positive impact on learning outcomes continues to exist in the long term, after graduation. The program can also have long run effects on students’ employment prospects, in two ways: (1) students pick up a different set of skills as compared to their counterparts at local universities, which might be desirable to employers, and (2) students graduate with an accredited American bachelor's degree from SNHU, which might be a signal to employers. The latter could be a positive or negative signal - positive due to perceived higher quality of education, or negative because the program is relatively new and unknown. A priori, we do not expect employers to receive a negative signal from the degree.

Multiple Treatment Arms Evaluated?

No

Intervention Timing

Intervention or Program Started at time of Registration?

Yes

Start Date:

11/01/2013

End Date:

Evaluation Method

Evaluation Method Overview

Primary (or First) Evaluation Method:

Matching

Other (not Listed) Method:

Additional Evaluation Method (If Any):

Difference in difference/fixed effects

Other (not Listed) Method:

Method Details

Details of Evaluation Approach:

The study takes advantage of treatment and matched comparison groups generated by previous evaluations examining learning outcomes for the Kepler 2013 and 2014 cohorts. These studies employed a matching strategy to build comparison groups. The target population contains graduates who are similar to graduates of the Kepler program across observable characteristics related to: a) the likelihood of enrolling into the Kepler program as measured by prior test scores and English language ability and b) projected academic success based on background characteristics often linked to academic achievement. We will report the pooled estimate of both cohorts, as well as the estimate for each cohort individually. Analysis of covariance (ANCOVA) will be used for all indicators. For the primary outcomes, we will perform multiple hypothesis adjustments to assess causal impact.

Outcomes (Endpoints):

Primary learning outcomes include: • Critical thinking • English skills • Computer literacy Primary employment outcomes: • Current monthly income or income when last employed • Monthly income from first job after graduation • Employment rate (each month since graduation) • Hours employed per week from current or most recent job(s) • Hours employed per week from first job after graduation Secondary employment outcomes: • Duration between graduation and beginning of employment • Employed in sector in which they studied • Whether student received first-choice job • Whether student attained higher degree • Employment satisfaction and performance • Anticipated future employment prospects • Perceptions of preparation for entry into workforce • Perceptions of academic experience • Professional network • Amount of debt • Cost of Kepler/University program per student

Unit of Analysis:

Individual

Hypotheses:

Kepler-SNHU’s blended learning tertiary education program in Rwanda will generate positive effects on post-graduation learning and employment outcomes for its graduates.

Unit of Intervention or Assignment:

Individual

Number of Clusters in Sample:

Number of Individuals in Sample:

373

Size of Treatment, Control, or Comparison Subsamples:

Treatment: 136, Control: 237

Data

Outcomes Data

Description:

Learning outcomes will be measured using two types of instruments: 1. Written academic style tests tailored to produce quantitative scores for students. 2. Online skills based tests that measure students’ critical thinking ability and computer literacy. Employment outcomes will be measured using an online survey instrument. All instruments will be administered by an IDinsight associate and IDinsight enumerators in March-April 2019.

Data Already Collected?

No

Data Previously Used?

Data Access:

Data Obtained by the Study Researchers?

Data Approval Process:

Approval Status:

Treatment Assignment Data

Participation or Assignment Information:

Yes

Description:

Data Obtained by the Study Researchers?

Data Previously Used?

Data Access:

Data Obtained by the Study Researchers?

Data Approval Process:

Approval Status:

Data Analysis

Data Analysis Status:

Study Materials

Upload Study Materials:

Registration Category

Registration Category:

Prospective, Category 1: Data for measuring impacts have not been collected

Completion

Completion Overview

Intervention Completion Date:

Data Collection Completion Date:

Unit of Analysis:

Clusters in Final Sample:

Total Observations in Final Sample:

Size of Treatment, Control, or Comparison Subsamples:

Findings

Preliminary Report:

Preliminary Report URL:

Summary of Findings:

Paper:

Paper Summary:

Paper Citation:

Data Availability

Data Availability (Primary Data):

Date of Data Availability:

Data URL or Contact:

Access procedure:

Other Materials

Survey:

Survey Instrument Links or Contact:

Program Files:

Program Files Links or Contact:

External Link:

External Link Description:

Description of Changes:

Study Stopped

Date:

Reason: