How does an employment and training program compare, in impacts and cost, to just giving people cash? In Rwanda, researchers worked with IPA to evaluate the impacts and cost-effectiveness of youth employment program, cash grants of equal cost to the funder, and a combination of the two on the employment status, time use, beneficiary income from wages and/or self-employment, value of productive assets owned by the beneficiary household, and household consumption, as well as a range of secondary outcomes and intermediate mechanisms. After 18-months (midline), the training program improved productive hours, assets, savings, and subjective well-being, while cost-equivalent cash transfers move all these outcomes as well as consumption, income, and wealth. In the head-to-head costing comparison, cash proves superior across a number of economic outcomes, while training outperforms cash only in the production of business knowledge. The 36-month endline survey will provide evidence on longer-term impacts.

The latest version of the paper can be downloaded here.

Policy Issue 

Research has found that giving cash to the poor without conditions or requirements has meaningful impacts on important life outcomes in the short term, such as child nutrition, schooling, mental health, teen pregnancy and HIV, microenterprise outcomes, consumer durables, and productive assets. The evidence on the long-term impacts of cash transfers is more mixed, but some studies have found substantial impacts. Given many studies in many different contexts showing that such household grants programs can be effective at reducing poverty and given the low cost and simplicity of distributing household grants, there is increasing interest in establishing household grants as the ‘index fund,’ or benchmark, in international development. Previous research has benchmarked food aid to cash, and evaluations have also benchmarked more complex, multidimensional programs against cash. This research contributes evidence on the relative cost-effectiveness of a youth employment program and cash grants of equal cost to the donor. The goal of the study is to provide transparent quantitative evidence that will be useful to the Government of Rwanda.

Context of the Evaluation 

The study took place in Rwamagana, Muhanga and Nyamagabe districts of Rwanda among poor and vulnerable youth (men and women), who were deemed eligible for the USAID Rwanda Youth Employment Project / Huguka Dukore, had a certain poverty status, and who expressed interest in participating in the program.1

In Rwanda, about 35 percent of the youth population is neither employed, in training, nor school, according to a 2017 report.2 The government has identified some of the major challenges facing youth  to be employable skills, mindset and attitudes, business capacity, and access to finance.3

Huguka Dukore is a 5-year project (2017-2021) that will provide 40,000 vulnerable youth (women and men) with employability skills by scaling up successfully proven Akazi Kanoze interventions, across 19 (of 30 total) districts countrywide, using a series of inclusive innovations that will invite more youth to participate in Rwanda’s historic transformation, particularly women, youth with disabilities and other vulnerable groups. Huguka Dukore follows USAID’s strategy on workforce readiness and skills training and is implemented by Education Development Center, Inc. (EDC).

The benchmarking cash transfer program was implemented by GiveDirectly, a US-based nonprofit that specializes in making unconditional household grants via mobile money.

Details of the Intervention 

Researchers worked with IPA to evaluate and compare the impacts and cost-effectiveness of the Huguka Dukore program, cash grants of equal cost to the funder, and a combination of the two programs. The evaluation was primarily interested in measuring impacts on the employment status, time use, beneficiary income from wages and/or self-employment, value of productive assets owned by the beneficiary household, and household consumption, as well as a range of secondary outcomes and intermediate mechanisms, including business knowledge, savings, subjective wellbeing, and wealth.

The study enrolled poor, underemployed youth who expressed willingness to enroll in a training program at baseline. Average yearly income in this population was about $190 a year on average. Of 2,275 individuals who attended an orientation meeting and signed up for Huguka Dukore, 1,967 met the program’s eligibility criteria. A further 119 could not be located either in the village of their stated residence, leaving a total of 1,848 youth who were enrolled in the study. After a baseline survey, conducted from December 2017-February 2018, thirteen public lotteries were used to randomly assign the youth into five groups:

  1. The Huguka Dukore program group
  2. A cash grant group (intended to be the same cost as Huguka Dukore)
  3. Cash grant and Huguka Dukore combined (to test if the interventions complement each other)
  4. A larger cash grant: About $750, which happened to be roughly equal to the cost of the combined arm
  5. Comparison group: Neither program was offered at the time of the study

Given that the total cost of the programs was not fully known before the study began, the research team conducted a detailed costing exercise prior to, and also after, the intervention period. The costing beforehand was used to estimate the total cost of the Huguka Dukore intervention, as well as the estimated overhead costs to GiveDirectly of providing household grants in this context.

This exercise arrived at a per-beneficiary cost of $452.47 of the Huguka Dukore program. However, the program ended up costing substantially less: $332 per person. Therefore researchers use regression adjustment to compare the program to a cash transfer costing the same amount, which would have delivered $255.04 to beneficiaries (see the paper for costing details).

The Huguka Dukore program was implemented for nine months, from January 2018 - November 2018 and the cash transfers were delivered between May 2018 - July 2018. The follow-up survey was conducted from July 2019 – August 2019, which was 18 months after the baseline survey (8-9 months after the program ended). A longer-term follow-up survey will be conducted 36 months after baseline (November 2020 – February 2021).

Results and Policy Lessons 

Main Findings: Huguka Dukore compared to no intervention

The main findings of evaluation of the Huguka Dukore program, relative to the comparison group, are as follows:

  • Youth experienced a surge in productive asset values, which rose to 154 percent higher than the comparison group average, a large and notable impact given the program made no material transfers to the beneficiaries.
  • The program also led to an increase in productive hours: Huguka Dukore was successful in driving a 3 hour increase to a base of 18.4 hours, an improvement of 16 percent.
  • However, youth who received the program were no more likely to be employed than the comparable youth who did not receive the program, nor did program youth experience higher incomes or consumption as a result of the program.
  • Average savings doubled.
  • Subjective wellbeing improved (based on a survey about happiness and life-satisfaction)
  • Business knowledge increased: participants performed better on a test of business knowledge built against the course curriculum.

Main Findings: Cash grants compared to no intervention

The main findings of the evaluation of the core cash grant amount (on average 14 months after transfers took place), relative to the comparison group, are as follows:

  • Youth in the cash group also experienced a surge in productive assets; values almost quadruple relative to the comparison group.
  • Youth experienced higher incomes and their household- and individual-level consumption increased.
  • Productive hours are non-linear in transfer amount, with the middle transfer amount leading to a significant 6.5 hour per week increase, and none of the other transfers having a significant impact. Youth in the large cash transfer arm achieve an insignificant 1.6-hour improvement. This is the first evidence suggesting that once transfers become sufficiently large they begin to reduce the incentive to work additional hours.
  • However, youth who received the program were no more likely to be employed than the comparable youth.
  • Average savings more than doubled
  • Subjective well-being improved
  • Net, non-land wealth increased by 90 percent

Main Findings: Huguka Dukore compared to cash grants

In the head-to-head comparison, the evaluation findings can be summarized as follows:

  • The cost-equivalent cash grant performed significantly better than Huguka Dukore at increasing monthly income, productive assets, subjective well-being, beneficiary consumption, and household livestock wealth.
  • Huguka Dukore was better at increasing business knowledge (the only outcome in which it outperformed cash)
  • In sum, over the 18 month horizon, youth benefited more from cash grants than from Huguka Dukore program across a range of indicators central to beneficiary economic welfare, while Huguka Dukore was more effective at generating business knowledge.
  • Neither Huguka Dukore or cash grants had a statistically significant impact on employment after 18 months.

Other noteworthy findings:

  • The evaluation did not find any complementarity between the two programs. Rather, in something of a challenge to ever-more complex bundled programs, each of these interventions has a distinct set of benefits that operated independently.
  • Nor did the evaluation find any ‘spillover’ effects on outcomes of non-beneficiaries in the same villages, though evidence suggests that take-up of HD is highest when that program is implemented with high geographic intensity.
  • Both interventions had a relatively consistent effect across richer and poorer, male and female, and across local labor market conditions.
  • While neither program significantly improved overall employment rates during the study period, a more detailed analysis shows that youth who received cash were more likely to move from wage labor into self-employment (they became more entrepreneurial), while Huguka Dukore beneficiaries became engaged in more off-farm wage labor (their training propelled them into wage jobs). In other words, at cost-equivalent levels, cash and training have launched youth into distinct forms of employment.

Sources

[1] The eligibility criteria Huguka Dukore are (a) ages ranging from 16 – 30 and (b) less than nine years of completed education. Because of the conditions placed on GiveDirectly by the Rwandan government, eligibility for the study was further limited to (c) households registered in Ubudehe poverty status 1 or 2. In order to provide a study that has compliance rates with the HD training that are as high as possible, we further restricted eligibility to those who (d) expressed interest in participating in the employment and entrepreneurship readiness training.

[2] National Institute of Statistics of Rwanda (NISR), Labour Force Survey August 2017 report, April 2018. P. 46 Available at: http://www.statistics.gov.rw/publication/labour-force-survey-report-august-2017

[3] Rwanda youth sector strategic plan 2013-2018. Available at: http://www.minecofin.gov.rw/fileadmin/templates/documents/sector_strategic_plan/Youth_Sub_Sector_SSP.pdf