Accumulating evidence suggests that pay-for-performance (P4P) contracts can elicit greater effort from incumbent civil servants, but less is known about how these contracts affect the composition of the public sector workforce. We provide the first experimental evidence of the impact of P4P on both the compositional and effort margins. In partnership with the Government of Rwanda, we implemented a ‘pay-for-percentile’ scheme (Barlevy and Neal 2012) in a novel two-tier experimental design. In the first tier, we randomly assigned teacher labor markets to either P4P or equivalent fixed-wage contracts. In the second tier, we implemented a ‘surprise’, school-level re-randomization, allowing us to separately identify the compositional effects of advertised P4P contracts and the effort effects of experienced P4P contracts. Our pre-analysis plan sets out a theoretical framework that helps to define a set of hypotheses, and conducts simulations on blinded data to develop high-powered tests. We find that P4P contracts did change the composition of the teaching workforce, drawing in individuals who were more money-oriented, as measured by a framed Dictator Game. But these recruits were not less effective teachers—if anything the reverse. On the effort margin, we observe substantial and statistically significant gains in teacher value added, mirrored in positive effects on teacher presence and observed pedagogy in the classroom. In Year 2, we estimate the total effect of P4P, across compositional and effort margins, to be 0.21 standard deviations of pupil learning. One quarter of this impact can be attributed to selection at the recruitment stage, with the remaining three-quarters arising from increased effort.
In Rwanda, we have continued our global tradition of rigorous, applicable research by building foundational research capacity and conducting evaluations in areas of pressing national concern. Examples of our work below offer promising insights into everyday issues that affect the lives of the Rwandan poor.
How do standard development programs compare to just giving people cash? In Rwanda, researchers conducted a randomized evaluation to shed light on this question. Villages were randomly assigned to one of four groups: they received either a USAID-funded, integrated WASH and nutrition program (with savings and asset transfer components), unconditional cash grants of equal cost to the donor, a larger cash transfer, or no program at the time of study. The transfers were funded by USAID and Google.org.
The evaluation measured impacts on five main health and economic outcomes: household dietary diversity, maternal and child anemia, child growth (height-for-age, weight-for-age, and mid-upper arm circumference), household wealth, and household consumption, as well as other secondary outcomes, such as savings.
After approximately one year:
» The integrated nutrition and WASH program had a positive impact on savings, a secondary outcome, among the eligible population, but did not impact any primary outcomes (household dietary diversity, maternal or child anemia, child growth, household consumption, or wealth) within the period of the study.
» An equivalent amount of cash (a cost to USAID of $142 per household) allowed households to pay down debt and boosted productive and consumption assets, but did not impact child health outcomes.
» A much larger cash transfer—of more than $500 per household—had a wide range of benefits: it not only increased consumption, savings, assets, and house values,but improved household dietary diversity and height-for-age, and decreased child mortality.
» The results suggest that, over the time period of the study, targeted programs focused on changing specific outcomes may be able to do so at lower cost than cash, but that large investments of cash can more rapidly affect some leading indicators of malnutrition.
» The results also suggest that large cash transfers impact not only the economic measures of consumption and wealth, but also dietary diversity, height-for-age, and child mortality, while small transfers appear to have more limited benefits.
Limited financial knowledge, skills, and confidence are associated with suboptimal financial behavior such as low rates of saving, limited usage of deposit and transactional accounts, and overindebtedness. The Government of Rwanda, the World Bank Group, and Innovations for Poverty Action (IPA) partnered to conduct a large-scale randomized evaluation that measured the impact of Phase One of the Financial Education through SACCOs program. The evaluation measured and compared the impacts of two program delivery models—autonomous vs. fixed trainer selection—on SACCO members’ financial knowledge, skills, attitudes, and behaviors.
- SACCO members attended more sessions of the Financial Education through SACCOs when SACCOs had autonomy to choose trainers from the local community (“autonomous selection”).
- SACCO members in this autonomous selection group showed improvements in financial knowledge, attitudes, and behaviors, including with respect to knowledge of key rules of thumb, attitudes that emphasize saving and responsible borrowing, and having—and strictly adhering to—a written budget and financial plan.
- They were also more likely to report saving regularly towards financial goals, and to deposit savings in the SACCO.
- However, when trainer profiles were predetermined and limited to individuals with formal roles at the SACCO (“fixed trainer selection”) these improvements were not observed. No improvements in either group were found on account usage, borrowing behavior, or financial security.
We present the results of a study designed to ‘benchmark’ a major USAID-funded child malnutrition program against what would have occurred if the cost of the program had simply been disbursed directly to beneficiaries to spend as they see fit. Using a three-armed trial from 248 villages in Rwanda, the study measures impacts on households containing poor or underweight children, or pregnant or lactating women, as well as the broader population of study villages. We find that the bundled health program delivers benefits in an outcome directly targeted by specific sub-components of the intervention (savings), but does not improve household dietary diversity, child anthropometrics, or anemia within the year of the study. A cost-equivalent cash transfer boosts productive asset investment and allows households to pay down debt. The bundled program is significantly better in cost-equivalent terms at generating savings and worse for debt reduction, while cost-equivalent cash drives more asset investment. A much larger cash transfer of more than $500 per household improves a wide range of consumption measures including dietary diversity, as well as savings, assets, and housing values. Only the large cash transfer shows evidence of moving child outcomes, with significant but modest improvements in child height-for-age, weight-for-age, and mid upper-arm circumference (about 0.1 SD). The results indicate that programs targeted towards driving specific outcomes can do so at lower cost than cash, but large cash transfers drive substantial benefits across a wide range of impacts, including many of those targeted by the more tailored program.
Background: Community health clubs are multi-session village-level gatherings led by trained facilitators and designed to promote healthy behaviours mainly related to water, sanitation, and hygiene. They have been implemented in several African and Asian countries but have never been evaluated rigorously. We aimed to evaluate the effect of two versions of the community health club model on child health and nutrition outcomes.
Methods: We did a cluster-randomised trial in Rusizi district, western Rwanda. We defined villages as clusters. We assessed villages for eligibility then randomly selected 150 for the study using a simple random sampling routine in Stata. We stratified villages by wealth index and by the proportion of children younger than 2 years with caregiverreported diarrhoea within the past 7 days. We randomly allocated these villages to three study groups: no intervention (control; n=50), eight community health club sessions (Lite intervention; n=50), or 20 community health club sessions (Classic intervention; n=50). Households in these villages were enrolled in 2013 for a baseline survey, then re-enrolled in 2015 for an endline survey. The primary outcome was caregiver-reported diarrhoea within the previous 7 days in children younger than 5 years. Analysis was by intention to treat and per protocol. This trial is registered with ClinicalTrials.gov, number NCT01836731.
Findings: At the baseline survey undertaken between May, 2013, and August, 2013, 8734 households with children younger than 5 years of age were enrolled. At the endline survey undertaken between Sept 21, 2015, and Dec 22, 2015, 7934 (91%) of the households were re-enrolled. Among children younger than 5 years, the prevalence of caregiver-reported diarrhoea in the previous 7 days was 514 (14%) of 3616 assigned the control, 453 (14%) of 3196 allocated the Lite intervention (prevalence ratio compared with control 0·97, 95% CI 0·81–1·16; p=0·74), and 495 (14%) of 3464 assigned the Classic intervention (prevalence ratio compared with control 0·99, 0·85–1·15; p=0·87).
Interpretation: Community health clubs, in this setting in western Rwanda, had no effect on caregiver-reported diarrhoea among children younger than 5 years. Our results question the value of implementing this intervention at scale for the aim of achieving health gains.