Helping Ugandan families save for school fees
The Boston Review recently hosted a forum beginning with the article “Small Changes, Big Results” by Rachel Glennerster and Michael Kremer. The forum featured seven responses, each focusing on a different concern, though as Glennerster and Kremer later noted, “One of the themes of the responses is that randomized evaluations are not the only method that can help inform policy in developing countries.”
Chloe O’Gara argued that randomized control trials “complement but do not replace the insights of other approaches – such as participatory rural appraisal, whereby an outsider facilitates conversation within a community about its problems – that aim to collect timely, valid, and useful information for program design.” This is a common critique of randomized control trials, as is the concern that they do not take into account the importance of context and that randomized interventions are immutable, unable to be influenced by other perspectives.
But there is no rule which states that randomized control trials should be implemented in isolation from other sources of information. In Uganda for example, IPA has drawn continuously on different sources and methods of data collection in the evaluation of a primary school-based savings program.
Throughout the implementation of the program, IPA has monitored attendance, enrolment and basic scholastic payments in all 136 schools involved in the study. In addition to this data collection, researchers are in constant discussion with school administrations and their pupils, building trust and collecting qualitative information to better understand the numbers.
Additionally, program implementation has offered great insight into the challenges facing schools and the various dynamics affecting their pupils. For example, it became clear during the first year of implementation that parental involvement was an important concern. We observed that in some schools children were saving on their own and that parents were unaware of the program, but that school fees and scholastic materials were most often funded by parents. Involving parents, it was believed, would be critical to keeping children in school.
We polled teachers in all schools and their responses supported our observations, not just in relation to saving but for education in general. The majority of teachers identified “parent sensitization” as the best way in which to improve primary education. Our research question is to determine whether savings can influence educational outcomes; attendance, the number of children with school materials, paying fees, and drop-out rates. In order to answer these questions, we have to first implement a savings program that works. We saw in focus group discussions, individual interviews and informal conversation that our program needed greater parental involvement in order to “work.”
For this reason, we designed and are in the process of implementing a parent outreach program as an extension of the savings program. This insight will also be highly valuable when we come to analyzing the results of the impact assessment.
The ability of randomized control trials to robustly demonstrate causal correlations between variables and thus determine the effects of programs is not found in any other form of evaluation. This is not to say that a randomized control trial cannot benefit from other forms of evaluation. As regards the primary school based savings program, with many different types of information and evaluation we are able to implement a better program and are better able to design and understand the results of the randomized control trial study.
Click here to learn more about this project at Global Giving.