Student learning levels across East Africa remain extremely low, despite more than a decade of major reforms and significant new investments in public education. To help generate rigorous evidence on what works, researchers are evaluating the impact of an education intervention that sends grants directly to schools and pays teachers a performance-based bonus.

Policy Issue 

Overall student-learning levels remain extremely low across East Africa, despite a decade of major reforms and significant new investments in public education. In Kenya, Tanzania, and Uganda, recent nationwide surveys show that large numbers of children are unable to read or do arithmetic at their grade level.1 Governments in the region are currently using two main approaches to improve the quality of education: strengthening teacher training and disbursing capitation grants, which provide funding to schools based on the number of students enrolled. However, the efficacy of these approaches in improving learning outcomes has not been established. In fact, several studies show that formal levels of teacher qualification are weakly correlated with performance.2 Capitation grant programs have not been rigorously evaluated, and many of these programs lack accountability measures and incentive structures. Evidence suggests that rewarding teachers or schools that achieve learning outcomes may be a more effective way to improve student learning and performance.3 This evaluation aims to add to the evidence on these topics.

Context of the Evaluation 

Educational outcomes in Tanzania have not improved in recent years, despite several new reforms and aid initiatives. Just 68 percent of Tanzanian children aged 10-16 can pass a basic numeracy test, 53 percent can pass a basic test in Kiswahili, and only 35 percent can pass an English literacy test.

In February 2013, Twaweza and IPA-affiliated researchers launched KiuFunza, an education initiative that seeks to discover through rigorous research what really works to help children learn. Twaweza, the implementing partner, is a nonprofit focusing on large-scale change in East Africa, with operations in Tanzania, Kenya and Uganda.

Details of the Intervention 

Researchers partnered with Twaweza to design and evaluate approaches aimed at improving learning outcomes for school children in the first three grades of school. The four-year evaluation has taken place in two phases.

Phase I

In the first two years of the project (2013-2014) researchers sought to evaluate the impact of: i) providing schools with extra resources in the form of capitation grants (a grant per child enrolled) and ii) providing teachers with financial incentives based on student performance. The evaluation aimed to measure the two approaches separately and in combination with each other.

Government primary schools in 10 districts in Tanzania were divided into three treatment groups and a comparison group:

  1. Capitation Grant: Each school received a grant of 10,000 Tanzanian shillings (US$ 5.84) per student enrolled, delivered in two disbursements per year. Grant funds were channeled directly to primary schools, and school stakeholders, including head teachers, teachers, school committees, village chairpersons, parents and students, were provided with information about the grants through informational meetings and written materials. (70 schools)
  2. Cash on Delivery: Kiswahili, English, and Math teachers in Grades 1-3 were eligible to receive 5,000 Tanzanian shillings for each student who passed a given subject test at the end of the year, implemented by the KiuFunza program, for a maximum of 15,000 Tanzanian shillings per student. Teachers were not penalized for students who did not pass. (70 schools)
  3. Combination: These schools received both the capitation grant and cash on delivery interventions. (70 schools)
  4. Comparison group: This group did not receive either the capitation grant or cash on delivery treatment. (140 schools)

Researchers used government test scores, results from the tests conducted by Twaweza, results from independent tests conducted by the researchers, school administrative records, and data from six surveys (three per year), to measure changes in school performance and student learning.

Twaweza and its partners in each district directly implemented all interventions, and Twaweza provided the funds for the grants and cash-on-delivery interventions.

Phase II

In 2015, the researchers and Twaweza dropped the capitation grant arm (see explanation under “results”) and added another incentive arm. Both intervention arms in the second phase of the program focus on teacher incentive programs, with two separate designs.

The two versions of the incentive program are labeled “levels” and “gains.” The levels incentive is based on the cash-on-delivery incentive from Phase I; teachers receive a bonus for each student that passes given subject test. Unlike the COD incentive from phase 1, however, teachers can earn funds even if a student does not pass the entire test. Teachers earn money based on each skill passed by a student; this means that more students are likelier to earn money for their teacher. Furthermore, some skills are ‘worth’ more than others, rewarding teachers who are able to effectively teach difficult skills (English paragraph reading, division, etc.) for their efforts.

The new, “gains” incentive rewards teachers for smaller, incremental learning gains. At the beginning of the school year, students are placed into ability groups based on their performance on the previous year’s Twaweza test. These ability groups span the breadth of the KiuFunza program. In this way, all students in an ability group begin at the same starting point, making the competition fair. At the end of the year, students who have improved the most in their ability groups will earn their teacher the most money. This new incentive is particularly meant to reward good teachers in schools where the initial learning levels of students are low (i.e. they are far from being able to pass the tests). 

Government primary schools in 10 randomly selected districts were randomly assigned to either the levels-based incentive, the gains-based incentive, or a comparison group that received no incentive program. Researchers are measuring the impacts of the two incentives over a two year period, both in relation to each other and in relation to no incentive, on student learning.

Results and Policy Lessons 

Preliminary findings and analysis (Phase I):

Analysis of Phase I findings showed that children in schools where both capitation grants and financial incentives were tested together (“combination schools”) showed significantly better learning outcomes than schools in the comparison group.

The bonus only had some impact, with teachers spending more time teaching (extra classes and additional test taking); it has shown a positive but not significant impact on learning outcomes. One possible explanation for this is the teacher’s lack of initial trust that the bonuses would actually delivered. After receipt of the bonuses in 2013, teachers responded enthusiastically and had more faith that the program would deliver on its promises.

In addition, the capitation grant was received in full by all schools in the sample. This change led to reports of more essential materials at schools, but it did not lead to any significant improvements in learning.

With complete funding reaching the schools as planned, the Ministry of Education and Vocational Training stated its intention to begin sending the grants directly to schools. (The current government policy had been to send capitation grants indirectly, through many district- and ward-level officials; this results in time delays, and less than the full amount reaching the school itself.) As a result of this stated policy change, researchers and the partner organization decided to discontinue the capitation grant component of the study, and further delve into effective teacher incentive programs through the two new intervention arms begun in Phase II of the project.   

Sources

[1] 2013 Uwezo East Africa Report: Are Our Children Learning?

[2] E.g. Woessmann, Ludger. “Cross-Country Evidence on Teacher Performance Pay.” Discussion Paper No. 5101, July 2010; Kane, T. J., J. E. Rockoff, and D. O. Staiger. (2008). "What Does Certification Tell Us About Teacher Effectiveness? Evidence from New York City," Economics of Education Review, 27, 615-631; Kleiner, M. (2000). "Occupational Licensing," Journal of Economic Perspectives, 14, 189-202.

[3] Kremer, Michael, and Alaka Holla. "Improving education in the developing world: what have we learned from randomized evaluations?" Annual Review of Economics 1 (2009): 513.