Evidence suggests that pay-for-performance (P4P) contracts can elicit greater effort from civil servants when designed well, but does advertising performance pay affect who applies for these public sector jobs? In Rwanda, researchers partnered with the Rwanda Education Board to compare the effects of two alternative ways of increasing primary teacher salaries, each with the same total costs to government: a P4P contract, which paid the top 20 percent of teachers in each district a bonus, and a fixed-wage contract. Preliminary results find that P4P contracts were effective in improving teaching quality and student learning. Most of the impact was driven by increased effort, as the contracts incentivized teachers to perform better. Some of the impact was also driven by recruitment, as the P4P contracts attracted somewhat different teachers than fixed-wage contracts.

Policy Issue 

Although access to education has greatly expanded across sub-Saharan Africa, improving education quality remains a challenge. Recruiting, motivating, and retaining skilled teachers is a step towards ensuring high-quality education. Pay-for-performance contracts, which reward teachers with bonuses for students’ high test scores, is one way that some school systems have tried to improve teachers’ performance, but evidence on the efficacy of performance pay contracts has been mixed.

In addition, at scale and over the long-term, performance pay teaching contracts may affect the composition of the teaching profession. Skilled teachers may be more willing to join the public sector under performance pay; less skilled individuals may be deterred from applying or may drop out. To date, there has been no rigorous evidence on how performance pay affects this selection into and out of the teaching profession.

This research aimed to provide new evidence on both questions. Can linking teacher performance to student learning outcomes improve teachers’ performance, and in doing so, contribute to student learning gains? How effective are performance pay contracts at recruiting skilled and motivated teachers?

Context of the Evaluation 

Rwanda is one of the top-performing countries in sub-Saharan Africa in terms of access to education. In 2015, net enrollment in primary education was at 97 percent. Yet, the U.K. Independent Commission for Aid Effectiveness (2012) found that the rapid expansion of primary education has led to a decline in educational outcomes. Recruiting and retaining qualified, skilled, and motivated teachers to improve education quality is a priority for Rwanda.

Several performance pay programs already exist in Rwanda’s public sector, and the government has expressed interest in reforming the incentive structure to make it more evidence-based. Under the imihigo system, public sector employees in other sectors receive financial rewards of up to five percent of their salary based on subjective performance evaluations. One evaluation found that a pay-for-performance program for primary health care clinics in Rwanda had substantial impacts on a range of health outcomes.1  In public schools, a substantial share of teachers’ existing salaries is made up of bonuses that are in theory discretionary and might be linked to performance. However, in practice, all teachers receive a fixed bonus amount.

Details of the Intervention 

Researchers worked with IPA, the IGC, and the REB to conduct a two-tiered randomized evaluation assessing whether performance-based contracts for teachers  improved students’ learning outcomes.

The study compared the effects of two alternative ways of increasing teacher salaries, each of which had the same total costs to government. The P4P contracts provided a bonus of RWF 100,000 (~15 percent of annual salary) to the top 20 percent of upper-primary teachers within a district. The fixed-wage contract instead provided a top-up of RWF 20,000 to all upper-primary teachers (grades P4, P5, and P6) in the school.

Tier 1: Advertised P4P versus fixed-wage contracts.

Researchers first randomly assigned teacher labor markets (defined as application pools to specific districts within specific teacher qualification types) to P4P or fixed-wage contracts.2 In a P4P labor market, potential applicants were told that recruits to new primary posts would receive the P4P contract for the 2016 and 2017 school years. Conversely, in a fixed-wage labor market, potential applicants were told that recruits to new primary posts would receive the fixed-wage contract for the 2016 and 2017 school years. The recruitment campaign had over 600 hiring lines (across both upper and lower primary) in six districts, comprising more than 60 percent of government planned hires for the year.

In this tier, researchers were interested in whether performance pay had any impact on who chooses to become a teacher (or enters the public sector). To do so, they looked at the characteristics of applicants and of hired teachers,3 as well as at the performance of those teachers in their eventual jobs.

Tier 2: Experienced P4P versus fixed-wage contracts.

The aim in the second tier was to randomize the schools to which REB had allocated the new primary posts to contracts. A school was included in the sample if it had at least one new post that was filled and assigned to an upper primary grade (grades 4, 5 and 6). Of the 164 schools in the second tier of the experiment, 85 were assigned to P4P and 79 were assigned to fixed-wage contracts. All upper primary teachers within each school received the new contract. At individual applicant level, this amounted to re-randomization and hence a change to the initial assignment for some new recruits. To ensure that new contracts paid as least as well as those advertised at the first tier, all new recruits were told that they would receive a retention bonus of 80,000 RWF if they remained in post during the 2016 school year. Teachers in P4P schools were also told that the 2016 performance award—determined by multiple teacher-input observations as well as beginning- and end-of-year student assessments—was conditional on remaining in post during the 2016 school year, and would be paid early in 2017.

In this tier, researchers were interested in whether performance pay had any impact on the behavior of individuals of who chose to become a teacher.

Results and Policy Lessons 

Preliminary Results*

After two years:

Working under P4P contracts motivated teachers to achieve better learning outcomes with their students than fixed-wage contracts. The improvement in student achievement was 0.09 standard deviations per year on average across the two years, and 0.16 standard deviations in the second year of the study. This means that in the second year, a student from the median (50th percentile) moved up to the 56th percentile of students.

P4P raised student learning outcomes by improving teacher presence and classroom conduct. Teacher presence was six percentage points higher among recruits who experienced the P4P contract compared to recruits who experienced the fixed-wage contract. (A sizeable impact given that baseline teacher presence was already 90 percent.) Teachers who experienced P4P were more effective in their classroom practices than fixed-wage contract teachers by 0.26 points, as measured on a four-point scale.4

Teachers recruited under P4P contracts were at least as effective as those recruited under fixed-wage contracts. Advertisement of the P4P contract changed the profile of hired teachers. Individuals recruited under the P4P contract were more money-oriented in a game that tested their intrinsic motivation compared to individuals recruited under the fixed-wage contract. However, these P4P recruits performed no worse than the fixed-wage recruits in terms of their presence, classroom conduct, or the learning outcomes of their students. The net effect of being recruited to, and then working under, a P4P contract was 0.21 standard deviations of learning gain among students in the second year.

P4P was popular among teachers. A post-evaluation survey asked teachers for their overall opinion about the idea of providing bonus payments on the basis of objective measures of performance. More than 1,300 teachers responded, of whom 78 percent had a favorable opinion of performance pay on such an objective basis; only 12 percent were unfavorable; 7 percent were neutral.

*These results are preliminary and may change after further analysis and/or peer-review.

Sources

1. Basinga, Paulin, Paul J Gertler, Agnes Binagwaho, Agnes L. B. Soucat, Jennifer R. Sturdy, and Christel M. J. Vermeersch, 2011. “Effect on maternal and child health services in Rwanda of payment to primary health-care providers for performance: An impact evaluation”. The Lancet, 377: 1421–1428.

[2] Districts solicited applications at the district-by-subject-family level, aggregating subjects into three subject families that correspond to the degree types issued by Teacher Training Colleges: math and science (TMS); modern languages (TML); and social studies (TSS). For this research a labour market is defined in terms of ‘district-by-subject family’. A total of 18 such markets are included in the study.

[3] Teacher attributes were measured through surveys with teachers, head teachers, and their deputies; an additional game-like experiment provided measures of teacher motivation on the job.

[4] The classroom observation tool that was used for this measurement was a locally adapted version of a “Charlotte Danielson” instrument and may be of interest to the government of Rwanda in relation to imihigo in education. For further details, see Danielson (2007), Enhancing professional practice: A framework for teaching.