Evaluating The Hunger Project’s Epicenter Scale-Up Project in Eastern Ghana

Given the interconnected challenges poor families often face, many practitioners in international development believe that breaking the cycle of poverty in extremely poor communities requires a multi-faceted and participatory approach. In rural Ghana, researchers evaluated the impact of a community-driven development program on community members’ health, education, nutrition, livelihoods, access to finance, and local governance.

Policy Issue:

Very poor households often face a set of interconnected challenges that may keep them in extreme poverty. For example, malnourished children may have difficulty succeeding in school; families without financial services may turn to predatory lenders when cash-strapped; poorer, less educated households may be less likely to participate in politics and, in turn, gain political power. Given these interconnected challenges, many practitioners in international development argue that breaking the poverty cycle requires a multi-faceted and participatory approach, and that tackling multiple issues at once can have a “multiplier effect” and quicken the pace of development. Yet there is relatively little evidence on the effectiveness of these types of participatory community driven development programs.

Context of the Evaluation:

The international NGO The Hunger Project (THP) implements a community-driven development program called the Epicenter Strategy in eight countries across Africa. It aims to mobilize rural communities to lead a series of self-help initiatives to reduce their own hunger and poverty. The multi-year program features four phases: an initial mobilization phase in which THP provides training to mobilize communities to commit to creating positive change, a construction phase in which communities partner with THP to construct the epicenter building, and the program implementation stage, and the transition to self-reliance stage.

Each epicenter building houses a clinic, a microfinance bank, a three-acre farm, a food storage unit, and either a kindergarten or library. To construct the centers, communities must contribute in cash or in kind, while THP provides some construction materials. Once a center is constructed, it provides community programs that address health, food security, education, agriculture, and household finance needs. After several years of operation, THP transitions out of supporting the epicenter with the goal that communities will fully take over its operation.

Details of the Intervention:

Researchers evaluated the impact of THP’s Epicenter Strategy on 36 outcomes in eight areas: community mobilization; gender equality; food security; literacy and education; health and nutrition; water, environment, and sanitation; livelihoods and microfinance; and governance. The evaluation took place in 13 districts in eastern Ghana in which THP had not worked prior to the evaluation.

Each district was divided into roughly eight clusters of about 10,000 people each. Fifty-five clusters were randomly assigned to receive the Epicenter Strategy via public lottery and 50 were assigned to the comparison group. Researchers randomly selected 20 households from two villages within each cluster in both the treatment and comparison groups (approximately 3,800 households) and measured changes in these households well-being over a five-year period.

The intervention consisted of active promotion of epicenters in the treatment clusters, in which THP conducted workshops and outreach activities with village volunteers, construction of the centers, implementation of programs, and transitioning to self-reliance (in which THP withdraws support).

Results and Policy Lessons:

Results forthcoming.

Encouraging Teacher Attendance through Monitoring with Cameras in Rural India

Getting teachers to attend school is sometimes more difficult than getting children there. Can monitoring tied to a salary incentive reduce teacher absenteeism in rural schools in India?  Will reduced absenteeism increase student achievement?  The answer is yes: monitoring made teachers show up, and students learned more.

Policy Issue: 

Over the past decade many developing countries have expanded primary school access, energized by initiatives such as the United Nations Millennium Development Goals, which call for achieving universal primary education by 2015. However, these improvements in school access have not been accompanied by improvements in school quality. Poor learning outcomes may be due, in part, to high absence rates among teachers, who often lack strong incentives to attend work. There have been relatively few rigorous studies that have evaluated successful interventions to address absenteeism, so little is known about how reduced absenteeism impacts other educational outcomes. If teachers are incentivized to show up to school, is that all they do- or once there do they teach? Do simple financial incentives undermine their motivation to teach well?

Context of the Evaluation: 

Despite booming economic growth and an improved educational infrastructure in many regions in India, primary education is lagging in many remote and marginalized communities. Sixty-five percent of surveyed children enrolled in grades 2 through 5 in government primary schools could not read a simple paragraph, and 50 percent could not do simple subtraction. Teacher absenteeism, a pervasive problem in these schools, may contribute to these poor educational outcomes. Disciplinary actions are rarely undertaken against absent teachers: in a survey of 3,000 Indian government schools, only one principal reported a teacher having been fired for poor attendance . This may account for the extremely high rate of teacher absence in India: in schools examined by this study, teachers attended classes about 65 percent of the time.

Details of the Intervention: 

This study estimates the effect of incentives on teacher attendance and of increased teacher attendance on students' attendance and abilities in math and language. Seva Mandir, a local NGO, worked with researchers to randomly selected 57 of their informal education centers for the intervention, and 56 for a comparison group. Ordinarily, teachers were paid a salary of Rs. 1,000 (about US$23) for at least 20 days of work per month, and this payment structure remained unchanged for the comparison schools. In the intervention schools, teachers received a Rs. 50 (US$1.15) bonus for each additional day they attended in excess of 20 days, and received a Rs. 50 fine for each of the 20 days they skipped work—however, the fine could not exceed Rs. 500 total, due to ethical and political concerns.. Thus, a monetary incentive was attached to teacher attendance. When these incentives were implemented, monthly pay in the treatment schools ranged from Rs. 500 to Rs. 1,300 (US$11.50 to US$29.50).  

In order to monitor teacher attendance, Seva Mandir gave each teacher a camera, along with instructions to have one student take a picture of the teacher and the class at the start and close of each school day. The camera's timestamp feature allowed Seva Mand  ir to determine when and for how long the teacher was at school.

Results and Policy Lessons: 

Impact on Teacher Attendance: The program resulted in an immediate and long lasting improvement in teacher attendance rates in treatment schools. Over the 30 months of the study, teachers at program schools had an absence rate of 21 percent, compared to 44 percent at baseline and 42 percent in the comparison schools. Even four years after the start of the program, teacher attendance remained significantly higher in treatment schools (72 percent attendance) than in comparison schools (61 percent), suggesting that teachers did not change their behavior simply for the evaluation – their response was almost entirely due to the financial incentives.  

Impact on Education: Teachers who were at school were just as likely to be teaching in treatment schools compared to comparison schools- they did not just show up for the picture and go home. Student attendance on days the teacher was there was similar in both groups, meaning that students in the treatment group received about 9 percentage points (or 30 percent) more days of instruction on average simply because their teachers were more likely to be at school. This corresponds to 2.7 more days of instruction time a month at treatment schools, and this effect is larger than that of other successful interventions that have been shown to increase child attendance. A year into the program, test scores in the treatment schools were 0.17 standard deviations higher than in the comparison schools. Two and a half years into the program, children from the treatment schools were also 10 percentage points (or 62 percent) more likely to transfer to a formal primary school, which requires passing a competency test.

Replicability in Formal Settings: The question arises as to whether the program can be instituted for regular teachers in government schools. Teachers in government schools are often more politically powerful than teachers in informal or private schools. Thus, it may prove difficult to institute a system in which government teachers would be monitored daily and their pay linked to attendance. However, the above evidence suggests that if teacher attendance can be improved this should flow through into improved test scores.

Selected Media Coverage:



Esther Duflo, Rema Hanna

Graduating the Ultra-Poor in Peru

More than one fifth of the world’s population lives on less than US$1.25 per day. While many credit and training programs have not been successful at raising income levels for these ultra-poor households, recent support for livelihoods programs has spurred interest in evaluating whether comprehensive “big push” interventions may allow for a sustainable transition to self-employment and a higher standard of living. To test this theory, in six countries researchers evaluated a multi-faceted approach aimed at “graduating” the ultra-poor from poverty. They found that generally the approach had long-lasting economic and self-employment impacts and that the long-run benefits outweighed their up-front costs. In Peru, the gains were smaller than in most other countries.

Policy Issue:

More than one fifth of the world’s population lives on less than US$1.25 per day. Many of these families depend on insecure and fragile livelihoods, including casual farm and domestic labor. Their income is frequently irregular or seasonal, putting laborers and their families at risk of hunger. Self-employment is often the only viable alternative to menial labor for the ultra-poor, yet many lack the necessary cash or skills to start a business that could earn more than casual labor.

In the past, many programs that have provided ultra-poor households with either credit or training to alleviate these constraints have not been successful at raising household income levels on average.  However, in recent years, several international and local nongovernmental organizations have renewed their support for programs that foster a transition to more secure livelihoods. Combining complementary approaches—the transfer of a productive asset, training, consumption support, and coaching— into one comprehensive program may help spur a sustainable transition to self-employment. To better understand the effect of these programs on the lives of the ultra-poor, researchers coordinated to conduct six randomized evaluations in Ethiopia,  GhanaHondurasIndiaPakistan, and Peru.

Context of the Evaluation:

In Peru, researchers partnered with implementing organizations Plan International-Peru and Asociación Arariwa. The study focused on households in the rural communities of Canas and Acomayo, located in the department of Cusco, which is among the poorest departments in Peru.[1] To select the poorest members of the communities, the project team conducted a Participatory Wealth Ranking, in which villagers collectively ranked households according to their wealth during a community meeting. Asociación Arariwa conducted a short survey afterwards to verify the results of the ranking.

Details of the Intervention:

Researchers conducted a randomized evaluation to test the impact of a two-year comprehensive livelihoods program (“the Graduation approach”) on the lives of the ultra-poor in Peru. The approach was first developed by the Bangladeshi NGO BRAC in 2002 and has since been replicated in several countries.

In Peru, an initial sample of 2,284 households was spread out across 86 villages. In half of the villages, households were randomized to either receive the program or not. The households that did not receive the program but had neighbors that did, served as a sub-comparison group to measure “spillover” effects.  The remaining 43 villages were randomly selected to be pure comparison villages (no one in the village received the program). The program consisted of six complementary components, each designed to address specific constraints facing ultra-poor households:

1. Productive asset transfer:One-time transfer of a productive asset valued at 1,200 PEN (2014 PPP US$854). Most participants (64 percent) chose guinea pigs, a quarter picked hens, and small number picked cattle (4 percent). 

2. Technical skills training: Training on running a business and managing their chosen livelihood. For example, households who selected livestock were taught how to rear the livestock, including vaccinations, feed and treatment of diseases.

3. Consumption support: Regular food support is a component of the Graduation approach, but in this study it was not unique to the treatment group. A governmental cash transfer program called Juntos already existed prior to the program. Households not enrolled in the Juntos program received monthly cash transfers of 100 PEN  (2014 PPP US$72). Households enrolled in Juntos (in both treatment and comparison groups) received cash transfers of 200 PEN (2014 PPP US$143.33) every two months from Juntos.

4: Health education: Information about nutrition, healthy practices, and prenatal health (delivered during training sessions).

5. Savings account:Households were encouraged to open savings accounts with Banco de Nacion or deposit group savings with Arariwa Microfinance.

6. Households visits: Home visits by Arariwa staff every six weeks over 24 months to provide to provide accountability, coaching, and encouragement.

The Graduation program began in early 2011 and continued until mid-2013. Researchers conducted the first endline survey after the program ended, as well as a second endline survey one year later (mid-2014).

Results and Policy Lessons:

Across all six countries, researchers found that the program caused broad and lasting economic impacts. Treatment group households consumed more, had more assets, and increased savings. The program also increased basic entrepreneurial activities, which enabled the poor to work more evenly across the year. While psychosocial well-being improved, these noneconomic impacts sometimes faded over time. In five of the six studies, long-run benefits outweighed their up-front costs. However, for households that received the Graduation program in Peru, one year after the Graduation program ended the gains were smaller, compared to most other countries:

Economic impacts: Households that received the program saw an 8 percent increase in food consumption (to 2014 PPP US$82.05 a month on average), but no significant increases in non-food consumption or durable good expenditures. They did not experience a significant increase in assets or food security, though they saved 2014 PPP US$220.10 a month on average, 26 percent more than households in the comparison group.

Self-employment: Households that received the program earned 2014 PPP US$307.30 in revenue from livestock on average, a 16 percent increase relative to the comparison group, but they did not experience an increase in agricultural income. Nor did they spend any more time on productive activities than comparison group households.

Psychosocial wellbeing: Households in Peru reported being physically in better health and happier than households that did not receive the program.

Political Involvement:Households did not experience significant gains in political involvement or women’s empowerment in Peru relative to the comparison group. 

Cost-benefit analysis: Compared to less comprehensive interventions, the Graduation program had relatively high up-front costs. Researchers calculated total implementation and program costs to be US$2,604 per household in Peru (2014 PPP US$5,742). However, estimated benefits from consumption and assets growth amount to 2014 PPP US$8,380 per household, representing an overall 146 percent return.

[1] “Descriptive Statistics from the Extreme Poverty Graduation Program Baseline in Cusco, Peru.” October 2011. Accessed at: http://www.microfinancegateway.org/sites/default/files/publication_files/tup_peru-baseline_report_pdf.pdf

Information and Community Mobilization in Rural India

Policy Issue: 
While primary school enrollment rates have risen sharply in much of the world, the quality of education remains low in many countries. Many children who attend school regularly are still unable to read or do basic arithmetic. For instance, a 2008 survey found that in rural India only 56 percent of children in grade 5 could read at the grade 2 level, and nearly 20 percent could not read beyond a single word. Community oversight and participation has been advocated to increase education quality. Does this strategy work, and if so how can community participation be encouraged? Is more direct action by communities to teach their children to read more effective?
Context of the Evaluation: 
In Uttar Pradesh, India’s most populous state, only 43.5 percent of children in grade 5 can read at the grade 2 level. In response to this problem, the government established Village Education Committees (VECs) in every village in 2001. VECs consist of the elected head of the village government, the head teacher of the local school, and three parents who are nominated by their community. These committees are responsible for monitoring school performance, allocating school resources, and hiring additional contract teachers in the event of overcrowding. 
Despite the promising aspects of this program, a survey conducted in 2005 indicated that 38 percent of VEC members did not readily identify as being part of the committee, and 25 percent did not even know they had this role. Only 3.6 percent of all VEC members knew they had the ability to request funds to hire additional teachers, which is one of the main prerogatives and responsibilities of the VEC. 
Details of the Intervention: 
Working jointly with Pratham (a local NGO) and the World Bank, researchers designed three interventions that were randomly assigned in 280 villages in four rural blocks in Jaunpur district, eastern Uttar Pradesh, a populous and educationally struggling area in India. These interventions served to determine if more information and encouragement to use the channels available to them would cause VECs and community members to demand and receive better services. They contrasted this with direct action to improve learning outside the official channels. 
Intervention 1: In 65 villages, Pratham staff started a series of conversations about education in small groups throughout the community. These conversations covered the current status of schools in the village, the quality of local schools, state mandated provisions for schools, mid-day meals, and local funds available for education. People were asked if they knew about the VEC and its membership and responsibilities. After two days of meetings in small groups, a community-wide meeting was held where people were encouraged to discuss and ask for information about the VEC, with information gaps filled in by Pratham’s field workers. VEC members also received a pamphlet on their roles and responsibilities from the Pratham staff.
Intervention 2: In addition to all the steps outlined above, communities in another 65 villages were trained and encouraged to conduct testing to see if children could read simple text and solve basic arithmetic problems. Volunteers put together a "report card" for each community, which was presented at the community-wide meeting. 
Intervention 3: In addition to the above two steps, Pratham officers taught volunteers in another 65 villages a simple technique for helping children learn to read. Volunteers were encouraged to start after-school reading classes-- they were invited to attend training sessions which lasted for four days, and staff returned an average of seven times to provide in-service training. The objective was to use Pratham-designed materials and local volunteers to supplement the normal curriculum, and improve literacy among village children. 
Comparison: Eighty-five villages received no treatment, serving as a comparison.
Results and Policy Lessons: 
Impact on Information Gaps: The average effect of all three treatments was an increase of 7.8 percentage points in VEC members who knew they could access public funds, and a 13 percentage point increase of members who had been properly trained. Parents were also 2.9 percentage points more likely to know that a VEC existed in their community.
Impact on Engagement: Despite these improvements in awareness, there was little difference between the VECs’ performance in treatment and comparison villages. The only significant difference was that 20 percent more contract teachers were hired in Intervention 2 villages (although not in Intervention 3 villages). Also, the intervention did not increase the level of engagement of parents with schools. Parents were no more likely to have visited the school or to have volunteered time or money in the treatment villages than in the comparison villages.
Impact on Reading: In 55 of the 65 Intervention 3 villages, volunteers ran more than 400 reading courses. The average child in an Intervention 3 village who could not read anything at baseline was 7.9 percent more likely to be able to read at least letters. Those who could read only letters at baseline were 3.5 percent more likely to read at least paragraphs or words, and 3.3 percent more likely to read stories if they were in an Intervention 3 village. These changes in average literacy across the village came despite the fact that only 8 percent of children, including 13 percent of those who could not recognize letters prior to the intervention, attended the classes. Provided that the effects of Intervention 3 are channeled entirely through attendance at the reading classes, comparing the endline reading levels of the comparison group with the Intervention 3 treatment effects described above reveals just how large these effects are:  all children who could not read at baseline but attended classes ended up being able to read letters at endline, and 98 percent of children who could read at the word or paragraph level was able to read at the story level. 
Intervention 3 was the only intervention which actually improved educational outcomes, by empowering individuals to improve teaching in their own communities. This suggests that enabling local action which does not depend upon large-group participation may be a means of directly affecting educational outcomes.  

Community Driven Development in Sierra Leone


While the accountability and inclusiveness of institutions are often considered key determinants of economic performance, there is little agreement about how institutions should be designed. Researchers evaluated the impact of a community-driven development program in Sierra Leone designed to establish more inclusive and accountable local decision-making infrastructure by providing villages with small development grants to be allocated by village committees. Results demonstrated the program successfully established village-level organizations and tools to manage development projects, but there was no evidence that the program led to fundamental changes in local institutions or decision-making.

Policy Issue: 

While the accountability and inclusiveness of institutions are often considered key determinants of economic performance, there is little agreement about exactly how institutions should be designed, how to move from a system of bad institutions to one with good institutions, and whether and how foreign donors can help in this process. One of the most popular strategies employed by donors to promote democratic and accountable institutions at the local level is “community driven development” (CDD). Typical CDD interventions combine flexible grants that communities can spend on local projects with requirements that decisions must be made in an inclusive and transparent manner and training on how to do this. The participation requirements aim to ensure that the projects funded reflect the needs of the community and facilitate learning by doing—i.e. the experiences gained in deciding how to spend project funds leave minority groups  better placed to participate in other community decisions after the project ends. While billions of dollars are spent on CDD programs, few studies provide rigorous evidence on their real-world impacts. Critics of CDD, and of decentralization in general, have raised the concern that decentralized funds will be captured or exploited by local elites.

Context of the Evaluation: 

Scholars argue that frustrations with government incompetence and corruption, as well as the exclusion of women and young men from decision-making in the traditional chieftaincy system that coordinates the provision of many local public goods, fueled violence during Sierra Leone’s recent civil war.  To both prevent a return to violence and to stimulate economic development, the Government of Sierra Leone implemented a number of reforms that give communities, and vulnerable groups within them, a greater voice in local decision-making. Alongside a national decentralization program that re-established district-level councils, the government piloted a community-driven development project that went one step  further by providing small grants to be administered by village development committees. This extension down to the village level aimed  to establish more inclusive and accountable local decision-making infrastructure, rebuild trust, promote collective action, and provide minority groups (particularly women and youth) with experience in managing projects and making decisions  within their community. Researchers and the Decentralization Secretariat collaborated to evaluate whether this pilot, called the “GoBifo” Project (or “Move Forward” in Krio),  acheived these goals. 

Details of the Intervention: 

Two hundred thirty-six villages from two ethnically and politically distinct districts were randomly allocated into a treatment group or a comparison group. Villages in the treatment group were regularly visited by a GoBifo facilitator, who helped community members create or revamp Village Development Committees (VDCs), set up bank accounts for the VDCs, establish transparent budgeting practices, and create village development plans that included specifics on how GoBifo grants would be used. The participation and inclusion of marginalized groups was central to this process – for example, each social group (women, youth, and adult men) came up with their own development plan, and these plans were then combined into a single unified vision. Women were often established as treasurer of the VDC and served as co-signatories on all project finances. A series of block grants totaling US$4,667  per community were given to implement local public goods and skills training projects that were identified in the village development plans. 

Household surveys, which covered participation in local decision-making, attitudes to minorities, and engagement in collective action, as well as demographic and socioeconomic information, were collected in late 2005 and again in mid-2009, along with village-level focus group discussions. In addition, three structured community activities (SCAs) were conducted in late 2009, shortly after GoBifo activities had ended, to capture any persistent impacts on collective action, participation of minorities, and elite capture. The SCAs were designed to measure how communities responded to concrete, real-world situations in three areas where GoBifo had sought to change behavior: (i) raising funds in response to a matching grant opportunity; (ii) making a community decision between two comparable alternatives; and (iii) allocating and managing an asset that was provided for free. 

Results and Policy Lessons: 

At the start of the evaluation, the authors and project team agreed a set of hypotheses they would test at the start of the evaluation (in 2005) and wrote out a plan on exactly how the data would be analysed before looking at the data. This prevented selective “cherry picking” of results from the 318 variables collected.

Project Implementation and Local Infrastructure Investment: The GoBifo project successfully established the village-level organizations and tools to manage development projects in nearly all cases. The distribution of project benefits within communities was equitable, leakage of project resources minimal, and minority participation high. 

GoBifo villages had a larger stock of higher quality local public goods, such as a functioning primary school or community grain-drying floor, than comparison areas. There was also more market activity in treatment communities, including the presence of more traders and items for sale, suggesting short-run economic gains. 

Institutional Change and Collective Action: There is no evidence that the program led to fundamental changes in local institutions or decision-making. Despite the fact that many women in treatment villages participated in GoBifo decisions, they were no more likely to voice an opinion in community meetings after the project ended or to play a leadership  role in other areas. Similarly, the establishment of a democratically elected village development committee that carried out multiple projects did not lead treatment villages to be any more successful at raising funds in response to a later matching grant opportunity.  Lastly, there were no program impacts on elite capture, although levels of capture were low in the research communities (at least as measured by the third SCA). 

Debates: The Impact of Voter Knowledge Initiatives in Sierra Leone

While political debates are often considered an integral part of campaign strategy, there is little definitive evidence on whether they affect how people vote. In Sierra Leone, researchers partnered with the civil society organization Search for Common Ground to evaluate how the dissemination of political information through debates impacts voter behavior, campaign spending, and the performance of elected politicians. The study found that watching debates substantially increased political knowledge, policy alignment, and vote shares for higher-quality candidates. The debates also encouraged politicians to invest more in their constituencies, both during the campaign and one year later.

Policy Issue:

Political debates are often considered an integral part of campaign strategy in contemporary democracies. Debates can reveal information about the relative quality of candidates and their differing policy positions. By creating a public record of their pre-election commitments, they may enhance the subsequent accountability of elected officials. In developing countries where voters may have very little information about candidates, publicizing debates could have a large impact on electoral outcomes. Yet there is little evidence on whether debates affect actual voting behavior. To address this gap, researchers examined the impact of showing public debates leading up to the 2012 parliamentary elections in Sierra Leone on voter knowledge, voter behavior, and politician behavior. This research sheds light on the above topics. In addition, it investigates if information swings voters from voting based on ethnicity to voting based on policy.

Context of the Evaluation:

Voting patterns in Sierra Leone have historically been based on ethnic ties and pre-existing party affiliations; the two largest political parties are closely associated with ethnic groups in the north or south. The country has 112 Parliamentary constituencies, in which one Member of Parliament (MP) represents approximately 40,000 residents in the national legislature. Search for Common Ground (SFCG), a nonpartisan civil society organization, operates programs designed to promote civic engagement and dialogue to build sustainable peace. The 2012 Parliamentary elections presented an opportunity for SFCG to test new electoral programs that could increase transparency, voter knowledge of candidates, and engagement between voters and MPs.

Details of the Intervention:

In the run-up to the November 2012 parliamentary elections in Sierra Leone, researchers partnered with SFCG to evaluate how the dissemination of political information through debates impacted voter knowledge and behavior, campaign spending, and the performance of elected politicians.

After identifying what they predicted would be 28 relatively competitive races, SFCG organized and filmed debates organized in a standardized format between the major candidates in a randomly selected half of these constituencies. After the debates, both the debate audience and an expert panel of government and civil society representatives separately determined the debate winner. From a total of 224 polling centers in the 14 constituencies where debates were filmed, 112 were randomly assigned to host public screenings of a video of the debate, between one and five weeks before the election. In total, an estimated 19,000 people viewed the videos. Surveys of voters before and after—in some cases only after—the screenings measured how their perceptions of candidates, knowledge of candidates, and voting intentions changed.

In a separate group of forty polling centers, the research team showed the debates to individual voters in order to test the effects of different kinds of information, such as policy positions, personal characteristics, or persuasive speeches, on voter behavior. Voters were randomly assigned to one of the following groups:

1.)   Full debate: 400 individuals watched the entire video of the debate on a personal handheld device.

2.)   Get-to-know-you only: 400 individuals watched a video clip of the candidates speaking informally about their hobbies and interests.

3.)   Radio report: 400 individuals listened to a journalistic summary of the main policy positions articulated by the candidates during the debates.

4.)   Surveyed comparison: 600 individuals were surveyed before the election, like those in groups 1-3, but were not shown any media.

5.)   Pure comparison: The rest of the voters registered at these polling centers viewed no media and were not surveyed until after Election Day.

On or immediately after Election Day, researchers conducted exit polls among a random subset of all voters. They used these polls, as well as the Election Commission’s official voting records, to measure voting outcomes. Once the elected MPs took office, they tracked their performance in both treatment and comparison constituencies.

Results and Policy Lessons:

Overall, exposure to the debates significantly increased voters’ political knowledge and changed their voting behavior. Candidates who participated in the debates increased campaign expenditures in communities where the debates were screened. The debates also caused politicians to engage with and invest more in their constituencies once they were in office.

Voter knowledge: The public debate screenings led to substantial improvements in voter knowledge, including general political knowledge as well as knowledge of specific candidates and their policy positions. For example, the proportion of voters who knew which candidate (if any) had been an MP in the past increased by 11 percentage points (from 49 percent to 60 percent) and the proportion of voters who could correctly identify candidates’ top priority for government spending doubled (from 14 to 29 percent).

Voting choices: Exposure to the public debate screening significantly increased policy alignment, measured as a match between the voter’s reported policy position and that of the candidate they voted for. Debate viewers were 9 percentage points more likely (from 43 percent to 52 percent) to have voted for a candidate whose top priority issue aligned with theirs. Policy alignment improved not only because voters became more likely to select candidates with views similar to their own, but also because voters changed their views to match those of their preferred candidate after watching them speak.

The debate screenings also made some voters more likely to vote for high-quality candidates. There was a 5-percentage-point increase in votes for the debate winner (as judged by the audience at the debate) relative to the comparison group. This increase was in part driven by voters who switched votes across ethnic-party lines when the debate winner was from a rival party. For voters historically aligned with the party of the debate winner, the screening had no effect on their vote choice.

The individual treatment arms aim to unpack which types of information—from “hard facts” about policy to more intangible information about charisma—drive these changes in voter behavior.  All three treatments (the full debate, radio report, and get-to-know you video), increased political knowledge, although the get-to-know-you was (as expected) only half as effective. Only the full debate, however, had a significant impact on voters’ policy alignment with their preferred candidates and their vote choice. The fact that the radio report treatment was as effective in building knowledge as viewing the debate, but only the full debate treatment impacted policy alignment, suggests that personality plays a key role in persuading voters.

Campaign spending: In communities that viewed the debates, candidates subsequently increased their level of campaign spending relative to the comparison group. Voters reported receiving more and more valuable gifts from the particular candidate, and the candidate visited the communities significantly more than in comparison communities.

Performance of elected officials: MPs who participated in the debates spent 2.5 times as much on verifiable development expenditures for their constituency. This corresponded to average gains of roughly US$6,000 per constituency from a base US$4,070 spent in comparison constituencies, and was close to the total of US$11,000 that MPs had available. In addition, these MPs conducted an average of 4.2 community visits, compared to 2.9 in the comparison group, and held 2.1 public meetings, compared to one in the comparison group. Constituents of MPs that participated in the debates were more likely to say their MPs were doing “a good job in promoting” various sectors in their constituency.  However, MPs who participated in the debates were not more likely to promote the sectors that they had ranked as  their top priority during the campaign, nor were they more active participants in Parliament than those who had not participated in the debates.

Policy implications: In sum, this research suggests that publicizing candidate debates can significantly increase voters’ political knowledge, which can then influence their vote on Election Day. Debates may also encourage politicians to invest more in their constituencies, both during the campaign and after they are elected.  Finally, this project demonstrates that publicized interparty debates can be a cost-effective and logistically feasible method of improving voters’ knowledge; fixed production costs for the debates were roughly US$5,000 each. In settings with higher mass media penetration, debates could be televised, reducing dissemination costs.

Northern Uganda Social Action Fund – Youth Opportunities Program

Youth unemployment is a persistent problem in the developing world, particularly in post-conflict settings, posing both economic and security issues. In growing, stable economies such as Uganda, what holds back youth from reaching their potential?  One theory suggests that youth unemployment is due primarily to the lack of sufficient capital to support entrepreneurship. If this is true, cash transfers or cheap credit could lead to a burst of self-employment. Evidence from other areas, such as studies on microcredit, suggests that alleviating these constraints with loans has little effect on earnings. In Northern Uganda, which is returning to peace after twenty years of war, the government’s Youth Opportunities Program offered cash transfers to groups of youth to increase employment and reduce conflict. Follow-up surveys two and four years later found a shift from agricultural work towards skilled trades and strong increases in income. Women in particular benefited from the cash transfers, with incomes of those in the program 84% higher than women who were not. There were no differences, however, in social outcomes such as community participation, aggression, and social cohesion.

See the full paper herea policy note for the World Bank here, and Chris Blattman’s blog discussion here.

Policy Question:
In developing countries, high unemployment - particularly among youth - is a pressing concern. Jobs, particularly higher-skilled labor and productive small enterprise, provide incomes and reduce poverty. For governments, transitioning from an economy based on small-scale agriculture to one based on entrepreneurship and production is critical for long-term growth. Employment is also seen as important for building social stability and political engagement in communities uprooted by long-term conflict.
One form of intervention offers cash in the hopes that youth will invest it in the training and assets to learn a trade or form a business. In the development community, anxiety persists over whether this is an effective approach: will youth with little or no financial or business training be able to direct the money towards successful long-term entrepreneurship?  Previous research also raises questions about the ability of women in particular to invest aid into increasing lifetime earnings, given occupational constraints and pressure to share windfalls.1
Uganda’s largest employment program sought to test if an intervention as simple as giving cash could help accomplish the country’s long-term economic and social goals for its youth.
Context of the Evaluation:
Twenty years of insurgency, instability and conflict led to high rates of poverty and unemployment in northern Uganda, but by 2005 a measure of peace and stability had returned to the region. The centerpiece of the post-conflict recovery plan was a decentralized development program, the Northern Uganda Social Action Fund (NUSAF). In 2006, to stimulate employment growth through self-employment, the government launched a new NUSAF component: the Youth Opportunities Program (YOP), which provided cash transfers to groups of young adults with the goal of encouraging trade-based self-employment. 
Description of the Intervention:
The YOP intervention had two official aims: to raise youth incomes and employment and to improve community reconciliation and reduce conflict. The program, targeted at youth from ages 16 to 35, required young adults from the same town or village to organize into groups and submit a proposal for a cash transfer to pay for: (i) fees at a local technical or vocational training institute of their choosing, and (ii) tools and materials for practicing a craft.
The average applicant group had 22 members. Group cash transfers averaged nearly UGX 12.8 million (US$7,108), and varied by both group size and group request. The average transfer size per member was UGX 673,026 (US$374) – more than 20 times the average monthly income of the youth at the time of the baseline survey.
Due to vast oversubscription, the 535 eligible groups were selected at random, using a lottery, to either receive the YOP program or be part of the comparison group. A baseline survey was conducted with 2601 individuals in 2008, and 87 percent were successfully followed and interviewed in the endline surveys two and four years later.
Results and Policy Lessons:
Overall, the program seemed to have strong economic effects. Four years later, beneficiaries of the YOP program had 41% higher income and were 65% more likely to practice a skilled trade, such as carpentry, metalworking, tailoring, or hairstyling. Hours worked were 17% higher, nearly entirely accounted for by these new professions – while most still farmed part-time, hours spent in agriculture were not different. They were also 40% more likely to keep records, register their business, and pay taxes.
Within the sample, gains were highest for those who had the highest initial credit constraints, those with fewest initial assets and access to loans. The effects were particularly strong for women. Women who received the cash grants four years later had 84% higher incomes than women who did not, while men were earning 31% more than their counterparts in the comparison group. This gender difference may reflect particular capital constraints faced by women.
While employment programs including this one are often implemented by governments with the aim of reducing social instability or promoting cohesion, the data show no evidence for impacts in these domains. After four years there were no measurable differences in cohesion, aggression, or community and political participation between participants in the YOP program and those in the comparison group.
Overall, the data show that the poor used the money effectively; investing in training and tools needed to start businesses and experienced a significant growth in income, even after four years. Even though impacts in social domains were negligible, the economic outcomes show the potential of alleviating capital constraints for spurring economic growth among the poor. 
Read the full paper here.
A midterm policy report here and policy note by the World Bank here were based on the initial 2-year follow up data.
[1] Fafchamps, M., McKenzie, D., Quinn, S., Woodruff, C., 2011. When is capital enough to get female microenterprises growing? Evidence from a randomized experiment in Ghana. Unpublished working paper.

Graduating the Ultra-Poor in Ethiopia

More than one fifth of the world’s population lives on less than US$1.25 per day. While many credit and training programs have not been successful at raising income levels for these ultra-poor households, recent support for livelihoods programs has spurred interest in evaluating whether comprehensive “big push” interventions may allow for a sustainable transition to self-employment and a higher standard of living. To test this theory, researchers evaluated a globally implemented “Graduation” approach to measure its impact on the lives of the ultra-poor. They found that the approach had long-lasting economic and self-employment impacts and that the long-run benefits outweighed their up-front costs. Here we summarize the Ethiopia site, which had similar effects as the other successful sites.

Policy Issue:

More than one fifth of the world’s population lives on less than US$1.25 per day. Many of these families depend on insecure and fragile livelihoods, including casual farm and domestic labor. Their income is frequently irregular or seasonal, putting laborers and their families at risk of hunger. Self-employment is often the only viable alternative to menial labor for the ultra-poor, yet many lack the necessary cash or skills to start a business that could earn more than casual labor.

In the past, many programs that have provided ultra-poor households with either credit or training to alleviate these constraints have not been successful at raising household income levels on average.  However, in recent years, several international and local nongovernmental organizations have renewed their support for programs that foster a transition to more secure livelihoods. Combining complementary approaches—the transfer of a productive asset, training, consumption support, and coaching— into one comprehensive program may help spur a sustainable transition to self-employment. To better understand the effect of these programs on the lives of the ultra-poor, researchers coordinated to conduct six randomized evaluations in  Ethiopia,  GhanaHondurasIndiaPakistan, and Peru.

Context of the Evaluation:

In Ethiopia, researchers partnered with the Relief Society of Tigray (REST) and the Dedebit Credit and Savings Institution (DECSI). The study focused on participants in Ethiopia’s food-for-work program who belonged to households without any outstanding loans and with at least one member capable of work. A local food security task force narrowed eligibility further by choosing those they considered to be the poorest members of their community. Within the sample, the median total per capita consumption was 2014 PPP US$1.22 per day, with two-thirds of households consuming less than US$1.25 per day. Two-thirds of households reported that not everyone in the household got enough food.

Details of the Intervention:

In partnership with local implementing organization REST and microfinance institution DECSI, researchers conducted a randomized evaluation to test the impact of a two-year comprehensive livelihoods program (“the Graduation approach”) on the lives of the ultra-poor. This approach was first developed by Bangladeshi NGO BRAC in 2002 and has since been replicated in several countries. From a sample of 925 households, researchers randomly assigned half to the treatment group, and half to the comparison group, which would not be eligible to receive the program.

The intervention consisted of five complementary components, each designed to address specific constraints facing ultra-poor households:

1. Productive asset transfer: One-time transfer of a productive asset valued at 4,724 Birr (2014 PPP US$1,228). Most (62 percent) of participants chose sheep and goats, while 24 percent selected oxen and 10 percent selected bees.

2. Technical skills training: Training on running a business and managing their chosen livelihood. For example, households who selected livestock were taught how to rear the livestock, including vaccinations, feed and treatment of diseases.

3. Consumption support: Regular food support is a component of the Graduation approach, but in this study it was not unique to the treatment group. The entire sample received food support (valued at a maximum 2014 PPP US$26 per month) through a separate food-for-work program.

4. Savings: Households had DECSI bank accounts opened and were required to regularly deposit savings totaling 4,724 Birr (2014 PPP US$1,228) over the two years of the program; households were unable to withdraw funds until they reached this threshold.

5. Home visits: Weekly home visits by REST staff to provide accountability, coaching, and encouragement.

The Graduation program began in 2010 and continued until May 2012. Researchers conducted a first endline survey immediately after the program ended, as well as a second endline survey around one year later.

Results and Policy Lessons:

Across all six countries, researchers found that the program caused broad and lasting economic impacts. Treatment group households consumed more, had more assets, and increased savings. The program also increased basic entrepreneurial activities, which enabled the poor to work more evenly across the year. While psychosocial well-being improved, these noneconomic impacts sometimes faded over time. In five of the six studies, long-run benefits outweighed their up-front costs. In Ethiopia, specifically, researchers found similar effects:

Economic impacts: One year after the Graduation program ended, average total monthly consumption among treatment households was 2014 PPP US$47.50, an 18.2 percent increase over households in the comparison group. Food and durable goods spending were also higher than in the comparison group, and more households reported having enough food every day. An asset analysis also revealed larger values for both productive and household assets a year after the program ended. Treatment households had 2014 PPP US$2,657 in total asset values, 68 percent more than comparison group households. For treatment group households, measures of financial inclusion also increased. Graduation program households reported borrowing and saving at higher rates than comparison group households, with a 372 percent increase in average total savings to 2014 PPP US$345.

Self-employment:One year after the Graduation program ended, treatment group households reported spending 43 minutes per day on productive activities in addition to the average four hours among comparison group households. Graduation households also experienced a two-fold increase in livestock revenue relative to comparison group households.

Psychosocial wellbeing:Generally, the Graduation program did not affect measures of physical or mental health. There were no changes in illness, happiness, or stress, although treatment group households did report a smaller likelihood of feeling anxious or worried in the last year.

Political Involvement:There is some evidence that participation in the Graduation program increased political involvement. One year after the program ended, 38 percent of treatment group households reported being a member of a political party (compared to 33 percent for the comparison group) and 57 percent attended a village meeting in the last year (compared to 52 percent for the comparison group). Participation in the Graduation program did not improve women’s decision-making power.

Cost-benefit analysis: Researchers calculated total implementation and program costs to be US$884 per household (2014 PPP US$4,157). However, estimated benefits of consumption and assets growth amount to 2014 PPP US$10,805 per household, representing an overall 260 percent return on investment.



Does Development Aid Undermine Political Accountability in Bangladesh?

There is little evidence to lend credence to or discredit the argument that development aid undermines political accountability. In Tanore district of Bangladesh, researchers tested the impact of providing external subsidies for sanitation projects on the behavior of local leaders and, subsequently, on constituents’ perception of their performance. They found that while constituents credited leaders for the sanitation program when the funding source was unknown, they did not credit local leaders when funding information was made transparent.

Policy Issue:

There are conflicting arguments about whether foreign aid benefits developing countries. Some argue that more aid is needed to address poverty, while others believe thataid undermines political accountability, making it hard for voters to distinguish between bad and good leaders. The latter theory assumes that constituents have difficulty distinguishing between the effects of leadership skill and externally-financed development aid. As a result, they are likely to give undue credit to their leaders for development programs that are actually supported by external aid. While some existing evidence supports the claim thatconstituents cannot always separate the actions of leaders from unrelated factors that increase community or individual well-being, there are also theories that suggest leader behavior and constituent perceptions of leaders’ performance may affect each other. Can providing information to constituents about the source of a benefit to their household or community help them correct misconceptions about their leaders’ performance?

Context of the Evaluation:

The study was conducted in the rural areas of the Tanore district in the north-west of Bangladesh. Though sanitation coverage has increased over the last decade, the district still lags behind in sanitation indicators.Of households surveyed, 31 percent reported open defecation or an unimproved latrine as their defecation site, and only 34 percent owned or had regular access to a hygienic latrine. In response to these sanitation issues in Bangladesh, organizations such as Wateraid- Bangladesh and Village Education Resource Center (VERC) work in collaboration with the poorest and most marginalized communities to set up lasting solutions to water, sanitation and hygiene problems.

This study was conducted in four of seven sub-districts (unions) within the Tanore district. Within each union, the highest level local leader is the Union Parishad (UP), who is elected by his constituents. The experiment covered all communities in these four unions to ensure that the program was concentrated enough in a geographic area to affect and track leader behavior.

Details of the Intervention:

Researchers conducted a large scale randomized evaluation  to test how leaders respond to externally funded sanitation programs in their communities, and how constituents subsequently assess their local leaders’ performance. The study sample consisted of 16,603 households in 97 villages. Each sub-district consisted of approximately 25 villages, and each village had 150-220 households. 

Villages were randomly assigned to receive one of two interventions: (1) a community motivation campaign, called the Latrine Promotion Program (LPP) or (2) the LPP plus subsidies for the purchase of hygienic latrines. A third randomly selected group received neither program and served as the comparison group.

Latrine Promotion Program (LPP):Implemented in collaboration with Wateraid- Bangladesh and Village Education Resource Center (VERC), LPP involved a multi-day exercise run by VERC health monitors to raise awareness of the problems of open defecation and non-hygienic latrines. Emphasizing that sanitation is a community level problem, the program focused on ending open defecation, and doing so through the use of hygienic latrines. 

LPP + Latrine Subsidies: A subsidy that covered 75 percent of the costs of the parts to install one of three types of hygienic latrines was randomly assigned by lottery to households in subsidy villages. Households were eligible to receive a subsidy if theyowned 50 decimals of land or less, which represents the poorest 75% of the population in these villages. Vouchers were distributed by public lottery approximately two weeks after the LPP campaign took place.

In order to test whether households update their beliefs based on new information, researchers ran a second information campaign in subsidy villages following the LLP and latrine voucher lottery. A subset of households was randomly selected to hear one of two scripts with details about the sanitation program. The first script attributed the intervention to a research project without explicitly mentioning the role of the leader, while the second script explicitly stated that the government played no role in funding the intervention or in selecting beneficiary villages.

Results and Policy Lessons:

Leaders responded to having an externally funded toilet subsidy program in their villages, and were more likely to spend time in subsidy villages. Reported satisfaction with leaders in subsidy villages was higher relative to villages only receiving the LPP program.  Nevertheless, while constituents credited leaders for the sanitation program when the funding source was unknown, they did not credit local leaders when funding information was made transparent.

Village level results:

Leader behavior:Leaders spent more time in subsidy villages after the program was implemented. Leaders showed up more often in villages where subsidies were implemented and also interacted more with the community once they showed up – constituents in LPP + subsidy villages were 10 percentage points more likely to have seen or interacted with their leaders than in LPP only villages. This supports the hypothesis that certain leaders respond to externally funded programs in their community, which in turn may affect constituent perception.

Constituent perception: Villages receiving only LPP showed some increases in sanitation investments, and greater overall satisfaction with access to sanitation relative to comparison villages. Moreover, providing subsidies led to greater investment in sanitation and even greater satisfaction.However,LPP communities became less satisfied with leader performance relative to comparison villages. On a scale of 1-10, reported satisfaction with leaders in LPP villages was 0.6 points less than in comparison villages. Researchers hypothesize that the information campaign armed constituents with greater knowledge of the problems of open defecation and highlighted the need for community level investment, which in turn led to greater political accountability. In contrast to the LPP only program, providing the subsidy had a significant positive effect on constituent perception of leader performance – reported satisfaction in subsidy and LPP villages was 0.6 points higher on the 10 point scale than in LPP-only villages.

Household level results:

Leader behavior:Within subsidy villages, winners of the latrine voucher were no more likely to see leaders than lottery losers. When it was clear that the allocation of vouchers was due to lottery, leaders had no incentive to spend more time with lottery winners over losers.

Constituent perception:  Within subsidy villages, winners are no more likely than losers of the subsidy to attribute credit to their leaders for meeting their sanitation needs. Constituents understand that the benefits of the public lottery are due to random chance. Additionally, informing villagers about the true source of the subsidy appeared to eliminate the excess credit that the residents of subsidy villages previously assigned to leaders. Hence, constituents incorrectly attributed credit to leaders when the source of a program remained uncertain;however, when villages were informed of the true source of the subsidies, households no longer assigned credit to leaders when credit is not due. These results indicate that political accountability is not easily undermined by development aid when information is transparent.

Campaigns Against Vote-Selling in the Philippines: Do Promises Work?

Vote-buying and vote-selling obstruct the democratic process, yet they remain pervasive in many developing democracies. Researchers asked voters in the Philippines to make a simple, unenforceable promise not to accept money from politicians or to promise to vote according to their conscience, even if they do accept money, to test the impact of promises on voters’ behavior. A majority of respondents made promises not to sell their votes. Researchers found that the promise significantly reduced vote-selling, cutting the number of people who sold their votes by 11 percentage points in the smallest-stakes election, but was not effective in the mayoral election with higher pay-outs. These results suggest that simply asking voters to promise not to sell votes can help reduce vote-selling in elections where vote-buying payments are typically small.

Policy Issue:

Vote-buying and vote-selling obstruct the democratic process, yet they remain pervasive in many developing democracies. In response, governments, international donors, and NGOs worldwide have directed significant attention and resources towards curbing vote-buying and vote-selling. Some strategies have aimed to stop politicians from offering money in exchange for votes, but this approach has often failed due to poor implementation and weak enforcement. Therefore, it has become common to focus anti-vote-buying efforts on voters themselves. This often involves asking voters to make promises or sign pledges to not accept money from politicians or their agents prior to elections. Another approach encourages voters to take the money being offered, but still “vote their conscience.” This evaluation aims to shed light on the impact of these two types of anti-vote-selling campaigns.

Context of the Evaluation:

The Philippines is an electoral democracy, but corruption and a lack of transparency continue to undermine democratic development. Elections have historically been marred by fraud, intimidation, and political violence.1 While vote-buying and selling has decreased in recent years, it is estimated that about 30 percent of Filipinos were offered money by a politician or local leader during the 2010 election campaign.

 Vote-buying is widespread in Sorsogon City, where this study took place. Most vote-buying in Sorsogon City occurs in the week leading up to election day. Candidate representatives approach households directly, offering money or goods in exchange for their vote. The size of payments differed differs greatly across races. In the mayoral races, payments per voter typically amounted to 250 to 500 Philippine pesos (US$5.57-$11.14), while those for city council races were in the range of 20 to 100 pesos (US$0.45-$2.23).

Details of the Intervention:

To evaluate how making promises impacts voters’ willingness to sell their votes, researchers partnered with IPA to conduct a randomized evaluation with 900 registered voters in Sorsogon City, the Philippines around municipal elections.

Before the municipal elections, all respondents were shown a three-minute video clip on a hand-held device. The video clip was part of a voter education campaign that used humor to encourage viewers to turn out to vote, vote for honest and competent candidates, and avoid vote-selling.2 All respondents were then asked to rank political candidates, indicating for whom they wanted to vote in the upcoming elections, and randomly assigned to one of three groups:

1.     Don't Take Money group: After watching the video, respondents were asked to make a promise not to take money from politicians.

2.     Vote Your Conscience group: After watching the video, respondents were asked to vote according to their conscience, even if they take money from politicians.

3.     Comparison group: These respondents only watched the video.

Following the elections, respondents were surveyed again.  Due to concerns respondents would misreport their true voting behavior, respondents were not asked directly if they had sold their votes. Instead, surveyors asked who they had voted for, and then compared their responses to those in the initial survey. Researchers used vote-switching as a proxy for vote-selling.

Results and Policy Lessons:

Overall, a majority of respondents made promises not to sell their votes. Among those who made promises in the city council race, where payments for votes are low, vote-selling decreased by 11 percentage points. However, in the mayoral race, where payments are higher, there was no change in vote-selling or vote-switching. This suggests that promises are not enough to make people feel committed to not accept large cash payments in exchange for voting one way or another.

Telling voters that they could accept money but should still vote according to their conscience did not reduce vote-selling in either race—in fact, it increased vote-selling. In the comparison group, 57 percent of respondents switched their vote at least once, compared to 50 percent in the promise group, and 62 percent in the conscience group (in both races).

These results suggest that simple interventions—such as asking voters to promise not to sell their vote—can help reduce vote-selling in small stakes elections where payments for votes are small. These results also suggest that telling voters they can accept money but should still vote according to their conscience may be counterproductive.

Related Paper Citation:

Hicken, Allen, Stephen Leider, Nico Ravanilla, and Dean Yang. "Temptation in Vote-Selling: Evidence from a Field Experiment in the Philippines." (2014).

[1]National Democratic Institute. Where we work: The Philippines

[2]The video clip features Mae Paner, a political activist and actress, as the fictional character “Juana Change.” 

The Impacts of Psychosocial Support and Cash for Work on Vulnerable Youth in Liberia

To foster economic productivity and stability in post-conflict countries, many development organizations have created economic programs and psychosocial programs for youth. Yet little evidence exists on ways to increase economic opportunities for young people and reduce their risk of participating in violence and risky behaviors. In this study, researchers evaluate the impact of both a cash-for-work program and a psychosocial support program for young people in urban Liberia. This research will contribute evidence on how to best enable highly vulnerable youth in post-conflict settings to pursue productive opportunities.

Policy Issue:

Youth development programs are widely viewed as a critical tool for preventing the re-escalation of violence in post-conflict countries. Ninety percent of violent onsets occur in countries with a previous conflict,1 and when conflicts end, young people have the potential to contribute to future development, but they may also be a force of instability in their societies.2 In situations of armed conflict, youth are more likely to be recruited into fighting forces, become targets for violence, be forced to generate a livelihood for themselves and others, and miss out on an education.3 Many organizations therefore identify youth development as an urgent priority for building peace and spurring economic development, and investment in such programs has risen substantially in recent years.4  Yet little evidence exists regarding the impact of these programs, and which interventions are most effective in increasing economic opportunities for youth or reducing their risk of participating in violence and risky behaviors. This study will contribute evidence to help fill this gap.

Context of the Evaluation:

Liberia emerged out of a 14-year long civil war in 2003, and the security situation remains fragile and economic development has been slow.5 In Liberia, where people between the ages of 15 and 35 years constitute over 60 per cent of the total population,6 the government has identified youth unemployment as a major challenge.7

Mercy Corps, a global aid agency, is implementing a three-year program in Liberia called Promoting Sustainable Partnerships for Economic Transformation, or PROSPECTS. The program combines life skills training and workforce development via apprenticeships, technical training, financial education and mentoring, and aims to help participants find gainful employment while also developing positive social ties with their communities. Mercy Corps has worked in Liberia since 2003.

PROSPECTS includes a Cash for Work (CFW) program that centers on paying youth to collect recyclables in their communities and a Sports for Change (SFC) program that combines sports with life skills sessions for groups of youth. These interventions are designed with the aim of preparing highly vulnerable youth in Monrovia, Liberia for the employment market by developing self-confidence and resilience.

Details of the Intervention:

This study tests the impacts of the Cash for Work and Sports for Change programs, and any synergistic effects of both programs, on risky and violent behaviors and labor outcomes of vulnerable youth. Researchers are carrying out the randomized evaluation with 3,000 out-of school youth, ages 16 to 25, in urban Monrovia.

Through a public registration and lottery, participants were randomly assigned to a group of peers. There were a total of four groups, which serve to compare different programmatic options:

  • Cash for Work program only
  • Sports for Change program only
  • Both Cash for Work and Sports for Change
  • No program

Both programs will run for approximately three months with two sessions typically held per week. Researchers will measure the different programs, and any synergistic effects of the two, on enabling vulnerable Liberian youth to become economically productive adults, and resist risky behaviors over a one-year period. Additionally, researchers will evaluate the cost-effectiveness of the programs, both together and separately.

Data will be collected on the participants’ stress levels (measured using biometric data), risk preferences, labor outcomes, aggression, and self-reported risky behaviors. Social network mapping – how youth interact with one another – will also be used to provide a rich perspective of linkages and ties of Liberian youth within and outside of their traditional communities.


Community Driven Development in the Philippines

Community-based approaches to development, also called community-driven development (CDD), seek to empower local communities to identify and implement the projects they most need. Researchers in this study in the Philippines are evaluating the impact of a national community-driven development program on governance, social capital, and socio-economic welfare.

Policy Issue:

Community-driven development (CDD) has become an increasingly common tool used by governments to address the needs of poor communities.  The CDD approach is characterized by the movement of responsibility over resources and planning decisions to local decision-makers in an effort to more accurately and efficiently identify the needs on the ground.  Empowering communities to take charge of their own development may also lead to long-term effects on how they perceive their own role in governance, with improvements in accountability, transparency, and the quality of decisions.   This study endeavors to provide an independent assessment of the impact of a national community-development program and to contribute to broader research about the socio-economic, governance, and social capital impacts of CDD programs.

Context of the Evaluation:

A keystone poverty reduction initiative of the Government of the Philippines is the Kapit-bisig Laban sa Kahirapan-Comprehensive Delivery of Social Services (KALAHI-CIDSS, or KC). KC is a CDD program implemented by the Department of Social Welfare and Development of the Philippines (DSWD).  KC targets poor communities in the country’s 48 poorest provinces (out of 81).  In 2011, KC received US$120 million in funding from the United States government’s Millennium Challenge Corporation Compact in the Philippines and $59 million in loan funding from the World Bank.  The MCC contracted IPA to carry out an impact evaluation of the program.

Details of the Intervention:

This study will evaluate the impact of the KC program on multiple outcomes, including access to key services; the quantity and quality of participation in local governance around decision-making and implementation; and knowledge and awareness of local governance.

The survey sample consists of 198 villages (one randomly selected per municipality) and 5,940 households (30 randomly selected per village). The 198 municipalities were paired within each province based on similar characteristics (99 pairs) and then randomly assigned through public lotteries to receive the KC program or serve as a comparison group.

The KC program trains the communities and their local governments, both at the village and municipal levels, in choosing, designing and implementing public projects called “subprojects”. This is done through a five-stage program known as the KC Community Empowerment Activity Cycle (CEAC). Roughly one-third of villages receive subprojects each year, although some villages may receive multiple subprojects and others none over the course of the project. Most subprojects are programmed to be implemented within six months, thus the stages of preparation, funding and implementation generally take nine to twelve months and are called a cycle. The same process is repeated over three one-year cycles, with cycles two and three having a condensed CEAC phase since communities have already become familiar with the project and process.

Data for the estimate of outcomes and impacts of KC will come from multiple rounds of data collection exercises, each involving several instruments including a household survey, village survey, a Structured Community Activity, qualitative focus groups, and qualitative key informant interviews. Each round of data collection will be implemented by Philippines-based survey firms and overseen by the Millennium Challenge Account-Philippines and IPA.

Results and Policy Lessons:

Results forthcoming.


The Impact of Women Policy Makers on Public Goods in India

Policy Issue:

In 2008, women accounted for 18 percent of parliament members worldwide, and only 13 countries had a female head of government. This underrepresentation has prompted governments to adopt reservation policies; assuring women will have a role in political leadership by reserving a certain proportion of seats for female candidates. Reservation policies clearly have a strong impact on women’s representation, and some evidence suggests that women and men have different policy preferences. However, very little is known about the actual impact of women’s representation on policy decisions.

Context of the Evaluation:

In 1993, a constitutional amendment in India called for a random one third of village council leader, or pradhan, positions to be reserved for women. The village council, which encompasses between five and fifteen villages, is responsible for the provision of local infrastructure – such as public buildings, water, and roads – and for identifying government program beneficiaries. Although all decisions in the village council are made by majority, the pradhan is the only full-time member and exercises significant control over the final council decisions. The village council is required to organize two village meetings per year, during which they present their proposed budget and report on their activities in the previous six months. The pradhan must also set up regular office hours where villagers can lodge complaints or requests.

Details of the Intervention:

Researchers studied the policy consequences of mandated representation by determining whether there was any difference in the provision of social services between male and female led village councils. As village councils were randomly selected to be reserved for women, differences in investment decisions can be attributed to the reserved status of the council.

Data was collected in two locations: Birbhum in West Bengal and Udaipur in Rajasthan. In Birbhum, data was collected in two stages. First, in each village council, researchers conducted an interview with the pradhan, asking about his or her family background, education, previous political experience, political ambitions, and the village council’s recent activities. In the second stage, from the 5-15 villages represented by each village council, three villages were randomly selected to be surveyed for available public goods and existing infrastructure. Researchers also collected minutes of the village meetings, and gathered data on what complaints or requests had been submitted to the village council in the last six months. Two years later, the same village level data was collected from 100 village councils in Udaipur. However, there were no pradhan interviews.

Results and Policy Lessons:

The results suggest that reservation affected policy choices. In particular, it affected policy decisions in ways that seem to better reflect women’s preferences. In West Bengal, women complained more often than men about drinking water and roads – 31 percent of women’s complaints were about drinking water and 31 percent were about road improvement, compared to 17 percent and 25 percent of men’s, respectively. These preferences were revealed in the investment decisions of reserved village councils. Village councils reserved for women, on average, invested in 9 more drinking water facilities and improved road conditions by 18 percent.

In Rajasthan, 54 percent of women’s complaints were about drinking water and 19 percent were about welfare programs, compared to 43 percent and 3 percent of men’s, respectively. Unlike in West Bengal, compared to men, women complained less frequently about roads. Only 13 percent of women’s complaints were about roads, compared to 23 percent of men’s. This breakdown of preferences was again revealed in the investment decisions of the village councils. Village councils reserved for women invested in 2.62 more drinking water facilities, on average, and made fewer improvements in road conditions, leading to an 8 percent deterioration.

Researchers investigated whether these results could be explained by the fact that female pradhans were inexperienced, that they perceived themselves as being less likely to be re-elected, or that they tended to come from more disadvantaged backgrounds than men. However, there was no evidence that the policy impact of the reservations was driven by features other than the gender of the pradhan.

Overall, these results indicate that a politician’s gender, and identity more generally, does influence policy decisions. This finding is likely to have implications beyond reservation policy.

Selected Media Coverage:

Improving Health Service Delivery Through Community Monitoring and Non-Financial Awards

We know little about whether non-financial incentives can improve the performance of health care workers in countries with cash-strapped health systems. In this study, IPA introduced two types of non-financial incentives to randomly selected health clinics in Sierra Leone—a bottom-up community monitoring program and a top-down non-financial awards competition–-to measure the impacts different types of non-financial incentives can have on health worker performance.

Policy Issue:

In many developing countries, the health sector suffers from a severe human resources problem due to staff shortages and absenteeism. The availability of health care workers is a crucial element of quality care and the existing high levels of absenteeism represent a major leakage in health sector resources. Policy-makers have focused their attention on performance-based financing to incentivize attendance and performance monetarily; however, the evidence on the impact of financial incentives in improving performance in the health sector is mixed. While some programs report positive results, others show little to no effect on attendance and outcomes.

In contrast, recent results have highlighted the power of non-financial incentives to reduce absenteeism and improve performance.  Evidence suggests that peer recognition and status-based incentives can be more motivational, less expensive and less likely to erode intrinsic motivation.[1]  In addition, another study implementing a community monitoring initiative in Uganda, in which community members and health workers jointly addressed obstacles to adequate healthcare provision. The study found that under-five mortality was 33 percent lower in treatment compared to comparison communities a year later, while utilization for general outpatient services was 20 percent higher.[2]

Yet, the finding that non-financial incentives such as community monitoring improve clinic performance leaves a crucial question unanswered: does community monitoring improve clinic performance because it is a bottom-up intervention which makes clinic personnel socially accountable to their immediate neighbors? Or does it work simply because clinic performance is being monitored and evaluated? The answer to this question is important as top-down monitoring may be potentially cheaper and more efficient than bottom-up monitoring; however, data on this crucial question is lacking.

Context of the Evaluation:

Sierra Leone’s health indicators are among the lowest in the world, and the country’s health system is plagued by such chronic worker absenteeism, resulting in part from a lack of accountability between service providers and patients, and the weak incentives healthworkers face. Alongside a national decentralization program introduced in 2004, the Government of Sierra Leone launched an ambitious policy in 2010 to institute free healthcare for pregnant women, new mothers and children under-five. The policy abolished user fees, while at the same time raising workers’ salaries. However, these reforms occurred without introducing institutional features to improve oversight of health workers or changing underlying incentive systems, leaving the health sector at risk of further weakening in response to rising demand for free health services.

Details of the Intervention:

This project evaluates two social accountability interventions aimed at improving health service delivery via community monitoring and the introduction of an incentive scheme to reward worker performance on the basis of non-financial awards. The 254 clinics taking part in the study have been assigned to participate in either intervention or act as a comparison, with one third of clinics allocated to each group.

The community monitoring intervention introduces health scorecards that provide information regarding the state of health care in each community, and facilitates interface meetings between community members and health facility staff. During these meetings, information about the state of healthcare is disseminated via a community scorecard and mutual commitments are made to improve services through a joint action plan addressing areas such as staff absenteeism, maternal mortality and vaccination rates. This framework aims to ensure participatory decision-making and hold both healthcare workers and the community mutually accountable, fostering increased access to and utilization of maternal and child health services. Researchers evaluate whether service quality and quantity improve due to the lower costs of collective action introduced through these meetings and the social accountability contract.  

The second intervention, non-financial incentives,facilitates a yardstick competition among groups of maternal and child health clinics, and rewards workers at the most improved facilities. The relative rankings of clinics on key measures of such as worker absenteeism, staff attitude and charging of illegal fees will be advertised publicly, and staff at winning clinics will receive letters of commendation from high-ranking politicians, and an award at a public ceremony.

The project is being conducted in partnership with the Government of Sierra Leone and the interventions have been designed with a self-sustainable model for scale-up through the Ministry of Health and Sanitation in mind. Researchers will assess the cost-effectiveness of each intervention, as well as their cost-effectiveness relative to one another, and findings will directly inform the government’s decision to scale up these interventions in future years.


Results forthcoming

[1]Ashraf, Nava, Oriana Bandiera, and Kelsey Jack. "No Margin, No Mission? A Field Experiment on Incentives for Pro-Social Tasks." Harvard Business School Working Paper, No. 12-008, August 2011.

Effect of Matching Ratios on Charitable Giving in the United States


Policy Issue: 

Throughout the world, charitable organizations are working to meet people’s basic needs and improve their quality of life, and these organizations often depend on the support of outside donors to finance their mission. Private donations comprised 75 percent of all charitable giving in the United States in 2008 and experts predict that the combination of increased wealth and an ageing population will lead to an even higher level of private gifts in the coming years. Such trends have left fundraisers, who typically rely on anecdotal evidence in lack of scientific evidence, divided as to the most efficient means to attract these dollars. While the economics of charity has been well studied on the “supply” side, critical gaps remain in our knowledge about the “demand” for charitable giving.

Context of the Evaluation: 

In the United States, private giving to charitable causes has grown significantly in the past several decades. Recent figures show that charitable gifts of money have been 2 percent or more of GDP since 1998 and currently more than 89 percent of Americans donate to charity. 

Among the tactics commonly used in fundraisers to solicit donations are matching gifts. A matching gift is a commitment by a donor to match the contributions of others at a given rate, up to the maximum amount the leadership donor is prepared to give. While the rate of matching is typically the result of an agreement between the fundraiser and the leadership donor, fundraising consultants ubiquitously note that increases in the matching ratio have power to influence contributions. Such conventional wisdom, however, is largely anecdotal as little scientific study has been completed to examine such demand side claims.


Details of the Intervention: 

This study seeks to explore the importance of price on charitable giving by measuring the comparative static effects of changes in rates of matching gifts. Partnering with a liberal U.S. nonprofit organization that works on social and policy issues relating to civil liberties, this study takes advantage of a capital campaign in which prior donors received direct mail solicitations seeking contributions. 

Individuals were randomly assigned to either a comparison group or a matching grant treatment group. All individuals received a letter identical in all respects except two: (1) the treatment letters included an announcement that a “concerned fellow member” will match their donation, and (2) the reply card included the details of the match. The specifics of the match offer were then randomized along three dimensions:

1. Price ratio of the match: 1:1, 2:1 or 3:1 (the 2:1 ratio means, for example, that for every dollar the individual donates, the matching donor contributes $2)

2. Maximum size of the matching gift across all donations: $25,000, $50,000, $100,000 or unstated

3. Example donation amount suggested to the donor equal to: (i) the individual’s highest previous contribution, (ii) 1.25 times the highest previous contribution, or (iii) 1.5 times the highest previous contribution

Because fundraising solicitations are sent to all 50 states, it is possible that utilitarian effects of contributing to a politically-motivated charity are different spatially due to the local political environments. To test for these effects, charitable giving data is merged with (i) demographic data, (ii) state returns from the 2004 presidential election and (iii) data on the frequency of the organization’s activities within each state.


Results and Policy Lessons: 

Matching Impact: Simply announcing that match money is available considerably increases the revenue per solicitation—by 19 percent. In addition, the match offer increases the probability that an individual chooses to donate by 22 percent. This finding supports anecdotal evidence among fundraising consultants on the efficacy of a matching mechanism.

While the match treatments relative to a comparison group increase the probability of donating, contrary to conventional wisdom, larger match ratios (2:1 and 3:1) relative to smaller match ratios (1:1) have no additional impact. This result directly refutes the rationale for using larger match ratios, and stands in sharp contrast to current fundraising practices.

Heterogeneous Treatment Effects: Study findings indicate that the results of the matching gift are driven by agents in states that voted for George Bush in the 2004 presidential election: the match increases the revenue per solicitation by 55 percent in “red” states whereas there was little effect observed in “blue” states. This result suggests that an individual’s political environment also has the capacity not only to influence the level of giving, but their responsiveness to different treatments.

1 Giving USA Foundation, “U.S. charitable giving estimated to be $307.65 billion in 2008,” http://www.philanthropy.iupui.edu/News/2009/docs/GivingReaches300billion....


Selected Media Coverage:
What Makes People Give? - New York Times
Bigger Matching Gifts Don't Produce More Donors - The Chronicle of Philanthropy
Dean Karlan, John List

Experiments to Improve Participation in a Recycling Program in Northern Peru

Policy Issue:
Economic growth in Latin America has come at the cost of increasingly acute environmental pressures. Expanding trade and consumption has led to increased waste generation and pollution requiring more developed solid waste disposal systems. Markets lack a price mechanism to internalize the environmental cost of this growth.  Policy makers often apply taxes, subsides or other mechanisms to attempt to align private incentives with public environmental preservation. Aside from altering financial incentives, growing evidence from psychology and behavioral economics research shows that behavior can successfully be influenced by leveraging social norms and emotions.  
Over 20,000 tons of solid waste are produced every day in Peru, most of which is dumped in waterways or informal trash heaps, making solid waste management an area of increasing concern for the country. PRISMA, a local NGO, operates a recycling program in Northern Peru whereby it trains and supports workers in forming associations that collect recyclables door to door from participating households. In addition to providing the informal workers with some initial tools and training, PRISMA further assists workers by canvassing the areas of operation to introduce the recyclers to the community and encourage the residents to segregate recyclable and take part in the recycling program. PRISMA was interested in identifying viable strategies to increase program uptake (34% at baseline), and reduce attrition of participating households from the program.
Description of Intervention:
Researchers worked with PRISMA to test a series of information messages aiming to improve take-up and participation its recycling program.
To improve take-up of households in communities where PRISMA planned to expand the program, researchers conducted a randomized evaluation to test the impact of different messages in eliciting program participation.  One week before the first PRISMA canvasser’s visit, a paper flyer was delivered with the a generic message about PRISMA’s program and one of nine specific messages eliciting pressures such as, social norm, peer comparison, conformity, authority, environmental or social benefits to increase participation.
Households that owned a cell phone and were willing to share their number (about 35% of the sample) received text messages once a week with the specific message in addition to the flyers. 
With a sample of 1,785 existing participants, researchers tested strategies to reduce program dropout and increase the amount and quality of the recyclables collected. Plastic bins were randomly distributed to households participating in the study. Some bins had a sticker specifying which items could be recycled.  Household that provided cell phone numbers were randomly assigned to receive either a generic or personalized SMS reminders to recycle or to serve as a comparison group without SMS reminders. SMS messages were sent the day before the weekly visit by the recycler, for a period of six weeks. Data collection for this component of the study lasted for eight weeks and included a careful accounting of the quantity and quality of recyclables received.
Results and Policy Lessons:
Results of Treatments on Program Enrollment: The results reveal no statistically significant effects from the different treatment messages. No significant impact on program take-up for the information campaign conducted through flyers alone or through flyers with text messages were found. Messages conveying social norms and applying social pressure were not successful in leveraging behavioral change. Two interpretations for this outcome are proposed by the researchers: a) these messages and norms were not relevant in this context, b) the large presence of informal recyclers operating outside of the program rendered separation of recyclables at the household level a non-issue.
Results of Treatments on Compliance: Households who received plastic bins turned in recyclables 3-8 percent more of the times and produced on average more (about 0.2 kg) and more valuable recyclables (about 0.1 pesos).  This finding suggests that convenience of storing recyclables is a barrier to greater program participation. 
The SMS reminders had no significant impact on the level and quality of participation of households in the program, suggesting that forgetfulness is not a serious constraints among households enrolled in the program. There was no clear difference in recycling compliance between households who received plain bins and those who received bins with explanatory stickers.

Peace Education in Rural Liberia

For new democracies and societies emerging from conflict, encouraging tolerance and dialogue, strengthening non-violent conflict resolution systems, and increasing understanding of human rights are key priorities. Governments and NGOs commonly try to change the political culture, civic values, and practices of conflict resolution at the local level through widespread dialogue, education, and information campaigns.  But do these dialogue and education programs actually work as intended? Do they change norms and behaviors, and if so, how? How are new patterns of conflict resolution formed?  And how do they contribute to national reconciliation? How do new state structures integrate with pre-existing local bodies to jointly support security goals and human rights, especially where traditional structures are in conflict with the later? In short, what programs are most useful in helping post-conflict countries achieve lasting peace?

Find a more in-depth policy report here.

Context of the Evaluation:

More than five years after the end of Liberia’s fourteen year civil war, underlying tensions between tribes, over land, and between youth and elders continue to pose threats to a fragile peace. The UN’s Peacebuilding Commission and the Government of Liberia are working together to promote non-violent dispute resolution and inter-group reconciliation, but how best to do this is unknown. This study was jointly designed by the UN, the government, and the research team to assess whether civic education and conflict resolution programs can contribute to this broader peacebuilding agenda.

Description of the Intervention and Evaluation:

Target communities for the program were identified within Liberia’s three most conflict-prone counties: Grand Geddeh, Lofa, and Nimba. The researchers randomly assigned half to receive the program as the “treatment” group, and half to not receive the program as a “control” group. The program took place in 67 villages and town quarters. The program mobilized and trained community members in order to achieve three main goals: (1) educate people on their rights and to respect the rights of others; (2) encourage community collective action towards shared goals; and (3) foster non-violent dialogue and conflict resolution. The program is notable for its intensity and reach: in each community, roughly 10% of adults participated in an eight-day long interactive workshop held over the course of several weeks. Workshops had between 20 to 30 participants, both men and women, were led by a professional facilitator, and were conducted in local dialects. Multiple workshops were held in most communities to reach the 10% coverage target.

Pre-program baseline data was collected in 2009, and the endline took place between 1 and 22 months after the program. Data came from more than 5,000 individuals with three main surveys. In each community, the team interviewed: 20 randomly-selected “community members”, 4 “community leaders”, and 3 people identified by local chiefs as potential trainees, including a “troublesome” person. The study measures the impact of attending the program on potential trainees, random community members, and community leaders, and the impact on the community of having the program take place in their community. An in-depth qualitative study in 14  of the communities was conducted alongside the randomized evaluation to determine the mechanisms of impact. The study focuses on four major outcome classes: community and political participation; attitudes to rights; civic knowledge, attitudes and beliefs; and the prevalence and resolution of conflict.

Results and Policy Lessons:

Community and political participationCommunity participation was measured through contributions to public goods and community projects, membership in groups (from farming to sports), membership in a peace group, and leadership in groups.  Across all measures, the only treatment effect was on membership in peace groups. On political participation, the only statistically significant treatment effect was seen on an index measuring whether potential trainees feel free to speak their minds to “big people” in the community and whether they feel community members have the right to speak out to elders: Those trained are 4% more likely to feel empowered to speak freely. This effect is concentrated among the “troublesome” individuals, who see a larger increase of about 8%.

Attitudes on human rights: Across multiple measures, nearly all the treatment effects are positive, indicating that respondents in trained communities generally report more progressive beliefs. For community members, however, these impacts are fairly close to zero. The impacts on trainees and leaders are modest in size – often in the range of 1 to 10% —and seldom statistically significant at conventional levels.

Civic attitudes and knowledge: The civic education component provided information on citizenship, civic rights and responsibilities and Liberia’s political structure. At endline, amongst potential trainees in the control group, only about 12% correctly understood the statutory requirements for citizenship. This understanding nearly doubled among treated trainees. Little change was seen in political knowledge, and the program also appears to have little to no impact on perceptions of equity in community governance as well as on perceptions towards the national government.

Prevalence and Resolution of Conflict: The most striking program impacts were on conflict and its resolution. In treatment communities (i.e. those that received the program), the evaluation found sizeable increases in non-violent inter-personal and inter-group disputes; suggestive evidence of a decrease in violent disputes; increasing levels of land conflict since the program, though also suggestions of lower rates of violence, and increased rates of dispute resolution and of satisfaction with those resolutions in trained communities. At the community level, leaders reported a 93% increase in conflicts (typically disputes and disagreements) between youth and elders in treated communities.  Treatment communities were also twice as likely to have a peaceful strike or protest, and three times as likely to have a witch killing (though the latter result is not statistically significant).

Violent strikes, protests, and inter-group violence are 59% less likely in treatment communities, however, though this result is not statistically significant (partly because the events are rare, making it difficult to estimate their prevalence precisely with such a small sample of communities).

In addition to the policy report detailing the impact evaluation results, the team has produced a second policy report analyzing patterns of conflict.

Decentralizing Education Expenditures: Primary School Community Grants in Niger

Policy Issue

In recent years school enrollment has risen dramatically in developing countries, prompting a renewed interest among policymakers in education management. Provision of educational services is centrally administered in much of the developing world, but evidence suggests that decentralized, locally administered services may be better suited to address low education quality. Local administration and oversight puts power into the hands of those with the most interest in seeing improvements in service delivery, and the best information about current education quality. By empowering local communities, decentralized management has the potential to combat systemic teacher absenteeism and reduce misallocation and corruption.

Context of the Evaluation

Niger has made significant strides in increasing access to education in the past decade, with primary enrollment increasing from 37% in 2000 to 54% in 2005. Yet the country still faces considerable challenges with regards to access, rates of primary school completion and the quality and management of the educational system at all levels. One problem, for example, is that schools are spaced far apart, making it difficult for the central government to effectively monitor them.

To address this problem the Ministry of Education mandated the creation of management committees (Committees de Gestion Scolaire, or COGES) for each school. The committee included the director of the school, as well as locally-elected community members. The COGES are responsible for monitoring teacher attendance and performance and managing both financial and material resources, such as the purchase of textbooks and supplies. The Ministry of Education introduced a pilot project to reinforce the capacity of the COGES through the provision of a yearly cash grant, which can be used to fund a part of its activities for the year, such as the construction of classrooms or the purchase of materials. These grants were provided under the belief that such financial support would spur communities to take a more active role in the management of schools.

Details of the Intervention

To test this belief, researchers, in collaboration with the World Bank and the Nigerian Ministry of Education, randomized the allocation of the grants. They observed schools which did and did not receive grants, to estimate their impact, as well as the impact of two supplementary accountability interventions on school management, student retention and learning achievement. One thousand schools in the regions of Tahoua and Zinder were randomly selected into treatment and comparison groups. The 500 schools in the treatment group each received an annual lump sum based on the number of classrooms in the school, averaging out to around US$2 per student. This grant size was chosen by the Ministry of Education with considerations of sustainability regarding a possible extension of the program. In addition to the grants themselves, two sub groups were randomly selected to receive complementary interventions designed to improve delivery and management of the grants.

One third of the treatment schools received financial monitoring in the form of a letter received at the beginning of the intervention telling the committee there was a 50% chance of a “financial control visit” at the end of the school year. Because of financial and capacity constraints, about 20% of the treatment schools received visits at the end of the year.

Another third of the treatment schools received expenditure restrictions in the form of a list of non-eligible expenditures, such as reimbursing school officials for travel, or spending money on things that can be produced locally.

Researchers used annual administrative survey data, pupil test score data, and a detailed financial questionnaire to determine the effect of the grants. The testing used a modified standardized test implemented in collaboration with the Ministry of Education Department of Education Assessment. The school data contained information on pupil and community participation, school resources and infrastructure and information on issues facing school governance. The functioning of the COGES, including how often they met, what they talked about, and whether they began improvement projects, was also monitored. 

Results and Policy Lessons

Supportive actions, which include parental financial and in-kind contributions, parent supervision of pupil attendance, and parent remedial action for pupil absenteeism, increased by 0.14 standard deviations in treatment communities. Most of this overall effect came from an increase in parental contributions to the school, which was 0.48 standard deviations higher in the treatment group. The overall impact was larger when the school committee was educated and was somewhat smaller when families live farther from school.

Management activities, which include parents’ association and school committee meetings, and whether the school committee is in charge of collecting fees and how they decide how the fees are spent, was 0.09 standard deviations higher in the treatment group. Specifically, the proportion of school committees in charge of collecting fees increased by 27 percent, the proportion of active parental association increased by 18 percent, and the frequency of school committee meetings increased by 5 percent.

Oppositional actions, which include supervising teacher attendance and sanctioning teachers, increased only in communities where the school committees were educated. In these communities, teacher supervision increased by 0.13 standard deviations.

Impact on school quality: The results indicate no short-term improvements in school quality, measured in terms of accountability and teacher effort. The lack of an impact on teacher effort is in line with the fact that very few of the schools undertook any sort of teacher supervision. However, one year after the treatment, the grants marginally improved (0.04 standard deviations) the material quality of schools. This improvement was largely driven by increases in the number of classrooms and construction of walls around the compound.

Impact on demand for education: The grant program led to a 0.17 standard deviation decrease in dropouts and a simultaneous 0.10 standard deviation increase in enrollment of students in Grade 2, indicating an increase in demand for education for young children.

Community Based Rangeland Management in Namibia

We conduct an impact assessment of the Community Based Rangeland and Livestock Management (CBRLM) program in Namibia.  This program is part of a larger set of interventions in the agricultural sector designed to reduce poverty among the population of the northern regions of the country.  Many people in the area rely on cattle production for their economic livelihoods, however overuse of the communal grazing areas and suboptimal grazing practices threaten the long-term viability of the land and contribute to persistent poverty.  

To increase the productivity of livestock and other animals using the land, the Namibia Millenium Challenge Compact funds a pilot program designed to help communities improve their livestock practices, address rangeland degradation, and improve market access.  

The evaluation is designed to test the impact of the various activities within the CBRLM intervention on household income, cattle productivity, and the condition of the rangeland. The intervention targets both inadequate information about appropriate cattle production practices and the social or other behavioral preferences of farmers.  At the moment, there is a collective action or “tragedy of the commons” problem – individual farmers are hesitant to reduce their herd’s impact on the rangeland because they are fearful that others will not follow suit which often results in overuse and degradation of the land.

Information Campaigns and Voters' Behavior in the 2009 Municipal Elections in Mexico

Policy Issue:

It is widely held that access to information is a vital component of democracy building and government accountability. A recent World Bank report[1] champions information as “a tool to empower citizens in developing countries to hold their public agents accountable.” Information flows, the report argues, not only enhance democratic participation, but also make democracy work for ordinary people. However, while evidence suggests that access to information may lead voters to punish corrupt incumbents, it is unclear whether this translates into increased support for challengers and higher political participation. In other words, information about corruption may not improve political accountability, if voters respond to it by withdrawing from the political process.


Despite optimistic views about fiscal decentralization in Mexico, local governments’ performance has remained poor. In 2008, for example, more than 80 percent of municipal governments’ resources were spent either on the bureaucracy or were unaccounted for. While elections should enable voters to discipline their mayors, a single-term limit is imposed on all elected officials in Mexico. Thus, the immediate fate of mayors is determined not by voters but by their political party. To reconcile the single-term limit with accountability, scholars have typically assumed that voters punish or reward the incumbent party for mayoral performance. However, there is little evidence that government performance impacts the subsequent election – previous work shows a strong entrenchment of incumbents from all political parties.

Further impeding voters’ ability to hold mayors accountable are widespread misconceptions about which public works and services municipal authorities are responsible for providing, as well as a lack of available information about the amount of money municipalities receive and how this money is spent. In an attempt to ensure greater municipal accountability, a 1999 constitutional reform established the Federal Auditor's Office (ASF). On a yearly basis, the ASF selects a sample of municipalities in each state to audit. The results of the audits are published in lengthy reports, which are made available online. Though public, these reports are rarely used by media or political parties in local campaigns because the release date of the reports is not aligned with the timing of elections.

Description of Intervention:

Researchers sought to assess the effects of information dissemination on participation in the 2009 municipal and congressional elections in Mexico. Approximately one week before Election Day, flyers with different kinds of information on municipal spending were delivered to all households within the boundaries of treated voting precincts. The first group of precincts received information about municipalities' overall spending; the second group received information about distribution of resources to the poor; and the third group received information about irregular, unauthorized, or unaccounted for spending. All flyers included a subtle advocacy message suggesting that voters raise questions with their mayors about the use of public funds and prompted people to think about the level of governments that was in charge of the provision of public infrastructure services. In total, 150 electoral precincts were randomly assigned to each of the three interventions, for a total of 450 treated precincts and 1910 precincts in the comparison group.

Researchers gathered demographic characteristics from census data, and then collected electoral results for each precinct from the electoral institutes for each state. This information was complemented by a follow-up survey collected ten days after the election.


Information about overall spending: Disseminating information about overall spending levels had no statistical impact on voter turnout when the mayor spent less than 75 percent of available funds. However, it decreased the incumbent vote share by 0.6 percentage points. When mayors spent more than 75 percent of the funds, releasing this information let to an increase in turnout of 1.9 percentage points, but had no significant impact on vote share.

Information about spending on poor: Turnout among voters who received information about levels of spending allocated to poor areas increased by 2.4 percentage points, but only when mayors spent less than 75 percent of available funds on poor areas. Surprisingly, releasing this information had a similar impact on both incumbent and challengers' vote shares - an increase of 1.5 and 1 percentage points, respectively. When mayors allocated more than 75 percent of resources to poor areas, releasing the information had no impact on turnout or vote share.

Information about corruption: Information about high levels of corruption appears to have had a significant negative impact on voter turnover. Turnout among individuals that received information about corruption decreased by 1.10 percentage points, which represents a 2 percent decrease in turnout. While the effect of information about corruption on the incumbent vote share was insignificant, the effect on the challengers' vote share was negative. These results suggest that while flows of information are necessary, they may be insufficient to improve political accountability, since voters may respond to information by withdrawing from the political process.

[1] Khemani, Stuti. 2007.  Can Information Campaigns Overcome Political Obstacles to Serving the Poor? The Politics of Service Delivery in Democracies. http://siteresources.worldbank.org/DEC/Resources/Khemani_CanInformationCampaignsOvercome.pdf.


Effect of Media on Voting Behavior and Political Opinions in the United States

There is substantial evidence that media sources have identifiable political slants, but there has been relatively little rigorous study into the impact of media on political views and behaviors.  IPA designed a natural field experiment to measure the effect of exposure to newspapers on political behavior and opinion.

Washington DC is served by two major newspapers, the Washington Times and the Washington Post. We found that those who received the Washington Post were eight percentage points more likely to vote for the Democratic candidate for governor than those assigned to the control group. We find similar but weaker evidence of shifts in public opinion on specific issues and attitudes.

Policy Issue: 

Citizens learn about politics and government primarily from television and newspapers. These media outlets can influence voters not only through the slant of a particular report, but also merely by choosing which to stories to cover. Recent studies suggest that media exposure can have a sizable impact in shaping the public’s political knowledge, attitudes, and behavior. However, these studies may have overestimated the impacts of media influence due to individuals’ tendency to seek out information that agrees with their pre-existing views. 

Context of the Evaluation: 

Prince William County in northeastern Virginia lies just 25 miles from Washington D.C. Here, the population is far enough away from the nation’s capitol so as not to be dominated by citizens involved professionally with politics, but close enough to be within the circulation of Washington’s conservative and liberal newspapers. The Washington D.C. metro area is served by two major newspapers, the conservative Washington Times and the more liberal Washington Post. The presence of a liberal and conservative paper serving the same region creates an opportunity to study the effect of media slant in a natural setting within a single population, which is subject to the same outside factors, such as political events and outcomes, and has a range of political leanings.

Details of the Intervention: 

This study takes advantage of this natural setting to measure the effect of political news content on people’s political behavior and opinions. Approximately one month prior to the Virginia gubernatorial election in November 2005, researchers administered a short survey to a random selection of households in Prince William County. 

From the 3,347 households of registered voters who reported that they received neither the Post nor the Times, researchers randomly assigned households to receive a free subscription to one of the two papers for ten weeks, or to the comparison group that was not sent either paper. A week after the election, a follow-up survey was administered asking individuals whether they voted in the November 2005 election, which candidate they selected or preferred, their attitudes toward news events of the previous weeks, and their knowledge about recent news events. Voter turnout data was also collected for the November 2005 and 2006 elections from state administrative records. 


Results and Policy Lessons: 

Impact on Political Knowledge: Receiving either paper produced no effect on knowledge of political events or stated opinions about those events, and there were no differences between the treatment and comparison groups in voter turnout for the 2005 gubernatorial election.  In November 2006, however, there was a 2.8 percentage point increase in voter turnout. It is surprising to see a result in 2006 but not in 2005. This could be a result of the post-election exposure to the remainder of the ten-week newspaper subscriptions, or the fact that 17 percent of the treatment group renewed their subscription after the free period ended.

Impact on Political Preference: Interestingly, receiving either newspaper led to an increase of support for the Democratic candidate. Despite the political slant of the newspapers, the effects were similar for the Post and the Times, resulting in an overall 7.2 percentage point increase in likelihood of voting for the Democratic candidate. This may be due to the fact that the Republican President’s approval ratings were falling over that period of time, or perhaps the Democratic candidate was conservative leaning. In either case, these results suggest that the informational effect of more exposure to news was stronger than the effect of its slant. 

Direct Democracy and Local Public Goods in Indonesia


Policy Issue: 

Basic public goods such as dependable roads and clean water infrastructure are routinely underprovided in much of the developing world. While the provision of these services is often centrally administered, many now advocate for decentralization and community involvement as a more effective approach. Local communities can have better information on what goods and services are needed, and may thus be better positioned to recognize and quickly respond to inefficiency or corruption in implementation. Recent years have witnessed a trend toward decentralization in developing countries; this increase in local participation in government decision-making has been facilitated by a wide variety of political reforms, but the implications of these political mechanisms are yet to be well understood.  

Context of the Evaluation: 

Over 13 percent of the population of Indonesia lives below the poverty line,  and improving the infrastructure in marginalized areas is a priority for the Indonesian government.1 An Indonesian government program supported by a loan from the World Bank, the Kecamatan Development Program (KDP), funds projects in approximately 15,000 villages each year. Each village receives an average of Rp. 80 million (US$8,800) for infrastructure projects. Participating subdistricts, typically containing 10-20 villages, receive an annual block grant for three years. Each village makes two proposals – one on behalf of the whole village and one proposed by woman’s groups – for small-scale infrastructure projects.

Typically, when it comes time for a village to decide upon its two KDP proposals, representatives from various hamlets come together to discuss the merits of, and to decide on, the village’s two project proposals. A typical meeting would have between 9 and 15 people representing the various hamlets, as well as formal and informal village leaders, with on average about 48 people attending in total out of an average village population of 2,200. While the program has effectively improved local infrastructure in many of these villages, it is unclear whether current procedure makes projects easily dominated by elites and under-provides for those community members who need improved services the most.   


Details of the Intervention: 

To investigate these issues, researchers conducted a randomized evaluation in 49 villages, all of which were preparing to apply for infrastructure projects. Each village was randomized into one of two different political processes through which they determined which project to propose: 32 villages would follow the traditional representative meeting-based process described above, and the remaining 17 would choose their KDP project proposals via a direct election-based plebiscite. At these plebiscites, villagers could directly vote on a list of potential projects. For the general project, all adults were eligible to vote, and for the women-specific proposal, only women could vote. Data was collected on the project preferences of all villagers, including the elite, as well as the location and type of projects selected.

Results and Policy Lessons: 

Impact on Project Type: The direct election process had little effect on the types of projects selected amongst the general project proposals, but had substantial positive effects on measures of citizen satisfaction with the political process. However, the elections did not change the probability that the general project would be located in a poor area. Direct elections on women’s projects, on the other hand, resulted in both increased satisfaction with the political process and increased probability that projects were located in poorer areas of the villages. 

Impact on Satisfaction Measures: Direct elections resulted in substantial changes in the community’s satisfaction with the political process. Overall, the plebiscites resulted in an increase of 21 percentage points of people who said that the project chosen was either very much or somewhat in accordance with their wishes, an increase of 18 percentage points of people who said they would benefit either very much or somewhat from the project, an increase of 10 percentage points of people who said they would use the project personally, and an increase in overall satisfaction with KDP by 13 percentage points. The elections also raised the probability that individuals stated they would contribute something (such as labor or money) to the project by 17 percentage points. Additionally, villagers in treatment locations were 19 percentage points more likely to correctly identify the type and location of the general village project, and 25 percentage points more likely to know these things about the woman’s project.  

[1] CIA World Factbook, “Indonesia,” https://www.cia.gov/library/publications/the-world-factbook/geos/id.html (accessed August 31, 2009).

Combating Corruption in Community Development in Indonesia

Policy Issue: 

Corruption plagues many developing countries where the world’s poorest live, and combating it continues to be an arduous task. Corruption acts like a tax, adding to the cost of providing public services and conducting business; it also creates potentially severe efficiency consequences as well. Many suggest the right combination of monitoring and punishments can control corruption, but often the very individuals tasked with monitoring and enforcing punishments may themselves be corruptible. Another approach to reducing corruption is community-level monitoring. Local community members have the most to gain from a successful anti-corruption program, and are thus believed to have better incentives to monitor than bureaucrats. However, there is little empirical evidence on the success of such strategies.  

Context of the Evaluation: 

An Indonesian government program supported by a loan from the World Bank, the Kecamatan Development Project (KDP), funds projects in approximately 15,000 villages each year. Each village receives an average of Rp. 80 million, (US$8,800), which they often use to surface existing dirt roads. KDP-funded projects are large relative to ordinary local government activities; in 2001, the average annual village budget was only Rp. 71 million (US$7,800), so receiving a KDP project more than doubles average local government expenditures. This large amount of money creates incentives for price inflation, collusion with suppliers, kickback for village leaders, and manipulation of wage payments. 

Two checks on corruption are built into KDP. First, funds are paid to village implementation teams in three installments. To receive the second and third payments, the teams must make accountability reports at an open meeting where they account for how they spent the money; only after that meeting has approved the accountability report is the next installment released. Second, each project has approximately a 4 percent chance of being audited by the government.


Details of the Intervention: 

To examine the role of community monitoring and government audits on corruption, researchers conducted a randomized evaluation in 608 Indonesian villages in East Java and Central Java, Indonesia’s most populous provinces. Each village in the study was about to start building a village road with KDP funding. Some villages were randomly selected to be told, after funds had been awarded but before construction began, that their project would subsequently be audited by the central government, increasing the likelihood of an audit from 4 percent to 100 percent. These audits carry the possibility of criminal action if corruption is detected, and the results of the audits are read publicly to an open village meeting, potentially resulting in social sanctions. 

To investigate the impact of increasing community participation in the monitoring process, two interventions were established to enhance participation at accountability meetings. Some villages were selected to have invitations to these meetings distributed throughout the community, encouraging direct participation in the monitoring process and reducing elite dominance of the process. In the second experiment, an anonymous comment form was distributed along with the invitations, providing villagers an opportunity to relay information about the project to be shared at the meetings, without fear of retaliation. (See chart below)

Experimental Treatments by Group:




Audit Probability


Accountability Meetings




Comment Cards
















  Participation I












  Participation II












  Audit & Participation I












  Audit & Participation























Corruption was measured by comparing the researcher’s estimate of what the project actually costs, determined by the quantity of materials used and estimate of material prices and wages paid on the project, to what the village reported it spent on the project on an item by item basis.


Results and Policy Lessons: 

The evidence suggests that increasing the probability of external audits substantially reduced missing funds in the project. Increasing the probability that a village was audited by the central government from 4 percent to 100 percent reduced missing expenditures by about eight percentage points, from 27.7 percentage points to 19.2 percentage points. One reason that the decrease was not larger is that a 100 percent audit probability does not imply that village officials face a 100 percent probability of detecting corruption and imposing a punishment. In fact, although auditors found violations of some type or another in 90 percent of the villages they visited, the vast majority of these violations were procedural in nature, and there were very few, if any, cases in which the auditors had enough concrete evidence to actually prosecute offenses. 

The invitations increased the number of people participating in the accountability meetings by an average of 14.8 people, or about 40 percent; slightly more than by including a comment form, since many villagers used the form as a substitute for attendance. Villages that received the invitations intervention were more likely to openly discuss corruption problems at the accountability meetings, and villages receiving both invitations and comment forms were more likely to take serious action at the meeting to resolve corruption-related problems. However, the magnitude of these changes in behavior at the meetings was small, and these treatments did not measurably reduce overall missing expenditures, yet they did have an effect on certain types of expenditures in some cases. For instance,  the invitations substantially reduced missing labor expenditures, and the comment forms did reduce missing expenditures in some cases, but only when they were distributed via schools, bypassing local officials. This study provides evidence that community participation, widely viewed as a panacea for development projects, impacts levels of corruption only under a limited set of circumstances, and pains must be taken to prevent elite capture for it to be an effective means of reducing corruption.

Extra Teachers in Kenya: Peer Effects, Pupil Teacher Ratios and Teacher Incentives

Policy Issue: 

The introduction of free primary education has raised primary school enrollment in many developing countries. However, the resulting overcrowding of schools, as well as the influx of new students with little or no preparation, poses new challenges to policymakers. For example, increased enrollment has often not been matched by increased numbers of government-salaried teachers. One method of lowering the pupil-teacher ratio, versions of which have been used by many governments, is to hire low-paid local contract teachers to supplement government-salaried teachers. However, there are concerns that these teachers may be less experienced and therefore less effective. Empowering the local community to monitor teachers' performance may help to overcome this problem, and may also increase civil-service teacher effort and subsequently student learning.

Context of the Evaluation: 

Historically, Kenyan schools have had two types of teachers – those hired through the Ministry of Education and those hired locally and informally by Parent Teacher Associations (PTAs). For civil-service teachers, promotion, transfers, and disciplinary measures are decided through the Ministry of Education, rather than by more locally accountable bodies. These teachers are represented by a strong union, have civil-service protection, and receive wages and benefits considerably above market levels. PTA teachers, while typically paid much less than their civil-service counterparts, have much stronger incentives, partly because they do not have union protection, but also because a good track record as a contract teacher can help them obtain a civil-service job. 

Incentives for civil service teachers, while weak, are largely based on their students’ scores on the primary school exit exam. Teachers may, therefore, have an incentive to focus on students they believe will perform better on the exam, while ignoring the weaker students who may be too far behind to catch up. 

In 2003, Kenya eliminated school fees for primary school. This led to an almost 30 percent increase in enrollment and greater heterogeneity in student preparation. Many of the new pupils were first generation students and had not attended preschools. Since parents were no longer required to pay fees, local school committees were generally unable to raise the funds necessary to hire PTA teachers to match the influx of students and bring the new students up to speed.


Details of the Intervention: 

In collaboration with International Child Support (ICS), researchers evaluated three nested interventions that addressed the large class sizes and heterogeneity in student preparation in the Kenyan school system: (1) the addition of locally hired contract teachers to reduce the pupil-teacher ratio; (2) the empowerment of parents within PTA committees through School-Based Management (SBM) training; and (3) the sorting of students by initial level of preparedness to reduce heterogeneity within the classroom.  

Out of 210 primary schools, 140 were randomly assigned to receive the Extra Teacher Program (ETP). The ETP program provided funding to hire a local contract teacher to address classroom overcrowding. The remaining 70 schools served as a comparison group. In each school, ICS held a meeting with parents and teachers to explain the program rules regarding the hiring of an additional teacher. School committees were responsible for hiring the contract teachers and were free to replace or keep the original contract teacher based on performance. The contract teachers were paid approximately one-quarter of the salary of regular civil service teachers, but had the same educational qualifications. 

In 2005, the first year of the program, the ETP contract teacher was assigned to grade 1. Two sections were created for grade 1, one taught by regular civil service teachers (with multiple teachers rotating in an out of the class, teaching different subjects) and one taught exclusively by the contract teacher. As a result, average class size in grade 1 was only 44 in ETP schools, compared to 82 in comparison schools. In the second year of the program, the ETP teacher moved to grade 2, such that the cohort of first graders that benefitted from the ETP program continued to benefit once in grade 2.

While in half of the ETP schools (the “non-tracking ETP schools”), students were divided into sections at random, in the other half (the “tracking ETP schools”), students were divided into sections based on students’ level of preparedness (as measured by exam scores during the first term). For all schools, which section was taught by the ETP contract teacher was then decided by random draw.

Finally, half of the schools assigned to ETP, including both tracking and non-tracking schools, were randomly selected to receive School-Based Management (SBM) training. The training was designed to empower parents (within the PTA committee) to ensure a fair and objective recruiting process for the ETP teacher, as well as to monitor teachers’ performance. Two parents of grade 1 students were asked to perform attendance checks on teachers on a regular basis. 

Standardized test covering math and literacy questions were administered to each school just before the program ended and again one year later. Four unannounced visits were also made to each school to measure teacher effort and to observe the classroom environment. 


Results and Policy Lessons: 

Learning Impacts: Providing school committees with funds to hire an extra teacher on a short-term contract had a positive effect on learning. Test scores of students in ETP schools were on average 0.22 standard deviations higher than those of students in comparison schools. However, not all students benefitted equally. The program impact varied a lot with the details of how it was implemented. In particular: 

  • The impact was much larger for students assigned to the ETP contract teacher (0.31 standard deviation gain) than for students assigned to the civil-service teachers (0.13 standard deviation). 
  • The impact was larger when the ETP program was accompanied with SBM training (0.25 standard deviation) than when it was implemented alone (0.19 standard deviation).
  • The impact was larger in tracking schools (0.28 standard deviation) than in non-tracking schools (0.16 standard deviation).
  • There was relatively rapid fade-out of these learning gains in the year after the program ended, except for the gains in tracking schools and the gains in SBM schools.


Teacher effort: A crude measure of teacher effort is how often the teacher is found in class and teaching during a random spot check. This rate is 58 percent among civil-service teachers in non-ETP schools, suggesting relatively low levels of efforts. Despite their low pay, contract teachers were found 27.8 percentage points more likely to be teaching during a random visit. As a result, their students learned more. In contrast, civil-service teachers in ETP schools responded to the introduction of the ETP program by decreasing their own effort: the probability that they would be found in class teaching during a random visit decreased from 58 to 45 percent. As a result, their students did not significantly gain in learning over students in comparison schools, despite being in much smaller classes. One of the reasons why the SBM program led to increased learning gains is that it mitigated the negative effort response by civil-service teachers.

Selection of Contract Teachers: Because a contract teacher position is a stepping stone for permanent civil-service positions, it is highly valuable (despite the low pay). As a result, existing civil-service teachers may want to hire their own relatives as contract teachers. In ETP schools without SBM training, 31 percent of contract teachers hired were relatives of existing teachers in the school. Those relatives tended to perform less well than other contract teachers. The second reason why the SBM program led to increased learning gains is that SBM increased the transparency of the contract-teacher recruiting process. Specifically, it reduced the number of contract teachers who were hired because they were related to an existing teacher by about half. 

Homogeneous classrooms: Compared to students in non-tracking ETP schools, students in tracking ETP schools were in more homogeneous classes. This homogeneity benefitted them, irrespective of whether they had started at the top, middle, or bottom of the distribution. Homogeneity is beneficial in part because it allows teachers to better tailor their materials to the level of their students, but also because it increases teachers’ attendance in the classroom. For those who started at the middle of the distribution, being assigned to the bottom track (instead of the top track) had no detrimental effect on their learning, because the decrease in peer quality was compensated by a positive effect of being at the top of the class. 

Overall, the results suggest that while hiring supplementary contract teachers can be part of the solution to Kenya’s teacher shortage, to get the most out of these teachers, implementation details matter. The biggest gains come when local school committees are empowered to oversee the recruitment process and to effectively monitor teachers, and when classes are structured so as to target instruction to students’ initial achievement level. 

Child Protection Knowledge and Information Network (CPKIN)

Policy Issue:

Children in post-conflict environments are a population of particular concern, exposed to violence, displacement, and death of family members. The impact of poverty means that child labor is common among children. Though no reliable data exists, indications are strong that sexual abuse and exploitation is a fact of life for many girls and teenage pregnancy has been identified as a child protection concern. In addition to the physical and mental health challenges posed by war experiences, these children remain susceptible to continued exploitation and abuse, often by those familiar to the child, during peace as refugees return home and community members cope with difficult pasts. 

Context of the Evaluation:

The situation of children in Sierra Leone continues to be precarious as 27% of the 2.7 million children are identified as vulnerable, lacking the protection of a primary care giver. Formal structures for child protection exist within police stations and local Child Welfare Committees (CWCs), but capacity and community trust in such institutions’ efficacy are limited in many rural areas[1]. To reduce violence against children, existing social structures,most paramount, section, and village chiefs, are frequently leveraged to respond to reports of abuse and to mobilize local protective factors.

To respond to the resource constraints faced by formal child protection systems in Sierra Leone, the Ministry of Social Welfare, Gender, and Children’s’ Affairs (MSWGCA) is experimenting with a policy framework that explicitly links local chiefs and community members to the formal child protection sector.  This policy was articulated in a Memorandum of Understanding (MOU) signed in 2010 by the MSWGCA, CWCs, and the Council of Paramount Chiefs. The MOU and focal person system are currently being implemented in Moyamba and Pujehun districts, with potential to expand the program nationwide.  The MOU creates a framework through which chiefs, MSWGCA, CWCs, and the Family Support Unit (FSU) of the Sierra Leone Police can collaborate to protect children. Village chiefs appoint a focal person for child welfare in a public meeting of village residents. Focal persons report cases of child abuse to chiefs and chiefs, with the assistance of focal persons, follow up with the formal child protection actors to ensure that cases of child abuse are addressed appropriately.

To help focal persons connect informal systems for child protection to the formal roles of the FSU and MSWGCA, UNICEF and other local child protection NGOs have developed a training program that introduces focal persons to general approaches to child protection, as well as to their roles and responsibilities under the MOU.  Despite the grounding of these trainings in both international best practices and local norms and understandings, key stakeholders have expressed concerns about whether one-time trainings will provide focal points with sufficient capacity to effectively serve as a link between communities, chiefs, and formal child protection systems.  At the same time, repeated trainings or sustained in-depth monitoring by international or national child protection agencies is not sustainable in the context of rural villages in Sierra Leone.

Details of the Intervention:

Mobile phones and SMS messaging provide one potential avenue for bridging the gap between the need for sustained support of focal points and the high costs of transportation in rural Sierra Leone.  To leverage the growing use of this technology in Sierra Leone, a consortium of non-profit, academic, government, and private-sector stakeholders have collaborated in developing the Child Protection Knowledge and Information Network (CPKIN).  CPKIN (pronounced as “See Pikin” or “See the Child” in Krio, a language spoken throughout the country) is a system that will be used to send focal persons automated text messages asking questions about the welfare of children in their village and prompting them to engage in discussions in their community to help answer these questions.  Focal persons will then be encouraged to send their answers back to the central CPKIN system using a free text message.  These text message questions and responses will be sent and received through a software program that makes it possible to manage, organize, and analyze high volumes of outgoing and incoming messages with a large list of recipients

There are several hypotheses underpinning the design of the CPKIN program. The first hypothesis is that the process of receiving questions, discussing the questions with community members, and sending answers will encourage focal persons to critically examine the situation of children in their communities and then to act with the resources available in their villages. A second hypothesis is that having a focal person who is active with respect to reporting abuses, taking proactive steps to improve the welfare of children, and engaging community members in discussions regarding children has the potential to cause dramatic shifts in community level knowledge, norms, and practices regarding child protection, which in turn may lead to greater connections between informal and formal child protection systems.

Given the novelty of the CPKIN system as a child protection intervention, it is necessary to assess the extent to which sending regular, open-ended text messages to village focal persons using this system can increase the capacity of focal points and their communities to proactively and reactively respond to local child protection issues. In particular, the effectiveness of CPKIN and the validity of the underlying hypotheses will be evaluated using a randomized evaluation in 140 villages in Moyamba and Pujehun districts. The randomization will occur at the village level, with the 70 villages randomly assigned to the CPKIN treatment, and the remaining 70 villages comprising the comparison group, in which there is no intervention. In treatment villages, the village focal person will receive a one-on-one training on the overall aims of CPKIN, how to receive and send CPKIN messages, and how to use CPKIN as a starting point for engaging their community members on issues concerning child protection.  Shortly after this training, focal persons will begin receiving regular CPKIN prompts.

Baseline and endline surveys regarding child protection and child welfare will be conducted before and after the six to eight month intervention in all 140 villages in order to assess the relative effectiveness of the CPKIN support system, with survey questions designed to elicit two kinds of information:

1.)    The capacity of key stakeholders (children, parents, focal persons, chiefs, FSU, and CWCs—where they exist) to identify and respond to child protection issues.

2.)    The protection and support for children within their communities, and their overall level of welfare (including the prevalence of both abuses and protective factors).

Results and Policy Lessons:

Results forthcoming.

[1] Human Rights Watch. (2011). World Report 2011


Primary Education Management in Madagascar

Policy Issue: 

Successful efforts to expand access to education in the developing world have not always translated into actual improvements in skills and learning for students. It remains an open debate as to whether top-down approaches are more effective in improving educational quality than approaches which promote beneficiary participation, such as parental monitoring. Top-down approaches can provide administrators with the tools necessary to better monitor their schools, but this assumes that they have the incentives to do so. Bureaucrats may have more incentives to improve the quantity, rather than quality, of education services since the benefits of improving quality are diffuse and harder to verify. The combined effects of these perverse incentives can result in a large number of children who are in the classroom, but are not learning. Promoting local accountability may be a useful means of improving schooling outcomes, if it can be determined which factors makes beneficiary participation effective at improving education delivery.

Context of the Evaluation: 

Madagascar divides its 2.7 million children into 15,000 public primary schools. Despite the significant increase in primary school enrollment following Madagascar's 2002 reforms and an influx of international financial support, resource allocation across schools remains inefficient, and better resource endowments rarely translate into better student performance. Only 63% of grade 5 children pass the primary-cycle exam, an assessment of the minimum level language and math knowledge presumed at this grade. District administrators face a performance review only every 3 years, and the subdistrict heads rarely face any credible threat of penalties or firing.

Details of the Intervention: 

Researchers, in collaboration with The Ministry of Education in Madagascar, ran a randomized experiment in 3,774 primary schools in 30 public school districts. These districts represented all geographic areas in the country, but were focused on schools with the higher rates of grade repetition.

All district administrators in treatment districts received operational tools and training that included forms for supervision visits to schools, and procurement sheets for school supplies and grants (district-level intervention). In some of these schools, the subdistrict head was also trained and provided with tools to supervise school visits, as well as information on the performance and resource level at each school (subdistrict-level intervention).

Lastly, several districts also introduced a school level intervention which involved parental monitoring through school meetings. Field workers distributed a ‘report card’ to schools, which included the previous year’s dropout rate, exam pass rate, and repetition rate. Two community meetings were then held, and the first meeting resulted in an action plan based on the report card. One example of the goals specified in the action plans was to increase the school exam pass rate by 5 percentage points by the end of the academic year. Common tasks specified for teachers included lesson planning and student evaluation every few weeks. The parent’s association was expected to monitor the student evaluation reports which the teachers were supposed to communicate to them. These tools allowed parents to coordinate on taking actions to monitor service quality and exercise social pressure on the teachers.

Results and Policy Lessons: 

Impact from Top-Down Approach: The interventions targeted at the district and subdistrict level had minimal effects on the administrator’s behaviors, and the schools and students under their responsibility. Though each tool – forms for supervision visits to schools and procurement sheets for school supplies and grants – was used by 90% of subdistrict heads and more than 50% of district heads, subdistrict heads visited their schools only slightly more often than those in the comparison group, an insignificant improvement. Teachers in both groups did not plan for lessons more, and no improvement in test scores was seen in the two years following the program.

Impact from Bottom-Up Approach: The interventions at the school level led to significantly improved teacher behavior. Teachers were on average 0.26 standard deviations more likely to create daily and weekly lesson plans and to have discussed them with their director. Test scores were 0.1 standard deviations higher than those in the comparison group two years after the implementation of the program. Additionally, student attendance increased by 4.3 percentage points compared to the comparison group average of 87%, though teacher attendance and communication with parents did not improve.


Decentralization: A Cautionary Tale - Public Finance in Kenya

Kenya’s education system blends substantial centralization with elements of local control and school choice.  This project looks at the system of incentives created by elements of decentralization.

Syndicate content
Copyright 2014 Innovations for Poverty Action. All rights reserved.