Information and Community Mobilization in Rural India

Can providing communities with more information about their local education system help improve schooling? This project evaluates the effect of providing people with information on the resources that should be available to them, the mechanisms through which they can produce change, and the current functioning of local schools. The interventions had no impact on community involvement, teacher effort, or learning outcomes. However, we did find that the training volunteers to teach children to read had a large impact on activity outside public schools. Local youths volunteered to be trained to teach, and children who attended these camps substantially improved their reading skills.

Show Project Evaluation Details
Hide Project Evaluation Details

Policy Issue:

While primary school enrollment rates have risen sharply in much of the world, the quality of education remains low in many developing countries. Many children who attend school regularly are still unable to read or do basic arithmetic. Community oversight and participation has been advocated to increase education quality. However there is currently little rigorous evidence as to whether this works, and how community participation can be encouraged. Is more direct action by communities to teach their children to read effective?

Context of the Evaluation:

In Uttar Pradesh, India’s most populous state, many districts have a literacy rate below 50%. In response to these problems, the government established Village Education Committees (VECs) in every village in 2001. VECs consist of the elected head of the village government (pradhan), the head teacher of the local school, and three parents who are nominated by their community. These committees are responsible for monitoring school performance, claiming public funds and hiring an additional contract teacher in the event of overcrowding.

Despite the promising aspects of this program, a survey conducted in 2005 indicated that 38% of VEC members did not readily identify as being part of the committee, and 25% did not even know they had this role. Only 3.6% of all VEC members knew they had the ability to request funds to hire another teacher, which is one of the main options open to the VEC.

Details of the Intervention: 

Working jointly with Pratham (a local NGO) and the World Bank, researchers designed three interventions that were randomly assigned in 280 villages in four rural blocks in Jaunpur district, eastern Uttar Pradesh, a populous and educationally struggling area in India. These interventions served to determine if more information and encouragement to use the channels available to them would cause VECs and community members to demand and receive better services. They contrasted this with direct action to improve learning outside the official channels.

  • Intervention 1: In 65 villages, field workers started a series of conversations about education in small groups throughout the community. These conversations covered the current status of schools in the village, the quality of local schools, state mandated provisions for schools, mid-day meals, and local funds available for education. People were asked if they knew about the VEC and its membership and responsibilities. After two days of meetings in small groups, a community-wide meeting was held where people were encouraged to ask for information from the VEC, with information gaps filled in by Pratham’s field workers. VEC members also received a pamphlet on their roles and responsibilities from the staff.
  • Intervention 2: In addition to all the steps outlined above, communities in another 65 villages were trained and encouraged to conduct testing to see if children could read simple text and solve basic arithmetic problems. Volunteers put together a "report card" for each community, which was presented at the community-wide meeting.
  • Intervention 3: In addition to the above two steps, Pratham officers taught volunteers in another 65 villages a simple technique for helping children learn to read. Volunteers were encouraged to start after-school reading classes, and staff returned an average of seven times to provide in-service training. The objective was to use Pratham-designed materials and local volunteers to supplement the normal curriculum, and improve literacy among village children, and 85 villages received no treatment, serving as a comparison.

Results and Policy Lessons:

Impact on Information Gaps: The average effect of all three treatments was an increase of 7.8 percentage points in VEC members who knew they could access public funds, and a 13 percentage point increase of members who had been properly trained. Parents were also 2.6 percentage points more likely to know that a VEC existed in their community.

Impact on Engagement: Despite these improvements in awareness, there was little difference between the VECs’ performance in treatment and comparison villages. The only significant difference was that 20% more contract teachers were hired in Intervention 2 villages (although not in Intervention 3 villages). Also, the intervention did not increase the level of engagement of parents with schools. Parents were no more likely to have visited the school or to have volunteered time or money in the treatment villages than in the comparison villages.

Impact on Reading: In 55 of the 65 Intervention 3 villages, volunteers ran a total of 400 reading courses. The average child in an Intervention 3 village who could not read anything at the baseline was 7.9% more likely to be able to read at least letters. Those who could read only letters at baseline were 3.5% more likely to read at least paragraphs or words, and 3.3% more likely to read stories if they were in an Intervention 3 village. These changes in average literacy across the village came despite the fact that only 8% of children, including 13% of those who could not recognize letters prior to the intervention, attended the classes. If we assume that all the improvement in the villages came from the reading classes then children who attended the classes must have seen very big improvements in reading. In particular, children who could not read at baseline but attended classes ended up being able to read letters at endline, and 98% of children who could read at the word or paragraph level were able to read at the story level.

This was the only intervention which actually improved educational outcomes, by empowering individuals to improve teaching in their own communities. This suggests that enabling local action which does not depend upon large-group participation may be a means of directly affecting educational outcomes. 

Abhijit Banerjee, Esther Duflo, Rachel Glennerster, Sendhil Mullainathan

Encouraging Teacher Attendance through Monitoring with Cameras in Rural India

 

Getting teachers to attend school is sometimes more difficult than getting children there.  Can monitoring tied to a salary incentive reduce teacher absenteeism in rural schools in India?  Will reduced absenteeism increase student achievement?  The answer is yes: monitoring made teachers show up, and students learned more too!

Show Project Evaluation Details
Hide Project Evaluation Details

Policy Issue: 

Over the past decade many developing countries have expanded primary school access, energized by initiatives such as the United Nations Millennium Development Goals, which calls for achieving universal primary education by 2015. However, these improvements in school access have not been accompanied by improvements in school quality. Poor learning outcomes may be due, in part, to high absence rates among teachers, who often lack strong incentives to attend work. There have been relatively few rigorous studies evaluating successful interventions to address absenteeism, so little is known about how reduced absenteeism impacts other educational outcomes. If teachers are incentivized to show up to school, is that all they do- or once there do they teach? Do financial incentives undermine their other motivation to teach well?

Context of the Evaluation: 

Despite booming economic growth and an improved educational infrastructure in many regions in India, primary education is lagging in many remote and marginalized communities. Sixty-five percent of surveyed children enrolled in grades 2 through 5 in government primary schools could not read a simple paragraph, and 50% could not do simple subtraction or division. Teacher absenteeism, a pervasive problem in these schools, may contribute to these poor educational outcomes. Disciplinary actions are rarely undertaken against absent teachers: in a survey of 3,000 Indian government schools, only one principal reported a teacher having been fired for poor attendance.1 This may account for the extremely high rate of teacher absence in India: in schools examined by this study, teachers attended classes only 60% of the time, and much of the time when they were in class they were not teaching.

Details of the Intervention: 

This evaluation estimates the effect of incentives on teacher attendance and of increased teacher attendance on students' attendance and abilities in math and language. Seva Mandir, a local NGO, worked with researchers to randomly select 57 of their informal education centers for the intervention, and 56 for a comparison group. Ordinarily, teachers were paid a salary of Rs. 1,000 (about US$22) per month, for 21 days of teaching. In the intervention schools, each teacher was guaranteed a base pay of Rs. 500, and was rewarded with Rs. 50 for each complete day taught. Thus, a monetary incentive was attached to teacher attendance. When these incentives were implemented, monthly pay ranged from Rs. 500 to Rs. 1,300. 

In order to monitor teacher attendance, Seva Mandir gave each teacher a camera, along with instructions to have one student take a picture of the teacher and the class at the start and close of each school day. The camera's timestamp feature allowed Seva Mandir to determine when and for how long the teacher was at school. This technological monitoring was a relatively cost-effective method to monitor teacher attendance, since visits by monitors were reduced from daily to once every three weeks.

Results and Policy Lessons: 

Impact on Teacher Attendance: The program resulted in an immediate and long lasting improvement in teacher attendance rates in treatment schools. Over the 30 months of the study, teachers at program schools had an absence rate of 21%, compared to 44% at baseline and 42% in the comparison schools. Absence rates stayed low after 14 months of the program, suggesting that teachers did not change their behavior simply for the evaluation – their response was almost entirely due to the financial incentives. 

Impact on Education: Teachers who were at school were just as likely to be teaching in treatment compared to comparison schools- they did not just show up for the picture and go home. Student attendance on days the teacher was there was similar in both groups, meaning that students in the treatment group received more days of instruction simply because their teachers were more likely to be at school. A year into the program, test scores in the treatment schools were 0.17 standard deviations higher than in the comparison schools. Two and a half years into the program, children from the treatment schools were also 62% more likely to transfer to a formal primary school, which requires passing a competency test. These outcomes are hopeful. Increasing teacher attendance improves educational outcomes, and cost-effective incentive programs like this could potentially improve the opportunities for children in developing countries such as India.

Cost-Effectiveness: Expressed in terms of cost per outcome, this program cost approximately 11 cents for each additional instruction day per child, $60 per additional school year, and $3.58 per 0.10 standard deviations of increased test scores. Thus, the camera program is a relatively cost-effective program, both in terms of increasing instruction time and in terms of increasing learning.

Replicability in Formal Settings: The question arises as to whether the program can be instituted for regular teachers in government schools. Teachers in government schools are often more politically powerful than teachers in informal or private schools. Thus, it may prove difficult to institute a system in which government teachers would be monitored daily and their pay linked to attendance. However, the above evidence suggests that if teacher attendance can be improved this should flow through into improved test scores. 

1 Chaudhury, Nazmul, Jeffrey Hammer, Michael Kremer, Karthik Muralidharan, and F. Halsey Rogers. “Missing in Action: Teacher and Health Worker Absence in Developing Countries.”Journal of Economic Perspectives, Vol.20:1 (Winter 2006): pages 91-116.

Esther Duflo

Peace Education in Rural Liberia

For new democracies and societies emerging from conflict, encouraging tolerance and dialogue, strengthening non-violent conflict resolution systems, and increasing understanding of human rights are key priorities. Governments and NGOs commonly try to change the political culture, civic values, and practices of conflict resolution at the local level through widespread dialogue, education, and information campaigns.  But do these dialogue and education programs actually work as intended? Do they change norms and behaviors, and if so, how? How are new patterns of conflict resolution formed?  And how do they contribute to national reconciliation? How do new state structures integrate with pre-existing local bodies to jointly support security goals and human rights, especially where traditional structures are in conflict with the later? In short, what programs are most useful in helping post-conflict countries achieve lasting peace?

Find a more in-depth policy report here.

Show Project Evaluation Details
Hide Project Evaluation Details

Context of the Evaluation:

More than five years after the end of Liberia’s fourteen year civil war, underlying tensions between tribes, over land, and between youth and elders continue to pose threats to a fragile peace. The UN’s Peacebuilding Commission and the Government of Liberia are working together to promote non-violent dispute resolution and inter-group reconciliation, but how best to do this is unknown. This study was jointly designed by the UN, the government, and the research team to assess whether civic education and conflict resolution programs can contribute to this broader peacebuilding agenda.

Description of Intervention and Evaluation:

Target communities for the program were identified within Liberia’s three most conflict-prone counties: Grand Geddeh, Lofa, and Nimba. The researchers randomly assigned half to receive the program as the “treatment” group, and half to not receive the program as a “control” group. The program took place in 67 villages and town quarters. The program mobilized and trained community members in order to achieve three main goals: (1) educate people on their rights and to respect the rights of others; (2) encourage community collective action towards shared goals; and (3) foster non-violent dialogue and conflict resolution. The program is notable for its intensity and reach: in each community, roughly 10% of adults participated in an eight-day long interactive workshop held over the course of several weeks. Workshops had between 20 to 30 participants, both men and women, were led by a professional facilitator, and were conducted in local dialects. Multiple workshops were held in most communities to reach the 10% coverage target.

Pre-program baseline data was collected in 2009, and the endline took place between 1 and 22 months after the program. Data came from more than 5,000 individuals with three main surveys. In each community, the team interviewed: 20 randomly-selected “community members”, 4 “community leaders”, and 3 people identified by local chiefs as potential trainees, including a “troublesome” person. The study measures the impact of attending the program on potential trainees, random community members, and community leaders, and the impact on the community of having the program take place in their community. An in-depth qualitative study in 14  of the communities was conducted alongside the randomized evaluation to determine the mechanisms of impact. The study focuses on four major outcome classes: community and political participation; attitudes to rights; civic knowledge, attitudes and beliefs; and the prevalence and resolution of conflict.

Results and Policy Lessons:

Community and political participation: Community participation was measured through contributions to public goods and community projects, membership in groups (from farming to sports), membership in a peace group, and leadership in groups.  Across all measures, the only treatment effect was on membership in peace groups. On political participation, the only statistically significant treatment effect was seen on an index measuring whether potential trainees feel free to speak their minds to “big people” in the community and whether they feel community members have the right to speak out to elders: Those trained are 4% more likely to feel empowered to speak freely. This effect is concentrated among the “troublesome” individuals, who see a larger increase of about 8%.

Attitudes on human rights: Across multiple measures, nearly all the treatment effects are positive, indicating that respondents in trained communities generally report more progressive beliefs. For community members, however, these impacts are fairly close to zero. The impacts on trainees and leaders are modest in size – often in the range of 1 to 10% —and seldom statistically significant at conventional levels.

Civic attitudes and knowledge: The civic education component provided information on citizenship, civic rights and responsibilities and Liberia’s political structure. At endline, amongst potential trainees in the control group, only about 12% correctly understood the statutory requirements for citizenship. This understanding nearly doubled among treated trainees. Little change was seen in political knowledge, and the program also appears to have little to no impact on perceptions of equity in community governance as well as on perceptions towards the national government.

Prevalence and Resolution of Conflict: The most striking program impacts were on conflict and its resolution. In treatment communities (i.e. those that received the program), the evaluation found sizeable increases in non-violent inter-personal and inter-group disputes; suggestive evidence of a decrease in violent disputes; increasing levels of land conflict since the program, though also suggestions of lower rates of violence, and increased rates of dispute resolution and of satisfaction with those resolutions in trained communities. At the community level, leaders reported a 93% increase in conflicts (typically disputes and disagreements) between youth and elders in treated communities.  Treatment communities were also twice as likely to have a peaceful strike or protest, and three times as likely to have a witch killing (though the latter result is not statistically significant).

Violent strikes, protests, and inter-group violence are 59% less likely in treatment communities, however, though this result is not statistically significant (partly because the events are rare, making it difficult to estimate their prevalence precisely with such a small sample of communities).

In addition to the policy report detailing the impact evaluation results, the team has produced a second policy report analyzing patterns of conflict.

Chris Blattman

Information Campaigns and Voters' Behavior in the 2009 Municipal Elections in Mexico

Policy Issue:

It is widely held that access to information is a vital component of democracy building and government accountability. A recent World Bank report[1] champions information as “a tool to empower citizens in developing countries to hold their public agents accountable.” Information flows, the report argues, not only enhance democratic participation, but also make democracy work for ordinary people. However, while evidence suggests that access to information may lead voters to punish corrupt incumbents, it is unclear whether this translates into increased support for challengers and higher political participation. In other words, information about corruption may not improve political accountability, if voters respond to it by withdrawing from the political process.

Show Project Evaluation Details
Hide Project Evaluation Details

Context:

Despite optimistic views about fiscal decentralization in Mexico, local governments’ performance has remained poor. In 2008, for example, more than 80 percent of municipal governments’ resources were spent either on the bureaucracy or were unaccounted for. While elections should enable voters to discipline their mayors, a single-term limit is imposed on all elected officials in Mexico. Thus, the immediate fate of mayors is determined not by voters but by their political party. To reconcile the single-term limit with accountability, scholars have typically assumed that voters punish or reward the incumbent party for mayoral performance. However, there is little evidence that government performance impacts the subsequent election – previous work shows a strong entrenchment of incumbents from all political parties.

Further impeding voters’ ability to hold mayors accountable are widespread misconceptions about which public works and services municipal authorities are responsible for providing, as well as a lack of available information about the amount of money municipalities receive and how this money is spent. In an attempt to ensure greater municipal accountability, a 1999 constitutional reform established the Federal Auditor's Office (ASF). On a yearly basis, the ASF selects a sample of municipalities in each state to audit. The results of the audits are published in lengthy reports, which are made available online. Though public, these reports are rarely used by media or political parties in local campaigns because the release date of the reports is not aligned with the timing of elections.

Description of Intervention:

Researchers sought to assess the effects of information dissemination on participation in the 2009 municipal and congressional elections in Mexico. Approximately one week before Election Day, flyers with different kinds of information on municipal spending were delivered to all households within the boundaries of treated voting precincts. The first group of precincts received information about municipalities' overall spending; the second group received information about distribution of resources to the poor; and the third group received information about irregular, unauthorized, or unaccounted for spending. All flyers, including those for the comparison group, included a subtle advocacy message suggesting that voters raise questions with their mayors about the use of public funds and prompted people to think about the level of governments that was in charge of the provision of public infrastructure services. In total, 150 electoral precincts were randomly assigned to each of the three interventions, for a total of 450 treated precincts and 1910 precincts in the comparison group.

Researchers gathered demographic characteristics from census data, and then collected electoral results for each precinct from the electoral institutes for each state. This information was complemented by a follow-up survey collected ten days after the election.

Results:

Information about overall spending: Disseminating information about overall spending levels had no statistical impact on voter turnout when the mayor spent less than 75 percent of available funds. However, it decreased the incumbent vote share by 0.6 percentage points. When mayors spent more than 75 percent of the funds, releasing this information let to an increase in turnout of 1.9 percentage points, but had no significant impact on vote share.

Information about spending on poor: Turnout among voters who received information about levels of spending allocated to poor areas increased by 2.4 percentage points, but only when mayors spent less than 75 percent of available funds on poor areas. Surprisingly, releasing this information had a similar impact on both incumbent and challengers' vote shares - an increase of 1.5 and 1 percentage points, respectively. When mayors allocated more than 75 percent of resources to poor areas, releasing the information had no impact on turnout or vote share.

Information about corruption: Information about high levels of corruption appears to have had a significant negative impact on voter turnover. Turnout among individuals that received information about corruption decreased by 1.10 percentage points, which represents a 2 percent decrease in turnout. While the effect of information about corruption on the incumbent vote share was insignificant, the effect on the challengers' vote share was negative. These results suggest that while flows of information are necessary, they may be insufficient to improve political accountability, since voters may respond to information by withdrawing from the political process.

[1] Khemani, Stuti. 2007.  Can Information Campaigns Overcome Political Obstacles to Serving the Poor? The Politics of Service Delivery in Democracies. http://siteresources.worldbank.org/DEC/Resources/Khemani_CanInformationCampaignsOvercome.pdf.

 

Ana De La O, Dean Karlan

Improving Health Service Delivery Through Community Monitoring and Non-Financial Awards

Policy Issue:

In many developing countries, the health sector suffers from a severe human resources problem due to staff shortages and absenteeism. The availability of health care workers is a crucial element of quality care and the existing high levels of absenteeism represent a major leakage in health sector resources. Policy-makers have focused their attention on performance-based financing to incentivize attendance and performance monetarily; however, the evidence on the impact of financial incentives in improving performance in the health sector is mixed. While some programs report positive results, others show little to no effect on attendance and outcomes.

In contrast, recent results have highlighted the power of non-financial incentives to reduce absenteeism and improve performance.  Evidence suggests that peer recognition and status-based incentives can be more motivational, less expensive and less likely to erode intrinsic motivation.[1]  In addition, another study implementing a community monitoring initiative in Uganda, in which community members and health workers jointly addressed obstacles to adequate healthcare provision. The study found that under-five mortality was 33 percent lower in treatment compared to comparison communities a year later, while utilization for general outpatient services was 20 percent higher.[2]

Yet, the finding that non-financial incentives such as community monitoring improve clinic performance leaves a crucial question unanswered: does community monitoring improve clinic performance because it is a bottom-up intervention which makes clinic personnel socially accountable to their immediate neighbors? Or does it work simply because clinic performance is being monitored and evaluated? The answer to this question is important as top-down monitoring may be potentially cheaper and more efficient than bottom-up monitoring; however, data on this crucial question is lacking.

Show Project Evaluation Details
Hide Project Evaluation Details
Context of the Evaluation:

Sierra Leone’s health indicators are among the lowest in the world, and the country’s health system is plagued by such chronic worker absenteeism, resulting in part from a lack of accountability between service providers and patients, and the weak incentives healthworkers face. Alongside a national decentralization program introduced in 2004, the Government of Sierra Leone launched an ambitious policy in 2010 to institute free healthcare for pregnant women, new mothers and children under-five. The policy abolished user fees, while at the same time raising workers’ salaries. However, these reforms occurred without introducing institutional features to improve oversight of health workers or changing underlying incentive systems, leaving the health sector at risk of further weakening in response to rising demand for free health services.

Details of the Intervention:

This project evaluates two social accountability interventions aimed at improving health service delivery via community monitoring and the introduction of an incentive scheme to reward worker performance on the basis of non-financial awards. The 254 clinics taking part in the study have been assigned to participate in either intervention or act as a comparison, with one third of clinics allocated to each group.

The community monitoring intervention introduces health scorecards that provide information regarding the state of health care in each community, and facilitates interface meetings between community members and health facility staff. During these meetings, information about the state of healthcare is disseminated via a community scorecard and mutual commitments are made to improve services through a joint action plan addressing areas such as staff absenteeism, maternal mortality and vaccination rates. This framework aims to ensure participatory decision-making and hold both healthcare workers and the community mutually accountable, fostering increased access to and utilization of maternal and child health services. Researchers evaluate whether service quality and quantity improve due to the lower costs of collective action introduced through these meetings and the social accountability contract.  

The second intervention, non-financial incentives,facilitates a yardstick competition among groups of maternal and child health clinics, and rewards workers at the most improved facilities. The relative rankings of clinics on key measures of such as worker absenteeism, staff attitude and charging of illegal fees will be advertised publicly, and staff at winning clinics will receive letters of commendation from high-ranking politicians, and an award at a public ceremony.

The project is being conducted in partnership with the Government of Sierra Leone and the interventions have been designed with a self-sustainable model for scale-up through the Ministry of Health and Sanitation in mind. Researchers will assess the cost-effectiveness of each intervention, as well as their cost-effectiveness relative to one another, and findings will directly inform the government’s decision to scale up these interventions in future years.

Results:

Results forthcoming


[1]Ashraf, Nava, Oriana Bandiera, and Kelsey Jack. "No Margin, No Mission? A Field Experiment on Incentives for Pro-Social Tasks." Harvard Business School Working Paper, No. 12-008, August 2011.

Oendrila Dube, Johannes Haushofer, Bilal Siddiqi

Northern Uganda Social Action Fund – Youth Opportunities Program

Youth unemployment is widely considered a threat to development and to security. In the least developed nations, where firms are rare, aid-based employment interventions commonly provide inputs into self-employment to reduce poverty and social instability. Such programs are rooted in at least three assumptions. The first is that poor people have agency and are capable of making informed economic decisions. The second is that the poor possess high returns to investments but are constrained from reaching those returns unaided. The third is that increased income reduces youth alienation and aggression. This evaluation looks for evidence of all three claims in Uganda’s largest employment program.

Find a more in-depth policy report here and a policy note by the World Bank here.

Context of the Evaluation:

Twenty years of insurgency, instability and conflict have led to high rates of poverty and unemployment in northern Uganda. By 2005, a measure of peace and stability had returned to the region. The centerpiece of the post-conflict recovery plan was a decentralized development program, the Northern Uganda Social Action Fund (NUSAF). To stimulate employment growth through self-employment, in 2006 the government launched a new NUSAF component: the Youth Opportunities Program (YOP), which provided cash transfers to groups of young adults for self-employment in trades.

Show Project Evaluation Details
Hide Project Evaluation Details

Description of the Intervention:

The YOP intervention had two official aims: raise youth incomes and employment; and improve community reconciliation and reduce conflict. The program, targeted at youth from ages 16 to 35, required young adults from the same town or village to organize into groups and submit a proposal for a cash transfer to pay for:  (i) fees at a local technical or vocational training institute of their choosing, and (ii) tools and materials for practicing a craft.

The average applicant group had 22 members. Group cash transfers averaged nearly UGX 12.8 million (US$7,108), and varied by both group size and group request. The average transfer size per member was UGX 673,026 (US$374) – more than 20 times the average monthly income of the youth at the time of the baseline survey.

Due to vast oversubscription, the 535 eligible groups were selected at random, using a lottery, to either receive the YOP program (treatment group) or not (comparison group). A baseline survey was conducted with 2601 individuals in 2008, and 87 percent were successfully followed and interviewed in the endline survey two years later. Researchers are then able to compare the impact of the intervention across the two groups on an array of economic and social indicators.

Results and Policy Lessons:

Mid-term results are now available 2 years after the intervention. Long-term results will be collected in 2012.

Grant Use and Investments: Overall, and rather remarkably, the vast majority of beneficiaries make the investments they proposed: most engage in vocational training and approximately two-thirds of the transfer appears to be spent on fees and durable assets (not including other startup costs or materials), suggesting that the fears over funds mislaid and misspent are confined to a minority of beneficiaries.

Relative to the comparison group, the average beneficiary received 405 more hours of training and acquired additional business assets worth UGX 656,000 (US$300) since the intervention – increases of 814 percent and 481 percent, respectively.

Employment, Incomes and Wealth:Beneficiaries experienced sizable economic impacts as a result of these investments. The average beneficiary was nearly 100 percent more likely to be engaged in skilled employment, and saw hours spent on market activities increase by roughly a third, relative to non-beneficiaries. Cash earnings increased by nearly 50 percent on average, and household wealth also increased relative to the comparison group, although by a lesser amount. Consistent with these income and wealth gains, treated subjects perceive themselves as doing economically better than fellow community members. They report increases in perceived wealth levels relative to the comparison group and similar increases in access to basic services in their community.

Social Cohesion, Engagement and Stability: In general, we see modest increases, of the order of 0 to 10 percent, in common community participation and other indicators of social and community support. Treated individuals are engaged more in community groups than comparison individuals, and are more likely to speak out at and mobilize for community meetings. Men who participated in YOP reported decreases in aggressive behavior in contrast to the comparison group. Conversely, female beneficiaries reported heightened levels of aggression and hostile behaviors, but it is important to note that overall levels of hostile behaviors are quite low, with fewer than 40 percent of men and women reporting any aggressive behaviors at all.

Governance and Corruption: While popular media reported some degree of leakage of YOP funds, the evaluation found little evidence of significant misuse of funds—fewer than 2 percent of groups reported funds being stolen before reaching the group and less than 1 percent reported the money never being disbursed amongst group members.

Lessons Learned: Overall, the mid-term results demonstrate that relatively unconditional cash transfers have the potential to be an efficient means of improving livelihoods and, potentially, improve social cohesion and stability.  

Consistent with other studies, findings indicate that many of the poor, especially males, have reasonably high returns to investment when capital is made available and without close supervision or conditionality. The results suggest that credit constraints and the lack of financial development in Uganda are substantial impediments to poverty alleviation. Cash grants or subsidized credit may be a means to achieve higher levels of stability and freedoms than otherwise available to the poor.

The results suggest further areas of research, such as unpacking the composite contributions of cash, assets and skills on improvements in livelihoods. Similarly, variation in grant size could help determine the extent to which individuals are credit-constrained. Future programs and evaluations provide an opportunity to learn more about which program models are most effective and most cost-effective.

The upcoming final phase of research will collect more detailed evidence on (i) the longitudinal effects of the YOP program, (ii) the distribution of economic impacts, (iii) why access to credit and capital does not appear to be the binding constraint on the youth, (iv) why larger per person grants do not lead to proportionally more impact, (v) the impact of employment on social stability, and (vi) how government transfers affect citizen attitudes and voter choices.

Chris Blattman

Extra Teachers in Kenya: Peer Effects, Pupil Teacher Ratios and Teacher Incentives

While the introduction of free primary education in many developing countries has been very successful at raising primary school enrollment, it has generally not been matched by increases in the number of government-salaried teachers, due to budget constraints. The resulting overcrowding of schools, as well as the influx of new students with little or no preparation, poses a new challenge to communities: how to ensure minimum quality of education.

Show Project Evaluation Details
Hide Project Evaluation Details

This project seeks to identify the main determinants of the quality of education, and how governments or donors can improve learning in this new context. Data is gathered from a pilot program that provides schools with funding to hire locally a contract teacher to address overcrowding. The contract teachers are paid approximately a quarter of the salary of regular civil service teachers. The NGO implementing the program is able to offer funding to 140 out of 210 eligible schools in the area; the 140 schools are randomly selected (treatment group) and remaining 70 serve as the control group. The treatment schools reduce class size in grade 1 by creating an additional section for that grade, taught by the contract teacher.

Of the schools that receive funding, half ("non-tracked schools") randomly assign students between the section taught by the contract teacher and the section taught by the regular civil service teacher. In the other half ("tracked schools"), students are assigned to sections based on levels of preparedness, and the contract teacher is randomly assigned to one of the two sections. Finally, some funded schools randomly receive additional training on local school committee oversight in order to monitor the locally-hired contract teacher.

By comparing various measures of education quality (student test scores and attendance, teacher attendance and effort) across schools, it is possible to isolate the effects of class size, local monitoring of teachers, and tracking by initial level of preparedness.

Esther Duflo, Pascaline Dupas, Michael Kremer

Community Driven Development in Sierra Leone

Policy Issue:

While the accountability and inclusiveness of institutions are often considered key determinants of economic performance, there is little agreement about exactly how institutions should be designed, how to move from a system of bad institutions to one with good institutions, and whether and how foreign donors can help in this process. One of the most popular strategies employed by donors to promote democratic and accountable institutions at the local level is “community driven development” (CDD). Typical CDD interventions combine flexible grants that communities can spend on local projects with requirements that decisions must be made in an inclusive and transparent manner and training on how to do this. The participation requirements aim to ensure that the projects funded reflect the needs of the community and facilitate learning by doing—i.e. the experiences gained in deciding how to spend project funds leave minority groups  better placed to participate in other community decisions after the project ends. While billions of dollars are spent on CDD programs, few studies provide rigorous evidence on their real-world impacts. Critics of CDD, and of decentralization in general, have raised the concern that decentralized funds will be captured or exploited by local elites.

Show Project Evaluation Details
Hide Project Evaluation Details
Context:

Scholars argue that frustrations with government incompetence and corruption, as well as the exclusion of women and young men from decision-making in the traditional chieftaincy system that coordinates the provision of many local public goods, fueled violence during Sierra Leone’s recent civil war.  To both prevent a return to violence and to stimulate economic development, the Government of Sierra Leone implemented a number of reforms that give communities, and vulnerable groups within them, a greater voice in local decision-making. Alongside a national decentralization program that re-established district-level councils, the government piloted a community-driven development project that went one step  further by providing small grants to be administered by village development committees. This extension down to the village level aimed  to establish more inclusive and accountable local decision-making infrastructure, rebuild trust, promote collective action, and provide minority groups (particularly women and youth) with experience in managing projects and making decisions  within their community. Researchers and the Decentralization Secretariat collaborated to evaluate whether this pilot, called the “GoBifo” Project (or “Move Forward” in Krio),  acheived these goals.

Description of Intervention:

Two hundred thirty-six villages from two ethnically and politically distinct districts were randomly allocated into a treatment group or a comparison group. Villages in the treatment group were regularly visited by a GoBifo facilitator, who helped community members create or revamp Village Development Committees (VDCs), set up bank accounts for the VDCs, establish transparent budgeting practices, and create village development plans that included specifics on how GoBifo grants would be used. The participation and inclusion of marginalized groups was central to this process – for example, each social group (women, youth, and adult men) came up with their own development plan, and these plans were then combined into a single unified vision. Women were often established as treasurer of the VDC and served as co-signatories on all project finances. A series of block grants totaling US$4,667  per community were given to implement local public goods and skills training projects that were identified in the village development plans.

Household surveys, which covered participation in local decision-making, attitudes to minorities, and engagement in collective action, as well as demographic and socioeconomic information, were collected in late 2005 and again in mid-2009, along with village-level focus group discussions. In addition, three structured community activities (SCAs) were conducted in late 2009, shortly after GoBifo activities had ended, to capture any persistent impacts on collective action, participation of minorities, and elite capture. The SCAs were designed to measure how communities responded to concrete, real-world situations in three areas where GoBifo had sought to change behavior: (i) raising funds in response to a matching grant opportunity; (ii) making a community decision between two comparable alternatives; and (iii) allocating and managing an asset that was provided for free.

Results and Policy Lessons:

The authors and project team agreed a set of hypotheses they would test at the start of the evaluation (in 2005) and wrote out a plan on exactly how the data would be analysed before looking at the data. This prevented selective “cherry picking” of results from the 318 variables collected.

Project Implementation and local infrastructure investment: The GoBifo project successfully established the village-level organizations and tools to manage development projects in nearly all cases. The distribution of project benefits within communities was equitable, leakage of project resources minimal, and minority participation high.

GoBifo villages had a larger stock of higher quality local public goods, such as a functioning primary school or community grain-drying floor, than comparison areas. There was also more market activity in treatment communities, including the presence of more traders and items for sale, suggesting short-run economic gains.

Institutional Change and Collective Action: There is no evidence that the program led to fundamental changes in local institutions or descision-making. Despite the fact that many women in treatment villages participated in GoBifo decisions, they were no more likely to voice an opinion in community meetings after the project ended or to play a leadership  role in other areas. Similarly, the establishment of a democratically elected village development committee that carried out multiple projects did not lead treatment villages to be any more successful at raising funds in response to a later matching grant opportunity.  Lastly, there were no program impacts on elite capture, although levels of capture were low in the research communities (at least as measured by the third SCA). 

Rachel Glennerster, Edward Miguel

Child Protection Knowledge and Information Network (CPKIN)

Policy Issue:

Children in post-conflict environments are a population of particular concern, exposed to violence, displacement, and death of family members. The impact of poverty means that child labor is common among children. Though no reliable data exists, indications are strong that sexual abuse and exploitation is a fact of life for many girls and teenage pregnancy has been identified as a child protection concern. In addition to the physical and mental health challenges posed by war experiences, these children remain susceptible to continued exploitation and abuse, often by those familiar to the child, during peace as refugees return home and community members cope with difficult pasts. 

Show Project Evaluation Details
Hide Project Evaluation Details
Context of the Evaluation:

The situation of children in Sierra Leone continues to be precarious as 27% of the 2.7 million children are identified as vulnerable, lacking the protection of a primary care giver. Formal structures for child protection exist within police stations and local Child Welfare Committees (CWCs), but capacity and community trust in such institutions’ efficacy are limited in many rural areas[1]. To reduce violence against children, existing social structures,most paramount, section, and village chiefs, are frequently leveraged to respond to reports of abuse and to mobilize local protective factors.

To respond to the resource constraints faced by formal child protection systems in Sierra Leone, the Ministry of Social Welfare, Gender, and Children’s’ Affairs (MSWGCA) is experimenting with a policy framework that explicitly links local chiefs and community members to the formal child protection sector.  This policy was articulated in a Memorandum of Understanding (MOU) signed in 2010 by the MSWGCA, CWCs, and the Council of Paramount Chiefs. The MOU and focal person system are currently being implemented in Moyamba and Pujehun districts, with potential to expand the program nationwide.  The MOU creates a framework through which chiefs, MSWGCA, CWCs, and the Family Support Unit (FSU) of the Sierra Leone Police can collaborate to protect children. Village chiefs appoint a focal person for child welfare in a public meeting of village residents. Focal persons report cases of child abuse to chiefs and chiefs, with the assistance of focal persons, follow up with the formal child protection actors to ensure that cases of child abuse are addressed appropriately.

To help focal persons connect informal systems for child protection to the formal roles of the FSU and MSWGCA, UNICEF and other local child protection NGOs have developed a training program that introduces focal persons to general approaches to child protection, as well as to their roles and responsibilities under the MOU.  Despite the grounding of these trainings in both international best practices and local norms and understandings, key stakeholders have expressed concerns about whether one-time trainings will provide focal points with sufficient capacity to effectively serve as a link between communities, chiefs, and formal child protection systems.  At the same time, repeated trainings or sustained in-depth monitoring by international or national child protection agencies is not sustainable in the context of rural villages in Sierra Leone.

Details of the Intervention:

Mobile phones and SMS messaging provide one potential avenue for bridging the gap between the need for sustained support of focal points and the high costs of transportation in rural Sierra Leone.  To leverage the growing use of this technology in Sierra Leone, a consortium of non-profit, academic, government, and private-sector stakeholders have collaborated in developing the Child Protection Knowledge and Information Network (CPKIN).  CPKIN (pronounced as “See Pikin” or “See the Child” in Krio, a language spoken throughout the country) is a system that will be used to send focal persons automated text messages asking questions about the welfare of children in their village and prompting them to engage in discussions in their community to help answer these questions.  Focal persons will then be encouraged to send their answers back to the central CPKIN system using a free text message.  These text message questions and responses will be sent and received through a software program that makes it possible to manage, organize, and analyze high volumes of outgoing and incoming messages with a large list of recipients

There are several hypotheses underpinning the design of the CPKIN program. The first hypothesis is that the process of receiving questions, discussing the questions with community members, and sending answers will encourage focal persons to critically examine the situation of children in their communities and then to act with the resources available in their villages. A second hypothesis is that having a focal person who is active with respect to reporting abuses, taking proactive steps to improve the welfare of children, and engaging community members in discussions regarding children has the potential to cause dramatic shifts in community level knowledge, norms, and practices regarding child protection, which in turn may lead to greater connections between informal and formal child protection systems.

Given the novelty of the CPKIN system as a child protection intervention, it is necessary to assess the extent to which sending regular, open-ended text messages to village focal persons using this system can increase the capacity of focal points and their communities to proactively and reactively respond to local child protection issues. In particular, the effectiveness of CPKIN and the validity of the underlying hypotheses will be evaluated using a randomized evaluation in 140 villages in Moyamba and Pujehun districts. The randomization will occur at the village level, with the 70 villages randomly assigned to the CPKIN treatment, and the remaining 70 villages comprising the comparison group, in which there is no intervention. In treatment villages, the village focal person will receive a one-on-one training on the overall aims of CPKIN, how to receive and send CPKIN messages, and how to use CPKIN as a starting point for engaging their community members on issues concerning child protection.  Shortly after this training, focal persons will begin receiving regular CPKIN prompts.

Baseline and endline surveys regarding child protection and child welfare will be conducted before and after the six to eight month intervention in all 140 villages in order to assess the relative effectiveness of the CPKIN support system, with survey questions designed to elicit two kinds of information:

1.)    The capacity of key stakeholders (children, parents, focal persons, chiefs, FSU, and CWCs—where they exist) to identify and respond to child protection issues.

2.)    The protection and support for children within their communities, and their overall level of welfare (including the prevalence of both abuses and protective factors).

Results and Policy Lessons:

Results forthcoming.


[1] Human Rights Watch. (2011). World Report 2011

 

Ryan Sheely

Community Based Rangeland Management in Namibia

Innovations for Poverty Action (IPA) is conducting an impact assessment of the Community Based Rangeland and Livestock Management (CBRLM) program in Namibia.  This program is part of a larger set of interventions in the agricultural sector designed to reduce poverty among the population of the northern regions of the country.  Many people in the area rely on cattle production for their economic livelihoods, however overuse of the communal grazing areas and suboptimal grazing practices threaten the long-term viability of the land and contribute to persistent poverty.  

Show Project Evaluation Details
Hide Project Evaluation Details

To increase the productivity of livestock and other animals using the land, the Namibia Millenium Challenge Compact funds a pilot program designed to help communities improve their livestock practices, address rangeland degradation, and improve market access.  

 

The evaluation is designed to test the impact of the various activities within the CBRLM intervention on household income, cattle productivity, and the condition of the rangeland. The intervention targets both inadequate information about appropriate cattle production practices and the social or other behavioral preferences of farmers.  At the moment, there is a collective action or “tragedy of the commons” problem – individual farmers are hesitant to reduce their herd’s impact on the rangeland because they are fearful that others will not follow suit which often results in overuse and degradation of the land.

Julian Jamison, Dean Karlan

Experiments to Improve Participation in a Recycling Program in Northern Peru

Economic growth in Latin America has come at the cost of increasingly acute environmental pressures. Expanding trade and consumption has led to increased waste generation and pollution requiring more developed solid waste disposal systems. Markets lack a price mechanism to internalize the environmental cost of this growth.  Policy makers often apply taxes, subsides or other mechanisms to attempt to align private incentives with public environmental preservation. Aside from altering financial incentives, growing evidence from psychology and behavioral economics research shows that behavior can successfully be influenced by leveraging social norms and emotions. 

Show Project Evaluation Details
Hide Project Evaluation Details

Policy Issue:

Economic growth in Latin America has come at the cost of increasingly acute environmental pressures. Expanding trade and consumption has led to increased waste generation and pollution requiring more developed solid waste disposal systems. Markets lack a price mechanism to internalize the environmental cost of this growth.  Policy makers often apply taxes, subsides or other mechanisms to attempt to align private incentives with public environmental preservation. Aside from altering financial incentives, growing evidence from psychology and behavioral economics research shows that behavior can successfully be influenced by leveraging social norms and emotions.  

 

Context:

Over 20,000 tons of solid waste are produced every day in Peru, most of which is dumped in waterways or informal trash heaps, making solid waste management an area of increasing concern for the country. PRISMA, a local NGO, operates a recycling program in Northern Peru whereby it trains and supports workers in forming associations that collect recyclables door to door from participating households. In addition to providing the informal workers with some initial tools and training, PRISMA further assists workers by canvassing the areas of operation to introduce the recyclers to the community and encourage the residents to segregate recyclable and take part in the recycling program. PRISMA was interested in identifying viable strategies to increase program uptake (34% at baseline), and reduce attrition of participating households from the program.

 

Description of Intervention:

Researchers worked with PRISMA to test a series of information messages aiming to improve take-up and participation its recycling program.

To improve take-up of households in communities where PRISMA planned to expand the program, researchers conducted a randomized evaluation to test the impact of different messages in eliciting program participation.  One week before the first PRISMA canvasser’s visit, a paper flyer was delivered with the a generic message about PRISMA’s program and one of nine specific messages eliciting pressures such as, social norm, peer comparison, conformity, authority, environmental or social benefits to increase participation. Households that owned a cell phone and were willing to share their number (about 35% of the sample) received text messages once a week with the specific message in addition to the flyers. 

With a sample of 1,785 existing participants, researchers tested strategies to reduce program dropout and increase the amount and quality of the recyclables collected. Plastic bins were randomly distributed to households participating in the study. Some bins had a sticker specifying which items could be recycled.  Household that provided cell phone numbers were randomly assigned to receive either a generic or personalized SMS reminders to recycle or to serve as a comparison group without SMS reminders. SMS messages were sent the day before the weekly visit by the recycler, for a period of six weeks. Data collection for this component of the study lasted for eight weeks and included a careful accounting of the quantity and quality of recyclables received.

 

Results

Results of treatments on program enrollment

The results reveal no statistically significant effects from the different treatment messages. No significant impact on program take-up for the information campaign conducted through flyers alone or through flyers with text messages were found. Messages conveying social norms and applying social pressure were not successful in leveraging behavioral change. Two interpretations for this outcome are proposed by the researchers: a) these messages and norms were not relevant in this context, b) the large presence of informal recyclers operating outside of the program rendered separation of recyclables at the household level a non-issue.

Results of treatments on compliance

Households who received plastic bins turned in recyclables 3-8 percent more of the times and produced on average more (about 0.2 kg) and more valuable recyclables (about 0.1 pesos).  This finding suggests that convenience of storing recyclables is a barrier to greater program participation. 

The SMS reminders had no significant impact on the level and quality of participation of households in the program, suggesting that forgetfulness is not a serious constraints among households enrolled in the program. There was no clear difference in recycling compliance between households who received plain bins and those who received bins with explanatory stickers.

Dean Karlan, Jeremy Shapiro, Jonathan Zinman

Effect of Matching Ratios on Charitable Giving in the United States

More than 89% of Americans donate to charity. Fundraisers are typically short on hard evidence about the best ways of attracting these dollars. We look at what happens when matching gifts are offered; a leadership gift that is conditionally committed by a donor to match the contributions of others at a given rate. We found that the match offer increases both the revenue per solicitation and the response rate.  However, larger match ratios (i.e., $3:$1 and $2:$1) relative to a smaller match ratio ($1:$1) had no additional impact.

Show Project Evaluation Details
Hide Project Evaluation Details

Policy Issue:

Throughout the world, charitable organizations are working to meet people’s basic needs and improve their quality of life, and these organizations often depend on the support of outside donors to finance their mission. Private donations comprise 75% of all charitable giving in the United States1 and experts predict that the combination of increased wealth and an aging population will lead to an even higher level of private gifts in the coming years. Such trends have left fundraisers, who typically rely on anecdotal evidence in lack of scientific evidence, divided as to the most efficient means to attract these dollars. While the economics of charity has been well studied on the “supply” side, critical gaps remain in our knowledge about the “demand” for charitable giving.

Context of the Evaluation: 

In the United States, private giving to charitable causes has grown significantly in the past several decades. Recent figures show that charitable gifts of money have been 2% or more of GDP since 1998 and currently more than 89% of Americans donate to charity.

Among the tactics commonly used in fundraisers to solicit donations are matching gifts. A matching gift is a commitment by a donor to match the contributions of others at a given rate, up to the maximum amount the leadership donor is prepared to give. While the rate of matching is typically the result of an agreement between the fundraiser and the leadership donor, fundraising consultants ubiquitously note that increases in the matching ratio have power to influence contributions. Such conventional wisdom, however, is largely anecdotal as little scientific study has been completed to examine such demand side claims.

Details of the Intervention:

This study seeks to explore the importance of price on charitable giving by measuring the comparative static effects of changes in rates of matching gifts. Partnering with a liberal U.S. nonprofit organization that works on social and policy issues relating to civil liberties, this study takes advantage of a capital campaign in which prior donors received direct mail solicitations seeking contributions.

Individuals were randomly assigned to either a comparison group or a matching grant treatment group. All individuals received a letter identical in all respects except two: (1) the treatment letters included an announcement that a “concerned fellow member” will match their donation, and (2) the reply card included the details of the match. The specifics of the match offer were then randomized along three dimensions:

(1) Price ratio of the match: 1:1, 2:1 or 3:1 (the 2:1 ratio means, for example, that for every dollar the individual donates, the matching donor contributes $2),
(2) Maximum size of the matching gift across all donations: $25,000, $50,000, $100,000 or unstated,
(3) Example donation amount suggested to the donor equal to: (i) the individual’s highest previous contribution, (ii) 1.25 times the highest previous contribution, or (iii) 1.5 times the highest previous contribution.

Because fundraising solicitations are sent to all 50 states, it is possible that utilitarian effects of contributing to a politically-motivated charity are different spatially due to the local political environments. To test for these effects, charitable giving data is merged with (i) demographic data, (ii) state returns from the 2004 presidential election and (iii) data on the frequency of the organization’s activities within each state.

Results and Policy Lessons: 

Matching Impact: Simply announcing that match money is available considerably increases the revenue per solicitation—by 19%. In addition, the match offer increases the probability that an individual chooses to donate by 22%. This finding supports anecdotal evidence among fundraising consultants on the efficacy of a matching mechanism.

While the match treatments relative to a comparison group increase the probability of donating, contrary to conventional wisdom, larger match ratios (2:1 and 3:1) relative to smaller match ratios (1:1) have no additional impact. This result directly refutes the rationale for using larger match ratios, and stands in sharp contrast to current fundraising practices.

Heterogeneous Treatment Effects: Study findings indicate that the results of the matching gift are driven by agents in states that voted for George Bush in the 2004 presidential election: the match increases the revenue per solicitation by 55% in “red” states whereas there was little effect observed in “blue” states. This result suggests that an individual’s political environment also has the capacity not only to influence the level of giving, but their responsiveness to different treatments.

1 Giving USA Foundation, “U.S. charitable giving estimated to be $307.65 billion in 2008,”http://www.philanthropy.iupui.edu/News/2009/docs/GivingReaches300billion....

Dean Karlan, John List

Comparing Charitable Fundraising Schemes in Germany

Policy Issue: 

In 2002, more than 241 billion dollars were given to charities in the United States, and 75% of charitable giving came in the form of individual donations. Despite the fact that these organizations often depend on the support of outside donors to finance their mission, fundraisers are divided as to the most efficient means of attracting donations. The disagreement arises in part because there is minimal evidence on the relative effectiveness of alternative fundraising schemes.

Show Project Evaluation Details
Hide Project Evaluation Details

Context of the Evaluation: 

The Bavarian State Opera House was founded in Munich, Germany in the 17th Century . In recent years, the Bavarian State Opera house has invested in the musical education of children to ensure that appreciation for the operatic art form does not disappear in future generations. The Opera’s most recent fundraising project is entitled “Stück für Stück” and strives to provide a musical education for school children from socially disadvantaged areas. Although the Bavarian State Opera has never previously used mailouts as a form of fundraising and the practice is not common in Germany, the program Stück für Stück will be partially funded by the individual donations received in response to a mail solicitation. The average recipient of the Opera’s solicitation purchased six tickets at an average cost of €86 per ticket in the year prior to the mailout.

Details of the Intervention: 

Researchers worked with the Bavarian State Opera to organize a mailout designed to elicit donations to support Stück für Stück. A total of 25,000 recipients were randomly selected from the opera’s database of customers who had purchased at least one ticket to attend either the opera or ballet, in the twelve months prior to the mailing.

Each addressee was randomly assigned to receive one of six mailing treatments:

  1. Comparison - A straightforward request for a donation.
  2. Lead Donor (Signaling) -  Information about an anonymous lead donor who committed €60,000 (over 400 times the average donation).
  3. 50% Matching -  An explanation of a matching scheme in which an anonymous donor would donate 50 cents for each euro donated by the recipient (up to a total of €60,000).
  4. 100% Matching - An explanation of a matching scheme in which an anonymous donor would donate an additional euro for each euro donated by the recipient (up to a total of €60,000).
  5. Matching for >€50 (Non-linear Matching) - An explanation that an anonymous donor would match each donation greater than €50 by donating an equivalent sum (up to a total of €60,000).
  6. Additional €20 for each donation (Leveraged Matching) - An explanation that an anonymous donor would contribute an additional €20 for each donation made, regardless of the size of the donation (up to a total of €60,000).

The lead gift was actually provided by an anonymous donor and each matching scheme was implemented. The opera did not have a fundraising target in mind, and none of the mailings indicated a specific financial target. Non-German residents, corporate donors, and formally titled donors were removed from the initial pool of 25,000 resulting in a total of 22,512 addressees.

Results and Policy Lessons: 

The study found that the average rate of response varied between 3.5% and 4.7% across the six mailing treatments. Providing information about the presence of a substantial lead donor nearly doubled the size of the average donation, while providing information about the existence of a matching scheme resulted in a decrease in the size of the average donation, and an offer to match donations in excess of €50 substantially increases the number of patrons donating between €51 and €60 (from 0 to 20.6%).

The results from each of the six treatments are summarized in the table below:

Treatment Description

Response Rate

Average Donation Given

Average Donation Received (including
matching funds)

1) Comparison

3.7%

74.3

74.3

2) Lead Donor

3.5%

132

132

3) 50% Matching

4.2%

101

151

4) 100% Matching

4.2%

92.3

185

5) Matching for >€50

4.3%

97.9

194

6) Additional €20 for each donation

4.7%

69.2

89.2

Despite the frequent use of matching schemes in fundraising campaigns, the results of the study indicate that the presence of a matching scheme reduces the size of the average donation. The total amount received including the donor’s matching funds increases, but in matching schemes, the donor’s money has the undesirable consequence of diminishing the total amount of funding that recipients are willing to contribute.

Imran Rasul

Decentralizing Education Expenditures: Primary School Community Grants in Niger

Policy Issue: 

In recent years school enrollment has risen dramatically in developing countries, prompting a renewed interest among policymakers in education management. Provision of educational services is centrally administered in much of the developing world, but evidence suggests that decentralized, locally administered services may be better suited to address low education quality. Local administration and oversight puts power into the hands of those with the most interest in seeing improvements in service delivery, and the best information about current education quality. By empowering local communities, decentralized management has the potential to combat systemic teacher absenteeism and reduce misallocation and corruption.

Show Project Evaluation Details
Hide Project Evaluation Details

Context of the Evaluation: 

Niger has made significant strides in increasing access to education in the past decade, with primary enrollment increasing from 37% in 2000 to 54% in 2005. Yet the country still faces considerable challenges with regards to access, rates of primary school completion and the quality and management of the educational system at all levels. One problem, for example, is that schools are spaced far apart, making it difficult for the central government to effectively monitor them.

To address this problem the Ministry of Education mandated the creation of management committees (Committees de Gestion Scolaire, or COGES) for each school. The committee included the director of the school, as well as locally-elected community members. The COGES are responsible for monitoring teacher attendance and performance and managing both financial and material resources, such as the purchase of textbooks and supplies. The Ministry of Education introduced a pilot project to reinforce the capacity of the COGES through the provision of a yearly cash grant, which can be used to fund a part of its activities for the year, such as the construction of classrooms or the purchase of materials. These grants were provided under the belief that such financial support would spur communities to take a more active role in the management of schools.

Details of the Intervention: 

To test this belief, researchers, in collaboration with the World Bank and the Nigerian Ministry of Education, randomized the allocation of the grants. They observed schools which did and did not receive grants, to estimate their impact, as well as the impact of two supplementary accountability interventions on school management, student retention and learning achievement. One thousand schools in the regions of Tahoua and Zinder were randomly selected into treatment and comparison groups. The 500 schools in the treatment group each received an annual lump sum based on the number of classrooms in the school, averaging out to around US$2 per student. This grant size was chosen by the Ministry of Education with considerations of sustainability regarding a possible extension of the program. In addition to the grants themselves, two sub groups were randomly selected to receive complementary interventions designed to improve delivery and management of the grants.

One third of the treatment schools received financial monitoring in the form of a letter received at the beginning of the intervention telling the committee there was a 50% chance of a “financial control visit” at the end of the school year. Because of financial and capacity constraints, about 20% of the treatment schools received visits at the end of the year.

Another third of the treatment schools received expenditure restrictions in the form of a list of non-eligible expenditures, such as reimbursing school officials for travel, or spending money on things that can be produced locally.

Researchers used annual administrative survey data, pupil test score data, and a detailed financial questionnaire to determine the effect of the grants. The testing used a modified standardized test implemented in collaboration with the Ministry of Education Department of Education Assessment. The school data contained information on pupil and community participation, school resources and infrastructure and information on issues facing school governance. The functioning of the COGES, including how often they met, what they talked about, and whether they began improvement projects, was also monitored. 

Results and Policy Lessons: 

Results forthcoming.

Decentralization: A Cautionary Tale - Public Finance in Kenya

Kenya’s education system blends substantial centralization with elements of local control and school choice.  This project looks at the system of incentives created by elements of decentralization.

Show Project Evaluation Details
Hide Project Evaluation Details

Michael Kremer

Primary Education Management in Madagascar

Policy Issue: 

Successful efforts to expand access to education in the developing world have not always translated into actual improvements in skills and learning for students. It remains an open debate as to whether top-down approaches are more effective in improving educational quality than approaches which promote beneficiary participation, such as parental monitoring. Top-down approaches can provide administrators with the tools necessary to better monitor their schools, but this assumes that they have the incentives to do so. Bureaucrats may have more incentives to improve the quantity, rather than quality, of education services since the benefits of improving quality are diffuse and harder to verify. The combined effects of these perverse incentives can result in a large number of children who are in the classroom, but are not learning. Promoting local accountability may be a useful means of improving schooling outcomes, if it can be determined which factors makes beneficiary participation effective at improving education delivery.

Show Project Evaluation Details
Hide Project Evaluation Details

Context of the Evaluation: 

Madagascar divides its 2.7 million children into 15,000 public primary schools. Despite the significant increase in primary school enrollment following Madagascar's 2002 reforms and an influx of international financial support, resource allocation across schools remains inefficient, and better resource endowments rarely translate into better student performance. Only 63% of grade 5 children pass the primary-cycle exam, an assessment of the minimum level language and math knowledge presumed at this grade. District administrators face a performance review only every 3 years, and the subdistrict heads rarely face any credible threat of penalties or firing.

Details of the Intervention: 

Researchers, in collaboration with The Ministry of Education in Madagascar, ran a randomized experiment in 3,774 primary schools in 30 public school districts. These districts represented all geographic areas in the country, but were focused on schools with the higher rates of grade repetition.

All district administrators in treatment districts received operational tools and training that included forms for supervision visits to schools, and procurement sheets for school supplies and grants (district-level intervention). In some of these schools, the subdistrict head was also trained and provided with tools to supervise school visits, as well as information on the performance and resource level at each school (subdistrict-level intervention).

Lastly, several districts also introduced a school level intervention which involved parental monitoring through school meetings. Field workers distributed a ‘report card’ to schools, which included the previous year’s dropout rate, exam pass rate, and repetition rate. Two community meetings were then held, and the first meeting resulted in an action plan based on the report card. One example of the goals specified in the action plans was to increase the school exam pass rate by 5 percentage points by the end of the academic year. Common tasks specified for teachers included lesson planning and student evaluation every few weeks. The parent’s association was expected to monitor the student evaluation reports which the teachers were supposed to communicate to them. These tools allowed parents to coordinate on taking actions to monitor service quality and exercise social pressure on the teachers.

Results and Policy Lessons: 

Impact from Top-Down Approach: The interventions targeted at the district and subdistrict level had minimal effects on the administrator’s behaviors, and the schools and students under their responsibility. Though each tool – forms for supervision visits to schools and procurement sheets for school supplies and grants – was used by 90% of subdistrict heads and more than 50% of district heads, subdistrict heads visited their schools only slightly more often than those in the comparison group, an insignificant improvement. Teachers in both groups did not plan for lessons more, and no improvement in test scores was seen in the two years following the program.

Impact from Bottom-Up Approach: The interventions at the school level led to significantly improved teacher behavior. Teachers were on average 0.26 standard deviations more likely to create daily and weekly lesson plans and to have discussed them with their director. Test scores were 0.1 standard deviations higher than those in the comparison group two years after the implementation of the program. Additionally, student attendance increased by 4.3 percentage points compared to the comparison group average of 87%, though teacher attendance and communication with parents did not improve.

 
Esther Duflo
Syndicate content
Empowered by:
Advomatic
Birocreative
Copyright © 2011 Innovations for Poverty Action. All rights reserved.