Information and Community Mobilization in Rural India

Policy Issue: 
While primary school enrollment rates have risen sharply in much of the world, the quality of education remains low in many countries. Many children who attend school regularly are still unable to read or do basic arithmetic. For instance, a 2008 survey found that in rural India only 56 percent of children in grade 5 could read at the grade 2 level, and nearly 20 percent could not read beyond a single word. Community oversight and participation has been advocated to increase education quality. Does this strategy work, and if so how can community participation be encouraged? Is more direct action by communities to teach their children to read more effective?
 
Context of the Evaluation: 
In Uttar Pradesh, India’s most populous state, only 43.5 percent of children in grade 5 can read at the grade 2 level. In response to this problem, the government established Village Education Committees (VECs) in every village in 2001. VECs consist of the elected head of the village government, the head teacher of the local school, and three parents who are nominated by their community. These committees are responsible for monitoring school performance, allocating school resources, and hiring additional contract teachers in the event of overcrowding. 
 
Despite the promising aspects of this program, a survey conducted in 2005 indicated that 38 percent of VEC members did not readily identify as being part of the committee, and 25 percent did not even know they had this role. Only 3.6 percent of all VEC members knew they had the ability to request funds to hire additional teachers, which is one of the main prerogatives and responsibilities of the VEC. 
 
Details of the Intervention: 
Working jointly with Pratham (a local NGO) and the World Bank, researchers designed three interventions that were randomly assigned in 280 villages in four rural blocks in Jaunpur district, eastern Uttar Pradesh, a populous and educationally struggling area in India. These interventions served to determine if more information and encouragement to use the channels available to them would cause VECs and community members to demand and receive better services. They contrasted this with direct action to improve learning outside the official channels. 
 
Intervention 1: In 65 villages, Pratham staff started a series of conversations about education in small groups throughout the community. These conversations covered the current status of schools in the village, the quality of local schools, state mandated provisions for schools, mid-day meals, and local funds available for education. People were asked if they knew about the VEC and its membership and responsibilities. After two days of meetings in small groups, a community-wide meeting was held where people were encouraged to discuss and ask for information about the VEC, with information gaps filled in by Pratham’s field workers. VEC members also received a pamphlet on their roles and responsibilities from the Pratham staff.
 
Intervention 2: In addition to all the steps outlined above, communities in another 65 villages were trained and encouraged to conduct testing to see if children could read simple text and solve basic arithmetic problems. Volunteers put together a "report card" for each community, which was presented at the community-wide meeting. 
 
Intervention 3: In addition to the above two steps, Pratham officers taught volunteers in another 65 villages a simple technique for helping children learn to read. Volunteers were encouraged to start after-school reading classes-- they were invited to attend training sessions which lasted for four days, and staff returned an average of seven times to provide in-service training. The objective was to use Pratham-designed materials and local volunteers to supplement the normal curriculum, and improve literacy among village children. 
 
Comparison: Eighty-five villages received no treatment, serving as a comparison.
 
Results and Policy Lessons: 
 
Impact on Information Gaps: The average effect of all three treatments was an increase of 7.8 percentage points in VEC members who knew they could access public funds, and a 13 percentage point increase of members who had been properly trained. Parents were also 2.9 percentage points more likely to know that a VEC existed in their community.
 
Impact on Engagement: Despite these improvements in awareness, there was little difference between the VECs’ performance in treatment and comparison villages. The only significant difference was that 20 percent more contract teachers were hired in Intervention 2 villages (although not in Intervention 3 villages). Also, the intervention did not increase the level of engagement of parents with schools. Parents were no more likely to have visited the school or to have volunteered time or money in the treatment villages than in the comparison villages.
 
Impact on Reading: In 55 of the 65 Intervention 3 villages, volunteers ran more than 400 reading courses. The average child in an Intervention 3 village who could not read anything at baseline was 7.9 percent more likely to be able to read at least letters. Those who could read only letters at baseline were 3.5 percent more likely to read at least paragraphs or words, and 3.3 percent more likely to read stories if they were in an Intervention 3 village. These changes in average literacy across the village came despite the fact that only 8 percent of children, including 13 percent of those who could not recognize letters prior to the intervention, attended the classes. Provided that the effects of Intervention 3 are channeled entirely through attendance at the reading classes, comparing the endline reading levels of the comparison group with the Intervention 3 treatment effects described above reveals just how large these effects are:  all children who could not read at baseline but attended classes ended up being able to read letters at endline, and 98 percent of children who could read at the word or paragraph level was able to read at the story level. 
 
Intervention 3 was the only intervention which actually improved educational outcomes, by empowering individuals to improve teaching in their own communities. This suggests that enabling local action which does not depend upon large-group participation may be a means of directly affecting educational outcomes.  

Encouraging Teacher Attendance through Monitoring with Cameras in Rural India

Getting teachers to attend school is sometimes more difficult than getting children there. Can monitoring tied to a salary incentive reduce teacher absenteeism in rural schools in India?  Will reduced absenteeism increase student achievement?  The answer is yes: monitoring made teachers show up, and students learned more too!

Policy Issue: 

Over the past decade many developing countries have expanded primary school access, energized by initiatives such as the United Nations Millennium Development Goals, which call for achieving universal primary education by 2015. However, these improvements in school access have not been accompanied by improvements in school quality. Poor learning outcomes may be due, in part, to high absence rates among teachers, who often lack strong incentives to attend work. There have been relatively few rigorous studies that have evaluated successful interventions to address absenteeism, so little is known about how reduced absenteeism impacts other educational outcomes. If teachers are incentivized to show up to school, is that all they do- or once there do they teach? Do simple financial incentives undermine their motivation to teach well?

Context of the Evaluation: 

Despite booming economic growth and an improved educational infrastructure in many regions in India, primary education is lagging in many remote and marginalized communities. Sixty-five percent of surveyed children enrolled in grades 2 through 5 in government primary schools could not read a simple paragraph, and 50 percent could not do simple subtraction. Teacher absenteeism, a pervasive problem in these schools, may contribute to these poor educational outcomes. Disciplinary actions are rarely undertaken against absent teachers: in a survey of 3,000 Indian government schools, only one principal reported a teacher having been fired for poor attendance . This may account for the extremely high rate of teacher absence in India: in schools examined by this study, teachers attended classes about 65 percent of the time.

Details of the Intervention: 

This study estimates the effect of incentives on teacher attendance and of increased teacher attendance on students' attendance and abilities in math and language. Seva Mandir, a local NGO, worked with researchers to randomly selected 57 of their informal education centers for the intervention, and 56 for a comparison group. Ordinarily, teachers were paid a salary of Rs. 1,000 (about US$23) for at least 20 days of work per month, and this payment structure remained unchanged for the comparison schools. In the intervention schools, teachers received a Rs. 50 (US$1.15) bonus for each additional day they attended in excess of 20 days, and received a Rs. 50 fine for each of the 20 days they skipped work—however, the fine could not exceed Rs. 500 total, due to ethical and political concerns.. Thus, a monetary incentive was attached to teacher attendance. When these incentives were implemented, monthly pay in the treatment schools ranged from Rs. 500 to Rs. 1,300 (US$11.50 to US$29.50).  

In order to monitor teacher attendance, Seva Mandir gave each teacher a camera, along with instructions to have one student take a picture of the teacher and the class at the start and close of each school day. The camera's timestamp feature allowed Seva Mand  ir to determine when and for how long the teacher was at school.

Results and Policy Lessons: 

Impact on Teacher Attendance: The program resulted in an immediate and long lasting improvement in teacher attendance rates in treatment schools. Over the 30 months of the study, teachers at program schools had an absence rate of 21 percent, compared to 44 percent at baseline and 42 percent in the comparison schools. Even four years after the start of the program, teacher attendance remained significantly higher in treatment schools (72 percent attendance) than in comparison schools (61 percent), suggesting that teachers did not change their behavior simply for the evaluation – their response was almost entirely due to the financial incentives.  

Impact on Education: Teachers who were at school were just as likely to be teaching in treatment schools compared to comparison schools- they did not just show up for the picture and go home. Student attendance on days the teacher was there was similar in both groups, meaning that students in the treatment group received about 9 percentage points (or 30 percent) more days of instruction on average simply because their teachers were more likely to be at school. This corresponds to 2.7 more days of instruction time a month at treatment schools, and this effect is larger than that of other successful interventions that have been shown to increase child attendance. A year into the program, test scores in the treatment schools were 0.17 standard deviations higher than in the comparison schools. Two and a half years into the program, children from the treatment schools were also 10 percentage points (or 62 percent) more likely to transfer to a formal primary school, which requires passing a competency test.

Replicability in Formal Settings: The question arises as to whether the program can be instituted for regular teachers in government schools. Teachers in government schools are often more politically powerful than teachers in informal or private schools. Thus, it may prove difficult to institute a system in which government teachers would be monitored daily and their pay linked to attendance. However, the above evidence suggests that if teacher attendance can be improved this should flow through into improved test scores.

Selected Media Coverage:

 

 

Esther Duflo, Rema Hanna

The Impacts of Psychosocial Support and Cash for Work on Vulnerable Youth in Liberia

To foster economic productivity and stability in post-conflict countries, many development organizations have created economic programs and psychosocial programs for youth. Yet little evidence exists on ways to increase economic opportunities for young people and reduce their risk of participating in violence and risky behaviors. In this study, researchers evaluate the impact of both a cash-for-work program and a psychosocial support program for young people in urban Liberia. This research will contribute evidence on how to best enable highly vulnerable youth in post-conflict settings to pursue productive opportunities.

Policy Issue:

Youth development programs are widely viewed as a critical tool for preventing the re-escalation of violence in post-conflict countries. Ninety percent of violent onsets occur in countries with a previous conflict,1 and when conflicts end, young people have the potential to contribute to future development, but they may also be a force of instability in their societies.2 In situations of armed conflict, youth are more likely to be recruited into fighting forces, become targets for violence, be forced to generate a livelihood for themselves and others, and miss out on an education.3 Many organizations therefore identify youth development as an urgent priority for building peace and spurring economic development, and investment in such programs has risen substantially in recent years.4  Yet little evidence exists regarding the impact of these programs, and which interventions are most effective in increasing economic opportunities for youth or reducing their risk of participating in violence and risky behaviors. This study will contribute evidence to help fill this gap.

 
Context of the Evaluation:

Liberia emerged out of a 14-year long civil war in 2003, and the security situation remains fragile and economic development has been slow.5 In Liberia, where people between the ages of 15 and 35 years constitute over 60 per cent of the total population,6 the government has identified youth unemployment as a major challenge.7

Mercy Corps, a global aid agency, is implementing a three-year program in Liberia called Promoting Sustainable Partnerships for Economic Transformation, or PROSPECTS. The program combines life skills training and workforce development via apprenticeships, technical training, financial education and mentoring, and aims to help participants find gainful employment while also developing positive social ties with their communities. Mercy Corps has worked in Liberia since 2003.

PROSPECTS includes a Cash for Work (CFW) program that centers on paying youth to collect recyclables in their communities and a Sports for Change (SFC) program that combines sports with life skills sessions for groups of youth. These interventions are designed with the aim of preparing highly vulnerable youth in Monrovia, Liberia for the employment market by developing self-confidence and resilience./p>

 
Details of the Intervention:

This study tests the impacts of the Cash for Work and Sports for Change programs, and any synergistic effects of both programs, on risky and violent behaviors and labor outcomes of vulnerable youth. Researchers are carrying out the randomized evaluation with 3,000 out-of school youth, ages 16 to 25, in urban Monrovia.

Through a public registration and lottery, participants were randomly assigned to a group of peers. There were a total of four groups, which serve to compare different programmatic options:

  • Cash for Work program only
  • Sports for Change program only
  • Both Cash for Work and Sports for Change
  • No program

Both programs will run for approximately three months with two sessions typically held per week. Researchers will measure the different programs, and any synergistic effects of the two, on enabling vulnerable Liberian youth to become economically productive adults, and resist risky behaviors over a one-year period. Additionally, researchers will evaluate the cost-effectiveness of the programs, both together and separately.

Data will be collected on the participants’ stress levels (measured using biometric data), risk preferences, labor outcomes, aggression, and self-reported risky behaviors. Social network mapping – how youth interact with one another – will also be used to provide a rich perspective of linkages and ties of Liberian youth within and outside of their traditional communities.

 

The Impact of Women Policy Makers on Public Goods in India

Policy Issue:

In 2008, women accounted for 18 percent of parliament members worldwide, and only 13 countries had a female head of government. This underrepresentation has prompted governments to adopt reservation policies; assuring women will have a role in political leadership by reserving a certain proportion of seats for female candidates. Reservation policies clearly have a strong impact on women’s representation, and some evidence suggests that women and men have different policy preferences. However, very little is known about the actual impact of women’s representation on policy decisions.

 
Context of the Evaluation:

In 1993, a constitutional amendment in India called for a random one third of village council leader, or pradhan, positions to be reserved for women. The village council, which encompasses between five and fifteen villages, is responsible for the provision of local infrastructure – such as public buildings, water, and roads – and for identifying government program beneficiaries. Although all decisions in the village council are made by majority, the pradhan is the only full-time member and exercises significant control over the final council decisions. The village council is required to organize two village meetings per year, during which they present their proposed budget and report on their activities in the previous six months. The pradhan must also set up regular office hours where villagers can lodge complaints or requests.

 
Details of the Intervention:

Researchers studied the policy consequences of mandated representation by determining whether there was any difference in the provision of social services between male and female led village councils. As village councils were randomly selected to be reserved for women, differences in investment decisions can be attributed to the reserved status of the council.

Data was collected in two locations: Birbhum in West Bengal and Udaipur in Rajasthan. In Birbhum, data was collected in two stages. First, in each village council, researchers conducted an interview with the pradhan, asking about his or her family background, education, previous political experience, political ambitions, and the village council’s recent activities. In the second stage, from the 5-15 villages represented by each village council, three villages were randomly selected to be surveyed for available public goods and existing infrastructure. Researchers also collected minutes of the village meetings, and gathered data on what complaints or requests had been submitted to the village council in the last six months. Two years later, the same village level data was collected from 100 village councils in Udaipur. However, there were no pradhan interviews.

 
Results and Policy Lessons:

The results suggest that reservation affected policy choices. In particular, it affected policy decisions in ways that seem to better reflect women’s preferences. In West Bengal, women complained more often than men about drinking water and roads – 31 percent of women’s complaints were about drinking water and 31 percent were about road improvement, compared to 17 percent and 25 percent of men’s, respectively. These preferences were revealed in the investment decisions of reserved village councils. Village councils reserved for women, on average, invested in 9 more drinking water facilities and improved road conditions by 18 percent.

In Rajasthan, 54 percent of women’s complaints were about drinking water and 19 percent were about welfare programs, compared to 43 percent and 3 percent of men’s, respectively. Unlike in West Bengal, compared to men, women complained less frequently about roads. Only 13 percent of women’s complaints were about roads, compared to 23 percent of men’s. This breakdown of preferences was again revealed in the investment decisions of the village councils. Village councils reserved for women invested in 2.62 more drinking water facilities, on average, and made fewer improvements in road conditions, leading to an 8 percent deterioration.

Researchers investigated whether these results could be explained by the fact that female pradhans were inexperienced, that they perceived themselves as being less likely to be re-elected, or that they tended to come from more disadvantaged backgrounds than men. However, there was no evidence that the policy impact of the reservations was driven by features other than the gender of the pradhan.

Overall, these results indicate that a politician’s gender, and identity more generally, does influence policy decisions. This finding is likely to have implications beyond reservation policy.

Selected Media Coverage:
 

Improving Health Service Delivery Through Community Monitoring and Non-Financial Awards

We know little about whether non-financial incentives can improve the performance of health care workers in countries with cash-strapped health systems. In this study, IPA introduced two types of non-financial incentives to randomly selected health clinics in Sierra Leone—a bottom-up community monitoring program and a top-down non-financial awards competition–-to measure the impacts different types of non-financial incentives can have on health worker performance.

 
Policy Issue:

In many developing countries, the health sector suffers from a severe human resources problem due to staff shortages and absenteeism. The availability of health care workers is a crucial element of quality care and the existing high levels of absenteeism represent a major leakage in health sector resources. Policy-makers have focused their attention on performance-based financing to incentivize attendance and performance monetarily; however, the evidence on the impact of financial incentives in improving performance in the health sector is mixed. While some programs report positive results, others show little to no effect on attendance and outcomes.

In contrast, recent results have highlighted the power of non-financial incentives to reduce absenteeism and improve performance.  Evidence suggests that peer recognition and status-based incentives can be more motivational, less expensive and less likely to erode intrinsic motivation.[1]  In addition, another study implementing a community monitoring initiative in Uganda, in which community members and health workers jointly addressed obstacles to adequate healthcare provision. The study found that under-five mortality was 33 percent lower in treatment compared to comparison communities a year later, while utilization for general outpatient services was 20 percent higher.[2]

Yet, the finding that non-financial incentives such as community monitoring improve clinic performance leaves a crucial question unanswered: does community monitoring improve clinic performance because it is a bottom-up intervention which makes clinic personnel socially accountable to their immediate neighbors? Or does it work simply because clinic performance is being monitored and evaluated? The answer to this question is important as top-down monitoring may be potentially cheaper and more efficient than bottom-up monitoring; however, data on this crucial question is lacking.

Context of the Evaluation:

Sierra Leone’s health indicators are among the lowest in the world, and the country’s health system is plagued by such chronic worker absenteeism, resulting in part from a lack of accountability between service providers and patients, and the weak incentives healthworkers face. Alongside a national decentralization program introduced in 2004, the Government of Sierra Leone launched an ambitious policy in 2010 to institute free healthcare for pregnant women, new mothers and children under-five. The policy abolished user fees, while at the same time raising workers’ salaries. However, these reforms occurred without introducing institutional features to improve oversight of health workers or changing underlying incentive systems, leaving the health sector at risk of further weakening in response to rising demand for free health services.

Details of the Intervention:

This project evaluates two social accountability interventions aimed at improving health service delivery via community monitoring and the introduction of an incentive scheme to reward worker performance on the basis of non-financial awards. The 254 clinics taking part in the study have been assigned to participate in either intervention or act as a comparison, with one third of clinics allocated to each group.

The community monitoring intervention introduces health scorecards that provide information regarding the state of health care in each community, and facilitates interface meetings between community members and health facility staff. During these meetings, information about the state of healthcare is disseminated via a community scorecard and mutual commitments are made to improve services through a joint action plan addressing areas such as staff absenteeism, maternal mortality and vaccination rates. This framework aims to ensure participatory decision-making and hold both healthcare workers and the community mutually accountable, fostering increased access to and utilization of maternal and child health services. Researchers evaluate whether service quality and quantity improve due to the lower costs of collective action introduced through these meetings and the social accountability contract.  

The second intervention, non-financial incentives,facilitates a yardstick competition among groups of maternal and child health clinics, and rewards workers at the most improved facilities. The relative rankings of clinics on key measures of such as worker absenteeism, staff attitude and charging of illegal fees will be advertised publicly, and staff at winning clinics will receive letters of commendation from high-ranking politicians, and an award at a public ceremony.

The project is being conducted in partnership with the Government of Sierra Leone and the interventions have been designed with a self-sustainable model for scale-up through the Ministry of Health and Sanitation in mind. Researchers will assess the cost-effectiveness of each intervention, as well as their cost-effectiveness relative to one another, and findings will directly inform the government’s decision to scale up these interventions in future years.

Results:

Results forthcoming



[1]Ashraf, Nava, Oriana Bandiera, and Kelsey Jack. "No Margin, No Mission? A Field Experiment on Incentives for Pro-Social Tasks." Harvard Business School Working Paper, No. 12-008, August 2011.

Effect of Matching Ratios on Charitable Giving in the United States

 

Policy Issue: 

Throughout the world, charitable organizations are working to meet people’s basic needs and improve their quality of life, and these organizations often depend on the support of outside donors to finance their mission. Private donations comprised 75 percent of all charitable giving in the United States in 2008 and experts predict that the combination of increased wealth and an ageing population will lead to an even higher level of private gifts in the coming years. Such trends have left fundraisers, who typically rely on anecdotal evidence in lack of scientific evidence, divided as to the most efficient means to attract these dollars. While the economics of charity has been well studied on the “supply” side, critical gaps remain in our knowledge about the “demand” for charitable giving.

 
Context of the Evaluation: 

In the United States, private giving to charitable causes has grown significantly in the past several decades. Recent figures show that charitable gifts of money have been 2 percent or more of GDP since 1998 and currently more than 89 percent of Americans donate to charity. 

Among the tactics commonly used in fundraisers to solicit donations are matching gifts. A matching gift is a commitment by a donor to match the contributions of others at a given rate, up to the maximum amount the leadership donor is prepared to give. While the rate of matching is typically the result of an agreement between the fundraiser and the leadership donor, fundraising consultants ubiquitously note that increases in the matching ratio have power to influence contributions. Such conventional wisdom, however, is largely anecdotal as little scientific study has been completed to examine such demand side claims.

 

Details of the Intervention: 

This study seeks to explore the importance of price on charitable giving by measuring the comparative static effects of changes in rates of matching gifts. Partnering with a liberal U.S. nonprofit organization that works on social and policy issues relating to civil liberties, this study takes advantage of a capital campaign in which prior donors received direct mail solicitations seeking contributions. 

Individuals were randomly assigned to either a comparison group or a matching grant treatment group. All individuals received a letter identical in all respects except two: (1) the treatment letters included an announcement that a “concerned fellow member” will match their donation, and (2) the reply card included the details of the match. The specifics of the match offer were then randomized along three dimensions:

1. Price ratio of the match: 1:1, 2:1 or 3:1 (the 2:1 ratio means, for example, that for every dollar the individual donates, the matching donor contributes $2)

2. Maximum size of the matching gift across all donations: $25,000, $50,000, $100,000 or unstated

3. Example donation amount suggested to the donor equal to: (i) the individual’s highest previous contribution, (ii) 1.25 times the highest previous contribution, or (iii) 1.5 times the highest previous contribution

Because fundraising solicitations are sent to all 50 states, it is possible that utilitarian effects of contributing to a politically-motivated charity are different spatially due to the local political environments. To test for these effects, charitable giving data is merged with (i) demographic data, (ii) state returns from the 2004 presidential election and (iii) data on the frequency of the organization’s activities within each state.

 

Results and Policy Lessons: 

Matching Impact: Simply announcing that match money is available considerably increases the revenue per solicitation—by 19 percent. In addition, the match offer increases the probability that an individual chooses to donate by 22 percent. This finding supports anecdotal evidence among fundraising consultants on the efficacy of a matching mechanism.

While the match treatments relative to a comparison group increase the probability of donating, contrary to conventional wisdom, larger match ratios (2:1 and 3:1) relative to smaller match ratios (1:1) have no additional impact. This result directly refutes the rationale for using larger match ratios, and stands in sharp contrast to current fundraising practices.

Heterogeneous Treatment Effects: Study findings indicate that the results of the matching gift are driven by agents in states that voted for George Bush in the 2004 presidential election: the match increases the revenue per solicitation by 55 percent in “red” states whereas there was little effect observed in “blue” states. This result suggests that an individual’s political environment also has the capacity not only to influence the level of giving, but their responsiveness to different treatments.

1 Giving USA Foundation, “U.S. charitable giving estimated to be $307.65 billion in 2008,” http://www.philanthropy.iupui.edu/News/2009/docs/GivingReaches300billion....

 

Selected Media Coverage:
What Makes People Give? - New York Times
Bigger Matching Gifts Don't Produce More Donors - The Chronicle of Philanthropy
 
 
Dean Karlan, John List

Experiments to Improve Participation in a Recycling Program in Northern Peru

Policy Issue:
Economic growth in Latin America has come at the cost of increasingly acute environmental pressures. Expanding trade and consumption has led to increased waste generation and pollution requiring more developed solid waste disposal systems. Markets lack a price mechanism to internalize the environmental cost of this growth.  Policy makers often apply taxes, subsides or other mechanisms to attempt to align private incentives with public environmental preservation. Aside from altering financial incentives, growing evidence from psychology and behavioral economics research shows that behavior can successfully be influenced by leveraging social norms and emotions.  
 
Context:
Over 20,000 tons of solid waste are produced every day in Peru, most of which is dumped in waterways or informal trash heaps, making solid waste management an area of increasing concern for the country. PRISMA, a local NGO, operates a recycling program in Northern Peru whereby it trains and supports workers in forming associations that collect recyclables door to door from participating households. In addition to providing the informal workers with some initial tools and training, PRISMA further assists workers by canvassing the areas of operation to introduce the recyclers to the community and encourage the residents to segregate recyclable and take part in the recycling program. PRISMA was interested in identifying viable strategies to increase program uptake (34% at baseline), and reduce attrition of participating households from the program.
 
Description of Intervention:
Researchers worked with PRISMA to test a series of information messages aiming to improve take-up and participation its recycling program.
To improve take-up of households in communities where PRISMA planned to expand the program, researchers conducted a randomized evaluation to test the impact of different messages in eliciting program participation.  One week before the first PRISMA canvasser’s visit, a paper flyer was delivered with the a generic message about PRISMA’s program and one of nine specific messages eliciting pressures such as, social norm, peer comparison, conformity, authority, environmental or social benefits to increase participation.
 
Households that owned a cell phone and were willing to share their number (about 35% of the sample) received text messages once a week with the specific message in addition to the flyers. 
 
With a sample of 1,785 existing participants, researchers tested strategies to reduce program dropout and increase the amount and quality of the recyclables collected. Plastic bins were randomly distributed to households participating in the study. Some bins had a sticker specifying which items could be recycled.  Household that provided cell phone numbers were randomly assigned to receive either a generic or personalized SMS reminders to recycle or to serve as a comparison group without SMS reminders. SMS messages were sent the day before the weekly visit by the recycler, for a period of six weeks. Data collection for this component of the study lasted for eight weeks and included a careful accounting of the quantity and quality of recyclables received.
 
Results and Policy Lessons:
Results of Treatments on Program Enrollment: The results reveal no statistically significant effects from the different treatment messages. No significant impact on program take-up for the information campaign conducted through flyers alone or through flyers with text messages were found. Messages conveying social norms and applying social pressure were not successful in leveraging behavioral change. Two interpretations for this outcome are proposed by the researchers: a) these messages and norms were not relevant in this context, b) the large presence of informal recyclers operating outside of the program rendered separation of recyclables at the household level a non-issue.
 
Results of Treatments on Compliance: Households who received plastic bins turned in recyclables 3-8 percent more of the times and produced on average more (about 0.2 kg) and more valuable recyclables (about 0.1 pesos).  This finding suggests that convenience of storing recyclables is a barrier to greater program participation. 
The SMS reminders had no significant impact on the level and quality of participation of households in the program, suggesting that forgetfulness is not a serious constraints among households enrolled in the program. There was no clear difference in recycling compliance between households who received plain bins and those who received bins with explanatory stickers.

Community Driven Development in Sierra Leone

 
Policy Issue: 

While the accountability and inclusiveness of institutions are often considered key determinants of economic performance, there is little agreement about exactly how institutions should be designed, how to move from a system of bad institutions to one with good institutions, and whether and how foreign donors can help in this process. One of the most popular strategies employed by donors to promote democratic and accountable institutions at the local level is “community driven development” (CDD). Typical CDD interventions combine flexible grants that communities can spend on local projects with requirements that decisions must be made in an inclusive and transparent manner and training on how to do this. The participation requirements aim to ensure that the projects funded reflect the needs of the community and facilitate learning by doing—i.e. the experiences gained in deciding how to spend project funds leave minority groups  better placed to participate in other community decisions after the project ends. While billions of dollars are spent on CDD programs, few studies provide rigorous evidence on their real-world impacts. Critics of CDD, and of decentralization in general, have raised the concern that decentralized funds will be captured or exploited by local elites.

 
Context of the Evaluation: 

Scholars argue that frustrations with government incompetence and corruption, as well as the exclusion of women and young men from decision-making in the traditional chieftaincy system that coordinates the provision of many local public goods, fueled violence during Sierra Leone’s recent civil war.  To both prevent a return to violence and to stimulate economic development, the Government of Sierra Leone implemented a number of reforms that give communities, and vulnerable groups within them, a greater voice in local decision-making. Alongside a national decentralization program that re-established district-level councils, the government piloted a community-driven development project that went one step  further by providing small grants to be administered by village development committees. This extension down to the village level aimed  to establish more inclusive and accountable local decision-making infrastructure, rebuild trust, promote collective action, and provide minority groups (particularly women and youth) with experience in managing projects and making decisions  within their community. Researchers and the Decentralization Secretariat collaborated to evaluate whether this pilot, called the “GoBifo” Project (or “Move Forward” in Krio),  acheived these goals. 

 
Details of the Intervention: 

Two hundred thirty-six villages from two ethnically and politically distinct districts were randomly allocated into a treatment group or a comparison group. Villages in the treatment group were regularly visited by a GoBifo facilitator, who helped community members create or revamp Village Development Committees (VDCs), set up bank accounts for the VDCs, establish transparent budgeting practices, and create village development plans that included specifics on how GoBifo grants would be used. The participation and inclusion of marginalized groups was central to this process – for example, each social group (women, youth, and adult men) came up with their own development plan, and these plans were then combined into a single unified vision. Women were often established as treasurer of the VDC and served as co-signatories on all project finances. A series of block grants totaling US$4,667  per community were given to implement local public goods and skills training projects that were identified in the village development plans. 

Household surveys, which covered participation in local decision-making, attitudes to minorities, and engagement in collective action, as well as demographic and socioeconomic information, were collected in late 2005 and again in mid-2009, along with village-level focus group discussions. In addition, three structured community activities (SCAs) were conducted in late 2009, shortly after GoBifo activities had ended, to capture any persistent impacts on collective action, participation of minorities, and elite capture. The SCAs were designed to measure how communities responded to concrete, real-world situations in three areas where GoBifo had sought to change behavior: (i) raising funds in response to a matching grant opportunity; (ii) making a community decision between two comparable alternatives; and (iii) allocating and managing an asset that was provided for free. 

 

Results and Policy Lessons: 

The authors and project team agreed a set of hypotheses they would test at the start of the evaluation (in 2005) and wrote out a plan on exactly how the data would be analysed before looking at the data. This prevented selective “cherry picking” of results from the 318 variables collected.

Project Implementation and Local Infrastructure Investment: The GoBifo project successfully established the village-level organizations and tools to manage development projects in nearly all cases. The distribution of project benefits within communities was equitable, leakage of project resources minimal, and minority participation high. 

GoBifo villages had a larger stock of higher quality local public goods, such as a functioning primary school or community grain-drying floor, than comparison areas. There was also more market activity in treatment communities, including the presence of more traders and items for sale, suggesting short-run economic gains. 

Institutional Change and Collective Action: There is no evidence that the program led to fundamental changes in local institutions or decision-making. Despite the fact that many women in treatment villages participated in GoBifo decisions, they were no more likely to voice an opinion in community meetings after the project ended or to play a leadership  role in other areas. Similarly, the establishment of a democratically elected village development committee that carried out multiple projects did not lead treatment villages to be any more successful at raising funds in response to a later matching grant opportunity.  Lastly, there were no program impacts on elite capture, although levels of capture were low in the research communities (at least as measured by the third SCA). 

Peace Education in Rural Liberia

For new democracies and societies emerging from conflict, encouraging tolerance and dialogue, strengthening non-violent conflict resolution systems, and increasing understanding of human rights are key priorities. Governments and NGOs commonly try to change the political culture, civic values, and practices of conflict resolution at the local level through widespread dialogue, education, and information campaigns.  But do these dialogue and education programs actually work as intended? Do they change norms and behaviors, and if so, how? How are new patterns of conflict resolution formed?  And how do they contribute to national reconciliation? How do new state structures integrate with pre-existing local bodies to jointly support security goals and human rights, especially where traditional structures are in conflict with the later? In short, what programs are most useful in helping post-conflict countries achieve lasting peace?

Find a more in-depth policy report here.

Context of the Evaluation:

More than five years after the end of Liberia’s fourteen year civil war, underlying tensions between tribes, over land, and between youth and elders continue to pose threats to a fragile peace. The UN’s Peacebuilding Commission and the Government of Liberia are working together to promote non-violent dispute resolution and inter-group reconciliation, but how best to do this is unknown. This study was jointly designed by the UN, the government, and the research team to assess whether civic education and conflict resolution programs can contribute to this broader peacebuilding agenda.

Description of the Intervention and Evaluation:

Target communities for the program were identified within Liberia’s three most conflict-prone counties: Grand Geddeh, Lofa, and Nimba. The researchers randomly assigned half to receive the program as the “treatment” group, and half to not receive the program as a “control” group. The program took place in 67 villages and town quarters. The program mobilized and trained community members in order to achieve three main goals: (1) educate people on their rights and to respect the rights of others; (2) encourage community collective action towards shared goals; and (3) foster non-violent dialogue and conflict resolution. The program is notable for its intensity and reach: in each community, roughly 10% of adults participated in an eight-day long interactive workshop held over the course of several weeks. Workshops had between 20 to 30 participants, both men and women, were led by a professional facilitator, and were conducted in local dialects. Multiple workshops were held in most communities to reach the 10% coverage target.

Pre-program baseline data was collected in 2009, and the endline took place between 1 and 22 months after the program. Data came from more than 5,000 individuals with three main surveys. In each community, the team interviewed: 20 randomly-selected “community members”, 4 “community leaders”, and 3 people identified by local chiefs as potential trainees, including a “troublesome” person. The study measures the impact of attending the program on potential trainees, random community members, and community leaders, and the impact on the community of having the program take place in their community. An in-depth qualitative study in 14  of the communities was conducted alongside the randomized evaluation to determine the mechanisms of impact. The study focuses on four major outcome classes: community and political participation; attitudes to rights; civic knowledge, attitudes and beliefs; and the prevalence and resolution of conflict.

Results and Policy Lessons:

Community and political participationCommunity participation was measured through contributions to public goods and community projects, membership in groups (from farming to sports), membership in a peace group, and leadership in groups.  Across all measures, the only treatment effect was on membership in peace groups. On political participation, the only statistically significant treatment effect was seen on an index measuring whether potential trainees feel free to speak their minds to “big people” in the community and whether they feel community members have the right to speak out to elders: Those trained are 4% more likely to feel empowered to speak freely. This effect is concentrated among the “troublesome” individuals, who see a larger increase of about 8%.

Attitudes on human rights: Across multiple measures, nearly all the treatment effects are positive, indicating that respondents in trained communities generally report more progressive beliefs. For community members, however, these impacts are fairly close to zero. The impacts on trainees and leaders are modest in size – often in the range of 1 to 10% —and seldom statistically significant at conventional levels.

Civic attitudes and knowledge: The civic education component provided information on citizenship, civic rights and responsibilities and Liberia’s political structure. At endline, amongst potential trainees in the control group, only about 12% correctly understood the statutory requirements for citizenship. This understanding nearly doubled among treated trainees. Little change was seen in political knowledge, and the program also appears to have little to no impact on perceptions of equity in community governance as well as on perceptions towards the national government.

Prevalence and Resolution of Conflict: The most striking program impacts were on conflict and its resolution. In treatment communities (i.e. those that received the program), the evaluation found sizeable increases in non-violent inter-personal and inter-group disputes; suggestive evidence of a decrease in violent disputes; increasing levels of land conflict since the program, though also suggestions of lower rates of violence, and increased rates of dispute resolution and of satisfaction with those resolutions in trained communities. At the community level, leaders reported a 93% increase in conflicts (typically disputes and disagreements) between youth and elders in treated communities.  Treatment communities were also twice as likely to have a peaceful strike or protest, and three times as likely to have a witch killing (though the latter result is not statistically significant).

Violent strikes, protests, and inter-group violence are 59% less likely in treatment communities, however, though this result is not statistically significant (partly because the events are rare, making it difficult to estimate their prevalence precisely with such a small sample of communities).

In addition to the policy report detailing the impact evaluation results, the team has produced a second policy report analyzing patterns of conflict.

Northern Uganda Social Action Fund – Youth Opportunities Program

Youth unemployment is a persistent problem in the developing world, particularly in post-conflict settings, posing both economic and security issues. In growing, stable economies such as Uganda, what holds back youth from reaching their potential?  One theory suggests that youth unemployment is due primarily to the lack of sufficient capital to support entrepreneurship. If this is true, cash transfers or cheap credit could lead to a burst of self-employment. Evidence from other areas, such as studies on microcredit, suggests that alleviating these constraints with loans has little effect on earnings. In Northern Uganda, which is returning to peace after twenty years of war, the government’s Youth Opportunities Program offered cash transfers to groups of youth to increase employment and reduce conflict. Follow-up surveys two and four years later found a shift from agricultural work towards skilled trades and strong increases in income. Women in particular benefited from the cash transfers, with incomes of those in the program 84% higher than women who were not. There were no differences, however, in social outcomes such as community participation, aggression, and social cohesion.

See the full paper herea policy note for the World Bank here, and Chris Blattman’s blog discussion here.

Policy Question:
In developing countries, high unemployment - particularly among youth - is a pressing concern. Jobs, particularly higher-skilled labor and productive small enterprise, provide incomes and reduce poverty. For governments, transitioning from an economy based on small-scale agriculture to one based on entrepreneurship and production is critical for long-term growth. Employment is also seen as important for building social stability and political engagement in communities uprooted by long-term conflict.
 
One form of intervention offers cash in the hopes that youth will invest it in the training and assets to learn a trade or form a business. In the development community, anxiety persists over whether this is an effective approach: will youth with little or no financial or business training be able to direct the money towards successful long-term entrepreneurship?  Previous research also raises questions about the ability of women in particular to invest aid into increasing lifetime earnings, given occupational constraints and pressure to share windfalls.1
 
Uganda’s largest employment program sought to test if an intervention as simple as giving cash could help accomplish the country’s long-term economic and social goals for its youth.
 
Context of the Evaluation:
Twenty years of insurgency, instability and conflict led to high rates of poverty and unemployment in northern Uganda, but by 2005 a measure of peace and stability had returned to the region. The centerpiece of the post-conflict recovery plan was a decentralized development program, the Northern Uganda Social Action Fund (NUSAF). In 2006, to stimulate employment growth through self-employment, the government launched a new NUSAF component: the Youth Opportunities Program (YOP), which provided cash transfers to groups of young adults with the goal of encouraging trade-based self-employment. 
 
Description of the Intervention:
The YOP intervention had two official aims: to raise youth incomes and employment and to improve community reconciliation and reduce conflict. The program, targeted at youth from ages 16 to 35, required young adults from the same town or village to organize into groups and submit a proposal for a cash transfer to pay for: (i) fees at a local technical or vocational training institute of their choosing, and (ii) tools and materials for practicing a craft.
 
The average applicant group had 22 members. Group cash transfers averaged nearly UGX 12.8 million (US$7,108), and varied by both group size and group request. The average transfer size per member was UGX 673,026 (US$374) – more than 20 times the average monthly income of the youth at the time of the baseline survey.
 
Due to vast oversubscription, the 535 eligible groups were selected at random, using a lottery, to either receive the YOP program or be part of the comparison group. A baseline survey was conducted with 2601 individuals in 2008, and 87 percent were successfully followed and interviewed in the endline surveys two and four years later.
 
Results and Policy Lessons:
Overall, the program seemed to have strong economic effects. Four years later, beneficiaries of the YOP program had 41% higher income and were 65% more likely to practice a skilled trade, such as carpentry, metalworking, tailoring, or hairstyling. Hours worked were 17% higher, nearly entirely accounted for by these new professions – while most still farmed part-time, hours spent in agriculture were not different. They were also 40% more likely to keep records, register their business, and pay taxes.
 
Within the sample, gains were highest for those who had the highest initial credit constraints, those with fewest initial assets and access to loans. The effects were particularly strong for women. Women who received the cash grants four years later had 84% higher incomes than women who did not, while men were earning 31% more than their counterparts in the comparison group. This gender difference may reflect particular capital constraints faced by women.
 
While employment programs including this one are often implemented by governments with the aim of reducing social instability or promoting cohesion, the data show no evidence for impacts in these domains. After four years there were no measurable differences in cohesion, aggression, or community and political participation between participants in the YOP program and those in the comparison group.
 
Overall, the data show that the poor used the money effectively; investing in training and tools needed to start businesses and experienced a significant growth in income, even after four years. Even though impacts in social domains were negligible, the economic outcomes show the potential of alleviating capital constraints for spurring economic growth among the poor. 
 
Read the full paper here.
 
A midterm policy report here and policy note by the World Bank here were based on the initial 2-year follow up data.
 
[1] Fafchamps, M., McKenzie, D., Quinn, S., Woodruff, C., 2011. When is capital enough to get female microenterprises growing? Evidence from a randomized experiment in Ghana. Unpublished working paper.

Impact of Voter Knowledge Initiatives in Sierra Leone

This study looks at the impact of developing and disseminating material on political candidates. In a context with low literacy levels and limited media penetration innovative approaches and different forms of media are shared with voters and evaluated to better understand the impact on voter knowledge and candidate selection.

 
Policy Issue:

Transparent and accountable government institutions are thought to be more effective at delivering important social services such as education and healthcare. However, there is little consensus over how best to enhance these aspects of governance, particularly in places where conflict has recently caused breakdown in democratic institutions. Evidence from Brazil and India suggests that increased information about politician performance can result in lower vote shares for low-performing or corrupt representatives. There is also evidence that town hall meetings, where representatives meet directly with constituents, can increase voter knowledge, turnout and support for participating candidates. While many interventions have tested the efficacy of these strategies at increasing basic voter knowledge and access to candidates, little work has been done where democratic institutions are nascent and where public information is limited. In such settings reliable information on candidates may be limited or non-existent, and thus requires significant effort to collect, compile and then convey such information to voters in a comprehensible manner. Debates may provide a feasible alternative which could work in many settings.

 

Context of the Evaluation:

Sierra Leone’s 2012 elections were hailed by international observers as generally peaceful, free, and fair.  In previous elections voting patterns in Sierra Leone have been overwhelmingly based on pre-existing party affiliations.  However, during the 2008 elections, people in Sierra Leone were more likely to vote against traditional party and ethnic affiliations in places where they had more information about candidates (for example, in local elections).  Many election-related social programs focus on logistics and informing people about the importance of voting, but as Sierra Leoneans become more familiar with the democratic process there is also room to help people learn more about the different candidates among whom they will be choosing. The 2012 election presented an opportunity to test new electoral programs that could increase transparency, voter knowledge of candidates, and voter engagement.

 
Description of the Intervention:

In the run-up to the November 2012 elections in Sierra Leone, implementing partner Search for Common Ground filmed debates between rival candidates for membership in parliament (MP). From a total of 264 polling centers, 112 were randomly assigned to receive community screenings of these debates, 40 received interventions that provided information to individual voters, and another 112 served as a comparison group. 

Firstly, debates were shown at almost 200 community screenings in polling centers across Sierra Leone, where they were seen by an estimated 19,000 people. Surveys of voters before and after they watched these debates measured how their perception of candidates, their knowledge of candidate positions, and their voting intentions were altered. 

In the 40 polling centers assigned to receive individually delivered information, individuals were allocated one of the following groups

  1. Debate: Individuals were shown the exact same debate screened in polling centers on a personal handheld device.

  2. Getting to Know You: Individuals were shown a “getting to know you” video of the same two candidates speaking informally about their hobbies and interests.

  3. Radio Report: Individuals listened to a recording of an independent moderator or journalist summarizing the main policy positions articulated by the two candidates during the debates.

  4. Thin Slice Evaluations: Individuals participated in a “lab” experiment where they were exposed to pairs of isolated images, voice recordings, and names of candidates from other constituencies across the country and asked to rate them along a variety of metrics, such as who they thought would be a better leader.

  5. Comparison Group: Individuals were surveyed, but not shown any media.

Evaluating and comparing these groups will allow researchers to disentangle the effects of different kinds of information, such as policy positions, personal characteristics, or persuasive speeches, on voter behavior.

On election day and the days following researchers administered a short exit survey to both comparison and treatment voters, assessing their knowledge about candidates, previous voting behavior, choices in the local and national election, and how they made their electoral choices. During 2013, researchers are conducting several follow-up activities, including further research on the use of video clips to disseminate electoral information, and interventions targeted at MPs to remind them of their electoral commitments.  

 
Results and Policy Lessons:

Project ongoing, results forthcoming

Decentralizing Education Expenditures: Primary School Community Grants in Niger

Policy Issue

In recent years school enrollment has risen dramatically in developing countries, prompting a renewed interest among policymakers in education management. Provision of educational services is centrally administered in much of the developing world, but evidence suggests that decentralized, locally administered services may be better suited to address low education quality. Local administration and oversight puts power into the hands of those with the most interest in seeing improvements in service delivery, and the best information about current education quality. By empowering local communities, decentralized management has the potential to combat systemic teacher absenteeism and reduce misallocation and corruption.

Context of the Evaluation

Niger has made significant strides in increasing access to education in the past decade, with primary enrollment increasing from 37% in 2000 to 54% in 2005. Yet the country still faces considerable challenges with regards to access, rates of primary school completion and the quality and management of the educational system at all levels. One problem, for example, is that schools are spaced far apart, making it difficult for the central government to effectively monitor them.

To address this problem the Ministry of Education mandated the creation of management committees (Committees de Gestion Scolaire, or COGES) for each school. The committee included the director of the school, as well as locally-elected community members. The COGES are responsible for monitoring teacher attendance and performance and managing both financial and material resources, such as the purchase of textbooks and supplies. The Ministry of Education introduced a pilot project to reinforce the capacity of the COGES through the provision of a yearly cash grant, which can be used to fund a part of its activities for the year, such as the construction of classrooms or the purchase of materials. These grants were provided under the belief that such financial support would spur communities to take a more active role in the management of schools.

Details of the Intervention

To test this belief, researchers, in collaboration with the World Bank and the Nigerian Ministry of Education, randomized the allocation of the grants. They observed schools which did and did not receive grants, to estimate their impact, as well as the impact of two supplementary accountability interventions on school management, student retention and learning achievement. One thousand schools in the regions of Tahoua and Zinder were randomly selected into treatment and comparison groups. The 500 schools in the treatment group each received an annual lump sum based on the number of classrooms in the school, averaging out to around US$2 per student. This grant size was chosen by the Ministry of Education with considerations of sustainability regarding a possible extension of the program. In addition to the grants themselves, two sub groups were randomly selected to receive complementary interventions designed to improve delivery and management of the grants.

One third of the treatment schools received financial monitoring in the form of a letter received at the beginning of the intervention telling the committee there was a 50% chance of a “financial control visit” at the end of the school year. Because of financial and capacity constraints, about 20% of the treatment schools received visits at the end of the year.

Another third of the treatment schools received expenditure restrictions in the form of a list of non-eligible expenditures, such as reimbursing school officials for travel, or spending money on things that can be produced locally.

Researchers used annual administrative survey data, pupil test score data, and a detailed financial questionnaire to determine the effect of the grants. The testing used a modified standardized test implemented in collaboration with the Ministry of Education Department of Education Assessment. The school data contained information on pupil and community participation, school resources and infrastructure and information on issues facing school governance. The functioning of the COGES, including how often they met, what they talked about, and whether they began improvement projects, was also monitored. 

Results and Policy Lessons

Supportive actions, which include parental financial and in-kind contributions, parent supervision of pupil attendance, and parent remedial action for pupil absenteeism, increased by 0.14 standard deviations in treatment communities. Most of this overall effect came from an increase in parental contributions to the school, which was 0.48 standard deviations higher in the treatment group. The overall impact was larger when the school committee was educated and was somewhat smaller when families live farther from school.

Management activities, which include parents’ association and school committee meetings, and whether the school committee is in charge of collecting fees and how they decide how the fees are spent, was 0.09 standard deviations higher in the treatment group. Specifically, the proportion of school committees in charge of collecting fees increased by 27 percent, the proportion of active parental association increased by 18 percent, and the frequency of school committee meetings increased by 5 percent.

Oppositional actions, which include supervising teacher attendance and sanctioning teachers, increased only in communities where the school committees were educated. In these communities, teacher supervision increased by 0.13 standard deviations.

Impact on school quality: The results indicate no short-term improvements in school quality, measured in terms of accountability and teacher effort. The lack of an impact on teacher effort is in line with the fact that very few of the schools undertook any sort of teacher supervision. However, one year after the treatment, the grants marginally improved (0.04 standard deviations) the material quality of schools. This improvement was largely driven by increases in the number of classrooms and construction of walls around the compound.

Impact on demand for education: The grant program led to a 0.17 standard deviation decrease in dropouts and a simultaneous 0.10 standard deviation increase in enrollment of students in Grade 2, indicating an increase in demand for education for young children.

Community Based Rangeland Management in Namibia

We conduct an impact assessment of the Community Based Rangeland and Livestock Management (CBRLM) program in Namibia.  This program is part of a larger set of interventions in the agricultural sector designed to reduce poverty among the population of the northern regions of the country.  Many people in the area rely on cattle production for their economic livelihoods, however overuse of the communal grazing areas and suboptimal grazing practices threaten the long-term viability of the land and contribute to persistent poverty.  

To increase the productivity of livestock and other animals using the land, the Namibia Millenium Challenge Compact funds a pilot program designed to help communities improve their livestock practices, address rangeland degradation, and improve market access.  

The evaluation is designed to test the impact of the various activities within the CBRLM intervention on household income, cattle productivity, and the condition of the rangeland. The intervention targets both inadequate information about appropriate cattle production practices and the social or other behavioral preferences of farmers.  At the moment, there is a collective action or “tragedy of the commons” problem – individual farmers are hesitant to reduce their herd’s impact on the rangeland because they are fearful that others will not follow suit which often results in overuse and degradation of the land.

Information Campaigns and Voters' Behavior in the 2009 Municipal Elections in Mexico

Policy Issue:

It is widely held that access to information is a vital component of democracy building and government accountability. A recent World Bank report[1] champions information as “a tool to empower citizens in developing countries to hold their public agents accountable.” Information flows, the report argues, not only enhance democratic participation, but also make democracy work for ordinary people. However, while evidence suggests that access to information may lead voters to punish corrupt incumbents, it is unclear whether this translates into increased support for challengers and higher political participation. In other words, information about corruption may not improve political accountability, if voters respond to it by withdrawing from the political process.

Context:

Despite optimistic views about fiscal decentralization in Mexico, local governments’ performance has remained poor. In 2008, for example, more than 80 percent of municipal governments’ resources were spent either on the bureaucracy or were unaccounted for. While elections should enable voters to discipline their mayors, a single-term limit is imposed on all elected officials in Mexico. Thus, the immediate fate of mayors is determined not by voters but by their political party. To reconcile the single-term limit with accountability, scholars have typically assumed that voters punish or reward the incumbent party for mayoral performance. However, there is little evidence that government performance impacts the subsequent election – previous work shows a strong entrenchment of incumbents from all political parties.

Further impeding voters’ ability to hold mayors accountable are widespread misconceptions about which public works and services municipal authorities are responsible for providing, as well as a lack of available information about the amount of money municipalities receive and how this money is spent. In an attempt to ensure greater municipal accountability, a 1999 constitutional reform established the Federal Auditor's Office (ASF). On a yearly basis, the ASF selects a sample of municipalities in each state to audit. The results of the audits are published in lengthy reports, which are made available online. Though public, these reports are rarely used by media or political parties in local campaigns because the release date of the reports is not aligned with the timing of elections.

Description of Intervention:

Researchers sought to assess the effects of information dissemination on participation in the 2009 municipal and congressional elections in Mexico. Approximately one week before Election Day, flyers with different kinds of information on municipal spending were delivered to all households within the boundaries of treated voting precincts. The first group of precincts received information about municipalities' overall spending; the second group received information about distribution of resources to the poor; and the third group received information about irregular, unauthorized, or unaccounted for spending. All flyers included a subtle advocacy message suggesting that voters raise questions with their mayors about the use of public funds and prompted people to think about the level of governments that was in charge of the provision of public infrastructure services. In total, 150 electoral precincts were randomly assigned to each of the three interventions, for a total of 450 treated precincts and 1910 precincts in the comparison group.

Researchers gathered demographic characteristics from census data, and then collected electoral results for each precinct from the electoral institutes for each state. This information was complemented by a follow-up survey collected ten days after the election.

Results:

Information about overall spending: Disseminating information about overall spending levels had no statistical impact on voter turnout when the mayor spent less than 75 percent of available funds. However, it decreased the incumbent vote share by 0.6 percentage points. When mayors spent more than 75 percent of the funds, releasing this information let to an increase in turnout of 1.9 percentage points, but had no significant impact on vote share.

Information about spending on poor: Turnout among voters who received information about levels of spending allocated to poor areas increased by 2.4 percentage points, but only when mayors spent less than 75 percent of available funds on poor areas. Surprisingly, releasing this information had a similar impact on both incumbent and challengers' vote shares - an increase of 1.5 and 1 percentage points, respectively. When mayors allocated more than 75 percent of resources to poor areas, releasing the information had no impact on turnout or vote share.

Information about corruption: Information about high levels of corruption appears to have had a significant negative impact on voter turnover. Turnout among individuals that received information about corruption decreased by 1.10 percentage points, which represents a 2 percent decrease in turnout. While the effect of information about corruption on the incumbent vote share was insignificant, the effect on the challengers' vote share was negative. These results suggest that while flows of information are necessary, they may be insufficient to improve political accountability, since voters may respond to information by withdrawing from the political process.


[1] Khemani, Stuti. 2007.  Can Information Campaigns Overcome Political Obstacles to Serving the Poor? The Politics of Service Delivery in Democracies. http://siteresources.worldbank.org/DEC/Resources/Khemani_CanInformationCampaignsOvercome.pdf.

 

Effect of Media on Voting Behavior and Political Opinions in the United States

There is substantial evidence that media sources have identifiable political slants, but there has been relatively little rigorous study into the impact of media on political views and behaviors.  IPA designed a natural field experiment to measure the effect of exposure to newspapers on political behavior and opinion.

Washington DC is served by two major newspapers, the Washington Times and the Washington Post. We found that those who received the Washington Post were eight percentage points more likely to vote for the Democratic candidate for governor than those assigned to the control group. We find similar but weaker evidence of shifts in public opinion on specific issues and attitudes.

Policy Issue: 

Citizens learn about politics and government primarily from television and newspapers. These media outlets can influence voters not only through the slant of a particular report, but also merely by choosing which to stories to cover. Recent studies suggest that media exposure can have a sizable impact in shaping the public’s political knowledge, attitudes, and behavior. However, these studies may have overestimated the impacts of media influence due to individuals’ tendency to seek out information that agrees with their pre-existing views. 

 
Context of the Evaluation: 

Prince William County in northeastern Virginia lies just 25 miles from Washington D.C. Here, the population is far enough away from the nation’s capitol so as not to be dominated by citizens involved professionally with politics, but close enough to be within the circulation of Washington’s conservative and liberal newspapers. The Washington D.C. metro area is served by two major newspapers, the conservative Washington Times and the more liberal Washington Post. The presence of a liberal and conservative paper serving the same region creates an opportunity to study the effect of media slant in a natural setting within a single population, which is subject to the same outside factors, such as political events and outcomes, and has a range of political leanings.

 
Details of the Intervention: 

This study takes advantage of this natural setting to measure the effect of political news content on people’s political behavior and opinions. Approximately one month prior to the Virginia gubernatorial election in November 2005, researchers administered a short survey to a random selection of households in Prince William County. 

From the 3,347 households of registered voters who reported that they received neither the Post nor the Times, researchers randomly assigned households to receive a free subscription to one of the two papers for ten weeks, or to the comparison group that was not sent either paper. A week after the election, a follow-up survey was administered asking individuals whether they voted in the November 2005 election, which candidate they selected or preferred, their attitudes toward news events of the previous weeks, and their knowledge about recent news events. Voter turnout data was also collected for the November 2005 and 2006 elections from state administrative records. 

 

Results and Policy Lessons: 

Impact on Political Knowledge: Receiving either paper produced no effect on knowledge of political events or stated opinions about those events, and there were no differences between the treatment and comparison groups in voter turnout for the 2005 gubernatorial election.  In November 2006, however, there was a 2.8 percentage point increase in voter turnout. It is surprising to see a result in 2006 but not in 2005. This could be a result of the post-election exposure to the remainder of the ten-week newspaper subscriptions, or the fact that 17 percent of the treatment group renewed their subscription after the free period ended.

Impact on Political Preference: Interestingly, receiving either newspaper led to an increase of support for the Democratic candidate. Despite the political slant of the newspapers, the effects were similar for the Post and the Times, resulting in an overall 7.2 percentage point increase in likelihood of voting for the Democratic candidate. This may be due to the fact that the Republican President’s approval ratings were falling over that period of time, or perhaps the Democratic candidate was conservative leaning. In either case, these results suggest that the informational effect of more exposure to news was stronger than the effect of its slant. 

Direct Democracy and Local Public Goods in Indonesia

 

Policy Issue: 

Basic public goods such as dependable roads and clean water infrastructure are routinely underprovided in much of the developing world. While the provision of these services is often centrally administered, many now advocate for decentralization and community involvement as a more effective approach. Local communities can have better information on what goods and services are needed, and may thus be better positioned to recognize and quickly respond to inefficiency or corruption in implementation. Recent years have witnessed a trend toward decentralization in developing countries; this increase in local participation in government decision-making has been facilitated by a wide variety of political reforms, but the implications of these political mechanisms are yet to be well understood.  

 
Context of the Evaluation: 

Over 13 percent of the population of Indonesia lives below the poverty line,  and improving the infrastructure in marginalized areas is a priority for the Indonesian government.1 An Indonesian government program supported by a loan from the World Bank, the Kecamatan Development Program (KDP), funds projects in approximately 15,000 villages each year. Each village receives an average of Rp. 80 million (US$8,800) for infrastructure projects. Participating subdistricts, typically containing 10-20 villages, receive an annual block grant for three years. Each village makes two proposals – one on behalf of the whole village and one proposed by woman’s groups – for small-scale infrastructure projects.

Typically, when it comes time for a village to decide upon its two KDP proposals, representatives from various hamlets come together to discuss the merits of, and to decide on, the village’s two project proposals. A typical meeting would have between 9 and 15 people representing the various hamlets, as well as formal and informal village leaders, with on average about 48 people attending in total out of an average village population of 2,200. While the program has effectively improved local infrastructure in many of these villages, it is unclear whether current procedure makes projects easily dominated by elites and under-provides for those community members who need improved services the most.   

 

Details of the Intervention: 

To investigate these issues, researchers conducted a randomized evaluation in 49 villages, all of which were preparing to apply for infrastructure projects. Each village was randomized into one of two different political processes through which they determined which project to propose: 32 villages would follow the traditional representative meeting-based process described above, and the remaining 17 would choose their KDP project proposals via a direct election-based plebiscite. At these plebiscites, villagers could directly vote on a list of potential projects. For the general project, all adults were eligible to vote, and for the women-specific proposal, only women could vote. Data was collected on the project preferences of all villagers, including the elite, as well as the location and type of projects selected.

 
Results and Policy Lessons: 

Impact on Project Type: The direct election process had little effect on the types of projects selected amongst the general project proposals, but had substantial positive effects on measures of citizen satisfaction with the political process. However, the elections did not change the probability that the general project would be located in a poor area. Direct elections on women’s projects, on the other hand, resulted in both increased satisfaction with the political process and increased probability that projects were located in poorer areas of the villages. 

Impact on Satisfaction Measures: Direct elections resulted in substantial changes in the community’s satisfaction with the political process. Overall, the plebiscites resulted in an increase of 21 percentage points of people who said that the project chosen was either very much or somewhat in accordance with their wishes, an increase of 18 percentage points of people who said they would benefit either very much or somewhat from the project, an increase of 10 percentage points of people who said they would use the project personally, and an increase in overall satisfaction with KDP by 13 percentage points. The elections also raised the probability that individuals stated they would contribute something (such as labor or money) to the project by 17 percentage points. Additionally, villagers in treatment locations were 19 percentage points more likely to correctly identify the type and location of the general village project, and 25 percentage points more likely to know these things about the woman’s project.  

[1] CIA World Factbook, “Indonesia,” https://www.cia.gov/library/publications/the-world-factbook/geos/id.html (accessed August 31, 2009).

Combating Corruption in Community Development in Indonesia

 
Policy Issue: 

Corruption plagues many developing countries where the world’s poorest live, and combating it continues to be an arduous task. Corruption acts like a tax, adding to the cost of providing public services and conducting business; it also creates potentially severe efficiency consequences as well. Many suggest the right combination of monitoring and punishments can control corruption, but often the very individuals tasked with monitoring and enforcing punishments may themselves be corruptible. Another approach to reducing corruption is community-level monitoring. Local community members have the most to gain from a successful anti-corruption program, and are thus believed to have better incentives to monitor than bureaucrats. However, there is little empirical evidence on the success of such strategies.  

 
Context of the Evaluation: 

An Indonesian government program supported by a loan from the World Bank, the Kecamatan Development Project (KDP), funds projects in approximately 15,000 villages each year. Each village receives an average of Rp. 80 million, (US$8,800), which they often use to surface existing dirt roads. KDP-funded projects are large relative to ordinary local government activities; in 2001, the average annual village budget was only Rp. 71 million (US$7,800), so receiving a KDP project more than doubles average local government expenditures. This large amount of money creates incentives for price inflation, collusion with suppliers, kickback for village leaders, and manipulation of wage payments. 

Two checks on corruption are built into KDP. First, funds are paid to village implementation teams in three installments. To receive the second and third payments, the teams must make accountability reports at an open meeting where they account for how they spent the money; only after that meeting has approved the accountability report is the next installment released. Second, each project has approximately a 4 percent chance of being audited by the government.

 

Details of the Intervention: 

To examine the role of community monitoring and government audits on corruption, researchers conducted a randomized evaluation in 608 Indonesian villages in East Java and Central Java, Indonesia’s most populous provinces. Each village in the study was about to start building a village road with KDP funding. Some villages were randomly selected to be told, after funds had been awarded but before construction began, that their project would subsequently be audited by the central government, increasing the likelihood of an audit from 4 percent to 100 percent. These audits carry the possibility of criminal action if corruption is detected, and the results of the audits are read publicly to an open village meeting, potentially resulting in social sanctions. 

To investigate the impact of increasing community participation in the monitoring process, two interventions were established to enhance participation at accountability meetings. Some villages were selected to have invitations to these meetings distributed throughout the community, encouraging direct participation in the monitoring process and reducing elite dominance of the process. In the second experiment, an anonymous comment form was distributed along with the invitations, providing villagers an opportunity to relay information about the project to be shared at the meetings, without fear of retaliation. (See chart below)

Experimental Treatments by Group:

 

Group

 

Audit Probability

 

Accountability Meetings

 

Invitations

 

Comment Cards

 

Villages

 

  Audit

 

100%

 

x

 

 

 

 

 

93

 

  Participation I

 

4%

 

x

 

x

 

 

 

105

 

  Participation II

 

4%

 

x

 

x

 

x

 

106

 

  Audit & Participation I

 

100%

 

x

 

x

 

 

 

94

 

  Audit & Participation
  II

 

100%

 

x

 

x

 

x

 

96

 

  Comparison

 

4%

 

x

 

 

 

 

 

114

Corruption was measured by comparing the researcher’s estimate of what the project actually costs, determined by the quantity of materials used and estimate of material prices and wages paid on the project, to what the village reported it spent on the project on an item by item basis.

 

Results and Policy Lessons: 

The evidence suggests that increasing the probability of external audits substantially reduced missing funds in the project. Increasing the probability that a village was audited by the central government from 4 percent to 100 percent reduced missing expenditures by about eight percentage points, from 27.7 percentage points to 19.2 percentage points. One reason that the decrease was not larger is that a 100 percent audit probability does not imply that village officials face a 100 percent probability of detecting corruption and imposing a punishment. In fact, although auditors found violations of some type or another in 90 percent of the villages they visited, the vast majority of these violations were procedural in nature, and there were very few, if any, cases in which the auditors had enough concrete evidence to actually prosecute offenses. 

The invitations increased the number of people participating in the accountability meetings by an average of 14.8 people, or about 40 percent; slightly more than by including a comment form, since many villagers used the form as a substitute for attendance. Villages that received the invitations intervention were more likely to openly discuss corruption problems at the accountability meetings, and villages receiving both invitations and comment forms were more likely to take serious action at the meeting to resolve corruption-related problems. However, the magnitude of these changes in behavior at the meetings was small, and these treatments did not measurably reduce overall missing expenditures, yet they did have an effect on certain types of expenditures in some cases. For instance,  the invitations substantially reduced missing labor expenditures, and the comment forms did reduce missing expenditures in some cases, but only when they were distributed via schools, bypassing local officials. This study provides evidence that community participation, widely viewed as a panacea for development projects, impacts levels of corruption only under a limited set of circumstances, and pains must be taken to prevent elite capture for it to be an effective means of reducing corruption.

Extra Teachers in Kenya: Peer Effects, Pupil Teacher Ratios and Teacher Incentives

 
Policy Issue: 

The introduction of free primary education has raised primary school enrollment in many developing countries. However, the resulting overcrowding of schools, as well as the influx of new students with little or no preparation, poses new challenges to policymakers. For example, increased enrollment has often not been matched by increased numbers of government-salaried teachers. One method of lowering the pupil-teacher ratio, versions of which have been used by many governments, is to hire low-paid local contract teachers to supplement government-salaried teachers. However, there are concerns that these teachers may be less experienced and therefore less effective. Empowering the local community to monitor teachers' performance may help to overcome this problem, and may also increase civil-service teacher effort and subsequently student learning.

 
Context of the Evaluation: 

Historically, Kenyan schools have had two types of teachers – those hired through the Ministry of Education and those hired locally and informally by Parent Teacher Associations (PTAs). For civil-service teachers, promotion, transfers, and disciplinary measures are decided through the Ministry of Education, rather than by more locally accountable bodies. These teachers are represented by a strong union, have civil-service protection, and receive wages and benefits considerably above market levels. PTA teachers, while typically paid much less than their civil-service counterparts, have much stronger incentives, partly because they do not have union protection, but also because a good track record as a contract teacher can help them obtain a civil-service job. 

Incentives for civil service teachers, while weak, are largely based on their students’ scores on the primary school exit exam. Teachers may, therefore, have an incentive to focus on students they believe will perform better on the exam, while ignoring the weaker students who may be too far behind to catch up. 

In 2003, Kenya eliminated school fees for primary school. This led to an almost 30 percent increase in enrollment and greater heterogeneity in student preparation. Many of the new pupils were first generation students and had not attended preschools. Since parents were no longer required to pay fees, local school committees were generally unable to raise the funds necessary to hire PTA teachers to match the influx of students and bring the new students up to speed.

 

Details of the Intervention: 

In collaboration with International Child Support (ICS), researchers evaluated three nested interventions that addressed the large class sizes and heterogeneity in student preparation in the Kenyan school system: (1) the addition of locally hired contract teachers to reduce the pupil-teacher ratio; (2) the empowerment of parents within PTA committees through School-Based Management (SBM) training; and (3) the sorting of students by initial level of preparedness to reduce heterogeneity within the classroom.  

Out of 210 primary schools, 140 were randomly assigned to receive the Extra Teacher Program (ETP). The ETP program provided funding to hire a local contract teacher to address classroom overcrowding. The remaining 70 schools served as a comparison group. In each school, ICS held a meeting with parents and teachers to explain the program rules regarding the hiring of an additional teacher. School committees were responsible for hiring the contract teachers and were free to replace or keep the original contract teacher based on performance. The contract teachers were paid approximately one-quarter of the salary of regular civil service teachers, but had the same educational qualifications. 

In 2005, the first year of the program, the ETP contract teacher was assigned to grade 1. Two sections were created for grade 1, one taught by regular civil service teachers (with multiple teachers rotating in an out of the class, teaching different subjects) and one taught exclusively by the contract teacher. As a result, average class size in grade 1 was only 44 in ETP schools, compared to 82 in comparison schools. In the second year of the program, the ETP teacher moved to grade 2, such that the cohort of first graders that benefitted from the ETP program continued to benefit once in grade 2.

While in half of the ETP schools (the “non-tracking ETP schools”), students were divided into sections at random, in the other half (the “tracking ETP schools”), students were divided into sections based on students’ level of preparedness (as measured by exam scores during the first term). For all schools, which section was taught by the ETP contract teacher was then decided by random draw.

Finally, half of the schools assigned to ETP, including both tracking and non-tracking schools, were randomly selected to receive School-Based Management (SBM) training. The training was designed to empower parents (within the PTA committee) to ensure a fair and objective recruiting process for the ETP teacher, as well as to monitor teachers’ performance. Two parents of grade 1 students were asked to perform attendance checks on teachers on a regular basis. 

Standardized test covering math and literacy questions were administered to each school just before the program ended and again one year later. Four unannounced visits were also made to each school to measure teacher effort and to observe the classroom environment. 

 

Results and Policy Lessons: 

Learning Impacts: Providing school committees with funds to hire an extra teacher on a short-term contract had a positive effect on learning. Test scores of students in ETP schools were on average 0.22 standard deviations higher than those of students in comparison schools. However, not all students benefitted equally. The program impact varied a lot with the details of how it was implemented. In particular: 

  • The impact was much larger for students assigned to the ETP contract teacher (0.31 standard deviation gain) than for students assigned to the civil-service teachers (0.13 standard deviation). 
  • The impact was larger when the ETP program was accompanied with SBM training (0.25 standard deviation) than when it was implemented alone (0.19 standard deviation).
  • The impact was larger in tracking schools (0.28 standard deviation) than in non-tracking schools (0.16 standard deviation).
  • There was relatively rapid fade-out of these learning gains in the year after the program ended, except for the gains in tracking schools and the gains in SBM schools.

Mechanisms:

Teacher effort: A crude measure of teacher effort is how often the teacher is found in class and teaching during a random spot check. This rate is 58 percent among civil-service teachers in non-ETP schools, suggesting relatively low levels of efforts. Despite their low pay, contract teachers were found 27.8 percentage points more likely to be teaching during a random visit. As a result, their students learned more. In contrast, civil-service teachers in ETP schools responded to the introduction of the ETP program by decreasing their own effort: the probability that they would be found in class teaching during a random visit decreased from 58 to 45 percent. As a result, their students did not significantly gain in learning over students in comparison schools, despite being in much smaller classes. One of the reasons why the SBM program led to increased learning gains is that it mitigated the negative effort response by civil-service teachers.

Selection of Contract Teachers: Because a contract teacher position is a stepping stone for permanent civil-service positions, it is highly valuable (despite the low pay). As a result, existing civil-service teachers may want to hire their own relatives as contract teachers. In ETP schools without SBM training, 31 percent of contract teachers hired were relatives of existing teachers in the school. Those relatives tended to perform less well than other contract teachers. The second reason why the SBM program led to increased learning gains is that SBM increased the transparency of the contract-teacher recruiting process. Specifically, it reduced the number of contract teachers who were hired because they were related to an existing teacher by about half. 

Homogeneous classrooms: Compared to students in non-tracking ETP schools, students in tracking ETP schools were in more homogeneous classes. This homogeneity benefitted them, irrespective of whether they had started at the top, middle, or bottom of the distribution. Homogeneity is beneficial in part because it allows teachers to better tailor their materials to the level of their students, but also because it increases teachers’ attendance in the classroom. For those who started at the middle of the distribution, being assigned to the bottom track (instead of the top track) had no detrimental effect on their learning, because the decrease in peer quality was compensated by a positive effect of being at the top of the class. 

Overall, the results suggest that while hiring supplementary contract teachers can be part of the solution to Kenya’s teacher shortage, to get the most out of these teachers, implementation details matter. The biggest gains come when local school committees are empowered to oversee the recruitment process and to effectively monitor teachers, and when classes are structured so as to target instruction to students’ initial achievement level. 

Child Protection Knowledge and Information Network (CPKIN)

Policy Issue:

Children in post-conflict environments are a population of particular concern, exposed to violence, displacement, and death of family members. The impact of poverty means that child labor is common among children. Though no reliable data exists, indications are strong that sexual abuse and exploitation is a fact of life for many girls and teenage pregnancy has been identified as a child protection concern. In addition to the physical and mental health challenges posed by war experiences, these children remain susceptible to continued exploitation and abuse, often by those familiar to the child, during peace as refugees return home and community members cope with difficult pasts. 

Context of the Evaluation:

The situation of children in Sierra Leone continues to be precarious as 27% of the 2.7 million children are identified as vulnerable, lacking the protection of a primary care giver. Formal structures for child protection exist within police stations and local Child Welfare Committees (CWCs), but capacity and community trust in such institutions’ efficacy are limited in many rural areas[1]. To reduce violence against children, existing social structures,most paramount, section, and village chiefs, are frequently leveraged to respond to reports of abuse and to mobilize local protective factors.

To respond to the resource constraints faced by formal child protection systems in Sierra Leone, the Ministry of Social Welfare, Gender, and Children’s’ Affairs (MSWGCA) is experimenting with a policy framework that explicitly links local chiefs and community members to the formal child protection sector.  This policy was articulated in a Memorandum of Understanding (MOU) signed in 2010 by the MSWGCA, CWCs, and the Council of Paramount Chiefs. The MOU and focal person system are currently being implemented in Moyamba and Pujehun districts, with potential to expand the program nationwide.  The MOU creates a framework through which chiefs, MSWGCA, CWCs, and the Family Support Unit (FSU) of the Sierra Leone Police can collaborate to protect children. Village chiefs appoint a focal person for child welfare in a public meeting of village residents. Focal persons report cases of child abuse to chiefs and chiefs, with the assistance of focal persons, follow up with the formal child protection actors to ensure that cases of child abuse are addressed appropriately.

To help focal persons connect informal systems for child protection to the formal roles of the FSU and MSWGCA, UNICEF and other local child protection NGOs have developed a training program that introduces focal persons to general approaches to child protection, as well as to their roles and responsibilities under the MOU.  Despite the grounding of these trainings in both international best practices and local norms and understandings, key stakeholders have expressed concerns about whether one-time trainings will provide focal points with sufficient capacity to effectively serve as a link between communities, chiefs, and formal child protection systems.  At the same time, repeated trainings or sustained in-depth monitoring by international or national child protection agencies is not sustainable in the context of rural villages in Sierra Leone.

Details of the Intervention:

Mobile phones and SMS messaging provide one potential avenue for bridging the gap between the need for sustained support of focal points and the high costs of transportation in rural Sierra Leone.  To leverage the growing use of this technology in Sierra Leone, a consortium of non-profit, academic, government, and private-sector stakeholders have collaborated in developing the Child Protection Knowledge and Information Network (CPKIN).  CPKIN (pronounced as “See Pikin” or “See the Child” in Krio, a language spoken throughout the country) is a system that will be used to send focal persons automated text messages asking questions about the welfare of children in their village and prompting them to engage in discussions in their community to help answer these questions.  Focal persons will then be encouraged to send their answers back to the central CPKIN system using a free text message.  These text message questions and responses will be sent and received through a software program that makes it possible to manage, organize, and analyze high volumes of outgoing and incoming messages with a large list of recipients

There are several hypotheses underpinning the design of the CPKIN program. The first hypothesis is that the process of receiving questions, discussing the questions with community members, and sending answers will encourage focal persons to critically examine the situation of children in their communities and then to act with the resources available in their villages. A second hypothesis is that having a focal person who is active with respect to reporting abuses, taking proactive steps to improve the welfare of children, and engaging community members in discussions regarding children has the potential to cause dramatic shifts in community level knowledge, norms, and practices regarding child protection, which in turn may lead to greater connections between informal and formal child protection systems.

Given the novelty of the CPKIN system as a child protection intervention, it is necessary to assess the extent to which sending regular, open-ended text messages to village focal persons using this system can increase the capacity of focal points and their communities to proactively and reactively respond to local child protection issues. In particular, the effectiveness of CPKIN and the validity of the underlying hypotheses will be evaluated using a randomized evaluation in 140 villages in Moyamba and Pujehun districts. The randomization will occur at the village level, with the 70 villages randomly assigned to the CPKIN treatment, and the remaining 70 villages comprising the comparison group, in which there is no intervention. In treatment villages, the village focal person will receive a one-on-one training on the overall aims of CPKIN, how to receive and send CPKIN messages, and how to use CPKIN as a starting point for engaging their community members on issues concerning child protection.  Shortly after this training, focal persons will begin receiving regular CPKIN prompts.

Baseline and endline surveys regarding child protection and child welfare will be conducted before and after the six to eight month intervention in all 140 villages in order to assess the relative effectiveness of the CPKIN support system, with survey questions designed to elicit two kinds of information:

1.)    The capacity of key stakeholders (children, parents, focal persons, chiefs, FSU, and CWCs—where they exist) to identify and respond to child protection issues.

2.)    The protection and support for children within their communities, and their overall level of welfare (including the prevalence of both abuses and protective factors).

Results and Policy Lessons:

Results forthcoming.



[1] Human Rights Watch. (2011). World Report 2011

 

Primary Education Management in Madagascar

Policy Issue: 

Successful efforts to expand access to education in the developing world have not always translated into actual improvements in skills and learning for students. It remains an open debate as to whether top-down approaches are more effective in improving educational quality than approaches which promote beneficiary participation, such as parental monitoring. Top-down approaches can provide administrators with the tools necessary to better monitor their schools, but this assumes that they have the incentives to do so. Bureaucrats may have more incentives to improve the quantity, rather than quality, of education services since the benefits of improving quality are diffuse and harder to verify. The combined effects of these perverse incentives can result in a large number of children who are in the classroom, but are not learning. Promoting local accountability may be a useful means of improving schooling outcomes, if it can be determined which factors makes beneficiary participation effective at improving education delivery.

Context of the Evaluation: 

Madagascar divides its 2.7 million children into 15,000 public primary schools. Despite the significant increase in primary school enrollment following Madagascar's 2002 reforms and an influx of international financial support, resource allocation across schools remains inefficient, and better resource endowments rarely translate into better student performance. Only 63% of grade 5 children pass the primary-cycle exam, an assessment of the minimum level language and math knowledge presumed at this grade. District administrators face a performance review only every 3 years, and the subdistrict heads rarely face any credible threat of penalties or firing.

Details of the Intervention: 

Researchers, in collaboration with The Ministry of Education in Madagascar, ran a randomized experiment in 3,774 primary schools in 30 public school districts. These districts represented all geographic areas in the country, but were focused on schools with the higher rates of grade repetition.

All district administrators in treatment districts received operational tools and training that included forms for supervision visits to schools, and procurement sheets for school supplies and grants (district-level intervention). In some of these schools, the subdistrict head was also trained and provided with tools to supervise school visits, as well as information on the performance and resource level at each school (subdistrict-level intervention).

Lastly, several districts also introduced a school level intervention which involved parental monitoring through school meetings. Field workers distributed a ‘report card’ to schools, which included the previous year’s dropout rate, exam pass rate, and repetition rate. Two community meetings were then held, and the first meeting resulted in an action plan based on the report card. One example of the goals specified in the action plans was to increase the school exam pass rate by 5 percentage points by the end of the academic year. Common tasks specified for teachers included lesson planning and student evaluation every few weeks. The parent’s association was expected to monitor the student evaluation reports which the teachers were supposed to communicate to them. These tools allowed parents to coordinate on taking actions to monitor service quality and exercise social pressure on the teachers.

Results and Policy Lessons: 

Impact from Top-Down Approach: The interventions targeted at the district and subdistrict level had minimal effects on the administrator’s behaviors, and the schools and students under their responsibility. Though each tool – forms for supervision visits to schools and procurement sheets for school supplies and grants – was used by 90% of subdistrict heads and more than 50% of district heads, subdistrict heads visited their schools only slightly more often than those in the comparison group, an insignificant improvement. Teachers in both groups did not plan for lessons more, and no improvement in test scores was seen in the two years following the program.

Impact from Bottom-Up Approach: The interventions at the school level led to significantly improved teacher behavior. Teachers were on average 0.26 standard deviations more likely to create daily and weekly lesson plans and to have discussed them with their director. Test scores were 0.1 standard deviations higher than those in the comparison group two years after the implementation of the program. Additionally, student attendance increased by 4.3 percentage points compared to the comparison group average of 87%, though teacher attendance and communication with parents did not improve.

 

Decentralization: A Cautionary Tale - Public Finance in Kenya

Kenya’s education system blends substantial centralization with elements of local control and school choice.  This project looks at the system of incentives created by elements of decentralization.

Syndicate content
Copyright 2014 Innovations for Poverty Action. All rights reserved.