The Hunger Project's "Epicenter" Scale up Strategy in Ghana

One of the greatest tragedies of extreme poverty is its intergenerational transmission. Poor, malnourished children do not develop normally, physically or cognitively; poor adolescents are unable to take proper advantage of educational opportunities and are more likely to engage in risky behaviors.  After generations of poverty, many families and communities cannot envision cooperation to break this cycle. The Hunger Project (THP) works towards tackling poverty in Africa by partnering with local people to establish community centers ("epicenters") offering a comprehensive range of services, from health and education, to agriculture, microfinance, water and sanitation, as well as fostering community spirit. This randomized evaluation of THP's multi-faceted program aims to assess the impact on the various outcomes it strives to improve.

Researchers from Yale University and IPA have partnered with THP to evaluate the long term-impact of this strategy on health, nutrition, income, gender roles, social cohesion and education. The Hunger Project plans to cover the entire Eastern Region of Ghana, however it is neither feasible nor desirable to build all 112 centers at once. A lottery is conducted within each district to determine which of the 112 communities are offered a center in the first years (treatment group).  Communities that do not win the lottery for early invitation, the comparison group, may receive an invitation a few years later.  A pre-intervention baseline survey of approximately 4000 households with over 20,000 individuals was completed in 2008 and a follow-up survey of the same households will be launched in early 2013.  The longitudinal nature of the survey allows us to examine if the effects of the centers are sustained over time and whether or not the strategy is financially sustainable. Generally, these centers aim to be economically sustainable within 5 years.

Information and Community Mobilization in Rural India

Policy Issue: 
While primary school enrollment rates have risen sharply in much of the world, the quality of education remains low in many countries. Many children who attend school regularly are still unable to read or do basic arithmetic. For instance, a 2008 survey found that in rural India only 56 percent of children in grade 5 could read at the grade 2 level, and nearly 20 percent could not read beyond a single word. Community oversight and participation has been advocated to increase education quality. Does this strategy work, and if so how can community participation be encouraged? Is more direct action by communities to teach their children to read more effective?
 
Context of the Evaluation: 
In Uttar Pradesh, India’s most populous state, only 43.5 percent of children in grade 5 can read at the grade 2 level. In response to this problem, the government established Village Education Committees (VECs) in every village in 2001. VECs consist of the elected head of the village government, the head teacher of the local school, and three parents who are nominated by their community. These committees are responsible for monitoring school performance, allocating school resources, and hiring additional contract teachers in the event of overcrowding. 
 
Despite the promising aspects of this program, a survey conducted in 2005 indicated that 38 percent of VEC members did not readily identify as being part of the committee, and 25 percent did not even know they had this role. Only 3.6 percent of all VEC members knew they had the ability to request funds to hire additional teachers, which is one of the main prerogatives and responsibilities of the VEC. 
 
Details of the Intervention: 
Working jointly with Pratham (a local NGO) and the World Bank, researchers designed three interventions that were randomly assigned in 280 villages in four rural blocks in Jaunpur district, eastern Uttar Pradesh, a populous and educationally struggling area in India. These interventions served to determine if more information and encouragement to use the channels available to them would cause VECs and community members to demand and receive better services. They contrasted this with direct action to improve learning outside the official channels. 
 
Intervention 1: In 65 villages, Pratham staff started a series of conversations about education in small groups throughout the community. These conversations covered the current status of schools in the village, the quality of local schools, state mandated provisions for schools, mid-day meals, and local funds available for education. People were asked if they knew about the VEC and its membership and responsibilities. After two days of meetings in small groups, a community-wide meeting was held where people were encouraged to discuss and ask for information about the VEC, with information gaps filled in by Pratham’s field workers. VEC members also received a pamphlet on their roles and responsibilities from the Pratham staff.
 
Intervention 2: In addition to all the steps outlined above, communities in another 65 villages were trained and encouraged to conduct testing to see if children could read simple text and solve basic arithmetic problems. Volunteers put together a "report card" for each community, which was presented at the community-wide meeting. 
 
Intervention 3: In addition to the above two steps, Pratham officers taught volunteers in another 65 villages a simple technique for helping children learn to read. Volunteers were encouraged to start after-school reading classes-- they were invited to attend training sessions which lasted for four days, and staff returned an average of seven times to provide in-service training. The objective was to use Pratham-designed materials and local volunteers to supplement the normal curriculum, and improve literacy among village children. 
 
Comparison: Eighty-five villages received no treatment, serving as a comparison.
 
Results and Policy Lessons: 
 
Impact on Information Gaps: The average effect of all three treatments was an increase of 7.8 percentage points in VEC members who knew they could access public funds, and a 13 percentage point increase of members who had been properly trained. Parents were also 2.9 percentage points more likely to know that a VEC existed in their community.
 
Impact on Engagement: Despite these improvements in awareness, there was little difference between the VECs’ performance in treatment and comparison villages. The only significant difference was that 20 percent more contract teachers were hired in Intervention 2 villages (although not in Intervention 3 villages). Also, the intervention did not increase the level of engagement of parents with schools. Parents were no more likely to have visited the school or to have volunteered time or money in the treatment villages than in the comparison villages.
 
Impact on Reading: In 55 of the 65 Intervention 3 villages, volunteers ran more than 400 reading courses. The average child in an Intervention 3 village who could not read anything at baseline was 7.9 percent more likely to be able to read at least letters. Those who could read only letters at baseline were 3.5 percent more likely to read at least paragraphs or words, and 3.3 percent more likely to read stories if they were in an Intervention 3 village. These changes in average literacy across the village came despite the fact that only 8 percent of children, including 13 percent of those who could not recognize letters prior to the intervention, attended the classes. Provided that the effects of Intervention 3 are channeled entirely through attendance at the reading classes, comparing the endline reading levels of the comparison group with the Intervention 3 treatment effects described above reveals just how large these effects are:  all children who could not read at baseline but attended classes ended up being able to read letters at endline, and 98 percent of children who could read at the word or paragraph level was able to read at the story level. 
 
Intervention 3 was the only intervention which actually improved educational outcomes, by empowering individuals to improve teaching in their own communities. This suggests that enabling local action which does not depend upon large-group participation may be a means of directly affecting educational outcomes.  

Roots and Remedies: Persistent poverty and violence amongst urban street youth in Liberia

Policy Issue:

Poor and underemployed youth can be found at the hearts of riots, revolutions, civil wars, and petty and organized crime. In post-conflict countries, where state capacity is weak, frustrations are many, and jobs are few, policymakers are particularly concerned about these youth’s potential to destabilize society. Liberia, which recently suffered through 14 years of civil conflict, has named “youth disempowerment” as one of two major threats to durable and lasting peace. Liberia’s 2009 Youth Fragility Assessment sums it up this way: “the youth… simply wish for… the prospect of some day earning an income, even a modest one. For many, this is the impossible dream... the challenge is to make it possible, soon and for everyone.” The stakes are extremely high. The World Bank writes: “while much of the world has made rapid progress in reducing poverty in the past 60 years, areas characterized by repeated cycles of political and criminal violence are being left far behind….," and calculates a civil conflict costs the average developing country roughly 30 years of GDP growth. A quarter of the world’s population (1.5 billion people) live in places plagued by recurring and endemic violence.

How can governments and NGOs raise employment and reduce the risk of violence among these poor and risky populations? Aid programs increasingly focus on helping youth through markets, especially through microenterprise development. The logic of this assistance, however, rests on the existence of market failures among the poorest of the poor: imperfect credit markets, or production discontinuities such as minimum start-up costs or low returns to small investments. Cash grants or credit are needed to achieve minimum scale. Street youth with no assets and weak social networks may be particularly vulnerable to this trap. But so far there has been little research proving the existence of market failures or the ability of aid to help.

Meanwhile, both psychologists and economists have begun to explore the extent to which behavioral skills – such as impulse control, time preferences for immediate vs. delayed gratification, risk aversion, conscientiousness, setting and keeping long range goals, and being deliberate in choices – contribute to poverty. In a war zone, being highly present-focused might indeed be the optimal survival strategy. During peacetime, however, the absence of such preferences could in theory constitute a second source of persistent poverty: a behavioral poverty trap, leading to low savings rates, wastage of any windfalls, and high-risk behavior including involvement in drugs, crimes, and violence. Importantly, core principles underlying much economic and psychological theory assume that such preferences are fixed in young adulthood, leading anti-poverty projects to take a paternalistic approach. Again, little research has critically examined these assumptions.

Counter to conventional wisdom, preliminary investigation suggests that a behavioral transformation program, akin to cognitive behavioral therapy, can be successful. This finding, if true, would be groundbreaking, challenging conventional economic and psychological models of behavior, which posit that preferences and behaviors are stable and difficult to change, especially among adults.

Context of the Evaluation:

The study is designed to disentangle how cash and capital constraints versus dysfunctional preferences and behaviors contribute to the poverty and violence of the young men and women living on Monrovia’s streets, and to create an inexpensive and scalable program that will reduce poverty, violence, and social instability among unstable youth in Liberia and beyond.

On the preferences and behaviors side, the questions are (a) What role do cognitive and behavioral traits play in persistent poverty and violence?; (b) Are these cognitive and behavioral traits malleable in adulthood, and is sustained cognitive behavior change possible?; and (c) Will changing them reduce poverty and violence? On the market failures side, the questions are (a) What role does the lack capital and credit play in persistent poverty and violence?; (b) Will unconditional cash transfers relieve this constraint and reduce poverty and violence?; and (c) Do capital constraints and cognitive and behavioral deficiencies interact, and must both constraints be relieved to reduce poverty and violence in sustained way?

Description of the Intervention:

This “Sustainable Transformation for Youth in Liberia” (STYL) program is an experimental program, being jointly run by the research team and two NGO partners: CHF International and NEPI. As of mid-2012, STYL will have enrolled approximately 1,000 youth. Youth are recruited from urban areas where large numbers of underemployed youth congregate, and are targeted for the program on the basis of exhibiting the following characteristics: persistently poor; homeless; lack of self-discipline; angry, hostile, depressed; idle and not busy with productive pursuits; involved in organized or petty crime, and/or conflict with the law; and getting drunk and/or high regularly.

The STYL study is currently experimentally evaluating two interventions, each on its own as well as in concert with the other.

A behavioral Transformation Program (TP), akin to cognitive behavioral therapy (similar to Alcoholics Anonymous) and life-skills programs. The TP has the aims of bolstering the cognitive and social skills necessary for entrepreneurial self-help, raising youth’s aspirations, and equipping the youth to reach them. The TP involves half-day sessions 3-times a week, for 8 weeks, held in groups of 20 led by 2 counselors. The curriculum includes modules on anger management, impulse control, future orientation and planning skills, and self-esteem.

An unconditional cash grant program, in which youth are given a large $200 one-time cash grant disbursement. How the grant is spent is entirely up to the recipient, though a grant orientation session provides some basic training on financial management and business planning.

Individual youth are randomly assigned to either receive the TP; the cash grant; the TP and then the cash grant; or neither.

The plan is to conduct both short-term and long-term endline surveys to capture treatment effects, through surveys and behavioral games. If the basic interventions are shown to be effective, the research team hopes to further improve program design through iterative tweaking and testing, including varying cash grant size and TP length and intensity, and trying additional potentially complementary interventions, in order to help policymakers achieve goals most cost-effectively.

Results and Policy Lessons:

Forthcominng

 

Media coverage of this project:

Chris Blattman Talks with NPR's Planet Money team here.

Chris Blattman and Paul Niehaus in Foreign Affairs here.

Jason Margolis interviews ex-combatants and researchers Tricia Gonwa and Chris Blattman.

Ex-combatant Reintegration in Liberia

For post-conflict societies, the challenges of reintegrating ex-combatants and war-affected youth are likely to far outlast and outsize the formal demobilization, disarmament and reintegration (DDR) of ex-combatants. These programs, conducted in war’s immediate aftermath, form an important part of a policymaker’s post-conflict toolkit. While ex-combatants receive special policy attention, poor and underemployed men are also widely considered a threat to political stability.

Find a more in-depth policy report here. 

Context of the Evaluation:

In Liberia, where the bulk of the population is young, poor, and underemployed, many rural youth continue to make their living through unlawful activities, including unlicensed mining, rubber tapping, or logging. Many of them are ex-combatants, and some remain in loose armed group structures, doing the bidding of their wartime commanders. While the security situation has steadily improved since 2003, the government, the UN, and NGOs fear that these youth are a possible source of instability, particularly in hotspot regions where mining, rubber tapping, or logging and the allure of “fast money” attract young men from around the country. These youth may also be recruited into regional conflicts as mercenaries. Agriculture is and will continue to be a major source of employment and income for rural Liberians. The international NGO Landmine Action (LMA, now known as Action on Armed Violence) runs an innovative and intensive agricultural training program, targeting ex-combatants and other high-risk youth in rural hotspots.

Description of the Intervention:

The LMA program is broader and more intensive than most ex-combatant reintegration programs, and is designed to rectify some of the main failings of prior demobilization programs: it is oriented towards agriculture (the largest source of employment in Liberia); it provides both human and physical capital; and it integrates economic with psychosocial assistance. It also targets youth at natural resource hotspots that presented the most immediate security concerns.

LMA took youth selected for the program to residential agricultural training campuses, where they received 3-4 months of coursework and practical training in agriculture, basic literacy and numeracy training, psychosocial counseling; along with meals, clothing, basic medical care, and personal items. After the training, counselors facilitated graduates' re-entry with access to land in any community of their choice.  Graduates received a package of agricultural tools and supplies, valued at approximately US$200. The program's total cost is approximately $1,250 per youth, excluding the cost of constructing the campuses. The program was designed to give youth a sustainable and legal alternative to illegal resource extraction, ease their reintegration into society, reduce the risk of re-recruitment into crime and insurrection in the future, and to improve security in hotspot communities.

LMA recruited twice as many youth as it had space for in its programs, and researchers randomly assigned half of the youth to treatment (receiving the program), and half to a comparison group (not receiving the program). By comparing these two groups 18 months after the program, researchers can see the effect of the intervention on agricultural livelihoods, shifts from illicit to legal employment, poverty, social integration, aggression, and potential for future instability.  Despite massive migration, 93% of the youth were found at the time of the endline survey. The qualitative study included observation and a series of interviews with 50 of the youth.

Results and Policy Lessons:

Engagement in agriculture: More than a year after completion of the program, program participants are at least a quarter more likely than the control group to be engaged in agriculture, and 37% more likely to have sold crops. Interest in and positive attitudes toward farming are also significantly higher among program participants. 

Illicit activities:The program had little impact on rates of participation in illicit activities like mining, but those who participated in the program do spend fewer hours engaged in illicit activities, as agricultural hours seem to substitute somewhat for hours spent in illicit activities.

Income, expenditures, and wealth:  There was a sizable increase in average wealth from the program, especially in household durable assets, but no change in current income (last week and last month), savings or spending for the average program participant. Overall, the evidence suggests that cash cropping provides periodic windfalls from sales, and that these are mainly invested in durable assets (and not necessarily in agricultural inputs or equipment).  Qualitative observations also suggest that access to markets may have been an important constraint on success.

Social engagement, citizenship, and stability:  There were small but positive improvements across most measures of social engagement, citizenship, and stability. While not all of the estimated impacts are large enough to be statistically significant, they nevertheless suggest a small but broad-based reduction in alienation and some gains in stability. The evidence on aggression and crime, however, does not point to a significant reduction in illegal or aggressive behaviors among program participants.

Interest and mobilization into the election violence in Cote d’Ivoire:Conflict broke out in Cote d’Ivoire shortly before the launch of the program evaluation.  Self reported rates of interest in the violence and mobilization were fairly low among the sample population, but they were especially low among program participants – they tended to report a third less interest in or links to recruiters and recruitment activities. Given the difficulty of shifting such behaviors, these impacts of the program are regarded as extremely promising.

For a policy memo with detailed results, as well as recommendations for reintegration, livelihoods, and poverty alleviation programs in Liberia, please see here.

Reconciliation, Conflict and Development: A Field Experiment in Sierra Leone

Reconciliation programs aim to help communities heal after war, but we know little about how effective these programs are and which approaches best serve post-conflict communities. In this study in Sierra Leone, researchers measure the impact of a community-based reconciliation program designed to address social divisions that remain following the 11-year civil war. Researchers will analyze how the program impacts individuals’ attitudes toward violence, the incidence of disputes and crimes in the community, and the mechanisms used to resolve disputes.

Policy Issuce:

Civil war has disrupted the lives of millions of people living in post-conflict countries in sub-Saharan Africa.  Communities are left with physical destruction to village infrastructure and social divisions amongst combatants and victims. How do community members, once in conflict with neighbors, move forward with the traumatic memories of war?  Every year, development dollars are allocated to transitional justice programming in these war-torn countries, yet the effects of this programming have not been addressed in development economics literature.

Context of the Evaluation:

Sierra Leone’s eleven-year civil war ended in 2002, affecting the entire country.  Despite the passage of almost a decade since the end of the war, many parts of the country have not fully recovered from the conflict.  Physical damage to public goods, such as buildings and water pipes, remains.  Many former combatants, who were children during the war, have not returned to their villages—possibly fearing they will not be accepted.

Fambul Tok International has developed a community-based reconciliation initiative to help communities rebuild. Fambul Tok’s programming provides an alternative to commonly accepted transitional justice mechanisms, such as retributive war crimes courts or truth and reconciliation commissions, both of which have had a presence in post-conflict Sierra Leone. Few community reintegration and reconciliation models, like that of Fambul Tok, have been rigorously evaluated, and evidence on best practices are lacking.

Description of the Intervention:

Fambul Tok’s approach to reconciliation engages community leaders to organize a forgiveness bonfire and cleansing ceremony in the village. At the ceremony, victims, perpetrators and witnesses all have the opportunity to publicly describe their experiences.  Perpetrators are given the opportunity to ask for forgiveness and victims the opportunity to forgive.  After the ceremony, the community is brought together through activities such as communal farming and designating a peace tree for dispute resolution. These ongoing activities are designed to promote social interconnectedness and build social capital.

One hundred sixty randomly selected villages will receive this intervention.  Twelve individuals in each of these villages, as well as in another 160 comparison villages that do not receive the program, will receive a baseline survey collecting data on socioeconomic status, trust, psychosocial well-being, social networks, and specific war-time experiences. An additional village-level survey will be administered in each village to a group of community leaders to garner information on the village’s war experiences and active community projects like communal farms and schools. A follow-up survey will be administered to the full sample one year later.

To understand how reconciliation affects conflict, researchers will analyze how the program impacts individuals’ attitudes toward violence, the incidence of disputes and crimes in the community, and the mechanisms used to resolve disputes. An assessment of whether reconciliation serves as a base for economic development will be conducted through analysis of how the program affects economic activity. The study will further measure whether individuals within treatment communities are more willing to work together or contribute resources for communal ends.

In addition to seeing if reconciliation improves social and economic outcomes, we hope to identify the mechanism through which it affects behavior.  We plan to identify the mechanism through the use of behavioral games, which will be conducted in a subset of the communities.  Behavioral games are complementary to respondent interviews, insofar as they measure differences in observed rather than reported behavior.  The behavioral experiments are designed to disentangle four different components of intra-group behavior: in-group altruism, out-group altruism, reputation and trust. By overlaying four different types of experiments, we can test for different mechanisms through which a change in perceptions and beliefs leads to changed behavior. 

Results and Policy Lessons

Results forthcoming. 

Urban Property Rights in Mongolia

Do property and land rights lead to better access to credit and increased investments in one’s land? It is widely assumed so, but there is little evidence to support this assumption. In this study, researchers go to Mongolia where many recent migrants to urban areas lack property rights. Researchers are evaluating the impact of two versions of a program that provides direct assistance to households seeking to privatize and register land plots. They will measure the program’s impact on the migrants’ access to credit, investment in land and housing, property values, labor market outcomes, and household income.

 
Policy Issue

Having a well-defined system of land and property rights is thought to be extremely important for increasing investment, as ownership of land may incentivize investment by ensuring that tenants receive the long-term returns from improving their land. Property rights may also be an important component in access to credit, since land can be used as a collateral asset and increase the likelihood of a borrower receiving a loan. Large land-titling policies have been undertaken in several developing countries, particularly in South America and Southeast Asia in order to increase land investments and income security. However, the linkage between property rights, access to credit, and increased investments has not been well-established empirically.

Context of the Evaluation

More and more poor rural Mongolians are abandoning traditional nomadic herding practices and migrating to the cities in search of better lives. The bulk of these migrants are moving to Mongolia’s three biggest cities – Ulaanbaatar, Erdenet and Darkhan – where they either settle in suburban “ger areas” or peri-urban rangeland areas, often creating informal settlements. Mongolian laws give ger area residents the right to obtain ownership to the land upon which they live. However, the complexity and expense of this process make it difficult to become an owner and thus use the land as a marketable asset.

The Urban Property Rights Project aims to improve the formal system for recognizing and transferring land rights to ger area residents. This effort includes legal and regulatory reform, upgrading the technology necessary for accurate land parcel mapping, and providing direct assistance to households to privatize and register their land plots. The project is being carried out in Ulaanbaatar and eight other regional centers. 

Details of the Intervention

This evaluation focuses on the land titling component of the project known as the Privatization and Registration of Ger Area Land Plots Activity. This component will provide direct assistance to 75,000 households seeking to privatize and register land plots in urban ger areas. A random subset of eligible houses in the area will be randomly chosen to receive door-to-door assistance with the registration process. This assistance will include support for both the necessary paperwork as well as the registration fees.     

After the program is implemented, the researchers will evaluate its impact on access to credit, investment in land and housing, property values, labor market outcomes and household income using both household level surveys and aggregate institutional data.

Results and Policy Lessons

Results forthcoming.

Experiments to Improve Participation in a Recycling Program in Northern Peru

Policy Issue:
Economic growth in Latin America has come at the cost of increasingly acute environmental pressures. Expanding trade and consumption has led to increased waste generation and pollution requiring more developed solid waste disposal systems. Markets lack a price mechanism to internalize the environmental cost of this growth.  Policy makers often apply taxes, subsides or other mechanisms to attempt to align private incentives with public environmental preservation. Aside from altering financial incentives, growing evidence from psychology and behavioral economics research shows that behavior can successfully be influenced by leveraging social norms and emotions.  
 
Context:
Over 20,000 tons of solid waste are produced every day in Peru, most of which is dumped in waterways or informal trash heaps, making solid waste management an area of increasing concern for the country. PRISMA, a local NGO, operates a recycling program in Northern Peru whereby it trains and supports workers in forming associations that collect recyclables door to door from participating households. In addition to providing the informal workers with some initial tools and training, PRISMA further assists workers by canvassing the areas of operation to introduce the recyclers to the community and encourage the residents to segregate recyclable and take part in the recycling program. PRISMA was interested in identifying viable strategies to increase program uptake (34% at baseline), and reduce attrition of participating households from the program.
 
Description of Intervention:
Researchers worked with PRISMA to test a series of information messages aiming to improve take-up and participation its recycling program.
To improve take-up of households in communities where PRISMA planned to expand the program, researchers conducted a randomized evaluation to test the impact of different messages in eliciting program participation.  One week before the first PRISMA canvasser’s visit, a paper flyer was delivered with the a generic message about PRISMA’s program and one of nine specific messages eliciting pressures such as, social norm, peer comparison, conformity, authority, environmental or social benefits to increase participation.
 
Households that owned a cell phone and were willing to share their number (about 35% of the sample) received text messages once a week with the specific message in addition to the flyers. 
 
With a sample of 1,785 existing participants, researchers tested strategies to reduce program dropout and increase the amount and quality of the recyclables collected. Plastic bins were randomly distributed to households participating in the study. Some bins had a sticker specifying which items could be recycled.  Household that provided cell phone numbers were randomly assigned to receive either a generic or personalized SMS reminders to recycle or to serve as a comparison group without SMS reminders. SMS messages were sent the day before the weekly visit by the recycler, for a period of six weeks. Data collection for this component of the study lasted for eight weeks and included a careful accounting of the quantity and quality of recyclables received.
 
Results and Policy Lessons:
Results of Treatments on Program Enrollment: The results reveal no statistically significant effects from the different treatment messages. No significant impact on program take-up for the information campaign conducted through flyers alone or through flyers with text messages were found. Messages conveying social norms and applying social pressure were not successful in leveraging behavioral change. Two interpretations for this outcome are proposed by the researchers: a) these messages and norms were not relevant in this context, b) the large presence of informal recyclers operating outside of the program rendered separation of recyclables at the household level a non-issue.
 
Results of Treatments on Compliance: Households who received plastic bins turned in recyclables 3-8 percent more of the times and produced on average more (about 0.2 kg) and more valuable recyclables (about 0.1 pesos).  This finding suggests that convenience of storing recyclables is a barrier to greater program participation. 
The SMS reminders had no significant impact on the level and quality of participation of households in the program, suggesting that forgetfulness is not a serious constraints among households enrolled in the program. There was no clear difference in recycling compliance between households who received plain bins and those who received bins with explanatory stickers.

Communal Sanitation Solutions for Urban Slums in Orissa, India

Can improved toilet facilities, combined with innovative accountability systems for maintenance, increase the use of community toilets in urban India?

Policy Questions:

In densely populated and rapidly growing countries, severe space constraints, poor utilities infrastructure, and temporary housing construction can render private household sanitation facilities infeasible. Improving communal toilets, which serve entire neighborhoods, may be a more feasible way to improve sanitation, health and well-being in such densely populated areas. However, these kinds of facilities face their own set of problems. Because the benefits of cleaner facilities extend beyond the individual, people may be unwilling to help with repair and maintenance. When the toilets then fall into disrepair, people often revert to open defecation, leading communal toilets to be abandoned. Can innovative systems of facility management help overcome these “collective action” problems and make communal toilets a sustainable option in urban slums?

Evaluation Context:

In the slums of Bhubaneswar and Cuttack in India, almost 45 percent of households use either public toilets, which are meant for a rotating population in commercial areas, or communal toilets, which serve a fixed residential population. However, the condition of these facilities is very poor. A preliminary survey showed that 53 percent of these toilets were either “dirty” or “very dirty”, and one in six facilities was completely non-functional. Households who were dissatisfied with the cleanliness of their community’s toilets were more likely to practice open defecation, and almost 30 percent of households reported doing so. Qualitative research suggests that these poor conditions may be caused by weak systems of accountability for toilet maintenance and repair.

Details of the Intervention:

This program sought to improve the physical infrastructure of community and public toilets, as well as to improve the associated management systems in order to ensure long-term maintenance. The physical infrastructure of a set of existing community toilets and a smaller set of existing public toilets will be updated to ensure that all have an adequate number of gender-separated toilets and washbasins; sufficient lighting and ventilation; and enough water for all services. A set of new toilets will also be constructed to these standards. A randomly chosen subset of both the community and public toilets will also be given enhanced, infrastructure, such as a space for bathing. Half of the improved community and public toilets, including both those with and without the enhanced infrastructure, will be randomly selected to be maintained by a private firm, while the remainder will be managed by the community according to a “constitution” that specifies responsibilities and rights.

In order to identify a solution that will produce the most attractive, sustainable and hygienic alternatives to open defecation for slum residents, researchers will test a variety of complementary household-level interventions, such as discount coupons for shared facilities and varying the pricing structure (monthly passes vs. pay-per-use). Researchers will also conduct a program of demand generation activities in a subset of communities around community and public facilities. These activities will be used to help communities notice the problems associated with open defecation and develop community cohesion to sanction it.

Researchers will collect data to measure take-up and maintenance of sanitation facilities over the life of the program. Household surveys will be used to examine satisfaction with the facilities, instances of diarrheal disease, and differential access within the household.

Results & Policy Lessons:

Project ongoing, results forthcoming.

Community Driven Development in Sierra Leone

 
Policy Issue: 

While the accountability and inclusiveness of institutions are often considered key determinants of economic performance, there is little agreement about exactly how institutions should be designed, how to move from a system of bad institutions to one with good institutions, and whether and how foreign donors can help in this process. One of the most popular strategies employed by donors to promote democratic and accountable institutions at the local level is “community driven development” (CDD). Typical CDD interventions combine flexible grants that communities can spend on local projects with requirements that decisions must be made in an inclusive and transparent manner and training on how to do this. The participation requirements aim to ensure that the projects funded reflect the needs of the community and facilitate learning by doing—i.e. the experiences gained in deciding how to spend project funds leave minority groups  better placed to participate in other community decisions after the project ends. While billions of dollars are spent on CDD programs, few studies provide rigorous evidence on their real-world impacts. Critics of CDD, and of decentralization in general, have raised the concern that decentralized funds will be captured or exploited by local elites.

 
Context of the Evaluation: 

Scholars argue that frustrations with government incompetence and corruption, as well as the exclusion of women and young men from decision-making in the traditional chieftaincy system that coordinates the provision of many local public goods, fueled violence during Sierra Leone’s recent civil war.  To both prevent a return to violence and to stimulate economic development, the Government of Sierra Leone implemented a number of reforms that give communities, and vulnerable groups within them, a greater voice in local decision-making. Alongside a national decentralization program that re-established district-level councils, the government piloted a community-driven development project that went one step  further by providing small grants to be administered by village development committees. This extension down to the village level aimed  to establish more inclusive and accountable local decision-making infrastructure, rebuild trust, promote collective action, and provide minority groups (particularly women and youth) with experience in managing projects and making decisions  within their community. Researchers and the Decentralization Secretariat collaborated to evaluate whether this pilot, called the “GoBifo” Project (or “Move Forward” in Krio),  acheived these goals. 

 
Details of the Intervention: 

Two hundred thirty-six villages from two ethnically and politically distinct districts were randomly allocated into a treatment group or a comparison group. Villages in the treatment group were regularly visited by a GoBifo facilitator, who helped community members create or revamp Village Development Committees (VDCs), set up bank accounts for the VDCs, establish transparent budgeting practices, and create village development plans that included specifics on how GoBifo grants would be used. The participation and inclusion of marginalized groups was central to this process – for example, each social group (women, youth, and adult men) came up with their own development plan, and these plans were then combined into a single unified vision. Women were often established as treasurer of the VDC and served as co-signatories on all project finances. A series of block grants totaling US$4,667  per community were given to implement local public goods and skills training projects that were identified in the village development plans. 

Household surveys, which covered participation in local decision-making, attitudes to minorities, and engagement in collective action, as well as demographic and socioeconomic information, were collected in late 2005 and again in mid-2009, along with village-level focus group discussions. In addition, three structured community activities (SCAs) were conducted in late 2009, shortly after GoBifo activities had ended, to capture any persistent impacts on collective action, participation of minorities, and elite capture. The SCAs were designed to measure how communities responded to concrete, real-world situations in three areas where GoBifo had sought to change behavior: (i) raising funds in response to a matching grant opportunity; (ii) making a community decision between two comparable alternatives; and (iii) allocating and managing an asset that was provided for free. 

 

Results and Policy Lessons: 

The authors and project team agreed a set of hypotheses they would test at the start of the evaluation (in 2005) and wrote out a plan on exactly how the data would be analysed before looking at the data. This prevented selective “cherry picking” of results from the 318 variables collected.

Project Implementation and Local Infrastructure Investment: The GoBifo project successfully established the village-level organizations and tools to manage development projects in nearly all cases. The distribution of project benefits within communities was equitable, leakage of project resources minimal, and minority participation high. 

GoBifo villages had a larger stock of higher quality local public goods, such as a functioning primary school or community grain-drying floor, than comparison areas. There was also more market activity in treatment communities, including the presence of more traders and items for sale, suggesting short-run economic gains. 

Institutional Change and Collective Action: There is no evidence that the program led to fundamental changes in local institutions or decision-making. Despite the fact that many women in treatment villages participated in GoBifo decisions, they were no more likely to voice an opinion in community meetings after the project ended or to play a leadership  role in other areas. Similarly, the establishment of a democratically elected village development committee that carried out multiple projects did not lead treatment villages to be any more successful at raising funds in response to a later matching grant opportunity.  Lastly, there were no program impacts on elite capture, although levels of capture were low in the research communities (at least as measured by the third SCA). 

Peace Education in Rural Liberia

For new democracies and societies emerging from conflict, encouraging tolerance and dialogue, strengthening non-violent conflict resolution systems, and increasing understanding of human rights are key priorities. Governments and NGOs commonly try to change the political culture, civic values, and practices of conflict resolution at the local level through widespread dialogue, education, and information campaigns.  But do these dialogue and education programs actually work as intended? Do they change norms and behaviors, and if so, how? How are new patterns of conflict resolution formed?  And how do they contribute to national reconciliation? How do new state structures integrate with pre-existing local bodies to jointly support security goals and human rights, especially where traditional structures are in conflict with the later? In short, what programs are most useful in helping post-conflict countries achieve lasting peace?

Find a more in-depth policy report here.

Context of the Evaluation:

More than five years after the end of Liberia’s fourteen year civil war, underlying tensions between tribes, over land, and between youth and elders continue to pose threats to a fragile peace. The UN’s Peacebuilding Commission and the Government of Liberia are working together to promote non-violent dispute resolution and inter-group reconciliation, but how best to do this is unknown. This study was jointly designed by the UN, the government, and the research team to assess whether civic education and conflict resolution programs can contribute to this broader peacebuilding agenda.

Description of the Intervention and Evaluation:

Target communities for the program were identified within Liberia’s three most conflict-prone counties: Grand Geddeh, Lofa, and Nimba. The researchers randomly assigned half to receive the program as the “treatment” group, and half to not receive the program as a “control” group. The program took place in 67 villages and town quarters. The program mobilized and trained community members in order to achieve three main goals: (1) educate people on their rights and to respect the rights of others; (2) encourage community collective action towards shared goals; and (3) foster non-violent dialogue and conflict resolution. The program is notable for its intensity and reach: in each community, roughly 10% of adults participated in an eight-day long interactive workshop held over the course of several weeks. Workshops had between 20 to 30 participants, both men and women, were led by a professional facilitator, and were conducted in local dialects. Multiple workshops were held in most communities to reach the 10% coverage target.

Pre-program baseline data was collected in 2009, and the endline took place between 1 and 22 months after the program. Data came from more than 5,000 individuals with three main surveys. In each community, the team interviewed: 20 randomly-selected “community members”, 4 “community leaders”, and 3 people identified by local chiefs as potential trainees, including a “troublesome” person. The study measures the impact of attending the program on potential trainees, random community members, and community leaders, and the impact on the community of having the program take place in their community. An in-depth qualitative study in 14  of the communities was conducted alongside the randomized evaluation to determine the mechanisms of impact. The study focuses on four major outcome classes: community and political participation; attitudes to rights; civic knowledge, attitudes and beliefs; and the prevalence and resolution of conflict.

Results and Policy Lessons:

Community and political participationCommunity participation was measured through contributions to public goods and community projects, membership in groups (from farming to sports), membership in a peace group, and leadership in groups.  Across all measures, the only treatment effect was on membership in peace groups. On political participation, the only statistically significant treatment effect was seen on an index measuring whether potential trainees feel free to speak their minds to “big people” in the community and whether they feel community members have the right to speak out to elders: Those trained are 4% more likely to feel empowered to speak freely. This effect is concentrated among the “troublesome” individuals, who see a larger increase of about 8%.

Attitudes on human rights: Across multiple measures, nearly all the treatment effects are positive, indicating that respondents in trained communities generally report more progressive beliefs. For community members, however, these impacts are fairly close to zero. The impacts on trainees and leaders are modest in size – often in the range of 1 to 10% —and seldom statistically significant at conventional levels.

Civic attitudes and knowledge: The civic education component provided information on citizenship, civic rights and responsibilities and Liberia’s political structure. At endline, amongst potential trainees in the control group, only about 12% correctly understood the statutory requirements for citizenship. This understanding nearly doubled among treated trainees. Little change was seen in political knowledge, and the program also appears to have little to no impact on perceptions of equity in community governance as well as on perceptions towards the national government.

Prevalence and Resolution of Conflict: The most striking program impacts were on conflict and its resolution. In treatment communities (i.e. those that received the program), the evaluation found sizeable increases in non-violent inter-personal and inter-group disputes; suggestive evidence of a decrease in violent disputes; increasing levels of land conflict since the program, though also suggestions of lower rates of violence, and increased rates of dispute resolution and of satisfaction with those resolutions in trained communities. At the community level, leaders reported a 93% increase in conflicts (typically disputes and disagreements) between youth and elders in treated communities.  Treatment communities were also twice as likely to have a peaceful strike or protest, and three times as likely to have a witch killing (though the latter result is not statistically significant).

Violent strikes, protests, and inter-group violence are 59% less likely in treatment communities, however, though this result is not statistically significant (partly because the events are rare, making it difficult to estimate their prevalence precisely with such a small sample of communities).

In addition to the policy report detailing the impact evaluation results, the team has produced a second policy report analyzing patterns of conflict.

The Impact of Women Policy Makers on Public Goods in India

Policy Issue:

In 2008, women accounted for 18 percent of parliament members worldwide, and only 13 countries had a female head of government. This underrepresentation has prompted governments to adopt reservation policies; assuring women will have a role in political leadership by reserving a certain proportion of seats for female candidates. Reservation policies clearly have a strong impact on women’s representation, and some evidence suggests that women and men have different policy preferences. However, very little is known about the actual impact of women’s representation on policy decisions.

 
Context of the Evaluation:

In 1993, a constitutional amendment in India called for a random one third of village council leader, or pradhan, positions to be reserved for women. The village council, which encompasses between five and fifteen villages, is responsible for the provision of local infrastructure – such as public buildings, water, and roads – and for identifying government program beneficiaries. Although all decisions in the village council are made by majority, the pradhan is the only full-time member and exercises significant control over the final council decisions. The village council is required to organize two village meetings per year, during which they present their proposed budget and report on their activities in the previous six months. The pradhan must also set up regular office hours where villagers can lodge complaints or requests.

 
Details of the Intervention:

Researchers studied the policy consequences of mandated representation by determining whether there was any difference in the provision of social services between male and female led village councils. As village councils were randomly selected to be reserved for women, differences in investment decisions can be attributed to the reserved status of the council.

Data was collected in two locations: Birbhum in West Bengal and Udaipur in Rajasthan. In Birbhum, data was collected in two stages. First, in each village council, researchers conducted an interview with the pradhan, asking about his or her family background, education, previous political experience, political ambitions, and the village council’s recent activities. In the second stage, from the 5-15 villages represented by each village council, three villages were randomly selected to be surveyed for available public goods and existing infrastructure. Researchers also collected minutes of the village meetings, and gathered data on what complaints or requests had been submitted to the village council in the last six months. Two years later, the same village level data was collected from 100 village councils in Udaipur. However, there were no pradhan interviews.

 
Results and Policy Lessons:

The results suggest that reservation affected policy choices. In particular, it affected policy decisions in ways that seem to better reflect women’s preferences. In West Bengal, women complained more often than men about drinking water and roads – 31 percent of women’s complaints were about drinking water and 31 percent were about road improvement, compared to 17 percent and 25 percent of men’s, respectively. These preferences were revealed in the investment decisions of reserved village councils. Village councils reserved for women, on average, invested in 9 more drinking water facilities and improved road conditions by 18 percent.

In Rajasthan, 54 percent of women’s complaints were about drinking water and 19 percent were about welfare programs, compared to 43 percent and 3 percent of men’s, respectively. Unlike in West Bengal, compared to men, women complained less frequently about roads. Only 13 percent of women’s complaints were about roads, compared to 23 percent of men’s. This breakdown of preferences was again revealed in the investment decisions of the village councils. Village councils reserved for women invested in 2.62 more drinking water facilities, on average, and made fewer improvements in road conditions, leading to an 8 percent deterioration.

Researchers investigated whether these results could be explained by the fact that female pradhans were inexperienced, that they perceived themselves as being less likely to be re-elected, or that they tended to come from more disadvantaged backgrounds than men. However, there was no evidence that the policy impact of the reservations was driven by features other than the gender of the pradhan.

Overall, these results indicate that a politician’s gender, and identity more generally, does influence policy decisions. This finding is likely to have implications beyond reservation policy.

Selected Media Coverage:
 

Impact of Voter Knowledge Initiatives in Sierra Leone

This study looks at the impact of developing and disseminating material on political candidates. In a context with low literacy levels and limited media penetration innovative approaches and different forms of media are shared with voters and evaluated to better understand the impact on voter knowledge and candidate selection.

 
Policy Issue:

Transparent and accountable government institutions are thought to be more effective at delivering important social services such as education and healthcare. However, there is little consensus over how best to enhance these aspects of governance, particularly in places where conflict has recently caused breakdown in democratic institutions. Evidence from Brazil and India suggests that increased information about politician performance can result in lower vote shares for low-performing or corrupt representatives. There is also evidence that town hall meetings, where representatives meet directly with constituents, can increase voter knowledge, turnout and support for participating candidates. While many interventions have tested the efficacy of these strategies at increasing basic voter knowledge and access to candidates, little work has been done where democratic institutions are nascent and where public information is limited. In such settings reliable information on candidates may be limited or non-existent, and thus requires significant effort to collect, compile and then convey such information to voters in a comprehensible manner. Debates may provide a feasible alternative which could work in many settings.

 

Context of the Evaluation:

Sierra Leone’s 2012 elections were hailed by international observers as generally peaceful, free, and fair.  In previous elections voting patterns in Sierra Leone have been overwhelmingly based on pre-existing party affiliations.  However, during the 2008 elections, people in Sierra Leone were more likely to vote against traditional party and ethnic affiliations in places where they had more information about candidates (for example, in local elections).  Many election-related social programs focus on logistics and informing people about the importance of voting, but as Sierra Leoneans become more familiar with the democratic process there is also room to help people learn more about the different candidates among whom they will be choosing. The 2012 election presented an opportunity to test new electoral programs that could increase transparency, voter knowledge of candidates, and voter engagement.

 
Description of the Intervention:

In the run-up to the November 2012 elections in Sierra Leone, implementing partner Search for Common Ground filmed debates between rival candidates for membership in parliament (MP). From a total of 264 polling centers, 112 were randomly assigned to receive community screenings of these debates, 40 received interventions that provided information to individual voters, and another 112 served as a comparison group. 

Firstly, debates were shown at almost 200 community screenings in polling centers across Sierra Leone, where they were seen by an estimated 19,000 people. Surveys of voters before and after they watched these debates measured how their perception of candidates, their knowledge of candidate positions, and their voting intentions were altered. 

In the 40 polling centers assigned to receive individually delivered information, individuals were allocated one of the following groups

  1. Debate: Individuals were shown the exact same debate screened in polling centers on a personal handheld device.

  2. Getting to Know You: Individuals were shown a “getting to know you” video of the same two candidates speaking informally about their hobbies and interests.

  3. Radio Report: Individuals listened to a recording of an independent moderator or journalist summarizing the main policy positions articulated by the two candidates during the debates.

  4. Thin Slice Evaluations: Individuals participated in a “lab” experiment where they were exposed to pairs of isolated images, voice recordings, and names of candidates from other constituencies across the country and asked to rate them along a variety of metrics, such as who they thought would be a better leader.

  5. Comparison Group: Individuals were surveyed, but not shown any media.

Evaluating and comparing these groups will allow researchers to disentangle the effects of different kinds of information, such as policy positions, personal characteristics, or persuasive speeches, on voter behavior.

On election day and the days following researchers administered a short exit survey to both comparison and treatment voters, assessing their knowledge about candidates, previous voting behavior, choices in the local and national election, and how they made their electoral choices. During 2013, researchers are conducting several follow-up activities, including further research on the use of video clips to disseminate electoral information, and interventions targeted at MPs to remind them of their electoral commitments.  

 
Results and Policy Lessons:

Project ongoing, results forthcoming

Does Microfinance Promote Generosity? Evidence from India.

Poor households often rely on the generosity of others to cope with periodic shocks to their health and income. Despite its pivotal role, there is substantial variation across poor communities in the level of observed proxies for generosity (such as transfers across households and measures of altruism and trust).

An important role for public policy may be to enable generosity and improve informal insurance in settings where resources are scarce and institutions are weak.

This project will evaluate the impact of an influential development program—microfinance—on generosity in poor communities.

Impact evaluations of policy initiatives or development projects rarely take into account measures of giving, trust, and mutual exchange, even though these are key elements of the overall well-being of individuals and communities. This project will exploit the randomized rollout of a local microfinance provider’s new branches in South India. It will conduct a large-scale household survey before and after microfinance interventions, in order to measure changes in self-reported measures of inter-household acts of generosity (financial transfers, gifts, interest-free loans, lending of durable products, etc.).

This project is part of the Science of Generosity research initiative at the University of Notre Dame.

Reduction of Gender-Based Violence Against Women in Cote d’Ivoire

By adding a component of gender discussion to Village Savings and Loan Associations (VSLAs) and including males in the proposed intervention, this study assesses the potential for improving the physical, social and economic outcomes for women in Côte d’Ivoire. Does including men in gender discussions in the context of financial decision-making help to reduce gender-based violence and increase economic outcomes for women?

Policy Issue:

Gender-based violence toward women and girls represents a violation of human rights and organizations like the International Rescue Committee (IRC) are actively engaging in rigorous research to determine the most effective ways to reduce this abuse.  Evidence from South Africa suggests that combining gender dialogue components to economic interventions can reduce partner violence.[1] Another program tying discussions of household dynamics to regular microfinance groups in Burundi also yielded promising findings[2]. While many programs that aim to reduce women’s exposure to gender-based violence focus solely on women, few interventions incorporate males to change their attitudes toward gender and their roles in the perpetuation of violence. This study addresses the potential to reduce gender-based violence within the post-conflict context by supplementing economic programs with a gender-training component for men and women.

Context of the Evaluation:

Côte d’Ivoire has experienced two civil wars in the past decade, most recently in November 2010 after a disputed presidential election. As of May 2011, the country has returned to a period of peace and internally displaced peoples (IDPs) are slowly returning to their homes. A recent report from Oxfam, Care and the Danish Refugee Council highlighted the potential humanitarian crisis IDPs represent in Cote d’Ivoire—over 450,000 Ivoirians remain displaced inside Côte d’Ivoire and abroad. About 58% of returnees and 82% of displaced Ivoirians report complete loss of revenue streams, and these populations remain highly susceptible to attacks, harassment and intimidation.

As part of its mission to respond to humanitarian crises and help post-crisis communities rebuild, the IRC uses Village Savings and Loan Associations (VSLAs) to increase savings opportunities and capital accumulation for self-selected groups of men and women. VSLA groups are composed of 15-30 community members who decide to contribute to a collective savings fund that members can borrow from, with the loan interest allowing the fund to grow over time. VSLA groups agree collectively on a payout date, when all members will receive a share of the common fund plus interest. The IRC provides initial training and resources such as a lockbox and notebooks, meaning that administrative costs are low and decision-making is driven entirely by the VSLA itself.

Description of the Intervention:

This study assesses the impact of a socio-economic program and discussion group on the incidence of physical and sexual violence and women’s individual agency. A randomized evaluation will be conducted in 24 villages across Côte d’Ivoire, reaching 48 newly-established VSLA groups. Village groups will be randomly assigned to either a treatment arm (gender dialogue program in addition to the VSLA program) or a comparison arm (VSLA program only).  Women will be surveyed before and after the program is completed, while men will be invited to participate in in-depth qualitative interviews. Additional midline quantitative and qualitative data collection activities were added to capture changes in household well-being as a result of the post-election instability in late 2010 and early 2011.

Gender dialogues will be facilitated through the IRC’s current structure for VSLA groups.  In addition to regular VSLA activities, groups assigned to the gender dialogue treatment will encourage men and women to discuss the processes by which economic decisions are made within the household and their current systems of control over household resources. By challenging gender norms within the dialogue groups in the context of savings and spending decisions, women’s position in the household may be strengthened over time.

Results:

Results forthcoming.


[1] (Proynk et al, 2006)

[2] (Iyengar and Ferrari 2011)

Discrimination in the Judicial System

Policy Issue:

In 2004, over 40% of sentenced inmates in the United States were African-American; with African-American males incarcerated at seven times the rate of white males. A long-standing principle of the United States’ justice system is that defendants should not be treated differently because of their race, codified in the “Equal Protection” clause of the 14th amendment to the Constitution. Differential sentencing or conviction rates by race could be evidence of a violation of this clause, making this is an important issue to address on legal grounds. However, it is possible that the observed racial gap in sentencing can be explained by other factors such as income, education level, or factors which are more difficult to observe. Establishing whether or not courts differentially treat minority defendants also has important social implications: such practices might further exacerbate social inequalities and could contribute to a self-confirming equilibrium where expectations of racial discrimination affect criminal behavior.

Context of the Evaluation:

Cook County, Illinois is the largest unified court system in the United States, with over 2.4 million cases processed each year in both civil and criminal courts. It is also a racially mixed urban area, with a population that is 48% white, 26% African-American, and 20% Hispanic. Illinois state courts are governed by sentencing guidelines, which provide suggested sentencing ranges by category of offense. Previous studies have found that guidelines do mitigate inter-judge sentencing variation, but not substantially. Judges in the Cook County court are initially appointed or elected, and subsequently subject to retention elections every six years.

Details of the Intervention:

Researchers use data from the state courts of Cook County, starting with data from felony cases initiated between 1995 and 2001. The racial breakdown in the data is 12% white, 72% African-American, and 16% Hispanic, reflecting substantially different rates of representation by race in the criminal justice system. Rather than asking whether there is a racial gap in sentencing, this study seeks to determine whether there are systematic differences across judges in the racial gap in sentencing – in other words, are certain judges more likely to mandate punishments for members of a particular race? If there were no racial bias, then we would expect the gap in sentencing for people of different races to be the same across judges. But when cases are assigned to judges randomly, any difference across judges in the sentencing gap may reflect people of a particular race receiving different treatment from particular judges.

For the primary analysis, data is restricted to defendants who are African-American or white (excluding the 16% classified as Hispanic). Data is further limited to those cases adjudicated by a subset of judges in the Cook County Criminal Courts who did not have any unusual circumstances (such as lengthy capital trials) that would have resulted in non-random assignment of cases. The random assignment of cases to judges is central to this study as it ensures that each judge receives the same types of defendants of different races. Once random case assignment has been established, it can be inferred that any differences in judicial decision patterns are due to differences across judges and not to differences in case or crime characteristics.

Results and Policy Lessons:

Evidence suggests that there is significant disparity in the racial gap in incarceration rates across judges, supporting the idea that at least some judges treat defendants differently based on their race.

Researchers observed an average incarceration rate of 38% for whites and 51% for African Americans, although as mentioned above this gap could be explained by factors which are difficult to observe. However, when moving from the 10th to the 90th percentile judge in terms of racial discrimination, the gap in incarceration rates increased by 18 percentage points, suggesting significant variance in the sentencing gap across judges. The estimated average difference in sentence length is 10 months, however this cannot statistically be distinguished from a situation where race played no role in sentence length.

Although judges differ in the degree to which race influences their sentencing, no evidence is found to suggest that observable characteristics such as judges’ gender or age significantly predict their treatment of members of different races. Similarly, no systematic pattern emerges with respect to work history (such as whether the judge ever worked in public defense). However, there is somewhat stronger evidence that the racial gap in sentencing is smaller among African-American judges. Further, judges who are harsher overall (as measured by incarceration rate) are more likely to sentence African-Americans to jail relative to whites.

Decentralizing Education Expenditures: Primary School Community Grants in Niger

Policy Issue

In recent years school enrollment has risen dramatically in developing countries, prompting a renewed interest among policymakers in education management. Provision of educational services is centrally administered in much of the developing world, but evidence suggests that decentralized, locally administered services may be better suited to address low education quality. Local administration and oversight puts power into the hands of those with the most interest in seeing improvements in service delivery, and the best information about current education quality. By empowering local communities, decentralized management has the potential to combat systemic teacher absenteeism and reduce misallocation and corruption.

Context of the Evaluation

Niger has made significant strides in increasing access to education in the past decade, with primary enrollment increasing from 37% in 2000 to 54% in 2005. Yet the country still faces considerable challenges with regards to access, rates of primary school completion and the quality and management of the educational system at all levels. One problem, for example, is that schools are spaced far apart, making it difficult for the central government to effectively monitor them.

To address this problem the Ministry of Education mandated the creation of management committees (Committees de Gestion Scolaire, or COGES) for each school. The committee included the director of the school, as well as locally-elected community members. The COGES are responsible for monitoring teacher attendance and performance and managing both financial and material resources, such as the purchase of textbooks and supplies. The Ministry of Education introduced a pilot project to reinforce the capacity of the COGES through the provision of a yearly cash grant, which can be used to fund a part of its activities for the year, such as the construction of classrooms or the purchase of materials. These grants were provided under the belief that such financial support would spur communities to take a more active role in the management of schools.

Details of the Intervention

To test this belief, researchers, in collaboration with the World Bank and the Nigerian Ministry of Education, randomized the allocation of the grants. They observed schools which did and did not receive grants, to estimate their impact, as well as the impact of two supplementary accountability interventions on school management, student retention and learning achievement. One thousand schools in the regions of Tahoua and Zinder were randomly selected into treatment and comparison groups. The 500 schools in the treatment group each received an annual lump sum based on the number of classrooms in the school, averaging out to around US$2 per student. This grant size was chosen by the Ministry of Education with considerations of sustainability regarding a possible extension of the program. In addition to the grants themselves, two sub groups were randomly selected to receive complementary interventions designed to improve delivery and management of the grants.

One third of the treatment schools received financial monitoring in the form of a letter received at the beginning of the intervention telling the committee there was a 50% chance of a “financial control visit” at the end of the school year. Because of financial and capacity constraints, about 20% of the treatment schools received visits at the end of the year.

Another third of the treatment schools received expenditure restrictions in the form of a list of non-eligible expenditures, such as reimbursing school officials for travel, or spending money on things that can be produced locally.

Researchers used annual administrative survey data, pupil test score data, and a detailed financial questionnaire to determine the effect of the grants. The testing used a modified standardized test implemented in collaboration with the Ministry of Education Department of Education Assessment. The school data contained information on pupil and community participation, school resources and infrastructure and information on issues facing school governance. The functioning of the COGES, including how often they met, what they talked about, and whether they began improvement projects, was also monitored. 

Results and Policy Lessons

Supportive actions, which include parental financial and in-kind contributions, parent supervision of pupil attendance, and parent remedial action for pupil absenteeism, increased by 0.14 standard deviations in treatment communities. Most of this overall effect came from an increase in parental contributions to the school, which was 0.48 standard deviations higher in the treatment group. The overall impact was larger when the school committee was educated and was somewhat smaller when families live farther from school.

Management activities, which include parents’ association and school committee meetings, and whether the school committee is in charge of collecting fees and how they decide how the fees are spent, was 0.09 standard deviations higher in the treatment group. Specifically, the proportion of school committees in charge of collecting fees increased by 27 percent, the proportion of active parental association increased by 18 percent, and the frequency of school committee meetings increased by 5 percent.

Oppositional actions, which include supervising teacher attendance and sanctioning teachers, increased only in communities where the school committees were educated. In these communities, teacher supervision increased by 0.13 standard deviations.

Impact on school quality: The results indicate no short-term improvements in school quality, measured in terms of accountability and teacher effort. The lack of an impact on teacher effort is in line with the fact that very few of the schools undertook any sort of teacher supervision. However, one year after the treatment, the grants marginally improved (0.04 standard deviations) the material quality of schools. This improvement was largely driven by increases in the number of classrooms and construction of walls around the compound.

Impact on demand for education: The grant program led to a 0.17 standard deviation decrease in dropouts and a simultaneous 0.10 standard deviation increase in enrollment of students in Grade 2, indicating an increase in demand for education for young children.

Community Based Rangeland Management in Namibia

We conduct an impact assessment of the Community Based Rangeland and Livestock Management (CBRLM) program in Namibia.  This program is part of a larger set of interventions in the agricultural sector designed to reduce poverty among the population of the northern regions of the country.  Many people in the area rely on cattle production for their economic livelihoods, however overuse of the communal grazing areas and suboptimal grazing practices threaten the long-term viability of the land and contribute to persistent poverty.  

To increase the productivity of livestock and other animals using the land, the Namibia Millenium Challenge Compact funds a pilot program designed to help communities improve their livestock practices, address rangeland degradation, and improve market access.  

The evaluation is designed to test the impact of the various activities within the CBRLM intervention on household income, cattle productivity, and the condition of the rangeland. The intervention targets both inadequate information about appropriate cattle production practices and the social or other behavioral preferences of farmers.  At the moment, there is a collective action or “tragedy of the commons” problem – individual farmers are hesitant to reduce their herd’s impact on the rangeland because they are fearful that others will not follow suit which often results in overuse and degradation of the land.

Paying for Environmental Services: An Experimental Study in Bolivia

Can financial incentives and information influence farmers to account for spillover effects of their cattle management practices?

Policy Issue

Some agricultural and farming practices create spillovers that affect others or the environment. These spillover effects, known as externalities, can create a wedge between the benefits a farming practice has to individuals and the effects it has on society as a whole. Although adoption of agricultural technologies that reduce the production of negative externalities, such as pollution or deforestation, is beneficial to society, such technologies will not be adopted if they don’t bring benefits to individual users. The standard policy solution in the face of such externalities is to change incentives so that private individuals benefit from use of socially responsible practices. In recent years, policymakers have advocated payments for environmental services as an incentive-based approach to internalizing the externalities of land use decisions, but there is little empirical evidence on the impacts of such programs.

Context of the Evaluation

This study takes place in Bolivia’s Rio Grande Protected Area, where cattle are both a  source of private income, and of negative agricultural productivity, health and environmental externalities. Most participants in the study area own both cattle and forest land. Through forest clearing for pasture and free range grazing in the forest, local cattle management practices generate significant negative externalities at local (watershed) and global (biodiversity, carbon) scales. Free range grazing cattle spend a significant amount of time standing in streams, where much of the best vegetation grows. This causes erosion of the stream banks, and contributes to landslides that close roads and block access to markets.

Details of the Intervention

Researchers partnered with Fundación Natura Bolivia, an organization that has worked in the study area for six years implementing payment for environmental services projects. This study will evaluate interventions designed to increase the adoption of more environmentally friendly cattle management practices, including the establishment of pastures with trees and water troughs, which reduce negative externalities related to health, productivity and environmental processes by reducing the time cattle spend grazing near streams.

In half of the eligible villages, financial incentives will be introduced in the form of conditional contracts for cattle management. The other half of the villages will be provided only with information about the harms associated with poor cattle management. In addition to evaluating effectiveness of incentives for conservation outcomes, data collection will focus on ancillary measures that help shed light on the process of collective action within the community.

In the short run, adoption of these management technologies will be costly to individual users, but has the potential to create benefits for everyone in the village by reducing erosion. In the long run (>5 years), the technology is expected to be both privately and socially beneficial, as the reductions in land degradation bring benefits to individual users. By varying the price incentives for adoption and the information about the externalities, the intervention will offer insights into whether incentives and information can help farmers adopt technologies that prevent negative externalities.

Results and Policy Lessons

Results forthcoming.

Police Performance and Public Perception in Rajasthan

Policy Issue: 

Research on economic development has increasingly focused on the importance of good governance, both as a precondition for economic development as well as a major factor directly affecting human welfare. Of all government functions, perhaps the most central is ensuring the safety of citizens with an effective police force. In order for police to function effectively they must maintain law and order, which requires the trust and understanding of the public. Yet in many developing countries, the police are plagued with problems such as inefficiency, corruption, and an insular police culture which hinders performance and creates negative public perceptions. 

Context of the Evaluation: 

The police force of Rajasthan is no exception, struggling to improve professionalism, transparency, and responsiveness. Survey results reveal that the public views the police as corrupt, lazy and unfair. The self-perception of police was also found to be negative, as they themselves felt overworked, unappreciated and victims of political manipulation. Although the Rajasthan Police attempted previous reforms to improve performance and morale, the results were never clear, partly due to the lack of quantitative outcome indicators and partly because the programs were administered selectively to police stations that tended to differ from the average.

Details of the Intervention:     

Recognizing these problems, the Rajasthan Police initiated an intervention with the researchers which aimed to enhance police performance, improve public opinion, and gather objective information about crime rates and performance. In response to survey data that identified issues of concern, the police (with input from the researchers) designed four interventions, randomly implemented in 150 police stations in 11 districts across Rajasthan, with 25 of those stations serving as a comparison.

All treated stations received in-service training at the Rajasthan Police Academy Jaipur, which included classes to improve the competence level and scientific techniques of 350 investigating officers, and soft skills training for all personnel, including skills such as communication, mediation, stress management, motivation, and team building. Second, all treatment stations also froze administrative transfers for one and a half years, since frequent transfers seemed to have adverse effects on personnel and their families.

Some stations also invited local volunteers, called community observers, to sit in the police station for three hours in the morning and evening, observing day-to-day activities. The objective was to increase public awareness of the roles of the police and improve police behavior through informal monitoring. Finally, some stations implemented a weekly day off and a duty rotation system. The entire staff in selected police stations received one day off every seven days. Additionally, each person was given the opportunity to perform all duties on a rotating basis, equally allocating tasks among personnel. This provided a more transparent and fair work environment, potentially reducing stress.

The impact of the project was measured using a baseline and endline survey administered to 7,981 randomly selected individuals in the 150 police station jurisdiction. This survey covered public opinion of the police, public experiences with crime, and past interactions with police. Then a baseline and endline survey of approximately 50% of all police personnel, 2,367 officers, was conducted in project centers to assess police job satisfaction, efficiency, and performance.

Results and Policy Lessons: 

Impact of weekly day off, duty roster, and community observer: Having a day off each week and rotating duties caused some effect in increasing staff morale, however these effects did not generate significant changes in perceived police performance. The community observer also had no effect on public perception of the police. 

Impact of police training: In contrast, increasing the number of trained officers from zero to 100 percent raised the probability that crime victims were satisfied with police investigation by 16-21 percentage points. Since, on average, only 39 percent of victims reported being satisfied with their investigation, these improvements represent about a twofold increase in victim satisfaction. Training had little impact on other police activities, such as registering cases, asking for bribes, making arrests, or recontacting the victim, which suggests that most of the effects of the training operated through the behavior of the police with crime victims rather than their increased investigative success.

Impact of increasing the duration of job postings: A small reduction in transfers also produced an increase in citizens’ and victims’ satisfaction. Stations with a freeze on transfers increased the amount of crime victims who were satisfied with their investigation by 14-29 percentage points. 

These results demonstrate that it is possible, using the correct methods, to affect the public image of the police in a relatively short period of time, using an affordable and easily implementable set of interventions, including training and a freeze on transfer. Within Rajasthan, the police have already expanded the training program to 8,000 more officers, and the Government of India is working to scale up reforms further.

Access to Transport in Rural Malawi

What impact does access to public transport have on the lives of those living in remote rural areas? This study provided free access to a bus operating between a rural village and a regional trading center. This is the first study on the impact of access to transportation using a randomized control approach. Surveys measured the impact of access to transport on labor market, agricultural, health, and behavioral outcomes.

Policy Issue: 

Transportation and access to services is generally considered to be a fundamental determinant of economic growth and a significant factor in an individual’s health, schooling, and economic status. Isolation is thought to limit opportunities in rural areas due to restricted access to goods, higher costs to transport crops and agricultural inputs, and the lack of demand for non-agricultural services. As a result, there has been a large push in the development community to increase investment in rural roads. However, this approach assumes that road condition is the main determinant of transport costs and that the existence of a good quality road will automatically lead to the provision of public transportation and improved access to services. There is little evidence to date, however, on the exact economic and social benefits of the provision of rural roads.

 
Context of the Evaluation: 

Malawi has one of the least developed  transportation infrastructures in Sub-Saharan Africa. Of all the unpaved roads in the country, 48 percent were judged to be in poor condition in 2005 and one study of road conditions in Malawi found that 38 percent of villages were not accessible by motorized vehicles for the five-month rainy season . 

The study was implemented in a cluster of five villages in rural Malawi. Prior to the study, no regular  motorized passenger transportation existed on the road between the villages and the closest market town, which was 17 kilometers  away. Travel outside the village cluster was infrequent, and most transport  was by foot or bicycle. There was a weekly market  in the closest town as well as an agricultural input supplier and crop purchaser, a government health clinic, a post office, ten maize mills, and dozens of kiosks that sold small items and food. While there were no permanent banking services in the market town, a mobile banking service made weekly visits to the market town to collect deposits and facilitate withdrawals.

 
Details of the Intervention: 

In order to understand how roads in relatively good condition in rural areas impacts public transportation provision, a daily minibus service that connected five rural villages and the nearby market town was subsidized over a four-month period.

The intervention was divided into two phases. In the first phase, which was intended only to measure take-up of public transportation, 100  households were randomly selected to receive a bus pass that allowed the members of that selected household to use the bus at zero cost to travel to and from the market town. In the second phase, to measure the impact of price on take-up, 406  households were randomly assigned to one of seven  bus pass prices. After completion of a baseline survey, a meeting was called at each of the five villages. At the meeting, one member of each household was asked to select a ticket from a bucket to determine the price of one round trip on the project bus. Prices varied between zero and 500 Malawi Kwacha (US$3.57).  After the household member was assigned the lottery ticket, the number was recorded and the price was stamped on the household’s bus pass. Any member of the household could use the pass but had to pay the stamped price.

On all trips a conductor was present to check bus passes, collect bus fares, monitor cargo loading, and conduct rapid trip-purpose surveys of all passengers. Minibus owners and drivers near the market town were also interviewed in order to obtain a broader understanding of the market for transportation in the region.

 
Results and Policy Lessons: 

Minibus usage and impact of subsidy: More than 60 percent of households used the bus at least once with an average of 2.85 inbound  rides per household. However, usage decreased rapidly with price. When free, approximately 47 percent of adults used the bus. This number dropped to roughly 6 percent when the price of one round trip was 300 Kwacha . In other words, compared to users who rode the bus for free, those who had to pay 50 Kwacha were 18 percent less likely; and those who had to pay 500 Kwacha were 47 percent less likely.  

Policy implications: Based on take-up and interviews with minibus owners and drivers, the results suggest that at any price, a bus operator will never  make enough revenue to break even on this road. If road construction or rehabilitation does not generate transport provision, than the investment will not lead to the provision of services for the local population and will have been largely wasted. It is important to consider transport services provision when considering infrastructure upgrades in rural areas.

Information Campaigns and Voters' Behavior in the 2009 Municipal Elections in Mexico

Policy Issue:

It is widely held that access to information is a vital component of democracy building and government accountability. A recent World Bank report[1] champions information as “a tool to empower citizens in developing countries to hold their public agents accountable.” Information flows, the report argues, not only enhance democratic participation, but also make democracy work for ordinary people. However, while evidence suggests that access to information may lead voters to punish corrupt incumbents, it is unclear whether this translates into increased support for challengers and higher political participation. In other words, information about corruption may not improve political accountability, if voters respond to it by withdrawing from the political process.

Context:

Despite optimistic views about fiscal decentralization in Mexico, local governments’ performance has remained poor. In 2008, for example, more than 80 percent of municipal governments’ resources were spent either on the bureaucracy or were unaccounted for. While elections should enable voters to discipline their mayors, a single-term limit is imposed on all elected officials in Mexico. Thus, the immediate fate of mayors is determined not by voters but by their political party. To reconcile the single-term limit with accountability, scholars have typically assumed that voters punish or reward the incumbent party for mayoral performance. However, there is little evidence that government performance impacts the subsequent election – previous work shows a strong entrenchment of incumbents from all political parties.

Further impeding voters’ ability to hold mayors accountable are widespread misconceptions about which public works and services municipal authorities are responsible for providing, as well as a lack of available information about the amount of money municipalities receive and how this money is spent. In an attempt to ensure greater municipal accountability, a 1999 constitutional reform established the Federal Auditor's Office (ASF). On a yearly basis, the ASF selects a sample of municipalities in each state to audit. The results of the audits are published in lengthy reports, which are made available online. Though public, these reports are rarely used by media or political parties in local campaigns because the release date of the reports is not aligned with the timing of elections.

Description of Intervention:

Researchers sought to assess the effects of information dissemination on participation in the 2009 municipal and congressional elections in Mexico. Approximately one week before Election Day, flyers with different kinds of information on municipal spending were delivered to all households within the boundaries of treated voting precincts. The first group of precincts received information about municipalities' overall spending; the second group received information about distribution of resources to the poor; and the third group received information about irregular, unauthorized, or unaccounted for spending. All flyers included a subtle advocacy message suggesting that voters raise questions with their mayors about the use of public funds and prompted people to think about the level of governments that was in charge of the provision of public infrastructure services. In total, 150 electoral precincts were randomly assigned to each of the three interventions, for a total of 450 treated precincts and 1910 precincts in the comparison group.

Researchers gathered demographic characteristics from census data, and then collected electoral results for each precinct from the electoral institutes for each state. This information was complemented by a follow-up survey collected ten days after the election.

Results:

Information about overall spending: Disseminating information about overall spending levels had no statistical impact on voter turnout when the mayor spent less than 75 percent of available funds. However, it decreased the incumbent vote share by 0.6 percentage points. When mayors spent more than 75 percent of the funds, releasing this information let to an increase in turnout of 1.9 percentage points, but had no significant impact on vote share.

Information about spending on poor: Turnout among voters who received information about levels of spending allocated to poor areas increased by 2.4 percentage points, but only when mayors spent less than 75 percent of available funds on poor areas. Surprisingly, releasing this information had a similar impact on both incumbent and challengers' vote shares - an increase of 1.5 and 1 percentage points, respectively. When mayors allocated more than 75 percent of resources to poor areas, releasing the information had no impact on turnout or vote share.

Information about corruption: Information about high levels of corruption appears to have had a significant negative impact on voter turnover. Turnout among individuals that received information about corruption decreased by 1.10 percentage points, which represents a 2 percent decrease in turnout. While the effect of information about corruption on the incumbent vote share was insignificant, the effect on the challengers' vote share was negative. These results suggest that while flows of information are necessary, they may be insufficient to improve political accountability, since voters may respond to information by withdrawing from the political process.


[1] Khemani, Stuti. 2007.  Can Information Campaigns Overcome Political Obstacles to Serving the Poor? The Politics of Service Delivery in Democracies. http://siteresources.worldbank.org/DEC/Resources/Khemani_CanInformationCampaignsOvercome.pdf.

 

Effect of Media on Voting Behavior and Political Opinions in the United States

There is substantial evidence that media sources have identifiable political slants, but there has been relatively little rigorous study into the impact of media on political views and behaviors.  IPA designed a natural field experiment to measure the effect of exposure to newspapers on political behavior and opinion.

Washington DC is served by two major newspapers, the Washington Times and the Washington Post. We found that those who received the Washington Post were eight percentage points more likely to vote for the Democratic candidate for governor than those assigned to the control group. We find similar but weaker evidence of shifts in public opinion on specific issues and attitudes.

Policy Issue: 

Citizens learn about politics and government primarily from television and newspapers. These media outlets can influence voters not only through the slant of a particular report, but also merely by choosing which to stories to cover. Recent studies suggest that media exposure can have a sizable impact in shaping the public’s political knowledge, attitudes, and behavior. However, these studies may have overestimated the impacts of media influence due to individuals’ tendency to seek out information that agrees with their pre-existing views. 

 
Context of the Evaluation: 

Prince William County in northeastern Virginia lies just 25 miles from Washington D.C. Here, the population is far enough away from the nation’s capitol so as not to be dominated by citizens involved professionally with politics, but close enough to be within the circulation of Washington’s conservative and liberal newspapers. The Washington D.C. metro area is served by two major newspapers, the conservative Washington Times and the more liberal Washington Post. The presence of a liberal and conservative paper serving the same region creates an opportunity to study the effect of media slant in a natural setting within a single population, which is subject to the same outside factors, such as political events and outcomes, and has a range of political leanings.

 
Details of the Intervention: 

This study takes advantage of this natural setting to measure the effect of political news content on people’s political behavior and opinions. Approximately one month prior to the Virginia gubernatorial election in November 2005, researchers administered a short survey to a random selection of households in Prince William County. 

From the 3,347 households of registered voters who reported that they received neither the Post nor the Times, researchers randomly assigned households to receive a free subscription to one of the two papers for ten weeks, or to the comparison group that was not sent either paper. A week after the election, a follow-up survey was administered asking individuals whether they voted in the November 2005 election, which candidate they selected or preferred, their attitudes toward news events of the previous weeks, and their knowledge about recent news events. Voter turnout data was also collected for the November 2005 and 2006 elections from state administrative records. 

 

Results and Policy Lessons: 

Impact on Political Knowledge: Receiving either paper produced no effect on knowledge of political events or stated opinions about those events, and there were no differences between the treatment and comparison groups in voter turnout for the 2005 gubernatorial election.  In November 2006, however, there was a 2.8 percentage point increase in voter turnout. It is surprising to see a result in 2006 but not in 2005. This could be a result of the post-election exposure to the remainder of the ten-week newspaper subscriptions, or the fact that 17 percent of the treatment group renewed their subscription after the free period ended.

Impact on Political Preference: Interestingly, receiving either newspaper led to an increase of support for the Democratic candidate. Despite the political slant of the newspapers, the effects were similar for the Post and the Times, resulting in an overall 7.2 percentage point increase in likelihood of voting for the Democratic candidate. This may be due to the fact that the Republican President’s approval ratings were falling over that period of time, or perhaps the Democratic candidate was conservative leaning. In either case, these results suggest that the informational effect of more exposure to news was stronger than the effect of its slant. 

Direct Democracy and Local Public Goods in Indonesia

 

Policy Issue: 

Basic public goods such as dependable roads and clean water infrastructure are routinely underprovided in much of the developing world. While the provision of these services is often centrally administered, many now advocate for decentralization and community involvement as a more effective approach. Local communities can have better information on what goods and services are needed, and may thus be better positioned to recognize and quickly respond to inefficiency or corruption in implementation. Recent years have witnessed a trend toward decentralization in developing countries; this increase in local participation in government decision-making has been facilitated by a wide variety of political reforms, but the implications of these political mechanisms are yet to be well understood.  

 
Context of the Evaluation: 

Over 13 percent of the population of Indonesia lives below the poverty line,  and improving the infrastructure in marginalized areas is a priority for the Indonesian government.1 An Indonesian government program supported by a loan from the World Bank, the Kecamatan Development Program (KDP), funds projects in approximately 15,000 villages each year. Each village receives an average of Rp. 80 million (US$8,800) for infrastructure projects. Participating subdistricts, typically containing 10-20 villages, receive an annual block grant for three years. Each village makes two proposals – one on behalf of the whole village and one proposed by woman’s groups – for small-scale infrastructure projects.

Typically, when it comes time for a village to decide upon its two KDP proposals, representatives from various hamlets come together to discuss the merits of, and to decide on, the village’s two project proposals. A typical meeting would have between 9 and 15 people representing the various hamlets, as well as formal and informal village leaders, with on average about 48 people attending in total out of an average village population of 2,200. While the program has effectively improved local infrastructure in many of these villages, it is unclear whether current procedure makes projects easily dominated by elites and under-provides for those community members who need improved services the most.   

 

Details of the Intervention: 

To investigate these issues, researchers conducted a randomized evaluation in 49 villages, all of which were preparing to apply for infrastructure projects. Each village was randomized into one of two different political processes through which they determined which project to propose: 32 villages would follow the traditional representative meeting-based process described above, and the remaining 17 would choose their KDP project proposals via a direct election-based plebiscite. At these plebiscites, villagers could directly vote on a list of potential projects. For the general project, all adults were eligible to vote, and for the women-specific proposal, only women could vote. Data was collected on the project preferences of all villagers, including the elite, as well as the location and type of projects selected.

 
Results and Policy Lessons: 

Impact on Project Type: The direct election process had little effect on the types of projects selected amongst the general project proposals, but had substantial positive effects on measures of citizen satisfaction with the political process. However, the elections did not change the probability that the general project would be located in a poor area. Direct elections on women’s projects, on the other hand, resulted in both increased satisfaction with the political process and increased probability that projects were located in poorer areas of the villages. 

Impact on Satisfaction Measures: Direct elections resulted in substantial changes in the community’s satisfaction with the political process. Overall, the plebiscites resulted in an increase of 21 percentage points of people who said that the project chosen was either very much or somewhat in accordance with their wishes, an increase of 18 percentage points of people who said they would benefit either very much or somewhat from the project, an increase of 10 percentage points of people who said they would use the project personally, and an increase in overall satisfaction with KDP by 13 percentage points. The elections also raised the probability that individuals stated they would contribute something (such as labor or money) to the project by 17 percentage points. Additionally, villagers in treatment locations were 19 percentage points more likely to correctly identify the type and location of the general village project, and 25 percentage points more likely to know these things about the woman’s project.  

[1] CIA World Factbook, “Indonesia,” https://www.cia.gov/library/publications/the-world-factbook/geos/id.html (accessed August 31, 2009).

Combating Corruption in Community Development in Indonesia

 
Policy Issue: 

Corruption plagues many developing countries where the world’s poorest live, and combating it continues to be an arduous task. Corruption acts like a tax, adding to the cost of providing public services and conducting business; it also creates potentially severe efficiency consequences as well. Many suggest the right combination of monitoring and punishments can control corruption, but often the very individuals tasked with monitoring and enforcing punishments may themselves be corruptible. Another approach to reducing corruption is community-level monitoring. Local community members have the most to gain from a successful anti-corruption program, and are thus believed to have better incentives to monitor than bureaucrats. However, there is little empirical evidence on the success of such strategies.  

 
Context of the Evaluation: 

An Indonesian government program supported by a loan from the World Bank, the Kecamatan Development Project (KDP), funds projects in approximately 15,000 villages each year. Each village receives an average of Rp. 80 million, (US$8,800), which they often use to surface existing dirt roads. KDP-funded projects are large relative to ordinary local government activities; in 2001, the average annual village budget was only Rp. 71 million (US$7,800), so receiving a KDP project more than doubles average local government expenditures. This large amount of money creates incentives for price inflation, collusion with suppliers, kickback for village leaders, and manipulation of wage payments. 

Two checks on corruption are built into KDP. First, funds are paid to village implementation teams in three installments. To receive the second and third payments, the teams must make accountability reports at an open meeting where they account for how they spent the money; only after that meeting has approved the accountability report is the next installment released. Second, each project has approximately a 4 percent chance of being audited by the government.

 

Details of the Intervention: 

To examine the role of community monitoring and government audits on corruption, researchers conducted a randomized evaluation in 608 Indonesian villages in East Java and Central Java, Indonesia’s most populous provinces. Each village in the study was about to start building a village road with KDP funding. Some villages were randomly selected to be told, after funds had been awarded but before construction began, that their project would subsequently be audited by the central government, increasing the likelihood of an audit from 4 percent to 100 percent. These audits carry the possibility of criminal action if corruption is detected, and the results of the audits are read publicly to an open village meeting, potentially resulting in social sanctions. 

To investigate the impact of increasing community participation in the monitoring process, two interventions were established to enhance participation at accountability meetings. Some villages were selected to have invitations to these meetings distributed throughout the community, encouraging direct participation in the monitoring process and reducing elite dominance of the process. In the second experiment, an anonymous comment form was distributed along with the invitations, providing villagers an opportunity to relay information about the project to be shared at the meetings, without fear of retaliation. (See chart below)

Experimental Treatments by Group:

 

Group

 

Audit Probability

 

Accountability Meetings

 

Invitations

 

Comment Cards

 

Villages

 

  Audit

 

100%

 

x

 

 

 

 

 

93

 

  Participation I

 

4%

 

x

 

x

 

 

 

105

 

  Participation II

 

4%

 

x

 

x

 

x

 

106

 

  Audit & Participation I

 

100%

 

x

 

x

 

 

 

94

 

  Audit & Participation
  II

 

100%

 

x

 

x

 

x

 

96

 

  Comparison

 

4%

 

x

 

 

 

 

 

114

Corruption was measured by comparing the researcher’s estimate of what the project actually costs, determined by the quantity of materials used and estimate of material prices and wages paid on the project, to what the village reported it spent on the project on an item by item basis.

 

Results and Policy Lessons: 

The evidence suggests that increasing the probability of external audits substantially reduced missing funds in the project. Increasing the probability that a village was audited by the central government from 4 percent to 100 percent reduced missing expenditures by about eight percentage points, from 27.7 percentage points to 19.2 percentage points. One reason that the decrease was not larger is that a 100 percent audit probability does not imply that village officials face a 100 percent probability of detecting corruption and imposing a punishment. In fact, although auditors found violations of some type or another in 90 percent of the villages they visited, the vast majority of these violations were procedural in nature, and there were very few, if any, cases in which the auditors had enough concrete evidence to actually prosecute offenses. 

The invitations increased the number of people participating in the accountability meetings by an average of 14.8 people, or about 40 percent; slightly more than by including a comment form, since many villagers used the form as a substitute for attendance. Villages that received the invitations intervention were more likely to openly discuss corruption problems at the accountability meetings, and villages receiving both invitations and comment forms were more likely to take serious action at the meeting to resolve corruption-related problems. However, the magnitude of these changes in behavior at the meetings was small, and these treatments did not measurably reduce overall missing expenditures, yet they did have an effect on certain types of expenditures in some cases. For instance,  the invitations substantially reduced missing labor expenditures, and the comment forms did reduce missing expenditures in some cases, but only when they were distributed via schools, bypassing local officials. This study provides evidence that community participation, widely viewed as a panacea for development projects, impacts levels of corruption only under a limited set of circumstances, and pains must be taken to prevent elite capture for it to be an effective means of reducing corruption.

Child Protection Knowledge and Information Network (CPKIN)

Policy Issue:

Children in post-conflict environments are a population of particular concern, exposed to violence, displacement, and death of family members. The impact of poverty means that child labor is common among children. Though no reliable data exists, indications are strong that sexual abuse and exploitation is a fact of life for many girls and teenage pregnancy has been identified as a child protection concern. In addition to the physical and mental health challenges posed by war experiences, these children remain susceptible to continued exploitation and abuse, often by those familiar to the child, during peace as refugees return home and community members cope with difficult pasts. 

Context of the Evaluation:

The situation of children in Sierra Leone continues to be precarious as 27% of the 2.7 million children are identified as vulnerable, lacking the protection of a primary care giver. Formal structures for child protection exist within police stations and local Child Welfare Committees (CWCs), but capacity and community trust in such institutions’ efficacy are limited in many rural areas[1]. To reduce violence against children, existing social structures,most paramount, section, and village chiefs, are frequently leveraged to respond to reports of abuse and to mobilize local protective factors.

To respond to the resource constraints faced by formal child protection systems in Sierra Leone, the Ministry of Social Welfare, Gender, and Children’s’ Affairs (MSWGCA) is experimenting with a policy framework that explicitly links local chiefs and community members to the formal child protection sector.  This policy was articulated in a Memorandum of Understanding (MOU) signed in 2010 by the MSWGCA, CWCs, and the Council of Paramount Chiefs. The MOU and focal person system are currently being implemented in Moyamba and Pujehun districts, with potential to expand the program nationwide.  The MOU creates a framework through which chiefs, MSWGCA, CWCs, and the Family Support Unit (FSU) of the Sierra Leone Police can collaborate to protect children. Village chiefs appoint a focal person for child welfare in a public meeting of village residents. Focal persons report cases of child abuse to chiefs and chiefs, with the assistance of focal persons, follow up with the formal child protection actors to ensure that cases of child abuse are addressed appropriately.

To help focal persons connect informal systems for child protection to the formal roles of the FSU and MSWGCA, UNICEF and other local child protection NGOs have developed a training program that introduces focal persons to general approaches to child protection, as well as to their roles and responsibilities under the MOU.  Despite the grounding of these trainings in both international best practices and local norms and understandings, key stakeholders have expressed concerns about whether one-time trainings will provide focal points with sufficient capacity to effectively serve as a link between communities, chiefs, and formal child protection systems.  At the same time, repeated trainings or sustained in-depth monitoring by international or national child protection agencies is not sustainable in the context of rural villages in Sierra Leone.

Details of the Intervention:

Mobile phones and SMS messaging provide one potential avenue for bridging the gap between the need for sustained support of focal points and the high costs of transportation in rural Sierra Leone.  To leverage the growing use of this technology in Sierra Leone, a consortium of non-profit, academic, government, and private-sector stakeholders have collaborated in developing the Child Protection Knowledge and Information Network (CPKIN).  CPKIN (pronounced as “See Pikin” or “See the Child” in Krio, a language spoken throughout the country) is a system that will be used to send focal persons automated text messages asking questions about the welfare of children in their village and prompting them to engage in discussions in their community to help answer these questions.  Focal persons will then be encouraged to send their answers back to the central CPKIN system using a free text message.  These text message questions and responses will be sent and received through a software program that makes it possible to manage, organize, and analyze high volumes of outgoing and incoming messages with a large list of recipients

There are several hypotheses underpinning the design of the CPKIN program. The first hypothesis is that the process of receiving questions, discussing the questions with community members, and sending answers will encourage focal persons to critically examine the situation of children in their communities and then to act with the resources available in their villages. A second hypothesis is that having a focal person who is active with respect to reporting abuses, taking proactive steps to improve the welfare of children, and engaging community members in discussions regarding children has the potential to cause dramatic shifts in community level knowledge, norms, and practices regarding child protection, which in turn may lead to greater connections between informal and formal child protection systems.

Given the novelty of the CPKIN system as a child protection intervention, it is necessary to assess the extent to which sending regular, open-ended text messages to village focal persons using this system can increase the capacity of focal points and their communities to proactively and reactively respond to local child protection issues. In particular, the effectiveness of CPKIN and the validity of the underlying hypotheses will be evaluated using a randomized evaluation in 140 villages in Moyamba and Pujehun districts. The randomization will occur at the village level, with the 70 villages randomly assigned to the CPKIN treatment, and the remaining 70 villages comprising the comparison group, in which there is no intervention. In treatment villages, the village focal person will receive a one-on-one training on the overall aims of CPKIN, how to receive and send CPKIN messages, and how to use CPKIN as a starting point for engaging their community members on issues concerning child protection.  Shortly after this training, focal persons will begin receiving regular CPKIN prompts.

Baseline and endline surveys regarding child protection and child welfare will be conducted before and after the six to eight month intervention in all 140 villages in order to assess the relative effectiveness of the CPKIN support system, with survey questions designed to elicit two kinds of information:

1.)    The capacity of key stakeholders (children, parents, focal persons, chiefs, FSU, and CWCs—where they exist) to identify and respond to child protection issues.

2.)    The protection and support for children within their communities, and their overall level of welfare (including the prevalence of both abuses and protective factors).

Results and Policy Lessons:

Results forthcoming.



[1] Human Rights Watch. (2011). World Report 2011

 

Property Tax Experiment in Punjab, Pakistan: Testing the Role of Wages, Incentives and Audit on Tax Inspectors' Behavior

Policy Issue

Revenue collection and public sector efficiency is a central question for developing countries. The low level of tax revenues raised in these countries can result in the under-provision of public goods, heightens vulnerability to economic crises, and may constrain growth. Poorer countries collect on average only two-thirds  or less of the amount of tax revenue as a fraction of GDP that richer countries do1, with an estimated $285  billion per year loss due to tax evasion. Past work on this topic attributes the weak performance of tax collection to the poor incentives for proper tax collection and administration. It is thought that incentives, accountability and monitoring can raise public sector efficiency, but there is little rigorous evidence on anti-corruption measures that systematically address the incentives faced by government bureaucrats.

Context of the Evaluation

Punjab is Pakistan’s most populous province with a population of over 80 million. International comparison reveals that the present level of property tax collection in Punjab is roughly a fifth of the level  of comparable countries. Evidence suggests that the main way tax evasion takes place is through several distortions such as granting exemptions to widows, the disabled, owners of small plots, retired federal and provincial government employees, and religious charitable institutions. Because officials may employ significant discretion in applying valuation to individual properties and determining exemptions, the system leaves considerable opportunities for leakages, collusion, and low collection.

Tax collectors in Pakistan are part of the provincial career bureaucracy with wages determined by salary band and length of service. Tax officials have few avenues for vertical mobility through promotions, and wage levels are not tied to performance in any way. As a result of these factors, tax officials suffer from low motivation, and evidence suggests that tax evasion and rent seeking are prevalent. 

Details of the Intervention

Geographical areas serviced by a set of tax collectors (called tax circles) were randomly assigned one of the following four wage and incentive programs (or no treatment):
Wage schemes:

Pure Wage Increase: the base salary of tax inspectors, clerks, and constables will be increased by a fixed amount. The usual monitoring and control systems from the Excise and Taxation Department (which also apply to the control group) remained in place in this and all other treatments.

Pure Wage Increase Plus Audit: In addition to the increase in wages, tax officials were told that a random sub-sample of the properties under their jurisdiction would be visited by an independent government unit outside the tax department and audited. Officials were told that they would be rewarded or penalized based on the accuracy of their audit relative to audits in other circles, as well as on taxpayer satisfaction with the quality of interactions with tax officials. The reward / penalty will be in the form of job assignments: the best performing circles would be given the highest priority in postings in the subsequent year, and the worst circles will be given lowest priority in reassignment in the subsequent year and denied the wage increase.

Output-Based Incentives: In this scheme, tax officers were rewarded on the basis of revenue collected. Specifically, a percentage of revenue collected in the circle above a historical benchmark will be given to the tax officers as performance honorarium. 

Output-Based Incentives Plus Audit: In this scheme, the output incentives were combines with the audits, with the penalty that if the audit performance is good/poor the circle officials are given highest/lowest priority in reassignment to tax circles and forfeit future incentive payments.

Results and Policy Lessons

Results forthcoming.

Primary Education Management in Madagascar

Policy Issue: 

Successful efforts to expand access to education in the developing world have not always translated into actual improvements in skills and learning for students. It remains an open debate as to whether top-down approaches are more effective in improving educational quality than approaches which promote beneficiary participation, such as parental monitoring. Top-down approaches can provide administrators with the tools necessary to better monitor their schools, but this assumes that they have the incentives to do so. Bureaucrats may have more incentives to improve the quantity, rather than quality, of education services since the benefits of improving quality are diffuse and harder to verify. The combined effects of these perverse incentives can result in a large number of children who are in the classroom, but are not learning. Promoting local accountability may be a useful means of improving schooling outcomes, if it can be determined which factors makes beneficiary participation effective at improving education delivery.

Context of the Evaluation: 

Madagascar divides its 2.7 million children into 15,000 public primary schools. Despite the significant increase in primary school enrollment following Madagascar's 2002 reforms and an influx of international financial support, resource allocation across schools remains inefficient, and better resource endowments rarely translate into better student performance. Only 63% of grade 5 children pass the primary-cycle exam, an assessment of the minimum level language and math knowledge presumed at this grade. District administrators face a performance review only every 3 years, and the subdistrict heads rarely face any credible threat of penalties or firing.

Details of the Intervention: 

Researchers, in collaboration with The Ministry of Education in Madagascar, ran a randomized experiment in 3,774 primary schools in 30 public school districts. These districts represented all geographic areas in the country, but were focused on schools with the higher rates of grade repetition.

All district administrators in treatment districts received operational tools and training that included forms for supervision visits to schools, and procurement sheets for school supplies and grants (district-level intervention). In some of these schools, the subdistrict head was also trained and provided with tools to supervise school visits, as well as information on the performance and resource level at each school (subdistrict-level intervention).

Lastly, several districts also introduced a school level intervention which involved parental monitoring through school meetings. Field workers distributed a ‘report card’ to schools, which included the previous year’s dropout rate, exam pass rate, and repetition rate. Two community meetings were then held, and the first meeting resulted in an action plan based on the report card. One example of the goals specified in the action plans was to increase the school exam pass rate by 5 percentage points by the end of the academic year. Common tasks specified for teachers included lesson planning and student evaluation every few weeks. The parent’s association was expected to monitor the student evaluation reports which the teachers were supposed to communicate to them. These tools allowed parents to coordinate on taking actions to monitor service quality and exercise social pressure on the teachers.

Results and Policy Lessons: 

Impact from Top-Down Approach: The interventions targeted at the district and subdistrict level had minimal effects on the administrator’s behaviors, and the schools and students under their responsibility. Though each tool – forms for supervision visits to schools and procurement sheets for school supplies and grants – was used by 90% of subdistrict heads and more than 50% of district heads, subdistrict heads visited their schools only slightly more often than those in the comparison group, an insignificant improvement. Teachers in both groups did not plan for lessons more, and no improvement in test scores was seen in the two years following the program.

Impact from Bottom-Up Approach: The interventions at the school level led to significantly improved teacher behavior. Teachers were on average 0.26 standard deviations more likely to create daily and weekly lesson plans and to have discussed them with their director. Test scores were 0.1 standard deviations higher than those in the comparison group two years after the implementation of the program. Additionally, student attendance increased by 4.3 percentage points compared to the comparison group average of 87%, though teacher attendance and communication with parents did not improve.

 

Testing the Effectiveness of Payments for Ecosystem Services to Enhance Conservation in Uganda

Policy Issue:

Deforestation contributes as much as 25 percent  of global greenhouse-gas emissions each year. Curbing deforestation in developing countries is potentially a very cost-effective way to reduce carbon emissions and address climate change. Recently, the United Nations launched a major initiative to pay developing countries for reduced deforestation. The program, known as REDD+ (Reducing Emissions from Deforestation and Degradation), may be incorporated into global carbon markets under the next international climate treaty, resulting in billions of dollars in payments from wealthier countries for forest conservation. However, despite growing interest and investment in reducing deforestation, surprisingly little research has been conducted on the most cost-effective ways to do so. One popular policy approach is payments for ecosystem services (PES), where participants receive payments if they comply with a set of conditions that are protective of the environment, such as refraining from cutting down trees on their land. PES programs are increasingly popular because of their perceived simplicity in comparison to alternative conservation interventions. It is important to evaluate this and other types of emission reduction interventions in order to determine the most cost-effective way of reducing carbon emissions.

Context of the Intervention:

Forest loss in Uganda is estimated to be about 2 percent per year, with an even higher rate on private land.[i] The project is based in the districts of Hoima and Kibaale, which are located in the equatorial zone of western Uganda, and have some of the highest deforestation rates in the country. These districts are predominantly rural, with a population density of about 97 persons/km2 and a combined population of around 750,000 inhabitants.  

While the environmental effects of deforestation are not limited to a particular portion of the population, the PES project is specifically targeted at private landowners who have forests on their land since they are the population with ownership rights over local forests and can decide whether or not to clear trees from their land. Forest owners might cut trees to clear land for growing cash crops such as tobacco and rice or to sell the trees as timber or for charcoal production. Cutting down trees provides the landowners with income, but is a threat to carbon storage as well as the survival of local wildlife (particularly the endangered chimpanzee population).

Description of the Evaluation:

One hundred forty villages in the Hoima and Kibaale districts of Uganda were randomly assigned to either the treatment or the comparison group. In the treatment villages, Chimpanzee Sanctuary and Wildlife Conservation Trust (CSWCT) staff members offered an incentive contract to each individual landowner, under which they will receive annual payments if they meet certain terms. Landowners are required to refrain from cutting trees on their land (with some exceptions built into the contract) and also to re-forest a portion of the land.  CSWCT employees monitor compliance with the contract by conducting random spot checks in the forest to look for things like newly cleared patched or fresh tree stumps.

The study will measure social and economic welfare as well as environmental outcomes, and will be linked with satellite imagery that assesses spatial patterns of forests.

Results and Policy Lessons:

Results forthcoming.



[i] National Environment Management Authority (NEMA) Uganda. “National State of Environment Report for Uganda 2007/2008”

Decentralization: A Cautionary Tale - Public Finance in Kenya

Kenya’s education system blends substantial centralization with elements of local control and school choice.  This project looks at the system of incentives created by elements of decentralization.

Syndicate content
Empowered by:
Copyright 2014 Innovations for Poverty Action. All rights reserved.