The Hunger Project's "Epicenter" Scale up Strategy in Ghana

One of the greatest tragedies of extreme poverty is its intergenerational transmission. Poor, malnourished children do not develop normally, physically or cognitively; poor adolescents are unable to take proper advantage of educational opportunities and are more likely to engage in risky behaviors.  After generations of poverty, many families and communities cannot envision cooperation to break this cycle. The Hunger Project (THP) works towards tackling poverty in Africa by partnering with local people to establish community centers ("epicenters") offering a comprehensive range of services, from health and education, to agriculture, microfinance, water and sanitation, as well as fostering community spirit. This randomized evaluation of THP's multi-faceted program aims to assess the impact on the various outcomes it strives to improve.

Researchers from Yale University and IPA have partnered with THP to evaluate the long term-impact of this strategy on health, nutrition, income, gender roles, social cohesion and education. The Hunger Project plans to cover the entire Eastern Region of Ghana, however it is neither feasible nor desirable to build all 112 centers at once. A lottery is conducted within each district to determine which of the 112 communities are offered a center in the first years (treatment group).  Communities that do not win the lottery for early invitation, the comparison group, may receive an invitation a few years later.  A pre-intervention baseline survey of approximately 4000 households with over 20,000 individuals was completed in 2008 and a follow-up survey of the same households will be launched in early 2013.  The longitudinal nature of the survey allows us to examine if the effects of the centers are sustained over time and whether or not the strategy is financially sustainable. Generally, these centers aim to be economically sustainable within 5 years.

Primary School Deworming in Kenya

Hundreds of millions of children worldwide are infected with parasitic worms. These worms are detrimental to children's health, their cognitive development, their education and their futures. Chronic illness caused by worm infections reduces literacy and adult productivity. 

Free deworming treatment substantially improved student attendance and health. The program also had significant "spillover" effects, improving health outcomes and attendance among students in neighboring primary schools. 

Including the spillover benefits of treatment, the cost of keeping a child in school one additional day is only US$0.02, which makes deworming considerably less expensive than any alternative method of increasing primary school participation.

Given the great success of this project, IPA is now working to Scale Up school-based deworming in partnership with Deworm the World.

<--break->Policy Issue

Intestinal helminths—including hookworm, roundworm, schistosomiasis and whipworm—infect more than one in four people worldwide and are particularly prevalent among school-aged children in developing countries. These intestinal worms are believed to have a negative impact on education, hindering child development as well as school attendance and reducing income later in life. These effects are especially pronounced in Africa, where nearly half of the total disease burden is due to infectious and parasitic diseases, including helminth infections. Existing randomized studies have focused primarily on the effects that these diseases have on cognitive performance, whereas outcomes of more direct interest to economists and policymakers—school attendance and enrollment, test scores, and ultimately, labor market outcomes—have yet to be thoroughly investigated.

Context of the Evaluation

Busia district is a poor and densely-settled farming region in western Kenya adjacent to Lake Victoria. Budalangi and Funyula divisions have some of the country’s highest helminth infection rates, in part due to the area’s proximity to Lake Victoria—schistosomiasis is easily contracted through contact with the contaminated lake water. Soil-transmitted helminths (STH), on the other hand, are transmitted through contact with or ingestion of fecal matter. This can occur, for example, if children do not have access to a latrine and instead defecate in the fields near their home or school, where they also play. One quarter of Kenyan student absenteeism is attributed to abdominal pains which likely due to intestinal helminth infections. In addition, older children may miss school to take care of siblings who are sick with helminth infections.

Details of the Intervention

This study evaluated the Primary School Deworming Project (PSDP), which was carried out by International Child Support in cooperation with the Busia District Ministry of Health. The program randomly divided 75 schools into three equal groups which were phased into treatment over three years.

Within each group, a baseline parasitological survey was administered to a random sample of pupils. Schools with worm prevalence over 50% were mass treated with deworming drugs every six months. Girls of reproductive age (thirteen and older) were not supposed to be treated due to concerns about the possibility of birth defects. Nonetheless, 19% of girls thirteen and older also received medical treatment, partly due to confusion about pupil age, and partly because several Kenyan public health nurses administered drugs to some older girls, judging the benefits to outweigh the risks. In addition to medicine, treatment schools received regular public health lectures, wall charts on worm prevention, and training for one designated teacher. The lectures and teacher training provided information on worm prevention behaviors—including washing hands before meals, wearing shoes and not swimming in the lake.

Results and Policy Lessons:

Impact on Infection Intensity: Deworming reduced serious worm infections by half amongst children in the treatment groups. Pupils that received treatment reported being sick significantly less often, had lower rates of severe anemia, and showed substantial height gains, averaging 0.5 centimeters.

Impact on School Attendance: Deworming increased school participation by at least 7 percentage points, which equates to a one-quarter reduction in school absenteeism. When younger children were dewormed, they attended school 15 more days per year, while older children attended approximately 10 more school days per year. The larger impact of treatment in lower grades may partially result from higher rates of infection among younger pupils.

Treatment Spillover: The entire community and those living up to 6 kilometers away from treatment schools benefited from “spillovers” of the deworming treatment. Spillover effects occur because medical treatment reduces the transmission of infections to other community members. Reductions in infection in non-treated children resulted in an additional 3 to 4 days of schooling per year. Although data was not collected on adults, it is also likely that older community members were able to work more days as a result of spillover effects.

No improvements in test scores were found as a result of the deworming. Additionally, evidence suggests that health education had a minimal impact on behavior, so that to the extent that the program improved health, it almost certainly did so through the effect of the medicines rather than through health education. Including the spillover benefits of treatment, the cost per additional year of school participation is US$3.27, considerably less than the cost of many alternative methods of increasing primary school participation.

 

Conditional Cash Transfers for Education in Morocco


Conditional Cash Transfers (CCTs) have been shown to increase investments in education and health, but their standard features make them expensive. Researchers conducted a randomized evaluation of a cash transfer program in Morocco to estimate the impact on attendance and enrollment of a “labeled cash transfer” (LCT): a small cash transfer made to parents of school-aged children in poor rural communities, not conditional on school attendance but explicitly labeled as an education support program. The program led to large gains in school participation and adding conditionality and giving cash transfers to mothers rather than fathers made almost no difference.
 
Policy Issue: 
Around the world, conditional cash transfer (CCTs) programs have been shown to influence households’ investment decisions regarding education and increase school attendance and educational attainment. One potential drawback of CCTs, however, is that two of their standard features, targeting—the process of defining and identifying eligible recipients—and conditionality—the requirement that recipients take certain actions, such as sending their children to school in order to receive the transfer—make them expensive to administer. Moreover, targeting and conditionality can also lead to the exclusion of the poorest segments of the population; targeting systems have been shown to be imperfect, often excluding many poor households; and conditionality means that those who cannot meet the conditions do not receive any benefit. While there is evidence that conditionality increases attendance, it is unclear how stringent the conditions must be. Can a cash transfer that encourages school enrollment without an explicit attendance requirement provide the small nudge necessary to increase educational achievements among disadvantaged communities?
 
Context of the Evaluation: 
Morocco is a lower middle income country: it’s GDP per capita was about $3,000 in 2011. Education levels in the general population are relatively low and in 2011only about 56 percent of the adult population was literate. While the national primary school enrollment rate is high, the rural primary school completion rate was still below 60 percent in 2008. In order to address these issues, the Moroccan Higher Council of Education (CSE) together with the National Ministry of Education (MEN) launched a nation-wide cash transfer program to encourage parents to keep their children enrolled in school.
 
Details of the Intervention: 
Researchers partnered with the Government of Morocco to evaluate the Tayssir program, a two-year pilot designed to increase student participation in primary school. Tayssir, which means “facilitation” in Arabic, made cash payments to parents in program communities with children aged 6 to 15. Parents had to formally enroll each of their children into the program. The pilot took place in 320 rural primary school sectors in the poorest areas within five of Morocco’s sixteen regions. The Tayssir pilot included two versions of the program:
 
Labeled cash transfer (LCT):  In this version of the program, families with children of primary school age could receive transfers whether or not their children attended school. In practice, since enrollment in the Tayssir program happened at schools, children enrolled in Tayssir were automatically registered and enrolled in school at the same time, but the transfers were not conditional on continued enrollment. The monthly amount per child increased as each child progressed through school, starting from US$8 for each child in grades 1 and 2 and increasing to US$13 for children in grades 5 and 6. The average transfer amount represented about 5 percent of the average household’s monthly consumption, which is small compared to a range of 6 to 27 percent for existing CCTs in middle-income countries.
 
Conditional cash transfer (CCT): In this version of the program, cash transfers were disbursed to parents of primary school-age children, as long as their child did not miss school more than four times each month.  The monthly transfer amounts were the same as those in the LCT program.
 
What’s more, in order to determine if the effectiveness of the transfers depends on the gender of the parent who received the transfer (the child's mother or father), in half of the school sectors sampled for Tayssir, mothers (the recipients in almost all CCT programs to date) received the transfers; while fathers received the transfers in the other half.
 
Thus, each school sector sampled for the study was randomly assigned to one of five groups:
  • LCT to father group (40 school sectors, 80 communities sampled for study)
  • LCT to mother group (40 school sectors, 80 communities sampled for study)
  • CCT to father group (90 school sectors, 180 communities sampled for study)
  • CCT to mother group (90 school sectors, 180 communities sampled for study)
  • Comparison group: no transfers (60 school sectors, 120 communities sampled for study)
Researchers collected information on student attendance and enrollment status for over 47,000 primary school aged children through unannounced visits to all schools. Comprehensive baseline and endline surveys gathered data on around 4,400 households. At endline a basic test was administered to measure arithmetic performance of one child per household. Finally, awareness surveys were administered mid-course to measure teachers’ and parents understanding of the program. 
 
Results & Policy Lessons:
Geographical targeting, by removing any ambiguity on eligibility, allowed for large program take-up. Around 97% of the study households in either the LCT or CCT communities had at least one child enrolled. Compliance with the random assignment of the gender of the transfer recipient was very high: it was close to 89% on average in schools where mothers had been designated as recipients, and around 80% in schools where fathers had been.
 
In terms of school participation, the Tayssir cash transfers had large impacts under all versions of the program, with slightly larger impacts for the LCT. After two years, the dropout rate among students enrolled in school at the start of the program in LCT schools was about 7 percentage points lower than the dropout rate of comparison schools (at 10 percent), a 70 percent decrease. Re-enrollment of those who had dropped out of school before the program almost doubled in LCT schools as compared to comparison schools, and the share of students who never enrolled in school fell by 43 percent. Performance on a basic arithmetic test improved but not significantly.
 
There was no difference in impacts between transfers issued to fathers and transfers issued to mothers. Making cash transfers conditional did not improve the effectiveness of the program either. If anything, it somewhat reduced it: relative to LCT schools, CCT schools had a slightly higher drop-out rate. Among students who were not enrolled at the start of the program, re-enrollment in CCT schools was lower than re-enrollment in LCT schools, perhaps because conditionality discouraged some households or teachers from enrolling weaker children in the program. That said, the rules of the program were overall poorly understood by local communities. After one year, in the CCT groups, only about half of parents interviewed knew transfers were conditional; in the LCT groups, only about half of parents interviewed knew transfers were unconditional. A weak understanding of what households must do to qualify for the cash transfer naturally weakens the potential for conditionality to matter.
 
The finding that adding conditionality or directing the cash transfers to mothers did not increase the program’s impact on student attendance or school enrollment is important. It likely is because the Tayssir program was framed as an educational support program, and perceived as an endorsement of the local schools, since headmasters were responsible for enrolling families. Data from the household surveys provides evidence that Tayssir, in all forms, increased parents’ belief that education was a worthwhile investment.
 
Overall, the results of the Tayssir experiment suggest that in some contexts unconditional but labeled transfers targeted at poor communities can provide parents with the small nudge necessary to increase attendance. This “nudge” is relatively cheap, due to both small transfers per child and small administrative costs. 

Girls Scholarship Program in Kenya

Approximately 85% of primary school age children in western Kenya are enrolled, however only about one-third of students finish primary school. Dropout rates are typically higher for girls. Results suggest that the Girls Scholarship Program led to persistent test score gains in pupils from treatment schools five years after the program. Girls from treatment schools were also more likely to be enrolled in school and to have attended some secondary school at the time of the long-term follow up survey.

 
Policy Issue: 

In many education systems, those who perform well on exams covering the material of one level of education receive free or subsidized access to the next level of education. Such merit-based scholarships are attractive to the extent that they can induce greater student effort, assuming that pupils are motivated to strive for scholarship opportunities. However, the role of student motivation in improving education outcomes is relatively poorly understood. Policymakers have frequently focused their attention on increasing school inputs or improving teacher attendance, assuming that students are motivated to take advantage of these improvements. Merit-based scholarships for girls may offer an alternative to increase female education, and more educated women tend to have healthier children and higher incomes. However, the assumption that pupils are inherently motivated to pursue education, and the effect that educational opportunities can have on female learning, are relatively unexplored.

 
Context of the Evaluation: 

Approximately 85 percent of primary school age children in western Kenya are enrolled in school, but only about one-third of students finish primary school. Dropout rates are typically higher for girls; in 2001 the 6th grade dropout rate was 10 percent for girls and 7 percent for boys among students in this study’s comparison schools at baseline. Primary schools charge fees to cover their non-teacher costs, including textbooks for teachers, chalk, and classroom maintenance (approximately US$6.40 per family per year). There are also additional fees for school supplies, textbooks, uniforms, and activities such as taking exams, and these costs may deter parents from sending children, especially daughters, to school. This project was introduced in part to assist families of high-achieving girls to cover these costs.

 
Details of the Intervention: 

The Girls’ Scholarship Program (GSP) was carried out by International Child Support (ICS) Africa, in two rural Kenyan districts, Busia and Teso. Out of a set of 127 schools, 64 were randomly invited to participate in a program which gave merit-based scholarships to 6th grade girls who scored in the top 15 percent on tests administered by the Kenyan government. For the next two years, winning girls received: (1) a grant of US$6.40 to cover school fees, paid to her school; (2) a grant of US$12.80 for school supplies paid directly to her family; and (3) public recognition at a school awards assembly held for students, parents, teachers and local government officials.

Academic achievement was captured in test scores, which are likely to be a good objective measure, and was not significantly affected by cheating. Exams in Kenya are administered by outside monitors, and district records from those monitors have no documentation of cheating. 

 

Results and Policy Lessons: 

Implementation: Poor existing attitudes towards outside intervention and an educationally disadvantaged population meant that some schools in the Teso district were resistant to the program. Particularly, stronger indigenous religious beliefs and a tradition of suspicion of outsiders caused implementation difficulties, which may have reduced program effectiveness there.

Test Score Effects: The program raised test scores by 0.19 standard deviations for girls enrolled in schools eligible for the scholarship. These effects were strongest among students in Busia, where the program increased scores by 0.27 standard deviations. There are no effects found in Teso. Large positive test score gains were also found among Busia girls with low chances of winning the award, suggesting that there were positive externalities on learning. The average program effect for girls corresponds to an additional 0.2 grades worth of primary school learning, and these gains persisted one full year following the competition. There is also evidence of positive program externalities on the entire class; boys (who were ineligible for the awards) saw scores increase by 0.08 standard deviations on average. 

Student Attendance: While the program impact on school participation is nearly zero among girls in the pooled Busia and Teso sample, the impact in Busia is positive at 3.2 percentage points. This corresponds to about a one-quarter reduction in school absenteeism.

Teacher Attendance: The program had a large impact on overall teacher attendance; in the pooled Busia and Teso sample there was a 4.8 percentage point increase in overall teacher attendance, and if these attendance gains were concentrated only among 6th grade teachers then this would imply a 7.6 percentage point increase in attendance. Once again, effects were larger in Busia—the impact on overall teacher attendance there was 7.0 percentage points, roughly halving overall teacher absenteeism. Teachers could potentially be gaming the system by diverting their effort towards students eligible for the program, but there is no difference in how often girls are called on in class relative to boys in the program versus comparison schools, indicating that program school teachers probably did not substantially divert attention to girls. This finding suggests that greater teaching effort was directed to the class as a whole.

Merit Scholarships and Inequality: The scholarship award winners did tend to come from relatively advantaged households, raising concerns about the distribution of benefits from this program. But in terms of student test score performance, the positive externalities affected all students, and were not concentrated amongst the most privileged. 

Parental Involvement Effects: Anecdotal evidence from teacher interviews suggests greater parental monitoring occurred in Busia as a result of the program. One Busia teacher mentioned that parents began to “ask teachers to work hard so that [their daughters] can win more scholarships.” Another Busia teacher asserted that parents visited the school more frequently to check up on teachers, and to “encourage the pupils to put in more efforts.” When teachers were asked to rate local parental support for the program, 90 percent of Busia teachers claimed that parents were either “very” or “somewhat positive;” in Teso the analogous rate was only 58 percent. Thus, the greater improvements in both student and teacher attendance and performance in Busia as compared to Teso suggest that merit scholarships are most effective in the presence of local parental accountability and involvement, either formal or informal. 

Information and Community Mobilization in Rural India

Policy Issue: 
While primary school enrollment rates have risen sharply in much of the world, the quality of education remains low in many countries. Many children who attend school regularly are still unable to read or do basic arithmetic. For instance, a 2008 survey found that in rural India only 56 percent of children in grade 5 could read at the grade 2 level, and nearly 20 percent could not read beyond a single word. Community oversight and participation has been advocated to increase education quality. Does this strategy work, and if so how can community participation be encouraged? Is more direct action by communities to teach their children to read more effective?
 
Context of the Evaluation: 
In Uttar Pradesh, India’s most populous state, only 43.5 percent of children in grade 5 can read at the grade 2 level. In response to this problem, the government established Village Education Committees (VECs) in every village in 2001. VECs consist of the elected head of the village government, the head teacher of the local school, and three parents who are nominated by their community. These committees are responsible for monitoring school performance, allocating school resources, and hiring additional contract teachers in the event of overcrowding. 
 
Despite the promising aspects of this program, a survey conducted in 2005 indicated that 38 percent of VEC members did not readily identify as being part of the committee, and 25 percent did not even know they had this role. Only 3.6 percent of all VEC members knew they had the ability to request funds to hire additional teachers, which is one of the main prerogatives and responsibilities of the VEC. 
 
Details of the Intervention: 
Working jointly with Pratham (a local NGO) and the World Bank, researchers designed three interventions that were randomly assigned in 280 villages in four rural blocks in Jaunpur district, eastern Uttar Pradesh, a populous and educationally struggling area in India. These interventions served to determine if more information and encouragement to use the channels available to them would cause VECs and community members to demand and receive better services. They contrasted this with direct action to improve learning outside the official channels. 
 
Intervention 1: In 65 villages, Pratham staff started a series of conversations about education in small groups throughout the community. These conversations covered the current status of schools in the village, the quality of local schools, state mandated provisions for schools, mid-day meals, and local funds available for education. People were asked if they knew about the VEC and its membership and responsibilities. After two days of meetings in small groups, a community-wide meeting was held where people were encouraged to discuss and ask for information about the VEC, with information gaps filled in by Pratham’s field workers. VEC members also received a pamphlet on their roles and responsibilities from the Pratham staff.
 
Intervention 2: In addition to all the steps outlined above, communities in another 65 villages were trained and encouraged to conduct testing to see if children could read simple text and solve basic arithmetic problems. Volunteers put together a "report card" for each community, which was presented at the community-wide meeting. 
 
Intervention 3: In addition to the above two steps, Pratham officers taught volunteers in another 65 villages a simple technique for helping children learn to read. Volunteers were encouraged to start after-school reading classes-- they were invited to attend training sessions which lasted for four days, and staff returned an average of seven times to provide in-service training. The objective was to use Pratham-designed materials and local volunteers to supplement the normal curriculum, and improve literacy among village children. 
 
Comparison: Eighty-five villages received no treatment, serving as a comparison.
 
Results and Policy Lessons: 
 
Impact on Information Gaps: The average effect of all three treatments was an increase of 7.8 percentage points in VEC members who knew they could access public funds, and a 13 percentage point increase of members who had been properly trained. Parents were also 2.9 percentage points more likely to know that a VEC existed in their community.
 
Impact on Engagement: Despite these improvements in awareness, there was little difference between the VECs’ performance in treatment and comparison villages. The only significant difference was that 20 percent more contract teachers were hired in Intervention 2 villages (although not in Intervention 3 villages). Also, the intervention did not increase the level of engagement of parents with schools. Parents were no more likely to have visited the school or to have volunteered time or money in the treatment villages than in the comparison villages.
 
Impact on Reading: In 55 of the 65 Intervention 3 villages, volunteers ran more than 400 reading courses. The average child in an Intervention 3 village who could not read anything at baseline was 7.9 percent more likely to be able to read at least letters. Those who could read only letters at baseline were 3.5 percent more likely to read at least paragraphs or words, and 3.3 percent more likely to read stories if they were in an Intervention 3 village. These changes in average literacy across the village came despite the fact that only 8 percent of children, including 13 percent of those who could not recognize letters prior to the intervention, attended the classes. Provided that the effects of Intervention 3 are channeled entirely through attendance at the reading classes, comparing the endline reading levels of the comparison group with the Intervention 3 treatment effects described above reveals just how large these effects are:  all children who could not read at baseline but attended classes ended up being able to read letters at endline, and 98 percent of children who could read at the word or paragraph level was able to read at the story level. 
 
Intervention 3 was the only intervention which actually improved educational outcomes, by empowering individuals to improve teaching in their own communities. This suggests that enabling local action which does not depend upon large-group participation may be a means of directly affecting educational outcomes.  

Encouraging Teacher Attendance through Monitoring with Cameras in Rural India

Getting teachers to attend school is sometimes more difficult than getting children there. Can monitoring tied to a salary incentive reduce teacher absenteeism in rural schools in India?  Will reduced absenteeism increase student achievement?  The answer is yes: monitoring made teachers show up, and students learned more too!

Policy Issue: 

Over the past decade many developing countries have expanded primary school access, energized by initiatives such as the United Nations Millennium Development Goals, which call for achieving universal primary education by 2015. However, these improvements in school access have not been accompanied by improvements in school quality. Poor learning outcomes may be due, in part, to high absence rates among teachers, who often lack strong incentives to attend work. There have been relatively few rigorous studies that have evaluated successful interventions to address absenteeism, so little is known about how reduced absenteeism impacts other educational outcomes. If teachers are incentivized to show up to school, is that all they do- or once there do they teach? Do simple financial incentives undermine their motivation to teach well?

Context of the Evaluation: 

Despite booming economic growth and an improved educational infrastructure in many regions in India, primary education is lagging in many remote and marginalized communities. Sixty-five percent of surveyed children enrolled in grades 2 through 5 in government primary schools could not read a simple paragraph, and 50 percent could not do simple subtraction. Teacher absenteeism, a pervasive problem in these schools, may contribute to these poor educational outcomes. Disciplinary actions are rarely undertaken against absent teachers: in a survey of 3,000 Indian government schools, only one principal reported a teacher having been fired for poor attendance . This may account for the extremely high rate of teacher absence in India: in schools examined by this study, teachers attended classes about 65 percent of the time.

Details of the Intervention: 

This study estimates the effect of incentives on teacher attendance and of increased teacher attendance on students' attendance and abilities in math and language. Seva Mandir, a local NGO, worked with researchers to randomly selected 57 of their informal education centers for the intervention, and 56 for a comparison group. Ordinarily, teachers were paid a salary of Rs. 1,000 (about US$23) for at least 20 days of work per month, and this payment structure remained unchanged for the comparison schools. In the intervention schools, teachers received a Rs. 50 (US$1.15) bonus for each additional day they attended in excess of 20 days, and received a Rs. 50 fine for each of the 20 days they skipped work—however, the fine could not exceed Rs. 500 total, due to ethical and political concerns.. Thus, a monetary incentive was attached to teacher attendance. When these incentives were implemented, monthly pay in the treatment schools ranged from Rs. 500 to Rs. 1,300 (US$11.50 to US$29.50).  

In order to monitor teacher attendance, Seva Mandir gave each teacher a camera, along with instructions to have one student take a picture of the teacher and the class at the start and close of each school day. The camera's timestamp feature allowed Seva Mand  ir to determine when and for how long the teacher was at school.

Results and Policy Lessons: 

Impact on Teacher Attendance: The program resulted in an immediate and long lasting improvement in teacher attendance rates in treatment schools. Over the 30 months of the study, teachers at program schools had an absence rate of 21 percent, compared to 44 percent at baseline and 42 percent in the comparison schools. Even four years after the start of the program, teacher attendance remained significantly higher in treatment schools (72 percent attendance) than in comparison schools (61 percent), suggesting that teachers did not change their behavior simply for the evaluation – their response was almost entirely due to the financial incentives.  

Impact on Education: Teachers who were at school were just as likely to be teaching in treatment schools compared to comparison schools- they did not just show up for the picture and go home. Student attendance on days the teacher was there was similar in both groups, meaning that students in the treatment group received about 9 percentage points (or 30 percent) more days of instruction on average simply because their teachers were more likely to be at school. This corresponds to 2.7 more days of instruction time a month at treatment schools, and this effect is larger than that of other successful interventions that have been shown to increase child attendance. A year into the program, test scores in the treatment schools were 0.17 standard deviations higher than in the comparison schools. Two and a half years into the program, children from the treatment schools were also 10 percentage points (or 62 percent) more likely to transfer to a formal primary school, which requires passing a competency test.

Replicability in Formal Settings: The question arises as to whether the program can be instituted for regular teachers in government schools. Teachers in government schools are often more politically powerful than teachers in informal or private schools. Thus, it may prove difficult to institute a system in which government teachers would be monitored daily and their pay linked to attendance. However, the above evidence suggests that if teacher attendance can be improved this should flow through into improved test scores.

Selected Media Coverage:

 

 

Esther Duflo, Rema Hanna

Teaching Socio-Emotional Skills to Schoolchildren in Lima

Research suggests that children who grow up in violent environments are more likely to use aggression to resolve conflicts, and that exposure to such violence can impact student learning. In addition, some studies indicate that it is important for children to build skills that enable them to develop sympathy and empathy for others, and maintain positive relationships. However, these socio-emotional skills are rarely incorporated into formal school curricula, especially in developing countries. In this study, Paul Gertler (UC Berkeley) and a team from the World Bank led by Ines Kudo measure the impact of a government program in Peru that teaches socio-emotional skills to schoolchildren on the students’ socio-emotional skills, learning, life satisfaction, and long-run labor outcomes, as well as school climate.

Policy Issue:

Research from developed countries suggests that children who grow up in violent schools, homes, or communities are more likely to use aggression to resolve conflicts, to be disruptive in school, and to become depressed. Moreover, exposure to violence in the home or at school can decrease student health and learning, increase criminal behavior, and even decrease employment in adulthood. Developing children’s socio-emotional skills, such as self-awareness and self-discipline, could be one way to reduce the harmful affects of growing up in violent neighborhoods by helping them negotiate emotional situations. However, there has been little research on the effectiveness of such school-based programs at improving students’ socio-emotional and cognitive skills, learning, or other outcomes.

 
Context of the Evaluation:

Violence at home, in school, and in communities is a problem that affects many children in Peru. Crime rates in Peru are among the highest in Latin America.[i] Gang violence and petty crimes are a specific concern in Lima, the capital.About 65 percent of the schools in metropolitan Lima and Callao, a neighboring city, experience high rates of violence.

According to the World Bank, the lack of socio-emotional skills among adults in Peru hinders their ability to become and stay employed.[ii] Moreover, this research suggests that socio-emotional skills are essential for improving the quality of life of all Peruvians, as people with higher socio-emotional skills have increased long-term income potential, improved health, are more engaged in their community, and ultimately, are happier.

 
Details of the Intervention:

Paul Gertler, an IPA-affiliated researcher from UC Berkeley, and a team from the World Bank led by Ines Kudo, partnered with the Ministry of Education to measure the effectiveness of Escuela Amiga, a large-scale government program designed to develop the socio-emotional skills of students in at-risk schools in Peru. The program will use a socio-emotional learning (SEL) toolkit developed by Peru’s Ministry of Education and the World Bank for students and teachers living or working in at-risk schools. Teachers receive a syllabus and other classroom materials designed to teach skills such as communication, empathy, learning, problem solving, and self-regulation. To help ensure the effective implementation of Escuela Amiga, the program includes:

1)    A one-year university training course for teachers about SEL and how to improve school climate;

2)    Teams of specialists that visit schools once a week to support teachers, students and parents in the adopting best practices; and

3)    After-school workshops, academic tutoring, sports, and other activities that involve parents and community actors.

The study will be carried out in 700 small and medium-sized primary and secondary public schools from 308 randomly selected clusters in 16 districts with high rates of violence. Half will be selected to receive the program, and the other half will not receive the program and serve as a comparison group.

The researchers will use surveys and administrative data to measure the effect of Escuela Amiga on learning outcomes, emotional well-being, and socio-emotional skills, as well as school climate, including levels of violence and bullying. These effects will be calculated controlling for cognitive skills and personality traits. In addition, the evaluation design will allow for a longitudinal analysis of long-run labor outcomes, health and risky behaviors (such as teen pregnancy, drug use, gang activity).

 
Results and Policy Lessons:

Results forthcoming.

 


[i] Latin America Public Opinion Project (2012). Political Culture and Democracy in Peru.  P.88

Retrieved from: http://www.vanderbilt.edu/lapop/peru/Peru-2012-Report.pdf

[ii] World Bank (2011). Strengthening skills and employability in Peru: final report. (The World Bank Research Report No. 61699- PE). Retrieved from http://www-wds.worldbank.org/external/default/WDSContentServer/WDSP/IB/2011/09/05/000356161_20110905010932/Rendered/PDF/616990ESW0whit0ficial0Use0Only00090.pdf http://documents.worldbank.org/curated/en/2011/05/14963817/strengthening-skills-employability-peru-final-report

Paul Gertler

HIV/AIDS Prevention Through Relative Risk Information for Teenage Girls in Kenya

Kenya's Ministry of Education has developed an AIDS curriculum for schools. However, this curriculum has not been effective in reducing the rates of infection and pregnancy. Information on the distribution of HIV infections by age and gender is not included in the official HIV curriculum for primary school. IPA evaluated the impact this information could have on teenager’s sexual decisions. Results found that girls exposed to the program were less likely to be pregnant in the next year. A follow-up survey is currently being conducted in order to measure longer-term impacts.

Policy Issue: 

The vast majority of HIV/AIDS cases occur in sub-Saharan Africa, where nearly 2 million people become infected with the virus every year. One quarter of these new HIV infections are among people under 25, and almost all are due to unprotected sex.  AIDS is incurable and no successful AIDS vaccine has been developed, so policymakers must focus on other preventative measures. Ensuring the adoption of safer sexual behavior among youth remains critical to preventing the transmission of this disease.

Context of the Evaluation:

Kenya has the eleventh largest HIV infected population in the world -- over 6 percent of Kenyans are infected.1  Children are seen as a “window of hope” in the fight against AIDS, because their sexual patterns are not firmly established. In an effort to prevent HIV infections in new generations, the Kenya Ministry of Education, Science, and Technology integrated HIV/AIDS education into the primary school curriculum in 2001. However, by 2003, this curriculum had not been fully implemented, likely due to teacher inexperience and discomfort with talking about this sensitive material.

Details of the Intervention:

In Kenya, as in most African countries, 25-year-old men are far more likely to have HIV than 16-year-old adolescent boys. This means that sexual relationships with older partners (often called “Sugar Daddies”) are particularly dangerous for adolescent girls.

Information on the distribution of HIV infections by age and gender is not included in the official HIV curriculum for primary school, however. To test the impact this information could have on teenager’s sexual decisions, ICS conducted a “Relative Risk Information Campaign” in 71 schools randomly selected among 328 primary schools involved in another HIV intervention evaluation. A trained project officer visited each of those 71 schools and, with the authorization of the teachers, spoke to Grade 8 students for a 40-minute period. Students were shown a 10-minute educational video on “sugar daddies”. The video screening was followed by an open discussion about cross-generational sex. During the discussion, the project officer shared the results of studies conducted in Kenya and Zambia and Zimbabwe on the role of cross-generational sex in the spread of HIV. In particular, the project officer wrote on the blackboard the detailed prevalence rates of HIV, disaggregated by gender and age group, in the nearby city of Kisumu, a place familiar to the students.

Results and Policy Lessons:

Impact on Unsafe Cross-Generational Sex: As a result of this intervention, the incidence of childbearing was reduced by 28 percent (from 5.4 percent of girls getting pregnant within a year, to 3.9 percent). This suggests that the intervention reduced the likelihood that girls engage in unsafe sex. Specifically, the intervention seems to have reduced unsafe cross-generational sex: the rate of childbearing with men five or more years older fell by 61 percent, with no offsetting increase in childbearing with adolescent partners. 

Cost-Effectiveness: This targeted approach cost US$98 per pregnancy averted. Researchers came up with several possible estimates of cost per HIV infection averted based off of various estimates of the ratio of the risk of HIV infection to the risk of cross-generational pregnancy; these estimates ranged from just under US$400 to almost US$2,000. These rough cost-effectiveness estimates compare favorably with other HIV prevention programs, such as treating sexually transmitted infections, voluntary HIV testing, and male circumcision.

 

1CIA World Factbook, “Kenya,” accessed June 6, 2012. https://www.cia.gov/library/publications/the-world-factbook/geos/ke.html.

Selected Media Coverage:
Pascaline Dupas

Improving Early Grade Learning Outcomes in Tanzania

Student learning levels across East Africa remain extremely low, despite more than a decade of major reforms and significant new investments in public education. To help generate rigorous evidence on what works, researchers are evaluating the impact of an education intervention that sends grants directly to schools and pays teachers a performance-based bonus.

Policy Issue:

Overall student-learning levels remain extremely low across East Africa, despite a decade of major reforms and significant new investments in public education. In Kenya, Tanzania, and Uganda, recent nationwide surveys show that large numbers of children are unable to read or do arithmetic at their grade level.1 Governments in the region are currently using two main approaches to improve the quality of education: strengthening teacher training and disbursing capitation grants, which provide funding to schools based on the number of students enrolled. However, the efficacy of these approaches in improving learning outcomes has not been established. In fact, several studies show that formal levels of teacher qualification are weakly correlated with performance.2 Capitation grant programs have not been rigorously evaluated, and many of these programs lack accountability measures and incentive structures. Evidence suggests that rewarding teachers or schools that achieve learning outcomes may be a more effective way to improve student learning and performance.3 This evaluation aims to add to the evidence on this topic.

Context of the Evaluation:

Educational outcomes in Tanzania have not improved in recent years, despite several new reforms and aid initiatives. Just 68 percent of Tanzanian children ages 10-16  can pass a basic numeracy test, 53 percent can pass a basic test in Kiswahili, and 35 percent can pass an English literacy test.

In February 2013, Twaweza, Tanzania’s Commission for Science and Technology, and IPA-affiliated researchers launched KiuFunza, an education initiative that seeks to discover through rigorous research what really works to help children learn. Twaweza, the implementing partner, is a nonprofit focusing on large-scale change in East Africa, with operations in Tanzania, Kenya and Uganda.

Details of the Intervention:

Researchers conducted a randomized evaluation over a two-year period to test the impact of performance-based grants for schools and teachers on student learning and performance. Government primary schools in 11 districts in Tanzania were divided into three treatment groups and a comparison group:

(1.)  Capitation Grant: Each school received a grant of 10,000 Tanzanian shillings (US$5.84) per student enrolled, delivered in two disbursements per year: the first in April or May and the second in July or August. Twaweza channeled grant funds directly to the primary schools, and provided information about the grants to school stakeholders, including teachers, school management committees, local politicians, parents and students via informational meetings and written materials. Researchers will measure what portion of funds actually reach schools, the level of citizen engagement in determining the use of funds, and, ultimately, the impact on learning outcomes. (70 schools)

(2.)  Cash on Delivery: Kiswahili, English, and Math teachers in Grades 1-3 were eligible to receive 5,000 Tanzanian shillings for each student who passed a given subject test at the end of the year, for a total of 15,000 Tanzanian shillings per student. Teachers were not penalized for students who did not pass. Researchers will measure the impact of incentivizing teachers on learning outcomes for students in Grades 1-3. (70 schools)

(3.)  Combination: These schools received both the capitation grant and cash on delivery interventions. This arm of the evaluation will reveal whether rewarding teachers for good student performance can create incentives to make better use of the capitation grants. (70 schools)

(4.)  Comparison group: This group did not receive either the capitation grant or cash on delivery treatment. (140 schools)

Researchers will use government test scores, results from the tests conducted by Twaweza, results from independent tests conducted by the researchers, school administrative records, and data from six surveys (three per year), to measure changes in school performance and student learning.

Twaweza and their partners in each district directly implemented all interventions, and Twaweza provided the funds for the grants and cash on delivery interventions.

Results and Policy Lessons:

Results forthcoming.



[1] Jones, et al. "Can your child read and count? Measuring learning outcomes in East Africa." Journal of African Economies 0 (0); 1-30. (2014).

[2] E.g. Kane, T. J., J. E. Rockoff, and D. O. Staiger. (2008). "What Does Certification Tell Us About Teacher Effectiveness? Evidence from New York City," Economics of Education Review, 27, 615-631; Kleiner, M. (2000). "Occupational Licensing," Journal of Economic Perspectives, 14, 189-202.

[3] Kremer, Michael, and Alaka Holla. "Improving education in the developing world: what have we learned from randomized evaluations?" Annual Review of Economics 1 (2009): 513; Woessmann, Ludger. “Cross-Country Evidence on Teacher Performance Pay.” Discussion Paper No. 5101, July 2010.

Evaluating Remedial Science Education in Lima, Peru

When young learners are performing below average in school, how can we close the learning gap? In this study, researchers go to Lima, Peru to evaluate the impact of a program that aims to help low-performing students, mostly from low-income urban households, master basic skills in science through targeted teaching that allows them to advance at their own pace. If the program proves effective, it may be expanded across Peru.
 
Policy Issue:
Recent evidence suggests low-performing students who have fallen behind in school respond positively to targeted, self-paced teaching aimed at mastering basic skills, and that this type of education can be effective at closing achievement gaps. Remedial education, as it is called, has improved short- and medium-term academic performance in various settings. This study will evaluate the impact of a remedial science education program for low-performing third graders in metropolitan Lima, Peru.
 
Context of the Evaluation:
Peruvian students consistently underperform in math and science. In the Second Regional Comparative Education Study (SERCE), for instance, Peruvian students scored below the Latin American average in both subjects. In third grade mathematics, more than half of the student population was below basic proficiency levels. Furthermore, in the 2012 application of the Program for International Student Assessment (PISA), Peruvian students ranked last, among the 65 participating countries, in both math and natural science. Moreover, about a third of all participating Peruvian students placed in the lowest proficiency level in science, meaning they had not mastered even the most basic skills. These assessments suggest that Peruvian students lack critical abilities, such as analyzing and synthesizinz information, applying new knowledge in real-life settings, and reasoning.
 
In an effort to improve science teaching and learning, Peru’s Ministry of Education has been collaborating with the Universidad Peruana Cayetano Heredia on diverse initiatives since 2010 to identify the best solutions for comprehensive K-College reform.
 
Universidad Peruana Cayetano Heredia, the main implementation partner, has a long track record of developing and implementing education initiatives, and is invested in pursuing rigorous research to help improve learning and achievement in Peruvian schools.
 
Details of the Intervention:
To test if remedial science education—taught by trained science tutors—improves students’ science test-score performance and attitudes towards science, researchers will carry out a randomized evaluation in which half of the lowest performing students in participating schools in metropolitan Lima receive a remedial science program.  Specifically, the program targets third-grade children whose prior science achievement places them in the bottom half of their class. Approximately 1,200 students in 49 randomly chosen public schools in Lima will participate in the study.
 
In the spring of 2014, third grade students from the selected schools took a baseline diagnostic science test. Of those who placed in the lower half of the scoring distribution, half were randomly selected to receive tutoring through a lottery that was stratified by school and gender.  The remaining half of the lower-scoring students were assigned to serve as a comparison group.
 
The first stage of the program will consist of tutor recruitment and pre-service training. Prior to the start of the school year, the university that is implementing the program will select potential tutors—preferably women that could serve as role models for girls—who are either science schoolteachers or college students majoring in pedagogy. The tutors will undergo a 20-hour introductory training course.
 
The second stage of the program is the remedial instruction itself. Tutoring groups will be composed of five to six students per tutor/school. Tutors will meet with students once a week at each school after regular hours between mid-June and November of 2014.
 
At the end of the tutoring program, students will take a test similar to the one used at baseline. Additionally, researchers will survey teachers and principals on their school’s climate and resources, and on their perspectives on classroom learning and their school’s science curriculum.
 
Results and Policy Lessons:
Results forthcoming. 
 

Technical Vocational Training in Mongolia

Can vocational education increase the wages of poor Mongolians? This study evaluates a project designed to improve technical skills and productivity to meet labor market demand in key industries (including, among others, construction, mining, electronics, mechanics, and transport). 

Policy Issue:

Youth underemployment, especially among less educated populations, has the potential to create significant social unrest and perpetuate poverty. However, little is known about how best to help youth find jobs and smooth the school-to-work transition, particularly in less developed countries. One would-be tool for expanding the labor market opportunities in these settings is vocational education, which could help students learn a trade and acquire the skills needed to take advantage of employment opportunities, and create successful small businesses. However, little is known about the actual benefits of vocational education in developing countries.

Context of the Evaluation:

Mongolia’s vocational education system has not evolved to serve the demands of a modern, private-sector led economy. The capacity of this system to teach core technical skills and provide critical labor information is weak, training equipment is limited and outdated, and instructors ill-prepared to teach. Essential partnerships between the government and business to ensure that students receive high quality, demand-driven training are largely absent, and credentialing systems are substandard. As a result, Mongolia tends to import skilled labor from other markets, leaving high rates of unemployment among unskilled young Mongolians.

Details of the Intervention: 

Researchers evaluated the Vocational Education Project, an initiative designed to address the gaps in Mongolia’s vocational education system, specifically by seeking to increase the wages of poor Mongolians by improving their technical skills and productivity to meet labor market demand in key industries (including, among others, construction, mining, electronics, mechanics, and transport). This will be done by (a) strengthening the institutional framework needed to support a demand-driven vocational education system through legal and policy reform and the creation of a Labor Market Information System, (b) defining industry training standards for occupations and translating these standards into a modern vocational education curriculum supported by new instructional materials and equipment, (c) developing 30 new career preparation tracks, and (d) improving teacher training and professional development.

This study will evaluate the overall impact of attending a Technical Educational Vocational Training (TVET) school, and the incremental impact of school equipment upgrades on both academic and labor-market outcomes. Equipment upgrades will complement the competency-based training curriculum and other components by providing the physical and technological resources that schools will need to instruct a new generation of skilled workers. This is the most expensive component of the program, and is expected to have the largest and most direct impact on student skill levels and employment outcomes.

Researchers are working with TVET schools to create comprehensive standards for admitting students.  Out of a pool of ‘qualified’ applicants, students will be chosen at random to gain admittance to the school. The randomized design will ensure that all qualified applicants will have an equal probability of receiving the vocational and technical training. Qualified applicants who were not admitted will serve as the comparison group. Surveys will be administered to the entire pool of applicants at the time of the application, one year after, and again at the time of graduation, including a standardized general knowledge test to measure skill levels and academic performance. Upon graduation, students will complete standardized trade-specific tests that measure knowledge and skills in specific trades. Shortly after graduation, another survey will be administered to collect information on labor-market outcomes, including wages and hours worked. This follow-up survey will continue annually for three years.

Results and Policy Lessons: 

Results forthcoming.

 

Starting a Lifetime of Saving: Teaching the Practice of Saving to Ugandan Youth

Improving financial literacy and access to bank accounts may help youth save, allowing them to meet current financial needs and invest in their futures. In Uganda, researchers evaluated whether offering financial education or group savings accounts to church-based youth groups increased savings. They found that total savings and income increased among youth offered financial education, group savings accounts, or both education and group accounts.

Policy Issue:

Promoting financial literacy and providing access to bank accounts have become popular approaches to help the poor save. Increased savings may help individuals meet day-to-day financial demands and invest in their futures. Furthermore, increasing the savings rate in the general population may help promote large-scale changes in a country’s economy by allowing increased investment in productive resources. In order to maximize the benefits of increased savings at both the individual and country level, it may be most effective to encourage youth to save. Young people may be more likely to adopt new habits, and they have many working years ahead of them. A growing body of literature investigates whether either financial education or bank access affect savings behavior. 

 
Context of the Evaluation:

Uganda has a very young population: in 2006, 52 percent of the country’s population was under 15 years old and 29 percent of the country’s adult population was between 15 and 34.1 In addition, Uganda has extremely low savings rates, even relative to its neighbors. Between 2001 and 2003, the average savings rate among Ugandan households was 5.2 percent, compared with an average rate of 12.7 percent in neighboring Kenya.2

Researchers partnered with the Foundation for International Community Assistance (FINCA) and the Church of Uganda in this evaluation. FINCA, whose mission is to provide financial services to the world’s lowest-income entrepreneurs, has worked in Uganda since 1992. The Church of Uganda is an Anglican church, representing the second largest religious group in the country. As of the 2002 census, 36 percent of the population considered themselves affiliated with the church. The Church maintains a large network of youth fellowship groups, based at village churches around the country. The youth groups participating in this study had an average of 40 members. The average age was 24.5 and 40 percent of members were female.

 
Details of the Intervention:

Researchers evaluated whether offering financial education or group savings accounts to Ugandan youth groups increased savings. The study involved 240 Church of Uganda youth groups, which were randomly assigned to receive financial education, a group savings account, both financial education and a savings account, or neither intervention. There were 60 youth groups in each arm of the study.

The curriculum for the financial education intervention was designed in partnership with Straight Talk Foundation and Freedom from Hunger. The ten-session, fifteen-hour curriculum taught concepts and skills for improving savings behavior, including role-playing the differences between saving and borrowing to achieve a goal, how to keep a budget, and strategies for successfully discussing sensitive topics around money.

Researchers partnered with FINCA to design a group savings account without fees and with simple account-opening procedures, which minimized common barriers to opening accounts. Each club had only one account and was responsible for maintaining a ledger with individual members’ savings. Clubs were also required to make a deposit within thirty days of opening the account and to maintain a minimum balance of 50,000 UGX (US$20).

 
Results and Policy Lessons:

Financial literacy: Members of youth groups receiving financial education had higher levels of financial knowledge, awareness, and numeracy. Youth in groups receiving financial education only scored 0.04 standard deviations higher than the comparison group on an index combining questions relating to financial literacy. Youth in groups receiving both financial education and group accounts scored 0.06 standard deviations higher than the comparison group. Youth in groups receiving account access only did not score any better than the comparison group.

Bank savings: Using administrative bank data on the group accounts offered in the intervention, researchers found that offering financial education in addition to account access increased savings more than offering the account alone. Averaging across groups receiving account access only and groups receiving account access plus financial education, only 14 percent of members used the account. However, those who did use the accounts saved non-trivial amounts: an average of 15,000 UGX  (US$6) in the account-only group and an additional 4,000-7,000 UGX (US$1.60-2.80) among those who also received financial education.

Total savings: All three interventions designed to promote savings increased participants’ total savings. This measure included saving by storing at home, by having another person hold the money, or by buying durable goods that could later be sold, in addition to savings held at a formal bank. In contrast to the administrative bank data, these results did not show that financial education and account access work together to promote savings, but rather that each approach can encourage increased savings on its own.

Income: Individuals in all three treatment groups reported earning 10-15 percent more income than individuals in the comparison group. However, researchers were unable to determine whether this effect resulted from individuals working more in order to increase their savings or from individuals using savings to make investments that generated income.

Are financial education and formal savings accounts complements or substitutes in the medium to long-term? Do the impacts on behavior and income persist over time? What are the mechanisms underlying the increase in earned income? To answer these questions, a follow-up of this evaluation three years after the commencement of the intervention is currently being conducted by IPA in Uganda, under the Financial Capability Research Fund. Results forthcoming.

 


[1]Uganda Bureau of Statistics (UBOS) and Macro International Inc. 2007. Uganda Demographic and Health Survey 2006. Calverton, Maryland, USA: UBOS and Macro International Inc. Page 11.

[2] Bank of Uganda research department, Sept. 14, 2005.  Found in “Savings Habits, Needs and Priorities in Rural Uganda.” Prepared by Richard Pelrine, Olive Kabatalya. Rural SPEED and Chemonics International.  Produced by USAID, September, 2005. 

Online Sexual Education for Schools in Urban Colombia

Can online sexual health education courses improve students’ sexual health knowledge, attitudes, and behavior? Do such courses also have positive effects on the peers of students who take the course? Researchers evaluated the impact of an online sexual health education course on the knowledge and sexual behavior of urban Colombian high school students. The education program led to significant impacts on knowledge and attitudes. No impacts were found on self-reported measures of behavior, but the program led to a reduction in incidence of sexually transmitted diseases among sexually active females. Moreover, results pointed to a significant increase for the treatment group in redemption of vouchers for condoms among students.

Policy Issue:

As young adults marry at older ages, they are more likely to have sex before marriage, increasing their risks of unwanted pregnancies and sexually transmitted infections (STIs). In low-income settings, adolescents face the added constraints of lower availability of information about safe sexual practices and restricted access to reproductive health services. Furthermore, sexual risk-taking in developing countries may have graver consequences because governments lack the resources and health systems to treat certain conditions.

In recent years, the rise of information and communication technology programs has changed school-based sexual health education.Some research[1] suggests that for mainstream subjects such as math and reading, computer-based instruction may not work as well as conventional teacher instruction. We examined whether this is true for sexual health education, or if the reverse was true.

 
Context of the Evaluation:

In Colombia, only 55 percent of sexually active females aged 15-17 used a condom in their first sexual encounter.[2] This level of risk-taking is reflected in the fertility rate among adolescents in Colombia of 74 births per 1,000, compared to 41 per 1,000 in the United States, 14 in Canada, and five in the Netherlands.[3] By age 19, 20 percent of female adolescents in Colombia have been pregnant while 16 percent are already mothers. In order to tackle this challenge, legislation establishing sexual education as obligatory in Colombian public schools was passed in 1994, and national public policy was drafted by 2003.

Profamilia is an internationally recognized non-profit provider of family planning and reproductive health services in Colombia, with over 33 clinics and 1,800 employees. A member of the International Planned Parenthood Federation since 1967, Profamilia is Colombia’s largest non-governmental organization focused on sexual health and reproductive health. Prompted by the deterioration of important adolescent sexual health indicators, such as teenage pregnancy rates, Profamilia’s education branch, Profamilia Educa, developed an online sexual education course for adolescents in public schools.

 
Details of the Intervention:

Researchers partnered with Profamilia to evaluate the impact of the online sexual health education course on the sexual health knowledge, behavior, and attitudes of Colombian high school students who took the course, as well as on their peers, who did not take the course. The study consisted of 138 ninth-grade classrooms from 69 public schools in 21 Colombian cities.

The course had five separate modules covering the topics of sexual rights, pregnancy/family planning and the use of contraceptives, STIs/HIV and the use of condoms, objectives in life and the role of sexuality (empowerment), and prevention of sexual violence.Profamilia implemented the course in the school’s computer lab with teacher supervision. In addition, students had access to the course from any Internet-enabled computer using a password-protected account. Students were also assigned an anonymous remote tutor in the central Profamilia offices who would answer questions online related to the material individually and confidentially.

Researchers collected data on the impact of the program both immediately after course completion and six months afterwards. Data was collected on students who took the course and students from comparison schools where the course was not offered. Also, to measure any effects on peers, researchers collected data on students who did not take the course, but went to schools where the course was offered. 

To complement the self-reported data, and to come as close as possible to measuring the actual use of condoms in a credible manner,researchers offered students a voucher for six condoms six months after the end of the study. Profamilia recorded which students redeemed their voucher at the local health clinic.

 
Results and Policy Lessons:

The online sexual health education course led to significant impacts on knowledge and attitudes and a 52 percent increase in condom redemption. Although fewer STIs were reported for females who reported being sexually active in an initial survey, there was no impact on self-reported behavior on average. However, the program led to a large increase in redemption of vouchers for condoms, suggesting that the indeed the program did change behavior. Notably, the impacts of the course intensified when a larger fraction of a student’s friends was also treated.

Knowledge: The course produced a 0.37 SD increase in overall knowledge one week after the intervention and a 0.38 SD increase in overall knowledge six months after the intervention. The lowest impact was found on the identification of situations of sexual violence - we found that treated beneficiaries were 0.11 SD more likely to correctly identify a situation of sexual violence.

Attitudes: The course produced significant effects of 0.24 SD in terms of attitudes one week after the intervention and 0.17 SD six months after, which suggests some decay in attitude impacts over time.

Sexual Behavior: The course did not change the average number of partners, frequency of sex, or rate of abstinence over the six months following the course. To go beyond self-reported measures, however, researchers found that 27 percent of treatment students redeemed the condom vouchers, compared to 18 percent of comparison students, a 52 percent increase. This result points to a significant increase in condom use.

Social Networks: Researchers documented a strong social reinforcement effect: the impacts of the course intensified when a larger fraction of a student’s friends was also treated. In particular, when full sets of friends were treated researchers found significant reductions in sexual activity, frequency of sex, and number of partners. Throughout the analysis, researchers did not find evidence of effects on peers who did not take the course, i.e. “spillovers.”

Cost-effectiveness: The marginal cost of the Profamilia course was approximately $14.60 per student. Compared to non-computer-based sexual health interventions in the United States, which range from $69 to more than $10,000 per student, the Profamilia course was extremely low cost.

Policy Lesson: Web-based education is a plausible alternative in a context of tight budget constrains in public education given the widespread availability of the Internet in schools throughout the world and accelerated improvements in software quality. In societies where teachers may be unwilling or unable to provide sexual education, online courses may also prove a useful and cost-effective substitute for in-person instruction. Furthermore, the cost-benefit analysis suggests that because online sexual health education programs are extremely low cost, their measurable benefits in terms of STI reductions make up for the costs.


[1] E.g. Barrera-Osorio, Felipe, and Leigh L. Linden. "The use and misuse of computers in education: evidence from a randomized experiment in Colombia." World Bank Policy Research Working Paper Series, Vol (2009).

[2] DHS (2010) “Encuesta Nacional de Demografía y Salud - ENDS Colombia 2010”.

[3] United Nations Statistics Division (2004) “Demographic Yearbook 2004”, New York: United Nations.

Improving Numeracy Instruction for Young Children in Peru

In rural Peru, 96 percent of children were not learning at their grade level in math in 2012. A Peruvian non-profit introduced a program that aimed to reverse children’s negative attitudes about math and improve students’ numeracy skills through a hands-on and interactive approach. Researchers evaluated the impact of the program for kindergartners, measuring outcomes after the program was completed and one year later.

Policy Issue:

In recent years, most regions of the world have achieved near-universal primary school enrollment. However, the increase in coverage has not corresponded with an improvement in educational quality. For example, in Peru, the net primary school enrollment rate was 94 percent in 2011.[1] However, nationwide tests in 2012 showed that 87 percent of seven-year-olds had not achieved the expected level of competency for their age in math; this number reached 96 percent in rural areas.[2] Furthermore, the 2012 Program for International Student Assessment (PISA)—which is a worldwide study of 15-year-olds’ scholastic performance—ranked Peru last, out of 65 countries, in the categories of math, science, and reading.[3]

In order to address such low test scores, particularly in the area of mathematics, some organizations have created innovative educational programs as an alternative to rote learning and memorization. This study assesses the impact of an innovative mathematics program in Peru.

Context of the Evaluation:

The Apoyo Institute—a Peruvian non-profit that carries out research and outreach on educational and social programs—developed the “Mathematics for All,” or Mimate, program to improve Peruvian students’ performance in mathematics. The program aims to reverse children’s negative attitudes about math through an approach that emphasizes hands-on and interactive learning. Pupils discover mathematical concepts through games, group or pair activities, and other interactive activities. Instructors encourage logical thinking rather than memorization and present math as a useful part of everyday life.

This study took place in public kindergarten classes in both urban and rural schools in the south-central Andean regions of Huancavelica and Ayacucho. These regions were among the poorest in Peru, and Ayacucho schools exhibited the lowest test scores in math nationally.

 
Details of the Intervention:

To evaluate the impact of the Mimate program on the numeracy skills of kindergartners in Peru, researchers carried out a randomized evaluation in partnership with the Inter-American Development Bank. Researchers randomly assigned 109 schools—with a total of 2,900 children—to either the treatment group, which received the program, or the comparison group, which did not receive the program.

In treatment group schools, the Mimate program was taught in 45-minute sessions two or three times a week. Each kindergartner received a set of materials, each with an individual box containing games and learning materials. At the start of the program, teachers underwent a day and a half of training on how to organize the teaching sessions and direct the activities. Four discussion groups of teachers were also organized during the year to discuss pedagogical methods and the implementation of the program. In addition, parents’ meetings were organized during the year to discuss the program's work plan and activities, and to encourage parents to ask their children about school daily.

Before and after the completion of the program, researchers tested the kindergartners’ ability to carry out mathematical tasks, such as recognizing shapes and colors and basic addition. Students were also tested in other areas, such as cognitive ability, oral comprehension, and basic writing.

 
Results and Policy Lessons:

Results forthcoming.



[1]The World Bank DataBank. http://data.worldbank.org/indicator/SE.PRM.NENR.

[2]Unidad de Mendición de la Calidad Educativa. “Evaluación Censal de Estudienantes 2012.” http://umc.minedu.gob.pe/?p=1405.

[3]OECD. 2014. “PISA 2012 Results in Focus.” http://www.oecd.org/pisa/keyfindings/pisa-2012-results-overview.pdf.

Accelerated Schooling for Children in Mali

Almost a quarter of all primary school age children are not attending school in Sub-Saharan Africa. In this study in southern Mali, researchers evaluated the impact of an accelerated learning program for out-of-school children on the students’ educational achievement, home life, and continuation with schooling.  This research aimed to contribute to cost-efficient policies for improved access to, and quality of, education in Mali and beyond.

Policy Issue:

Sixty-seven million children were out of school across the globe in 2009, and nine of the 17 countries with the highest numbers of out-of-school children are located in Sub-Saharan Africa.[1]While remarkable progress has been made in expanding access to primary schooling in Sub-Saharan Africa—the number of out-of-school children decreased from about 43 million to 30 million between 1999 and 2009—23 percent of all primary school-age children still remain out of school across the region.[2] One way to address this problem may be accelerated learning programs for out-of-school children that aim to quickly bring them up to grade level and enrolled, or re-enrolled, in the government school system. This study evaluates the impact of such a program.

Context of the Evaluation:

Roughly 3 million school-aged children in Mali, Niger, and Burkina Faso are not enrolled in school. National survey data from Mali reports that only 47 percent of rural eligible children were enrolled in the first level of schooling in 2006. This underscores the urgent educational policy problem of how to effectively engage out-of-school children.

The Strømme Foundation, a Norwegian development organization, created a Speed School program to respond to the high percentage of out-of-school children in Mali, Niger, and Burkina Faso. The educational program was developed by education curriculum experts and aims to provide out-of-school children ages 8-12 with an accelerated nine-month curriculum, and to ultimately transfer them into the government primary school system.

Details of the Intervention:

To evaluate the impact of the Speed School program on the students’ educational achievement, home life, and continuation with schooling, researchers carried out a randomized evaluation over the course of the 2012-2014 school years in the Koulikoro and Sikasso regions of southern Mali. Seventy-seven randomly-selected villages participated in the study, with 46 receiving the program and 31 serving as the comparison group. IPA collected data on children and households in villages participating in the program and in comparison villages for two years, between mid-2012 and mid-2014.

The Speed School program started with the creation of local Speed Schools Management Committees, consisting of women and men. The committee’s role was to advocate for the importance of education, encourage children to attend Speed School regularly, and closely collaborate with parents and teachers, forming a locally-owned education network. Children were taught to read and write in their mother tongue during the first two months, and then an continued with an accelerated curriculum in French. The pedagogical approach was designed to encourage children to actively participate. Teachers offered intensified learning support. At the end of the program, students could enter grades 3, 4 or 5 in the government primary school system, depending on their test scores.

 
Results and Policy Lessons:

Results forthcoming.



[1]UNESCO Institute of Statistics. “Out-of-School Children: New Data Reveal Persistent Challenges.” June 2011. Available at: http://www.uis.unesco.org/FactSheets/Documents/FS12_2011_OOSC_EN.pdf

[2]Ibid. 

Andrew Dillon

Negotiating a Better Future: The Impact of Teaching Negotiation Skills on Girls' Health and Educational Outcomes

In Sub-Saharan Africa, young girls drop out of school at higher rates than boys. Parents often invest more in sons than in daughters, by allowing them more resources for education, such as school fees, and time away from house chores for studying. Adolescent girls are also more likely to contract HIV from older, more sexually active male partners, on whom they often depend for financial resources. Girls’ education and negotiation skills for women are therefore viewed as important tools for reducing school dropout rates, early pregnancies and the HIV rate among young women. This study, conducted in the capital of Zambia, assesses the impact of teaching girls negotiation skills on health and education outcomes.

Policy Issue:

When young girls drop out of school, they are often unable to develop the skills necessary to support themselves. They often rely on male partners for resources, and those partners often demand sex in return for financial support.  Such relationships are prevalent across sub-Saharan Africa, leaving young girls highly vulnerable to HIV infection and unwanted pregnancy, evidenced by the two-to-one ratio of HIV rates among young women versus their male counterparts. 1 The World Health Organization has identified negotiation skills for women and expanded efforts to keep girls in school as critical tools for reducing HIV rates among women in sub-Saharan Africa. 2 Designing school curriculum to provide girls with a stronger education and new skill sets has the potential to change gender dynamics and improve health outcomes for this vulnerable population.

Context of the Evaluation:

School data for Zambia shows a dramatic decline in female enrollment from primary to secondary school years.3  While this drop is normally attributed to the commencement of school fees in the eighth grade, a closer look reveals that school dropout rate increases prior to the fee increase. In grade five, the drop-out rate is three times higher for girls than boys. 4

This project tests the impact of negotiation training in addition to the current school curricula on HIV/AIDS, health, and education outcomes among Zambian girls. Through a randomized controlled trial, this study analyzes whether negotiation skills that allow a girl to reshape her understanding of a conflict and her communications with others, can ultimately result in more favorable resource allocations.

Details of the Intervention:

This study isolates the impact of teaching information versus teaching negotiation by layering two interventions on top of a "social capital" program, including time with other girls in a safe space.

About 2,400 grade eight girls from across 20 schools in Lusaka will be randomly assigned to participate in one of three two-week programs.  About 120 girls will be engaged per school, with roughly 40 girls in each program:

  • Social capital: girls meet after school to play games; receive  a snack notebooks, and pens

  • Information: girls meet after school to learn information on HIV and importance of schooling and to play games, also receive a snack, notebooks, and pens

  • Negotiation plus information: girls receive above program plus negotiation training

The Negotiation Curriculum is structured by four principles: "Me," or identifying one’s own interests and options in conflict situations; "You," or identifying the other person’s interests, needs, and perspective; "Together," or identifying shared interests and small trades; and "Build," or developing win-win solutions.  The curriculum also accounts for some negotiations in which it is necessary to be patient, or "Take 5," and others in which the only outcome to keep the girl safe and healthy is to walk away and not negotiate.

Outcome measures will measure both the size and source of impact, capturing transformations in the girl's capabilities, her interactions with others, and the outcomes of those interactions:

  • Survey data: Self-perception, outcomes of arguments and discussion, reported locus of control, intra-household allocations, and sexual risk exposure. Impact on the family measured through parent and sibling surveys to see if gains in participant well-being come at the expense of other family members.

  • Real outcomes (administrative data from schools): Rates of pregnancy, school attendance and advancement, and potentially STI/HIV rates

  • Behavioral measures: Take-up of an additional opportunity that requires child-parent negotiation, altered willingness to pay for schooling by parents, responses to negotiation scenario or partner game.

Results and Policy Lessons:

Results forthcoming.  If successful, this program curriculum could be scaled up countrywide in partnership with the Ministry of Education to increase schooling attainment and lower HIV infections at a relatively low cost.

For more information about this project, click here.

To watch a video about this project, click here.

[1] (UNAIDS (2010) "UNAIDS report on the global AIDS epidemic" p.183)

[2] WHO's Gender, Inequalities, and Health (2009): http://www.who.int/gender/hiv_aids/en/

[3]UNICEF (2011) "State of the World's Children." p.107

[4]Zambia DHS 2007, p. 21

Nava Ashraf

The Zambian Early Childhood Development Project

Despite an increased international interest in child development, representative data on child development is still remarkably scarce, particularly from Sub-Saharan Africa. For this project, researchers from Harvard University and the University of Zambia partnered with the Zambian Ministry of Education, the Examination Council of Zambia and UNICEF to develop and evaluate a comprehensive instrument for assessing Zambian children’s physical, socio-emotional, and cognitive development before and throughout their schooling careers. The project has thus far demonstrated that comprehensive child assessments are feasible. Longer-term follow-studies are planned to assess both the validity of the tool and to identify the most important domains of child development for schooling outcomes in a Sub-Saharan African context.

Policy Issue:

Early childhood care and education remains underdeveloped in much of the developing world, though early educational experiences may have a significant impact on future learning. A large number of studies have investigated the impact of early childhood experiences on children’s developmental, health, and educational outcomes in developed countries, yet relatively little evidence is available on early childhood development in Sub-Saharan Africa. This research responds to this knowledge gap, aiming to improve understanding of child development in a Sub-Saharan Africa context.

Note: This study is not a randomized controlled trial.

Context of the Evaluation:

In 2009, the Zambian Ministry of Education, the Examination Council of Zambia, UNICEF, the University of Zambia, and the Center on the Developing Child at Harvard University launched the Zambian Early Childhood Development Project (ZECDP), a collaborative effort to measure child development in general, and to measure the improvements in child development achievable through large health programs like Zambia’s nationwide Rollback Malaria program. In order to comprehensively measure children’s development prior to school entry, the ZECDP created an instrument for assessing children’s physical, socio-emotional, and cognitive development before and throughout their schooling careers—the first assessment tool of its kind in Zambia.

 
Details of the Intervention:

Researchers and early childhood development stakeholders from the University of Zambia, the Ministry of Health, the Ministry of Education, and UNICEF developed and evaluated a comprehensive instrument for assessing Zambian children’s physical, socio-emotional, and cognitive development before they enter the formal schooling system.

Completed in May 2010, the Zambian Child Assessment Test (ZamCAT) combines existing child development measures with newly developed items in order to provide a broad assessment of children of preschool age in the Zambian context. The ZamCAT features tasks and tests to measure seven fundamental domains of child development: fine motor skills, language (expressive and receptive), non-verbal reasoning, information processing, executive functioning, socio-emotional development and task orientation.

After two rounds of piloting, a first cohort of 1,686 children born in 2004, from randomly selected households across 73, was assessed between July and December 2010. In 2011, successful follow-up occurred with 1,250 of those children. IPA collected data during an additional follow-up in June-August 2012. The 2012 survey covered 945 children and their caregivers in 53 of the study clusters. Trained surveyors visited the 945 randomly selected children and their caregivers at home, and conducted a one-hour long skill assessment with children followed by an one-hour interview with their caregiver to capture children’s socioeconomic and health background as well as children’s exposure to early learning programs.

 
Results and Policy Lessons:

Results from the ZECDP suggest a stark socioeconomic gradient in children’s development prior to entering school. In the absence of national preschool programs, only a relatively small fraction of Zambia children has access to early childhood care and learning prior to entering school, further increasing developmental differences generated by limited nutrition and exposure to infectious disease in the first years of children’s lives.

The research team plans to follow up with children from both the 2010 and 2012 cohort when they complete primary school to further validate the instrument and to identify the most critical aspects of child development in this context.

Gunther Fink

Smoothing the Cost of Education: Primary School Saving in Uganda

Even when there are no official school fees, the financial burden of purchasing uniforms, books, and other school supplies prevents low-income students from remaining in school. In Uganda, researchers tested whether a school-based savings program improved academic performance and reduced dropout rates by enabling students and their families to save for school-related expenses. A version of the program that labeled savings for educational purposes, rather than fully committing money to educational expenses, increased the amount students saved, expenditures on educational supplies, and test scores.
 
Policy Issue:
Although many countries in Sub-Saharan Africa have close to universal primary school enrollment, many students drop out before completing primary school or fail to continue to secondary school. While children drop out for a number of reasons, financial concerns are often an important factor. Even when governments eliminate school fees, there are still many costs associated with attending school. Providing basic school supplies such as uniforms, pens, pencils, and workbooks is often a significant challenge for low-income families. Furthermore, these families may lack access to formal savings services, making it difficult to set aside money for education. Even when families do have some savings, there is no guarantee they will use the money for educational expenditures. This evaluation assesses the impact of a school-based savings program that aims to encourage students and their parents to save for educational expenses.
 
Context of the Evaluation:
Uganda’s primary school enrollment rates have greatly increased since the government began providing free universal primary education. Retaining pupils, however, is more difficult and as few as 32 percent of children entering primary school complete all seven grades. While the government covers the cost of teachers and schools, many Ugandan primary schools require uniforms, and families are responsible for providing school supplies such as stationary and workbooks. The financial strain of buying these supplies is often too high for the family to sustain, and is cited as a major reason for children dropping out of school.
 
Description of the Intervention:
Researchers partnered with the Private Education Development Network (PEDN) and FINCA Uganda to implement and test the “Super Savers” program in public primary schools. Children in grades five through seven, the final three years of primary school, were given the opportunity to deposit money into lockboxes on a daily or weekly basis. The money was deposited into the school’s bank account at the end of each trimester. The bank accounts did not earn interest. At the beginning of the next trimester, bank representatives returned to the school to disburse the funds. On the day the funds were paid out, PEDN organized a small market at each school where students could purchase school supplies or school services such as practice exams or tutoring sessions.
 
Schools were randomly assigned to have students’ savings returned in one of two ways:
  • Voucher payout: students received their savings in the form of a voucher that could only be used to buy supplies or school services at the market set up at the school. This created a binding commitment to spend savings on educational expenditures.
  • Cash payout: students received their savings in cash, which meant they could spend the funds either at the market set up at the school or however else they chose. 
Students were notified of the kind of payout they would receive at the beginning of the program. There were 39 schools in each group, and an additional 58 schools served as a comparison group  received no savings account.
 
Half of the schools in each payout group were also randomly assigned to receive parent outreach, in which workers from PEDN hosted a workshop for sixth- and seventh-grade parents to describe the various ways they could support their children’s education and to promote the savings program as a tool to help families finance school expenditures.
 
Results and Policy Lessons:
Researchers found that students deposited significantly more when their savings were returned in cash, rather than vouchers. On average, students in schools that received cash payouts deposited between 2,200 and 2,340 Ugandan shillings, while the average student who received voucher payouts deposited between 1,120 and 1,180 shillings.
 
The purpose of the voucher payouts was to commit students to spend their savings on educational expenses. Cash payouts, on the other hand, imposed no restrictions on the use of savings, but did provide a weak commitment to spend savings on educational expenses by basing the savings program in schools and timing payouts to correspond with markets for school supplies. This weaker commitment may have appealed to students who value flexibility on how to spend their savings, while the voucher treatment’s stronger commitment may have discouraged them from saving.
 
When combined with parent outreach, students who received cash payouts were significantly more likely to have a complete set of school supplies. They also had test scores that were 0.11 standard deviations higher than the comparison group. There were no significant positive effects on school supplies or test scores among students who received cash payouts without parent outreach or among students who received vouchers, with or without parent outreach. These results suggest that combining cash payouts from savings accounts with parental outreach can lead households to spend savings on education and improve student learning.
 

Fighting Procrastination among Loan Officers in Colombia

Policy Issue:

It often hard for individuals to balance current needs against long-term goals, or to meet self-imposed deadlines without an external nudge in the right direction. Most scholarly research has focused on procrastination problems in the context of individual choices and decisions; in contrast, efficient mechanisms for fighting procrastination in firms and organizations have been little-studied. In the context of development, it is likely that microcredit banks could operate more efficiently if their loan officers spread their workload evenly over time, resulting in a more stable cash flow for the bank—but loan officers, like many people, have a tendency to procrastinate.

Context:

Bancamia, a for-profit microfinance bank in Colombia, specializes in access to finance for micro-enterprises, low- and middle-income segments, rural populations, and women. At the time of this intervention, Bancamia had close to US $160 million in outstanding loans to over 180,000 clients. Although the bank’s loan officers are paid on commission, they commonly postpone credit collection efforts and sourcing new clients until the last two weeks of each month, just before bonuses are calculated. As a result, they report exhaustion and high stress levels close to the monthly deadline. Loan officer procrastination also creates cash management problems for Bancamia, as there is a mismatch between the timing of credit repayment at the beginning of the month and credit disbursement at the end.

Description of Intervention:

Researchers partnered with Bancamia to introduce incentives for loan officers to shift their work earlier in the month under an initiative called the “Madrugador” (or “Early Riser”) program. Prior to the intervention, Bancamia’s payment structure for loan officers included a commission on new loans, with weekly targets. For every week that loan officers failed to meet their targets, they were punished with a 5 percent reduction in their potential commission. Despite this payment structure, which was already designed to spread the workload over time, loan officers were not able to keep self-imposed deadlines. As a result, most were in effect giving up part of their paycheck each month.

In contrast to the punishment-based incentive system, the intervention tested emphasized positive rewards and frequent goal reminders. Under the Madrugador program, small prizes like movie tickets or restaurant coupons were awarded weekly to loan officers who met quotas for new loans early in the month, and branch managers were asked to remind their employees about meeting weekly goals. Researchers also printed and distributed the “Madrugador Monthly Bulletin,” featuring the best performers in each participating branch and a reminder about the program’s grand prize, a free vacation for the top performer overall.

Half of Bancamia’s 61 branches were randomly selected to participate in the incentive program, while the other half served as a comparison group. After three months, the Madrugador program was expanded to include incentives for branch managers, including as well as loan officers, including small prizes if their branch met 50 percent of the set goals, grand prizes if their branch performed best overall, and acknowledgement in the monthly bulletin.

Results:

The introduction of short term goals, positive reminders and incentives led to a significant change in the way loan officers allocated their time and improved performance. The effect of these small nudges became significant, however, only when branch managers were included in the intervention. There was an average 18 percent increase in the sourcing of new loans in the first two weeks of each month and a 30 percent improvement in the achievement of target goals above the comparison group. Before the intervention, only about 15 percent of loans were disbursed during the first two weeks of the month, while one-third of loans were made during the very last week. After the Madrugador Program ended, however, loan officers began procrastinating again, and the improvement in workload distribution disappeared.   

During the intervention, loan officer compensation rose 25 percent (an increase of US$152-191, independent of program prizes) relative to the comparison group. This was not because loan officers made more loans total over the course of the month, but because the program helped them to spread their workload across the month to take advantage of the existing commission payment structure. Even though they knew they could make more money if they did not procrastinate, loan officers needed support from an organizational structure that provides short-term goals and reminders about when to exert effort in order to use their time optimally.

Bancamia’s loan officers in the treatment group also reported significantly higher job satisfaction and lower stress levels with the leveled-out workload. Because the project was highly popular with employees, Bancamia is restarting the Madrugador program on its own.

Antoinette Schoar

Improving the Design of Conditional Transfer Programs: Evidence from a Randomized Education Experiment in Colombia

Policy Issue: 
Over the past decade many developing countries have expanded primary school access, energized by initiatives such as the United Nations Millennium Development Goals, which call for achieving universal primary education by 2015. High student absenteeism is a considerable challenge, however, since families can face many barriers and opportunity costs in sending their children to school. School enrollment issues are thought to be more pronounced among girls, low income families and older children. Despite the importance of education, academics and policy makers are still far from understanding what determines whether or for how long children are educated. Conditional cash transfer programs have proven effective at improving education outcomes in some settings (notably Mexico's PROGRESA/Oportunidades program); however their impact on other goals and in other environments may be very different
 
Context of the Evaluation: 
Colombia is a relatively typical middle-income, Latin American country. Child mortality is relatively low at 21 per 1,000 births, and only 18 percent of the population lives on less than two dollars a day. Similar to many middle-income countries, school attendance in Colombia is close to 100 percent for younger children, but declines substantially after the age of 13. Average attendance is 92 percent among 15 year olds, 90 percent among 16 year olds, and 80 percent among 17 year olds. The drop is considerably faster for low-income individuals: by age 17 the attendance rate falls to 65 percent in this group. Moreover, in 2003, individuals from the bottom of the Colombian poverty index represented nearly 74 percent of those not properly enrolled in school. The primary reason cited for dropping out is the cost of education. Students must pay to enroll each year and pay for required items like uniforms, books, and supplies. Monthly education costs fluctuate between US$13 and US$22 – a relatively large expense considering that the poorest families in Bogota earn less than US$750 a year.
 
Details of the Intervention: 
In 2005, the city of Bogota established the Conditional Subsidies for School Attendance (“Subsidios Condicionados a la Asistencia Escolar”) program in an effort to improve student retention, lower drop-out rates and reduce child labor. Three different incentive structures for a new conditional cash transfer system (Subsidios) were piloted in two localities within the city:
 
Basic model: Participants received US$15 per month, conditional on the child attending school at least 80 percent of the required days that month. The total annual value of the transfer (US$150) is three times more than students’ reported average labor market earnings, and is slightly more than the average annual reported education expenditure (US$125). Students were removed from the program if they failed to reach the next grade twice, failed to meet the attendance target in two successive periods, or were expelled from school.
 
Savings treatment: This variant alters the timing of cash transfers by saving a portion of the monthly stipend on a bank account to be paid out at once. The monthly US$15, which is still conditional on an 80 percent attendance rate, is split into a direct payment of US$10 and a ‘savings’ component of US$5 which is held in a bank account. The accumulated funds are made available to families at the end of the year, just before enrollment for the subsequent grade level. This one-time payment of accumulated savings is unconditional on further attendance rates, i.e. once a child has met the attendance target in one month, the banked US$5 will be paid to the family at the end of the year regardless of attendance in future months. This structure creates a mechanism for families to save money for annual enrollment costs which might otherwise be an obstacle to students’ continuing from one grade level to the next.
 
Tertiary treatment: In addition to providing an incentive to attend school, this treatment provided incentives to graduate and then to matriculate to a higher education institution. Like in the savings treatment, in the short term, the monthly subsidy is reduced from US$15 to $10. However, upon graduating, students earned the right to receive a transfer of US$300, equivalent to 73 percent of the average cost of a year at vocational school, if they enrolled in a higher educational institution within one year.
 
Results and Policy Lessons: 
By conducting two different experiments, this research finds that changing the structure of cash transfer programs can have a significantly different effect on enrollment and attendance rates, as well as graduation, enrollment in secondary school and matriculation to tertiary institutions. The first experiment compares the ‘basic’ with the ‘savings’ treatment, while the second experiment evaluates the ‘tertiary treatment’.
 
On average, all of the designs significantly increase attendance, generating gains of 3 to 5 percentage points. Despite the reduced bi-monthly transfers, the savings and tertiary treatments increase attendance rates by at least as much as the basic treatment. However, these two non-standard treatments are superior to a basic cash transfer when considering enrollment rates at both the secondary and tertiary levels. Within secondary school, simply postponing part of the transfer increases re-enrollment by 4 percentage points compared with no significant change for students in the basic treatment. The tertiary treatment increases secondary enrollment by 3.7 percentage points. The savings and tertiary treatments also appear to succeed in increasing the matriculation to tertiary institutions. The savings treatment increases enrollment by 9.4 percentage points while the tertiary treatment proves particularly effective, increasing matriculation by 48.9 percentage points.
 
Moreover, this study identifies the specific groups for which the different interventions have greatest effects. The difference, for example, in performance of the basic and savings treatments is entirely driven by their effects on the most at-risk students. The savings treatment is especially effective at improving enrollment of the lowest income students and students with the lowest participation rates. In comparison, the basic treatment has little effect on these groups of students. The tertiary treatment performs similarly to the savings treatment with a much more significant impact on students unlikely to re-enroll. This suggests that modifications in the structure of these interventions can also help target the program by better meeting the needs of those students most likely to drop out of school.  
There is some evidence that participation in these programs may cause potentially worrisome reallocation of resources within the household. For example, such spillover effects are evident in the finding that siblings (particularly sisters) of treated students work more and attend school less than those in families that received no treatment. Such findings suggest that families may decide to concentrate resources towards the sons and daughters that have become program beneficiaries, and away from those children that did not.
 
Overall, this research shows that experimenting with the design of incentive programs may have substantial benefits in terms of the efficacy of these programs. Simply postponing some of the cash transfers to a large lump-sum paid at the time of re-enrollment increases enrollment in both secondary and tertiary institutions without reducing daily attendance – particularly so for the poorest students and those most at-risk of dropping out. Moreover, incentivizing on graduation rather than on attendance alone has proven to be particularly effective, leading to higher levels of daily attendance and higher levels of enrollment at secondary and tertiary levels.
 
Related Papers Citations: 
Barrera-Osorio, Felipe, Marianne Bertrand, Leigh Linden, and Francisco Perez-Calle. 2011. "Improving the Design of Conditional Transfer Programs: Evidence from a Randomized Education Experiment in Colombia." American Economic Journal: Applied Economics. 3(April): 167-95.

 

Marianne Bertrand

Exploring Early Education Programs in Peri-urban Settings in Africa: Summary findings from Nairobi, Kenya

Summary
Innovations for Poverty Action (IPA) performed a study of preschools in a slum of Nairobi, Kenya, in May and June 2013. The study aims to present descriptive details on the access and quality of preschools in this growing sector as part of a four-city project including similar work in South Africa, Nigeria and Ghana, launched and sponsored by the UBS Optimus Foundation.
 
Note: This is not an impact evaluation, but a scoping study in four African cities designed to support future research. You can see the full Nairobi report here (PDF), the full 4-city report here (PDF), and the main page with links to the other summaries here.
 
Data collection was conducted in the Mukuru slum area, a large industrial community in the south east of Nairobi, where an estimated 75% of children aged 3-6 live in an informal dwelling, generally a tin structure. With the aim of documenting the scale, cost and quality and preschool education in this area, 221 household surveys, 29 headmaster surveys and 32 classroom observations were conducted.
 
Findings:
 
Large preschool participation rates, even among the poorest
Preschools abound in Nairobi and can be found on many streets in slum neighborhoods. Over 80% of 4 and 5 year olds in the Mukuru area are attending preschools, with no significant gender gap. Children in the poorest quintiles still have participation rates over 70%. 
 
These high attendance figures are achieved despite the fact that 41% of the 3-6 year-olds in the area live in households with a daily income of less than $2.50 per capita. All school-related costs come to about KES 1,500 ($18) per month per child on average.
 
Many private preschool options
The preschool sector is largely dominated by the growing private school industry: an estimated 94% of preschool students in the study area of Mukuru are attending private preschools. Parents generally give a high priority to sending children to preschool for primary school preparation, and put a great emphasis on academic study starting as soon as at age 3. 
 
We also find strong evidence that parents perceive more expensive private schools as superior to low cost private schools, and private preschools to be superior to public preschools. On average, parents estimate that attending a low cost private preschool instead of a public preschool would be associated with higher educational achievement and a 33% greater income at the age of 30.  
 
The average caregiver interviewed knows of 4.9 preschools that their child could walk to, which shows the large set of options that parents have when choosing a preschool. The vast majority of private primary schools have attached preschools. Amongst the major factors caregivers consider when selecting a school are proximity, teacher quality, fee level and school test results.
 
Gaps in infrastructure and services
The observed preschool classrooms had adequate infrastructure, with basic learning materials, seating options and teacher supervision in most settings. The average preschool student in Mukuru is in a class of 27 students, with a student teacher ratio of 32:1.
 
Classroom observations revealed very few cases where students had insufficient seating. An estimated 50% the preschools attended by the children in our sample have access to electricity, 87% have latrines, 66% have a playground or open space, and 65% are enclosed by a fence or a wall. Many preschools have little in the way of health or nutritional provision.
 
Strong emphasis on academic instruction
Notably, classroom observations revealed that 100% of instruction was teacher led, where the teacher provided instruction at the front of the class to students at desks. Children are taught literacy and numeracy, are given exams, and are ranked within the class from as early as age 3. Learning goals at young ages significantly outstrip those in place in Europe or America, and the teaching style of preschools mimics that of primary schools. In contrast, education experts underline the importance of developing a wide range of skills in preschool years, with equal emphasis being placed on social development, creativity, problem solving and emotional development.  
 
There was no shortage of very basic learning materials, with an average of 100 exercise books per class. However, materials with additional content such as textbooks, storybooks, activity books, art materials or toys were generally limited or absent, which is in line with the strong academic emphasis of all preschools. 
 
Conclusion
These findings indicate overall that the low cost private schools movement, particularly developed in urban Kenya, seems to also be reaching preprimary students. Most parents are aware of the value of early education, even in very low income areas, and a large majority of 3-6 year olds are attending academically-oriented preschools. There is evidence suggesting, however, that cost remains a barrier to good quality preschools, and that preschools might benefit from improved facilities and a more diverse curriculum focused on developing a broader range of skills. 
 
IPA is eager to identify cost-effective programs successful at improving access and quality of preschool services with both public and private sector partners. For questions on IPA’s work in the early education sector, please contact Loïc Watine (lwatine@poverty-action.org).
Loïc Watine

Exploring Early Education Programs in Peri-urban Settings in Africa: Summary findings from Lagos, Nigeria

Summary
Innovations for Poverty Action (IPA) conducted a scoping study of the preprimary education sector in Agege, Lagos in August and September 2013. The study aims to present details on access to and quality of preprimary education in this growing sector as part of a four-city project including similar work in Kenya, South Africa and Ghana, launched and sponsored by the UBS Optimus Foundation. Study results show that a large number of pre-primary options exist in Agege and that a large majority of young children are attending preschools in general, and private preschools in particular.  
 
Note: This is not an impact evaluation, but a scoping study in four African cities designed to support future research. You can see the full Lagos report here (PDF), the full 4-city report here (PDF), and the main page with links to the other summaries here.
 
Study background and design
The Nigerian education system includes four years of preschool: “Kindergarten” for ages 2-3 and 3-4 and “Nursery” for ages 4-5 and 5-6.”
 
The Agege Local Council Development Area (LCDA) is an area of about 5 km2 located next to the airport in the north west of mainland Lagos, and is home to a population of over 200,000. Unlike other ‘slums’ in major African cities, most homes are made of brick or concrete. 
 
126 household interviews and 18 headmaster surveys were conducted in Agege with the aim of documenting the scale, cost and quality of preprimary education in this area. The data collected is representative of the LCDA of Agege as a whole.
 
Findings:
 
Preschool participation rates are very high, but are significantly lower for the poorest 
There are a large number of preschool options in Agege, and enrollment rates are very high. About 70% of 3 year olds go to preschool, and more than 90% of 4-6 year olds go to either preschool or primary school. All preschool children attend 5 days a week and more than 95% are at school for more than 25 hours in an average week.
 
This high participation is achieved despite the fact that an estimated 48% of 3-6 year olds live in households that are below the poverty line of 1 dollar a day. Yet, preschool participation rises with both household income and education level of adults within the household: children in the poorest quintile are significantly less likely to attend preschool (73%, versus 90-96% in the other quintiles). There is no significant gender gap on attendance rates for children aged 3 to 6.
 
Preprimary school-related costs, about half of which are school fees with the other half including mainly books and food, average about 4,400 Nairas ($27) per month per child, though this varies significantly by income group.
 
Preschools are mostly private, and most preschools are attached to a primary school
An estimated 82% of preschool students in Agege go to a private preschool.  Almost all preschools are attached to a primary school, and almost all primary schools have a preschool attached.
 
Parents have a relatively large number of options when choosing a school for their child.  The average number of preschools caregivers know of that their child could walk to is 3.6 (compared to 2.7 health centers). The major factors caregivers consider when selecting a preschool are teacher quality and proximity, though curriculum and cost are also mentioned.  
 
However, even if most caregivers do not mention cost as the number one reason for choosing a specific preschool, 49% of the children who walk to school are not being sent to the preschool that their parents consider to be the best within walking distance, most commonly due to cost.  This indicates that ability to pay still seems to act as a significant constraint when choosing a school.
 
Parents hold preschool education in high regard, particularly for private schools
Parents do seem to view preschools as educational establishments rather than just daycare centers. When asked their main motivation for sending their child to preschool, for 86% it was to learn skills and be prepared for primary school. Only 1 caregiver said that the main reason was that there was no-one at home to take care of them.
 
We also find strong evidence that parents perceive private schools to be better than public schools, and more expensive private schools to be superior to low cost private schools. Parents expect that attending a low cost private preschool instead of a public preschool would be associated with higher educational achievement and a 38% greater income at the age of 30.
 
Class sizes are small in private preschools, but the provision of facilities and services is mixed
The average number of pupils per teacher in private preschool is just 11. No school that was visited had a teacher/student ratio greater than 1:20. Over 90% of private school teachers have completed secondary school, and almost 75% have a degree, but none have ECD-specific training. 56% of private school teachers have 3 or fewer years of experience.
 
Wider school infrastructure is generally relatively satisfactory; all schools offering preprimary grades had latrines on the premise, and most also have electricity and some health facilities. A large number of schools, however, lack play materials and a playground, and most do not provide any food.
 
Teaching and learning academically oriented
School materials are geared towards an academic style of learning. All private schools claimed to have both exercise books and textbooks in all preschool classrooms. 90% also claimed to have storybooks, but fewer than half had any toys for preschool use.
 
Homework is generally set from the earliest years of preschool, and learning goals are particularly ambitious, with the average private headmaster saying that children should know single-digit numbers and the full alphabet by age 3 and a half. The majority of private preschools in the sample conduct exams, most even with the youngest age group (age 2-3).  Exams and most teaching are in English, despite the Government’s official policy that a child’s mother tongue should be the principle medium of instruction.
 
Conclusion
In Agege there is a large preschool sector with high attendance rates for both boys and girls. Most parents are aware of the value of early education and spend considerable amounts to send their children to relatively formal and very academically-oriented private preschools. There is evidence suggesting, however, that cost remains a barrier to attending good quality preschool.
 
The profusion of private schools that this study illustrates remains fairly unregulated. Supporting the improvement of teaching quality and expansion of quality preschools are key challenges for the Nigerian education system.
 
IPA is eager to identify cost-effective programs successful at improving access and quality of preschool services with both public and private sector partners. For questions on IPA’s work in the early education sector, please contact Loïc Watine (lwatine@poverty-action.org).
Loïc Watine

Exploring Early Education Programs in Peri-urban Settings in Africa: Summary findings from Accra, Ghana

Summary
Innovations for Poverty Action (IPA) performed a scoping study of preschools in Ashaiman, Accra in September and October 2013. The study aims to present details on access to and the quality of preschools as part of a four-country study including similar work in Nairobi, Johannesburg and Lagos, launched and sponsored by the UBS Optimus Foundation. The results show that a large number of preschool options exist in Ashaiman, particularly in the private sector, and that an overwhelming majority of young children are attending these preschools.
 
Note: This is not an impact evaluation, but a scoping study in four African cities designed to support future research. You can see the full Accra report here (PDF), the full 4-city report here (PDF), and the main page with links to the other summaries here.

 
Study background & design
The Ghanaian early education system includes four years of preprimary education. Children enter Nursery at age 2, after which they enter Kindergarten, and enter primary school at age 6. All children attend 5 days a week and spend on average 41 hours per week at preschool. 
 
Ashaiman is a town of roughly 200,000 residents, located 30 kilometers east of the Ghanaian capital Accra’s city center. Although Ashaiman is clearly regarded as a ‘slum area’ by Ghanaians, most dwellings are permanent structures made of bricks or concrete.
 
Data collection was conducted in Ashaiman with the aim of gathering data on the scale, cost and quality of preprimary education. In total, 286 household interviews, 30 headmaster surveys and 40 classroom observations were conducted. 
 
Findings:
 
Large preschool participation rates, even among the poorest
There are a large number of preschool options in Ashaiman, and participation rates are very high. More than 80% of 3 year olds attend preschool, and more than 90% of 4-6 year olds are attending either preschool or primary school, with no significant gender gap. 
 
High attendance is achieved across all levels of income, despite the fact that about 30% of 3-6 year olds live in households live below the poverty threshold of 2.50 dollars per capita. 
 
Options exist, but costs still a barrier
Parents have a relatively large number of options when choosing a school for their child. The average caregiver knows of 3.6 preschools that their child could walk to. 
 
The major selection factors caregivers consider in a preschool are proximity, cost and teacher quality. 60% of parents whose children walk to school are not sending their child to the school they consider to be the best within walking distance, most commonly due to cost. Preprimary school-related costs, including mostly fees, food, and books averaged $38 dollar per month per child. 
 
Parents value preschool education, particularly of private schools
Parents view preschools as educational establishments rather than daycare centers. 80% of parents said their main motivation for sending their child to school was for the child to learn skills and be prepared for primary school. Only 12% said that the main reason was that there was no-one at home to take care of them.
 
We also find evidence that parents perceive private schools to be better than public schools, and more expensive private schools to be superior to low cost private schools. An estimated 91% of preschool students in Ashaiman go to a private preschool. 
 
Content and teaching academically oriented
Materials and teaching style are geared towards an academic style of learning, with the prevalence of forward-facing desks and exercise books. There is considerable variation, however, in provision of materials within classrooms; the responsibility of buying school books generally rests with parents and within most classes a minority of pupils remain without learning materials. Toys and play materials are limited; only 26% of children attend schools that had any toys for preschool use.
 
Homework is generally assigned as of age 3.5 on average, and learning goals are very ambitious. The average ages by which headmasters consider that children should know the single-digit numbers and the full alphabet are 3.6 and 3.7 years respectively. Almost half (45%) of private schools have exams for 3 year olds and 90% have exams for 6 year olds. 
 
Gap between government plan and reality
23 of the 24 private schools in the sample said they were registered with either Ghana Educational Services (GES) or the Ashaiman Municipal Council.
 
While the Ghanaian government has recently formulated the ambitious and comprehensive “Operational Plan to Scale-Up Quality KG Education in Ghana,”classroom realities depart from this vision in two key ways:
1) Rote learning techniques have not yet been overtaken by the Government’s espoused child-centered teaching model.
2) Around three-quarters of observed classrooms used primarily English for instruction, despite the Government’s literacy acceleration plan for 90% of kindergarten instructional time to be in national languages.
 
In order to bring classroom behavior more in line with government rhetoric, GES has stipulated that teacher training is their number one priority within the preschool sector. They are currently seeking appropriate models of mass transformational training, which will target Ghana’s 27,000 untrained preschool teachers.
 
Conclusion
While Ashaiman has a large, well-attended preschool sector, private providers dominate and   evidence suggests that cost remains a barrier to attending higher quality institutions. While Ghana has advanced educational policies, classroom realities do not always reflect the government’s vision. The high numbers of untrained teachers and overwhelming usage of English to teach numeracy and literacy are of particular concern. The Government has recently published an ambitious plan to deal with these issues which, if successfully implemented, could cement Ghana’s status as a leader in preschool education services in Africa.
 
IPA is eager to identify cost-effective programs successful at improving access and quality of preschool services with both public and private sector partners. For questions on partnering on impact evaluations in Ghana specifically please contact Chase Stafford (cstafford@poverty-action.org), or for general questions regarding IPA’s work in the early education sector, please contact Loïc Watine (lwatine@poverty-action.org).
Loïc Watine

Exploring Early Education Programs in Peri-urban Settings in Africa: Summary findings from Johannesburg, South Africa

Summary
Innovations for Poverty Action (IPA) performed a study of preschools in Soweto, Johannesburg in July and August 2013. The study aimed to present details on access to and quality of preschools as part of a four-city project including similar work in Kenya, Nigeria and Ghana, launched and sponsored by the UBS Optimus Foundation. Results show that a large number of preprimary options exist and a majority of young children are attending preschool in this area. These preschools may be more oriented towards broad developmental learning rather than the academic approaches prevalent in other study areas.
 
Note: This is not an impact evaluation, but a scoping study in four African cities designed to support future research. You can see the full Johannesburg report here (PDF), the full 4-city report here (PDF), and the main page with links to the other summaries here.
 
Study background and design
In South Africa, a typical distinction is made between the 0-5 age range, for which the main involvement of the government is through subsidies of not-for-profit ECD centers, and the 5-6 age range, for which the public sector is more directly involved through grade R classes attached to public primary schools.
Given the general focus of this multi-country study on early education programs for children aged 3 to 6, this report encompasses both 3-5 year olds in pre-grade R centers, and 5-6 year olds in grade R. 
 
Soweto is located about 15 km southwest of Johannesburg, with a population of over 1 million. Given the focus of the study on the poor and high income inequality in Soweto, this study selected the 8 poorest of the area’s 46 wards. In South Africa, most urban poor actually live in formal settlement parts of townships, in part due to the large government-provided housing system. In the study area, only about one fourth of the households live in informal settlements.
 
Data collection was conducted in Soweto with the aim of documenting the scale, cost and quality of preprimary education (i.e. both preschool and grade R) in this area. In total 238 household interviews, 30 headmaster interviews and 26 classroom observations were conducted.
 
Findings: 
 
Preprimary participation rates vary across the age range and are significantly lower for the poor 
60% of 3 and 4 year olds in the study areas attend school, and more than 80% of 5 and 6 year olds. There is no significant gender gap on attendance rates for children aged 3 to 6. However, preschool attendance rises with both household income and educational attainment of adults within the household. In the poorest quintile, only about half of children go to school, as opposed to 91% in the wealthiest quintile.
 
Pre-grade R are mostly non-profit, but grade R are mostly public
71% of pre-grade R students in the study area go to a private preschool (largely NPOs) but an estimated 74% of grade R students go to a public grade R.
 
Parents have a relatively large number of preschool options, but fewer grade R options: the average number of preschools caregivers knew of within walking distance was 3.3, but only 2.1 grade R options. 
 
The major factors caregivers consider in selection are proximity and cost, though teacher quality, curriculum and facilities are also mentioned. 
 
Preprimary school-related costs average just above 500 rands ($51) per month per child. The government’s child support grant is being claimed for 74% of the children attending preprimary in our sample, but little other financial assistance is available from schools or other bodies. 
 
Parents have a high view of preprimary education
Parents seem to view preschools as educational establishments rather than just daycare centers; both preschool and grade R are viewed as important investments in a child’s future. Respondents estimated that relative to children who had not attended preprimary education, earnings at age 30 would increase by 75% for attending grade R and preschool, and 56% for just attending grade R.
 
We also find strong evidence that parents perceive more expensive private schools as superior to low cost private schools
 
Provision of facilities and services is generally good, though uneven
Of the preschool classrooms observed there was decent infrastructure with basic learning materials, play materials, and teacher supervision in most settings. Wider school infrastructure was also generally acceptable. All schools had latrines and a fence. All except one had at least some electricity and all but two had a playground.
 
All but one of the schools in our sample provided at least one meal per day to students (and over 90% provide both breakfast and lunch). Most schools also had first aid kits and immunization programs.
 
According to their headmasters, 85% of teachers in our study area have completed an ECD-specific training course. This breaks down into 96% of public sector teachers and 79% of private sector teachers. 
 
Less academically-oriented style than other Sub-Saharan countries 
Classrooms appeared geared towards learning-through-play rather than formal academic styles. Children spent considerable time playing on the floor or outside, and around half the preschool classrooms had no desks, chairs or uniforms. Schools generally had a decent number and variety of toys available.
 
Grade Rs are larger and more academically oriented than preschools. Children in grade R have more learning materials, are more likely to be sitting in rows facing the front, and tend to have better qualified and experienced teachers. These differences may reflect both a difference in age range and also the fact that grade Rs are predominantly public, while pre-grade R are predominantly private.
 
Registered schools are better resourced
Of private preprimaries in the study, 62% were registered with the Department of Social Development and/or the Department of Basic Education. Registered schools tended to be larger, have higher teacher salaries, have more learning materials and receive more financial support.
 
Conclusion
The preprimary education sector in Soweto is large and relatively well-attended, though the market may not be saturated. Most parents are aware of the value of early education, and a majority of 3-6 year olds are attending preprimary schools. However, Children from the poorest backgrounds are significantly less likely to attend school and, if attending, are more likely to be attending schools with poor resources.
 
IPA and its partner the Abdul Latif Jameel Poverty Action Lab (J-PAL Africa), are eager to identify cost-effective programs successful at improving access and quality of preschool services with both public and private sector partners. For questions on partnering on impact evaluations in South Africa specifically, contact J-PAL Africa (jpalafrica@povertyactionlab.org) at the University of Cape Town. For questions regarding IPA’s work in the early education sector please contact Loïc Watine (lwatine@poverty-action.org).
Loïc Watine

The Sisters of Success Mentoring and Girls’ Groups in Liberia

As girls pass through adolescence, a number of factors influence whether they complete secondary school, avoid teenage pregnancy, and develop the life skills, attitudes, behaviors and relationships that will set them on a path to a healthy and productive adulthood. This evaluation investigates whether being part of a mentorship and life skills program, “Sisters of Success,” during early adolescence improves outcomes for girls in Liberia’s capital city, Monrovia.

Policy Issue:

Adolescent fertility rates in Sub-Saharan Africa are substantially higher than other regions of the world, with 115 births per 1,000, compared to 72 births per 1,000 in Latin America and just 19 births per 1,000 in Europe.[1] The gender gap in education is also significant, with West and Central Africa having the largest gender gap in education of all regions in the world.[2] Increasing the number of girls who complete secondary school, and reducing early motherhood, are common policy goals across many countries in Sub-Saharan Africa. This evaluation will contribute evidence to policymakers on effective programming to reduce school drop-out and teen pregnancy. Secondly, Liberian policymakers, and NGOs working with Liberia, have noted that life skills are fundamental to individuals’ labor market success, but there is little evidence on the impact of life skills training, or the impact of enhanced life skills on real world outcomes. One factor behind this evidence gap is that life skills training is typically delivered together with vocational training, credit, or even cash transfers, thus making it impossible to isolate the impact of the life skills training itself. This evaluation will help fill this evidence gap.

 
Context of the Evaluation:

In Liberia, the adolescent fertility rate is 117 births per 1,000 girls, ages 15-19.[3] Meanwhile, within Liberia, the gender gap in school attendance is high. Only 60 percent of girls complete primary school in Liberia, compared to 71 percent of boys, and 19 percent of men have completedsecondary school or higher, but only 8 percent of women have accomplished the same.[4]

The Sisters of Success (SOS) programis taking place in an urban area of 1.1 million people. The SOS program’s goals are for girls to adopt healthy behaviors, build confidence and self-esteem; learn and practice their rights; begin to develop savings and financial literacy habits; increase their community participation and involvement; and help them work towards their own personal development goals, among others. SOS is funded by Nike, and coordinated by the International Rescue Committee (IRC), in partnership with two local organizations, EDUCARE, and Planned Parenthood Association of Liberia (PPAL), and volunteer mentors, drawn from the same communities as the girls they mentor.

 
Details of the Intervention:

Researchers hope to find out whether the Sister of Success mentoring program improves specific outcomes for adolescent girls, including their likelihood of staying in school or returning to school, and their likelihood of engaging in risky sexual behavior and becoming pregnant as minors. The study will also deliver evidence on the characteristics of girls for whom the program is more or less effective.      

Sisters of Success will recruit and match girls ages 12-15 with mentors. Each mentor will be randomly matched with ten mentees. Approximately 2,880 girls will participate in the randomized evaluation, with half becoming mentees and half serving as a comparison group.

SOS mentors and mentees meet in “sisterhood sessions,” comprised of two mentors and 20 mentees, which meet twice a month over the course of 15 months. The program also includes extracurricular activities in which larger groups of mentors and mentees do activities together. SOS mentors, who are unpaid, are intended to serve as trusted individuals, friends, advisors, coaches, guides, teachers, and role models for the mentees.

Researchers will collect data on a wide range of topics, such as who and what influences girls to leave school; the social and economic factors that influence when girls first have sex and birth their first child, the number and type of partners girls choose, and their use of contraception. The study will also measure the impact of the SOS program on girls labor market activities and earnings. In addition, researchers will evaluate the relative cost-effectiveness of the SOS program as compared to other policy options. Finally, if the program is effective, this study will pinpoint the key mechanisms that make an impact on girls’ outcomes.

 
Results and Policy Lessons:

Results forthcoming.

 


[1] Population Reference Bureau. “Trends in Adolescent Fertility A Mixed Picture.” http://www.prb.org/Publications/Articles/2013/adolescent-fertility.aspx

[2] http://www.ungei.org/gap/reportWafrica.html

[3] World Bank. Adolescent Fertility Rate. Available at:  http://data.worldbank.org/indicator/SP.ADO.TFRT

[4] UN Development Group. “Fact Sheet: Empowering Women in Liberia.” Available at: http://www.undg.org/docs/11143/genderemail.pdf

Returns to Apprenticeship Training in Ghana

Youth unemployment—an acute problem in Sub-Saharan Africa—is overwhelmingly considered to have long-term negative implications, both on individuals’ quality of life and on broader socio-economic development outcomes. This study assesses the impact of a government apprenticeship program in Ghana that pairs young people who have limited education with master trainers operating small businesses for a one-year training period. Researchers are evaluating the targeting and recruitment process, the benefits for apprentices and trainers, the program’s cost-effectiveness, and to what extent performance-based incentives increase the quality of training apprentices receive. Results from this study will assist policymakers in Ghana to formulate sound policy to address youth unemployment. 

Policy Issue:

Youth unemployment is an acute problem in low-income countries, and especially in Sub-Saharan Africa.[1] Young people account for 60 percent of the unemployed in Sub-Saharan Africa, and 72 percent of young people between the ages of 15 and 24 live below the $2 a day poverty line.[2] A lack of necessary skills is often cited as contributing to high unemployment. Job training programs, and in particular apprenticeship training with private-sector informal firms, could expand labor market opportunities for young people by providing them with relevant on-the-job experience and market-ready skills.

Despite their promise, program evaluations of job training programs around the world have been disappointing and job training has been shown to provide mediocre labor market returns on average. One reason for these lackluster effects may be wide variability in the quality of training. In Ghana, as in other poor countries, productivity (and likely, training ability) varies greatly across businesses. In addition, though non-institutional, on-the-job training is otherwise believed to be the most effective option, it potentially introduces new issues where the incentives of training providers may be misaligned with those of trainees or the government agency implementing the program. In particular, apprentices provide low-cost labor to businesses via their on-the-job experience, and this dual role could incentivize training providers to seek to retain apprentices at a skill level that is profitable for the business but makes them less than employable elsewhere.

Context of the Evaluation:

In 2012, 26 percent of young people in Ghana between the ages of 15 and 24 were unemployed.[3] The skills deficit in Ghana is partly driven by an education system in which a large number of students fail to progress beyond junior high school. Access to senior high school is based on performance in national examinations and places are often limited; in 2011/2012 only 50 percent of qualified students transitioned to senior high school, and the gross enrollment ratio for senior high school was only 37 percent.[4]  The gross enrolment ratio in public technical and vocational training institutions was just 3.5 percent. Limited capacities at government schools and public and private training institutions combined with costly fees prevent many young people from furthering their education and improving their skills.

The National Apprenticeship Program (NAP) is a training program initiated by the Council for Technical and Vocational Education and Training (COTVET), and implemented by district-level coordinators of the Ghana Education Service, in close partnership with craft-specific trade associations. The program is based on the traditional three-year apprenticeship model, which is extremely widespread in Ghana, except that the NAP training period has been shortened to one year in order to reduce the amount of time before apprentices enter the labor market. Similar to the traditional apprenticeship model, COTVET has committed to paying master trainers and to providing a toolkit to each participating apprentice.

The program is mostly targeted at young people who are unable to continue their education beyond Junior High School. Recruitment of both trainees and master trainers  was conducted at the district level.

Details of the Intervention:

To estimate the returns of the apprenticeship program, and to examine how it can work most effectively, researchers are implementing a randomized evaluation in 32 nationally representative districts across all 10 regions of Ghana. Researchers assigned a random half of the 4,601 applicants to participate in the apprenticeship program. Treatment apprentices were then randomly assigned to master trainers’ businesses that matched their trade and geographic preferences. The businesses and their owners have a wide range of characteristics to enable researchers to measure if there are certain business-level characteristics associated with higher quality training (as measured by self-reported progress throughout the program, and apprentice attendance, retention, and completion) and higher labor market returns upon completion.

Additionally, a random half of master trainers will be offered cash incentives to encourage them to instruct their apprentices well. This additional cash incentive program is designed to encourage trainers to ensure a proficient skill level within the specified time period.

Results and Policy Lessons:

Results forthcoming.


[1] Africa Development Forum (2014). Youth Employment in Sub-Saharan Africa. Washington D.C.:  The International Bank for Reconstruction and Development/The World Bank. [Online].  Available at: <http://documents.worldbank.org/curated/en/2014/01/19342178/youth-employment-sub-saharan-africa-vol-2-2-full-report>. [Accessed: 8th May 2014].

[2] World Bank (2009). Africa Development Indicators 2008/09. Youth and Employment in Africa: The Potential, the Problem, the Promise. Washington D.C:  World Bank. [Online].  Available at: <https://openknowledge.worldbank.org/handle/10986/12350>. [Accessed: 8th May 2014].

[3] African Economic Outlook (2012). Ghana 2012 Paris: OECD[Online]. Available at: http://www.africaneconomicoutlook.org/fileadmin/uploads/aeo/PDF/Ghana%20Full%20PDF%20Country%20Note.pdf. [Accessed: 25th June 2014].

[4] Ministry of Education (2012).  Education Sector Performance Report.  Accra: Ministry of Education. [Online].  Available at: http://www.ndpc.gov.gh/GPRS/Dist%20and%20Sec%20APR%202012/SECTOR%202012%20APRs/Ministry%20of%20Education.pdf. [Accessed: 25th June 2014].

Isaac Mbiti

Math in My School: Improving Basic Math Skills in Paraguay

Basic “pre-math” skills in young children have been shown to be important for developing later mathematics competency. In Paraguay, where math scores are lower than other Latin American countries, and where there is great variation in math abilities among young children, the government adapted a pre-math curriculum into audio lessons in kindergarten classrooms. Results showed an increase in math scores and narrowing of gaps between many demographic groups. 

Policy Issue:
An early basic understanding of numbers, “pre-math,” in young students has been shown to be an important foundation for later math learning.1 Tests show that students in Latin America lag behind the rest of the world in math and science skills, and within Latin America, Paraguayan students consistently underperform, with more than half of third-graders not achieving basic grade-level competency.Within Paraguay, large performance gaps exist between boys and girls, between schools in different geographic areas, and between schools with multi-grade classrooms and classrooms devoted to single grades. The government hopes to find a program to improve early understanding of numbers and basic math while closing the existing gaps between different socio-demographic groups.
 
Context:
The Cordillera region of Paraguay, located just east of the nation’s capital, reflects the diversity of the country. In Cordillera, like in much of the country, people speak Spanish, Guaraní, or a mix between the two called Jopará. Schools range from instructing in Spanish completely to not at all and the number of students in kindergarten classes ranges from one to over 40. Generally, students in small, rural, Guaraní speaking schools have lower access to educational resources and test lower than their peers, whereas students in larger, urban and Spanish speaking schools test higher. The data also showed a significant gender gap between boys and girl’s math abilities, and 90 percent of surveyed teachers responded that they were unable to teach all topics in the preschool math curriculum.
 
Details of the Intervention:
The Paraguayan government, in an effort to increase math abilities and close achievement gaps, reviewed a series of math initiatives around the world and chose to adapt New York City’s Big Math for Little Kids program. The Ministry of Education and Culture and Inter-American Development bank adapted the curriculum to the Paraguayan context and languages, and turned it into audio lessons on CD so that it could be used in classrooms regardless of variation in individual teachers’ preparation or pedagogical skill. The program is called Tikichuela: Mathematics in My School, and teachers were trained to play the 30-40 minute lessons and in tutoring for the curriculum. The lessons were played in class four days a week, with the fifth day for the teacher to review or provide extra assistance based on observed needs. Lessons were also implemented using complementary materials such as student workbooks with exercises instructed by the characters in the lessons, along with physical objects for counting and other illustrative material to help assimilation of mathematical concepts. The Tikichuela program contrasts with the traditional curriculum which previous research has found uses long lectures focused on rote memorization of basic concepts, without attention to children internalizing an understanding of how mathematics concepts work in everyday life.3
 
To evaluate the new program’s effectiveness, all 265 schools in the Cordillera region with at least one single grade kindergarten (i.e. not combined with other grades) were randomized, with 131 schools assigned to receive the program and 134 continuing on as normal, to serve as the comparison group. At the beginning of 2011 a baseline survey was conducted of the students in both sets of schools evaluating basic numeracy and early math skills, along with other areas such as reading and writing, which the program could conceivably also affect. Teachers, principals and parents also answered questions on socio-demographic information for students and schools, and data was supplemented with a qualitative evaluation. In 2012 the same evaluations were conducted on the new students in the same classrooms, and a one year follow up survey was given to the original students.
 
Results and Policy Lessons:
The 2011 implementation lasted a relatively short five months, but in that time students who received the Tikichuela program increased their performance 0.16 standard deviations in mathematics skills, an effect equivalent to moving from the 50th percentile to 57th percentile, relative to the comparison group. The effects of the program were strongest for those in schools with lower initial test scores – smaller, rural, and Guaraní-speaking schools. It benefitted the classrooms and individual students who tested lower originally, those in the bottom three quartiles at the baseline improved relative to those in the comparison group, while those in the top quartile showed no change.
 
The program raised math scores for both boys and girls, but boys benefitted more. Boys improved 0.21 standard deviations (about 9 percentile points), and girls 0.13 standard deviations (about 6 percentile points) over the comparison group. In light of this finding, the second year of implementation adjusted the program and preliminary analysis suggests the gender gap was closed. Analysis of the second year data is ongoing.
 

[1] Geary, David C., Mary K. Hoard, Lara Nugent and Drew H. Bailey. 2013. “Adolescents’ Functional Numeracy Is Predicted by Their School Entry Number System Knowledge.”  Plos One 8 (1): 1-8.  
 
[2] MEC (Ministry of Education and Culture). 2010. “Informe preliminar SNEPE.” Asunción, Paraguay.
 
[3] Emma Naslund-Hadley, Ernerto Martinez, Armando Loera and Juan Manuel Hernández-Agramonte, 2012. El Camino hacie el éxito en matemáticas y ciencias: Desafios y triunfod en Paraguay. Banco Inter-Americano de Desarrollo (BID).
Susan Parker

Evaluating the Efficacy of School Based Financial Education Programs

Could financial literacy training for children lay a foundation for good financial decisions and a better quality of life in adulthood? If so, what type of training works best? In this study, IPA partnered with Aflatoun, a Dutch non-governmental organization, to evaluate the impact of two forms of financial education on primary school children across Ghana. 

Policy Issue:

Research on financial knowledge and behavior indicates that individuals in both developed and developing countries around the world lack adequate knowledge to make informed financial decisions. In response to evidence that financial literacy is correlated with well-being, many service providers, donors, and policymakers have begun including financial training and business education as part of their broader anti-poverty strategies. Intuitively, financial education provides useful tools to people of all ages, yet empirical evidence for this impact is meager and often mixed. This project tests two financial education curricula for primary school students. Specifically, it measures the impact of financial education on student behavior attitudes, and outcomes.

Context of the Evaluation:

Saving and finances are part of daily life for many youth, yet traditional school curricula often overlook the specific issues and challenges students encounter with money. This curricular gap represents a missed opportunity for students and teachers. Aflatoun, a Dutch non-governmental organization providing social and financial education to 540,000 children in 33 countries, operates a voluntary after school club in Ghana for primary and junior high schools. Aflatoun uses a uniquely designed “social and financial education curriculum” to improve children’s saving habits as well as financial attitudes and self-esteem. Aflatoun’s training on handling money, saving on a regular basis, and spending responsibly aims to teach children, at a young age, lessons and behaviors that they will carry with them throughout their lives.

Aflatoun operates in collaboration with local partners to implement its programs. Two project partners in Ghana - the Women and Development Project (WADEP) and the Netherlands Development Organization (SNV) - trained instructors and managed program implementation. SNV Ghana worked with three other implementing partners in two regions to train teachers and monitor the implementation of clubs: Berea Social Foundation (Western Region), Support for Community Mobilization Projects and Programs  (Western Region), and Ask Mama Development Organization (Greater Accra Region).

Details of the Intervention:

The study included 5,000 primary school students aged 9 - 14 in 135 public schools in semi-urban and rural Ghana, including 30 schools in Greater Accra, 60 in Volta, and 45 in Western District. One-third of the schools in each region were randomly assigned to each of three different groups: the Aflatoun program, Honest Money Box (HMB) intervention, or a comparison group without treatment.

The Aflatoun curriculum includes lessons about planning, budgeting, saving, proper spending, as well as self-esteem building exercises. It uses songs, games, and worksheets, which put children at the center of the learning process. Aflatoun also adapts its messages and activities to the context of the countries in which it operates, focusing on cultural heritage and community in order to foster a collective sense of empowerment among participant children. The HMB intervention, in contrast, is solely focused on financial education and is designed to provide a comparison for Aflatoun’s unique social and attitudinal curriculum. IPA developed the HMB intervention as a group savings scheme with a financial literacy curriculum. Some of the topics covered in the curriculum include: What is Money?, Saving and Spending, Planning and Budgeting, and Entrepreneurship, as well as lessons in how to use the Money Box, a lockbox that stores group savings.  

To implement the two programs, local partner organizations trained approximately 200 teachers (two teachers in each selected school). Teachers instructed two multi-grade clubs, with an average of 54 students per club, and delivered the assigned curriculum, in addition to providing a secure storage space for the money saved, generally in the teacher’s locked office. Clubs met, on average, once a week after school at a time decided by the members. Students saved money from their pocket change and recorded transactions on individual passbooks. IPA and partner organizations monitored the teachers to ensure that implementation met pre-determined standards.

The evaluation was conducted over the course of one school year.  Between 20 and 40 children per school were chosen to be surveyed.. The baseline survey  was conducted in September 2010 and the endline in August 2011. The surveys collected data on financial well-being of students and their families, cognitive function, and perspectives on savings and time and risk preference. The endline survey captured the same information as the baseline, in addition to a financial education endline assessmentand a psychosocial module to understand students’ outlooks and levels of self-control.

Surveys: 

The surveys used are available here in .doc format:

Baseline Student Survey

Endline Student Survey

Aptitude Assessment

Shop Games

Results:

Presentation from the “Impact and Policy Conference” in Bangkok, September 2012.

Analysis is ongoing, results forthcoming.

 

 

Teaching Science and Environment in Peru

Students in Peru have historically ranked poorly in math and science in comparative tests across South America. The Peruvian government is seeking to test a new science curriculum which moves away from traditional rote memorization to hypothesis testing and inquiry. In two pilots in Lima Province, a random sample of classrooms’ teachers received training in a new style of teaching, and the classrooms were provided with hands-on materials to work with. Data from the first pilot suggest an improvement in one of the science modules, but only for boys who were already in the higher end of the distribution on the baseline test. Data from the second pilot, designed to improve implementation and provide further evidence, is forthcoming.
 
Policy Issues:
In recent years the Peruvian education system has vastly improved its coverage, reaching populations that previously had no access. Unfortunately this increase in coverage has not resulted in improvements in educational quality, with tests in South America such as PISA and SERCE ranking Peruvian students low in math and science.1Additionally, science teaching in Peru relies largely on memorization, rather than critical thinking or practical exercises.
 
The Peruvian Ministry of Education is seeking to strengthen their national plan for basic environmental and natural sciences education. To this end, they worked with the Inter-American Development Bank on randomized evaluations in 2010 and 2012 for a new inquiry-based science curriculum (implemented by the Instituto Von Braun in 2010 and by the Universidad Cayetano Heredia in 2012). The studies examined the effects on student achievement of a new science curriculum for Lima Province public schools based on critical reasoning and hypothesis testing.
 
Context:
The intervention took place in Lima province, with students representing both the “sierra” mountain and “costa” coastal regions. Classrooms were sampled from urban, peri-urban, and rural areas, and students tended to be from low income families, with about a third of the students also reporting they worked on a family’s farm or microenterprise during the school year.
 
Description of Intervention
The studies looked at the effects of a new style of teaching based on scientific reasoning with a first pilot in 2010, followed by a 2012 pilot designed to replicate it and improve implementation. The new curriculum used experiments and practical activities in class to teach scientific reasoning (formulating hypotheses, testing them and analyzing results critically) and encouraged students to discover fundamental concepts of the lessons through hands-on scientific examination. In the first pilot, a set of LEGOs was distributed for classroom exercises, and classrooms were encouraged to grow vegetable gardens if materials were locally available. The new curriculum also introduced science fairs at the end of the year for students to showcase the experiments they had conducted, sometimes using recycled materials.
 
4,634 third-grade students in 104 schools from around Lima Province participated, data came from tests covering science (the 3 main topics covered by the traditional science curriculum: “the human body,” “the environment,” and “physical world”), reading and writing abilities, and mathematics. Surveys of school administrators measured characteristics of each school, and surveys of teachers collected data on individual students, as well as any changes in their own perceptions of how to best teach science. Classrooms were also videotaped for analysis of how well each classroom implemented the new style, and additional analysis of differences in teaching styles across classrooms and regions.
 
The 2012 study sought to improve on implementation of 2010 study and was implemented by Universidad Cayetano Heredia (UCH). In this second pilot, teacher training was implemented more uniformly, and intensified, with weekly training in groups and classroom visits focused on pedagogy and implementation of the classroom exercises.
 
Results
Results are currently available for the 2010 pilot, which was not implemented completely because of logistical problems. A positive impact was found on test scores for boys only in one of the three modules of the science test, the “physical world” module, with an increase of 0.18 standard deviations among students in the classrooms with the new teaching style. Furthermore, the effect was concentrated among boys who were above the median at baseline. The 2012 follow-up included a qualitative evaluation to examine these differences in impacts in more detail.
 
The 2010 intervention was only implemented partially and appears to have been only partly successful in achieving its intended goal. The 2012 follow up was designed to examine the effects of the full program and in greater detail.
 
Read more about the findings and recommendations for future policies in a paper for the Inter-American Development Bank here.

1Citation: OCDE (2010), Pisa 2009 Results: What Students Know and Can Do - Student Performance in Reading, Mathematics and Science (Volume I). SERCE (2008), Primer Reporte.

After-School Girls Empowerment and Satellite-Transmitted Classes in Ghana

Recent efforts to increase primary school education enrollment in developing countries have been extremely successful, yet major challenges persist in improving educational outcomes. In sub-Saharan Africa, high drop out rates, especially for girls, as well as student and teacher absenteeism are major impediments to learning. Many students, especially girls from low-income families in rural areas, miss so much school that they become chronic repeaters. This study assesses the impact of a program that aims to improve student retention and learning outcomes for marginalized pupils in Ghana through distance learning and an after-school girls’ empowerment program.

Policy Issue:
Like many sub-Saharan African nations, Ghana is facing significant challenges in its educational system. One study revealed that teachers in Ghana only spend 38 percent of school hours “engaged in learning.”[i] A driving factor behind diminished learning time is the high rate of teacher and student absenteeism. Teacher absenteeism averages 27 percent annually and as many as 40 percent of children may not be present at school on a given day.[ii]  Students from low-income households miss school the most, and achievement for these students is especially low. Many poor children also repeat, so more over-age children are poor children.[iii] Girls, especially those from low-income families, drop out of school at a higher rate than boys, with early pregnancy playing a large role.[iv] Though almost the same number of boys and girls enter primary school in Ghana, only two girls for every three boys complete high school.
 
The Making Ghana Girls Great (MGCubed) program, implemented by the Global Education Management System (GEMS), is based on the premise that diminished learning time, due to high rates of teacher and student absenteeism, contributes to low learning levels, and that a lack of female role models contributes to a high drop out rate for girls.
 
The program introduces distance learning and after-school girls club called the Wonder Women, both led by facilitators from the community, most of them female. Researchers will evaluate the impact of the interactive distance learning classes, transmitted via satellite, and the after-school program on learning outcomes. The program targets marginalized boys and girls aged 7-14 who are currently enrolled in school, and girls who have dropped out of school or never attended school. The marginalized pupils are defined as those who are above the average age for their class, those who have more than five siblings, those who walk more than 30 minutes to reach school, or those who have a history of not attending school.
 
This evaluation aims to assist policymakers in identifying interventions that ensure optimum retention, increase enrollment, and improve learning for marginalized students, and for girls in particular.
 
Details of the Intervention:
To measure the impact of the MGCubed program on learning outcomes and student retention, IPA will evaluate the distance learning and Wonder Women programs over a three-year period, measuring any changes in the test scores, stated aspirations, and confidence levels of participants over time.
 
The sample size is 147 schools, of which 70 will receive interventions and 77 will serve as the comparison group. Within the 70 treatment schools, 80 studentswill receive the in-school component, and 40 of those students who are girls, plus 10 out-of-school girls, will participate in the after-school Wonder Women program. All students are randomly selected from lists of marginalized pupils from each school.
 
The two treatment groups are, therefore, as follows:
 
1.     Students who receive in-school distance learning lessons only. Forty students at all treatment schools will be taught English and math lessons for two hours per day.
 
2.     Students who receive the in-school distance learning lessons and the after-school girls’ club called Wonder Women. Forty female students at each treatment school will be taught English    and math lessons for two hours per day and will also participate in the after-school program. Ten out-of-school girls will also be invited to participate in the program.
 
The in-school distance-learning classes, which are interactive and transmitted via satellite, provide the consistent presence of a remote teacher, digital content, and a student-centered pedagogical design. It is designed to improve the quantity of instructional hours students receive, as well as the quality of the instruction.
 
The after-school program will run two hours per day for four days of the week. While this program also aims to increase the instructional input hours received by the students, its aim is to empower the marginalized girls to have greater confidence in themselves and to stay in school, or for those who have dropped out, to re-enroll.
 
Trained facilitators will be recruited from teaching staff or from secondary school graduates in the community, with a preference for women facilitators, to lead both distance-learning program and the Wonder Women program.
 
The intervention is taking place in regions that Ghana Education Services has recognized as regions with both known educational needs and operational access. The specific districts are Ada East, Ada West, Shai Osudoku and Ningo Prampram, in the Greater Accra region, and Kadjebi and Nkwanta South in the Volta Region.
 
Data to be collected include test scores, quantitative data on students’ background information, goals and confidence level for girls.
 
 
Results and Policy Lessons:
Results forthcoming.
 
 
 

[i] Abadzi, Helen. "Instructional time loss and local-level governance." Prospects 37.1 (2007): 3-16.
 
[ii]  "What Works in Girls’ Education in Ghana." Prepared by Camfed Ghana. January, 2012. P.8 
 
 
[iv]Gyan, Charles. "The Effects of Teenage Pregnancy on the Educational Attainment of Girls at Chorkor, a Suburb of Accra."Journal of Educational and Social Research 3, no. 3 (2013): 53.
 
Christopher Ksoll

Returns to secondary schooling in Ghana

By offering means-tested secondary school scholarships to youth in Ghana, and comparing school enrollment and longer-term outcomes in health and employment between scholarship winners and a similar group of youth not offered a scholarship, this study aims to provide evidence on the barriers to enrollment and estimate the longer-run returns to secondary education.  Preliminary evidence suggests that the secondary school completion rate will be at least 30 percentage points higher among scholarship recipients, suggesting substantial financial barriers for Ghanaian youth who gain admission into secondary school but fail to enroll.

Policy Issue:

Over the past decade, many developing countries have expanded primary school access, energized by initiatives such as the United Nations Millennium Development Goals, which call for achieving universal primary education by 2015.  As some progress has been made towards this goal, an important question for policy-makers is emerging: how quickly should access to secondary education expand? Although human capital is considered to be an important driver of growth and development, and the role of primary education has been well studied and understood, there is very little evidence of the benefits of secondary education. It is sometimes hypothesized that secondary education could have a much larger impact than primary education on long-run earnings, health, fertility, gender equality, and civic and political participation. However, expanding secondary education is significantly more expensive than providing free primary education.

Evaluation Context:

While primary school completion rates have boomed in recent years, secondary school enrollment across sub-Saharan Africa has remained low, at 34% in 2007 according to UNESCO1. The direct costs of fees and materials (around US$350 over four years in Ghana) as well as the opportunity costs to families of taking youth out of the workforce may be contributing factors to low secondary school enrollment.

Description of the Intervention:

The sample for this long-term research consists of a cohort of 2,068 students (evenly split between males and females) who earned admission into a 4-year secondary school (“senior high school”) but had not enrolled by the Fall 2008 due to financial constraints. Out of these 2,068 students, 682 students were selected (by lottery) to receive a scholarship that covered 100% of the tuition and fees at a local public senior high school. The scholarships were announced during the 2008/2009 academic year and over 75% of scholarship winners enrolled in senior high school that year. Most of them are expected to graduate in June 2012. The goal is to compare the outcomes of those offered the scholarship (the treatment group) with those who did not win the lottery (the comparison group) for at least 10 years in order to estimate the impacts of lowering the financial barriers to secondary school enrollment and the returns to secondary education.

At the beginning of the study, a baseline survey was administered to all participating youth and their guardians. At the time, the youth were 17 years old on average.  Study participants were given a cell phone, and once a year, we attempt to reach all respondents over the phone in order to update their contact information and ask for their current schooling status and location. If they cannot be reached over the phone, we attempt to find them in person by going to their home area.

The first of three extensive follow-up surveys will be administered to the entire sample in the Fall 2012, when most study participants will be out of school. Subsequent follow-ups will be administered in 2015 and 2018. Follow-up surveys will include, among others, modules on labor market outcomes, health, marriage and fertility, time and risk preferences, technology adoption, civic participation, etc.

Preliminary Results and Policy Lessons:

As follow-up data has not yet been collected, lessons on the impact of increased access to secondary education cannot yet be made. Preliminary results concerning educational attainment are the following:

Among those who did not win a scholarship (the comparison group), 20% enrolled in senior high school in the school year 2008/2009. The enrollment rate among them had risen to 38% by the end of the 2010/2011 school year. This rate was higher among boys (44%) than girls (34%). Part of the lower enrollment rates among girls comes from the fact that a subset of girls in our study sample had been out of school for more than a year already at the time the study started.

Among scholarship winners (the treatment group), the enrollment rate was 75% in the school year 2008/2009, almost four times that in the comparison group. Three years later, the enrollment rate was still twice as high among those that received scholarship compared to those who did not, at 73% overall (81% among boys and 64% among girls).

HIV Prevention Among Youths: Evidence from a Randomized Controlled Trial in Kenya

Policy Issue: 

The vast majority of new HIV infections occur in sub-Saharan Africa, where nearly 2 million people become infected with HIV/AIDS every year. Forty-five percent of these new HIV infections occur among people under 25 years old, and nearly all of them are due to unprotected sex. Ensuring the adoption of safer sexual behavior among youth is critical to keeping the new generations free of HIV.

The objective of this project is to examine, through a large randomized controlled trial, the impact of two HIV prevention strategies among a youth population in Kenya. The two strategies to be tested are: Voluntary Counseling and Testing for HIV (VCT), and condom distribution.

  • VCT is a critical entry point for access to HIV/AIDS treatment and care, and is being scaled up in many countries. But VCT could also be a powerful prevention tool. By providing personalized counseling as well as information about high-risk behaviors, VCT could motivate people to adopt safer sexual behavior and prevent transmission of HIV. This could be particularly important for adolescents and young adults, who typically have had their sexual debut but might not have perfect information about HIV risk. They are often still HIV-negative, and might be better able to change their sexual behavior.

  • Despite strong evidence of the biological effectiveness of the male condom as an HIV prevention strategy, condom use continues to remain low in many countries. Several factors, such as low availability, cost, lack of education about condoms and how to use them, and relationship factors contribute to low usage.  This study examines whether free and easy access to a large quantity of condoms can result in a reduction of risky behaviors and a decline in transmission of STIs among youth.

Context of the Evaluation: 

Kenya has the 10th largest HIV infected population in the world – nearly 7% of Kenyans are infected.1 The study is being implemented in four districts of Kenya’s Western Province (Butere, Mumias and Bungoma South and Bungoma East), spanning an area of approximately 50,000 square kilometers. Since about 45% of all new infections worldwide occur in youth aged 15-24 years, this study focuses on young people (both men and women). The sample is composed of approximately 10,000 youths (half of them female) who were 17 to 22 years old in 2009.

Despite the expanding implementation of VCT, an estimated 80% of Kenyans living with HIV are unaware of their status. Take-up of VCT in traditional settings (such as government health centers) is low. As such, several alternative models of VCT service provision, including mobile VCT, workplace VCT and home-based VCT are being explored. This study has used both mobile VCT and VCT within homes, and has achieved a very high take-up (85% of the people who were offered VCT accepted it).

Condom usage in Kenya is also relatively low. Only 24% of women aged 15-49 who reported multiple partners in the last 12 months used a condom during their last sexual encounter. Despite significant efforts to increase availability of free male condoms, recent data suggest that condom distribution remains low, with on average 0.71 condoms distributed per eligible person per year.

Details of the Intervention: 

A detailed baseline survey was administered to 10,420 youths (about ½ of them girls) between March 2009 and July 2010. All respondents were tested for Herpes (HSV-2) and for HIV (via anonymous linked testing) during the baseline. The prevalence rates for HSV-2 and HIV were 8.5% and 0.5%, respectively.

Among those surveyed at baseline, 25%  had been randomly pre-selected to be offered VCT, 25% had been randomly pre-selected to receive free condoms, and 25% had been randomly pre-selected to receive both VCT and free condoms.

Those pre-selected for VCT were offered VCT right after the baseline survey had been administered. Eighty-seven percent of them consented.  The consent rate was slightly higher among girls than among boys.

Those pre-selected for free condoms were offered three boxes of 50 condoms each, right after VCT (if also sampled for VCT) or right after the baseline survey had been administered. Not all respondents offered condoms took them. Seventy-one percent took all 150 condoms, 19% took only some of them, and the remainder, 10%, refused to take any condoms. The acceptance rate was much higher among boys. While 87% of boys took all the condoms and only 5% took none, only 52% of girls took all 150 condoms and 15% took none.

A follow-up survey will be conducted in 2011-2012. The survey will include detailed questions on sexual behavior, including sexual debut, number and type of partners, and condom use, as well as detailed questions on beliefs regarding HIV transmission, own HIV status, and own exposure to risk. Crucially, the follow-up survey will also include HSV2 and HIV testing.

Results: 

Results forthcoming.

1 CIA World Factbook, “Kenya”. Available at https://www.cia.gov/library/publications/the-world-factbook/geos/ke.html

Northern Uganda Social Action Fund – Youth Opportunities Program

Youth unemployment is a persistent problem in the developing world, particularly in post-conflict settings, posing both economic and security issues. In growing, stable economies such as Uganda, what holds back youth from reaching their potential?  One theory suggests that youth unemployment is due primarily to the lack of sufficient capital to support entrepreneurship. If this is true, cash transfers or cheap credit could lead to a burst of self-employment. Evidence from other areas, such as studies on microcredit, suggests that alleviating these constraints with loans has little effect on earnings. In Northern Uganda, which is returning to peace after twenty years of war, the government’s Youth Opportunities Program offered cash transfers to groups of youth to increase employment and reduce conflict. Follow-up surveys two and four years later found a shift from agricultural work towards skilled trades and strong increases in income. Women in particular benefited from the cash transfers, with incomes of those in the program 84% higher than women who were not. There were no differences, however, in social outcomes such as community participation, aggression, and social cohesion.

See the full paper herea policy note for the World Bank here, and Chris Blattman’s blog discussion here.

Policy Question:
In developing countries, high unemployment - particularly among youth - is a pressing concern. Jobs, particularly higher-skilled labor and productive small enterprise, provide incomes and reduce poverty. For governments, transitioning from an economy based on small-scale agriculture to one based on entrepreneurship and production is critical for long-term growth. Employment is also seen as important for building social stability and political engagement in communities uprooted by long-term conflict.
 
One form of intervention offers cash in the hopes that youth will invest it in the training and assets to learn a trade or form a business. In the development community, anxiety persists over whether this is an effective approach: will youth with little or no financial or business training be able to direct the money towards successful long-term entrepreneurship?  Previous research also raises questions about the ability of women in particular to invest aid into increasing lifetime earnings, given occupational constraints and pressure to share windfalls.1
 
Uganda’s largest employment program sought to test if an intervention as simple as giving cash could help accomplish the country’s long-term economic and social goals for its youth.
 
Context of the Evaluation:
Twenty years of insurgency, instability and conflict led to high rates of poverty and unemployment in northern Uganda, but by 2005 a measure of peace and stability had returned to the region. The centerpiece of the post-conflict recovery plan was a decentralized development program, the Northern Uganda Social Action Fund (NUSAF). In 2006, to stimulate employment growth through self-employment, the government launched a new NUSAF component: the Youth Opportunities Program (YOP), which provided cash transfers to groups of young adults with the goal of encouraging trade-based self-employment. 
 
Description of the Intervention:
The YOP intervention had two official aims: to raise youth incomes and employment and to improve community reconciliation and reduce conflict. The program, targeted at youth from ages 16 to 35, required young adults from the same town or village to organize into groups and submit a proposal for a cash transfer to pay for: (i) fees at a local technical or vocational training institute of their choosing, and (ii) tools and materials for practicing a craft.
 
The average applicant group had 22 members. Group cash transfers averaged nearly UGX 12.8 million (US$7,108), and varied by both group size and group request. The average transfer size per member was UGX 673,026 (US$374) – more than 20 times the average monthly income of the youth at the time of the baseline survey.
 
Due to vast oversubscription, the 535 eligible groups were selected at random, using a lottery, to either receive the YOP program or be part of the comparison group. A baseline survey was conducted with 2601 individuals in 2008, and 87 percent were successfully followed and interviewed in the endline surveys two and four years later.
 
Results and Policy Lessons:
Overall, the program seemed to have strong economic effects. Four years later, beneficiaries of the YOP program had 41% higher income and were 65% more likely to practice a skilled trade, such as carpentry, metalworking, tailoring, or hairstyling. Hours worked were 17% higher, nearly entirely accounted for by these new professions – while most still farmed part-time, hours spent in agriculture were not different. They were also 40% more likely to keep records, register their business, and pay taxes.
 
Within the sample, gains were highest for those who had the highest initial credit constraints, those with fewest initial assets and access to loans. The effects were particularly strong for women. Women who received the cash grants four years later had 84% higher incomes than women who did not, while men were earning 31% more than their counterparts in the comparison group. This gender difference may reflect particular capital constraints faced by women.
 
While employment programs including this one are often implemented by governments with the aim of reducing social instability or promoting cohesion, the data show no evidence for impacts in these domains. After four years there were no measurable differences in cohesion, aggression, or community and political participation between participants in the YOP program and those in the comparison group.
 
Overall, the data show that the poor used the money effectively; investing in training and tools needed to start businesses and experienced a significant growth in income, even after four years. Even though impacts in social domains were negligible, the economic outcomes show the potential of alleviating capital constraints for spurring economic growth among the poor. 
 
Read the full paper here.
 
A midterm policy report here and policy note by the World Bank here were based on the initial 2-year follow up data.
 
[1] Fafchamps, M., McKenzie, D., Quinn, S., Woodruff, C., 2011. When is capital enough to get female microenterprises growing? Evidence from a randomized experiment in Ghana. Unpublished working paper.

Computer-Assisted Learning Project with Pratham in India

Policy Issue: 

Over the past decade many developing countries have expanded primary school access, energized by initiatives such as the United Nations Millennium Development Goals, which call for achieving universal primary education by 2015. However, there are major concerns about the quality of public education in developing countries—with achievement surveys indicating low levels of learning even for children who have attended school regularly for many years. While much is now known about how to get children into school, much less is known about how to improve school quality in a cost-effective way. While many schools rely on rote learning and memorization, how realistic is it to introduce more interactive learning approaches? How important is a pedagogical approach that adapts to the level of the child? Can technology help achieve these goals in a cost-effective way?

Context of the Evaluation:

In India 44 percent of children aged 7 to 12 could not read a basic paragraph and 50 percent could not do simple subtraction in 2005, although most were enrolled in school. Even in urban India, the learning levels are very low. In Vadodara, a major city in the Indian state of Gujarat, only 19.5 percent of the students enrolled in grade 3 could correctly answer questions testing grade 1 math competencies.  Computer-assisted learning (CAL) could be an effective supplement to regular instruction, as good education software can be reproduced at a nominal cost, and well-designed educational games can sustain interest and curiosity as well as adapt to the learning level of the child. Excitement over CAL is particularly strong in India, where the high-tech sector is successful, but schools have little guidance about if or how they should use computer based educational assistance.

Details of the Intervention:

Taking advantage of a government program that placed four computers each in 80 percent of primary schools in Vadodara, education-oriented NGO Pratham designed a program that supplemented classroom instruction with CAL. The study asked if CAL improves students' skill levels, if the effects would persist over time, and how cost-effective the program would be compared to alternatives.

Pratham introduced the CAL program over two years. Fifty-five schools were randomly assigned to receive the intervention while 56 served as a comparison. Students in treatment schools received basic instruction on how to use the computers, and then spent two hours per week of shared time (two students per computer) working independently with educational software. This software consisted of self-paced games in the local language, designed to improve basic math skills. The students were tested at the beginning, middle, and end of the school year, in both math and language skills. The experiment was repeated the following year by providing the CAL program to the schools that had served as a comparison group in the first year.

Results and Policy Lessons:

Impact on Education: Students who participated in the CAL program had higher math scores on average compared to the comparison group. In the first year math scores increased by approximately 0.36 standard deviations, and in the second year, by 0.54 standard deviations, a substantial achievement when compared to other education interventions. There was no measurable impact on language scores, suggesting that the introduction of computers did not have spillover effects on learning in other subjects. The improvement in math scores persisted to some extent after one year, but further research is needed to fully assess long-run impacts.   

Comparative Cost-Effectiveness: The CAL program was tested at the same time as a remedial tutor-based program, Balsakhi. The CAL program was shown to be highly effective in raising students' skill levels in math, but was less cost-effective than the tutor-based Balsakhi program—the Balsakhi program cost about US$2.25 per student per year, whereas the CAL program cost about US$15.18 per student per year.

Selected Media Coverage:

Balsakhi Remedial Tutoring in India

Policy Issue: 

Over the past decade many developing countries have expanded primary school access, energized by initiatives such as the United Nations Millennium Development Goals, which call for achieving universal primary education by 2015. However, improvements in school access and enrollment may not always translate into improved learning outcomes for all students if the quality of education is poor. Current research has identified several cost-effective ways to increase student attendance, but much less is known about how to improve education quality and student learning in a cost-effective way. Many schools rely on rote learning and memorization, but can lessons which are more tailored to children’s learning level improve achievement? How important is a pedagogical approach which adapts to the level of the child?

Context of the Evaluation: 

A 2005 survey found that 44 percent  of Indian children age 7 to 12 could not read a basic paragraph and 50 percent could not do simple subtraction even though most were enrolled in school. Even in urban India, the learning levels are very low in Vadodara, a major city in the Indian State of Gujarat, only 19.5 percent of the students enrolled in grade 3 could correctly answer questions testing grade 1 math competencies. Ironically, the difficulty in improving the quality of education may be complicated by success in getting more children to attend school, as in many cases neither the pedagogy nor the curriculum has been adapted to take into account the quantity and characteristics of the influx of new children.

Details of the Intervention: 

In conjunction with education-oriented NGO, Pratham, researchers evaluated the Balsakhi Program, a remedial education intervention implemented in 122 public primary schools in Vadodara and 77 schools in Mumbai. A tutor (balsakhi), usually a young woman recruited from the local community and paid a fraction of the cost of civil-service teachers (US$10-15 per month), worked with children in grades 2, 3 and 4 who were identified as falling behind their peers. The instructor typically met with a group of approximately 15-20 of these children who were taken out of the regular classroom into a separate class for two hours of the four-hour school day each day. Instruction focused on the core competencies the children should have learned in the first and second grades, primarily basic numeracy and literacy skills. The instructors were provided with two weeks of initial training and a standardized curriculum that was developed by Pratham.  

In the 2001 school year in Vadodara, approximately half of the schools were given a tutor for grade 3, and the other half were given a tutor for grade 4, while in Mumbai during that same year, approximately half of the schools received a tutor for grade 3, and the other half received a tutor for grade 2—in both cities, which school received which tutor was randomized. In 2002, the schools were given a tutor for the previously untreated grade. In determining program impact, grade 3 students in schools that only received a tutor for grade 4 were compared to grade 3 students in schools that had tutors for grade 3, and so on. Academic achievement was measured through two annual tests, administered at the start and end of the academic term.

 
Results and Policy Lessons: 

Impact on Education: The program had substantial positive impacts on children’s academic achievement. In both Vadodara and Mumbai, the Balsakhi program significantly improved overall test scores; by 0.14 standard deviations in the first year and 0.28 standard deviations in the second year, with the largest gains in math. Moreover, the weakest students, who were the primary target of the program, gained the most. Researchers estimate that the entire effect of the program was due to a very large (0.6 standard deviations) improvement in average test scores among the children who were sent for remedial education. In contrast, there was no measurable impact for their classroom peers, who did not receive remedial tutoring, but were “treated” with smaller class sizes and a more homogenous classroom.

Balsakhi Turnover: There was rapid turnover among the balsakhi tutors, with each tutor staying on average for just one year, typically until they got married or got another job. Despite the high turnover among tutors, the program still resulted in significant gains in student learning, which suggests that the success of the program did not depend on a handful of very determined and enthusiastic individuals. 

Cost-Effectiveness: The Balsakhi program was very inexpensive, since the main cost of the program was the tutors' relatively small salaries. Overall, the Balsakhi program cost approximately US$2.25 per child per year, significantly less than the cost per child of a Computer Assisted Learning program, evaluated by Pratham at the same time. In terms of cost per improvement in test scores, researchers estimate an attractive cost-effectiveness of about US$0.67 per standard deviation increase in test scores. The Balsakhi program has since been adapted, re-evaluated, and scaled up across India.

 

Selected Media Coverage:

 

 

 

Teacher Incentives Based on Students' Test Scores in Kenya

Does linking teachers' pay to students' test performance improve educational outcomes, or just increase “teaching to the test”? This study examines the effects of a teacher incentives program on both teacher behavior and student test scores in Kenya. Student test scores increased significantly during the study period, but evidence suggests that this improvement came through test-preparation sessions outside of normal class hours. Test score improvements dropped off after the program was completed, implying there were little to no spillover effects of the test preparations onto actual learning.

Policy Issue:

Over the past decade, many developing countries have expanded primary school access, energized by initiatives such as the Millennium Development Goals which call for achieving universal primary education by 2015. However, these improvements in school access have not been accompanied by improvements in school quality. Poor learning outcomes may be due, in part, to high absence rates and low effort among teachers, who often lack strong incentives to perform well at work. Some argue that linking teachers' pay to students' performance may be a way to increase teacher effort; opponents argue this will result in “teaching the test” rather than better teaching of the curriculum.

Context of the Evaluation: 

Learning outcomes in Kenya are poor: although the vast majority of Kenyan children attend primary school, less than half complete their primary education in rural areas. Government officials and policy makers are hopeful that improved education quality will enable individuals to attain higher levels of education. But more schools and an improved curriculum can only go so far – students must be met with a motivated teacher in their classroom. This is often not the case in Kenya; in Busia and Teso districts, teachers are absent on average 20 percent of the time. Teachers’ salaries depend on their education and experience, with no opportunity for performance-based promotion, which appears to result in a system with no incentives to teach well.

Details of the Intervention:

This study examines the effects of a teacher incentives program on both teacher behavior and student test scores in Kenya. Out of 100 schools which the Ministry of Education designated as particularly in need of assistance, 50 were randomly selected for the treatment, while the other 50 served as a comparison group.  In collaboration with International Child Support (ICS), researchers designed and evaluated a program in Busia and Teso districts that provided prizes to teachers in grades 4 to 8 based on the performance of their school as a whole on the annual district exams. ICS offered in-kind prizes that ranged in value from 21 to 43 percent of the typical teacher’s monthly salary, values that were comparable to merit pay programs in the United States.

All teachers who taught grades 4 through 8 in the 50 treatment schools were eligible for prizes. The program awarded “Top-Scoring,” and “Most Improved” prizes, as well as second, third and fourth place awards to teachers in winning schools. The program penalized teachers for dropouts by assigning low scores to students who did not take the exam, preventing them from selecting only the most qualified students to take the test. In all, prizes were awarded to 24 of the 50 participating schools.

Data were collected on many types of teacher effort – attendance, homework assignment, pedagogical techniques, and holding extra exam preparation sessions – as well as on student test scores after the end of the program. The program ran for two years beginning in 1998, with 1996 exam scores used to measure improvements.

Results and Policy Lessons:

By the program’s second year, student test scores increased significantly in treatment schools. However, evidence suggests that this improvement did not necessarily occur through the intended channel of regular classroom teaching. Teacher attendance and student dropout and repetition rates did not improve, and no changes were seen in either homework assignment or pedagogy. Instead, teachers were more likely to conduct test-preparation sessions outside of normal class hours. Prior to the program, treatment schools were slightly less likely to offer test preparations, but after the introduction of the program, treatment schools were 4.2 percentage points more likely to conduct prep sessions in the first year and 7.4 percentage points more likely in the second. Furthermore, there was evidence that much of the increase in test scores was due to improved test-taking skills such as strategies for handling multiple-choice questions or “cramming” for tests that are prone to memorization, as opposed to increased overall knowledge.  Lastly, test score improvements dropped off after the program was completed, implying there were little to no spillover effects of the test preparations onto actual learning.

Michael Kremer

Teacher Training and Free Uniforms for HIV Prevention in Primary Schools in Kenya

Policy Issue:

The vast majority of HIV cases occur in sub-Saharan Africa, where nearly 2 million people become infected with HIV/AIDS every year. One quarter of these new HIV infections are among people under 25, and almost all are due to unprotected sex.  AIDS is incurable and no successful AIDS vaccine has been developed. Ensuring the adoption of safer sexual behavior among youth is critical to preventing the transmission of this disease. However, there is surprisingly little evidence concerning the relative effectiveness of different programs to reduce risky sexual behavior.

Context:

Kenya has the 10th largest HIV infected population in the world – nearly 7% of Kenyans are infected.[i] Children are seen as a “window of hope” in the fight against AIDS, because their sexual patterns are not firmly established. In an effort to prevent HIV infections in new generations, in the late 1990s UNICEF and the Kenya Institute of Education jointly developed an AIDS education curriculum, including student and teacher handbooks. However, by 2003, this curriculum had not been fully implemented, likely due to teacher inexperience and discomfort with talking about this sensitive material.

Description of the Intervention:

This evaluation tested two interventions to reduce risky sexual behavior: training teachers on the existing HIV curriculum, and reducing the costs of schooling by providing free uniforms. The 328 study schools were randomly assigned to one of four groups of about 82 schools. Each of the four groups of schools received a different set of programs:

In groups 1 and 3, three teachers were trained on HIV/AIDS and on how to teach the HIV curriculum. The curriculum covers facts about the disease, and encourages abstinence until marriage and faithfulness afterwards. It also teaches life skills, such as how to say “no” to unwanted or unsafe sexual relations.

In groups 2 and 3, children already enrolled in sixth grade classes were given a free uniform. Implementers also announced that students still enrolled in school the following year would be eligible for a second uniform, and distributed uniforms again the following year.

All in all, group 1 schools received the teacher training program only, group 2 schools received the uniforms program only, group 3 schools received both programs, group 4 received no program at all and thus served as a comparison group.

To evaluate the impact of the two programs on sexual behavior and sexual health, survey data was collected on youths’ sexual behavior. Such survey data can be subject to reporting biases, however. It was therefore important to complement this data with an objective measure of the incidence of unprotected sex, which is the main mode of HIV transmission in Kenya. Two such measures were considered: (1) childbearing rates and (2) STI infection rates. Childbearing rates were monitored regularly between 2003 and 2010. STI infection rates (specifically, Herpes and HIV infection rates) were measured during a long-term follow-up in 2009-2010.

Results:

Impact of Teacher Training only: Training teachers on how to implement the national HIV/AIDS curriculum greatly increased the likelihood that teachers teach about HIV in the classroom. Two years after the training students whose teachers had been trained had greater knowledge about the disease and also reported more tolerant attitudes toward those with AIDS. However, the intervention did not reduce childbearing rates among girls, suggesting that it did not decrease the likelihood that girls engage in unprotected sex. It also did not reduce the risk of STI as measured after 6-7 years.

Impact of Free Uniforms only: Free school uniforms led students to stay enrolled for significantly longer, and reduced the incidence of teen marriage and teen pregnancy. Girls who benefitted from free uniforms were not less likely to have an STI after 6-7 years, however, suggesting that some of the adolescent girls in the free uniforms program, while less likely to engage in committed relationships that lead to pregnancy and marriage, might have engaged in casual relationships.

Combined Impact: In schools that received both free uniforms and teacher training on the HIV/AIDS curriculum, the reduction in drop-outs and teenage pregnancy among girls was lower than that observed in schools that received free uniforms only. This suggests that the curriculum’s emphasis on abstinence until marriage may have persuaded some girls who would have delayed marriage thanks to the free uniforms to instead privilege committed relationships, where pregnancies are more likely. On the other hand, the two programs combined led to a significant reduction in the risk of STI. This suggests that among girls who chose to delay marriage in order to stay in school with the free uniform, the HIV curriculum convinced some to abstain altogether in order to avoid the STI risk associated with casual relationships.


[i] CIA World Factbook, “Kenya” https://www.cia.gov/library/publications/the-world-factbook/geos/ke.html (accessed August 25, 2009). 

Decentralizing Education Expenditures: Primary School Community Grants in Niger

Policy Issue

In recent years school enrollment has risen dramatically in developing countries, prompting a renewed interest among policymakers in education management. Provision of educational services is centrally administered in much of the developing world, but evidence suggests that decentralized, locally administered services may be better suited to address low education quality. Local administration and oversight puts power into the hands of those with the most interest in seeing improvements in service delivery, and the best information about current education quality. By empowering local communities, decentralized management has the potential to combat systemic teacher absenteeism and reduce misallocation and corruption.

Context of the Evaluation

Niger has made significant strides in increasing access to education in the past decade, with primary enrollment increasing from 37% in 2000 to 54% in 2005. Yet the country still faces considerable challenges with regards to access, rates of primary school completion and the quality and management of the educational system at all levels. One problem, for example, is that schools are spaced far apart, making it difficult for the central government to effectively monitor them.

To address this problem the Ministry of Education mandated the creation of management committees (Committees de Gestion Scolaire, or COGES) for each school. The committee included the director of the school, as well as locally-elected community members. The COGES are responsible for monitoring teacher attendance and performance and managing both financial and material resources, such as the purchase of textbooks and supplies. The Ministry of Education introduced a pilot project to reinforce the capacity of the COGES through the provision of a yearly cash grant, which can be used to fund a part of its activities for the year, such as the construction of classrooms or the purchase of materials. These grants were provided under the belief that such financial support would spur communities to take a more active role in the management of schools.

Details of the Intervention

To test this belief, researchers, in collaboration with the World Bank and the Nigerian Ministry of Education, randomized the allocation of the grants. They observed schools which did and did not receive grants, to estimate their impact, as well as the impact of two supplementary accountability interventions on school management, student retention and learning achievement. One thousand schools in the regions of Tahoua and Zinder were randomly selected into treatment and comparison groups. The 500 schools in the treatment group each received an annual lump sum based on the number of classrooms in the school, averaging out to around US$2 per student. This grant size was chosen by the Ministry of Education with considerations of sustainability regarding a possible extension of the program. In addition to the grants themselves, two sub groups were randomly selected to receive complementary interventions designed to improve delivery and management of the grants.

One third of the treatment schools received financial monitoring in the form of a letter received at the beginning of the intervention telling the committee there was a 50% chance of a “financial control visit” at the end of the school year. Because of financial and capacity constraints, about 20% of the treatment schools received visits at the end of the year.

Another third of the treatment schools received expenditure restrictions in the form of a list of non-eligible expenditures, such as reimbursing school officials for travel, or spending money on things that can be produced locally.

Researchers used annual administrative survey data, pupil test score data, and a detailed financial questionnaire to determine the effect of the grants. The testing used a modified standardized test implemented in collaboration with the Ministry of Education Department of Education Assessment. The school data contained information on pupil and community participation, school resources and infrastructure and information on issues facing school governance. The functioning of the COGES, including how often they met, what they talked about, and whether they began improvement projects, was also monitored. 

Results and Policy Lessons

Supportive actions, which include parental financial and in-kind contributions, parent supervision of pupil attendance, and parent remedial action for pupil absenteeism, increased by 0.14 standard deviations in treatment communities. Most of this overall effect came from an increase in parental contributions to the school, which was 0.48 standard deviations higher in the treatment group. The overall impact was larger when the school committee was educated and was somewhat smaller when families live farther from school.

Management activities, which include parents’ association and school committee meetings, and whether the school committee is in charge of collecting fees and how they decide how the fees are spent, was 0.09 standard deviations higher in the treatment group. Specifically, the proportion of school committees in charge of collecting fees increased by 27 percent, the proportion of active parental association increased by 18 percent, and the frequency of school committee meetings increased by 5 percent.

Oppositional actions, which include supervising teacher attendance and sanctioning teachers, increased only in communities where the school committees were educated. In these communities, teacher supervision increased by 0.13 standard deviations.

Impact on school quality: The results indicate no short-term improvements in school quality, measured in terms of accountability and teacher effort. The lack of an impact on teacher effort is in line with the fact that very few of the schools undertook any sort of teacher supervision. However, one year after the treatment, the grants marginally improved (0.04 standard deviations) the material quality of schools. This improvement was largely driven by increases in the number of classrooms and construction of walls around the compound.

Impact on demand for education: The grant program led to a 0.17 standard deviation decrease in dropouts and a simultaneous 0.10 standard deviation increase in enrollment of students in Grade 2, indicating an increase in demand for education for young children.

Training and Technical Assistance for New Entrepreneurs in Morocco

Little empirical evidence exists on the effectiveness of business training and support. In particular, the extent to which such a support program might raise revenues and augment survival rates of new enterprises is unclear. The impact evaluation of the pilot phase of the Enterprise Support Project will measure the effectiveness of training, technical assistance and personal coaching provided to small businesses and revenue generating activities (AGRs) throughout Morocco in order to inform the decision of whether to scale-up the program after its pilot phase. The evaluation will provide critical information to program implementers, the Millennium Challenge Corporation (MCC) and the Agence du partenariat pour le progrès (APP) in addition to the government of Morocco regarding the effectiveness of the project and ways in which it might be improved.

The objective is to measure the specific contribution of the training and support program on key business outcomes. In particular, the principle research questions the evaluation is designed to address are:

  • Did the Enterprise Support Project contribute to an increase in revenues and profits of the participating businesses/AGRs?
  • Did the project improve survival rates of businesses/AGRs?
  • Did the program affect employment?
  • Were the benefits generated by the project greater than project costs?

 

 

Brain Drain in Ghana

"Brain drain", or the emigration of skilled workers, is one of the most common concerns African countries have about migration. Yet migration, broadly speaking, plays a significant role in economic development in the form of remittances and continued interaction of migrants with their home countries.

In order to provide more empirical evidence on the determinants and effects of skilled workers' migration, we propose to collect primary data on 1976-2004 cohorts of top high (secondary) school students in Ghana.

The project will analyze various aspects of the "brain drain" issue, including the reasons for migration of the highly skilled and the channels through which highly skilled migration affects the sending country (such as whether there is any evidence for involvement in trade facilitation, knowledge transfer, the level of remittances sent, etc.).  The project will also provide insights into the optimal design of education policy when facing increasingly globalized labor markets.

David McKenzie

Empowering Girls in Rural Bangladesh

What is the best way of empowering adolescent girls? Rights-based campaigns? Skills training? Researchers are cooperating with the Bangladesh office of Save the Children USA to evaluate a broad range of interventions.

Policy Issue: 

Throughout the world, cultural stigmatism has often excluded women from receiving education or earning income equivalent to men, though educating a woman increases her financial independence and dramatically improves the chances that her children will survive, be better nourished and better educated. For less developed nations in particular, women may embody a previously untapped source of human capital, and those countries that have embraced more aggressive policies in regards to gender equality in education can be expected to return greater social and economic benefits.

Context of the Evaluation:

When it comes to education, social mobility, and marriage choices, adolescent girls in Bangladesh appear to face numerous barriers. In recent years, school enrollment rates have improved, but girls are still often forced to drop out of school, to be married off at a young age. A recent UNICEF study shows that almost 50% of girls are married by the age of 15, and 60% are mothers by the age of 19. Effectively, this harms their health, education, and future decision-making prospects as well as their future income-generating potential and their status within the family. While some female-empowerment programs focus on rights-based campaigns or skill-building activities within the community, little is known about the relative costs and effects these different programs have on the social and economic empowerment of adolescent girls.

Details of the Intervention:

Researchers are cooperating with the Bangladesh office of Save the Children USA to implement a broad range of interventions, aimed at empowering adolescent girls in southern Bangladesh. The Kishoree Kontha (KK) project operates through small peer-led sessions in Safe Spaces (spaces within a community where adolescent girls can safely meet on a regular basis). Out of a sample of 460 target villages, 307 villages have been randomly selected to receive one of four intervention packages. The remaining villages will serve as the comparison group. The four intervention packages are:

  1. Basic Package: This package provides literacy and numeracy training for illiterate girls, as well as study support and educational mentoring for school-going girls. It also provides social competency training, which includes information on health, rights, and general negotiation and social skills.
  2. Livelihoods Package: This package complements the Basic package with additional sessions on financial livelihood readiness. Rather than providing direct vocational training, these sessions build entrepreneurial and budgeting skills that are applicable to a wide variety of financial opportunities in the community.
  3. Full Package: This package includes all the sessions of the Livelihood package, but adds a direct incentive to delay marriage until the legal age of 18 years old. All girls in targeted villages, between 15 and 17 years old, will be eligible to receive approximately 16 liters of cooking oil per year, on the condition that they remain unmarried. This amount surpasses the financial cost associated with delaying marriage that families must pay in the form of dowry.
  4. Delayed Marriage Package: This package provides only the incentives to delay marriage, as described above, without any peer-led sessions in the communities. As such, it will allow researchers to disentangle the effects of direct conditional incentives from changes in attitudes and skills.
Results and Policy Lessons: 

Results forthcoming.

Vocational Education Voucher Delivery and Labor Market Returns in Kenya

How effective is vocational training in Western Kenya? This study leverages a longitudinal survey to examine the impact on individuals with different childhood and background characteristics. Individuals are invited to apply (and then randomly selected) for a tuition voucher. The study will also measure the demand for vocational training, and the difference between public and private sector training institutes. 

Policy Issue: 

Youth underemployment, especially among less educated populations, has the potential to create significant social unrest and perpetuate poverty. However, little is known about how best to help youth find jobs and smooth the school-to-work transition, particularly in less developed countries. One would-be tool for expanding the labor market opportunities in these settings is vocational education, which could help students learn a trade and acquire the skills needed to take advantage of employment opportunities, and create successful small businesses. However, credible research on the economic returns to vocational schooling in poor countries remains scarce.

Context of the Evaluation:

The introduction of free primary education in Kenya in 2003 prompted a large influx of pupils previously not enrolled in school. As these pupils complete their primary schooling in the coming years, Kenya will face unprecedented numbers of primary school graduates competing for limited seats in traditional secondary schools. Vocational education is one promising avenue for these youths. Increasing vocational education has the potential to help address unemployment issues for new primary school graduates by providing students with the skills needed for employment, even without obtaining a secondary school diploma. Vocational training may also help to address the problem of underemployment among youth who exited the academic schooling system before 2003. However, there is little empirical evidence on the impacts of vocational education on employment, salary, or consumption for individuals with different characteristics and backgrounds, for instance, by gender, family background, or individual cognitive ability.

Details of the Intervention:

The program was launched in 2008 with the recruitment of 2,163 out-of-school Kenyan youths (roughly 18 to 26 years old) who had participated in the Kenya Life Panel Survey. In order to be included in the sample, applicants were required to attend two meetings on the program, participate in a short survey regarding their expected returns, complete a form ranking their preferred schools and courses, and submit a letter of support from either a local official or a school official. Applicants were then entered into a lottery to determine if they would receive a vocational training voucher worth up to approximately US$325, an amount sufficient to fully, or almost fully, cover the tuition costs for most private vocational education programs and government-run rural village polytechnics.

Among the voucher winners, a randomly selected half received vouchers that could be used only in government supported public vocational training institutes, while the other half received unrestricted vouchers that could be used in either public centers or in the local private vocational training sector. Once winners were informed of which type of voucher they were eligible for, these young adults were able to access the tuition voucher, greatly reducing the cost (close to zero) of vocational education. The impact evaluation tracks these individuals (both voucher winners and non-winners) over time to evaluate take-up of vocational education, and its impacts on labor market and other life outcomes. The division of the treatment group into those with restricted versus unrestricted vouchers, combined with detailed panel data collection of individuals and institutions, will allow researchers to estimate the additional labor market returns of having access to the private vocational training sector. 

Results and Policy Lessons: 

Results forthcoming.

 

Extra Teachers in Kenya: Peer Effects, Pupil Teacher Ratios and Teacher Incentives

 
Policy Issue: 

The introduction of free primary education has raised primary school enrollment in many developing countries. However, the resulting overcrowding of schools, as well as the influx of new students with little or no preparation, poses new challenges to policymakers. For example, increased enrollment has often not been matched by increased numbers of government-salaried teachers. One method of lowering the pupil-teacher ratio, versions of which have been used by many governments, is to hire low-paid local contract teachers to supplement government-salaried teachers. However, there are concerns that these teachers may be less experienced and therefore less effective. Empowering the local community to monitor teachers' performance may help to overcome this problem, and may also increase civil-service teacher effort and subsequently student learning.

 
Context of the Evaluation: 

Historically, Kenyan schools have had two types of teachers – those hired through the Ministry of Education and those hired locally and informally by Parent Teacher Associations (PTAs). For civil-service teachers, promotion, transfers, and disciplinary measures are decided through the Ministry of Education, rather than by more locally accountable bodies. These teachers are represented by a strong union, have civil-service protection, and receive wages and benefits considerably above market levels. PTA teachers, while typically paid much less than their civil-service counterparts, have much stronger incentives, partly because they do not have union protection, but also because a good track record as a contract teacher can help them obtain a civil-service job. 

Incentives for civil service teachers, while weak, are largely based on their students’ scores on the primary school exit exam. Teachers may, therefore, have an incentive to focus on students they believe will perform better on the exam, while ignoring the weaker students who may be too far behind to catch up. 

In 2003, Kenya eliminated school fees for primary school. This led to an almost 30 percent increase in enrollment and greater heterogeneity in student preparation. Many of the new pupils were first generation students and had not attended preschools. Since parents were no longer required to pay fees, local school committees were generally unable to raise the funds necessary to hire PTA teachers to match the influx of students and bring the new students up to speed.

 

Details of the Intervention: 

In collaboration with International Child Support (ICS), researchers evaluated three nested interventions that addressed the large class sizes and heterogeneity in student preparation in the Kenyan school system: (1) the addition of locally hired contract teachers to reduce the pupil-teacher ratio; (2) the empowerment of parents within PTA committees through School-Based Management (SBM) training; and (3) the sorting of students by initial level of preparedness to reduce heterogeneity within the classroom.  

Out of 210 primary schools, 140 were randomly assigned to receive the Extra Teacher Program (ETP). The ETP program provided funding to hire a local contract teacher to address classroom overcrowding. The remaining 70 schools served as a comparison group. In each school, ICS held a meeting with parents and teachers to explain the program rules regarding the hiring of an additional teacher. School committees were responsible for hiring the contract teachers and were free to replace or keep the original contract teacher based on performance. The contract teachers were paid approximately one-quarter of the salary of regular civil service teachers, but had the same educational qualifications. 

In 2005, the first year of the program, the ETP contract teacher was assigned to grade 1. Two sections were created for grade 1, one taught by regular civil service teachers (with multiple teachers rotating in an out of the class, teaching different subjects) and one taught exclusively by the contract teacher. As a result, average class size in grade 1 was only 44 in ETP schools, compared to 82 in comparison schools. In the second year of the program, the ETP teacher moved to grade 2, such that the cohort of first graders that benefitted from the ETP program continued to benefit once in grade 2.

While in half of the ETP schools (the “non-tracking ETP schools”), students were divided into sections at random, in the other half (the “tracking ETP schools”), students were divided into sections based on students’ level of preparedness (as measured by exam scores during the first term). For all schools, which section was taught by the ETP contract teacher was then decided by random draw.

Finally, half of the schools assigned to ETP, including both tracking and non-tracking schools, were randomly selected to receive School-Based Management (SBM) training. The training was designed to empower parents (within the PTA committee) to ensure a fair and objective recruiting process for the ETP teacher, as well as to monitor teachers’ performance. Two parents of grade 1 students were asked to perform attendance checks on teachers on a regular basis. 

Standardized test covering math and literacy questions were administered to each school just before the program ended and again one year later. Four unannounced visits were also made to each school to measure teacher effort and to observe the classroom environment. 

 

Results and Policy Lessons: 

Learning Impacts: Providing school committees with funds to hire an extra teacher on a short-term contract had a positive effect on learning. Test scores of students in ETP schools were on average 0.22 standard deviations higher than those of students in comparison schools. However, not all students benefitted equally. The program impact varied a lot with the details of how it was implemented. In particular: 

  • The impact was much larger for students assigned to the ETP contract teacher (0.31 standard deviation gain) than for students assigned to the civil-service teachers (0.13 standard deviation). 
  • The impact was larger when the ETP program was accompanied with SBM training (0.25 standard deviation) than when it was implemented alone (0.19 standard deviation).
  • The impact was larger in tracking schools (0.28 standard deviation) than in non-tracking schools (0.16 standard deviation).
  • There was relatively rapid fade-out of these learning gains in the year after the program ended, except for the gains in tracking schools and the gains in SBM schools.

Mechanisms:

Teacher effort: A crude measure of teacher effort is how often the teacher is found in class and teaching during a random spot check. This rate is 58 percent among civil-service teachers in non-ETP schools, suggesting relatively low levels of efforts. Despite their low pay, contract teachers were found 27.8 percentage points more likely to be teaching during a random visit. As a result, their students learned more. In contrast, civil-service teachers in ETP schools responded to the introduction of the ETP program by decreasing their own effort: the probability that they would be found in class teaching during a random visit decreased from 58 to 45 percent. As a result, their students did not significantly gain in learning over students in comparison schools, despite being in much smaller classes. One of the reasons why the SBM program led to increased learning gains is that it mitigated the negative effort response by civil-service teachers.

Selection of Contract Teachers: Because a contract teacher position is a stepping stone for permanent civil-service positions, it is highly valuable (despite the low pay). As a result, existing civil-service teachers may want to hire their own relatives as contract teachers. In ETP schools without SBM training, 31 percent of contract teachers hired were relatives of existing teachers in the school. Those relatives tended to perform less well than other contract teachers. The second reason why the SBM program led to increased learning gains is that SBM increased the transparency of the contract-teacher recruiting process. Specifically, it reduced the number of contract teachers who were hired because they were related to an existing teacher by about half. 

Homogeneous classrooms: Compared to students in non-tracking ETP schools, students in tracking ETP schools were in more homogeneous classes. This homogeneity benefitted them, irrespective of whether they had started at the top, middle, or bottom of the distribution. Homogeneity is beneficial in part because it allows teachers to better tailor their materials to the level of their students, but also because it increases teachers’ attendance in the classroom. For those who started at the middle of the distribution, being assigned to the bottom track (instead of the top track) had no detrimental effect on their learning, because the decrease in peer quality was compensated by a positive effect of being at the top of the class. 

Overall, the results suggest that while hiring supplementary contract teachers can be part of the solution to Kenya’s teacher shortage, to get the most out of these teachers, implementation details matter. The biggest gains come when local school committees are empowered to oversee the recruitment process and to effectively monitor teachers, and when classes are structured so as to target instruction to students’ initial achievement level. 

Flipcharts and School Inputs in Kenya

Flipcharts are thought to promote learning in several ways, and may appeal to a broader range of students with different learning styles. Flipcharts may be particularly attractive in the rural Kenyan setting, where textbooks are too expensive for most students and many students have limited proficiency in English, the medium of instruction in Kenya and the language in which all Kenyan textbooks are written.

The estimated impact of flipcharts on student test scores was found to be close to zero. A parallel non-randomized analysis estimates impacts five to ten times larger than the estimates based on the randomized trial. This suggests that the non-randomized estimates are subject to serious upward bias

 
Policy Issue:

While 85% of primary school-aged children are enrolled in school worldwide,1 these higher levels of enrollment may not be translated into increased learning for students in developing countries, where education quality is often low. Poor education may be due, in part, to lack of proper school materials, which are often undersupplied to poorer or more remote areas. There is a widespread belief that the provision of school materials can substantially improve educational outcomes in rural, underdeveloped areas, however there is little empirical evidence to confirm this.

 

Context of the Evaluation: 

The vast majority of Kenyan children attend primary school, although in rural areas less than half complete their primary education. The schools in this study are located in Busia and Teso, two neighboring agricultural districts on the border with Uganda, both of which have below-average income for Kenya. Flipcharts and other visual aids are rare in schools in these areas, and less than one third of the schools had any flipcharts before the study. Even textbooks are rare: in grade 8, which is selective, about 40% of students had textbooks in math and English, but 15% or less had textbooks in science and other subjects. In lower grades, textbooks are even rarer.

Flipcharts are thought to promote learning in several ways, and may appeal to a broader range of students with different learning styles. Flipcharts may be particularly attractive in the rural Kenyan setting, where textbooks are too expensive for most students and many students have limited proficiency in English, the medium of instruction in Kenya and the language in which all Kenyan textbooks are written.

Most analyses of the effect of educational inputs are based on retrospective studies, which compare schools with different levels of inputs. One potential weakness of this approach is that observable inputs like textbooks and flipcharts may be correlated with unobserved variables that affect educational outcomes. For example, if parents who provide better home environments (a typically unobserved characteristic) tend to organize to obtain more observed school inputs for their children, then estimates of the effect of those school inputs on test scores may be biased upwards. This evaluation compares retrospective and prospective estimates of the effect of flipcharts in Kenyan primary schools.

 

Details of the Intervention: 

In the prospective study, a Dutch NGO, ICS, provided five sets of flipcharts—large, poster-sized charts with instructional material that can be mounted on walls or placed on easels—to each of 89 randomly selected Kenyan primary schools. The intervention included flipcharts covering mathematics, geography, health, and two charts for science (agriculture and general science), as well as a teacher’s guide for science. The charts are not kept in the classroom, but rather are brought in when they are relevant to the day’s lesson, and can therefore be used in more than one classroom on any given day. In practice, the grade 7 and 8 teachers have priority over the usage of the charts, and account for roughly 60-75% of total use.

The data available for this study are the 1997 and 1998 scores from the Kenya Certificate of Primary Education (KCPE) exam, which is taken at the end of grade 8, and the scores of grade 6-8 students on practice exams taken throughout 1997 and 1998. Test scores from schools that received the flipchart intervention were compared to the comparison group. Pre-intervention data on school performance across all subjects was also examined.

The retrospective analysis uses data for 100 schools involved in a separate study that provided textbooks and grants for flipcharts to randomly selected schools. The effect of flipcharts and other inputs was estimated using the 1998 practice and KCPE exam scores for grades 6-8.

 

Results and Policy Lessons:

The comparison between retrospective and prospective estimates of the effect of flipcharts in Kenyan primary schools finds that randomized prospective estimates are much smaller than retrospective estimates. Despite a large sample size and two years of follow-up data, in the prospective study the estimated impact of flipcharts on student test scores was found to be close to zero.

In contrast, using the retrospective framework, several conventional ordinary-least-squares (OLS) estimates suggest that flip charts raise student test scores by 20 percent of a standard deviation. This suggests that retrospective estimates are subject to serious upward omitted variable bias, even when controlling for other observable inputs. The relationship between school inputs and educational outcomes is critical for educational policy; however policy should reflect prospective study findings as retrospective studies may produce misleading results. 

1 UNICEF, “Child info, Statistics by Area/Education,” http://www.childinfo.org/education.html.

Michael Kremer

The Illusion of Sustainability: Comparing Free Provision of Deworming Drugs and Other "Sustainable" Approaches in Kenya

Policy Issue: 

Intestinal helminths—including hookworm, roundworm, schistosomiasis and whipworm—infect more than one in three people worldwide and  at least 800 million of these are school-age children. Worms are believed to have a negative impact on child development, and can contribute to lower educational attainment and income later in life. Intestinal worms can be effectively treated with low-cost drugs, but treatment must be continued indefinitely to prevent re-infection. Finding sustainable approaches to providing deworming drugs is a pressing research question, as most deworming interventions are currently financed by external institutions. Practices such as health education or cost-sharing may be able to increase program sustainability, but there is little systematic evidence on this matter.

Context of the Evaluation: 

Busia district is a poor and densely-settled farming region in western Kenya adjacent to Lake Victoria. This area has some of the country’s highest helminth infection rates; upwards of 90 percent of children aged 6-18 are infected. This is in part due to the area’s proximity to Lake Victoria—schistosomiasis is easily contracted through contact with the infected lake water. Other types of helminths can be transmitted through contact with or ingestion of fecal matter. This can occur, for example, if children do not have access to a latrine and instead defecate in the fields near their home or school, areas where they also play. 

The prevention and treatment of infectious diseases such as worms is a priority for health officials, and more efficient and sustainable programs could enable the delivery of health care to a larger number of people. Advocates of improving sustainability concentrate on health education, community mobilization, and cost-recovery from program beneficiaries, to complement the more standard practice of subsidizing health products.

Details of the Intervention: 

This study evaluated the Primary School Deworming Project (PSDP), which was carried out by NGO International Child Support in 75 schools, randomly divided into three groups (1, 2, and 3) and phased into treatment over several years. All schools with helminth prevalence over 50 percent were treated periodically with albendazole, as well as with praziquantel if the local prevalence of schistosomiasis was high enough.

Cost-Sharing:  In 2001, 25 of the 50 Group 1 and Group 2 schools were randomly selected to pay user fees for deworming treatment. Two thirds of the schools participating in cost-sharing received albendazole at a cost of US$0.40 per family, and one third received both albendazole and praziquantel (depending on the local prevalence of schistosomiasis) at a cost of US$1.30 per family. The per family basis of the fee introduced within-school variation in the per-child cost of deworming, since households have different numbers of children.

Health Education: In addition to medicine, all treatment schools received regular public health lectures, wall charts on worm prevention, and training for two teachers from each school. The lectures and teacher training provided information on worm prevention behaviors—including washing hands before meals, wearing shoes, and not swimming in the lake. 

Verbal Commitments: A verbal commitment "mobilization" intervention asked people to verbally commit in advance to adopt the deworming drugs. 

Social Learning: A questionnaire was conducted in 2001 to test whether households with more “social links” to schools which received early treatment would be more likely to take deworming drugs. Respondents were asked about the friends and relatives they speak with most frequently about child health issues, and the degree of “linkage” to treatment schools was established on this basis.

Results and Policy Lessons: 

Cost-Sharing Intervention: The introduction of a small fee for deworming drugs led to an 80 percent reduction in treatment rates, consistent with the hypothesis that people have low private valuation for deworming. Take-up dropped sharply when going from a zero price to a positive price, but was not sensitive to the exact positive price level, suggesting that it may be counter-productive to charge small positive prices for the treatment of infectious diseases. 

Health Education & Verbal Commitment Impact: An intensive school health education intervention had no impact on worm prevention behaviors. Child health is likely to be worsened to the extent that funds are diverted from medical treatment into health education in this setting. Asking people in advance whether they planned to take deworming drugs also had no impact on adoption.

Social Learning: Individuals in treatment schools who had more extensive social networks, and therefore presumably had more information about deworming drugs, were significantly less likely to consent to take the drugs. For each additional social link to a family that had already received treatment, a family’s child was 3.1 percentage points less likely to take the drugs, and  these individuals were also more likely to believe the drugs are “not effective.” Negative social effects on take up are especially large for families with more knowledge about deworming, which may be due to overly optimistic prior beliefs about the net private benefits of the drug. A high proportion of deworming benefits flows not to the treated child or her family, but to others in the local community through positive externalities; the lower take-up among those with more deworming knowledge may reflect their understanding of this fact. 

Overall, findings suggest that socially desirable health technologies with low private benefits may not spread on their own, due to low private estimations of the benefits that are reinforced through social networks.

Textbooks and Test Scores in Kenya

Policy Issue: 

Education in developing countries is often hindered by large class sizes and low teaching quality. Over the past decade many developing countries have expanded primary school access, energized by initiatives such as the United Nations Millennium Development Goals, which call for achieving universal primary education by 2015. But increased quantity has not always been matched with improved quality, and only 60 percent of appropriately aged children gain the skills necessary to attend secondary school worldwide. In sub-Saharan Africa, that number drops to only a quarter.1 There is a widespread belief that the provision of textbooks can substantially improve educational outcomes in developing countries, but there is little empirical evidence as to whether providing textbooks alone can overcome the effects of other systemic problems such as high teacher-student ratios and frequent absenteeism.

Context of the Evaluation: 

In Kenya, 50 percent of families live below the poverty line, and high teacher absenteeism is reported in schools.2 Nevertheless the overwhelming majority of children start primary school. But some schools appear to promote only strong students to grade 8 in order to maintain high average scores on secondary school entrance exams, and those not promoted often drop out. Many students also drop out in earlier grades – 35 percent of the surveyed students in grade 3 dropped within the next three years. 

Primary school is provided at a low cost, with some expenses, such as textbooks, paid by parents. Because of this cost, schools in Busia and Teso usually have textbooks for teachers to use, but few textbooks for children. At the start of this study, 80 percent of students in the sample were in classrooms with less than one English textbook for every 20 students, and the analogous figures for math and science textbooks were 78 percent and 89 percent, respectively. Government officials and NGOs are hopeful that textbook provision will improve test scores, encouraging higher grade attainment and overall improved educational outcomes.

Details of the Intervention: 

In 1995, International Child Support (ICS) began a program to improve primary education in Kenya’s Busia and Teso districts by providing additional official government textbooks. J-PAL affiliates, in an effort to determine the impact which textbooks have on students, teachers, and overall learning, evaluated this program. ICS randomly divided 100 needy, low performing schools into four even groups. In each year, program schools were compared to those not yet receiving the program.

Program Design

 

Distribution of Textbooks

School Year

Schools

Group 1

Math books provided to grades 3, 4, 5, 6 and 7

English provided to 3, 5 and 7

Science and Agriculture provided to grade 8

1996

25

Group 2

Grant equal to US$2.65 per student, 43% of which was spent on textbooks

1997

25

Group 3

Similar grant in 1998 school year

1998

25

Group 4

Similar grant in 2000 school year

2000

25

Sharing textbooks is common in Kenya, and two or three students typically share a workspace. Hence, a 60 percent textbook per pupil ratio was provided in English and science, and a 50 percent ratio was provided in math, giving nearly all students shared access to a textbook. A pre-test in the form of a district exam was administered to all schools before the textbooks were distributed, and district exam scores were collected at the end of each of the subsequent program years for comparison data. Classroom activities, attendance and dropout rates were also monitored.

Results and Policy Lessons: 

This study found no evidence that textbook provision increased average test scores, or that it reduced either grade repetition or dropout rates. Textbooks increased progression to secondary school for eighth graders but did not reduce grade repetition or raise attendance in lower grades. This provides evidence for the hypothesis that the program mostly benefited strong students, since only strong students reach grade 8 and have a hope to progress to secondary school, while many students in the lower grades were actually unable to read the textbooks. Students in grade 8, a selective and academically strong group, were 5 percentage points more likely to enter secondary school in the second program year than comparison school students.  

Kenyan students are extremely heterogeneous in their family background, preparation for schooling, and economic circumstances. The Kenyan curriculum remains oriented to elites with language of instruction in English, most students' third language. Textbooks are meant to pair with this curriculum, arguably better suited toward elite students and ill-suited for academically weaker students. The evidence from this evaluation supports such a picture of the Kenyan education system, and suggests that better suited materials might produce achievement gains in a wider section of the population. 

1 UNICEF, “Basic Education and Gender Equality,” http://www.unicef.org/girlseducation/index_bigpicture.html
2 As of 2000. CIA World Fact Book, “Kenya,” https://www.cia.gov/library/publications/the-world-factbook/geos/ke.html

Michael Kremer

Balwadi deworming in India

Policy Issue: 

Nearly 40% of children in Africa and Asia suffer from iron deficiency anemia (IDA), which can result in weakness, stunted physical growth, and a compromised immune system. Intestinal helminths (worms) cause chronic intestinal blood loss which contributes to iron deficiency anemia. Worms are prevalent among children in developing countries and are believed to have a negative impact on education, impacting child cognitive and physical development as well as school attendance. Estimates suggest that the impact of iron deficiency anemia—through both physical and cognitive channels—could be as large as 4% of GDP on average in less developed countries, yet there is little rigorous work by economists on the effects of anemia on economic development.

Context of the Evaluation: 

Like other developing nations in the region, iron deficiency anemia and Vitamin A deficiency affect many of India’s children. Over 69% of preschool aged children in urban Delhi are anemic and 30% suffer from intestinal worms, contributing to the high prevalence of malnutrition. In 2005, 46% of children were found to be underweight, and 38% were found to have stunted growth. Children in this study typically came from families of poor migrant laborers, and have a particularly high risk of anemia and other nutritional deficiencies.

Details of the Intervention: 

This study evaluated the impact of NGO Pratham’s preschool nutrition and health project in the slums of Delhi, India. The program delivered a package consisting of iron and Vitamin A supplementation and deworming drugs to 2-6 year old children through an existing preschool network.

Two hundred preschools with a total of 2,392 children were randomly divided into three treatment groups, which were gradually phased into the program over two years. The deworming drugs were taken at “health camps” held at the preschool approximately every three months. Preschool teachers in treatment schools were instructed to administer daily iron doses for thirty school days following each health camp. Children in both treatment and comparison groups were also administered Vitamin A supplements, which in addition to other health benefits, promotes the absorption of iron.

A household survey was administered to a random 30 percent of the child population from each preschool both at the baseline and then again before the final group was phased into the program. Hemoglobin (Hb) tests (to measure anemia) and parasitological tests (to measure the presence of worms) were administered in conjunction with the household survey. Child height and weight were measured during each health camp, and participation data was collected during monthly, unannounced visits to each preschool.

Results and Policy Lessons: 

Child Weight Gain: Large gains in child weight—roughly 0.5 kg on average—were found in the treatment schools relative to comparison schools over the two-year study period. No gains in average child height were found, but this pattern makes sense from a clinical standpoint: iron supplementation is thought to reduce acute malnutrition in the short-run by improving the absorption of micronutrients and increasing appetite, but improvements in chronic malnutrition are not expected over short periods.

Impact on School Attendance: Average preschool participation rates increased sharply by 5.8 percentage points among treated children, reducing preschool absenteeism by roughly one fifth.

Weight gains and school-participation improvements were most pronounced for sub-groups with high baseline anemia rates, in particular, for girls and children in low socioeconomic status areas.

Given the low cost of the intervention (averaging approximately US$1.70 per additional year of schooling induced for one child), these results suggest that the package of iron, Vitamin A and deworming drugs is a highly cost-effective means of improving child school participation and health in a poor urban setting where anemia and worm infections are widespread.

Edward Miguel

Primary Education Management in Madagascar

Policy Issue: 

Successful efforts to expand access to education in the developing world have not always translated into actual improvements in skills and learning for students. It remains an open debate as to whether top-down approaches are more effective in improving educational quality than approaches which promote beneficiary participation, such as parental monitoring. Top-down approaches can provide administrators with the tools necessary to better monitor their schools, but this assumes that they have the incentives to do so. Bureaucrats may have more incentives to improve the quantity, rather than quality, of education services since the benefits of improving quality are diffuse and harder to verify. The combined effects of these perverse incentives can result in a large number of children who are in the classroom, but are not learning. Promoting local accountability may be a useful means of improving schooling outcomes, if it can be determined which factors makes beneficiary participation effective at improving education delivery.

Context of the Evaluation: 

Madagascar divides its 2.7 million children into 15,000 public primary schools. Despite the significant increase in primary school enrollment following Madagascar's 2002 reforms and an influx of international financial support, resource allocation across schools remains inefficient, and better resource endowments rarely translate into better student performance. Only 63% of grade 5 children pass the primary-cycle exam, an assessment of the minimum level language and math knowledge presumed at this grade. District administrators face a performance review only every 3 years, and the subdistrict heads rarely face any credible threat of penalties or firing.

Details of the Intervention: 

Researchers, in collaboration with The Ministry of Education in Madagascar, ran a randomized experiment in 3,774 primary schools in 30 public school districts. These districts represented all geographic areas in the country, but were focused on schools with the higher rates of grade repetition.

All district administrators in treatment districts received operational tools and training that included forms for supervision visits to schools, and procurement sheets for school supplies and grants (district-level intervention). In some of these schools, the subdistrict head was also trained and provided with tools to supervise school visits, as well as information on the performance and resource level at each school (subdistrict-level intervention).

Lastly, several districts also introduced a school level intervention which involved parental monitoring through school meetings. Field workers distributed a ‘report card’ to schools, which included the previous year’s dropout rate, exam pass rate, and repetition rate. Two community meetings were then held, and the first meeting resulted in an action plan based on the report card. One example of the goals specified in the action plans was to increase the school exam pass rate by 5 percentage points by the end of the academic year. Common tasks specified for teachers included lesson planning and student evaluation every few weeks. The parent’s association was expected to monitor the student evaluation reports which the teachers were supposed to communicate to them. These tools allowed parents to coordinate on taking actions to monitor service quality and exercise social pressure on the teachers.

Results and Policy Lessons: 

Impact from Top-Down Approach: The interventions targeted at the district and subdistrict level had minimal effects on the administrator’s behaviors, and the schools and students under their responsibility. Though each tool – forms for supervision visits to schools and procurement sheets for school supplies and grants – was used by 90% of subdistrict heads and more than 50% of district heads, subdistrict heads visited their schools only slightly more often than those in the comparison group, an insignificant improvement. Teachers in both groups did not plan for lessons more, and no improvement in test scores was seen in the two years following the program.

Impact from Bottom-Up Approach: The interventions at the school level led to significantly improved teacher behavior. Teachers were on average 0.26 standard deviations more likely to create daily and weekly lesson plans and to have discussed them with their director. Test scores were 0.1 standard deviations higher than those in the comparison group two years after the implementation of the program. Additionally, student attendance increased by 4.3 percentage points compared to the comparison group average of 87%, though teacher attendance and communication with parents did not improve.

 

The Impact of Distributing School Uniforms on Children's Education in Kenya

Policy Issue:

Children in developing countries face numerous barriers to accessing basic education. According to the World Bank, school fees (fees for books, materials and some exams) are among the major obstacles to universal primary education in developing countries. Several countries in sub-Saharan Africa have taken strides towards meeting the Millennium Development Goal of universal primary education by 2015 by eliminating school fees,but other significant costs remain, including the cost of providing a school uniform for a child. Many governments and NGOs have sought to overcome this barrier by offering free or subsidized uniforms to particular students, often as part of a “child sponsorship” program. However, no prior studies have examined the impact of providing school uniforms on school participation and education outcomes.

Context of the Evaluation:

In Kenya, students were required to pay school fees to attend school through 2002. In January 2003, a new government policy eliminated fees, which had amounted to approximately US$4 in the Busia area, and also provided basic textbooks. Students were, however, still required to purchase and wear uniforms, and historically, students who did not wear uniforms could be sent away from school at the discretion of the principal. School uniforms cost between US$4.33 and $7.33 for girls and between US$5.40 and $7.33 for boys. Amongst sampled primary schools in rural Busia, 78% of students were likely to be wearing uniforms, but only 5% to be wearing shoes.

Description of Intervention:

The NGO ICS-Africa operates a Child Sponsorship Program (CSP) in Western Kenya, in which children sponsored by donors in the Netherlands and elsewhere receive school fees and school uniforms. Some portion of the sponsorship fee paid by the donors is used to give general assistance to the school: grants for construction, health care programs, and agricultural programs. In 2002 ICS-Africa planned to phase CSP into several new schools and agreed to randomize the aspects of the program directed at individual children. ICS used a lottery to distribute some of the child sponsorships within each of twelve primary schools in western Kenya.

The Kenyan school year runs from January to December. In mid-January 2002, ICS organized a census of children in standards one through four of the twelve selected schools. Based on that census, ICS selected all children who had experienced one or both parent deaths (orphans) to receive sponsorships.It then used a lottery to randomly select the remaining beneficiaries, and studied the effects on this population.

Schools immediately received some basic benefits from being sponsored. A pair of ICS nurses visited each school several times a year and provided basic first aid to any child or local adult who requested it. An agricultural representative organized student clubs to grow crops on the school grounds. In fall 2002, each school received a sizeable grant for classroom construction, for desks, and for books. The only individual benefit that sponsored children received was a school uniform; in June 2002, uniforms were distributed to all sponsored children who were still in school.

An attendance dataset recorded pupil attendance at each of the twelve schools from 2002 through the end of 2004. Attendance was gathered as field officers made unannounced visits to each school multiple times each year and recorded whether each child was present. From these multiple visits, an annual child attendance average is collected.

Results and Policy Lessons:

Results indicate a strong positive impact of receiving a school uniform on student school participation. Giving a uniform reduces school absenteeism by 6.4 percentage points (43%) from a base of 15% school absenteeism. The effect is 3.4 percentage points larger for students who did not have a uniform at the baseline. This is a major reduction in absenteeism from a baseline school attendance level of 85%.

The program appears to have had a positive impact on test scores in 2003, raising average test-scores of recipients by one quarter of a standard deviation. While the average test scores of uniform recipients are still observed to be higher two years after the program started by one fifth of a standard deviation, the effect is less precisely estimated.

The average effect of the program is an increase in school participation of 0.064 years per treated child. The average cost of a school uniform is 436.86 Kenyan shillings (US$5.82). Thus, the cost of increasing school by one year is $5.82 / 0.064, or US$90.94. While this is more than other interventions, such as deworming, which were tested in this area, it is still considerably less than the cost of many programs which give beneficiaries cash, rather than goods.

Michael Kremer

Kenyan Life Panel Survey

The Kenyan Life Panel Survey (KLPS) builds on an existing longitudinal dataset of educational, health and demographic information for approximately 6800 pupils in Western Kenya collected from 1998-2003, and extends it for another 6 years. In particular KLPS seeks to examine the long-run impact of a recent school-based health program - the Primary School Deworming Project - which provided free treatment for intestinal helminthes (worms) to pupils in 75 rural primary schools phased in over 5 years.  The project found that deworming had significant health and nutritional impacts, as well as leading to dramatic gains in school attendance and enrollment. After five years, educational attainment was significantly higher among early treatment school children. Evidence from KLPS linking child health gains (from deworming) and adult human capital formation could be used to justify increased investment in child health and nutrition programs.

The KLPS tracks individuals throughout Kenya using a rigorous two stage tracking system. During the first round of household survey data collection, IPA made direct contact with nearly 85% of target individuals. Round 2 survey data collection is currently underway.

Decentralization: A Cautionary Tale - Public Finance in Kenya

Kenya’s education system blends substantial centralization with elements of local control and school choice.  This project looks at the system of incentives created by elements of decentralization.

Syndicate content
Copyright 2014 Innovations for Poverty Action. All rights reserved.