We conduct field experiments to probe the conceptual foundations of approaches to poverty alleviation, and conduct randomized program evaluations to generate insights into what works--and why--in fighting poverty.

Why we evaluate
Evaluating development programs is a critical step in increasing our understanding of the types of interventions that lift people out of poverty--and helps us to direct resources towards interventions that have been proven to work. Through evaluation we can identify successful programs, and just as importantly, it helps the programs themselves to learn which particular strategies have the greatest impact, and for which specific types of clients.

Without a rigorous evaluation it is impossible to tell to what extent changes in people's lives are attributable to a given program. Participants may be affected by various interventions in their region, macroeconomic growth or contraction, or shocks such as favorable or unfavorable harvests. Thus even if participants' welfare is improving it may not be due to the program in question (or conversely, if welfare isn't improving it might have been worse in the absence of the program).

Evaluations help us clarify the impact of the program itself, and how well it works for outcomes of interest: changes in income, health, and education; for women, for children, for the very poor, and so on. Where a particular intervention is particularly effective it can be shared and adopted by other programs around the world, and where an intervention fails to deliver the desired impact it can be retooled (and retested) or dropped in favor of a more effective strategy.

Why we only use randomized trials to evaluate programs
We use randomized trials, the methodology routinely used in medicine to test the effects of drugs before they are made widely available, to evaluate programs that address issues central to enhancing lives and promoting economic growth and development, including childhood education, health care, water, microfinance, agriculture, and gender equity and empowerment.

We use randomized trials because they address completely the complex challenge of identifying the impact of a social program, making it possible to isolate the program effects from other factors. Recent innovations have made it easier to use randomized trials in ways compatible with the practices and priorities of policy practitioners in developing countries; consequently, many more successful randomized evaluations are now conducted.

Before researchers identified ways to integrate randomized trial techniques smoothly into the operations of development programs, the standard technique for measuring program impact was to compare the outcomes of participants to a group of similar non-participants. But this method does not produce a reliable estimate of the impact of the program because it compares people who chose to participate in the program to people who did not, or it compares people in a village which the program chose to serve, to people in a village the program for some reason skipped over. Such comparisons do not account for the fact that subtle differences between the people compared may have--with or without the program--resulted in vastly different outcomes.

We now know that the subtle differences between the people who select into programs and those who do not can have enormous implications for the conclusions we draw about the effectiveness of particular interventions. For instance, in a study of village banks in Thailand, economist Brett Coleman found that participants in a program designed for the poor tended to be wealthier than non-participants, even before they joined the program. Non-participants may refuse to join because they are afraid, participants may join because they are more motivated, non-participants may be excluded because their village is too far from the main road, participants may be included because they are poor or have poor initial outcomes.

All these, and any other subtle difference, may have similarly significant consequences on the results of the evaluation. If we, for example, compare program participants to non-participants in a poorer village we could overstate the impact of the program. On the other hand, if we compare them to non-participants in a wealthier village we could understate the impact, or make an effective program appear harmful to participants!

We can be certain that the only difference between program participants and non-participants is the effect of the program itself when program participants are randomly selected from a potential population of participants (such as individuals, communities, schools or classrooms). By integrating randomized control techniques into development programs we can be assured that, on average, participants are not systematically different from non-participants, and any differences in their outcomes are due to the program.

Studies which have calibrated the accuracy of non-experimental techniques against randomized controlled trials have found that even the best efforts of experienced program evaluators cannot match the scientific results of randomized control trials (cf. Gary Burtless: "The Case for Randomized Field Trials in Economic and Policy Research"). What's more, as the results of randomized evaluation do not rest on subtle statistical assumptions, it is easier for non-experts to understand the findings and implications and use them in designing or implementing interventions. The challenge is now to make randomized trials standard for measuring the impact of poverty programs--just as they are in testing medicine--and ensuring that policy decisions are made on the basis of the concrete, rigorous, and practical evidence yielded by randomized trials.

In the past few years, researchers at IPA and other organizations, notably from one of our partner organizations, the Abdul Latif Jameel Poverty Action Lab at MIT, have shown that randomized evaluations can be conducted in a wide variety of settings without distracting the operations of the program, often at lower cost than traditional evaluations. Randomized trials have already been used to generate precise estimates of the effectiveness of a wide variety of interventions, including new savings products in the Philippines, business training in Peru, and marketing techniques in South Africa.

Our standards for determining what works are high. We extend the scientific approach to development, beyond applying the most rigorous methods available to development programs, by replicating experiments in different contexts and locales.

Re-evaluating under different contexts allows us to determine the extent to which findings and insights from the original experiment generalize. This allows us--and the decision makers who use our results to guide their policy decisions--to conclude with greater certainty that the findings were not due to idiosyncrasies of place, time, and people in the original experiment.

This part of our work speaks to the depth of our policy interest, for re-evaluation yields not so much the novel answers to question of academic interest, but the answers as to what strategies really work, answers that policymakers and other practitioners need.

IPA strives to transform its findings and insights into innovative action. We disseminate the evidence we generate to development practitioners. And, where appropriate, we work closely with partners to facilitate the replication of effective programs to other areas of the world.