New book out by IPA President & Founder Dean Karlan and Jacob Appel
When we read about a successful program that mitigates poverty or improves health, we seldom consider how much work went into its evaluation and how many ways the process could go wrong. Drawing on their own and others' extensive experience in the field, Karlan and Appel provide vivid examples of failure to help future evaluators avoid common pitfalls.
-Paul Brest, Stanford Law School
At IPA, we believe we can all learn a lot from what has not worked as well as from what has. We are constantly learning from our own failures and improving our methods for collecting data in the field. We believe it is important to be transparent when plans go awry, and that sharing our failures with others is a public good, improving the likelihood that others will succeed in the complicated task of conducting high quality research in challenging, and often unpredictable, settings.
In Failing in the Field, IPA Founder Dean Karlan and former IPA research associate Jacob Appel delve into the common causes of failure in field research so that researchers and decision-makers might avoid similar pitfalls in future work. Drawing in part on IPA’s experiences implementing research in developing countries, this book delves into failed studies and helps guide practitioners as they embark on their research.
From experimental design and implementation to analysis and partnership agreements, Karlan and Appel show that we can learn important lessons from failures at every stage. They describe five common categories of failures, review six case studies in detail, and conclude with some reflections on best (and worst) practices for designing and running field projects, with an emphasis on randomized controlled trials.
Cracking open the taboo subject of the stumbles that can take place in the implementation of research studies, Failing in the Field is a valuable “how-not-to” handbook for conducting fieldwork and running randomized controlled trials in development settings.
More praise for Failing in the Field:
“Winston Churchill described success as 'stumbling from failure to failure with no loss of enthusiasm.' That certainly describes my own career. This book not only tells researchers and policy makers like me that we are not alone, it also tells us what we can learn from our mistakes. A great read." – Christopher Blattman, University of Chicago
"This unusual and refreshing book is a good complement to the existing literature on impact evaluation and documents the practical issues involved in implementing randomized control trials. The authors look at real examples of what goes wrong in the field and provide a nice framework for thinking about how to avoid failures." - Rachel Glennerster, coauthor of Running Randomized Evaluations
"Using a rich set of examples, Failing in the Field describes failures that occur because the design or implementation of a research study does not yield data that can answer the questions it was intended to. By showing that mistakes in research design can be systematic, this book could benefit many students before they embark on their own studies. It was a pleasure to read." - Karla Hoff, World Bank
About the Authors
Dean Karlan is the Frederic Esser Nemmers Distinguished Professor of Economics and Finance at Northwestern University's Kellogg School of Management and Co-Director of the Buffett Institute's Global Poverty Research Lab. He is the Founder of Innovations for Poverty Action and Co-Founder (with Elijah Goldberg) of ImpactMatters.
Jacob Appel previously worked with Innovations for Poverty Action, and obtained his MPA at Princeton University's Woodrow Wilson School of Public and International Affairs in 2017. Karlan and Appel are the coauthors of More Than Good Intentions: How a New Economics Is Helping to Solve Global Poverty.