Download
Theory of Change: Laying the Foundation for Right-Fit Data Collection The first step in designing a right-fit data collection strategy is to create a solid theory of change. A theory of change is a clear visual map that represents how a program will make an impact on the world. It illustrates what goes into a program, what gets done, and how the world is expected to change as a result. A theory of change supports right-fit data collection in several ways: by pointing organizations to the elements of the program they need to track to ensure it is operating as planned; by providing a foundation for impact measurement by differentiating the outputs to be tracked from the outcomes to be measured using a credible counterfactual; and by generating credible research questions. The Goldilocks Initiative does not offer a complete manual on building a theory of change—many resources exist for that—but here we break down the basics of creating a theory of change and explain how a clear theory, to...
Type:
Goldilocks Toolkit
Date:
January 05, 2016
Download
A mobile phone in every person’s pocket will soon be a reality. What does this mean for development organizations? This report reviews the current state of mobile technology for survey and telephonic data collection, activity monitoring, and impact measurement. It also addresses the notion of crowdsourcing, and the various ways it is used to improve organizational decision-making.
Type:
Goldilocks Toolkit
Date:
January 05, 2016
Download
Theory of change is the foundation of a right-fit monitoring and evaluation strategy. A theory of change provides a road map that outlines how a program will work to change outcomes and deliver impact. It identifies the key assumptions of the program and risks to successful implementation, and helps organizations pinpoint the data they need to collect. When done well, a theory of change should also enjoy widespread buy-in from staff throughout the organization. This helps to focus data collection activities on the most important questions about implementation and impact. Building a theory of change with solid theoretical foundations and widespread buy-in requires organizations to invest time and resources into a process with multiple steps and participation at all levels of the program. Chapter 3 of The Goldilocks Problem outlines the key elements that make up a theory of change and walks through several examples. In this article we outline some of the preparatory work needed to guide...
Type:
Goldilocks Toolkit
Date:
January 05, 2016
Download
One key message of the Goldilocks Initiative is that impact evaluation is not for everyone. Yet, even when measuring impact is not feasible, social enterprises and non-profits can still answer important questions about their programs using rigorous measurement techniques. One of these is techniques rapid-fire testing: randomized trials that compare the effect of related interventions on a single, immediate (or short-term) outcome. This method is used to test operational issues and aims to influence immediate outcomes, such as product take-up, program enrollment, loan repayment, and attendance, among others. In rapid-fire tests, participants are randomized into different treatment groups (and sometimes, but not necessarily, a pure control group) and exposed to variations in a program’s design or message. The outcome of interest (usually program take-up or use) is measured and compared across treatment and control groups. Often outcomes are measured administratively, so that there is no...
Type:
Goldilocks Toolkit
Date:
January 05, 2016
Download
Digital Green: Addressing Measurement Challenges in Agricultural Technology Programs The use of information and communications technology (ICT) in agricultural services is becoming increasingly common. These technologies—radio, SMS, television, video, and Internet services—have the potential to help smallholder farmers increase their incomes1 by making it easier for them to learn about and adopt new farming methods, grow higher-value crops, or connect with new markets. Digital Green, an international non-profit organization based in India, uses locally-produced videos and in-person facilitation to share knowledge about improved agricultural and nutrition practices. The program aims to help rural communities across South Asia and Sub-Saharan Africa understand and adopt better agricultural and nutrition practices, and the ultimate goal of the program is to have a positive impact on individual well-being. Digital Green is currently working in nine states in India, and also in Afghanistan,...
Type:
Goldilocks Toolkit
Date:
January 04, 2016
Download
Root Capital: Measuring the Impact of Financing Small and Growing Agricultural Businesses Agriculture accounts for one-third of gross domestic product and three-quarters of employment in sub-Saharan Africa, yet agricultural yields and productivity are the lowest of any region of the world.1 While there are many efforts underway to improve the incomes of poor farmers, a number of market weaknesses hamper these efforts, including poor infrastructure, lack of agricultural support services and credit, and difficulty accessing international markets. Root Capital is an impact investor that seeks to address some of the market problems affecting the rural poor. It provides loans and financial management training to small and growing agricultural businesses, which buy directly from smallholder farmers and sell to larger distributors. Root Capital’s assistance is designed to help small and growing agribusinesses to buy better quality products at higher volumes, and with greater consistency from...
Type:
Goldilocks Toolkit
Date:
January 04, 2016
Download
Splash: Finding the Right Timing for Impact Evaluation Lack of safe drinking water, inadequate sanitation, and poor hygiene contribute to poor health outcomes for children in many countries across the world. Based in Seattle, WA, the NGO Splash provides safe water and hygiene education to approximately 290,000 children in schools and orphanages in urban areas of Asia and Africa. Splash provides urban institutions with water purification systems, teaches children about hygiene, and also works to make safe water provision sustainable by working with local partners to scale up its programs. The program started in 2007 in Cambodia, China, and Nepal and has since expanded to Ethiopia, India, and Thailand. Splash is eager to learn about the impact of its program on the health and development of schoolchildren through a rigorous impact evaluation. Before an organization can measure its impact, however, it is important it has a strong monitoring and evaluation system in place, and in 2015, whe...
Type:
Goldilocks Toolkit
Date:
January 04, 2016
Download
TulaSalud: Challenges of Measuring Impact for a Mobile Health program Rapid expansion of mobile technology across developing countries has ushered in the emergence of the “mobile health” sector, or mHealth, which refers to the use of mobile technology in medical care. TulaSalud is an NGO based in Guatemala that implements an mHealth program that aims to improve health services for rural populations. Founded in 2008 with support from the Tula Foundation, TulaSalud operates in close partnership with the Ministry of Health in Alta Verapaz, one of the poorest regions in the country. Alta Verapaz is characterized by chronic malnutrition and high rates of maternal and infant mortality. TulaSalud uses a mobile application to help community health workers (CHWs) collect patient information and provide basic healthcare services. Health outcomes in Alta Verapaz have improved considerably since TulaSalud introduced the program, but the precise contribution of the mHealth platform to this progress...
Type:
Goldilocks Toolkit
Date:
January 04, 2016
Download
IPA assembled this set of resources for use in designing and running an impact evaluation. Beginning with the need for a theory-driven evaluation, and ending with a set of concrete tools to use in running an evaluation, they cover a range of practical materials useful for organizations that are considering a rigorous impact evaluation. This set of resources cover the following topics: theories of change and impact evaluations determining the right timing for an evaluation selecting the right evaluation methodology designing and managing a randomized evaluation calculating sample size and doing power calculations mobile data collection administrative data data management and analysis For organizations with strong internal technical capacity, these resources will provide valuable practical guidance for designing and implementing a credible impact evaluation. And for organizations without the internal capacity to design and run an impact evaluation, these resources will help key staff mem...
Type:
Goldilocks Toolkit
Date:
January 04, 2016
Download
IPA has compiled this set of resources on data collection and storage to aid in the development of monitoring and evaluation plans. When used in conjunction with a solid theory of change, these resources can help to ensure the collection of credible, actionable, and transportable data. The topics covered include: Designing survey instruments Generating credible survey questions Power calculations Using administrative data for evaluations Using mobile or paper-based surveys Data and code management
Type:
Goldilocks Toolkit
Date:
January 04, 2016
Download
IPA assembled this list of resources for use in designing a program and developing a theory of change. The following resources can be useful throughout the design phase of your program—showing you how others have defined similar problems and in what contexts, which interventions others have tried, and how large the effects of these programs have been found to be. This will be useful in designing the program. These resources can also be useful for thinking ahead to a future evaluation. Many of them describe: the research methods, such as the evaluation type, sample size and sampling strategy; the research questions; the timing of the evaluation; and the outcomes measured. There is no one-size-fits-all evaluation plan, but reviewing what others have done and how they have overcome specific challenges can help you think through options for your own evaluation strategy. Though far from exhaustive, the resources here are a good place to start when designing your program or theory of change....
Type:
Goldilocks Toolkit
Date:
January 04, 2016
Download
IPA assembled this list of resources for use in designing a monitoring system and understanding monitoring as good management. This set of textbook chapters, articles, blog posts, and organizational guides provides a range of information and perspectives on monitoring and management. They draw from a variety of sectors, from health to education to nonprofit and business management. These resources cover: designing a process-evaluation (monitoring) plan methods for monitoring service utilization and program organization detailed steps in building a monitoring system monitoring as management building organizational culture around monitoring strengthening the use of data in program monitoring Taken together, these resources will aid in the development of a monitoring plan and system that reflects CART principles and incorporates useful management practices. If thoughtfully applied, they should help develop an organizational culture built around monitoring and data, one that will help to i...
Type:
Goldilocks Toolkit
Date:
January 04, 2016
Download
IPA has compiled this set of resources on the theory of change and its relationship to program design. They take readers from needs assessments and the importance of the theory of change, through practical discussions of how to develop a theory of change and identify assumptions. These resources cover the following topics: importance of a theory of change needs assessment developing a theory of change identifying assumptions using a theory of change in program design and monitoring theory of change for donors and grantees A solid theory of change is the foundation of strong program design and a sound monitoring and evaluation strategy. These resources go into the depth necessary to build a theory of change and link it to a CART monitoring strategy.
Type:
Goldilocks Toolkit
Date:
January 04, 2016
Download
One Acre Fund: Theory of Change for the Adoption of New Technologies Over the last decade, agricultural productivity of smallholder farmers throughout Africa has remained stagnant – often constrained by lack of access to improved agricultural technologies and well-functioning markets. One Acre Fund is an East Africa-based NGO that seeks to address barriers to improved agricultural productivity of smallholder farmers and to reduce poverty. Its core program offers farmers a set of services and agricultural inputs that include financing, agricultural training (extension services), and post-harvest storage. The organization currently serves approximately 280,000 farmers in Kenya, Rwanda, Burundi, and Tanzania and aims to reach one million farmers by 2020. One Acre Fund’s measurement approach focuses on activity tracking and monitoring to assess how their model works in practice. This tracking data is used to improve the program and to scale it to new areas. The organization enhances the cr...
Type:
Goldilocks Toolkit
Date:
January 04, 2016
Download
Unconditional cash transfers are increasingly recognized to be a cost-effective strategy for reducing poverty. This is in large part due to a small but growing body of research that shows significant positive welfare effects of programs that provide cash to the poor with no strings attached. This subject sparked a debate in the international development field about an alternative approach to improving social welfare, as compared to traditional forms of aid. GiveDirectly is a New York-based nonprofit that provides secure direct cash transfers to the extreme poor in Kenya and Uganda primarily via mobile phone. Since it was founded in 2009, GiveDirectly has grown rapidly: as of 2013, it had transferred $1.5 million to recipients, with $3.4 million scheduled for delivery. The organization has enrolled nearly 10,000 recipients to date and planned to move $20 million during FY2015. Beyond the immediate goal of transforming lives of the poor by providing direct cash transfers, the founders of...
Type:
Goldilocks Toolkit
Date:
January 02, 2016
English
In the previous IPA Health Bulletin (August 2015) we discussed the discernible positive differences in household behaviors and child health in districts where the Community Health Assistants were recruited using career incentives (“Career CHAs”) in comparison with those recruited using community incentives (“Control CHAs”). Over the past months IPA has been meeting with government stakeholders - including the MOH HR Technical Working Group - and presenting what these results mean for the cost-benefit of providing career incentives for the CHAs. Here is a brief summary of that presentation.
Country:
Program area:
Date:
January 01, 2016
English
We implement an artefactual field experiment in rural Malawi to study revisions of prior choices regarding future income receipts. This allows examination of intertemporal choice revision and its determinants. New tests provide evidence of self-control problems for some participants. Revisions of money allocations toward the present are positively associated with refined measures of present-bias from an earlier survey, and with the randomly assigned closeness in time to the first possible date of money disbursement. We find little evidence that revisions of allocations toward the present are associated with spousal preferences for such revision, household shocks, or the financial sophistication of respondents.
Country:
Type:
Working Paper
Date:
January 01, 2016
English
We implemented a randomized intervention among Malawian farmers aimed at facilitating formal savings for agricultural inputs. Treated farmers were offered the opportunity to have their cash crop harvest proceeds deposited directly into new bank accounts in their own names, while farmers in the control group were paid harvest proceeds in cash (the status quo). The treatment led to higher savings in the months immediately prior to the next agricultural planting season, and raised agricultural input usage in that season. We also find positive treatment effects on subsequent crop sale proceeds and household expenditures. Because the treatment effect on savings was only a small fraction of the treatment effect on the value of agricultural inputs, mechanisms other than alleviation of savings constraints per se are needed to explain the treatment’s impact on input utilization. We discuss other possible mechanisms through which treatment effects may have operated.
Country:
Program area:
Type:
Published Paper
Date:
January 01, 2016
English
Currently very few conflict-affected countries have met a single Millennium Development Goal. The need for effective peacebuilding programmes is acute, as development indicators are dramatically low and poverty levels are dramatically high in these conflict-affected areas. This scoping paper by Annette N Brown, Faith McCollister, Drew B Cameron, and Jennifer Ludwig reviews the supply of and demand for evidence from impact evaluations and systematic reviews on peacebuilding interventions.  The analysis focused on three inputs, which were presented according to a common framework of intervention and outcome categories developed by key stakeholders working in the area of peacebuilding. The first two inputs, a review of current and recent programming across 25 intervention categories and the results of a stakeholder survey, provide information on the demand for more and better evidence. The third input is a 3ie evidence gap map, which illustrates the evidence base of impact evaluations and...
Program area:
Type:
Report
Date:
December 31, 2015
Spanish A4
En el marco de la gestión pública por resultados, se viene fortaleciendo una cultura de seguimiento y evaluación en el país, lo cual permite la generación de evidencia para la formulación de políticas públicas de efectiva intervención para el desarrollo y bienestar de la población. En ese contexto se crea la Red Peruana de Monitoreo y Evaluación-REDPERUME, como una organización integrada por actores públicos y privados, que promueve la institucionalización de una cultura de seguimiento y evaluación, contribuyendo de este modo a la efectividad de las políticas, planes, programas y proyectos. Esta Red surge hacia finales del año 2010, como un grupo adscrito a la Red Latinoamérica y del Caribe de Monitoreo y Evaluación-REDLACME, a iniciativa del Ministerio de la Mujer y Poblaciones Vulnerables-MIMP. A la fecha cuenta con 286 integrantes de diversos Sectores del Estado, Organizaciones Privadas y de la Sociedad Civil, encontrando en la Red un espacio vivo para el intercam...
Country:
Type:
Report
Date:
December 04, 2015

Pages