The Research Methods Initiative is a collaboration between Innovations for Poverty Action and the Global Poverty Research Lab at Northwestern University that supports systematic studies into how to improve the methods and measurement of key outcomes in global poverty research.
IPA manages over 300 active studies in over 20 countries led by more than 575 researchers, and the Global Poverty Research Lab (GPRL) is an academic hub that uses empirical evidence to address the challenges of overcoming poverty and improve wellbeing in the developing world. With such a volume of research, IPA and GPRL have faced recurring questions regarding the best ways to ensure quality and consistency in field studies.
The Research Methods Initiative was created to help answer these questions by designing and implementing methodological studies across many projects, building and collaborating with a network of interested researchers, and developing technical products and training. IPA and GRPL’s research informs policy, but quantitative research can be biased without careful measurement and estimation. Improving these methods and measurement tools will improve data quality, promote innovation in methods and measurement, and provide higher confidence in research results.
The Research Methods Initiative will be comprehensive in examining sources of measurement error in research studies including in questionnaire design, sampling, field work implementation and validation of key indicators. The initiative will be organized around three themes: (1) Research Design, (2) Questionnaire Design, and (3) Fieldwork Implementation and Data Quality.
Theme 1: Research Design
At the heart of empirical analysis is the problem of establishing causal relationships in the data that we collect and from which we can provide policy recommendations that are effective. Randomized control trials have provided an important new tool in addressing identification in research design, but we can learn more. Examples of work within this theme will include innovations in RCT designs, the implications of different sampling strategies and statistical power in research designs, and how we can design for replication and scale.
Theme 2: Questionnaire Design
This theme will focus on how survey instruments are designed and the consequences of alternative choices that a researcher may make in how data is collected. Errors in measurement may bias key variables which have important consequences not only in the representation of population level characteristics, but also the empirical relationships estimated. Examples of work within this theme will include estimating relative biases in alternative questionnaire designs from recall periods, question framing, proxy rules, and alternative units of analysis. We will also explore integrating technology in person to person interviews to improve measurement and the benefits and costs of using standardized modules.
Theme 3: Fieldwork Implementation and Data Quality
Theme 3 will focus on implementation decisions that limit non-random measurement error in data collected from recruitment, training, monitoring and data validation. Studies under this subtheme have the potential to yield insights on enumerator labor markets, interdisciplinary insights into personal interviewing, as well as practical tools to be integrated as best practices within IPA and other data collection organizations. Examples of work under this theme will explore enumerator effects including how recruitment, training and motivation of enumerators improves data quality. The Research Methods Initiative will facilitate the standardization of IPA’s quality assurance methods and study the effects of old and new data quality tools. Researchers are interested to know how these data quality tools impact the reliability of data. The tools include observational visits, audio audits, backchecks, high frequency checks, nightly monitoring reports and machine learning to recognize data falsification.
The Research Methods Initiative will leverage IPA’s sector programs, such as Financial Inclusion and Social Protection, to study and improve the performance of sector-specific modules. More granular research is also possible, such as how changing the phrasing on one particular survey question affects responses.
By drawing on the wealth of local knowledge and surveying experience IPA has built through its country offices, the Research Methods Initiative is leading the way in the development of accurate, standardized measurement tools for anti-poverty studies.