IPA’s extensive experience in managing academic-level research surveys for randomized trials has led it and MIT’s Jameel Poverty Action Lab to jointly develop a standard set of procedures and processes for managing projects. These protocols include specific requirements for adhering to human subjects regulations, developing survey instruments, fielding data collection teams, implementing data quality reviews, and producing and cleaning datasets for analysis. IPA requires all of its projects to comply with these research protocols. In support of projects, IPA’s Research & Knowledge Management Department:

  • Works closely with Country Directors, Research Managers, Survey Coordinators and Data Coordinators throughout IPA’s country programs to assist project teams in research implementation and management, and ensure compliance with IPA Research Protocols
  • Supports and advises IPA project staff prior to survey launch to identify the best hardware and software options, create budgets, hire or train programmers and field staff, and develop work plans and timelines
  • Provides resources and technical support for projects. For projects administering electronic surveys, this support may include programming, testing, and field management of electronic survey instruments, as well as protocols and tools for tracking progress and quality-checking incoming data in real-time
  • Provides assistance and support for computer assisted interviews through Windows and Android devices


IPA's Research Protocols

*Research Protocols are currently under review, expect changes in Q1 2016.


Research Design

Protocols:
 

  • Monitor Treatment & Control compliance.
  • Randomization
    • Field Randomization must be audited
    • Randomization code must be documented and able to be replicated

Recommendations:

  • Power calculations documented and shared

Project Management

Protocols:
 

  • All project activities are budgeted for and approved in advance.
  • Documentation
    • Project Manual describing the entire project
    • Project documents are clearly labeled and organized
    • Sample tracking file to monitor attrition across treatment and control.

Recommendations:

  • Follow the recommended folder structure.
  • Upload final documentation to the project workspace on Box.

Human Subjects

Protocols:
 

  • IRB approval before any activity involving human subjects
  • IRB training certificates for all personnel who will handle the data and identified surveys in the field
  • Administer informed consent to every respondent
  • All field officers (including surveyors) sign confidentiality forms
  • Full local and international IRB compliance

Recommendations:

  • IRB application submitted at least 6 weeks before data collection begins
  • Reference the informed consent checklist for enumerators to describe key human subjects considerations

Full IRB requirements and procedures can be viewed here.


Measurement & Questionnaire Design

Protocols:
 

  • Informed consent included with every survey
  • Unique ID for every data point
  • Translation
    • Written: The questionnaire must be back-translated.
    • Oral: The survey team creates a dictionary of key terms and their translations.
  • All final questionnaires shared on Box.
  • All questions are piloted before data collection begins.
  • PII is removable from paper questionnaire.

Recommendations:

  • Pilot data is entered and analyzed.
  • Use two unique ids for each data point.
  • Questionnaires are reviewed by IPA’s Questionnaire Review Committee.

Data Collection

Protocols:
 

  • Training
    • There is a survey manual for each survey.
    • Appropriate length of training depending on survey length and complexity.
  • Team Structure
    • Appropriate ratio of surveyors for each field supervisor.
    • Appropriate ratio of field teams per field manager.
  • Monitoring
    • Accompaniments: each surveyor is occasionally observed by a supervisor.
    • Editing & Logical Checks: A dedicated person who edits every paper survey or checks the data every day.
    • Back checks are done with multiple back check instruments that cover the entire survey.
    • Random Spot Checks: Done weekly by an independent monitor on every surveyor.
  • Documentation
    • Data collection plan submitted for review of team structure and monitoring plan
    • Keep a master tracking document that tracks all field interviews separately from the data.
    • Write survey notes to document decisions made during data collection.
    • Attrition rates are documented throughout the survey.
  • Securely store and transport of PII in both paper surveys and electronic data.

Recommendations:

  • Train more surveyors than will be hired.
  • Surveyors are administered tests and quizzes during training to assess their knowledge of the questionnaire.
  • Accompaniments: A minimum of 10% of respondents surveyed by each surveyor are observed by a supervisor.
  • Back checks: 10 % or 40 surveys per week (whatever is higher). A bare minimum of 10 questions or 10% of your total survey instrument.
  • Random Spot Checks: At least 15% by senior project management and more during the first two weeks.
  • Each monitoring activity is scheduled and designed so that each surveyor is observed, editing and back checked, at minimum, once a week.
  • Monitoring feedback, including back check analysis, happens daily as part of team meetings. 

Data Entry & Digital Data Collection

Protocols:
 

  • Software encrypts PII or remains offline.
  • Double entry and reconciliation of all data.
  • An independent audit is performed on double-entered data
  • Security
    • PII is stored separately from original data (both paper and electronic) and encrypted.
    • All data is securely backed up offsite.
    • All transmissions of data are secure.


Recommendations:

  • Survey Specifications and data entry/CAI programs submitted
  • Data available in a common format (.dta, .sav, .csv) with associated codebook
  • All final datasets made publicly available after completion of proposed

Data Cleaning & Analysis

Protocols:
 

  • Reconciliation (with entries, audits, field tracking data), high-frequency checks
  • De-identify data before beginning analysis.
  • Properly comment all .do files
  • Use relative references in all .do files.
  • Properly name and label all variables.

Administration - Finance, Human Resources, Legal

Protocols:
 

  • Finance & Grants: Budgets, actuals, reporting.
  • Human Resources: Follow safety and security protocols.
  • Legal: Obtain approval before signing MOUs.
  • Communications: Obtain PI approval before publishing results externally (blogging, social media, etc)
  • Policy: Document the cost of the intervention and intervention materials.