A recent profile of work by John Ioannidis caused a bit of a stir, showing that most medical research is likely to be wrong due to publication bias and small samples. On the CGAP blog Jonathan Morduch (NYU and the Financial Access Initiative) argues that these concerns are less important for microfinance RCTs, but we do have other things to worry about. 

on this score, Ioannidis’s criticisms are far less of a concern when it comes to microfinance.  We have tighter theoretical understandings, a smaller set of hypotheses, and, usually, bigger samples. But don’t relax completely.  Microfinance research has its own set of concerns.

Namely replication, external validity, and whether we are even able to ask the most interesting questions. 

The problem with imperfect studies is that you don’t know how big the bias is (but you know the bias could be really big).  RCTs have taken us a huge step forward, and promise to deliver clean estimates of various slices of microfinance impact.  In the end, the big question is not the one Ioannidis asks (are the results apt to be right or wrong?).  The big questions concern how the particular results matter to our understanding of microfinance broadly.

Worth reading in full.