May 31, 2011

The media attention surrounding the release of More Than Good Intentions and Poor Economics has led to a few criticisms of RCTs cropping up around the interwebs:

Whats that, you want a handy 5-bullet-point definitive rebuttal list? Sure thing;

  1. RCTs can't answer every question. Nobody ever said that. 
  2. Having said that, there are probably more questions that they can answer than you originally thought. 
  3. Of the set of questions that they can answer, they are probably the best way of answering them. This shouldn't be too controversial a claim. 
  4. Billions of dollars, and probably the vast majority of aid spending, still goes on projects that have not been rigorously evaluated. So we aren't quite yet in the position to be worried about over-kill. 
  5. Yes we should be careful about extrapolation and generalizing. But this also applies to other sources of evidence, and is not an argument against relying on evidence to the fullest extent possible, or generating as much new evidence as is cost-effective.


Tom Murphy just sent me remarks on the "evaluation fad" by Jonathan C. Lewis at the Microfinance USA Conference

Dennis Whittle says we need to "make the beneficiary king". 

Sasha Dichter laments extrapolation from too few studies.

Ian Thorpe suggests that RCTs are going through a familiar "hype cycle."

Do add any other links we may have missed here.