July 07, 2009

How has Michael Jackson's death helped advance the cause of IPA? I've just come back from rural Cusco, Perú, where we were training credit officers for our newest project, where we use video and radio to help teach village bank clients concepts related to financial literacy. Earlier, I had talked to the credit officers about the importance of randomized trials, mentioned results from previous studies, outlined the indicators we were thinking of measuring, and explained relevant details about the assignment of treatment and control banks. I am sure I reminded them of a sanctimonious middle school history teacher, and bored them to oblivion.

Then the head of training for our partner organization divided the credit officers into four groups. Using folded up pieces of paper, she had them draw lots so they would be assigned to "treatment" and "control." The treatment groups got to go inside and watch Michael Jackson's "Earth Song" video. The control groups were sent outside, where I watched them sidle up to the windows for a surreptitious peek and listen by the door in an attempt to find out what the heck was going on.
She then handed out a three question survey about Michael Jackson, and posted responses for everyone to see. Those who did not see the video (and who had not wandered off to do errands while waiting to be let back in) wrote that he was a pop star with a lot of personal problems. The "treatment group," on the other hand, wrote that he cared about the environment, the poor, social justice, and so on. "You see!" She exulted. "Our training video had an impact on attitudes about Michael Jackson!"

Okay, so this particular mini experiment had sample size issues, spillover problems, and some attrition bias. Was it successful at capturing the attention of the staff members? Definitely. Was I as successful? Probably not.

Final score: Michael Jackson, 1, sanctimonious middle school history teachers, 0.