March 25, 2010

 

We at IPA have recently been delving into the world of online advertising to help us spread the gospel of rigorous impact research.  Being who we are, we could not resist this opportunity to run a field experiment.  We designed one that would help us optimize our advertising strategy while also settling an important score: which academic institution's rep pulls the most weight in cyberspace? Our ad was simple:

Poverty Research

Breakthroughs to Fight Poverty

By [randomized] Researchers

 

Inside the brackets in the third line, Google ads then randomly inserted one of nine university names, one of three acronyms (IPA, JPAL, or FAI) , one of three "impostor" acronyms (ITA, GTAM, and MAI) that were phonetically similar to the real acronyms, or one of three generic words (university, top, and academic).

               Needless to say, we got some interesting results.  Surprisingly, Nobel factory UChicago fared the worst among all university brand names that we tested.  Interestingly, albeit we will admit this was not our prior intended subsample analysis, turns out that the cities of New York and Chicago were both the laggards in this race; NYU, Columbia, Northwestern, and UChicago had the lowest response rates among universities tested.  The top ones were Dartmouth, MIT, Yale and Harvard, in order.

               As you can see, generic keywords "university," "top" and "academic," were significantly more effective than the ones naming schools or organizations. 

Among the acronyms, FAI fared the best, outperforming Princeton, Harvard, and Yale.  J-PAL narrowly edged out its counterpart IPA, and was right on the heels of Ivies Princeton and Harvard.  But before any acronym organization gets too excited, we must note there was no significant difference between real and impostor acronyms.  Although J-PAL and FAI marginally outperformed their impostors, IPA actually had a slightly lower click-through rate than "impostor" ITA.  These findings unfortunately temper our enthusiasm at IPA and its fellow research groups apparently having as big a rep in cyberspace as the top Ivies. 

"Real" vs. "Impostor" - Pooled

 

Coef.

Std. Err.

t

P>t

95% CI

real

0.0000784

0.000214

0.37

0.714

-0.0003405

0.000497

punct

0.0002146

0.000217

0.99

0.322

-0.0002101

0.000639

_cons

0.0105644

0.000184

57.44

0

0.0102039

0.010925

n (# obs)

957,443

         

IPA vs ITA

 

Coef.

Std. Err.

t

P>t

95% CI

real

-0.000244

0.000512

-0.48

0.633

-0.0012464

0.000758

punct

0.0007773

0.000511

1.52

0.128

-0.0002246

0.001779

_cons

0.0101261

0.000361

28.03

0

0.0094181

0.010834

n (# obs)

239,902

         

FAI vs. MAI

 

Coef.

Std. Err.

t

P>t

95% CI

real

0.0001344

0.000338

0.4

0.691

-0.0005283

0.000797

punct

-0.0002938

0.000338

-0.87

0.385

-0.0009563

0.000369

_cons

0.0108223

0.000308

35.14

0

0.0102188

0.011426

n (# obs)

319,573

         

J-PAL vs GTAM

 

Coef.

Std. Err.

t

P>t

95% CI

real

0.0000617

0.000367

0.17

0.866

-0.0006568

0.00078

punct

0.000635

0.000367

1.73

0.083

-0.0000836

0.001354

_cons

0.0105066

0.000317

33.1

0

0.0098846

0.011129

n (# obs)

397,968

         
           
             

To increase the probability that we would offend someone with this post, we also ran a descriptive regression examining the correlation between select university attributes and its apparent popularity in cyberspace.  Its results come with the disclaimer that there are only nine data points and no identification strategy whatsoever, so readers are warned to interpret this regression at their own risk: 

   

Coef.

Std. Err.

T

P>|t|

95% Conf.

 

rank

0.003283

0.00462

0.71

0.529

0.01142

0.017985

 

selectivity

-0.00767

0.004306

-1.78

0.173

0.02138

0.006031

 

endowment

-0.00214

0.003241

-0.66

0.556

0.01246

0.008175

 

attractiveness

0.080691

0.059113

1.37

0.266

0.10743

0.268816

 

drug use

0.069523

0.06215

1.12

0.345

0.12826

0.267311

 

Constant

0.718469

0.282063

2.55

0.084

0.17918

1.61612

 

N (# obs.)

9

         

The dependent variable is the percentage of ad views that actually resulted in clicks.  "Rank" indicates the 2010 US News and World Report ranking of each school.  "Selectivity" is the 2010 undergraduate acceptance rate in percentage points, endowment is endowment size in billions, "attractiveness" is an index measuring how attractive the undergraduate females were (according to a survey of students and alumni from collegeprowler.com), and "drug use" is an index measuring the prevalence of undergraduate drug use (higher scores indicate lower prevalence, again from collegeprowler.com).  Interestingly, selectiveness is the most significant right-hand variable; the negative point estimate indicates that more exclusive schools got higher click rates.  Second is female attractiveness, which is positively correlated with performance.  Low reported prevalence of drug use is correlated with more clicks.

And, as we are now learning, adding 2010 to your headline makes a bigger difference than anything we tested here.