BIBLIOGRAPHY

Publications, working papers, and other research using data resources from IPUMS.

Full Citation

Title: Site Selection Bias in Program Evaluation

Citation Type: Working Paper

Publication Year: 2012

Abstract: Site selection bias occurs when the probability that partners adopt or evaluate a program is correlated with treatment effects. I test for site selection bias in the context of the Opower energy conservation programs, using 111 randomized control trials (RCTs) involving 8.6 million households across the United States. Predictions based on rich microdata from the first ten replications substantially overstate efficacy in the next 101 sites. There is evidence of two positive selection mechanisms. First, local populations with stronger preferences for environmental conservation both encourage utilities to adoptthe program and are more responsive to the treatment. Second, program managers initially target treatment at the most responsive consumer sub-populations, meaning that efficacy drops when utilities expand the program. While it may be optimal to initially target an intervention toward the most responsive populations, these results show how analysts can be systematically biased when extrapolating experimental results, even after many replications. I augment the Opower results by showing that microfinance institutions(MFIs) that run RCTs differ from the global population of MFIs and that hospitals that host clinical trials differ from the national population of hospitals.

User Submitted?: No

Authors: Allcott, Hunt

Series Title:

Publication Number: 18373

Institution: National Bureau of Economic Research

Pages:

Publisher Location: Cambridge, MA

Data Collections: IPUMS NHGIS

Topics: Other

Countries:

IPUMS NHGIS NAPP IHIS ATUS Terrapop