Full Citation
Title: How Transparency and Reproducibility Can Increase Credibility in Policy Analysis
Citation Type: Dissertation/Thesis
Publication Year: 2017
ISBN:
ISSN:
DOI:
NSFID:
PMCID:
PMID:
Abstract: The analysis of public policies, even when performed by the best non-partisan agencies, often lacks credibility (Manski, 2013). This allows policy makers to cherry- pick between reports, or within a specific report, to select estimates that better match their beliefs. For example, in 2014 the Congressional Budget Office (CBO) produced a report on the effects of raising the minimum wage that was cited both by opponents and supporters of the policy, with each side accepting as credible only partial elements of the report. Lack of transparency and reproducibility (TR) in a policy report implies that its credibility relies on the reputation of the authors, and their organizations, instead of on a critical appraisal of the analysis. This dissertation translates to policy analysis solutions developed to address the lack of credibility in a different setting: the reproducibility crisis in science. I adapt the Transparency and Openness Promotion (TOP) guidelines (Nosek et al, 2015) to the policy analysis setting. The highest standards from the adapted guidelines involve the use of two key tools: dynamic documents that combine all elements of an analysis in one place, and open source version control (git). I then implement these high standards in a case study of the CBO report mentioned above, and present the complete analysis in the form of an open-source dynamic document. In addition to increasing the credibility of the case study analysis, this methodology brings attention to several components of the policy analysis that have been traditionally overlooked in academic research, for example the distribution of the losses used to pay for the increase in wages. Increasing our knowledge in these overlooked areas may prove most valuable to an evidence-based policy debate.
Url: https://www.rand.org/content/dam/rand/pubs/rgs_dissertations/RGSD300/RGSD394/RAND_RGSD394.pdf
User Submitted?: No
Authors: Hoces de la Guardia, Fernando
Institution: PARDEE RAND GRADUATE SCHOOL
Department:
Advisor:
Degree:
Publisher Location:
Pages:
Data Collections: IPUMS CPS
Topics: Other
Countries: