Publication bias in the social sciences since 1959: Application of a regression discontinuity framework

Abstract

While publication bias has been widely documented in the social sciences, it is unclear whether the problem aggravated over the last decades due to an increasing pressure to publish. We provide an in-depth analysis of publication bias over time by creating a unique data set, consisting of 12340 test statistics extracted from 571 papers published in 1959-2018 in the Quarterly Journal of Economics. We, further, develop a new methodology to test for discontinuities at the thresholds of significance. Our findings reveal, that, first, in contrast to our expectations, publication bias was already present many decades ago, but that, second, bias patterns notably changed over time. As such, we observe a transition from bias at the 10 percent to bias at the 5 percent significance level. We conclude that these changes are influenced by increasing computational possibilities as well as changes in the acceptance rates of scientific top journals.