Kevin Zollman (Carnegie Mellon University)
Published scientifc results might be in error. That is the nature of inductive reasoning, it is never certain. For a given set of interests, there will be an appropriate balance between the importance of the result (if true) and the risk of error (if false). But, the career motivation of scientists might encourage them to err on the side of publishing too quickly. This is especially serious in cases where the result might make a “big splash.” The replication crises observed in psychology, medicine, and related felds may be due, in part, to this confict between social goals and individual career motivations (Heesen 2017a). To some extent we must be willing to tolerate a certain level of error, that is the nature of inductive inference. However, if we want to encourage scientists to be more careful, we must counteract the motivation to publish potentially erroneous results. There are essentially two avenues that the scientifc community might pursue. First, we could make it structurally more difcult to publish potentially erroneous results. For example, a recent proposal suggested changing the p-value required for publication (Benjamin et al. 2017). Second, we might socially sanction scientists whose research is overturned in order to efectively deter the motive to rush publication in the frst place. The negative reactions to replication failures and retractions show that such social sanctions do exist (Lu et al. 2013). In this paper, I focus on this second potential solution. I argue that social sanctions by themselves may not be adequate. Through a simple decision-theoretic model, I illustrate how a scientist might actively game a punishment system in order to have a successful career despite only publishing results she knows will eventually be overturned. Because there is time between the publication of a result, and its eventual rebuttal, the scientist has a brief window of positive rewards. In many sciences, fame translates into resources for more research. If the scientist is able to translate her transient success into funding ever more exciting research, she can then produce new erroneous results that will swamp the negative criticism from the impending replication failures. I call this strategy “the scientifc Ponzi scheme.” In this paper, I provide a mathematical model of the situation and give the conditions under which it is possible. The example illustrates, I claim, a central difculty with the deterrence strategy for preventing the intentional publication of misleading results
