Funding, Publication, and the Credit Economy in Science

Carl Bergstrom (University of Washington), Kevin Gross (North Carolina State University)
A large fraction of scientifc research funding is allocated through a system of grant proposals and awards. We use the economic theory of contests to analyze the scientifc efciency of this process. Investigators participate in contests to write high-quality proposals. Funding agencies use these contests not as a mechanism for extracting work from participants, but rather as a screening mechanism intended to reveal the most promising research projects. As a frst approximation, the work that investigators do in proposal preparation provides no extrinsic value to the funder. We fnd that the efort researchers expend in preparing proposals may be comparable to the total scientifc value of the additional funding. The problem may be exacerbated as the fraction of funded proposals drops. When investigators derive non-scientifc utility from their funding successes (in the forms of e.g., hiring, promotion, tenure, or reputation), the net efect of a funding program on scientifc productive can be negative. We suggest that partial lotteries for funding may ameliorate the problem by reducing the intensity of competition and the extra-scientific benefits associated with funding success.

Carole J. Lee (University of Washington, Seattle)
Within the scientific community, there is a common understanding that its reward system drives problematic behavior and that, in order to improve science, the scientific community must coordinate across institutions to change how credit is assigned to individual scientists. But, how do we go about identifying how to assign credit to any given behavior? This talk will explore the philosophical challenges associated with assigning credit — drawing practical examples from the current context of science's complex and dynamic culture — with an eye towards identifying how individuals and institutions can resolve ambiguities in the assignment of credit.

Cailin O'Connor (University of California, Irvine), James Owen Weatherall (University of California, Irvine)
In this talk, we ask: how can funding agencies and scientific communities make use of credit incentives to protect the public from industry? Low powered studies are more likely to spuriously support erroneous conclusions. To industry interests every individual study that happens to show that, say, smoking is safe becomes a powerful tool in shaping public belief. Science needs to shape internal credit granting systems to remove incentives to publish many, low-powered 'least-publishable units'. This reasoning also means that funding bodies should allocate money to a few very high-powered studies, rather than splitting it up into many small grants.

Remco Heesen (University of Cambridge), Liam Kofi Bright (Carnegie Mellon University)
Pre-publication peer review should be abolished. We consider the effects that such a change will have on the social structure of science, paying particular attention to the changed incentive structure and the likely effects on the behavior of individual scientists. We evaluate these changes from the perspective of epistemic consequentialism. We find that where the effects of abolishing pre-publication peer review can be evaluated with a reasonable level of confidence based on presently available evidence, they are either positive or neutral. We conclude that on present evidence abolishing peer review weakly dominates the status quo.

Kevin Zollman (Carnegie Mellon University)
Published scientifc results might be in error. That is the nature of inductive reasoning, it is never certain. For a given set of interests, there will be an appropriate balance between the importance of the result (if true) and the risk of error (if false). But, the career motivation of scientists might encourage them to err on the side of publishing too quickly. This is especially serious in cases where the result might make a “big splash.” The replication crises observed in psychology, medicine, and related felds may be due, in part, to this confict between social goals and individual career motivations (Heesen 2017a). To some extent we must be willing to tolerate a certain level of error, that is the nature of inductive inference. However, if we want to encourage scientists to be more careful, we must counteract the motivation to publish potentially erroneous results. There are essentially two avenues that the scientifc community might pursue. First, we could make it structurally more difcult to publish potentially erroneous results. For example, a recent proposal suggested changing the p-value required for publication (Benjamin et al. 2017). Second, we might socially sanction scientists whose research is overturned in order to efectively deter the motive to rush publication in the frst place. The negative reactions to replication failures and retractions show that such social sanctions do exist (Lu et al. 2013). In this paper, I focus on this second potential solution. I argue that social sanctions by themselves may not be adequate. Through a simple decision-theoretic model, I illustrate how a scientist might actively game a punishment system in order to have a successful career despite only publishing results she knows will eventually be overturned. Because there is time between the publication of a result, and its eventual rebuttal, the scientist has a brief window of positive rewards. In many sciences, fame translates into resources for more research. If the scientist is able to translate her transient success into funding ever more exciting research, she can then produce new erroneous results that will swamp the negative criticism from the impending replication failures. I call this strategy “the scientifc Ponzi scheme.” In this paper, I provide a mathematical model of the situation and give the conditions under which it is possible. The example illustrates, I claim, a central difculty with the deterrence strategy for preventing the intentional publication of misleading results