Loading Session...

The Replication Crisis: How Should Science Change?

Session Information

Both scientists and the general public are highly concerned about the explosion of replication failures of published scientific studies. This replication crisis affects a variety of disciplines, ranging from studies in social psychology to clinical trials in medical research. While the crisis raises multiple philosophical questions, the philosophy of replication is still an under-explored topic, with few exceptions occurring mostly in scientific venues. Our symposium addresses key philosophical questions about scientific replicability and makes constructive proposals for solving the crisis: conceptual analysis of the nature of replications (i.e., "what is a replication?"), adopting new measures for quantifying statistical evidence, and changing the culture and incentive structures of the scientific enterprise. The symposium consists of five presentations (20 minutes) and it contrasts the perspectives of scientists active in the replication crisis with philosophers of statistics and scientific method.

03 Nov 2018 09:00 AM - 11:45 AM(America/Los_Angeles)
Venue : University (Fourth Floor Union Street Tower)
20181103T0900 20181103T1145 America/Los_Angeles The Replication Crisis: How Should Science Change?

Both scientists and the general public are highly concerned about the explosion of replication failures of published scientific studies. This replication crisis affects a variety of disciplines, ranging from studies in social psychology to clinical trials in medical research. While the crisis raises multiple philosophical questions, the philosophy of replication is still an under-explored topic, with few exceptions occurring mostly in scientific venues. Our symposium addresses key philosophical questions about scientific replicability and makes constructive proposals for solving the crisis: conceptual analysis of the nature of replications (i.e., "what is a replication?"), adopting new measures for quantifying statistical evidence, and changing the culture and incentive structures of the scientific enterprise. The symposium consists of five presentations (20 minutes) and it contrasts the perspectives of scientists active in the replication crisis with philosophers of statistics and scientific method.

University (Fourth Floor Union Street Tower) PSA2018: The 26th Biennial Meeting of the Philosophy of Science Association office@philsci.org

Presentations

What is a Replication?

Philosophy of Science 09:00 AM - 09:30 AM (America/Los_Angeles) 2018/11/03 16:00:00 UTC - 2018/11/03 16:30:00 UTC
Edouard Machery (University of Pittsburgh)
There is very little discussion of what a replication is and of its function. In this talk, I develop such a general account ("the re-sampling account of replication"): A replication is an experiment that resamples experimental components. On this basis , I end up rejecting the very distinction between direct and conceptual replication, as it is usually drawn and I conclude that the debate between the relative value of direct and conceptual replications is meaningless.
Presenters
EM
Edouard Machery
University Of Pittsburgh

Improving Scientific Rigor

Philosophy of Science 09:30 AM - 10:00 AM (America/Los_Angeles) 2018/11/03 16:30:00 UTC - 2018/11/03 17:00:00 UTC
Ferric Fang (University of Washington)
Research reproducibility is increasingly questioned. Explosive growth of the literature and hypercompetition among scientists are not conducive to good scientific practices. Proposals to make research more reproducible emphasize scientific rigor with little guidance as to what that means and how it can be achieved. Casadevall and Fang propose a framework for scientific rigor: (1) redundancy in experimental design; (2) sound statistical anaysis; (3) recognition of error; (4) avoidance of logical traps; and (5) intellectual honesty (MBio, 2016). Although this can guide research education, such efforts will be ineffective unless accompanied by a cultural change that rewards rigorous work over impact.
Presenters
FF
Ferric Fang
University Of Washington

The Division of Replication Labor

Philosophy of Science 10:15 AM - 10:45 AM (America/Los_Angeles) 2018/11/03 17:15:00 UTC - 2018/11/03 17:45:00 UTC
Felipe Romero (University of Groningen)
The social structures that govern scientists' work undermine replication and therefore self-correction. How can we intervene in such structures to make science more replicable? To address this question, I introduce the notion of "self-corrective labor schemes" —ways of organizing replication efforts in the scientific community. I propose a novel scheme, which I call the professional scheme. This scheme recommends creating a distinct reward system for scientists whose primary function is to do confirmation/replication labor. I show the advantages of the professional scheme over extant scheme proposals and how different parties involved in the research process should support it.
Presenters
FR
Felipe Romero
University Of Groningen

Degrees of Corroboration as a Cure to NHST

Philosophy of Science 10:45 AM - 11:15 AM (America/Los_Angeles) 2018/11/03 17:45:00 UTC - 2018/11/03 18:15:00 UTC
Jan Sprenger (University of Turin)
Statistical significance tests cannot quantify evidence in favor of the null or default hypothesis. Hence, many experimental findings which do not speak "significantly" against the null do not get published and end up in the proverbial "file drawer", contributing to bias and lack of replicability. My contribution addresses this problem by explicating the concept of degree of corroboration (of the null hypothesis), combining Popperian intuitions with modern Bayesian---and frequentist---statistics. Degrees of corroboration cure most deficits of significance tests: they allow for a more nuanced and less misleading assessment of the null hypothesis than p-values or confidence intervals.
Presenters
JS
Jan Sprenger
University Of Turin

Beyond Replication

Philosophy of Science 11:15 AM - 11:45 AM (America/Los_Angeles) 2018/11/03 18:15:00 UTC - 2018/11/03 18:45:00 UTC
Simine Vazire (University of California, Davis)
In this talk I will illustrate how scientists' failure to adhere to scientific norms has contributed to the replicability crisis. I will discuss how we can increase our adherence to these norms, from changing individual researchers' practices to reforming peer review and scientific institutions. The goal of these reforms is to increase the transparency of scientific research and level the playing field so that all research is held to the same standards of transparency. This transparency would enable more effective criticism, and therefore more skepticism of unsubstantiated claims. This, in turn, would lead to a more credible scientific literature.
Presenters
SV
Simine Vazire
UC Davis
1049 visits

Session Participants

Online
Session speakers, moderators & attendees
University of Pittsburgh
University of Groningen
University of Washington
University of Turin
KU Leuven
No attendee has checked-in to this session!
65 attendees saved this session

Session Chat

Live Chat
Chat with participants attending this session

Questions & Answers

Answered
Submit questions for the presenters

Session Polls

Active
Participate in live polls

Need Help?

Technical Issues?

If you're experiencing playback problems, try adjusting the quality or refreshing the page.

Questions for Speakers?

Use the Q&A tab to submit questions that may be addressed in follow-up sessions.