Ravit Dotan (University of California, Berkeley)
I argue that non-epistemic values are essential to theory choice, using a theorem from machine learning theory called the No Free Lunch theorem (NFL).
Much of the current discussion about the influence of non-epistemic values on empirical reasoning is concerned with illustrating how it happens in practice. Often, the examples used to illustrate the claims are drawn from politically loaded or practical areas of science, such as social science, biology, and environmental studies. This leaves advocates of the claim that non-epistemic values are essential to assessments of hypotheses vulnerable to two objections. First, if non-epistemic factors happen to influence science only in specific cases, perhaps this only shows that scientists are sometimes imperfect; it doesn’t seem to show that non-epistemic values are essential to science itself. Second, if the specific cases involve sciences with obvious practical or political implications such as social science or environmental studies, then one might object that non-epistemic values are only significant in practical or politically loaded areas and are irrelevant in more theoretical areas.
To the extent that machine learning is an attempt to formalize inductive reasoning, results from machine learning are general. They apply to all areas of science, and, beyond that, to all areas of inductive reasoning. The NFL is an impossibility theorem that applies to all learning algorithms. I argue that it supports the view that all principled ways to conduct theory choice involve non-epistemic values. If my argument holds, then it helps to defend the view that non-epistemic values are essential to inductive reasoning from the objections mentioned in the previous paragraph. That is, my argument is meant to show that the influence of non-epistemic values on assessment of hypotheses is: (a) not (solely) due to psychological inclinations of human reasoners; and (b) not special to practical or politically loaded areas of research, but rather is a general and essential characteristic for all empirical disciplines and all areas of inductive reasoning.
In broad strokes, my main argument is as follow. I understand epistemic values to be heuristics for choice that are presumed to make it more likely that the chosen theory is true. Learning algorithms are ways to induce general hypotheses from a given dataset. As such, they are procedures for theory choice – they are ways to choose the one hypothesis that best fits the data. The NFL determines that all learning algorithms, i.e. all ways to conduct theory choice, have the same average performance when averaging over all possible datasets. This entails that, if we don’t restrict the possible datasets, all ways of choosing between hypotheses have the same expected performance. That is, they are equally likely to produce true hypotheses. This includes all ways to choose between hypotheses, and in particular traditional epistemic heuristics like simplicity and even random guessing. When averaging over all possible data sets, all choice procedures are equally non-epistemic. Moreover, I argue that theory choice essentially involves non-epistemic values even when we restrict the range of admissible datasets, as we do in real life.
