Mike Dacey
Experiments in comparative psychology typically aim to test a default model against an alternative. Morgan’s Canon dictates that researchers prefer models that posit the simplest processes. This is often interpreted by analogy to null hypothesis statistical testing (NHST): the simpler model should be the default (Andrews & Huss 2014). Morgan’s Canon has faced considerable criticism lately, and the two proposed replacements set up the central tension of this paper. One replacement, contextual null choice, accepts the general default model framing while choosing nulls/defaults case by case (Mikhalevich 2015, Mikhalevich, Powell, & Logan 2017). The other, evidentialism, rejects defaults altogether in favor of a more holistic inference to the best explanation (Sober 2005, Fitzpatrick 2008).
I argue for a version of evidentialism over any view that retails the default model framing (even if one wishes to retain Morgan’s Canon in a weaker form). I do so by first undermining the analogy that supports the default model framing, then demonstrating that it has problematic effects. The analogy between default models and NHST fails to respect the difference between statistical hypotheses and substantive hypotheses. Statistical hypotheses specify a distribution of a certain feature (the thing to be measured); substantive hypotheses are models that motivate the statistical hypotheses and, potentially, explain them. The inferential gap between statistical and substantive hypotheses looms large in comparative psychology, because in comparative work it’s often the case that any model can be consistent with many possible specific experimental outcomes. In such cases, the failure of any statistical hypothesis does not entail the failure of any substantive hypothesis. The analogy that supports the default model framing does not hold: statistical nulls can be (and should be) chosen without treating any model as the default.
Additionally, the default model framing has problematic effects, distorting the weighting of evidence, and systematically biasing experimental practices. One option mentioned above, contextual null choice involves choosing nulls based on the available evidence. While this is a step in the right direction, it means that a model will gain the same privileged status of “null” whether it wins by an inch or a mile. This distorts the weighting of evidence. Chosing default models biases practice by supporting the ‘associative/cognitive’ distinction that has become problematic in the field (Buckner 2011, 2017, Dacey 2016, 2017).
No model should be treated as a ‘default.’ Understanding how any particular experimental finding impacts the credibility we should lend to a particular model requires a more inclusive inference to the best explanation, as described by evidentialism.
