Occam's razor – simplicity and accuracy

Suppose we have two models for the same data/problem and both seem to explain the data equally as well. Which model should we choose? There is a guiding principle or heuristic known as Occam's razor that loosely states that if we have two or more equivalent explanations for the same phenomenon, we should choose the simpler one. There are many justifications for this heuristic; one of them is related to the falsifiability criterion introduced by Popper, another takes a pragmatic perspective since simpler models are easier to understand than more complex models, and another justification is based on Bayesian statistics. Without getting into the details of these justifications, we are going to accept this criterion as a useful rule of thumb for the moment, something that sounds reasonable.

Another factor we generally should take into account when comparing models is their accuracy, that is, how well the model fits the data. We have already...