Occam's razor kills a few more fairies

While I was cramming for my machine learning test last night, I came across an interesting passage on induction and Occam's razor. Induction is the red-headed stepchild of deduction, deduction being reasoning logically from firm premises to new conclusions. For instance, premises like "Guys are scum" and "You're a guy" imply the conclusion that "You are scum" as long as you believe the premises. The philosophical quest for deductive thinkers is to discover unshakeable premises and go from there. One of the most famous such premises is "I think, therefore I am."

Out here in the real world, we have no such foundational premises, or we don't know how to find them (unless someone in the know tells us, which is what revelatory religion is all about). To revelation and deduction, we add induction, which is trying to find the common thread in a dizzying array of experiences. Induction never really ends, because as far as we have seen (ha ha), the universe is a continually surprising place, and snips that common thread as fast as we can spin it. What this also means is that information we learn by induction is never unshakeable; the process of induction is a bit more risky than safe, justifiable deduction (in fact, it is famously difficult to justify induction, after Hume). We might be learning probabilities, or we might be finding red herrings, or we might be stumbling across fundamental truths. But we never really know, because we never really think that we have seen everything.

It is impossible to number the many possible theories about what we have seen and experienced. Witness the wild array of sciences that continually proliferate surrounding the physical (and theoretical and mathematical and computational) universe. We will never come to the end of them. When two theories conflict, like the gravitational theory and the gravity fairy theory, which one should we believe?

One way to decide between competing theories, not a law exactly, but a sort of heuristic, is called Occam's razor, after William of Occam (also spelled Ockham, I think). It has many paraphrases. I believe the original one was in Latin and meant literally "Do not multiply entities without cause". You could also say "The simplest explanation is the best explanation" or "Favor the shortest theory".

The idea is that you can lay out two theories, and if one explains all the things the other theory does, but also requires the existence of gravity fairies while the latter doesn't, we should believe the latter theory. Another extended example is materialism vs. theism. The classic design argument says that the universe is so complex that it implies a creator; the materialist response is "who created God?" Once the theist says that God is uncreated and eternal to escape an infinite regress ("who created the thing that created God?" ad nauseam), the materialist asks "why can't the universe be uncreated and eternal?" and then says, "Your God doesn't add anything to our concept of the universe, and by Occam's razor, we shouldn't multiply entities without cause, so God doesn't exist. Snap!"

Once we get beyond trivial thought experiments like these, though, we encounter a lot of problems; what does "entity" mean in Occam's razor? How about "simple"? Isn't the shortest theory "For the Bible tells me so"? Worse, in Theory A, we have magic fairies which don't exist in Theory B, but on the other hand, in Theory B we have fields and forces that somehow influence all of space-time at arbitrary distances that don't exist in Theory A. In fact, you might not even be able to compare these theories so directly; it might instead be impossible to explain Theory A in terms of Theory B because the concepts just don't translate. In these cases, which theory should we pick?

Thomas Kuhn, a philosopher of science, essentially advocates a pragmatic approach; in some sense, science is what scientists do and what scientists say it is. A scientific theory works if it surprises you and enables you to do things you couldn't do before. A theory is interesting if it is beautiful or elegant, if it slides into place and clears up many mysteries in a straightforward and devilishly clever way.

But there are no obvious measuring sticks for these qualities, either. Eventually even Kuhn has to throw up his hands and say that the best theory is the one that all the scientists in a discipline eventually vote for. It allows some new possibilities but eliminates others. It still circumscribes that corner of the universe, just with a different shape than the old theory. It has the allegiance of the crowd, but so had the other theory. The crowd is committed to good theories, to clear description, to quantification and anomaly, to elegance, beauty, and truth, to verification by reproducible experiment. But even this doesn't really get to the heart of why one scientific theory is better than another one.

It's amazing that new technology appears with aplomb when the philosophical foundations of science itself seem so slippery to catch hold of, and ordinary scientists can study their stars or DNA or President's correspondences without worrying much about these concerns. But the explanation is obvious once you think about it:

Science fairies.