Welcome to Insight Axis, where I make connections between practical philosophy, technology, books, science, and more. I’m Zan - follow me on Substack.
I assume most of my readers have heard of Occam's razor. For those who haven't, it's a philosophical rule of thumb attributed to 14th Century theologian, William of Ockham. In his Summa Totius Logicae, he wrote:
"It is futile to do with more things that which can be done with fewer."
Since then, Ockham's words have morphed into the more recognisable Occam's razor:
"The simplest explanation is usually the best one."
I want to explain to you why I think Occam's razor is a dangerous double-edged blade. First, I'll show you why you can only use this razor in specific situations. Then I'll show you how it doesn't necessarily favour simplicity - it could just be deferring complexity to other objects.
Occam's razor is a precision tool, not a panacea
Since Occam's razor is about explanations, I want to take some time to define what a good explanation is. I like to think of an explanation as a container for knowledge, and my thoughts on this are heavily influenced by David Deutsch. In The Beginning of Infinity, Deutsch suggests that a good explanation is:
Deep - it aligns with more fundamental explanations of how the world works.
Broad - it can explain a range of phenomena of the same class.
Hard to vary - changing a small part of the explanation would change its predictions.
Fundamentally, if an explanation is deep, broad and hard to vary, it's a good explanation that's likely to stand the tests of science.
But good explanations and simple explanations are not the same thing.
Blindly applying Occam's razor to competing theories puts you at risk of choosing a wrong explanation for the sake of its simplicity. An explanation should never be chosen just because it's simpler than the alternative. It should be chosen because it's better than the alternative.
Some things are just hard to explain, and Occam's razor might point us away from the more complicated, but better explanations of the world. For example, Einstein's theory of relativity is more complex and powerful than Newton's simpler theory of gravity. In the pursuit of accurate knowledge, choosing Newton's theory over Einstein's for the sake of simplicity would be wrong.
So when should you use Occam's razor?
You can apply Occam's razor when you have two competing explanations that are comparable in depth, breadth and variability.
For example, if you can't run an experiment that helps you prove one theory and disprove the other, you might want to apply the razor and pick the simpler explanation. Alternatively, if one competing theory has more unexplained assumptions, you might use Occam's razor to eliminate it.
In fact, a better way to frame Occam's razor would be as Deutsch describes it in The Fabric of Reality:
"Do not complicate explanations beyond necessity"
With this definition, you'd be right to say that good explanations are never unnecessarily complex. This is not the same as the standard formulation of the razor, which conflates the simplest explanation with being the best one.
Occam's razor transfers complexity, but doesn’t always eliminate it
Continuing with my inspiration from Deutsch, let's expand on a thought experiment he walks through in the Fabric of Reality.
Let's assume we're living before 1900, and humanity is not yet airborne, let alone space-borne.
Over last few hundred years, we’d have derived equations to describe how the planets move across the night sky. These theories and equations imply that all these wonderful planets actually exist. (For simplicity of language, I'll ignore the sun and other stars - but the end-result would be the same if they were included through the entire argument.)
Now let's construct a competing theory, which says "The planets are actually projections on the screen of a massive planetarium all around Earth."
To refute this competing theory, you might ask "well, if there’s only a planetarium, and there are no real planets, then why do our equations explain their elliptical orbits around the sun?"
I could answer "The computer program of the planetarium makes the pixellated planets move on the screen as if the planets have elliptical orbits around the sun". According to Deutsch, a similar argument was actually used by the Church against Galileo when he refuted their geocentric theory of the planets. Essentially, the Church did not mind that Galileo or other scientists had equations that predicted the movement of planets as if they orbited the sun; they just didn’t accept that their equations could be taken as seriously explaining the true nature of planets’ orbits.
Anyway, in this example we have two competing explanations, and 2 sets of associated objects.
Explanation 1:
There are equations that explain the elliptical orbits of planets around the sun.
Objects associated with Explanation 1:
9 (or 8 if you hate Pluto) complex and absurd planets, ranging from hot, massive balls of gas, all the way to tiny balls of ice. All these planets are made of several elements in unique proportion, and some even have fancy rings and multiple moons. We also have massive raging ball of fire in the middle of our solar system called the sun. By extension, this theory also accepts the absurdity of stars, galaxies, nebulae, black holes and other space phenomena as being real.
Explanation 2:
There is a massive planetarium-like dome that surrounds Earth
The planets wiggle around on the planetarium's screen as if they had elliptical orbits around the sun.
Objects associated with Explanation 2:
A massive planetarium like dome, and a (big but not unimaginable) computer which programs the pixels on the screen to look like planets, and a brighter set of pixels for the sun. By extension, this theory also means that stars, galaxies, nebulae, black holes and other space phenomena are just pixels on a single, curved screen.
Deutsch correctly says that by using his rendition of Occam's razor, you should accept Theory 1 over Theory 2. This is because in Theory 1, you only have to explain the orbits of the planets. However in Theory 2, you'd have to explain the planetarium screen, its associated computer, and then also the wiggling of the planets on the screen by using equations that you’d derive for Theory 1 anyways. You can't explain Theory 2 without the knowledge contained in Theory 1. This makes Theory 2 unnecessarily complex compared to Theory 1, with less explanatory power. Therefore it's correctly refuted using the razor.
What Deutsch doesn't really labour is that by accepting the simpler Theory 1, you accept far more complex objects than if you accept Theory 2. All Occam's razor has done in this case is favour a simpler (and better) explanation, at the expense of more complex objects.
This is the correct application of Occam's razor. Many times, people will wrongly use the razor to accept simpler objects by using far-reaching explanations. Instead, the opposite should be done.
A real-world example: macroeconomics
Armed with this idea of Occam's razor transferring complexity from explanations to objects, we can see the field of macroeconomics with a new perspective.
Macroeconomists are known for their almost comedic ability to always disagree with each other, and this isn't new. Each macroeconomist favours their own complex, not-so-good explanation of what's going on in the world. Their complex explanations are accompanied by simple objects: rationally acting humans, corporations and governments. That's a complete inversion of Occam's razor. Instead, I'm in favour of a newer take on macroeconomics that Andrew Lo outlines in his book, Adaptive Markets.
Lo adopts a relatively simple, Darwinian explanation of market dynamics. He takes inspiration from evolutionary game theory to underpin the phenomena we see on the global economic stage. The consequence? He accepts wild varieties of current and future complex and irrational entities which phase in and out of the economy based on their survival strategy and the economic climate, in ways that are not well described by standard macroeconomics. No rationality is assumed on the part of market agents, and no simple objects are taken for granted.
Final thoughts
Beware of Occam's razor, for it can be incredibly powerful, and maybe not in ways you think.
Don't fear a good but simple explanation just because it might require you to accept the existence of complex objects. It might get you closer to the truth than the opposite trade.
Recommended reading from Substack:
𐄷
explores an analogy between mass and consciousness.💬
brings together philosophy and linguistics in the dawn of humanity.😢
shares a touching family story.🧸
blows my mind by combining the concepts of countable and uncountable sets with the english language.🔍
examines the unspoken myth of successful, self-made individualism in today’s world.
Recommended reading beyond Substack:
Adaptive Markets: Financial Evolution at the Speed of Thought - Andrew Lo
If you enjoyed this essay, please like, subscribe, and share it with others.
I learn from my readers, so leave a comment with your thoughts, and I’ll find you there!
"Deutsch suggests that a good explanation is:
Deep - it aligns with more fundamental explanations of how the world works.
Broad - it can explain a range of phenomena of the same class.
Hard to vary - changing a small part of the explanation would change its predictions."
1. There's little mention of prediction or falsifiability. That's a rather extraordinary omission, compared to everyone else's philosophy of science. (The two are of course closely related, because failure to predict is the gold standard of falsification)
2. Almost everyone wants Depth .. ultimately correspondence to.reality itself. But it can't be measured directly .. that's possibly the major problem in scientific realism. And the reason philosophy of science tends to emphasise simplicity, in particular, is as a proxy for Depth/Correspondence. (Predictiveness is a virtue in its own right).
3. The substitution of Hard to Vary for Simple is another novelty of Deutsch's approach. However, it is not an improvement.
Scientific epistemology has a distinction between realism and instrumentalism. According to realism, a theory tells you what kind of entities do and do not exist. According to instrumentalism, a theory is restricted to predicting observations. If a theory is empirically adequate, if it makes only correct predictions within its domain, that's good enough for instrumentalists. But the realist is faced with the problem that multiple theories can make good predictions, yet imply different ontologies, and one ontology can be ultimately correct, so some criterion beyond empirical adequacy is needed.
Realism is "deep", instrumentalism is "shallow". Deutsch is on the realist side, because of his concern with Depth, lack of interest in prediction, and disdain for induction.
Now, in most forms of scientific epistemology, simplicity is key to obtaining realism over and above empirical adequacy. You can have a set of theories which all make good predictions, and the same predictions, but have different ontological interpretations, so they cannot all be correct representations of the reality.
There are multiple simplicity criteria, but not multiple truths. So you need the right simplicity criterion.Minimally, it needs to be able to single out a "best" theory from the candidates.
A theory is easy to vary if part of it isn't doing anything: that's a part that can be removed, or substituted. Removing redundant parts makes theories simpler, so picking hard to vary theories is picking simpler theories, all other things being equal. So the HTV criterion is a simplicity criterion. Bur the HTV criterion, while it removes N theories that have redundant parts, leaves M theories that don't -- and M is likely to be greater than one. So it doesn't hone in right on the truth -- admittedly a high bar! -- and is also less selective than most rivals.
Another problem is that Kludges, patches and epicycles, are not redundant!
A theory can be baroquely complex, yet consist entirely of elements that are doing useful work -- each epicycle in a Ptolemaic theory adds some accuracy.
In an era where nothing is what it seems, the use of Occam's razor to determine what the supposed truth and reality is, brings about a dangerous misconception, and those who use Occams razor in this manner will most definitely end up cutting themselves repeatedly.