On life and computability

Few would disagree that we live in an age where making fruitful progress in neuroscience requires a deeply interdisciplinary approach. This involves a successful translation of the core ideas from one discipline to another. One of the most beautiful and unifying ideas that was given a clear articulation by those working in the physical sciences in the 20th century is the idea of computation, alongside the closely related idea of information. It is, however, an idea that is to this day still not always communicated effectively, and that hasn’t yet sunk into our society as an inherited metaphysical preconception to the degree that other equally important ideas, such as the Newtonian recognition of the unifying power of mathematical descriptions, have.

I frequently encounter harsh resistance to the application of a computational worldview within the biological sciences and in philosophy of mind, which I believe is driven by three main causes: a misunderstanding of what computation actually means, misuse of the idea by many of those who do subscribe to it, and internal psychological resistance to what this idea tells us about ourselves.

It is my intention here to address these three points clearly, and to show that this new metaphysics is not only of great utility when applied correctly within the sciences, but also deeply beautiful when embraced fully, in both its objective and personal scope.

First of all I want to present a formulation of computation as simulation. This is a slightly different route to the classical one famously developed by Alan Turing in his 1936 paper ‘On Computable Numbers, with an Application to the Entscheidungsproblem’, and I choose this route with the express intention of keeping our minds flexible as to what types of substrates we will end up considering as performing computation.

We begin by taking an abstract dynamical-systems based view of physics, in which some system is described by a mathematical object, such as a vector in a well-defined vector space, and whose evolution in discrete or continuous time is known via, say, a first-order differential equation. We put our system into some initial state and see what happens as it evolves. In many such combinations of physical laws and initial state will have extremely simple behaviour: a ball will roll down a hill then reach the bottom, where it will remain still forever, if nothing changes. Such systems can remain simple even with large perturbations to their initial state. Others will be in practical terms impossible to predict: chaotic systems like the weather are extremely sensitive to their initial conditions, such that even minor changes in the initial state will lead to drastic changes as the system evolves.

Some systems, however, will have interesting dynamics in such a way that it is impossible to say what state they will be in at some future time without first seeing what state they will be in at all successive intermediate times. To figure out the trajectory of such a system, therefore, we need to do some work. And this is when we utilise one of the greatest tricks that nature has devised. It turns out that for any dynamical system, our laws of physics are rich enough that we can always find another system, with a one-to-one mapping to this original system, such that its dynamics are identical. If we can make this other system substantially more practical to set in motion, then we can just run the simpler system and use our mapping to see how the original system will unfold. This picture encompasses both the traditional view of simulation, but also of symbolic, pen and paper computation, where the mapping goes from the symbols to the physical system, and the dynamical laws are the laws by which the symbols are manipulated by the mathematician.

The fact that this is always possible is a very deep fact indeed about nature, but we can push things further. It may sound like, at this point, that for any given problem, we need to set up a new toy version of this problem, which doesn’t sound easy. However, it actually turns out that there exist systems in the world with just the right amount of sensitivity to their initial conditions that we can make relatively small perturbations to their initial state, and we end up simulating wildly different physical systems - in fact, any other possible system in our physical universe. This is known as the universality of computation, or the Church-Turing thesis. This is an insanely practical fact - with very modest changes in our simulator’s initial state, such as reconfiguration of the bits of RAM in a von Neumann computer¹, we can simulate anything, from an electron in a magnetic field, to an ideal gas of millions of particles, to planetary orbits, and up to the expansion of the universe under the dynamics of general relativity.

There is just a minor caveat here that, understandably, the size of the state of the simulator has to grow with the difficulty of the problem, and we can devise problems of arbitrary difficulty, so when thinking about the limits of computation theoretically we typically allow the size of the state to grow as needed, without bound.

To recap: our physical world consists of a state, and a dynamical evolution rule. We have argued that we can partition this state, such that a mapping from one partition to another holds under the partitions’ respective dynamical evolution rules, and that we can construct special partitions that can be mapped to any other partition with just a minor perturbation of its initial state. I will shortly argue that this is such an insanely useful fact about the world that it has been exploited countless times not just by us, but also by natural selection. But first, I want to give a final flavour of why this fact is so remarkable.

When we set up two such partitions, they can, at first glance, have drastically different dynamical evolution laws. There seems to be nothing in common between the circuit logic occurring in a von Neumann computer and the laws of general relativity, for example. And yet, our mapping does exist. This raises the question: does the system we are simulating even have to be a part of our physical universe? The answer is a resounding no! And this bizarre fact catapults us into a staggeringly large space of possible sets of physics - the possible physical worlds that machines in our universe can map to.

Another natural question to ask is what universes can map back onto ours? The answer is once again, a shockingly high number. Thinking down these lines led figures such as John von Neumann, John Conway, and Stephen Wolfram to work on cellular automata, which can be thought of as being the simplest possible physical universes with non-trivial dynamics. The simplest cellular automata are 1D universes, discrete in time and space, with binary states and local evolution laws. Now, we can obviously simulate them. The question is, can they simulate us? Out of all 256 such universes, some of them can, such as the famous Rule 110 that was proved to be capable of universal computation in 2004 by Matthew Cook. The fact that such a minimal universe is capable of completely simulating our entire world is one of the most absurd facts about nature I have ever encountered. Nature appears to hit universality extraordinarily quickly.

What this all points to, is that there exists a staggeringly large space of different sets of physics that are in a profound sense equivalent to each other. It implies that it is just as useful to study the abstract patterns of computation and information as it is to study the underlying substrate that implements them.

As a side note, its worth mentioning that not all physical universes fall into this equivalence class. The cellular automata that just sets all states to zero and remains there forever does not have the richness necessary to simulate our world. Similarly, we can conceive of worlds that our world can’t simulate. This was one of Turing’s great triumphs in his seminal 1936 paper. The halting problem is the canonical example of an uncomputable but well defined operation, and we can conceive of worlds that include ‘oracles’ that solve the halting problem, so it therefore is possible to describe physical universes whose dynamics we cannot simulate. Furthermore, we can conceive of universes that we could in principle simulate, but that it would take an unacceptable amount of time and/or space to do so: for example, a universe whose dynamics depend on solving NP-hard problems. Under this very physicalist interpretation, then, the field of computational complexity theory corresponds to grouping all possible laws of physics into equivalence classes under which these mappings can be constructed either in principle, or under reasonable constraints in time and space. It is therefore, in a sense, more fundamental a field than fundamental physics.

We now return to the questions of how this is relevant to biologists and neuroscientists, to cases where I feel this perspective can be applied well, and how to avoid the pitfalls of an overly eager application of a computational worldview.

I believe the importance of a computational perspective to the life sciences is twofold: first, nature will have been wise to exploit the simulatibility of physical systems at all levels of its hierarchy. Second, science is always a careful balancing act between generalisation and specification, and a computational perspective, by virtue of being the most generalising possible perspective in the sciences, can provide us with a toolkit in this direction that is challenging to come by otherwise.

Beginning with the first point: we recall that, as the universality of certain systems within Rule 110 demonstrates, computers are shockingly easy to come by. However, many systems within physics are not computers: the orbits of planets in our solar system, for example, probably doesn’t have the richness of dynamics necessary to simulate other interesting systems. This, however, could not be further from the truth for biological systems, where complexity abounds. The dynamics of gene regulatory networks, of tissue morphogenesis, of complex metabolic pathways, and of biological neural networks, have ample space for computation to arise, and tower above Rule 110 in this regard.

The most obviously provable example of a biological universal computer is, well, you. If you so desired, you could absolutely read a manual detailing the operation of a Turing machine and replicate its dynamics with pen and paper. Additionally, since you are composed of matter, demonstrating that you belong to a class higher than this - that is, that you do something fundamentally non-computational and cant be simulated - would require finding something non-computational in the fundamental laws of physics, then demonstrating how this pertains to the operation of your brain. This would involve a shakeup in physics orders of magnitude more rattling than the discovery of quantum mechanics. Proposals along these lines, from Cartesian interactionism where non-computational causation is mediated through the pineal gland, to modern proposals such as Penrose’s Orch-OR theory in which this occurs through non-computational wavefunction collapse affecting processes in neural microtubules are, to put it lightly, a bit far-fetched².

With that said, universality is probably not the correct yardstick for many biological processes. Biological systems have to solve a specific set of problems efficiently, robustly, and flexibly, and they will be very tailored to their problem domain, with universality arising in certain instances as only a byproduct of this. There are many incredibly useful frameworks in which to formalise this idea, including the historical field of cybernetics, control theory, reinforcement learning, and active inference. My current feeling is that our fields are still nascent enough that a unifying framework is perhaps premature and that it would be wise to get a grasp on as many such frameworks as possible.

That said, there’s an important feature that all of them share in common: they all involve modelling their environment as a central implicit or explicit computation. This is summarised eloquently in Conant and Ashby’s good regulator theorem, which states that ‘Every good regulator of a system must be a model of that system’. All biological systems interact with a complex environment, both inside and outside the organism in which they exist, and have preferred states over both themselves and their environment that they want to maximise. By the good regulator theorem, this involves modelling their environment, which means finding a mapping and internal state configuration from the system doing the regulating to the environment that maintains its evolution adequately. This is, by definition, what computation is.

Thinking of biological systems as performing computation is key because it unlocks a whole new way of thinking: algorithmic thinking. Indeed, it should go without saying that the most important, overarching idea in the life sciences, that of evolution by natural selection, is an algorithm, proposed by Darwin long before the substrates that implemented it were uncovered. Algorithmic thinking allows us to generalise and see connections we wouldn’t have otherwise, and can play a deeply complementary role in our thinking in addition to thinking in the world of substrate and implementation.

Examples abound so readily that giving one feels almost silly, but I will nonetheless. Bacteria, such as E. coli, have preferred states of high nutrients and low toxins. They can sense these nutrients and toxins with surface receptors, and can perform a few actions, including tumbling, and moving in a straight line, propelled by their flagella. Crucially, these systems operate together to implement a biased random walk algorithm: bacteria sample their environment over time, and if conditions are improving, they continue to more forwards, while if they worsen, they will perform a tumble and try again in a new direction. The net effect is that bacteria maximise their time in regions of high nutrients and successfully avoid toxins. Implicit in this system is a model of the environment that says that preferred regions of higher nutrients lie up their concentration gradients. Studying the receptor proteins and flagella biomechanics is not enough to normatively explain the behaviour of this system: the key lies in thinking algorithmically.

There are, however, pitfalls that can arise when one is overly eager in applying an algorithmic perspective within the life sciences. Of course, the implementation level of understanding is equally important in its own right. You often hear it argued that a computational perspective is too overly-generalising to be of the use, and we need to acknowledge the warm, wet, sub-optimal messiness of real biological systems to actually understand them. I think this argument is actually deeply misguided, but it arises from the misconception that by ‘computation’, we are referring to von Neumann computers, and that we are trying to draw parallels here where there are none. What I have tried to communicate in this piece is that a computational perspective is for more wide reaching than the traditional one rooted in modern digital computers. Indeed, I believe much of the most fascinating current work in biology and neuroscience looks directly at how algorithms can be performant at implementing their computations in the face of all the biological messiness we encounter: how biological systems can be incredibly flexible, intelligent, and robust in new problem spaces, and in the face of perturbations. So I see the problem here as lying not in the over-application of computational thinking, but of an insufficiently diverse understanding of what ‘computation’ actually means.

In the present day, we are essentially only exposed to von Neumann computers. If I write code and it has a typo, it will not compile and will fail to run. Our systems are inflexible but extremely well engineered and adept at the tasks we have designed them for. But they comprise just a tiny portion of the extremely large space of possible computing devices, almost none of which we are exposed to, or if we are, we typically don’t think of them as performing ‘computation’.

The tides are most definitely shifting with modern machine learning, as a new generation will grow up against a backdrop of far more flexible and creative computational systems. But even they run on the same substrate as classical computer programs, and don’t necessarily have the capity to help us expand our minds into a full appreciation of the vast space of computation that I’m envisioning. In truth, this full appreciation would take an exploration of all possible architectures or substrates under all possible computable laws of physics, which is of course absurd.

What we can do, as humans, is broaden our perspectives just a bit by studying non-conventional architectures. I’m thinking along the lines of neuromorphic computing, older analog computing designs, in-memory computing, reversible computing, reservoir computing, and much more. Anything that expands our horizon away from traditional stored-program, sequential execution-based von Neumann computing.

Beyond this, I think we need to be deeply humble in the face of how large this space is, how little we know, and how much nature still has to teach us. In particular, there is so much going on in biology that could be viewed best through an algorithmic lens; we just typically miss it because we don’t have well-established frameworks in place for us to see it. Nature has had a long time to find ingenious substrates and creative solutions to its problems, and we are all the more wiser for paying attention, with the generalisable framework we have explored ready at hand to clarify a given process as and when necessary. I wish I could say more on this here, but I dont have the expertise to do it justice, so I refer you to the entire body of work of the brilliant developmental biologist Michael Levin, who has as keen an eye as any for recognising diverse and non-trivial computational processes at play in unusual biological substrates.

To bring this all back to neuroscience, I’m pleased to say that, at least in the mathematically-minded end of the community, everything I’ve said here is well-established gospel, preached famously by David Marr in the late 70s and early 80s. Marr’s three levels of analysis consist of the computational, algorithmic, and implementation levels. These form the bedrock from which our discipline operates.

By contrast, the machine learning community has taken the most simplified possible single neuron model, the McCulloch-Pitts neuron, as descriptive enough of the implementation level, and have run with it, concerning themselves with hardware only from an efficiency and performance perspective. This is, of course, producing dazzling results, and I’m not here to criticise it. But my personal inclination as a natural scientist at heart is to remain open-minded to the levels within the biological hierarchy at which important implementation could be occurring, and to the level of detail needed to fully descsribe the brain. There may well still be plenty of room at the bottom, as Feynman graciously expressed, for interesting processes to occur, and nature may well have exploited these in ways that are still genuinely insightful and novel to us.

More than anything, neuroscience is about the art of keeping ourselves grounded firmly across the three levels, and of striking that delicate dance between generalisation and specification, and between theory and observation, just right.

In these concluding remarks I want to make things slightly more personal. The discovery of the universality of computation, and of the conclusion from this that you, too, are a computational process, can be destabilising and bewildering. It’s an idea that is frequently met with harsh resistance. I think that modern adherents of interactionist dualism and related ideas like Orch-OR give in to these uneasy feelings too readily. We have a deep, innate desire for our selves to occupy a special place in the universe - for our minds to somehow be distinct from matter, and for us to be greater than mere machines. We must treat these biases with the utmost caution. Every major scientific revolution has made our place in the universe smaller, from Copernicus through Newton to Darwin, and finally now to Turing. But each of these have also only made the universe a bigger, wilder, and ultimately more beautiful place. I have zero doubt in my mind that the revolution in our worldview taking place right now seeks to make entry into the final refuge in which our sense of separateness remains: our own minds. And we’d be wise to pay attention to this shift. As debates around the personhood of AI systems, and of the intrinsic value of non-human biological intelligences, start to abound with increasing fervour, I certainly don’t want to be the one persecuting Galileo for showing us our true place.

Throughout this confusing and potentially distressing paradigmatic shift, I take refuge in the Buddhist teaching of Anattā: the doctrine of not-self. The world as I see it is utterly teeming with a diverse array of computers, of agents, of life - not as separated selves disconnected from matter, but as a unified, interconnected whole, and we are in the miraculous position of being able to understand this and gain insight from this deep truth. To me, nothing else could be more awe-inspiring.