Computational Functionalism
Discussion
Anil Seth:
"Brains Are Not Computers."
"The very idea of conscious AI rests on the assumption that consciousness is a matter of computation. More specifically, that implementing the right kind of computation, or information processing, is sufficient for consciousness to arise. This assumption, which philosophers call computational functionalism, is so deeply ingrained that it can be difficult to recognize it as an assumption at all. But that is what it is. And if it’s wrong, as I think it may be, then real artificial consciousness is fully off the table, at least for the kinds of AI we’re familiar with.
Challenging computational functionalism means diving into some deep waters about what computation means and what it means to say that a physical system, like a computer or a brain, computes at all. I’ll summarize four related arguments that undermine the idea that computation, at least of the sort implemented in standard digital computers, is sufficient for consciousness.
First, and most important, brains are not computers. The metaphor of the brain as a carbon-based computer has been hugely influential and has immediate appeal: mind as software, brain as hardware. It has also been extremely productive, leading to many insights into brain function and to the vast majority of today’s AI. To understand the power and influence of this metaphor, and to grasp its limitations, we need to revisit some pioneers of computer science and neurobiology.
Alan Turing towers above everyone else in this story. Back in the 1950s, he seeded the idea that machines might be intelligent, and more than a decade earlier, he
formulated a definition of computation that has remained fundamental to our technologies, and to most people’s understanding of what computers are, ever since.
Turing’s definition of computation is extremely powerful and highly (though, as we’ll see, not completely) general. It is based on the abstract concept of a Turing machine: a simple device that reads and writes symbols on an infinite tape according to a set of rules. Turing machines formalize the idea of an algorithm: a mapping, via a sequence of steps, from an input (a string of symbols) to an output (another such string); a mathematical recipe, if you like. Turing’s critical contribution was to define what became known as a universal Turing machine: another abstract device, but this time capable of simulating any specific Turing machine — any algorithm — by taking the description of the target machine as part of its input. This general-purpose capability is one reason why Turing computation is so powerful and so prevalent. The laptop computer I’m writing with, as well as the machines in the server farms running whatever latest AI model, are all physical, concrete examples of (or approximations to) universal Turing machines, bounded by physical limitations such as time and memory.
“The very idea of conscious AI rests on the assumption that consciousness is a matter of computation.” Another major advantage of this framework, from a practical engineering point of view, is the clean separation it licenses between abstract computation (software) and physical implementation (hardware). An algorithm (in the sense described above) should do the same thing, no matter what computer it is running on. Turing computation is, in principle, substrate independent: it does not depend on any particular material basis. In practice, it’s better described as substrate flexible, since you can’t make a viable computer out of any arbitrary material — cheese, for instance, isn’t up to the job. This substrate-flexibility makes Turing computation extremely useful in the real world, which is why computers exist in our phones rather than merely in our minds.
At around the same time that Turing was making his mark, the mathematician Walter Pitts and neurophysiologist Warren McCulloch showed, in a landmark paper, that networks of highly simplified abstract neurons can perform logical operations (Figure 2). Later work, by the logician Stephen Kleene among others, demonstrated that artificial neural networks like these, when provided with a tape-like memory (as in the Turing machine), were “Turing complete” — that they could, in principle, implement any Turing machine, any algorithm.
Put these ideas together, and we have a mathematical marriage of convenience and influence, and the kind of beauty that accompanies simplicity. On the one hand, we can ignore the messy neurobiological reality of real brains and treat them as simplified networks of abstract neurons, each of which just sums up its inputs and produces an output. On the other hand, when we do this, we get everything that Turing computation has to offer — which is a lot.
The fruits of this marriage are most evident in its children: the artificial neural networks powering today’s AI. These are direct descendants of McCulloch, Pitts and Kleene, and they also implement algorithms in the substrate-flexible Turing sense. It is hardly surprising that the seductive impressiveness of the current wave of AI reinforces the idea that brains are nothing more than carbon-based versions of neural network algorithms.
But here’s where the trouble starts. Inside a brain, there’s no sharp separation between “mindware” and “wetware” as there is between software and hardware in a computer. The more you delve into the intricacies of the biological brain, the more you realize how rich and dynamic it is, compared to the dead sand of silicon.
Brain activity patterns evolve across multiple scales of space and time, ranging from large-scale cortical territories down to the fine-grained details of neurotransmitters and neural circuits, all deeply interwoven with a molecular storm of metabolic activity. Even a single neuron is a spectacularly complicated biological machine, busy maintaining its own integrity and regenerating the conditions and material basis for its own continued existence. (This process is called autopoiesis, from the Greek for “self-production.” Autopoiesis is arguably a defining and distinctive characteristic of living systems.)
Unlike computers, even computers running neural network algorithms, brains are the kinds of things for which it is difficult, and likely impossible, to separate what they do from what they are.
Nor is there any good reason to expect such a clean separation. The sharp division between software and hardware in modern computers is imposed by human design, following Turing’s principles. Biological evolution operates under different constraints and with different goals. From the perspective of evolution, there’s no obvious selection pressure for the kind of full separation that would allow the perfect interoperability between different brains as we enjoy between different computers. In fact, the opposite is likely true: Maintaining a sharp software/hardware division is energetically expensive, as is all too apparent these days in the vast energy budgets of modern server farms.
“The more you delve into the intricacies of the biological brain, the more you realize how rich and dynamic it is, compared to the dead sand of silicon.”
This matters because the idea of the brain as a meat-based (universal) Turing machine rests precisely on this sharp separation of scales, on the substrate independence that motivated Turing’s definition in the first place. If you cannot separate what brains do from what they are, the mathematical marriage of convenience starts to fall apart, and there is less reason to think of biological wetware as there simply to implement algorithmic mindware. Evidence that the materiality of the brain matters for its function is evidence against the idea that digital computation is all that counts, which in turn is evidence against computational functionalism.
Another consequence of the deep multiscale integration of real brains — a property that philosophers sometimes call “generative entrenchment” — is that you cannot assume it is possible to replace a single biological neuron with a silicon equivalent, while leaving its function, its input-output behavior, perfectly preserved.
For example, the neuroscientists Chaitanya Chintaluri and Tim Vogels found that some neurons fire spikes of activity apparently to clear waste products created by metabolism. Coming up with a perfect silicon replacement for these neurons would require inventing a whole new silicon-based metabolism, too, which just isn’t the kind of thing silicon is suitable for. The only way to seamlessly replace a biological neuron is with another biological neuron — and ideally, the same one.
This reveals the weakness of the popular “neural replacement” thought experiment, most commonly associated with Chalmers, which invites us to imagine progressively replacing brain parts with silicon equivalents that function in exactly the same way as their biological counterparts. The supposed conclusion is that properties like cognition and consciousness must be substrate independent (or at least silicon-substrate-flexible). This thought experiment has become a prominent trope in discussions of artificial consciousness, usually invoked to support its possibility. Hinton recently appealed to it in just this way, in an interview where he claimed that conscious AI was already with us. But the argument fails at its first hurdle, given the impossibility of replacing any part of the brain with a perfect silicon equivalent.
There is one more consequence of a deeply scale-integrated brain that is worth mentioning. Digital computers and brains differ fundamentally in how they relate to time. In Turing-world, only sequence matters: A to B, 0 to 1. There could be a microsecond or a million years between any state transition, and it would still be the same algorithm, the same computation.
By contrast, for brains and for biological systems in general, time is physical, continuous and inescapable. Living systems must continuously resist the decay and disorder that lies along the trajectory to entropic sameness mandated by the inviolable second law of thermodynamics. This means that neurobiological activity is anchored in continuous time in ways that algorithms, by design, are not. (This is another reason why digital computation is so energetically expensive. Computation exists out of time, but computers do not. Making sure that 1s stay as 1s and 0s stay as 0s takes a lot of energy, because not even silicon can escape the tendrils of entropy.)
What’s more, many researchers — especially those in the phenomenological tradition — have long emphasized that conscious experience itself is richly dynamic and inherently temporal. It does not stutter from one state to another; it flows. Abstracting the brain into the arid sequence space of algorithms does justice neither to our biology nor to the phenomenology of the stream of consciousness.
Metaphors are, in the end, just metaphors, and — as the philosopher Alfred North Whitehead pointed out long ago — it’s always dangerous to confuse a metaphor with the thing itself. Looking at the brain through “Turing glasses” underestimates its biological richness and overestimates the substrate flexibility of what it does. When we see the brain for what it really is, the notion that all its multiscale biological activity is simply implementation infrastructure for some abstract algorithmic acrobatics seems rather naı̈ve. The brain is not a Turing machine made of meat."