Intelligence Is Not Fungible

On Creating New State Spaces

Return of the Sun by Odd Nerdrum (1986)

Amidst the AI revolution we are living through, business leaders and tech-CEOs have said the future of work entails humans focusing on so-called ‘human skills’ while leaving technical ‘non-human’ work to machines (e.g., science, engineering, mathematics, etc). But this dichotomy of ‘human vs non-human skills’ contains myriad conceptual missteps. Who is to say technical skills are not human? There is no Platonic universal schema that allows us to discern which skills fall into human vs non-human camps. So far, we have only seen scientists, engineers and other technicians who, in all appearances, are ‘effectively human.’ Even when an LRM or any computational tool are used for technical work, a human being is always interpreting (and reporting) the results. Or else, we would never know of them, nor would it make sense if not analysed through a public world, i.e., regardless of it being a human or machine, a proposition, sentence or an output is sensible only if it is grounded in the accessibility conditions of a community with preexisting social practices, entwined norms, our intersubjective symbolic world and even prelinguistic we-experiences. Such oversights stem from a misapprehension about intelligence. Two unfounded fallacies are to assume [1] intelligence is fungible and [2] intelligence is private.

Currency is fungible: if I give a friend $20 and they give me two $10 notes, I still have $20. There is no change in the amount of money I have, as the indistinguishable units of currency are interchangeable. But this is not the case for intelligence, i.e., there are no distinct units of intelligence to be transmitted. A teacher cannot ‘transfer’ intelligence to a student, nor can organisational intelligence be directly downloaded by a new member from existing members; instead, they have to partake in an epistemic community and become enculturated within a collective, relationally emergent, non-fungible intelligence specific to an agent-environment context. In that vein, the so-called ‘context window’ in LLMs is a misnomer. To assume context is information in a database to be cumulatively ingested into a computational system commits a mereological fallacy akin to a neuroscientist saying the left hemisphere does logical reasoning or the ‘brain is minimising free energy.’ It is a categorical error to attribute psychological functions of a human being to mechanisms in the brain, and so is attributing the phenomenon of context—that intelligent agents can attune themselves to—to mathematically quantifiable information or bits in a digital computer. In other words, the new member in an organisation is picking up on unsaid norms, social cues, rituals, etc., and is already experiencing a pre-given world the moment they become part of a community. This enculturation, however, is not a matter of the new ‘Agent A’ merging with a shared state space of the other agents that can be easily abstracted and represented on, let’s say, a factor graph, but intelligence has a peculiar property in which it can create whole new state spaces while accounting for the phylogeny and constraints of a given space it’s situated within. For instance, Einstein’s discovery did not simply lead us to update our prior beliefs through a process like Bayesian inference, but rather to entirely revise our shared concept of space, which is a fundamental reframing beyond inferring from posterior probability. At a more immediate horizon, such revisions are frequently seen in modern market economies as they respond to new technologies, e.g., the internet, cloud computing, and LLMs, which radically displace their dynamics. Intelligence, in that regard, is the ability to manoeuvre the ‘unknown unknowns’ of this world that cannot be found in a database or reduced to a recipe like an algorithm.

That being said, it is a mistake to conceive of intelligence as a private property of an agent. Doing so is repeating the mistake of physicists of the past who believed that ‘hot’ and ‘cold’ were separate quantities and that there existed fluid-like ‘temperature particles’ that flowed between matter, until James Prescott Joule discovered the relationship between heat and mechanical work, which laid the foundations of thermodynamics. Temperature, while physically existing, is also not a ‘physical thing’ but rather a property related to the kinetic energy of particles.

Temperature is analogous to intelligence: it is clearly an observable process that agents enact, yet lacks its own independent physicality and invariably exists within an embodied substrate (e.g., a human being) and within a social world (e.g., an organisation). Nor can this process be decoupled from its observable, meaning, a sequestered phenomenon at work that we can point to and say ‘that entity is intelligent’ or ‘that is an intelligent process.’ The moment we do so, we lose an integral part of the phenomena we are investigating, and eventually the notion itself becomes entirely meaningless, not just wrong but nonsensical. Therefore, in analysing intelligence, we can learn from the 20th-century philosophy of science’s demarcation problem, which recognised that there is no logical absolute rule demarcating science from non-science; scientific thought is not an autonomous form of reasoning but inseparable from the broader, preexisting, and evolving human (and conceivably non-human) thought. Likewise, intelligence is best analysed publicly by its social use, not as relativistic subjectivism but through an intersubjective objectivity that emerges in an already-given world within which intelligent processes are situated. As intelligence is a practice, not an abstract procedure computed in a ‘Leibnizian calculus ratiocinator’ or decidable in a quasi-Hilbertian program.

The position that intelligence is best analysed in terms of its social use is not, however, a functionalist position borrowing from the philosophy of mind, despite its practical, ostensibly functional approach. The simplest definition of functionalism is ‘the mind is only what it does.’ But the Intelligence-is-a-practice contention is not an ontological position, nor is it attempting to develop a metaphysic. Instead, it is purely an exercise of conceptual clarification concerned with the use and misuse of the concept of intelligence and where it can be applied meaningfully. If this view is thought of as functionalist, then all of the criticisms and arguments against functionalism by Block, Searle, Putnam, and the like rightly apply. Moreover, functionalism presupposes an idealised function, whether cognitive or machinic, to meet a Platonic criterion for a phenomenon being considered a mental state, or, in our case, an indication of intelligence. But this cannot apply to a position grounded on contingency. If a calculus emerges that allows us to adjudicate whether a process carrying out a function is intelligent, it does not have an intrinsic meaning contained within it that universally exists in any Platonic sense, but is a result of social use that conditions such a method of reason or algorithm. Functionalism, when applied to intelligence, then commits the aforementioned demarcation fallacy.

An AI researcher claiming an artificial neural network (ANN) is reasoning and therefore a system instrumented with ANNs is intelligent is not wrong insofar as their benchmarks are set up according to the informal rules of the computer science lab, which organically emerge from the everyday use of concepts such as ‘reasoning’ or ‘intelligence.’ But if, due to the system carrying out a reified contingent function X, they consequently conclude that the system is reasoning the way a human being would, as a psychologist, poet, or non-scientific layperson uses the term, they are committing a conceptual trespass that functionalism in the philosophy of mind has been vulnerable to. A trespass no different to a neuroscientist pointing at a neural pathway and claiming “This is where memory lies!” despite the notion of memory being polymorphous with vastly different meanings depending on the context and phylogeny, e.g., memory as understood in a cognitive neuroscience lab vs its use in Marcel Proust’s In Search of Lost Time. In other words, to speak of reasoning in general based on an extraneous function is nonsensical. In addition to a ‘semantic bleed,’ such a conceptual stretching also holds an implicit metaphysical belief: they commit the reductionist fallacy of a cognitivist who presupposes a homology between scientific models and an undisclosed true reality solely based on a functional output with no account for the community’s commonplace pre-scientific contributions that ground a respective scientific model and allow a scientist to speak of it meaningfully.