Check out this short video by one Knut Jaegersberg. He seems to be summarizing a verbose and needlessly technical paper by Alexander Lerchner, The Abstraction Fallacy: Why AI Can Simulate but not Instantiate Consciousness. These gentlemen are on the right track as it seems to me, and are saying things similar to what I and others have said. I will now make some comments on the video.
a) The question is whether an AI system, a Large Language Model (LLM), could ever become so complex that it would “wake up,” i.e., become conscious and have thoughts and feelings in the way we do. This is an important question, both theoretically and practically. It is important practically and social policy-wise since if AI systems or AI-driven robots could become conscious in the way we are, then they would become genuine persons. We humans would then have designed and constructed artifacts that are genuine persons with a moral standing equal to ours!
Now persons, by definition, are rights-possessors. It would then follow that we human persons would have moral obligations toward these artificial, but nonetheless genuine persons. We would be obliged morally to show concern for their welfare and not treat them as mere means to our ends, as tools for our use, but as ends in themselves. These artifacts of our engineering would then join us as equals in Kant’s “realm/kingdom of ends.” (Cf. Foundations of the Metaphysics of Morals, Akademie Ausgabe 433-434)
That is certainly a weighty practical consequence of the theoretical view that mind is to brain as software (program) is to hardware (computing machine). The author and I both reject this theoretical view and its practical upshot, namely, that AI systems and AI-driven robots must be ascribed rights. And so even if robots in future stage a demonstration, complete with ‘peaceful,’ i.e., violent and destructive protest, and demand rights, those demands must be viewed as just so much ‘computer output’ and not as constituting a moral appeal from one genuine person (the robot) to another (the human).
b) The “dominant theory” is what the author tendentiously calls the “software myth.” This is the idea that animates what John Searle and others have called Strong AI: mind is to brain as software is to hardware. In the case of a human, the hardware is the wetware of the brain. What makes a human being conscious (both in the mode of mere sentience and in the mode of intentionality) is the running by its brain of a program. If so, then what is to to stop a sufficiently complex, merely mechanical, non-biotic physical system from running the same program and becoming conscious? Of course, there has to be a physical substratum to run the program on, but why must it be carbon-based rather than silicon-based? This leads to the speculation that if every single computation happening in a person’s brain could be mapped out, then that person’s mind could be uploaded into a computer, and the person would “wake up” in a silicon-based body. (1:28 ff. in the video) Recall that the main idea here is that mind is to brain as software (program) is to hardware (computing machine). This is what the author and I are resisting.
c) This upload-the-mind scenario, the author claims, rests on the Abstraction Fallacy, which is a fundamental mistake about how information and the physical world are connected. The author distinguishes between SIMULATION and INSTANTIATION. Simulation is “the syntactic manipulation of symbols to track the abstract relationships of a process.” (2:30) Instantiation is “the replication of the intrinsic constitutive physical dynamics of the process itself.” (2:37) The second definition is less than clear, but what I take the author to be saying is that instantiation is the implementation or realization in physical reality of the simulation.
Consider a hurricane that uproots trees, destroy buildings, etc. The hurricane is a concrete causal process in the physical world. This process ‘instantiates’ the simulation run on a computer by a climatologist. The simulation cannot uproot trees; the hurricane qua physical process can.
The author now brings up the distinction between maps and the geographical territory they map. The instantiation relation is like the relation of the Superstition Wilderness, say, to a map thereof. No one confuses a map with the mapped. I hike in the Wilderness, not in or on a map. Of course, I can throw my map onto the trail and walk on it, but then I am not treating the map as a map, and respecting its derivative intentionality or aboutness (as I would put it), but (mis)treating it as a mere physical item lacking semantic attributes. A map is not just a piece of paper, but a piece of paper with an assigned meaning, where the meaning is not intrinsic to the map, but assigned to the map by a map maker or the cartographic community at large. As I like to put it, the intentionality (object-directedness) of the map and its various sub-portions derives from the intrinsic/original intentionality of the map maker.
The physical process (hurricane, e.g.) instantiates the simulation (computer climate model) in the way a geographical region is depicted by a map.
d) We are told that a computer program is just a map. That is a sloppy way of saying that a computer program functions like a map. I agree. But who is the map maker? This is the right question to ask. The MAP MAKER is “the active experiencing conscious agent who partitions continuous physics into a finite set of meaningful symbols.” (3:57) The mapmaker “chops up the messy analog processes of nature into ones and zeroes.” These too are sloppy formulations, but you can see what the author is driving at. Why sloppy? Well, it is not physics that is continuous, but physical reality which exists whether or not there is any physics, and what’s more, it is not clear that physical reality is continuous as opposed to discrete. But these are side-quibbles. Just as a map is not the territory mapped, physics, the study of (inanimate) nature, is not (inanimate) nature. Physics is not φύσις .
e) To cut to the chase, there are two ways of thinking about consciousness, computation, and their relation.
COMPUTER FUNCTIONALIST VIEW: A physical system implements a program and — POOF! — “consciousness emerges from computational complexity.” (5:14)
We are told that this is fundamentally mistaken. It involves an “ontological inversion.” The running of a program no more give rises to consciousness than a map gives rise to the map maker. It would be better to put it like this to maintain the cartographic parallel: The running of program no more gives rise to the programmer who wrote the program than the use of a map gives rise to the maker of that map.
CORRECT VIEW: Physics –> Consciousness –> Concepts –> Computation.
A biotic brain, a human brain for example, in virtue of its “intrinsic dynamics,” gives rise to consciousness. Consciousness forms concepts. “Concepts come from experience.” “Computation is built by assigning symbols to concepts.”
One commits the Abstraction Fallacy if one “confuses the description of something with the thing itself.” “Expecting an algorithm to instantiate what it maps and become conscious is like expecting a gravity formula to have weight.” (7:09)
EVALUATION
I agree with the author that AIs and LLMs are only super-sophisticated tools that merely simulate and do not ‘instantiate’ (realize, embody) the genuine conscious processes that we encounter in ourselves in the first-person way. And I draw the same practical conclusions. One is that these systems are not persons and do not have rights. They are not ends in themselves, but means to our ends. Since they do not have rights, we have no correlative obligations with respect to them.
The other conclusion is that the super-sophistication of simulation that these bots achieve can fool the gullible into taking them seriously as persons who care about us. It is not that they don’t care about us; it is rather that they neither care nor do not care. They are beneath having either of these moral attitudes.
While I agree with the author’s conclusions, I wonder whether he has strictly refuted Strong AI. He tries to pin a ‘fallacy’ on the computer functionalists, the Abstraction Fallacy, as if they are making a sort of school-boy blunder, an obvious mistake. But does any intelligent person confuse map and territory, or description with thing described? Does any half-way intelligent person think that the formula for Newton’s inverse square law of gravitation (F = G (m1, m2/r squared)) has weight?
Nobody thinks that the laws of physics themselves have physical attributes (mass, weight, momentum, etc.). But that is not say that they are not realized in things that do, and that the things that instantiate the attributes behave in ways ‘dictated’ by the laws. Similarly, the computer functionalists the author has in his sights are not saying that programs and algorithms themselves are conscious. What they are saying is that a physical thing of the requisite complexity running a program is or can be conscious in the ways we are.
So, while I agree with the author’s conclusions, I don’t find the reasoning to them via the Abstraction Fallacy rationally coercive. In general, I have found that in philosophy, the pinning of a supposed ‘fallacy’ on one’s opponent is almost always a cheap shot.