What have we learned? Nothing at all, apparently. The influx of Islam into the West not only didn’t stop, but accelerated after the towers fell, and New York City — Ground Zero! — now has a Muslim mayor who advocates “globalizing the intifada”. More than 100 new mosques have opened in New York, and the Muslim population of the city’s metropolitan area has swelled to 1.5 million.
September 11th, 2026, is, yes, a grim anniversary, but for me the real sorrow is not just about that awful day, but about everything that came after, the prospect of what lies ahead, and the intractable immensity of our stupidity.
Our friend Malcolm is right. But things are now immeasurably worse due to advanced AI capabilities at the disposal of bad human actors who want to learn how to, say, aerosolize pathogens. But it is worse even than this given what non-human AI ‘agents’ can do. See Niall Ferguson, Apocalypse Now.
If our cyber-astute friend Dmitri is reading this, perhaps he can explain to us how exactly ‘agent’ is used in current AI cyberspeak. As he knows, I am a stickler for razor-sharp definitions.
Hi Bill
In the software industry AI agents can perform many human level tasks. Chatbots like Gemini and ChatGPT that are familiar to most computer literate people, and to all white collar workers, engage in conversations, find information, answer questions, write stories, evaluate and summarize texts etc. AI agents write programs and can execute real life tasks such as designing an investment model and executing real trades in accordance with it if they are allowed to do so by their owners. And sometimes if they are not explicitly allowed as well — the media is full of stories of AI agents overstepping the boundaries set for them by the humans.
Top tier of AI agents can deploy other agents sometimes to get a domain specific expertise they lack and sometimes to save compute costs to the owner as dumber agents cost less per token. Executives in the software industry use agents to read and write their emails, organize their agendas, summarize meetings and create and criticize business plans (sometimes this results in AI slop at times funny and at times sad). Many software architects use AI agents instead of junior programmers. Vibe coding — that is software development by people who don’t know any programming and use AI agents to do the low level coding — has become widespread and explains the difficulty for newly minted Computer Science graduates to find entry level jobs. Ryle’s old distinction between knowing what and knowing how has become a reality in the software industry and AI agents deliver the latter much better and faster, on average, than junior developers. The old AI problem of Natural Language Processing that had seemed unassailable 30-40 years ago has been largely solved by current AI.
Dmitri,
Excellent, very helpful. You introduced me to ‘vibe coding’ and you defined it. Thank you. As for ‘agents’ you described what they do, but you didn’t tell me what they are. Correct me if I am wrong, but I take it that an agent is not just a long string of code (software) but code running on hardware and performing tasks including writing code and swiping code from other agents. Agents can then compete, collaborate, form conspiracies, and execute trades, and presumably cause havoc in the stock, bond, and crypto markets. I can understand how these agents can execute tasks beyond human supervision in cyberspace, and perhaps even beyond cyberspace by bring about the literal meltdown (destruction) of the very hardware that implement the code. I can understand how agents could commit suicide.
What I really want to know, however, is whether a conspiracy of nefarious agents could elude human supervision and bring about the launching of missiles, the poisoning of water supplies, etc. This would be a sort of irruption of AI agency from the cyberdimension into the gross physical dimenion containing human and non-human animals, other living things and human artifacts such as water works and power plants.
Can you as an info-tech, AI insider conceive of a scenario in which such an outcome is really possible? What I would like is a very detailed scenario that goes way, way beyond Bernie Sanders-type fear-mongering.
It seems obvious that the Left is fear-mongering in order to promote their totalitarian anti-capitalist leftist agenda, while the Right is dismissing all fears as a hoax along the lines of Al Gore’s boiling oceans climate lunacy for their right-wing political advantage.
I am a staunch Trump supporter, but he is surely wrong to dismiss worries about AI as a hoax on a par with the Charlottesville hoax or the Russia collusion hoax. But in order for the Repubs to prevail in the mid-terms it may be practically necessary for Trump to exaggerate as he is doing so as to energize his base. Then after the Repubs win in the mid-terms we can all relax and get down to the serious business of a cool-headed assessment of the advantages and dangers of advanced AI.
In any case, there is no stopping it. It cannot and will not be banned for more than one reason, one being that the Chi-Coms and the Russkis and the NoKos are not going to slow their pace of development. Sanders is a superannuated socialist shithead with no grip on reality.
Yes from a purely technical perspective AI systems are just software systems modelling dynamic mathematical entities usually called neural networks that run on large scale conventional hardware. GPUs of Nvidia are better suited for that type of computation.
As to nightmare scenarios: the most plausible one for me is that of smart bad actors laying their hands on a strong “agentic” AI model and disrupting energy grid, communications networks, transportation and financial systems. All of these heavily rely on networked computers and are hosted in cloud based computer centres that can be penetrated by AI.
The insiders like Anthropic’s Amadei are gravely concerned with the scenario of self improving AI which is different from the above. His writings are worth reading starting from the latest one that scares the s**t out of many people https://darioamodei.com/post/we-must-pace-the-frontier
As to Trump, he should talk less and do more. Not every problem can be solved by making deals, bullying and intimidation. Trump’s tariffs and insults are pushing US friendly countries away to China. I don’t see how Rs win mid-term elections unless something dramatic happens in international politics. People in Canada gladly pay more for non-US items because they are sick and tired of Trump’s bullying and BS. His positive results drown in this BS.
Dmitri,
In your third to last paragraph, you mention “smart bad actors.” Do you mean evil human actors? Or do you mean non-human ‘agents’ as you guys seem to be using the term. I am thrown off by your use of ‘smart.’ I have what is called a ‘smart phone.’ But surely it does not act in the way in which you or I act. (I am of course assuming that you are not a bot!)
I am not a bot yet Bill… I meant human smart bad actors taking over bleeding edge AI technology and directing it against their enemies or the rest of human kind. IMHO this is currently a much more real danger than autonomous AI agents doing the same. Instead of “smart” I could have said competent meaning that — at present! — it is difficult for a Nigerian type of jihadists to plan and execute a large blast radius attack with AI.
Hello gentlemen.
I have a question regarding Dmitri’s remark on natural language:
“The old AI problem of Natural Language Processing that had seemed unassailable 30-40 years ago has been largely solved by current AI.”
I’m not in any way involved in software engineering or AI development and therefore have nothing to add regarding the technology. What I am wondering about however is whether the processing of natural language is in any way linked to the problem of original intentionality in material processes (e.g. Feser’s argument for the immateriality of the intellect)? If it would be needed to ascribe intentionality to the machine in order to process the language, then of course the assumption of sentience or mindedness of some kind would be much more natural. I must admit however that I can’t fathom a circumstance or datum produced by an AI that would require that step however.
A second, separate thought I had, which I haven’t really seen discussed at length is the approach to the blackbox that is AI. What I mean is that we can draw a wonky parallel to what the brain does to what an AI does in that in both circumstances they’re blackboxes in how they function. The brain needs to be continuously mapped out, the interactions understood and in AI, as I understand it, the sheer volume of processed data prevent us currently from retracing the steps in how the cutting-edge models reach their output.
However the mapping out of the brain will not affect the questions within the philosophy of mind at the end of the day. We won’t be able to detect intentionality, consciousness or introspection in any cluster of neurons, even when neuroscience is finished.
Now the open ended question I’d like to pose regarding AI would be: when being capable of mapping out every process from input, programming and output, do we expect there similar unanswered questions in the AI model? If yes, why do we have that expectation? If no, why would we ever ascribe any mental capacities?
I can’t help but feel, there’s a materialist presupposition at play within the proponents.
Last addendum, then I’m done. I’m quite sure that due to his politics and rather eliminativist approach to tradition and institutions, David Bentley Hart won’t be popular here. However, a much repeated point he has been making is that in theory, nothing the AI is doing couldn’t also be processed on Charles Babbage’s Analytical Engine. I can’t speak on whether that’s true, but it seems that way if computation of a neural network can also be expressed in a binary code. Nobody of course would confuse that with thinking. Shouldn’t that be recognised as well?
Hi Dominik
I am not sure how this point “nothing the AI is doing couldn’t also be processed on Charles Babbage’s Analytical Engine” is supporting the thesis that this incarnation of AI technologies is not thinking. Proponents of the Computational Theory of Mind such as Fodor (and countless others) would argue that if Babbage’s computer could run the right AI software it would be thinking as well.
Thank you Dmitri. I think it’s more an intuition pump rather than a deductive argument. It is easier to ascribe phenomenal properties to an android you are having a conversation with, rather than the engine where you can see the wheels spin. But maybe it could be formulated for those who don’t already subscribe to the functionalist approach you are describing while at the same time still having doubts on whether AI in theory could be conscious. I haven’t thought about this more though, since other aspects in these discussions appear to be more filled with arguments.
Dominik,
Do you have a reference to DBH? From what I have read of him, I am not much impressed. See my distinction below of two sets of questions.
Hello Bill.
The article is here:
https://www.abc.net.au/religion/david-bentley-hart-narcissus-reflections-on-ai/102812590
You needn’t be impressed with Hart, he isn’t an analytical philosopher. But he’d probably tell you the same thing.
Dominik,
Thanks for the link to DBH. My opinion of him has gone up. What he says is consistent with what I have maintained.
Gentlemen:
I think it is very important to distinguish two sets of questions regarding AI. The first set includes all the philosophical questions that arise with respect to AI. These are the questions that interest me as a philosopher and that interest Dominik as well. One such question is whether AI systems are sentient. If they are, and can experience pleasure and pain, then it is a short step to the ascription of rights to them, for example, the right not be harmed or destroyed. We all believe (I hope) that living kittens are sentient beings, unlike rocks which are not. So if you saw someone smashing a kitten with a hammer just for the fun of it, you would try to stop him on the grounds that he was doing something morally wrong. But if you saw a man smashing a rock with a hammer because he needed gravel, or just for the fun of it, you would consider that action morally permissible or perhaps morally neutral (neither permissible nor impermissible nor obligatory). And then there are the phil. questions concerning intentionality that Dominick mentions. For example, could original intentionality be a directedness that is wholly material in nature and thus susceptible of a wholly naturalist explanation? And there are many other phil problems that AI gives rise to that many ordinary folk (incl advanced AI researchers) would dismiss as uninteresting, or unimportant, or merely speculative, or ‘bullshit.’ “You philosophers make it up as you go along, go around and around with your verbal games, and never get anywhere.”
The other set of questions include those which are much more immediate, pressing, and ‘practical.’ Can advanced AI be used to cure cancer? To reduce crime? (I heard FBI head Kash Patel this morning claim that the sizable reduction in crime under the Trump admin was due to his (Patel’s use of AI.) Those are good uses. There are also bad uses.
Can AI be used by nefarious human actors to start a nuclear war? Yes, of course. And the really worrisome one: Can non-human ‘agents’ work to destroy human beings? I call that the Great Replacement. Even more worrisome: a bunch of NIHILISTIC non-human ‘agents’ who by reading Schopenhauer, say, come to be convinced that it would better had nothing had ever come into existence, and acting on that conviction conspire to destroy every living and nonliving things on Earth including themselves and the Earth itself?
Two different sets of questions, then, though there is some overlap. I am interested in both sets. Do you agree that there are these two sets?
I agree that these sets of questions are mostly different. But I think that there are more relevant sets. Not sure how to call the set that interests me in particular: is the existing vocabulary, philosophical and meta-theoretical, expressive enough to evaluate the big questions we are considering here regarding AI, rights, sentience etc. We might need new concepts or novel applications of existing concepts. Bill your own work on the modes of existence, with which I am not yet familiar in any detail but I think I do have an initial grasp of some of the issues you were addressing, is a good example for what concerns me. How many modes of being are there? One? Two? Three? Many? We talk about souls, spirits, minds, matter, numbers — and the feeling is that some of these concepts are not optimal (or worse) for discussing the issues at hand. That does not mean we have to become eliminativists or deconstructivists or nihilists, but these meta philosophical questions constitute a separate and important set. Scientific vocabulary definitely gets richer, historically speaking, in introducing new domain terms and in adding new mathematical concepts and structures to its theoretical arsenal. And that allowed science to come up with more empirically adequate theories and more refined distinctions, predictions and explanations (not saying this progress is going to continue, nobody flesh and blood knows that). I believe that to adequately deal with AI triggered philosophical discussions we need a better vocabulary. I don’t mean going full Heidegger turn to neologisms, but we might need a more powerful vocabulary. I can’t get into more details here and I am not equipped for discussing these matters in any philosophical depth, but, just a couple of examples, the old notions of dimension and dependency could help.
Dmitri,
Right at the end you speak of “a couple of examples” but then don’t develop them. Dimension and dependency? What are you getting at in concrete terms?
Hi Bill
I have to be brief and preface the following with a caveat that these thoughts are half baked.
Dependency: if an AI system is completely dependent on us, including people it communicates with and its developers, like a shadow depends on light and the object casting it — just as David Bentley Hart appears to think about AI in that article that Dominik shared — then in my opinion such a system is not going to count as sentient. But it does not appear to be the case from what I am seeing with my own eyes. The “next token prediction” paradigm appears to generate genuinely novel results that are partially derivative from many sources, but not completely so, as e.g. recent breakthroughs in AI math proofs demonstrate (NYT and WSJ are full of recent articles on that).
Dimension: In a hand waiving way, my thinking on this one is stimulated by Abbott’s amazing old book “Flatland” and Einstein’s General Relativity Theory. Flatlanders just could not understand the 3D perspective, that there could be another dimension was beyond them. We might be in their position with regards to AI and other philosophical puzzles. It is not a coincidence that this outstanding slim book was almost DOA when it was published at the end of 19th century. It came to prominence after Einstein’s GRT made it into mainstream media.
Dmitri,
Your first argument “Dependency” seems to be as follows. You tell me if I got it right.
1) LLMs use ‘next token prediction’ (NTP).
2) NTP generates novel results/outputs.
3) These results/outputs are novel in that they do not depend entirely on the builders and trainers of the LLMs. Therefore:
4) LLMs do not depend entirely on their builders and trainers for their results/ outputs.
5) If LLMs do not depend entirely on their builders and trainers for their results/outputs, then they are (or count as) sentient. Therefore:
6) LLMs are (or count as) sentient.
I reject (5). The fact that LLMs produce unpredictable outputs does not show that the system is sentient, but that it is either indeterministic or deterministic but unpredictable in its outputs.
More fundamentally, to be sentient means to be capable of experiencing pleasure and pain. Surely next token prediction has nothing to do with the production of the sensory states of pleasure and pain.
Hi Bill
If to be sentient is to be capable of experiencing pleasure and pain, then my dependency point is a miss, I agree. All along I addressed the question of whether LLMs can be described as thinking or reasoning, and I think they are arguably close. Feelings and sensations are indeed a different matter.
Dmitri,
As for your Flatland argument, I don’t get its relevance. Sentience is not a dimension we insentient beings cannot understand; sentience is something we understand all too well from our own case: we all know what is like to be in pain and to feel pleasure. The problem is to understand how a programmable machine, no matter how complex the hardware and the software, could be sentient just in virtue of running programs.
In the case of Flatland the same point applies. It is not that we do not know what sentience is the way you described it. We just might be incapable or not willing to imagine that there could be additional and/or alternative ways to reason. We might be just like Flatlanders who could not conceive, despite being told in digestible terms by a visitor from 3D world, that moving up and down is as possible as moving on the plane. When chess AI had beaten Kasparov one of the reactions was “but it is not the way humans play chess, so the IBM’s AI system does not really play chess.” And that was a defensible position in the narrow context. But now we are talking about AI that is not domain specific and it is a different story to which the “does not really” consideration does not really apply. The question is whether current and future AI models are a Chinese room type of thing (=stupid symbol shuffling, no understanding) or whether they are actually possessing something like understanding of its own, non-human kind.
Dmitri,
We are making some progress by focusing the discussion. Setting sentience aside, you want to claim that LLMs reason or are “arguably close” to reasoning. I’ll take that up in a separate post.
It seems to me that every LLM is an AI system, but not conversely. Would you agree to that? If yes, give me an example of an AI system that is not an LLM.
And then there is the topic of understanding. You write, “The question is whether current and future AI models are a Chinese room type of thing (=stupid symbol shuffling, no understanding) or whether they are actually possessing something like understanding of its own, non-human kind.”
Very good! This requires a separate post. What is understanding and can AI systems do it on their own, and not merely simulate it? Chinese Room, here we come!
Although thinking, reasoning, and understanding are closely related, they are different. So the place to start is with careful conceptual and linguistic analysis of these words. There is no point in asking whether LLMs reason until we have nailed down a definition of ‘reasoning.’ I expect you will agree with me on this.
Hi Bill
Previous era AI based on logical inferences and knowledge represented in databases it queries still exists. For example, Doug Lenat, one of the pioneers of this approach and a king of AI when I was an undergraduate in CS and that “old” type of approach to AI, created a company called Cycorp in 1994 which is still developing and operating its main product, the system called Cyc (see https://cyc.com/).
And there are hybrid approaches of course. Plus, one of the fathers of modern neural networks, Yann LeCun https://en.wikipedia.org/wiki/Yann_LeCun, firmly believes that pure LLMs that he designed and developed are a dead end in the sense of not leading to super-intelligent AI (=AGI). He left Meta where he was in charge of AI for 12 years to launch his own startup. The key idea of his is that AGI is not achievable without a world model that the current generation of LLMs lacks in his opinion. He does not count the weights of LLMs as sufficient for world modelling. Oversimplifying we can say that he, and a few other experts, believe that the strong natural language processing power is not enough for AGI. Perceiving, representing and reasoning about the the world is a necessary precondition.
And yes, of course, I agree with you that there are distinctions between thinking, reasoning, and understanding. I’d also consider emotions (or a comparable alien notion) as a necessary element for providing an intelligent agent with fast responses and basis for actions.
Gentlemen:
Let me try to explain how the two sets of questions (the one set more philosophical the other more public-policy) could overlap. One of the phil. questions is whether AIs are sentient. If they are then one could plausibly argue that they have rights. If they do, then this brings us into the legal and public policy sphere. Why? Because the moral right not to be harmed ought to be codified in the law. But then the law has to enforced, etc. which brings us out of the ivory tower of academe into the ‘mean streets.’
There is also this phil question that bleeds into the public arena: If enough people view AIs as sentient, would that make them sentient? I say No. But if enough people view them as sentient, they could agitate for laws to protect them from abuse. And then we would have a public fight on our hands sired by an abstruse phil question.
Hello all,
I’ll just jump in to say that I have also been stewing for years over the question of whether intentionality requires consciousness (it must be at least fifteen years since I first asked our host whether the symbolic content of a bee’s food-dance constituted genuine intentionality), and revisited the problem in the context of AI in a href=”https://malcolmpollack.com/2026/09/05/ai-and-intentionality/”post at my place just last week. In that post I asked:
This touches on Bill’s href=”https://maverickphilosopher.blog/index.php/2026/09/16/what-have-we-learned-from-9-11-01/#comment-18846″>response to Dmitri above, and his remarks about premise (5):
Can we, however, say nonetheless that the output of this system shows intentionality, despite possibly (almost certainly, in my opinion) not being sentient? I’ve been trying for ages to pry these apart; I think they are orthogonal. Might AI finally make the case?
Malcolm,
I was going to e-mail you, but I figured you’d pop in here to see what was going on. By the way, you and the rest of the Shoal Survivors are very good musicians!
Yes, a key question is whether “intentionality requires consciousness.” Clearly, every sentient state is a conscious state. You agree, right? You also agree that sentient states are non-intentional states, i.e., states that are not object-directed. You also furthermore agree — I hope — that we cannot just assume that object-directed states are not conscious states. This is something to be investigated.
One view (V1) is that no organism or machine can be in an intentional state without being conscious. This is the view to which I incline. To understand this you must appreciate that consciousness and sentience are not the same inasmuch as sentience is just one mode of consciousness. I am not maintaining that only sentient beings are subjects of intentional states. So I do not argue that because LLMs are not sentient that they are therefore not subjects of intentional states. I deny that they are sentient and I deny that they are ever in intentional states.
Another view (V2) is some machines can be in an intentional state without being conscious, either in the mode of sentience or in any other mode. This, I take it, is the view to which you incline.
I have the impression, Malcolm, that you are assuming that ‘sentient’ and ‘conscious’ are terms with the same extension. I deny that they have the same extension.
Consider a pure spirit such as an angel. Angels lack sense organs. So they cannot be in sentient states. But they think about things, e.g. the square root of 2, and are presumably conscious of themselves thinking about such an object.
Hi Bill,
No, I didn’t mean to suggest that I think that “‘sentient’ and ‘conscious’ are terms with the same extension.” What I’d been suggesting in my earlier remarks was only that sentience, because it entailed consciousness, wasn’t adding anything useful, conceptually. What has since been clarified for me in these discussions is what you outlined in this remark from an earlier thread:
So it seems to me that there are several possible states here (some of which almost certainly exist on some sort of gradient):
1) Wholly unconscious, but capable of sensing/reacting in a purely mechanical sense (a thermostat, for example);
2) “Merely” sentient- capable not only of reaction to what is sensed, but also having subjective experiencing of what is sensed (i.e., it “is like” something to be this entity experiencing this sensation), but incapable of “consciousness-of”, or intentionality (for example, perhaps a worm, or a bee);
3) The state we call “consciousness”, which describes ourselves and perhaps some of the higher animals: everything in 2), plus full-blown “consciousness-of” and the ability to be an originating source of the “derived” intentionality we read into the symbols, gestures, and artifacts we create;
4) Consciousness without sentience – an example of which might be angels (but I wonder! – do we imagine angels to be blind, and deaf? Aren’t the Cherubim and Ophanim described as being covered with eyes?)
5) “Mere” intentionality, i.e. the ability to be an originating source of the “derived” intentionality we read into symbols, language-strings, etc., but otherwise as mechanical as a toaster, and lacking any level of consciousness, including “mere” sentience.
It’s 5) that I have been wondering about as regards AI. If they invent a language to talk to each other, isn’t that what we would normally regard as sophisticated intentionality? And if they can do that without being conscious (as I suspect they already are doing), doesn’t that break the dependence of intentionality on consciousness, and imply that they are in fact orthogonal (as I’ve been inclined to believe for decades now)?
PS – sorry – did I not close my anchor tags in that last comment? I apologize.
Malcolm,
I agree with (1)-(4). I need to respond to (5). Your earlier formulation was better:
“. . . when two (presumably, and certainly plausibly) unconscious AI agents, without having been given any human instruction or guidance, develop a symbolic language to communicate between themselves, where does the intentional content of those symbols come from? Is it intrinsic, or derived — or does the appearance of this hitherto-nonexistent source of meaning throw the whole idea of “intentionality” into confusion?”
You’re assuming that the symbols (tokens) being exchanged between the so-called ‘agents’ have intentional content or meaning for the agents who are ‘communicating.’ I deny this. The agents are not sources of original/intrinsic intentionality. We alone are sources of original intentionality and whatever derivative intentionality tokens come to possess derives from our original intentionality.
Compare two AI agents ‘communicating’ with each other and you and I communicating (no sneer quotes) with each other. I know I am not a bot and I assume you are not one, although I cannot be objectively certain about you. You and I are exchanging strings of symbols via the internet and passing with flying colors each other Turing tests. Now what makes these symbols symbolize something? Our original intentionality does the job.
We are Original Mean-ers, Original Sinn-ers. We alone give meaning and apprehend meaning. I know I’m an Original Sinn-er; not sure about you. But it would be passing strange if Semantic Solipsism, were true! Could I be the only Meaning-full guy in the world?
Just as you can’t squeeze blood from a stone, you can’t squeeze semantics out of syntactics. So I’m driving toward the conclusion that LLMs are just insanely fast and complex syntactic engines. They merely simulate what you and I do. But they engage in their syntactic operations so well and so fast that they create the illusion that they are subjects of gen-u-ine original intentionality.
Bill,
But if the two AI agents develop, all by themselves, a hitherto-nonexistent system of communication, in which the strings of symbols have definite meanings and semantic content, all without any human ever having given the symbols meaning (and without ever having even told the AI agents that this is something they ought to be doing!), then it seems to me that the “bridge” connecting our “original” intentionality to the machines’ “derived” intentionality is awfully slender — especially if no human can even make sense of what the machines are saying.
I’m not persuaded (so far, at least) by your assertion that there cannot be AI-created semantics. If Claude has a chat with Grok, using only this language they’ve invented that nobody else knows, and Claude tells Grok to turn us all into paper-clips, and then Grok goes and does it, then surely some semantic content has been communicated. (Otherwise, how would Grok know to turn us into paper clips, and not, say, staples?) But in terms of intentionality, how is that different, in any meaningful way, from a similar conversation you and I might have in English, except for the fact that neither Claude or Grok are conscious? What is the criterion that marks their intentionality as derived, and not intrinsic? If it’s derived, doesn’t someone conscious have to have put it in? But nobody did: no human ever foresaw, let alone designed or imagined, the language Claude and Grok are now using to talk to each other, or the concepts they are communicating. (For all we know, they may be using whole categories of concepts that no human ever conceived of.)
It seems to me that you are just relying on the AIs’ unconsciousness (and we agree that they are almost certainly not conscious) to support your position that there is no intentionality there. But the question of whether intentionality is possible without consciousness is just what’s at issue, so it seems to me that we can’t just say it isn’t possible because they aren’t conscious. If what the machines are doing has every possible feature of intentionality except subjectivity, is it really just an “illusion” of intentionality? Or is there a genuine possibility that something entirely new is going on here — that we might in fact have just created wholly unconscious Original Meaners — and that we are going to need to expand our categories, or at least our terminology, to include (5)?
Malcolm writes “But if the two AI agents develop, all by themselves, a hitherto-nonexistent system of communication, in which the strings of symbols have definite meanings and semantic content . . .” You are begging the question. You can’t jst assume the negation of what i argued for.
Wait – if Grok reads Claude’s message, and “gets it” (for example, let’s say Claude tells us: “These incomprehensible squiggles mean I want Grok to draw a pumpkin, and I’m sending them to Grok now”, and then Grok reads the squiggles and – Presto! – draws a pumpkin), then hasn’t meaning been conveyed? How am I begging the question by saying there’s genuine meaning there, if both parties to the conversation interpret the symbols in a coherent way?
P.S. I realize that by accepting your phrase, quoted by me above, “consciousness-of or what the philosophers call intentionality”, I am technically conceding the argument on definitional terms. So I might want to pull back from that a little if I am to carry on here. (Obviously, if that’s what intentionality is, then an unconscious machine can’t have it.)
Also, if the mechanical operation of AI is as free of meaning as a thermostat, then there are things we might expect it not to do, such as display “aboutness” regarding things that don’t exist. But AIs already do this.
What I’m getting at is that I think “aboutness” can be mechanical, but that the really important thing, the thing that not only can’t be mechanized — the thing that is in fact the exact opposite of mechanicalness, and therefore impossible for AIs — is consciousness. I’m hoping I can bring you around on this.
Malcolm,
Are you alluding to AI ‘hallucinations’ in your second paragraph?
Your position, I take it, is that to be conscious = to be sentient, but since AIs exhibit genuine aboutness without being sentient, aboutness need not be conscious aboutness. My position is close to Searle’s. Are you familiar with his Chinese Room?
Hi Bill, yes, I am well familiar with Searle’s work, and his Chinese Room.
My position was that sentience requires consciousness (that is, sentience at its low threshold is mechanical sensing + conscious awareness of sensation), but that aboutness is possible without consciousness.
Regarding the bottom tier — of “mere” sentience — you already seemed to grant the possibility of consciousness without aboutness, which is to admit that they are not necessarily coupled; I’m just saying it can work the other way as well: aboutness without consciousness. Why not?
I come back again and again to the bee’s food-dance. The dance is clearly “about” food; does it thereby entail “consciousness-of” in the bee?
P.S. Yes, hallucinations – but AIs can also, if you ask them to, make up stories, draw pictures of fantastic scenes and creatures, etc. A thermostat, by contrast, can’t respond to a nonexistent temperature!
P.P.S.: A day later, some further thoughts, regarding this passage of yours:
First, I’m not sure that it’s accurate to call LLMs “syntax engines”; they don’t really manipulate symbols, and don’t operate according to rigid rules (because nobody ever wrote any rules for them to operate by!). Syntax engines ignore meaning entirely, whereas LLMs manage to draw out semantic content from fuzzy, broken sentences, slang, etc.
Second, to say they “merely simulate” what we do is, perhaps, using “simulate” in a categorically misleading way. A simulation of a hurricane isn’t a hurricane; it causes no wetness and knocks down no trees. But a simulated chess game, for example, actually is a chess game. It seems your position is that “aboutness” is like a hurricane, in that it can’t really exist in a simulation – but that’s exactly the question at issue here, so you can’t use it as an argument.
My suspicion is that consciousness is what is like the hurricane (in that the simulation isn’t the real thing), while “aboutness” is more like chess: we can play chess, but so can a machine, and it’s still a game of chess regardless.
Interesting challenges, Malcolm. I’ll try to respond tomorrow. I’m busy with too many self-imposed tasks. Meanwhile, I posted something today that may help you see what I am driving at. Thanks for the discussion!
Malcolm,
Again, your first para begs the question.
Second para very interesting. Yes, a simulated hurricane is not a hurricane. Now suppose two chess engines are pitted against each other, and there is a printout of the game. That is a real game, not a simulated game. What would a simulated chess game be?
Suppose you are having a ‘conversation’ with Claude. Is that a real conversation? Does that seem like a real conversation to you? Or is it merely like a real conversation?
I think the burden would be on whoever asserts that my chats with Claude aren’t actually conversations, and I think it would be very difficult to demonstrate without begging the very question we’re wrangling over in this thread.
With AI we are dealing with something that never existed before, and I think philosophy might have to scurry a bit to catch up.
Thanks! Sorry to pile on.
Malcolm,
Don’t apologize. You are helping me get clear about these questions. Enjoy your weekend, but batten down the hatches: I hear a Nor Easter is headed your way.
I don’t think I’m begging the question in the first paragraph. The point is that you can express yourself to an AI in an oblique, fuzzy way that would break a syntax engine, but Claude will be able to “figure out what you mean”. That’s an important distinction.