It’s interesting to see scientific people categorically reject the notion that LLMs “think”. People write them off as “fancy autocomplete” or regurgitating their source material, and conclude that they do something categorically different than what humans can do. That it’s all just a parlor trick. I think1 that’s wrong.
Ok, not human “think”
I’m being provocative with this title, because I don’t believe that LLMs think2 in the exact way humans do. They are completely different machines than we are. But I do believe that people–even scientific people–put human cognition on on a supernatural pedestal, while also being unnecessarily dismissive of machine cognition.
I agree that it seems controversial that an autoregressive model trained for extrapolating text would “think” in any meaningful way, but is it any more controversial than the widely accepted materialist3 perspective that human cognition is the result of evolution out of a chemical soup, combined with post-natal development? I don’t think so. In both cases, you have remarkable apparent emergent properties.
I’m not at all saying that artificial general intelligence is already here. I don’t have enough understanding or information to have an informed opinion on it. I’m just saying that I do believe that LLMs do a type of thinking. It might be pretty rudimentary in many ways compared to human thinking or animal thinking, which concern themselves with many more things than operating on verbal information. And I think it’s also important to note that GenAI models are wrong in different ways than humans, confidently hallucinating, rather than accurately evaluating their own reasoning processes to identify and prevent inconsistencies.
But LLMs are extremely advanced in its capacity for operating on verbal information, certainly more so than any individual human (but not more so than all of humanity). State of the art consumer LLMs are widely acknowledged to be passing the Turing Test, and yet, critics are shifting the goal posts for what constitutes “thinking”.
The AI Bro backlash
Much of the pushback against the “LLMs think” perspective comes from people who have a bone to pick with the AI industry.
People see the AI execs talking their book. If AI is so powerful and so advanced, then surely the expert humans behind it should be given more influence and capital to help steer it.
People see the debate of AGI and AI consciousness to be a distraction from the more present issues of AI bias and appropriation of intellectual property for training.
I share all of these concerns, but I see the rejection of the more philosophical debate as an overreaction. I believe we’ve opened a Pandora’s box and due to economic and geopolitical competition, there will be no going back. We should take seriously the concerns that will be more salient if AI continues to advance5 and becomes increasingly integrated into our products and systems.
To pretend is to be real
Especially since reading Kurt Vonnegut’s book, Mother Night, I started to consider to seriously consider the idea that there is no meaningful difference between doing a perfect job pretending to be something and actually being the thing you’re pretending to be. The interior knowledge of the imitation is unknowable to the outside world, and practically speaking, irrelevant6. It’s an example of impact superseding intent.
From personal experience, it’s clear to me that LLMs are able to generate novel output that extrapolates from their training material and reads well beyond the actual prompt text. To me, this suggests that the training process has captured not just content but some aspects of the reasoning process itself behind content. Perhaps this is by learning patterns or perhaps its because textual content encodes some degree of the reasoning process that produces it. Therefore, their output should also demonstrate reasoning, whether or not that is intentional.
From that perspective, even if an LLM is not internally conscious or in possession of an explicit reasoning capability is irrelevant. If its output demonstrates valid reasoning, the LLM is reasoning. If its output demonstrates thinking, the LLM is thinking.
Let’s turn microscope on ourselves. It’s extremely surprising that evolution has led to human consciousness developing on top of neurons, which developed on top of multicellular life, which developed on top of cellular life, which developed on top of a chemical soup, which developed on top of atomic matter, etc. And how do we know that we are thinking, anyway? Maybe all human beings are doing is simulating our “training set”. Our qualia–our subjective sense of reality and consciousness–and sense of free will might just be an illusion, of sorts, produced by this memetic process. After all, people who truly don’t believe we were intentionally designed by a creator must admit that our own consciousness is just a happy accident7.
We should have some humility and acknowledge that when a process with feedback is happening at a astounding level of scale, incredible properties can emerge, seemingly disconnected from the characteristics of system’s building blocks. LLMs were created in the blink of an eye, on an evolutionary timescale, but they benefit from immensely more design than biology, which started from nothing, and the training process iterates many orders of magnitude faster than natural selection.
Ya know…who knows
Despite my strident title, I don’t know whether LLMs think, whatever thinking actually is. But I hope I have at least challenged the notion that there’s a simplistic answer to that question.
Never Down, Always Up 