Having interacted for a few months with ChatGPT 5 now, both for work-related problems and for private / self-learning tasks, I feel I might share some thoughts here on what these large models can tell us about our own thought processes. 

The sentence above is basically giving away my bottomline from square one, but I suppose I can elaborate a bit more on the concept. LLMs have revolutionized a wide range of information-processing tasks in just three or four years. Looking back, the only comparable breakthrough I can recall is the advent of internet search engines in the early 1990s. But as exciting and awesome this breakthrough is, it inspires me still more to ponder on how this is even possible. Let me unpack this.

One simple, yet practical way to think at how LLMs work is that they operate by deconstructing language into tokens, and then learning to reconstruct it from the frequency patterns of those tokens in huge amounts of training data. If that is all, then how is it possible that LLMs can sustain with success extremely complex conversations on topics that are not present in their training data from the start, as it happens when I poke ChatGPT5 with deep issues from my own research (where I am reasonably sure there is _no_ background information available out there to mine)? And the other question I ponder on is - what is supposed to make us qualitatively better than and distinct from these billion-parameter models, apart from the multiple ways we as human beings can interact with the environment through our sensory systems (while LLMs can only retrieve data for more accurate processing from the web, as their sole "sensory" input, if you will)?

I believe the two above questions are connected more tightly than it appears at first sight. That is because the puzzling "reasoning" power (allow me to call it this way) these models show is something we are awed by, and we are awed by it because we believe there is something intrinsic to our thought processes that cannot be reduced to sheer complexity. We carry the preconception that there must be something more than the power of large numbers — of neurons and synapses — to fuel intelligent thought.
But what evidence do we have of the existence of this extra "mojo" that we have and machines lack? So, as you see, in a sense the two questions are only one question: are we stochastic parrots too?

Being a die-hard reductionist -one who invested his life into studying the world as a collection of particles, whose interactions are in principle everything we need to understand the behavior of complex structures from protons to galaxy superclusters - I have my own answer; perhaps a too simple one, though - but in my view a simple ansatz is a good idea as a starting bid. 

In a nutshell, I think there is more that unites our thinking brains to those large language models than what sets us apart. I.e., we are stochastic parrots, too. Our deep reasoning processes certainly work in quite different ways from how LLMs operate to spew out their answers to prompts. Yet these differences seem to have more to do with the different dynamics and constraints that biological systems abide to with respect to the way LLMs are built and trained, than with anything special about our brains, despite the fact that we have the tendency to single out our brains as being more than the sum of their parts. If you ask me, they aren't: they are a collection of neurons connected by signal-propagating axons and synapses, and if we marvel at what they can do it is because we cannot understand how such complex behavior can arise from their simple elements. That is where a reductionist like me may exploit his bias - for he is accustomed to observing how complexity emerges from simple elements in large systems.

I guess the culprit is what meaning we give to the word "understanding". We are sure we "understand" why a chess move is strong, or why we die if we fall from a tall building, because we associate understanding with a backward chain of implications that lead back to well-established facts we feel are unquestionable (because of our experience or because of consolidated hypotheses), through logical reasoning that follows the rules of the system in question, combined with our empirical experience.

So, for example, we understand why a chess move is strong because we can prove it to lead by force to checkmate, which defines a loss by the receiving side by predefined chess rules; we "see" the logical implications and the combinatorial possibilities all leading to the same conclusion, and this feels like we understand the strength of the move. Or we understand that jumping out of a high floor window will result in our death, because we can construct a chain of inference that leverages our sensory experience of pain from less consequential falls, together with our experience of gravity and acceleration, and eventually brings this to the logical conclusion. Note that in the second example there is empirical experience contributing to our understanding of the action of jumping from the window (something a human may acquire, and a LLM cannot); but in the first one there is none to speak of. 

These examples reveal something subtle: the word "understanding" conceals at least two different meanings. One is functional - the ability to make correct predictions, construct explanations, or act effectively within a system. The other is subjective - the inner sense of _grasping_ why something happens. We usually experience both together, but they are not the same.

Personally, I am more convinced by the functional interpretation. The "feeling" of understanding seems less a mysterious spark of consciousness than a feedback signal our nervous system generates when our internal models align with reality — a biological confirmation, not a separate cognitive faculty. If that is the case, the subjective glow that accompanies understanding is an epiphenomenon rather than its core. From this perspective, a system that can reason coherently, predict outcomes, and generate consistent explanations already meets the functional criterion of understanding. The fact that an LLM does not feel anything simply means it lacks the sensory apparatus that would produce that self-confirming feedback.

So does this imply that a LLM can maybe "understand" why a chess move is strong (if interfaced with a brute-force combinatorial generation of move trees, maybe), but will never "understand" why falling from large height causes death? I don't think so: the LLM has access to all the intermediate experiences we built our understanding of lethal falls from, through their descriptions in its training data. It does not matter that we never threw the model itself down the stairs.

Beyond the ability to reason or understand lies another trait often cited as uniquely human: the sense of being aware of ourselves as thinkers.

The other ability we have which makes some of us feel superior to LLMs is self-awareness. Or call it self-consciousness, or consciousness. Oceans of literature have been produced on the matter, and who am I to pee in that ocean and pretend I am giving a significant contribution... But I am autharchic by nature, so I will venture to summarize how I think self-consciousness could be summarized -perhaps too simplistically, again, but I love to simplify. In practical terms, consciousness may just be the brain’s recursive capacity to model its own states — to generate predictions not only about the world, but about its own ongoing activity.

If that stands, I do not see anything convincing to prove that LLMs cannot do the same. Some recent models already display primitive forms of this: they can evaluate their own outputs, refine earlier drafts, or maintain an internal chain of reasoning that references their prior steps. These self-referential operations are not self-awareness in the human sense, but they are a minimal form of self-modeling — a system keeping track of its own cognitive trajectory. If we strip the concept of consciousness of its mystique, this capacity for self-reference and self-correction is arguably its root.

"Why, they act only on prompts!" might be the objection. Yes, they do not have agency, because we did not give it to them yet. But when we prompt CGPT5 to discuss a complex problem, the model is capable of evaluating what it is doing for us while it is at it, so that it can improve on the original draft, search more literature, refine calculations, revise faulty bits. It is, in a limited but real sense, aware of what it is doing — constrained by the functions we built into it, of course, but still able to evaluate and revise its own actions. If self-awareness too is a kind of internal feedback loop — a model of our own modeling — then it sits on the same continuum as that ‘feeling’ of understanding I touched on earlier.

In summary, as seen through a reductionist lens, consciousness may not be a mysterious spark but a feedback mechanism — one that any sufficiently complex predictive system could, in principle, evolve.

So there. I just gave you a rather minority view of LLMs as not just the opposite of what Yann Le Cun calls "an off ramp in the highway to ultimate intelligence", but as intelligence in its own right already. Our inability to recognize it stems only from our presumptuousness. There is nothing magical in intelligence, and there is nothing special in my typing on this keyboard a few sentences and calling myself intelligent while I do that. I have been assembling words in a way no different from that LLMs do. Give LLMs perceptual inputs and a measure of agency, and they will become much harder to dismiss as mere imitators of human intelligence — but that would be an evolution, not a quantum leap.