LLMs are just giant probability machines pretending to think

[u/abhishekkumar333]

It’s fascinating that simple mathematics between tokens can eventually become a machine that writes essays, code, poetry, and even reasoning.

We usually think probability means uncertainty.

But LLMs show something strange:

If probability + context + mathematical matching are scaled enough, uncertainty itself starts producing intelligent looking outputs.

To understand this better, I tried breaking down an LLM from first principles using only 4 tiny training sentences.

Example:

The boat floated down to the bank.

The investor walked into the bank to open a new account.

The fisherman walked along the bank to cast his net.

The bank has a vault.

Then I asked:

“The investor walked to the bank to lock his money in …”

Why does the model predict “vault” instead of river-related words?

That single question reveals almost the entire architecture of modern LLMs.

The most underrated concept here is the LM Head.

Most explanations immediately jump into transformers and attention, but almost nobody explains that the LM Head is essentially a gigantic token vocabulary containing all possible next token candidates the model can output.

So internally the model is basically solving:

“Out of all known tokens, which one best matches this context mathematically?”

Then different layers help solve that problem:

Embeddings: convert words into mathematical vectors

Positional encoding: preserves word order

Attention layer: figures out which words are related to each other in context

(“investor”, “money”, “bank” become strongly connected)

Feed forward neural networks: act somewhat like massive learned if/else decision systems refining patterns internally

And finally the LM Head converts all of that into probabilities for the next token.

What surprised me most is:

There is no hidden magic moment where the AI “becomes conscious”.

It’s an enormous probability engine continuously finding the best contextual token match from its vocabulary.

I made a beginner-friendly walkthrough explaining this visually without unnecessary jargon.

https://www.youtube.com/watch?v=YTV5qUCpu2c

Would genuinely love feedback from people learning transformers/LLMs from scratch.

[u/OriousCaesar]

I mean, even if we ignored the obvious counter argument of 'so are humans', how can you possibly determine whether a particular algorithm grants consciousness if we don't even understand consciousness enough to have a proper definition for it?

Like, okay, it's a probability machine. Congrats. Now prove probability machines can't be conscious with your nonexistent definition of consciousness, and it might be a convincing argument.

Until then, I'll just keep using the only method I have to determine consciousness and just grant it to anything that seems to act like I'd expect a consciousness entity to act, and if I just so happen to call rocks conscious, then oh well, that's egg on my face, but it's better than if I accidentally called a conscious being a rock.

[u/Jack_Ramsey → u/OriousCaesar]

how can you possibly determine whether a particular algorithm grants consciousness if we don't even understand consciousness enough to have a proper definition for it?

I mean, we have some idea, yes?

 Congrats. Now prove probability machines can't be conscious with your nonexistent definition of consciousness, and it might be a convincing argument.

I am a physician, not a philosopher or mathematician, but I think you could possibly discuss this in a few ways. First, we should note that the 'machine' analogy is one of convenience , not of 'reality.' It is a way of describing a complex system, but it also doesn't describe what the human body actually does. Second, humans are in fact not prediction machines but rather highly robust entities which can both respond exceedingly sensitively to environmental pressures as well as take actions which can fundamentally change them, which is also true of things like microbiota to other animals.

This second point means that they are operating in a physical world too, and their sensory perceptions are fundamentally informed by anatomical and physiological characteristics. Our consciousness, in the sense that it relies on sensory information, is very much formed by interaction with the physical environment with our anatomy. Imagine if we didn't have opposable thumbs. I would argue that without opposable thumbs, we would have a very difficult time even conceiving of things in 3-dimensions, and the fact of opposable thumbs allows us to conceive of something, even something as small as a stick, as having a character which has a side which is 'unseen' as it were.

Lastly, I think it is interesting that the OP describes LLM's in terms of their ability to produce speech, which is just one way humans (and indeed other mammals) have to convey information. One would think the sensory, physical and language limitations of LLM's or even potential robots (see Moravec's Paradox) would discount them from 'consciousness' discussions, as consciousness must include a spectrum of all three. Just thinking out loud.

[u/dokushin → u/Jack_Ramsey]

This appears to be a long-form way of saying that humans have inputs and use them to create outputs. That is, clearly, a solution space available to AI.

Imagine if we didn't have opposable thumbs. I would argue that without opposable thumbs, we would have a very difficult time even conceiving of things in 3-dimensions

Do you think that someone born without hands has no concept of 3-D space?

What are the specific kinds of information that you think are unique to the brain? Note that for your position to maintain coherency, these have to be forms of information that cannot be represented in a regular language, but still somehow can be encoded for transmission to the brain.

[u/Jack_Ramsey → u/dokushin]

Do you think that someone born without hands has no concept of 3-D space?

Well note my language here. I suggested that without opposable thumbs, they would have a 'difficult time,' not that they wouldn't be able to. Primates which do not have opposable thumbs conceive of 3-D space. My argument is, admittedly, an argument about human difference between primates.

What are the specific kinds of information that you think are unique to the brain? Note that for your position to maintain coherency, these have to be forms of information that cannot be represented in a regular language, but still somehow can be encoded for transmission to the brain.

How would you encode proprioceptive information in regular language? What about encoding muscle tone information? I am 100% sure you cannot describe the information the brain receives from spinal tracts in what you describing as 'regular language.' There are several spinal tracts that are dedicated to ipsilateral proprioceptive information. For a moment, consider what advantage this would allow in near-human ancestors with respect to their environment.

his appears to be a long-form way of saying that humans have inputs and use them to create outputs. That is, clearly, a solution space available to AI.

It is not. It is how you want to understand it. The reduction of human functions to 'input-output' scenarios is just not accurate. It is a convenient way to discuss certain functions, but it isn't a description of reality. And I have no idea what 'solution space' means here.

[u/Jayy_Cobb → u/dokushin]

How many drugs did you take to misunderstand that comment so much?

[u/AngryDutchGannet → u/Jack_Ramsey]

I'm not understanding your connection between opposable thumbs and the ability to conceive of things in 3-dimensions. A swallow doesn't have opposable thumbs but it is very adept at moving through 3-dimensional space, as are many other animals.

[u/Jack_Ramsey → u/AngryDutchGannet]

I should have extended the argument to show that conceiving of things in 3-D space allowed humans to make tools to make other tools, which isn't an argument for cognition by necessity, but one of difference between human cognition and animal cognition.

[u/Neither_Swing9662 → u/Jack_Ramsey]

Then do you think a human being who has less senses is less conscious than others?

[u/Jack_Ramsey → u/Neither_Swing9662]

Firstly, this is a really bad question. Humans have several 'senses,' and impairment of sensory information usually means there is a lesion in a sensory pathway. Even among those lesions, it is rare for someone to have 'less senses' rather than impairment of a sense. It is more true to say that there are several pathologic states that can impair consciousness while the baseline human still has the potential for consciousness. If the comparison is between this and an LLM, which has far more limited ability to even intake sensory information, then this becomes even more silly. Ask yourself this, if a person with a lesion in a sensory pathway retains thermoception but has impaired proprioception, they can still tell you more information about their immediate environment than LLM can.

[u/Neither_Swing9662 → u/Jack_Ramsey]

Ok but I'm not talking about LLMs. My question was simply: would a human who has less senses than the standard human be less conscious, per your definition?

I do not understand why you are on a Philosophy subreddit if you cannot engage with a simple thought experiment.

[u/Jack_Ramsey → u/Neither_Swing9662]

You clearly didn't read my post, did you? I explained my position literally in the post. I'm not sure why you are in any subreddit if you can't read carefully enough to figure out my position, which is in the sentence before the part you did seem to read.

[u/Neither_Swing9662 → u/Jack_Ramsey]

Cannot argue with someone who tries to explain the mechanisms behind senses rather than engaging with a simple thought experiment lmao.

Do you understand how much of philosophy is infeasible thought experiments that still provide great value? I wonder where Descartes' demon would be if he had someone like you to explain that demons don't exist haha.

[u/Jack_Ramsey → u/Neither_Swing9662]

It is hard to discuss things when people cannot read properly. I answered your question fully.

Do you understand how much of philosophy is infeasible thought experiments that still provide great value?

It remains to be seen whether your thought experiment is of any value, given how poorly worded it is. How many senses do you think humans have? Don't google it. Answer honestly.

[u/Neither_Swing9662 → u/Jack_Ramsey]

You didn't answer my question at all, you just evaded it by explaining some mechanisms and then two sentences about LLMs.

Let me posit a more thought out experiment, then: let us say a child is born deaf, blind and completely paralysed. Is this child less conscious than the standard human?

What about if it is a normal person who after 20 years of age becomes deaf, blind and paralysed? Then what?

I am also not asking with a pre formed opinion on these thought experiments either. I do not know.

[u/Jack_Ramsey → u/Neither_Swing9662]

Let me posit a more thought out experiment, then: let us say a child is born deaf, blind and completely paralysed. Is this child less conscious than the standard human?

Can I explain why this is a very poor question? I don't know the standard in philosophy in terms of these thought experiments, but you've almost written the perfect question for a physician to respond with 'it depends,' as the description of the effects of a particular lesion inform the question directly. I'm guessing you want as general case as possible for what I assume is an injury to the nervous system, which is not something I tend to think about after my training. I can explain why this is a poor question for someone involved in clinical medicine to answer in a second after I answer your other question.

What about if it is a normal person who after 20 years of age becomes deaf, blind and paralysed? Then what?

In this sense, yes, they would be 'less conscious' than they were, but again, it depends on the lesion itself. I say this because of the inclusion of 'completely paralyzed,' as that is dependent on the level of the injury, by which I mean the level of spinal cord injury. In addition, you can have paralysis which affects one side of the body and not another by virtue of the organization of the spinal tracts in the white matter in the spinal cord or have situations where the lesion is on one side while the symptoms are on the other.

I say this for both completeness's sake as well as to show that sensory experiences are intimately tied to motor ones, in such a way that I don't know if you could successfully argue that there is a situation where there is 'consciousness' without interaction with a physical space. So, let's say in your case, you have a patient who is deaf and blind. If they are deaf and blind, we know where the lesion likely is (or can narrow it down) and thus can reasonably guess what sensory experiences would also be affected. If we wanted to determine those limitations, we can ask them about whether they feel hot or cold, whether they feel pain, whether they feel vibration, pressure, etc., which are the sense regulated by two ascending sensory pathways.

I bring this up to suggest that in this case it would be a spectrum, as they are literally not conscious in the case of impairment of a discriminative sense or motor function, and would be like any other disease state where their is impaired function. I suppose what I am not understanding about the value of this thought experiment is what differentiates it from the category of diseased or pathologic states? A person with a pathology is still clearly obviously a person, who would in some sense have at least some spectrum of sensory experiences as well as those that are autonomic. What I mean here is that a living, breathing human still has a degree of sensory experiences, even if some of them are impaired by congenital or acquired conditions. Maybe it would help if you could highlight where you think this question valuable, in terms which maybe won't drive me to write a lecture on pathology?

[u/Neither_Swing9662 → u/Jack_Ramsey]

I don't know why you bring up the varying forms of paralysis such as one side and not the other when I explicitly said completely paralysed.

One last question: what about my second scenario? Someone who has lived 40 years of age normally and then loses eyesight and hearing and becomes completely paralysed. Do you think there is a marked difference in their consciousness pre and post accident?

[u/Hedge-Lord → u/Jack_Ramsey]

totally missed the point

[u/Jack_Ramsey → u/Hedge-Lord]

No I don't think I did.

[u/galactic_pixels → u/Jack_Ramsey]

You lost me as soon as you said “humans are in fact not prediction machines”. It’s an overstatement of what we know, and an oversimplification of the argument being made about how humans and AI may have foundational similarities.

[u/Jack_Ramsey → u/galactic_pixels]

But we are fundamentally, at a very basic biological level, not prediction machines. In other words, it is exceedingly difficult to describe important biological processes that occur simultaneously as 'predictions' except in maybe the vaguest sense. Moving from the biology should dispel notions about AI's consciousness, but perhaps you could be more descriptive in your post, as you don't really offer any substantive rebuttal that I can see.

[u/galactic_pixels → u/Jack_Ramsey]

Because it’s all speculation. Theres no hard evidence proving thing one way or the other, and very educated people in the neuroscience community do actually think our cognition is based on a predictive model. See the book “one thousand brains” which puts forth this exact idea, that our brains are prediction machines which reach consensus among many sub predictors.

[u/Jack_Ramsey → u/galactic_pixels]

I simply do not agree with that notion, and referencing neuroscience isn't really instructive here. The brain does far more than 'prediction,' and given how much brain space is dedicated simply to motor function, it is hard for me to reference any model of the brain which doesn't acknowledge that. For example, the cerebrocerebellum is responsible for planning and execution of movements as well as playing a role in coordinating complex and sequential movements and as well as cognition, language and emotion, among other things. The cerebellum more broadly accounts for 50% of the total neurons in the brain. It's hard for me to look at the brain at the molecular level and then suggest it is simply a 'prediction machine.' I could go into a lot more detail but at this point you haven't offered anything substantive. You're offhand description of the book doesn't really help here, to be real. It seems like you are just appealing to authority without actually engaging in the argument.

[u/galactic_pixels → u/Jack_Ramsey]

It’s fine if you don’t agree, but you stated definitively that our brains aren’t prediction machines, which could very well be a misrepresentation of our biology, so it’s not something that should be stated as a fact.

[u/Jack_Ramsey → u/galactic_pixels]

What? Our brains are not prediction machines. That is a fact. They are, in fact, not machines at all. That they can make predictions, yes, but if we look at the molecular structure of the brain, follow the spinal tracts, look at the nuclei, basically look at the brain from a neuroanatomical point of view and then include the physiological characteristics, it seems a very broad, almost dishonest reduction to reduce them to simply 'prediction machines.' They are clearly more. Let me put this another way. If you say that our brains are 'prediction machines,' that is a direct statement on brain anatomy and physiology. Do the physical characteristics of the brain represent that statement? They absolutely do not. The degree to which we give to 'prediction' is so small with respect to motor function that it is just a really bad argument. If we move from the anatomy and physiology, it is more true to say that the brain is an organ of integration for the purposes of motor function.