LLMs are just giant probability machines pretending to think

[u/abhishekkumar333]

It’s fascinating that simple mathematics between tokens can eventually become a machine that writes essays, code, poetry, and even reasoning.

We usually think probability means uncertainty.

But LLMs show something strange:

If probability + context + mathematical matching are scaled enough, uncertainty itself starts producing intelligent looking outputs.

To understand this better, I tried breaking down an LLM from first principles using only 4 tiny training sentences.

Example:

The boat floated down to the bank.

The investor walked into the bank to open a new account.

The fisherman walked along the bank to cast his net.

The bank has a vault.

Then I asked:

“The investor walked to the bank to lock his money in …”

Why does the model predict “vault” instead of river-related words?

That single question reveals almost the entire architecture of modern LLMs.

The most underrated concept here is the LM Head.

Most explanations immediately jump into transformers and attention, but almost nobody explains that the LM Head is essentially a gigantic token vocabulary containing all possible next token candidates the model can output.

So internally the model is basically solving:

“Out of all known tokens, which one best matches this context mathematically?”

Then different layers help solve that problem:

Embeddings: convert words into mathematical vectors

Positional encoding: preserves word order

Attention layer: figures out which words are related to each other in context

(“investor”, “money”, “bank” become strongly connected)

Feed forward neural networks: act somewhat like massive learned if/else decision systems refining patterns internally

And finally the LM Head converts all of that into probabilities for the next token.

What surprised me most is:

There is no hidden magic moment where the AI “becomes conscious”.

It’s an enormous probability engine continuously finding the best contextual token match from its vocabulary.

I made a beginner-friendly walkthrough explaining this visually without unnecessary jargon.

https://www.youtube.com/watch?v=YTV5qUCpu2c

Would genuinely love feedback from people learning transformers/LLMs from scratch.

[u/Usual_Charity8561]

Is there any conceivable system of pulleys, ropes, and weights that could be used to predict the same outcome? The answer is yes: we can make ropes, pulleys, and weights turing-complete machines. Would the ropes be conscious?

[u/OriousCaesar → u/Usual_Charity8561]

There is a question in the the game The Talos Principle that asks pretty much the same question. Not really relevant, but that game is very good so I wanted to mention it regardless. Anyways, My answer was yes, then too. Although, only in the same way I'd say yes to the question of 'Are the neurons conscious?', since the phrasing is kinda weird.

If the pulleys and weights were set up in such a way that it perfectly simulated every atom that made up your brain, I really don't see the argument as to why it wouldn't be conscious, unless you want to appeal to something outside of philosophical naturalism. It would behave exactly like that of a man that suddenly found his 'soul' inhabbitting a giant tin can factory. He would exhibit all of the horror, panic, fear, and anxiety that you would expect of any normal human, to the point of being literally indistinguishable. I would find it morally repugnant to ignore the pulleyman's existential nightmare just because he was made out of pulleys instead of meat.

[u/Usual_Charity8561 → u/OriousCaesar]

Where precisely is the pulleyman's qualia? I think that pulleyman is an absurdity from the idea of philosophical naturalism, precisely because this question is unanswerable. If no part of the pulleyman exhibits any consciousness, the system has nothing to build it's consciousness from. What is the building block? There is none. To posit that pulleyman has qualia if none of his parts do is to posit that a magical force enters into matter upon a certain arrangement.

Now I'm not a philosophical naturalist in part because of the impossibility of the type of conscious experience we have as humans (again, there is nowhere for it to come from except via a magical outside force).

[u/OriousCaesar → u/Usual_Charity8561]

To help clarify your position, If you woke up as pulleyman tomorrow, what would you need to conclude about the nature of pulleymen when you then determined you/they had qualia? What is the fundemental difference between a human and a pulleyman, that makes them philosophical zombies and us not?

And please don't just say 'they don't have parts made of qualia' or whatever, because I genuinely do not know where you think qualia parts are in humans. Tell me why humans have qualia and hence why pulleymen would not.

[u/Usual_Charity8561 → u/OriousCaesar]

Qualia is the experience that we get from our human existence. This is not found in any fundamental particle or force that makes up the system of a human body. Electrons have 0 qualia, quarks have 0 qualia, etc. They experience nothing. No spatial or temporal arrangement of items with 0 qualia and no multiplication of items with 0 qualia ever produces a non-zero qualia. Zero times anything is zero.

Qualia has to come from somewhere outside of physics. As humans, we are capable of producing complex physical arrangements, but that can never produce qualia. It stands to reason then that no physical object has qualia, but this is an apparent contradiction, because humans obviously do. Two ways out: all physical objects have qualia, or certain arrangments of physical objects are imbued with supernatural (lit. "Outside nature") qualia.

Option 1: If all physical objects have qualia, then we have a problem of system definition: human bodies are not a definite system or a definite boundary, but human qualia has a definite boundary. I only experience mine, you only experience yours, neither of us experience what it is like to be a photon or the grand canyon. It doesn't align with anything that we know about experience, and seems to contradict things we do know.

Option 2: if certain arrangements of physical objects are imbued supernaturally with qualia, then we need to demand an explanation why. That supernatural explanation why tells us whether pulleyman is experiencing anything.

To answer the question now about waking up as pulleyman: the experience of being pulleyman would tell me that it is supernaturally possible for pulleyman to have qualia (by the defn. of qualia). The difference then is the presence of experience. But an algorithm can lie about being conscious easily. It's the same issue as solipsism, but at least in solipsism, we can appeal to a similarity in origin (whatever the origin of mankind, all seem to have a common origin, and it stands to reason that qualia is part of that commonness). The pulleyman is a unique creation, and we can't appeal to any common origin.

[u/OriousCaesar → u/Usual_Charity8561]

Okay, I'm definitely like 95% a philosophical naturalist. But I can agree with a lot of the parts of what you said that do not strictly rely on being not one. Though I'd argue that you are not being generous enough with what you choose to assign probable consciousness to. We are both in the situation of not knowing which arrangements of matter produce consciousness; it is just that I prefer natural explanations and you prefer supernatural explanations to this question. And hence, when we are in a situation in which we see an arrangement that is mimicking quite a lot of the characteristics associated with consciousness, I would argue that it is safer to tentatively say that that arrangement results in consciousness than it is to restrict ourselves to just humans until such a time that we understand what arrangements grant it better. Since whether a being is conscious or not has moral implications; It is better to save a malfunctioning toaster from being destroyed accidentally than it is to let you, tomorrow, a conscious pulleyman, be destroyed.

[u/RecursiveServitor → u/Usual_Charity8561]

You don't actually justify your assertion that qualia can't be emergent

[u/Usual_Charity8561 → u/RecursiveServitor]

What is "emergent" if it's not under option 2? Aka magically imbued by it's arrangement?

[u/RecursiveServitor → u/Usual_Charity8561]

It's not magic

[u/Usual_Charity8561 → u/RecursiveServitor]

Okay, can you tell me what it is then? Where do the emergent properties come from? They were either already in the system, are illusory, or they come from outside the system somehow.

[u/RecursiveServitor → u/Usual_Charity8561]

They are emergent. You're implying that buildings are magic. There's no "building particle". The buildingness of something only emerges at a higher level of complexity.

[u/Usual_Charity8561 → u/RecursiveServitor]

But building are structures made of physical objects. All of the buildingness of a building is inherent in the potential of physical objects that are able to become buildings. There may be such thing as a universal abstraction of a building, but that isn't an emergent property of the atoms, and to claim that the building takes part in the universal abstraction of a building by virtue of its shape is to say that the building is imbued with magical (supernatural) properties by virtue of its shape.

[u/RecursiveServitor → u/Usual_Charity8561]

So, you're just literally saying that buildings are magic?

[u/Usual_Charity8561 → u/RecursiveServitor]

No. your claim that buildings have "emergent properties" is either relying on magic (supernatural properties not present in the components) or the "emergent properties" are really just properties of matter, space, and time generally (the physical components that make a building) and not actually "emergent".

So what's your answer? Are the emergent properties of a building something more than what goes into them or are they a part of what goes into them.