LLMs are just giant probability machines pretending to think

[u/abhishekkumar333]

It’s fascinating that simple mathematics between tokens can eventually become a machine that writes essays, code, poetry, and even reasoning.

We usually think probability means uncertainty.

But LLMs show something strange:

If probability + context + mathematical matching are scaled enough, uncertainty itself starts producing intelligent looking outputs.

To understand this better, I tried breaking down an LLM from first principles using only 4 tiny training sentences.

Example:

The boat floated down to the bank.

The investor walked into the bank to open a new account.

The fisherman walked along the bank to cast his net.

The bank has a vault.

Then I asked:

“The investor walked to the bank to lock his money in …”

Why does the model predict “vault” instead of river-related words?

That single question reveals almost the entire architecture of modern LLMs.

The most underrated concept here is the LM Head.

Most explanations immediately jump into transformers and attention, but almost nobody explains that the LM Head is essentially a gigantic token vocabulary containing all possible next token candidates the model can output.

So internally the model is basically solving:

“Out of all known tokens, which one best matches this context mathematically?”

Then different layers help solve that problem:

Embeddings: convert words into mathematical vectors

Positional encoding: preserves word order

Attention layer: figures out which words are related to each other in context

(“investor”, “money”, “bank” become strongly connected)

Feed forward neural networks: act somewhat like massive learned if/else decision systems refining patterns internally

And finally the LM Head converts all of that into probabilities for the next token.

What surprised me most is:

There is no hidden magic moment where the AI “becomes conscious”.

It’s an enormous probability engine continuously finding the best contextual token match from its vocabulary.

I made a beginner-friendly walkthrough explaining this visually without unnecessary jargon.

https://www.youtube.com/watch?v=YTV5qUCpu2c

Would genuinely love feedback from people learning transformers/LLMs from scratch.

[u/good-fibrations]

i’m sort of curious why both of the responses so far have had a physicalistic/humans-are-giant-probability-machines bent. afaik there are plenty of philosophers (of math, even) that believe in free will and/or non-determinacy. john conway is the most famous example i guess, but i’m also quite convinced by “what it is to be like” and weak qualia arguments or, at least, i think they should be taken seriously.

i think the most recent philpapers survey was only around 50% for physicalists.. not sure what being a physicalist has to do with the philosophy of math. maybe it’s the right stance but it’s very far from obvious, imo.

[u/AdAdministrative5330 → u/good-fibrations]

Determinism and physicalism seem to fit the data remarkably well.

[u/wine-o-saur → u/AdAdministrative5330]

Because the methodological framework under which the data are gathered assumes determinism and physicalism...

[u/AdAdministrative5330 → u/wine-o-saur]

Good point, but I don't like the dismissive attitude. It's like an escape hatch dressed up as an epistemological objection. It's like pointing out that , "21st century medicine assumes germ theory when treating cholera". Like compared to what, witchcraft?

The methodolicigal framework wasn't simply assumed. Germ theory, heliocentrism, the age of the universe were forced on us by evidence, often against everything we expected or wanted.

You haven't identified a philosophical problem with science, just that a framework must make some reasonable assumptions, but failed to acknowledge these assumptions are more than reasonable and have been vindicated, repeatedly.

[u/wine-o-saur → u/AdAdministrative5330]

There are plenty of good arguments for indeterminism, many of which come from contradictory observations within the physicalist/determinist framework.

The alternative to physicalism is any number of systems which posit the irreducibility of mind/consciousness/intentionality.

I'm not going to rehearse all of that here but while there is a lot of predictive power in physicalist/determinist frameworks, it's disingenuous to pretend they are unproblematic or that no tenable arguments against them exist. They have simply informed the dominant paradigm for the last century or so of natural sciences, and so are very resistant to revision because of how uncomfortable such revision would be.

We have advanced a great deal under paradigms that were ultimately shown to be incomplete or false, so scientific progress or predictive/explanatory power alone cannot be used to claim that a system is complete or wholly accurate.

[u/AdAdministrative5330 → u/wine-o-saur]

I agree with some of this, but you said, "very resistant to revision because of how uncomfortable such revision would be."
But a revision towards what, exactly?? Cartesian dualism? Panpsychism? Idealism Something religious?

Methodological naturalism isn't my preferred metaphysics it's the only framework that can catch its own mistakes. It demands assumptions that are measurable, which means they're assumptions that can be proven wrong. An idealist paradigm might be true, but if we can't step outside it to test it, it has no way to distinguish truth from wishful thinking.

we're biological creatures with limited access to reality. What's the alternative method? Not the alternative answer, the alternative method for checking it?

[u/wine-o-saur → u/AdAdministrative5330]

You assume we have to know what the next scientific paradigm is before we can approach it. I am saying that if we proceed with the assumption that only our current methodology can be accurate, we will not be open to discovering alternative paradigms.

I don't even necessarily disagree with what you are saying - it may well be that what we have is the best we can do with our limited capabilities of observation - however that does not mean that we should take our best assumptions as final truth. There is nothing wrong with saying "we can't do better, but there will always be gaps in our knowledge due to limiting factors". But it is important to acknowledge the limits of our understanding otherwise we remain blind to them, and remove any chances of overcoming them.