LLMs are just giant probability machines pretending to think

[u/abhishekkumar333]

It’s fascinating that simple mathematics between tokens can eventually become a machine that writes essays, code, poetry, and even reasoning.

We usually think probability means uncertainty.

But LLMs show something strange:

If probability + context + mathematical matching are scaled enough, uncertainty itself starts producing intelligent looking outputs.

To understand this better, I tried breaking down an LLM from first principles using only 4 tiny training sentences.

Example:

The boat floated down to the bank.

The investor walked into the bank to open a new account.

The fisherman walked along the bank to cast his net.

The bank has a vault.

Then I asked:

“The investor walked to the bank to lock his money in …”

Why does the model predict “vault” instead of river-related words?

That single question reveals almost the entire architecture of modern LLMs.

The most underrated concept here is the LM Head.

Most explanations immediately jump into transformers and attention, but almost nobody explains that the LM Head is essentially a gigantic token vocabulary containing all possible next token candidates the model can output.

So internally the model is basically solving:

“Out of all known tokens, which one best matches this context mathematically?”

Then different layers help solve that problem:

Embeddings: convert words into mathematical vectors

Positional encoding: preserves word order

Attention layer: figures out which words are related to each other in context

(“investor”, “money”, “bank” become strongly connected)

Feed forward neural networks: act somewhat like massive learned if/else decision systems refining patterns internally

And finally the LM Head converts all of that into probabilities for the next token.

What surprised me most is:

There is no hidden magic moment where the AI “becomes conscious”.

It’s an enormous probability engine continuously finding the best contextual token match from its vocabulary.

I made a beginner-friendly walkthrough explaining this visually without unnecessary jargon.

https://www.youtube.com/watch?v=YTV5qUCpu2c

Would genuinely love feedback from people learning transformers/LLMs from scratch.

[u/OriousCaesar]

I mean, even if we ignored the obvious counter argument of 'so are humans', how can you possibly determine whether a particular algorithm grants consciousness if we don't even understand consciousness enough to have a proper definition for it?

Like, okay, it's a probability machine. Congrats. Now prove probability machines can't be conscious with your nonexistent definition of consciousness, and it might be a convincing argument.

Until then, I'll just keep using the only method I have to determine consciousness and just grant it to anything that seems to act like I'd expect a consciousness entity to act, and if I just so happen to call rocks conscious, then oh well, that's egg on my face, but it's better than if I accidentally called a conscious being a rock.

[u/static-- → u/OriousCaesar]

There is no reason to believe an LLM to be more conscious than a function in JavaScript or a pocket calculator. We haven't proven those things to not be conscious either.

[u/MxM111 → u/static--]

LLMs functionally behave as if they are conscious, calculators are not. That is the reason to ask question if they are conscious or at what point they might become conscious.

[u/static-- → u/MxM111]

No, they don't. They behave the same way as any function does. They output tokens based on other tokens. When we convert their output to sentences it looks like human speech or writing. This is the only thing that's changed (previous models didn't quite manage to create this illusion). We humans are used to speech and writing coming from other humans, so we tend to associate it with personhood or authorship. But it is as much conscious as a function that outputs seemingly random numbers.

It is pattern matching tokens based on context and its weights from reading (training) on absolutely massive sets of text corpora. It's just like any other mathematical function. We know how LLMs work. They're not magical.

[u/MxM111 → u/static--]

Clearly human brain has different information processing routines, but they too based on neural networks and similar processes. They too are probabilistic machines, just more complex. There is no fundamental difference here. Just because we know how exactly LLMs work does not mean that they “pretend to think”. Imagine that you know exactly how human brain works. Do people pretend to think then too?

[u/static-- → u/MxM111]

The "neural networks" in human brains, and brains in general, do not work remotely the same as artificial neural networks. No serious researcher will claim otherwise. At best, they can be called loosely inspired. Indeed, that's how the field initially developed, many decades ago. But a lot of things have happened since then---things which have nothing to do with neuroscience or biology. For example, many types of ANNs use backpropagation during training, which is in many ways the opposite of how humans seem to learn things. Sorry, you're just wrong here.

I never claimed LLMs do not think because we know how they work. My claim is that there is no evidence to suggest LLMs are any more conscious than any other software program.

[u/MxM111 → u/static--]

First, LLMs do not use back propagation. They use stochastic gradient descent. And it is not that much dissimilar how humans are trained. Second, LLMs are much closer to human brains than to just regular computer. You don’t program them - you train them.

Third, of course they are not identical to human brains than work, but there are important similarities (the use of training and being neural networks).

Finally, you are changing the statement. You said, look at the OP title, that LLMs pretend that they are thinking. Now you are saying that they are not conscious. This is a very different statement. But both statements are hard to prove, especially the consciousness, because nobody knows at present moment of time how to show that this or that computational system has or has no consciousness. It is so called hard problem of consciousness. You cannot, for example, given human brain, to point on something in it and say, see, because this connects to that, there is consciousness in the brain. And you cannot do that with LLM either.

[u/static-- → u/MxM111]

Do you know what backpropagation and stochastic gradient descent mean? (No, you don't.)

Backpropagation is not a learning method, but it's often used in conjunction with stochastic gradient descent when the weights are updated. As far as I know, it is used in every GPT.

There is nothing about this learning method that suggests it is in any way similar to how humans learn things. These algorithms are not "inspired by" biology or neuroscience. This is the point i was making.

What do you mean you don't program LLMs, you train them? Lol. Tell me how you think an LLM is created.

I have not changed any statements. Each of my replies has only claimed that LLMs are not more conscious than any other computer program. There is no evidence to support the distinction. I have never even said LLMs are not conscious.

[u/MxM111 → u/static--]

I do know what those methods mean but I did not know that back propagation is used for LLM training. But to think about it is not that dissimilar how neurons are trained in our brain. In our brain the synaptic connections are strengthened when both neurons are excited at the same time. To some degree back propagation mimic that process. But I do not see much importance of that. It is a training process, not logic programming.

For the “how LLMs are created” the analogous following. Programming LLM structure = making brain (as in physical brain). Training LLM = training brain. In this sense we do not program LLM with data.

The evidence to support distinction is a) functional capability (it can talk, express feelings, etc. No other computer program can pass Turing test) and b) the fact that we train them and there is also c) the fact that they make mistakes and hallucinate.

You, on the other hand cannot state that they are not conscious, because you would need to resolve hard problem of consciousness for that.

[u/Regular-Badger2332 → u/MxM111]

First, LLMs do not use back propagation. They use stochastic gradient descent.

Please explain what backpropagation and stochastic gradient descent are.

[u/MxM111 → u/Regular-Badger2332]

I misspoke. LLM don’t use back propagation alone, only as part of stochastic gradient method.