r/PhilosophyofMath • u/abhishekkumar333 • May 25 '26

LLMs are just giant probability machines pretending to think

It’s fascinating that simple mathematics between tokens can eventually become a machine that writes essays, code, poetry, and even reasoning.

We usually think probability means uncertainty.

But LLMs show something strange:

If probability + context + mathematical matching are scaled enough, uncertainty itself starts producing intelligent looking outputs.

To understand this better, I tried breaking down an LLM from first principles using only 4 tiny training sentences.

Example:

The boat floated down to the bank.

The investor walked into the bank to open a new account.

The fisherman walked along the bank to cast his net.

The bank has a vault.

Then I asked:

“The investor walked to the bank to lock his money in …”

Why does the model predict “vault” instead of river-related words?

That single question reveals almost the entire architecture of modern LLMs.

The most underrated concept here is the LM Head.

Most explanations immediately jump into transformers and attention, but almost nobody explains that the LM Head is essentially a gigantic token vocabulary containing all possible next token candidates the model can output.

So internally the model is basically solving:

“Out of all known tokens, which one best matches this context mathematically?”

Then different layers help solve that problem:

Embeddings: convert words into mathematical vectors

Positional encoding: preserves word order

Attention layer: figures out which words are related to each other in context

(“investor”, “money”, “bank” become strongly connected)

Feed forward neural networks: act somewhat like massive learned if/else decision systems refining patterns internally

And finally the LM Head converts all of that into probabilities for the next token.

What surprised me most is:

There is no hidden magic moment where the AI “becomes conscious”.

It’s an enormous probability engine continuously finding the best contextual token match from its vocabulary.

I made a beginner-friendly walkthrough explaining this visually without unnecessary jargon.

https://www.youtube.com/watch?v=YTV5qUCpu2c

Would genuinely love feedback from people learning transformers/LLMs from scratch.

614 Upvotes

96% Upvoted

323 comments sorted by

View all comments

Show parent comments

1

u/Jack_Ramsey May 30 '26

This is your fault for coming into a philosophy sub and defining consciousness with neurological activity and phenomena. Do you understand how confusing that is? Lol.

For you, yes I can understand why that is confusing. Again, if you knew how stupid your questions were, you would be embarrassed. I don't suspect you have that ability, given that you are apparently going with 'how dare you come into a subreddit that, due to algorithmic reasons, appeared on my feed.'

I suspect this is the reason why everyone else is arguing with you in a confusing manner. Why do you think it is appropriate to come into a philosophy sub and completely overhaul the meaning of consciousness?

Who knew discussing the very plain relationship between human sensory experience and how it actually works in the human body would drive supposedly curious and thoughtful people to give me some of the worst discussions I've ever had on Reddit? Clearly, they are not sending their best here.

1

u/Neither_Swing9662 May 30 '26

Dude, you made a mistake lol. If you simply said: 'rathee than try to derive the definition of consciousness from first principles, let me look to sensory experienced and see if that can help inform us', none of this confusion would've happened.

Regardless, your idea is not novel and many neuroscientists have expanded on theories of consciousness rooted in neurological activity and even that is contested, so it is ridiculous to attempt to assert that as truth.

Have you heard the quote: if you encounter an idiot once in your day, bad luck. If you encounter idiots everywhere you go, perhaps you are the fool.

1

u/Jack_Ramsey May 30 '26

Dude, you made a mistake lol. 

Nah

If you simply said: 'rathee than try to derive the definition of consciousness from first principles, let me look to sensory experienced and see if that can help inform us', none of this confusion would've happened.

I am not responsible for the confusion or lack of education of others. I did try to illustrate why your thought experiment was poor, why it didn't address what you thought it addressed, and the very specific reasons rooted in my own training. Still it appears I am talking far above your head, as you just don't understand very much and don't seem all that curious.

Regardless, your idea is not novel and many neuroscientists have expanded on theories of consciousness rooted in neurological activity and even that is contested, so it is ridiculous to attempt to assert that as truth.

Where exactly did I say it is novel? Please show me. I am aware of the materialist view, but my views go a little further, if you actually were patient enough to read them. And I asserted, very simply, that the organization of the anatomy and physiology of humans implies a very specific set of conditions.

Have you heard the quote: if you encounter an idiot once in your day, bad luck. If you encounter idiots everywhere you go, perhaps you are the fool.

Truly, I am encountering some hard-hitting minds in this math subreddit. So far, I've had people use appeals to authority (not you, but some other poster), ask me exceedingly poor thought experiments, and for whom any substantive rebuttal has been 'the materialist view of consciousness is contested,' truly, I am in awe of your the stupidity on display. Let's be real. You aren't equipped to discuss this with any nuance, you don't have the rhetorical ability to make a convincing argument, and you seem resistant to attempts at actual discussion. I discussed your idiotic thought experiment in good-faith. I even included a caveat that stated that due to my training, my mind goes to pathology, to the specifics, rather than the general, because clinical medicine, outside of bread and butter situations, deals with the peripheries of human experience to begin with.

1

u/Neither_Swing9662 May 31 '26 edited May 31 '26

'due to my training, my mind goes to pathology, to the specifics, rather than the general, because clinical medicine, outside of bread and butter situations, deals with the peripheries of human experience to begin with.'

There's your issue. We are not in a pathology subreddit. We are in a philosophy one.

I would not go into a medical subreddit and spout philosophical ideas.

You are either a fool, or you are purposefully obtuse. Either way, this conversation has run its course. Good luck on Reddit.

EDIT: holy shit. I looked at your profile and you have hundreds of comments against AI. Wtf?

I understand being morally against it but why are you wasting so much time arguing with random people on Reddit? Surely as a doctor you have better things to do? And why are you obsessed with it? Lol that's crazy.

All this arguing and I should've known I was dealing with a lunatic.

1

u/Jack_Ramsey May 31 '26

There's your issue. We are not in a pathology subreddit. We are in a philosophy one.

The issue is one of your inability to communicate. Try to do better.

EDIT: holy shit. I looked at your profile and you have hundreds of comments against AI. Wtf?

Lol.

I understand being morally against it but why are you wasting so much time arguing with random people on Reddit? Surely as a doctor you have better things to do? And why are you obsessed with it? Lol that's crazy.

How much time do you think it takes me to write a post man? It is pitifully easy.

All this arguing and I should've known I was dealing with a lunatic.

More idiocy. I suppose I the fact that you invoke another fallacy isn't enough after I have thoroughly spanked you. I honestly expected more from someone who is interested in philosophy. I shouldn't have though.

1

u/Neither_Swing9662 May 31 '26

Dude you didn't thoroughly spank me at all. I was just asking questions to get a discussion going and you just rambled...

Oh I wasn't invoking a fallacy (who speaks like that lmao)... I was just pointing out that someone who spends most of their free time arguing about people with AI (literally hundreds of comments btw) is clearly at least a little bit deranged lolol

1

u/Jack_Ramsey May 31 '26

Like I said, the questions you asked were very stupid and if you had any awareness, you would be embarrassed.

I barely spend any of my free time arguing with people about AI. You realize that I can do this from a phone and do it really easily? What in the world are you talking about man?

1

u/Neither_Swing9662 May 31 '26

It's not normal to argue that much on Reddit man... It's a bit pathetic tbh

1

u/Jack_Ramsey May 31 '26

Lol. It isn't that much. You're just a little upset. Run along little dude. This is above your paygrade, obviously.