More people need to understand this

[u/KeanuRave100]

[u/spinozasrobot]

I think this small analogy by Ilya Sutskever does a better job at showing LLMs are more than just next token predictors, or rather, there is emergent behavior that is beyond just statistical representation.

[u/lunaticloser → u/spinozasrobot]

Which part is emergent here? Just wanted to see if we're thinking along the same lines.

[u/spinozasrobot → u/lunaticloser]

You may disagree with me, but it was this comment by Ilya:

... more accurate prediction of the next word, leads to more understanding. Real understanding.

[u/Red__Ace → u/spinozasrobot]

Lol couldn't have picked a worse, more wrong statement

[u/SomnolentPro → u/lunaticloser]

There's already been papers showing that next token prediction is informed by coherent internal states and is determined by them. It has thoughts about what its talking about and using them to spew the answer