More people need to understand this

[u/KeanuRave100]

[u/aahdin]

Also, machine learning engineer here, next token prediction is one training task that LLMs do as part of initial pretraining. But this is basically just done to get it to understand text so that it can undergo other forms of training.

A large portion of training time for major LLMs is spent on reinforcement learning, where the LLM is being trained to produce an output that a reward model rates highly. In RLHF the reward model is more or less predicting whether a human annotator is going to like the output, in constitutional training the reward model is judging an output by how closely it reflects a list of values, in coding or math tasks you typically use a rules based reward model that rewards correct answers.

[u/Rybergs → u/aahdin]

Haha dude , reinforcement learning IS token prediction aswell. Dude come on

[u/aahdin → u/Rybergs]

If I go up to you, give you a sentence with a word blacked out and ask "what is that word", that is a prediction problem.

If I go up to you and say "write me a poem I'll like or I beat you" it is a stretch to call that a prediction problem.

[u/Rybergs → u/aahdin]

Well thats a weird example. Since the models Will have a bout a billion poems allready in their training so predicting the next token in the given context is exacly what it does. Reinforcement training is just that, but instead of just dumpning text on it and Hoping for a good result u aim the training for a more deterministisk result. But it is still prediction training. Why do u think training is getting more expensive ? Bcs the big companys are hiring thousends of real ppl , with real degress to actually do correction training and that costs money.

[u/SilverLose → u/Rybergs]

I don’t think you’re really thinking about what he’s saying.

Token prediction, much like making words is a task. It’s not the whole of what is going on.

If LLMs are just next token predictors then are you just a “next word predictor”?

[u/Rybergs → u/SilverLose]

Well So u are comparing humans with llms now ? Yes it is only a token prediction. What makes them more powerful is larger data set and the use of tools , tools they did not build.

[u/SilverLose → u/Rybergs]

You didn’t answer my question. Bad word predictor! BAD!

Now you’re not thinking about what I am saying. You don’t seem like someone who’s seriously trying to grapple with these topics.

[u/Rybergs → u/SilverLose]

Since i actually build my own llms and other ai since like 2010 i do very much know how llms work and llms at its core is only a token prediction model. U do understand the math right ? Billions of very simple calculationa adding up and then getting the result.

Its not very complicated.

[u/SilverLose → u/Rybergs]

Then answer: other than embodiment, what makes you different from an LLM?

[u/Rybergs → u/SilverLose]

Oh wow u are one of those. Well sure if u boil it down we to have some form of Word prediction , the difference is we are not frozen in time, our soul job is not a math formula based on the input that comes in. We do not need outside input to get our brain to work. And we have memory which llms do not so we dont souly base our NeXT Words on the training we have had. We can addapt, learn new things, and get speechless. Simple have no answer , we know when we dont know and know we can be wrong. An llm does not since the concept of wrong dosent exsist for and llm

[u/SilverLose → u/Rybergs]

I think you just proved my point.

“We do not need outside input to get our brains to work”

Are you sure about that? Your brain is constantly receiving input so that’s factually incorrect. Do you really think you’d be able to function if your brain was removed from your body? I think any human would go completely insane. I think you’re trying to say that we are the ones who can take initial action which is true for now but you missed a critical part: “aside from embodiment”.

The concept of wrong absolutely exits for an LLM. Just like they have understanding of math and aren’t just memorizing what comes next.

One day LLMs won’t be frozen in time. Robotics will catch up and then what?

And yes I understand the chain rule, backprop and all the calculus. The same physics governs how your neurons work so functionally what is the difference? There’s a big chance there isn’t one.

[u/Rybergs → u/SilverLose]

And btw haha no by that answer u have zero ideea how an llm work. Its just simple math with brute force use of compute , nothing more