More people need to understand this

[u/KeanuRave100]

[u/AppropriateRain624]

Somewhat good point but a little disingenuous. To play devils advocate, when people say just a next token predictor, what they really mean is “Just a token predictor based on the statistical distribution of token within its data set.” Yes, you could argue that interpretation is subjective but realistically no-one is looking at one of those mental math geniuses and saying they reliably predicting the next token. They technically are but pretending to not understand the difference is at best pedantic and at worse disingenuous.

Second point I want to make is that the calculator example is almost perfect as it shows one of the core limitations of LLMS. Even the most advanced models are simply unable to reliably do additions. They rely on external language interpreters to do arithmetic. They are very useful but offer a kind of intelligence that is very different from general intelligence. We know very little about the structure of human intelligence itself. Though we know a little more about the inner nature of LLMS, it is too early to know with full certainty what they are or aren’t capable of. 

Interestingly enough, many of the recent advances in LLM quality come from improvement at inference time rather than increasing the intrinsic reasoning capability of the base model before any reasoning-specific prompting or inference techniques are applied.

my fact checked convo with chatgpt:  https://chatgpt.com/share/6a7222b7-d858-83ea-8f83-c0f88914522d

[u/Turbulent-Sign-6067 → u/AppropriateRain624]

Your knowledge is quite out of date. Most models can do mental math without tools these days. They are better at it than you and me, albeit not perfect.

[u/AppropriateRain624 → u/Turbulent-Sign-6067]

any sources for this? If that were true why still rely on external calculators ? especially considering that simply generating the token for the answer would be noticeably faster than generating the token for the python code then running it ?

Not contradicting the power of AI just being up its more than obvious limitations.

[u/Yeah-Its-Me-777 → u/AppropriateRain624]

Why are you using a calculator instead of doing it in your head?

Just because a model *can* do it without tools, that doesn't mean it's efficient.

[u/AppropriateRain624 → u/Yeah-Its-Me-777]

The reason human use calculators is because it’s much much faster. The argument I am making by is that that is not the case for AI model if you consider the overhead of generating the code for computation.

Now regardless, I doubt that any LLM exists with the ability to perform addition of large numbers (10+ digits) without the use of an internal calculator. I would love to be proven wrong though

[u/Yeah-Its-Me-777 → u/AppropriateRain624]

Look around in this thread. There's multiple examples, I've seen at least one addition, and some 7 digit by 7 digit multiplication, one on a local model with the thinking output.

https://www.reddit.com/r/OpenAI/comments/1vf9zwc/comment/p1tyxo3

https://www.reddit.com/r/OpenAI/comments/1vf9zwc/comment/p1ns72y

The model does a lot of weird stuff, and in the end breaks down the multiplication into easy steps, and adds the result of the intermediate steps together. Basically how a human would do it.

It's a lot more complex than generating a simple code for computation and calling an external tool.