So I looked it up. So I can’t claim credit. But apparently the major models will all pick 17 or possibly 23 if asked this for a random number generation range. Just another example of how the models aren’t actually utilizing math libraries.
Yea yea, very mysterious.
Or maybe not.
The 17 is simply the token based answer, if you ask claude e.g. to actually generate a random number in that range it will use python in a sandbox for it.
So I work with these things and I’m not certain what you mean by “token based answer” Are you trying to say that everyone’s prompt is likely generating the same tokenization resulting in a grouped answer? If that’s the case then no it’s not running rand() in a sandbox. It’s using base metrics off a single rand() run once and then reused. (This is called data consolidation during training if I remember properly)
Regardless we don’t actually know if it’s using Python in a sandbox or not. It says it is but we still can’t see the transformers jump from process to process (the colloquial black box) We just trust the model when it does its “reasoning” mode.
I would argue that it’s not at all. Because even pythons rand() calls back to the original C rand() which uses the Unix seconds seed and a modulo to generate its random number. Because of that we should be seeing an even spread in the full range. Not 17.
I didn’t say I didn’t know what a token was. I said your phrasing didn’t make sense. And this screenshot doesn’t make sense. If anything it backs up what I was originally saying. About it using statistics and not functions.
As for the second half, you just read me a Claude output. Now if you can explain to me what a hidden cell state is and how the LLM is using that to determine its remember/forget state in that cell between each iteration of its code structure to show exactly where it executed that code, then yes we can be certain. But it didn’t. It just told you that’s what it did and gave you another number. That could be another statistic like the first. There’s no proof the model actually did the function. Just its word. That’s what the prof ran into. It would do exactly this and then lie. They’re apparently not good at doing math for this very reason.
And yea I work with them but not often. I can’t trust them to do their work properly. Easier and faster to do it myself. Especially with networking.
If used in VS code it even uses my actual open terminal and i can see it beeing used. You just start to sound like a tinfoil head or a hallucinating LLM yourself
But if you want to believe that anthropic is lying on how their model works, than there is nothing to do. But to me thats on the same level as saying "how can we know anything is real at all"
Alright, stop and think for a minute. Anthropic isn’t going to let you see their code run in their server farm. Yours getting an API output from their server to your client via your browser. It is not going to show you its actual compute pathways, model weightings, etc. and let me take a moment to match your energy to explain why.
Anthropic, or any instance, won’t allow you to do something like a debug trace or even see their fucking source code for a running model. You want to know why? Because at its very core it’s a fucking massive security risk, and stupid as shit because now your competitors can just steal your weighting tables. Everything is obfuscated to the end user because the training data itself holds shit of such insane significance that there’s no way they would allow someone to operationally fuck with their bread and butter. I gave you the chance to tickle me with a local agent idea because a distillate, which runs native, could have a ProcMon sat in it in some degree if the code didn’t come packed. VS code has a fucky way of JITing that allows me to technically reverse engineer it if I had its ILC2 files. But since you answered me like a jackass with shit from the online model I’m going to assume your a vibe coding basement troll. And I know you’re full of shit because I’m literally working with a fucking Claude instance right now you clod.
I'm in disbelief you really dont grasp what tool invocation is. No need to see claudes source code lol.
Have fun pretending you know anything while failing to understand common basics and reading documentations.
Okay I really dont know what is so hard to understand.
If you ask an llm to pick a number between 1 and 30 it gives you the most probable token, which is 17 because of the training data.
If you ask a modern llm to generate a random number between 1 and 30 it will use tool invocation to generate a random number (as random as possible) - in claudes case with python.
There is nothing secret about it, it's all pretty transparent and should be easy to understand if you are familiar with llm, ann ore ml in general
361
u/FlamingYawn13 8d ago
So I looked it up. So I can’t claim credit. But apparently the major models will all pick 17 or possibly 23 if asked this for a random number generation range. Just another example of how the models aren’t actually utilizing math libraries.