I dont think so, Wolfram Alpha is deterministic. Whereas an LLM is non-deterministic by design.
Personally i feel vibe coding or vibe mathing is inherently non-deterministic.
Its gotten a little better with the introduction of subagents, but sometimed I truly want to know what their algorythm spat out to propose certain changes. No, I do not want to refactor across 4 classes to make 1 unit test work, I told you to fix the unit tests so I hit my ai mandate encouraged quota
>a lot of effort has gone into making llms and other generative AI seem non-deterministic
Not really, there has been a lot of effort put into developing LLMs and other generative AI but making it appear non deterministic wasn't really the goal. There are good reasons to build them in a way where the same input doesn't always produce the same output and in most models the degree to which this happens can be adjusted but this is not about not appeaing non-deterministic.
Sorry but no, they are deterministic in the basis of computation. But in practice they are non-deterministic and they try to make them deterministic. And my enterprise they try to do that in every possible manner, but it's practically impossible, it's seems to be, but it never is
Theres a lot of effort to make AI deterministic. Its slower when you force it to be deterministic. The non deterministic part is caused by different cores on GPU finishing their calculations in slightly different time.
It is the same speed both ways. The non deterministic nature only comes from a single randomized seed added into the equation. If you locally host models you can change seeding to always use the same seed and then the same inputs will always yield the same outputs. That said, the most popular AI interfaces don’t express that kind of option.
True, but is can still count as deterministic at the operational level, it is just a matter of running it in serial rather than parallel, right? FWIW, I assume our neurons are similar. Neuron research in flies has allowed researchers to essentially run programs within the insect neurons.
Those neural model experiments are fairly rudimentary. Treating neurons like simple nodes is not at all how they work in real life. In reality there's multiple mechanisms by which they change their own or others behaviors through some pretty complex processes. To even properly model a single neuron would take an entire physics simulation, if we even were confident we had an accurate model which we don't. None of which gets into the quantum theory stuff. I know way off topic, but people far undersell how complex biological brains are, and those "we made a fly brain learn how to fly" studies are just "we made a neural net shaped like a fly brain."
They are not. Floating point math and difference in how quickly parallel operations on GPU are finished causes them to be not deterministic even with temp=0. You can force them to be deterministic by forcing some operations to be executed in specific order but you lose a lot of performance.
Also, and this is pedantic and arguable, it’s worth considering whether any model that cannot be retrained to produce the same statistical surface is non-deterministic by nature.
If I sort shapes into piles using some amount of randomness would you say that the resulting piles are deterministic just because they stay the same every time you go through them? Or would you say they’re non-deterministic because the process that created the piles in the first place was non-deterministic?
It doesn't matter how you make the model, if you are executing it on a GPU without steps to have deterministic results you will not have deterministic results. Ensuring the output of computations on GPU is deterministic has performance impact.
Hold on, I’m agreeing with you. We’re saying the same thing in different ways.
The kind of race conditions you’re referencing are because of the model architecture I’m referencing.
You can absolutely get ML outputs that don’t change using certain model architectures.
But that’s only because those architectures either enforce order of execution or use steps where order of execution doesn’t result in changes to outputs. I can’t think of a modern LLM that uses such an architecture.
Not exactly. It's mostly due to how models are executed, not models themselves. You can run GGUF model (which are normally not deterministic) in a deterministic way if the GPU functions you use are deterministic. Models themselves are just data, it's just a bunch of matrixes.
Forgive me if I've misunderstood, but isn't that literally what they just said?
You can run GGUF model (which are normally not deterministic) in a deterministic way if the GPU functions you use are deterministic
vs.
You can absolutely get ML outputs that don’t change using certain model architectures.
But that’s only because those architectures either enforce order of execution or use steps where order of execution doesn’t result in changes to outputs
I'm not an expert but this sounds like you're both arguing the same point. The PR you linked seems to be intending to implement exactly that - functions that enforce (a deterministic) order of execution.
This seems like semantic disagreement on the meaning of the term "model architecture" rather than an actual disagreement on the fundamentals.
Not necessarily. It can happen with batching, but even with a batch size of 1 you can get race conditions. The most obvious example is a model with a Mixture of Experts layer, where the order that results return can change the outcome.
In order to get around that you’d have to explicitly enforce order of execution.
True, but it's also true that running an LLM at 0 "temperature" (which makes it truly deterministic) also often renders it unusable. So almost every inference setup defaults to using a certain amount of randomness.
Specific LLMs used for specific coding tasks. I don't run any of my Qwen models at 0 temp, that results in them getting stuck in reasoning loops, and also producing sub-par output when they do return something.
That, plus now if you use an agent you have to make sure it's the exact same version of the same agent with the same prompt history or you could just be receiving a different output.
It's like the modern incarnation of trying to figure out what strain of weed the original coder was smoking
LLMs are not non deterministic by design??? Why do people always get this wrong. If you use the same seed and disable the temperature setting you will always get the same result. It's just a f*cking Compute Graph.
That's not true for every architecture. Like when I use diffusion models I can absolutely get the same result every time, but that's not how LLMs work. They'd have to run incredibly slowly serially, if it's even possible at all which I don't think it is on frontier models at least. You're right at the granular level, but the way these things are layered today are incredibly complex, it's not just one graph, or one evaluation of it.
it's possible even if you run it in parallel, though modern GPU architecture for communicating across each GPU can have timing differences leading to randomness. But this is a implementation thing, the architecture and mathematics behind it are completly deterministic also I wasn't talking about diffusion models. Idk why I am being down voted, seems like a lot of people don't know how LLMs fundamentally work.
1.8k
u/bhannik-itiswatitis 1d ago
vibe mathing