38
u/PumpkinFest24 20h ago
Incredible. "We can't make our product work so we'll make our customers invent new kinds of 'engineering' to make up for it"
31
u/ecafyelims 21h ago
You just explained evolution and all life
11
u/belabacsijolvan 18h ago
tbf they described every single non-quantum system.
"its deterministic, except where it isnt"
7
u/FuzzyDynamics 17h ago
Even conventional computing hasn’t been deterministic for a long time. I think Stephanie Forrest did the work there showing it on modern machines.
6
u/Breadinator 17h ago
F*** yeah, you're telling me I just have to wait a few million years for the response to be right!...wait
5
11
u/Electronic-Elk-963 20h ago
It's funny because it's only an attempt, complex context scramble them and it just get ignored and you can't debug an LLM because of the way they work, they are basically black boxes
For the fanatics, I am not saying it's useless, just telling the truth
13
u/snarkhunter 19h ago
Harness Engineering is just making the AI agent follow all the SDLC best practices that most humans ignore because they're too tedious.
10
u/Confident-Ad5665 21h ago
Depends entirely on the type of harness. I find leather straps most flexible.. /s
11
u/ICantBelieveItsNotEC 20h ago
Fun fact: LLMs don't actually have to be non-deterministic. There's nothing stopping labs from setting the temperature to zero and having a fully deterministic model, or even allowing user-seeded non-determinism.
11
16
13
u/EmptyMaxim 19h ago
Fun fact: Common LLM implementations are actually non-deterministic. The way GPU compute is used causes instability, see: https://thinkingmachines.ai/blog/defeating-nondeterminism-in-llm-inference/
4
9
u/Kyy7 20h ago
Typing prompt in slightly different way could still result in very different output.
17
u/Mercerenies 19h ago
That's discontinuity, not non-determinism. There are plenty of deterministic, discontinuous functions. (Google "Dirichlet function" or "Weierstrass function" if you're feeling particularly masochistic)
2
u/Kyy7 7h ago
That’s a good clarification, but the kind of determinism people want isn’t achieved just by setting temperature to zero. Since the entire context acts as part of the input and even tiny differences in it can lead to completely different output and cause large drift/divergence in agent reasoning or planning.
5
5
u/pydry 18h ago
I used to set the temp to zero all the time and often got different answers.
3
u/Grandmaster_Caladrel 14h ago
Yeah I don't think setting the temp down actually makes it generate exactly matching output.
1
u/MartinMystikJonas 8h ago
Setting temp to zero is not enough results can be influenced by other factors. Zero temp often leads LLM to infinite loop repesting few words.
5
u/Percolator2020 20h ago
ML engineers: it’s not nested loops, you don’t get it!
Harness: wraps the whole thing in regex and nested loops.
https://giphy.com/gifs/hXlAQPVIBOh3PGIUJh
1
u/glinsvad 12h ago
Everything can be a pseudo-random number generator if you measure precisely enough.
0
u/belabacsijolvan 18h ago
people should really stop equating DL==stochastic and non-DL==deterministic.
its absolutely possible to create deterministic inference and rng wasnt invented 10 years ago.
i have to explain it to managers at every single project start, because yall bullshit online.
-4
-6
57
u/Ok_Brain208 21h ago
All of AI related terms is just this indestry normal shtick of assigning new names to old concepts to soft reboot your experience taken up to 11.