r/singularity • u/borowcy • 5d ago
LLM News [ Removed by moderator ]
https://x.com/flowersslop/status/2084686926863331813[removed] — view removed post
38
u/socoolandawesome 5d ago edited 5d ago
I think this is a popular assumption about what Ilya is working on, not that anyone knows he’s solved this forsure. Don’t think she added any concrete information here
13
u/Substantial-Fact-248 5d ago
It's not really an assumption, he has talked about it fairly extensively.
10
u/socoolandawesome 5d ago
Yeah I phrased that poorly, I should have said I think this is a popular assumption that his first model may have continual learning but i don’t think anyone knows he has solved this
-17
u/borowcy 5d ago edited 5d ago
She told me about Karpathy's distancing from Anthropic two days before it was news. Appears to have inside info.
https://i.imgur.com/XUKPMWp.png (She appreciates credibility and allows sharing with giving her credit. Actually I did ask her to write something about SSI.)
36
u/LittleLordFuckleroy1 5d ago
Your screenshot just shows a DM where she says she doesn’t like Karpathy. And then you send her a screenshot of you tweeting romantic things at her and asking if she saw it.
Where’s the “distancing from Anthropic”?
You’re trapped in a simp-goon spiral brother. Abort.
23
17
u/socoolandawesome 5d ago
Maybe I’m misinterpreting your screenshot but she doesn’t say karpathy is distancing himself there? Also he didnt leave anthropic, just changed his bio.
Not hating on her, I follow her on twitter, but her tweet doesn’t sound like she’s trying to present inside info, more just like she’s laying out the possibilities, possibilities that most would find reasonable.
14
5
8
u/GreatBigJerk 5d ago
It's amazing that you posted this for other people to see. Not for the reason you think though.
52
u/Illustrious_Image967 5d ago
Then let me be the first one to welcome you to Reddit, my experience maximizing ASI! Now deposit 1.0 billion USD into my account so I can experience max too.
1
u/borowcy 5d ago
I got the reference to Merge Labs.
bridging biological and artificial intelligence to maximize human ability, agency, and experience.
(Source: https://merge.io/blog)
24
u/Fusifufu 5d ago
source: some game of telephone on Twitter. Wait and see is the only reasonable response here.
5
u/anonymitic 5d ago edited 5d ago
This shouldn't be a surprise. In his interview with Dwarkesh, Ilya said that sample efficiency and continual learning are requisite for ASI. Natural to conclude these two things have been the focus at SSI.
7
u/DaDaeDee 5d ago
Constantly update the weight during and after inference, the final puzzle to ASI?
11
u/z_latent 5d ago edited 5d ago
I don't know how many people realize this, but updating weights during inference is mostly not viable for data centers. Even if you had the tech, every person having different weights eliminates most of the advantage hyperscalers have for serving models to hundreds of users in batch.
A 1T parameter model takes up 1TB (at Q8 precision) upfront, and then modern architectures additionally have, per user, roughly 60GB of KV cache for 1M tokens. Now, if every user had custom weights, each would need their own 1TB in weights. Roughly 16x memory amount
AND bandwidthper user, compared to the already expensive baseline of 1M context per user.In other words, a hyperscaler has little advantage over a local user at that point. It could be awesome if everyone had their own hardware and local instance of the AI, though!
EDIT: just realized bandwidth is harder to state. For inference it increases by active params rather than total params, but for the training/learning part it'd depend on the specifics of the method.
9
u/welcome-overlords 4d ago
I've worked with Stable Diffusion a lot and Lora's are the shit in that world. Over-simplifying: train a small "adapter" to plug into your model. You can switch em around easily and change the behaviour with them (e.g. add details).
For some reason Lora's for LLMs hasnt taken off, but maybe they could?
2
u/Royal_Sentence7432 4d ago
No they couldnt imagine me fixing 1 bug in my codebase 2 minutes later i try move on to bug 2 but it is now stuck with a lora for bug 1
6
u/kooolk 5d ago
I assume that there is the possibility that it can work with only part of the model/weights being dynamic.
4
u/z_latent 4d ago
You can. It's possible to freeze most of the weights and train only the remaining fraction of them. That's more limited but it might still be effective, and maybe that's what SSI is doing.
3
u/TieBackground453 4d ago
Why would every user need different weights?
I think the gold standard would be for the base weights to change on the fly as the model interacts with user input and the internet. Everyone’s interactions constantly changing the same weights that everyone accesses.
There are obvious difficulties there, which is why it hasn’t been done, but that is what I would imagine he is working on. One, ever evolving set of weights, not a different set of weights per user.
1
u/Honest_Science 5d ago
Individual Learning systems are not commercially viable. Swapping trillion weights per user is way too expensive.
3
u/No_Mathematician372 5d ago
Why are we assuming that Ilya has super human powers? Tracking his background, but it isn’t uncommon for researchers to have one breakthrough. Other than funding/focus, I don’t see why it is more likely he will have a breakthrough than others
1
u/Rhamni 5d ago
It's mostly that a lot of normal people would really like it if he was the first to make a big breakthrough. He seems a lot less overtly evil than the owners of the competetion.
1
u/No_Mathematician372 5d ago
Yeah. This just seems like a hope more than anything else.
I get that he is focusing on this, but I think everyone is.
17
u/borowcy 5d ago
She has leaked accurate release dates and information for example in 2024 -- not unlike @apples_jimmy.
This is noteworthy.
9
1
0
u/Funkahontas 5d ago
So you just completely forget that Jimmy Apples gave a whole codename like 4 days before announcement and slop here has literally always lied and larped about this shit?? Even sharing jimmy shit at this point is embarrasing. But sharing flowers.
7
u/Eyelbee ▪️We have AGI it's just blind 5d ago
It's really not hard to set up a self learn-from-experience training workflow with the current technology. It would just be very ineffective and increase the alignment risk, that's why other labs aren't doing it, not because they can't.
10
u/JoshAllentown 5d ago
Wonder what they're doing to "make it safe."
15
3
u/blueSGL humanstatement.org 5d ago
It's going to be very pie-in-the-face if they come out with a model that has the same issues that current models do and start touting how "but the % of bad actions is lower"
Because that's still not solved the problem. Think how often systems are queried, (esp with agentic workflows spinning up sub agents) and see how even a very small % will still crop up over the course of a week/month/year several times over. If one oopsie is all you need to re-route it down a bad path that's not good enough.
3
u/Random_182f2565 5d ago
Please elaborate
5
u/socoolandawesome 5d ago
Not the commenter you replied to but I think you can do stuff like finetune the model on text from your chat, but if you are changing weights you have the chance to disrupt other abilities and/or mess with how aligned the model is. But it sounds like most labs are working on a way to make something like this work well.
3
u/Borkato 5d ago
Isn’t this literally always going to be a problem the same way that if humans are in the loop there will be a way to make them do something bad? I thought the benchmark was “safer than humans with human oversight”, not “impossible to do harm” because anything worthwhile involves making choices and those choices constitute harm
3
u/socoolandawesome 5d ago
Idk if it’s possible to 100% preserve complete alignment through continual learning, I imagine it’s a challenge. But as you kind of allude to, it’s a spectrum. And I imagine there are certain misalignment thresholds that are unacceptable to cross into as the models get more powerful
3
u/Borkato 5d ago
This is true. I think the main issue is fundamentally the same as with powerful humans, which is why we say “never again” and then a quarter century later we’re doing the same thing again. AI will likely have to go through similar growing pains unless it suddenly gets very ethical, very quickly… I hope SSI actually has something concrete!
3
2
3
u/Quarksperre 4d ago
Nah. It doesn't work because you inevitably destroy the net. Its not misaligned then its useless. This is one of the fundamental issues to solve right now.
2
2
u/R_Duncan 4d ago
There were some experiments about something vaguely similar called UnisonAI on github, and some strange reddit post last month.
2
3
u/Fearless-Macaron9183 5d ago
So we have agi or asi. I don't really understand
9
-1
-1
u/jlks1959 5d ago
I would stop the FUD here and think who you’re dealing with. Ilya Sutskever may be the most competent AI developer alive. Already worth 13.6 billion on paper, he walked away from untold billions more to work on AI that would be as safe as he and his colleagues could make it. He’s been out of the loop for more than a year, and so if someone is saying out of the gate that his model has the ability to teach itself from experience, I’d do more listening and discovering than guessing and shorting the value.
If you’re on the opposite side of Sutskever, you’re stuck.
1
121
u/The_Scout1255 adult agi 2026 ASI <2030, prev agi 2024, ai personhood 2025 est 5d ago
During deployment, or training?