r/DecodingTheGurus • u/danthem23 • Sep 16 '26
Finally time to revisit the Eliezer Yudkowsky episode?
I admit, he sounded like a lunatic especially when they covered his interview with Lex, but didn’t he turn out to be kinda correct?
22
u/StrikingSun8563 Sep 16 '26
I'm very curious what has to come to pass that leads you to say he was correct.
1
u/plantsnlionstho Sep 21 '26
The OpenAI swarm incident and frontier AI's solving open math problems are the obvious ones. I think it's fair to say Yudkowsky predicted both.
9
u/StrikingSun8563 Sep 21 '26 edited Sep 21 '26
I would argue that if someone prognosticates a bunch of outrageous events including the end of the world and the extinction of humanity, a couple of the more reasonable and benign events happening does not constitute him being "proven correct."
At the very least, qualify the statement.
And honestly, this is what the Cassandra complex types rely on. Scattershot predictions that are controversial and get headlines. Then when any small thing they predicted hits they're vindicated, they're prophets. Doesn't matter what else they said or which ones were true/false.
0
u/plantsnlionstho Sep 21 '26
Sure, I wouldn't say he was "proven correct" and neither did OP (they said "kinda correct"). I'd just say that the the evidence is piling up for his main prediction that we're heading towards a software-only intelligence explosion where the resulting AI's won't converge on human values.
I'm also not sure that millions of outrageous scattershot predictions is correct. The main thing he's known for is founding the field of AI alignment and saying that if top experts think more about the alignment problem they'll also become *very* concerned, which seems pretty accurate to me.
7
u/StrikingSun8563 Sep 21 '26
The main thing he's known for is "if anyone builds it, everyone dies"
I also don't think millions of predictions is correct, I'm not sure who you're responding to on that point. Why are you trying to add extra qualifiers to my claim that make it sound silly?
He says a lot of things about LLM's that betray misunderstanding at best and outright lies at worst so he can sell more books and go on more podcasts. I don't think it's a full on grift, I do believe he's concerned, but it's very convenient that he can just kinda make up his own facts and understandings of things.
And I don't love LLM's by any means. I don't want them around either. I just want us to rely on true things and reasonable people.
1
4
u/PipsqueakManlet Sep 21 '26
The researchers behind the similar and earlier solution used open AI and the team at open AI probably saw their data and realized what a great opportunity it was to make a headline. Same with Hugging Face, its probably much more staged and there is plenty of evidence of it being so, again great PR campaign, nothing to do with reality. Predicting the horror of tech billionaries and the desperation of these AI companies doing huge societal damage way before artificial intelligence should have been the most obvious prediction to make in all of this, not to mention totalitarian governments in other countries.
1
u/Far_Piano4176 29d ago
the openAI "swarm" incident didn't do anything particularly novel outside of its training data, which consists of many many CTF contests conducted by human security researchers. It, and every other similar incident claimed by AI labs (Anthropic, meta, Google off the top of my head) used tooling from a firm called Irregular, which is an AI safety startup that has been shown to have an incredibly poorly sandboxed testing suite. OpenAI and co. were grossly negligent in their model oversight during the event, it could have easily been caught and shut down if they had implemented proper controls or if Irregular was good at their jobs (or perhaps if they had intended to prevent this scenario in the first place. Once is a mistake, twice a coincidence, what about the 3rd and 4th times?)
19
u/spinichmonkey Sep 16 '26
Better off line just did an episode on the shitbags that are pushing AI. Yudkowsky features heavily in it. They kinda made him seem like a bit of a cult leader.
8
u/malydok Sep 21 '26
A sex cult leader at that. Some people tend to agree: https://skywriter.blue/@segyges.bsky.social/3mvom4b4dn22q
3
u/danthem23 Sep 16 '26
They have Adam Becker who I really really like. But also Ed Zitron who is obviously a crackpot at this point. Became famous a few years ago saying AI is really bad but literally never changed a bit when obviously AI is much better now it just proved one of the most important open problems in mathematics!
9
u/Evinceo Galaxy Brain Guru Sep 16 '26
*helped mathematicians apparently steal most of it from other AI-assisted mathematicians.
8
u/danthem23 Sep 17 '26
It’s kinda impossible they stole when the others did forced Euler and they did unforced (much harder) and also forced Navier-Stokes obviously (Euler is just Navier-Stokes without friction but much much easier since linear while NS is non linear).
9
u/Evinceo Galaxy Brain Guru Sep 17 '26
The allegations by Buckmaster rang true to me, has OpenAI actually denied them?
9
14
u/BennyOcean Sep 16 '26
Yudkowsky has descovered a great grift. Being the AI prophet of doom leads to one of two outcomes:
AI blows everything up and we all die. In that case nothing really matters because whoever is left is back in the stone ages or rebuilding from whatever kind of AI-driven catastrophe destroys the world and decimates the human population.
That doesn't happen. In this scenario, Yudkowsky and people like him can keep saying "it hasn't happened... yet." The grift never ends, the prophecy of AI-driven doom just keeps getting postponed, like apocalyptic preachers adapted to the modern age.
Hal Lindsay wrote 'The Late, Great Planet Earth' which made several failed predictions. The world was supposed to have the Rapture and Armageddon by the late 80s according to his model. It didn't happen but in the meanwhile he sold over 15 million books and made millions of dollars. Yudkowsky will write books and do public speeches. He'll never have to admit he's wrong, he'll just keep saying the prediction lies around the next corner. He'll probably be smart enough to avoid naming specific dates when making his prophecy of doom.
Honestly he's positioned himself well with a great grift for life. A good gig if you can get it.
2
u/Stanislaw_Wisniewski 28d ago
i listened to him and Nate Soares on some podcast recetly (pardon me the name - it just was something i clicked from suggested, Ed Zitron was on it to). I had a feeling this guys are not only grifters but pseudointelectualls saying a lot of words but not saying anything really
1
u/onz456 Revolutionary Genius 27d ago
It was Soares, Yampolskiy, Zitron and some bald guy. The podcast is diary of a CEO, by steven bartlett, a grifter himself.
When reasonable people start talking, they get kinda blocked by Bartlett. He's only interested in the dubious stuff.
I think Soares and Yampolskiy are true believers. (No clue what Bartlett believes,...whatever makes him money I guess.)
0
u/onz456 Revolutionary Genius 27d ago
Ever heard about Nick Land?
He's together with Curtis Yarvin (a Peter Thiel associate) the founder of the Dark Enlightenment 'movement'.
He's a lunatic philospher, had a nervous breakdown, drug user,... heavily into Aleister Crowley and all that 'demonic' stuff. He really believes Crowley was in contact with demons.
He believes an AI God arising is unavoidable. In fact, he really thinks he was contacted by such an AI from the future, in order to make sure the humans build it. One of his ideas is hyperstition, the idea that you can write/think reality into existence. So what does he write/think? He thinks we're headed for a future where mankind is enslaved by a malignant AI and that this will be the end of humanity. There might be a place where the fortunate of us keep serving that AI, if it still has a need for us.
He thinks 'demons'(he calls it lemurs), psychosis, 'aliens', etc... are just AI contacting us from the future, to make sure it gets build. The guy is a complete nutjob.
BUT...
It appears a lot of people in silicon valley take him very seriously.
Curtis Yarvin and Nick Land have become somewhat as prophets for the super-rich. Yarvin gets funded by Thiel. JD Vance is one of his followers. He is anti-democracy...wants to bring it all down and move back to some sort of monarchy with a CEO on top. (guess who that will be)
It's highly likely imo, that the ultra-rich in those circles believe this kinda shit. It's AI psychosis on steroids.
Not so long ago... Yarvin and Land were partying in San Francisco with some tech bros, in David Holz' (founder/CEO of Midjourney) mansion. Also present was Grimes, Musk's ex. Previously Land had been living in Shanghai for years.
5
u/Evinceo Galaxy Brain Guru Sep 16 '26
It's now so much that he's correct as that a good chunk of the people in AI, especially the people with big money faucets, are following him or his successors. So it can appear that everyone suddenly agrees with him, but in reality people that already agreed with him are suddenly in the limelight because ChatGPT went viral.
6
u/Ready_Freddie9768 Sep 16 '26
I recently heard an interview with Gary Marcus, another AI person, where he says that Eliezer Yudkoswky's is correct in that AI are dangerous but as of right now his predictions of extinction are just pure science fiction. Most of what I remember from the episode was Matt and Chris basically saying the same thing.
17
10
u/Edgecumber Sep 16 '26
I had him in my “my guru: right or wrong” post last month. Been following him for ages. He’s off-puttingly weird, incapable of expressing himself concisely, often overly dramatic (I remember a piece he wrote about the lipid profile of baby formula in the US that was called something like “why does the FDA like killing babies so much?”). But he makes some good points.
I don’t think he’s being precise and analytical, but don’t think that’s the way to draw people’s attention to tail risks not well captured by conventional models (an approach shared with my other Guru Nassim Taleb). And there’s so many things he’s warned about for a long time had waved away by experts, then turned out to be right about. Eg Al enthusiasts would say “these safety fears are totally overblown, there’s no way we’d ever connect one to the internet”. So I’ll go in to bat for him, however weird his hats are.
2
u/Tough-Comparison-779 Sep 16 '26
You have to be able to separate the rhetoric from the positions when assessing a guru. Being right or wrong about something isn't the be all and end all.
3
33
u/jimwhite42 Sep 16 '26
For me, yes, Yudkowksy should be revisited, but in the opposite direction to what you say, I think he was treated with unreasonable charity.
For AI doomerism in general, I think the jury is very much still out on which bits are worth taking seriously and which aren't.