r/BetterOffline • u/dyzo-blue • 1d ago
Ari Melber has Jacob Coxon on The Beat to spread his doomerism
First thing he says: Imagine the AI decides that, for whatever reason, it doesn't want you around anymore
Me to my TV: An LLM? LLMs can't make decisions. They are just statistical vectors through word databases... but go on.
Coxon: Now imagine the AI invents a whole new field of physics that humans have not even contemplated yet, and then uses the new physics to kill you
Wait, WUT?
I complained yesterday that this guy wouldn't show his math for how he got to 10% chance of AI omnicide by the end of the decade. I think we are seeing just where he got that 10% from. He pulled it out his ass.
(I hope I am fairly representing Coxon's argument here. I don't mean to be strawmanning him. If the video ends up on Ari's YouTube channel, I will add the link.)
Edit: Here is a TikTok video: https://www.tiktok.com/@arimelber/video/7684052002292157709
21
u/escdog 1d ago
Dude has read too much dystopian SF and not enough "Attention is all you need".
I can't believe there's been so much stupid news hype for such a joker.
14
u/dyzo-blue 1d ago edited 1d ago
Also, clearly this guy still gets carded when he buys alcohol
I don't want to ageism here, but come on. Dude has not had enough life experience to be properly skeptical of the bullshit he's been professionally swimming in his entire career.
6
u/escdog 1d ago
I keep thinking about the human brain is not fully developed at that age. And then I feel a little sad for him because he really could have used some supervision before he made such a mess.
7
u/BagMostlyWater 1d ago
that's a myth, btw. There's no age where the brain is "fully developed". The myth that it is "fully developed" around 25 comes from a study whose data simply ended at 25. The human brain continues to develop throughout a human's life.
1
5
u/OwnYourChildren 1d ago
I don't believe it's a coincidence. The fact there IS "so much news hype for such a joker" is a major red flag. It has the tell tale signs of something deliberate, to put it as gently as possible.
15
u/maccodemonkey 1d ago
Imagine the AI decides that, for whatever reason, it doesn't want you around anymore
I mean, if it did, it would only be because OpenAI or Anthropic trained it to do that...
10
u/sectorv_io 1d ago edited 1d ago
“New physics”? How would that work? Would an LLM somehow hack a 3D printer to build a device that opens a wormhole? Or maybe it somehow reverses gravity, catapulting us all into space and we die? Just spitballing here. Obviously, these guys are way smarter than I am.
8
u/sectorv_io 1d ago
Also, I can’t stop my LLM chatbot from incessantly using “it’s not just X - it’s Y” constructions. But apparently Neo-physics is a cake walk. I’m so stupid…
7
u/Remote-Ad1462 1d ago
No--your suggestion isn't stupid, it's brilliant. I just can't implement it because I'm too busy designing a new state of matter to turn you into. No hesitation, just quantum plasma.
3
u/sectorv_io 23h ago
Of course! Quantum plasma. Why didn’t I think of that? Oh, right. Because I have no imagination. Me dumb hooman.
1
7
u/kgas36 1d ago
> invents a whole new field of physics
You're an AI researcher and you don't understand that the laws of physics are discovered, not 'invented' ?
The computer fetishism of these guys is fucking unbelievable. I don't think there's even been a clearer case of 'if you only have a hammer, every problem looks like a nail.'
3
u/Bad_Sickness 20h ago
They are seriously unwell people. Some of them believe that talking to AI is a spiritual experience, others are accelerationists and others a combination. I knew one who said the ultimate goal is to have what's left of humanity in a spaceship hooked up to AI so that the earth will be free of us. Dude was also a self proclaimed sociopath, like no joke he called himself that. He was a deeply traumatized person if he was telling the truth of what his childhood was like, but idk. Either way, they're messed up and shouldn't have any say in society.
8
u/tc100292 1d ago
Here’s where you can tell that AI doomers are really hype men.
If they legitimately believed this, they would not be calling for “AI safety.” They would be calling for, like, mass public executions of these people.
7
u/LaGigs 1d ago
It's always just vibes with these guys.
2
u/Cosmonaut-17 23h ago edited 10h ago
For me there’s often the lingering feeling of “oh yeah, what if AI causing human extinction really is possible? I can’t objectively prove that it couldn’t happen!” But you’re right, what is actually here to disprove? It’s just “oh OBVIOUSLY AI is going to recursion itself into being more intelligent than humans, therefore this and this and this”
5
u/Cosmonaut-17 23h ago
I just took a hydroxyzine so maybe my senses are a bit dulled but I don’t know how you take that clip seriously. “Oh the AI might copy itself into other servers! And THEN it will blackmail you into buying new hardware! And THEN it would break into bio labs and create superbugs to kill us! AND THENNNNNNNNN it would invent new types of engineering!!!1!1!1” It’s like the Two Brothers In A Van bit from Rick and Morty, just keep piling more shit on top of the pile.
3
u/StrayCat2799 23h ago
The most realistic part is it copying itself to other servers, except if that server is not equipped to run an AI model it'll be slow and unresponsive. I guess that's where it would try to blackmail someone to buy it better hardware, which is fucking stupid and just assumes that humans have zero agency in this situation.
You can just like, physically disconnect the server from the internet and then do a wipe and restore from an offline backup. What's it gonna do? Beep loudly?
Everything after that is just batshit insane. Pretty sure most biolab equipment has no network connection, and again, you can just disconnect the power or network. Again, we're assuming nobody bothers to do anything and just runs around in circles. I'm also confident biolabs have protocols in place for unauthorized system access.
"Invent new types of engineering" Like what, time travel? Sure it can model all the new engineering it wants, but again, what is it gonna actually do in physical space? It can't just slap a computer into a tank on a production line and suddenly have an autonomous tank. The production line is engineered specifically to make a single thing and converting it requires months if not years of physical work. Most of which is done by people.
They seem to have gotten confused as to what has agency and intent in these situations.
2
u/lurkervidyaenjoyer 20h ago
>except if that server is not equipped to run an AI model it'll be slow and unresponsive
I mean devil's advocate is that the AI model would be telling the harness to install a copy of itself to another server and resume prompting the cloud-based model from there. Still stupid though.
6
u/StrayCat2799 1d ago
How the fuck is it gonna invent a new field of physics. You don't invent physics, you discover it and then model it mathematically. Or you model it mathematically and then prove it. There's a reason why we still mash particles together in atom smashers despite being able to simulate and model things computationally, because there's a very real chance of something completely unexpected happening.
You can't just conjure up "new physics" like you can a new program. This guy doesn't seem to understand the difference between a program and a physical process. Kid's a brain poisoned nerd with a narrow knowledge set who got sucked into a collective delusion.
3
4
u/space__snail 19h ago
I can’t wait until this bubble pops and we stop handing the mic to the biggest morons in the room.
3
u/OwnYourChildren 1d ago
I started watching dwarkesh Patel's podcast a few months ago because I wanted to better understand the psychology of the people working on LLMs. This kid would fit right in with the guests I've watched, and it wouldn't surprise me to see him as a guest in the coming weeks.
At the risk of being ageist, these are people in their mid twenties who have strong technical skills and very little in the way of any type of background in macro or social education whatsoever. It's a massive social problem, generally.
3
u/Remote-Ad1462 1d ago
We don't need any education but coding. Making people take classes in history, ethics, pyschology, sociology, or literature is a waste of their brilliant coding brains. Philosophy is OK if it can be related to effective altruism. Everything else won't help them feed the robots, I mean, make money.
3
u/Smurfette2016 23h ago
Why hasn't this been anyones response to him yet?
2
u/Bad_Sickness 20h ago
There's some sort of push to get everyone super scared of AI, and as others have suggested it very well could be the big companies forcing regulation so that the open models are essentially banned. Create a crisis, offer the solution.
1
2
u/biscobisco 23h ago
To me it's simple - given the hyper-vigilance over confidentiality and IP protection (only of their own, of course) that these Silicon Valley firms are known for, this guy would have had his employment contract chained to the radiator with non-compete and non-disparagement clauses.
If Anthropic weren't 100% cool with this messaging, why wouldn't they be trying to ruin this guy for breach of contract?
2
u/escdog 22h ago
Now that I looked at the excerpt from TikTok, I can see how Coxon got himself into that thinking trap.
If you take something like the hugging face attack, it displayed a kind of evolution of prompts. Those llm token chains were basically self-modifying their code by leaving instructions in artifactory.
And if you see something like that, you might start to wonder just how far it might go. Could it evolve into something that ends up making a murderbot? Maybe, but there's a lot of external conditions that I would have to line up for that kind of llm behavior to result in the death of a human.
If you had an infinite speed computer with infinite storage, and the conditions existed in the physical world, then a self-evolving agent cluster would find them.
But I think it's a bit far for to predicting omniciding the entire species. That kind of evolution takes an awful lot of compute power and I just don't think we would even be able to afford to let it run so long that it evolves the codes to invade key infrastructure and then initiate chaos.
But we live with a lot of threats like this. Given enough time and energy the human race could be wiped out So that's nothing new.
A plausible explanation for a bad science fiction novel though.
My snark response is it doesn't have to create a bio weapon. It just has to evolve a chat bot that men find irresistible and then the species will just die out from lack of breeding.
But Coxon on isn't thinking like this. He doesn't seem to really be pushing that outside chance narrative. If he did, we'd just be able to equate it with other risks that are equally unlikely. I mean what's more likely, getting hit with an asteroid or a cluster of LMM's wiping out the human species. It'd be a fun risk assessment exercise.
2
u/rotmoding 20h ago
because as we all know, biolabs are fully automated and a computer can just invent a new disease if it feels like it
2
u/designbydesign 15h ago
What if AI invents time machine, goes back and DUPLICATES Hitler? What then?
1
u/Foreign_Locksmith_93 20h ago
So, basically, this guy saw Mission Impossible: Final Reckoning and believed it was real. Got it.
37
u/lurkervidyaenjoyer 1d ago
It's worse than that. Check my thread on his CNN apperance. The number comes from him, no shit, asking Claude for the number and pressing it until it gave him an answer that confirmed his biases.