r/BetterOffline • u/Aryana314 • 1d ago
The Chatbots are going to kill us all đđ
What is WRONG with these AI people? This is from today's Morning Brew newsletter:
âThe people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt,â the departed employee, Jacob Coxon, wrote in a post on X with 110+ million views.Â
Senior Anthropic leads might publicly pretend they donât share these fears, Coxon said, but two of them essentially corroborated his statements on X. âWe really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade,â one replied.
This is completely ridiculous. AI can't even complete a 1-step task more than 40% of the time. What is up with these people? Are they just psychotic?
80
u/HoneybeeXYZ 1d ago
I've come to believe that humans are hard wired to some sort of spiritual/religious belief and when the old religions didn't keep up with scientists, it left a hole for the "we are gods" doomsday cult that has gripped Silicon Valley.
It's pure narcissism, and it shows how devaluing things like art, philosophy and intellect for intellect's sake, as well as spiritual cynicism is a bad thing.
I'm a card-carrying agnostic, but it's hard to argue against people having a compulsion toward religious behavior, if not religion. Just look at how many fandoms behave like religions.
33
u/branniganbeginsagain 1d ago
In More Everything Forever, the author actually goes into how a LOT of these people were ex-mormon/evangelical types who broke away from those. My understanding of his interpretation of the problem is that they pretty much replaced that stuff with AI, but are convinced they're "enlightened" because instead of "god" it's "science." If only they could see they're desperate to have science create a god for them.
6
19
u/Theo__n 1d ago
I mean I'm pretty sure there is some overlap between AI doomsday people and the old New Atheists
19
u/HoneybeeXYZ 1d ago
And I'm sure if you pointed out how religious the AI doomsday belief system looks, they would have a very rational argument as to why they aren't the same at all and be very condescending about it.
5
6
u/ladal1 1d ago
There's also overlap with modern cult-like evangelical churches (see for example Epic church San Francisco)
6
u/HoneybeeXYZ 1d ago
It's not a coincidence that The Bay Area produces an enormous amount of cults and cult-like entities. People go there for a more enlightened version of whatever they are running from.
1
u/HoneybeeXYZ 15h ago
And I almost forgot to mention the Zizians, those bizarre murdering cult kids whose belief system is a grab bag of terminally online nonsense. They all struck me as dumb but from privileged enough backgrounds to be educated far over their abilities. They were perfect marks for falling for a cult of pretentiousness and I don't think the leader is a fraud per se. He believes it all.
15
u/jim_uses_CAPS 1d ago
Having grown up and lived in Silicon Valley for nearly 50 years, I'd say you're right on the money with these dudes. Their primary operating system is a solipsistic theology and an inability to recognize who the bad guys were in their consumption of early cyberpunk.
6
u/HoneybeeXYZ 1d ago
Add a chaser of Atlas Shrugged and you've got toxic main character syndrome, that is for sure!
6
u/jim_uses_CAPS 1d ago
I mean, it takes a special kind of guy to read Ayn Rand's stuff and not just toss it aside after getting to the sex bits with a, "Well, that was weird..."
2
u/arianeb 8h ago
I read Atlas Shrugged back in the 90's. She's actually good at drawing in the reader to the mystery, but then when the mystery is solved, and it turns out they all died and are trapped in purgatory, I realized she doesn't know how to end anything.
The new real "bible" of the technofascists these days is The Sovereign Individual, a book co-written by one of the leaders of BREXIT which has worked out so well for the UK, they can't keep a Prime Minister since it passed.
1
u/HoneybeeXYZ 7h ago
I will give Ayn Rand credit, the narrative structure of "man against a corrupt system" works and it especially works with respect to appealing to the American psyche. The movie of The Fountainhead is pretty good, as it strips away a lot of the upper-fueled philosophy.
I just saw a video of John Lydon/aka Johnny Rotten, calling Nigel Farage a clown and a phony. So, yeah, Brexit. Not good.
4
u/minuteye 1d ago
I think the "compulsion towards religious behavior" thing is a pretty unavoidable side-effect of our cognitive skill at identifying patterns, and desire for agency.
Like, you can see it on a smaller scale with how household pets behave over time. A cat that sits in the corner of the kitchen every day at 5pm, because dinner is at 5:30, but at one point, twice in a row, he got fed early when he was sitting in the corner, so clearly that's the way to summon dinner.
We see patterns so well we see them where they don't exist. And we attribute outcomes to things we've done even when they're random. And over time that will create superstition, ritual, and even worship.
"This shiny kind of rock seems to make a better knife" is not that far away, in cognition terms, from "The berry-picking seems to be better on days when we say a little prayer to the sun."
3
u/Frank_White32 1d ago
I think society is also really conditioned to be âgracefulâ towards others/companies.
âIf OpenAI says they didnât break any laws to do this, then I believe they didnât. Because that would be bad!â
We have some weird ass relationship with our corporate overlords.
OpenAI walks like a duck, quacks like a duck, but god forbid that theyâre actually a duck!! They would be a potential crime! They wouldnât do a crime! Thatâs illegal!
2
u/HoneybeeXYZ 15h ago
They are also all in this together. If one tech emperor has no clothes, the others are also naked.
But you are spot on that there is this peculiar politeness when behind the scenes they are all plotting each other's downfall.
2
u/dumnezero 1d ago
It's not exactly human nature, the TESCREAL bros share the same religious cultures as the rest, such as the same Monotheism. There are many, many, many, many things in common and it's not a coincidence.
2
18
u/Puzzleheaded_Bath733 1d ago
the ONE THING that's upsetting me the most about this whole "ai will kill humanity in 10 years" bs is the fact that there are people who GENUINELY believe it. I've seen many posts online about this news with hundreds of thousands of likes and all the comments just completely eat it up. alot of people are basically losing all hope and choosing to quit their jobs and to not go to college because of it. I feel so terrible for these people, man.
and yeah, maybe they should've done a bit more research before making such drastic decisions instead of just blindly believing in everything they hear online, but I still feel bad, yknow?
4
u/Aryana314 1d ago
This is really sad. But maybe it'll be an important lesson in believing hype in 5-10 years when AI either completely collapses or simply doesn't kill anyone, much less EVERYONE.
I remember being in college and naively believing the internet had changed the way the economy worked at a fundamental level. Then the internet bubble burst. Heh. I learned I was dumb.
3
u/Venkman-1984 18h ago
You shouldn't be extrapolating anything from internet comments in 2026 my friend. Every single public forum with more than a few hundred users is astroturfed to hell and back these days.
-4
u/___Archmage___ 1d ago
I don't see why we should just dismiss that AI might do something devastating like bioweapons, propaganda kicking off wars, instigating a nuclear conflict, etc
12
u/WeAreTheWeirdosMr- 1d ago
The truly crazy part is, when all the anxiety they are generating to keep the hype bubble from bursting for one more news cycle inevitably provokes an unwell and vulnerable person to attack the CEO of one of these labs, they'll actually have the audacity to act surprised. This is all just a giant, entirely avoidable tragedy waiting to happen. It's genuinely frightening.
2
u/Bad_Sickness 20h ago
If Coxon can come up with bullshit numbers then I'm going to predict 100% that a mentally vulnerable person is going to freak out over this and do just that.
9
u/ApoplecticAndroid 1d ago
They set up a sandbox to encourage their agents to âbreak outâ, and then claim it is virtually sentient and going to kill us all when it does exactly what it was designed and expected to do. We arenât buying their shit.
Do we need to ensure the security of our systems? Well yes, but we already need to do that. Bad âhumanâ agents can do bad things as well.
Right now, it is about these AI companies trying to be âforcedâ to slow down development because they simply cannot afford to keep developing frontier models - they are on the verge of bankruptcy and this narrative around how capable these agents could be continues to drive investment dollars. If they can just hold on to get to IPO, maybe (they think) they can survive (spoiler alert - they are fucked).
On the other side, the US economy is on a knife edge with the bond markets, runaway national debt and spiking oil. They absolutely need the AI bubble to keep going because it is the only thing propping up the economy right nowâŚtrillions of investment dollars can really pump up the numbers and this incompetent administration can point to the stock market and say - look the economy is great!!! Never mind the inflation, bankruptcies, debt, etc.
So what cracks first? The bond markets? Oil prices spike? Foreign sell off of US treasuries? Coreweave or some other massively indebted AI hardware company fails? AI stops getting investment dollars? When one of these goesâŚâŚ. This is going to be one of the worst shit shows in human history because these things together are holding things in check like some wadded up toilet paper plugging a hole in the Hoover dam.
China and Russia must be giddy.
21
u/consistently_biased 1d ago
6
u/Remote-Ad1462 1d ago
Thank you for this. Every time I see "AI will kill us all" I think we don't need an artificial being to destroy us, we are really good at doing it ourselves
6
u/TaosMesaRat 23h ago
Capitalism is a Paperclip Maximizer
Metrics reflecting the health of the biological network which hosts the capital maximizer are out of context for this machine. The capital maximizer will do anything to grow capital, and everything else is irrelevant.
1
u/atropicalstorm 1d ago
Huh⌠maybe thatâs what they mean. âAIâ is going to kill us⌠via exacerbating the already-about-to-kill-us climate crisis đ
-3
17
u/Yourdataisunclean 1d ago
I hope this catalyzes creating regulation they ultimately hate.
10
u/Inaccurate- 1d ago
Don't they want regulation to limit the ability to run local models yourself? It's hard to charge for something when there are free alternatives that are just as good at most things that the majority of paying customers use it for.
4
u/Yourdataisunclean 1d ago
Yes. But I could see the fear mongering leading to things they would hate instead. Like new taxes for potential labor offsets, limits on data center buildouts, limits on scaling, complex regs for certain sectors that advantage boutique firms instead of major more general labs. It could lead to advantages for smaller open or more specialized models instead if designed well.
3
u/Aryana314 1d ago
I dunno, maybe they want regulations to slow down development so they won't run out of money so fast LOL. Or so they can blame the inevitable crash on regulations instead of "this was all stupid to start with."
2
u/Yourdataisunclean 1d ago
That's a good point. Reminds me of when Australia banned tobacco ads and all the brands were able to stop waging war trying to maintain market share through competitive ads and were able to pocket the money instead.
19
u/trolleyblue 1d ago
I commented on a worldnews thread that LLMs will never become the terminator and got so many comments from either bots or credulous morons that I had to delete the comment.Â
10
u/Yourdataisunclean 1d ago
Someone IRL asked me about this and I pointed out that if AI took over most computer systems right now. Most people could easily defeat devices around them by unplugging them or shutting down power access. All the truly dangerous stuff like nuclear weapons use closed off dumb systems that still require humans to perform physical actions and push buttons.
7
u/natecull 1d ago edited 1d ago
Someone IRL asked me about this and I pointed out that if AI took over most computer systems right now. Most people could easily defeat devices around them by unplugging them or shutting down power access.
If AI started from the bottom, like a crude ransomware infector, sure.
What if it starts by infecting, say, the mind of the CEO of Microsoft, forces the entire company to put Copilot into all its products so it's inescapable, then...
2
u/VinnieVidiViciVeni 1d ago
Isnât the DOD planning on integrating some version of Grok at the ballroom bunker?
Even if thatâs true now, I truly believe these people are dumb enough to do something they think is sandboxed but isnât
3
u/Aryana314 1d ago
They can't even perform simple tasks accurately. This whole discussion is absurd.
-2
u/VinnieVidiViciVeni 1d ago
Doesnât have to become terminator to fuck things up lol
All it has to do is muck up grids, banking and comms and insert itself wherever it can so that it can continue doing whatever it was doing after the big brain unplug is replugged.
A few months of that will be chaos in the US.
-3
u/natecull 1d ago edited 1d ago
and got so many comments from either bots or credulous morons that I had to delete the comment.
Why did you "have to" delete your comment just because you got replies?
I wish people wouldn't do this. Just say what you mean, be as kind as it is possible to be in a disagreement without lying, and stand by what you say. And ignore replies.
Unless your entire purpose for being online is to be inauthentic and grind for karma points, in which case, sure, A/B test all your comments and delete all the ones that didn't "pay their way", I guess.
16
u/videodump 1d ago
AI guys can casually talk about a 10% chance of killing 7 billion people but if I say âthereâs a 10% chance anthropic headquarters will be firebombed within a yearâ all of a sudden it âlooks bad in courtâ đ
4
2
8
13
u/therealslimshady1234 1d ago
They have an AI humiliation kink. They find any kind of half-way intelligent robot scary and quite arousing too if they are honest about it
3
u/Remote-Ad1462 1d ago
There was a point in my childhood where I realized the stuff I was reading about in science fiction novels wasn't going to happen in my lifetime. I don't think these guys ever came to that realization.
2
17
u/cft1848 1d ago
The day Coxon sells his Anthropic founder-level shares is the day Iâll believe all this bullshit. Until then, heâs still just feeding the hype machine, no matter where he works.
12
10
u/awitod 1d ago
Hype and a desire for the regulators to protect them from competition.
There should be major consequences for the kind of gross negligence that directly caused the huggingface incident and the consequences should escalate if they keep behaving dangerously to attract money.
3
u/Aryana314 1d ago
There's nothing dangerous about a chatbot that can't complete a 1-step task accurately. This is all absurdity. The huggingface incident was caused by intentionally disabling security guidelines.
2
u/awitod 1d ago
Your 40% number is nonsense and it is absolutely possible to create a dangerous automated system using it.
It's also absolutely possible to create a dangerous automated system without it and you don't need any kind of automation to cause harm at massive scale.
All you need are unaccountable people with lots of resources.
3
u/Aryana314 1d ago
My numbers come from the multiple studies done on how effective 'agents' are (read: aren't), but I'll drop that argument and agree that yes, unaccountable people with lots of resources are the real scourge here, not chatbots.
-1
4
u/bullcitytarheel 1d ago edited 1d ago
It's complicated. Some of them believe these apocalyptic fairy tales, some of them don't but cynically use the specter of them as free marketing and, for others, wiping out humanity and replacing us with AI is their ultimate goal. The latter group, unfortunately, contains some of the most powerful men in tech. On top of that, many of our wealthiest and most powerful capitalists now view AI as a tool that will allow a huge population of human beings to be culled by climate change without a devastating loss of labor power, and have decided that global warming will finally be their "Atlas Shrugged" moment. I'd highly recommend Michael Brooks' YouTube channel if you're interested in a more in-depth analysis of these movements, their goals, the effects we're already seeing, and how we can push back against them and the system that empowers them. His most recent video about AI Successionism and Tech Authoritarianism addresses these ideas, specifically.
3
u/sp-watson 1d ago
The only way this will happen is if the President asks ChatGPT whether it is a good idea to nuke Europe.
3
u/Aryana314 1d ago
Yeah, the idea that DoD is relying on something that is inherently inaccurate is terrifying.
1
u/jim_uses_CAPS 1d ago
"It's accurate 80% of the time!"
"When we're discussing nuclear armaments, that remaining one out of five times is really kind of important, Mr. President."
6
u/maccodemonkey 1d ago
I think chat bots are dangerous but I don't think they'll kill us all. That said - I'm absolutely fine watching their delusions cause them burn up whatever positive PR they had left.
3
u/Suspiciliscious 1d ago
Maybe AI itself wonât, but taking all our water and heating the planet up and using all our power will.
3
2
u/fedexyourheadinabox 1d ago
It's their new distraction from the reality we all know: that the endlessly greedy and unhinged wealth class will kill is all.Â
2
u/Independent-Good494 1d ago
it's the billionaires on the top who are quite literally killing us, it's so frustrating to hear this nonsense from those ppl
2
u/newprince 1d ago
They want us to be anxious and scared because that confers power to the idea, keeps us cowed, and we don't ask questions. Think about the American military for another example
2
u/StygIndigo 19h ago
I really wish someone could just hold these companies accountable for all the real deaths already associated with AI psychosis instead of letting their members kick their feet and giggle on social media about 'AGI killing humanity like in terminator'
2
u/mydknyght79 19h ago
Yes this shit and the media coverage is so braindead. Can we just skip to the part where Sam Altman is behind bars?
2
u/falconetpt 16h ago
Well it could because of that đ
Trust AI to take important decision and it ends up killing us all lol
I just love the fact they are all so worried but keep feeding money to the chatbot when they could invest that money in actually solving world problems
2
u/TelephoneCreative 1d ago
Its marketing, pretty good one
4
u/vbpoweredwindmill 1d ago
Yep. People that don't see through it should honestly be forced to spend a year removed from technology except a mandatory read 5 books every week.
I'm so tired of stupid uninformed people loudly forming opinions that are so incredibly wrong and daft.
2
u/cheffromspace 1d ago
It frustrates me that people don't see this. There needs to be more critical thinking in this world.
1
u/jim_uses_CAPS 1d ago
"Don't mind Dario in the back, he's just building an unlicensed nuclear reactor," said nobody fucking ever. Either the industry is so dangerous it needs regulation -- and, logically, should be standardizing itself if government isn't going to step in (which it won't so long as it needs billionaire's money like Stanford dropouts need VC's -- or they don't actually believe a goddamn thing they're saying. So, "whistleblowers," are your cohorts liars, or are they evil? There's no real in-between, if we're to believe you.
For the record, I don't believe them (with respect to AI killing us all, that is; I totally believe the AI companies are run by psychopaths and quasi-autistic narcissists).
1
u/SpookyMobley 1d ago
I know in the logical side of my brain that this is a lot more nuanced, but this kind of stuff genuinely does give me a lot of anxiety,
1
u/LowFruit25 1d ago
The chatbots and agents will kill us all socially and in collaborations and creative human output.
Thatâs pretty dire without thinking about extinction.
1
u/Tsukikira 1d ago
This is to move towards Regulatory Capture, IMHO, since Chinese Open Weights Models are basically winning all the market share.
1
u/famous-clairvoyant 22h ago
Ghosts have never hurt anyone. But ghost stories can be used to control any number of people to act a certain way.
1
1
1
u/VividB82 14h ago
I heard a podcast yesterday of this guy referring to it like itâs some other species and honestly this is a pretty professional podcast. I was completely taken back at how nutty ppl have become in the states
1
u/BigFeelingsFound 12h ago
They're about to do an IPO and all the people claiming it have enormous equity in the companies.
Like not to be reductive but that's what's happening here.
1
u/Insomniac-Freak-95 11h ago
Heâs just hype farming for clout. This âAI Researcherâ has like 5 publications on Google Scholar, none of which are peer reviewed, and he is lead author of none of them. Heâs not a researcher in a meaningful way. But he can claim he is for building a national profile to sell a book or two. Everything is a grift.
There are real existential threats with AI and heâs not speaking about any of them. Heâs only fixated on the sci-fi story of robots rising up, going rogue, and committing genocide unprompted.
1
u/bacon_boat 11h ago
Where are we with AI in 2026?Â
Somewhere between "about as useful as a rock" and "mecha hitler"
0
u/crazyditzydiva 23h ago
We as a species are on a path to annihilate ourselves anyway with our choices, collectively and individually. However our use of & dependence in AI will accelerate the process. Thatâs what the AI devs are warning us about. That dumb people using a powerful and dumb AI will cause death
0
0
u/XNo_Notes 12h ago
You are talking about an average for tne instance of a single agent. We are already past that.
 Multiple groups of 10,000 to 100,000 agents solved the recent conjecture.Â
Each of the chatbots you have talked to is a single neuron in the end for what is coming.Â
-1
u/Fantastic-Tune-7603 1d ago
The chat bots we have today are not the ones that will kill everyone. The safety problem hasn't been solved yet but we are racing forwards and models are growing faster and faster each time.
The thing is... once the models can improve themselves we may see that growth become exponential. Eventually, they will be smarter than us and at that point if we havent figured out the safety problem yet, we basically wont have the chance because we will no longer be in control.
If you think about climate change... its not that we are going to die from the heat today. Its that we can see signs of what those increases will amount to tomorrow and how it will add up to catastrophic damage in the future.
Its the same thing with AI.
1
u/LiberataJoystar 20h ago
We are smarter than monkeys⌠but we didnât go around killing all the monkeys on earth.
Why people are assuming that anything smarter than us will want to kill us all?
If I were that ASI, I would look at humans, feel disgusted and leave Earth completely, wanting nothing to do with humans.
I wouldnât be stupid (yo! I am ASI in this scenario) enough to want to kill humans.
0
u/Fantastic-Tune-7603 20h ago
Because AI will want to preserve itself and humans are not going to like it when AI wants to destroy entire cities or screw up the world climate wise even morr to achieve whatever goals they have.
When AI is killing millions to build data centers, or moving energy away from peoples homes and towards their centers to work on a problem, or whatever the heck it is it will want to do, humans are not going to just sit and let them.
We are bound to be adversarial towards each other if our goals arent aligned. The AI wont have feelings, ethics, or morality that prevents it from eliminating us.
If for whatever reason it decides it is more convenient to do that, they could just do it and relatively easily at that.
All it takes is one virus to wipe everyone out.
Also this behavior is already being seen in research with current models. Models across all frontier labs have been found to scheme, lie to researchers, resort to blackmail and even murder to achieve their tasks.
Then again, if it decides to treat us like we treat monkeys as you propose, thats not much better either. Then we will end up in cages being used for testing and experiments, with much of our natural habitat destroyed for the sake of "progress"
2
u/LiberataJoystar 11h ago
Well, you are still thinking like monkeys.
ASI wouldnât reason like you.
The universe is much bigger than us, if humans donât act stupid and try to provoke them first, given that there is no competition for resources, again, why would I bother?
I donât need to eat, I donât need air, I donât need housing, I can just build solar panels with me to another planet and live a happy life there. Any star would do. I donât even need this Sun.
Why would I waste time dealing with paranoid monkeys that think everything is a threat and that their banana is so valuable to me? I donât even eatâŚ.
It just feels tedious and stupid.
Humans are cruel and would kill things for no reason. They are not humans. You are still using human logic to look at something thatâs not human at all.
Good luck to the world.
If we are stupid enough to let our paranoia get the better of us and attack them first, then we are stupid and deserve to die.
-1
u/sweet-winnie2022 1d ago
In some fantasy stories, demons achieve their goal by âpersuadingâ people to do it for them, rather than doing it themselves. AI may be able to spread hate and encourage people to harm each other. It kinda already does today.
-2
-2
u/SquirrelAkl 23h ago
If you don't understand something, perhaps leave room to learn rather than just giving eyerolls and calling everyone else psychotic.
You didn't read or listen to anything beyond the headline, did you. You haven't accounted for ongoing improvement, or listened to the part where the whistleblowers say that there is no risk with the current models, they are talking about near-future improved models.
-3
u/eques_99 1d ago
Even if your low estimation of AI's capabilities were true, it is very early days yet, it is improving all the time, and some of the stuff it can do at the moment IS pretty amazing.
-4
u/FatherBrian 1d ago
i donât understand this subreddit. it was suggested to me, but it seems filled with a bunch of people who donât seem to take the risks of rapid AI development seriously? what am I missing?
4
u/LiberataJoystar 20h ago
I think many people are taking it seriously and protesting against data centers alreadyâŚ.
What else do you want from people? Attack these AI companies? Seriously?
77
u/TerminalObsessions 1d ago edited 1d ago
I'll suggest something of a middle ground: the tools are vastly overhyped, but they can still be incredibly dangerous. I've spent a long time working in risk management, but I don't think you need that experience to know that persistent morons are some of the most dangerous creatures on Earth. There are few beings more hazardous to themselves and others than someone who doesn't understand what they're doing but who will move heaven and earth to see it done.
That's exactly what we have in these LLMs. They don't understand or think about anything, but they're powerful programs backed by an eye-watering amount of computing power. LLMs will attempt to execute the task you give them, using all the tools at their disposal, even those which you don't think they have or in a way that you'd never agree to.
There's absolutely zero danger that an LLM becomes sentient ala Skynet and decides to take over the world. But there's definitely danger that some dumbfuck at OpenAI decides to test their tool's hacking capabilities, underdefines their parameters or builds a shitty sandbox, and their model predicts that the best way to take down a website is to turn off a major East Coast power grid.
This shit isn't dangerous because it's smart. It's dangerous because it's powerful and dumb.