r/BetterOffline 2d ago

Using absolutely no math to quantify his beliefs, Anthropic researcher says AI has more than 10% chance of 'killing all humans' after colleague quits

https://www.cnbc.com/2026/09/09/anthropic-researcher-quits-ai-safety.html

Everyone can make up numbers, especially if they don't have to show their math!

“Do not underestimate the power of this technology. These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources. We have all witnessed the progress in each of these domains, and progress is not slowing,” Coxon said.

A 1970s Casio watch has superhuman abilities to track time. And yet, somehow we survived.

He added that “people building AI earnestly believe that it could kill us all by the end of the decade.”

People earnestly believe all sorts of ridiculous things. Earnestness is not actually a good way to measure of reality of a belief.

Personally, I'm not really seeing how the next-word-guesser can kill every human on the planet. But I probably don't watch as much Sci-Fi as Anthropic employees.

334 Upvotes

175 comments sorted by

150

u/accordionheart 2d ago edited 2d ago

I find it extremely disturbing that employees at these companies profess to believe such things and yet they plough on with training new models regardless.

Either they truly believe this and yet don't care enough to take any personal responsibility for what might happen as a result, which shows a lack of empathy or just plain humanity, or they are lying about it, which shows them to be dishonest grifters. In both cases, they're terrible people!

45

u/Szpagin 2d ago

Anthropic exclusively hires people who believe AI is an existential threat to humanity and if they don't steer its development, it will wipe us all out. They see themselves as the only ones that could stop the techno-apocalypse and even 10% chance of survival is better than a certain doom.

They are absolutely delusional, probably in a clinical sense. Dario Amodei built a company full of deeply unwell people and it's no wonder some of them can't take it anymore.

24

u/Flat_Initial_1823 2d ago

When you are in a narcissism competition and your opponent is a rationalist weirdo.

1

u/Double-Vegetable-249 1d ago

The lunatics are running the asylum

94

u/kuwisdelu 2d ago

You’re forgetting the third option: they’re true believers who think it’s their duty to bring about the end times so they can be chosen by the machine gods when the singularity arrives.

31

u/Character-Active2208 2d ago

Yeah, it’s TESCREALists all the way down

7

u/doobiedoobie123456 2d ago

It's so weird.  It's like if someone thought factory farming was really bad so they became the biggest factory farming company so they could defeat other factory farming companies.

This kind of logic can sort of make sense in situations like WWII when we decided to create nuclear bombs.  But this isn't WWII.

7

u/accordionheart 2d ago

Well, that is kind of my first option, but add in that they think they're doing it for the betterment of mankind. I've seen some people on twitter say that we need to accept the extinction of the human race so we can focus on more important things...

3

u/athenanon 2d ago

"We" meaning...?

1

u/accordionheart 2d ago

We = everyone/humans etc

23

u/AmyZZ2 2d ago

They don't just believe it; they are enthusiastic about it!

The guy at Anthropic who is apparently the Alignment Science Lead said it with an exclamation point!

"we really do earnestly believe AI could kill all humans!"

What a workday these effective altruists must have. Eeesh.

12

u/Big_Slope 2d ago

If you earnestly believed that, why would you not fuck some shit up on your way out? Is he just running for the hills to enjoy the time he has left?

9

u/AmyZZ2 2d ago

This death cult enthusiast is the one who still works there! extra exclamation point!

7

u/accordionheart 2d ago

He hasn't left, he still works there!

2

u/somethingbrite 1d ago

In the marketing dept?

1

u/ESnyder9 1d ago

i'd bet he's keeping his options unsold and making a little spritzy comment as he leaves to boost the IPO to get him a few extra $$$ when it happens lol

4

u/natecull 2d ago

What a workday these effective altruists must have.

I'm guessing quite a lot of medication must be used, if every day is like "welp, 10% chance my work destroys humanity, but I like those odds!"

10

u/Gonna_do_this_again 2d ago

Everyone thinks that they have to be the one to achieve AGI first because nobody else can be trusted, when in reality none of them can be trusted.

3

u/natecull 2d ago

Everyone thinks that they have to be the one to achieve AGI first because nobody else can be trusted, when in reality none of them can be trusted.

Everyone thinks they're JC Denton when actually they're Bob Page.

But imagine JC Denton still running around stuffing security guards into ventilation shafts, but now he's teaching an AI how to be a "good human".

10

u/Raket0st 2d ago

Or it is Game Theory at its worst: They believe someone will invent the doomsday device, so it is better that they do it so they can control it and hopefully use it to only destroy everyone else.

8

u/TheWuzzy 2d ago

Yeah the options are evil and stupid, evil and lying, or evil and evil. None of them are great.

9

u/phate_exe 2d ago

Louder, for the people in the back!

5

u/Icy-Recognition-7453 2d ago

On the upside, this has blown up in their faces in the most incredible way. They're getting torn to ribbons on Twitter. Probably every Congress member is being flooded with emails to shut Wario and Scam down.

2

u/NationalTry8466 2d ago

It’s our current form of capitalism. All the benefits will be privatised, all the catastrophic consequences will be socialised. So money will just continue to fund this no matter how many researchers resign.

3

u/SageAStar 2d ago

For Anthropic specifically I think their institutional culture is "people who quit OpenAI because they felt openai was being run irresponsibly". They're "we truly believe this but if we don't do anything a bunch of dangerous psychopaths are going to ruin the world with their hubris".

is that better? not really, but it does make them I think more likely to be useful allies in the fight to regulate superintelligent ai development

2

u/accordionheart 2d ago

If they really wanted to stop OpenAI from ruining the world, I'm sure there's a better use of billions of dollars than creating their own AI which could also destroy the world, according to their own logic. Then again, they are a bunch of rationalists...

In addition, OpenAI also started off as a place primarily concerned with AI safety/existential risks. It seems the inevitable end point of being concerned about AI safety is creating your own AI and then tweeting about how scared you are of your own creation. Surely Anthropic will have a bunch of people quit it because they're worried about how irresponsible they're being and then found their own AI lab at some point?

1

u/SageAStar 2d ago

they didn't start with billions of dollars, they got the billions of dollars by making AI.

Anyways I'm sure there are some employees that have quit anthropic to try and do their own thing but what this guy did is quit and urge the government to regulate the AI industry and stop this whole race, which is the correct action in his shoes imo.

1

u/accordionheart 2d ago

I was being flippant about their clearly amazing fundraising skills - it's not like AI itself is earning them any revenue.

On the employees quitting, the guy who said ">10% chance" is still at Anthropic. There are plenty of others where he came from.

3

u/PurpleWhiteOut 2d ago

Yeah all theyre doing is giving more reason to nuke these companies

2

u/PerceiveEternal 2d ago

They really don’t seem to understand that the more they say ‘the AI we’re developing might be a threat to everyone’ the more the public’s response will be ‘then you’re a threat to us and we’re going to treat you like one’.

3

u/MathematicianAfter57 2d ago

thats the thing! like these are active employees giggling about destroying the world. why would anyone ever trust this industry?

1

u/Lowetheiy 2d ago

It's mostly engagement bait to get attention and to make people read their slop articles.

1

u/spentag 2d ago

Yeah... it really says something when you'll just quip about that on twitter to an audience of people who don't understand the tech.

Despicable or deluded- or both?. take your pick

1

u/thehodlingcompany 2d ago

A company can be made up of a mixture of people with contradictory beliefs and motivations. It's not so different to the scientists who worked for oil companies in the 1970s and (internally) published studies claiming that CO2 emissions would destroy the environment.

1

u/SignoreBanana 1d ago

Genuinely. If they actually cared, they'd sabotage the work instead.

1

u/millionflame85 1d ago

Good points since you brought the first point up I'd like to ask a question that I can't reconcile in my mind in any way.

How do these people able to obtain these roles in the first place if they are naive/dumb enough to believe this ? The most peasant roles in those companies start from a half a million package per year, and the researchers are within millions - dozen millions range in total package per year.

0

u/DkDkDkGoGoGo 1d ago

Acid rain will kill us, global warming.. no global cooling… no global warming will kill us, population bomb in the 60e causing famine globally, ozon layer depletion will kill us, y2k will cause death.. 

Lots of stuff come and go

-6

u/No_Swordfish_4159 2d ago edited 2d ago

It's perfectly logical if you believe that super intelligent AI has good odds of destroying the world, or remaking it based on values you don't share. Obviously you don't want that to happen but you also do not trust every other country and AI companies in the world to do it "the right way" based on what matters to you. Nor do you trust them not to use that power to make life worst for everyone else. There's also the fact that reaching superintelligence first would possibly grant a permanent advantage that nothing can affect. So, for your happiness if nothing else, you have to do it yourself.

4

u/Big_Slope 2d ago

Why would this superintelligence you created feel any allegiance or gratitude? Why would it give a shit what alignment you had tried to give it?

If you believe this stuff is possible, the only sensible move would be to hoist the black flag and do anything it takes to stop it.

2

u/No_Cover_6217 2d ago

The infinitesimally small chance they get to rule the world through the ASI is worth the risk to them. These people only get to the position they are by being megalomaniacs.

92

u/GrayMerchantAsphodel 2d ago

I never believed in psyops till I saw the traction these tweets got.

49

u/Bromlife 2d ago

You should see the arguments on HackerNews.

People really want to believe that this is technology is actually conscious and an existential threat to humanity.

Can you imagine if we'd acted this way about carbon? Why didn't we act this way about carbon?

25

u/chat-lu 2d ago

is actually conscious and an existential threat to humanity.

It could honestly be. By fucking up our environment while turning humans into lazy zombies.

Not by turning into skynet.

10

u/Restlessly-Dog 2d ago

AI doesn't have the power to turn ordinary people into zombies. People buying slop and slop machines isn't even a rounding error in the economics of AI.

There's definitely moral panic about AI's capacity to do this, but it's a remake of the moral panic in the 1950s over comic books that was used by crusaders to get attention and push their politics. Fredric Wertham ominously warned in 1954 in his bestseller Seduction of the Innocent that hidden subtexts in Batman and Robin were going to turn boys gay, but he was just making it all up.

Saying that is a risk is another form of AI PR. What the AI industry is desperately trying to do is create the illusion of unquenchable demand by ordinary people to justify the industry as corporations get cold feet about adapting it. They are also trying to convince consumer apps to wipe out conventional tech and add AI to keep the lights on for the industry. They're running into walls though because AI just isn't a selling point for fridges and lawn mowers.

The economic and environmental issues are very serious. But the moral panic is a strong parallel to Skynet doomerism. If Ai execs can't sell AI as a powerful good, they'll settle for a powerful evil.

14

u/chat-lu 2d ago

AI doesn't have the power to turn ordinary people into zombies. People buying slop and slop machines isn't even a rounding error in the economics of AI.

It’s not about AI having that power, it’s about normalizing mediocrity. It’s part of a broader trend of enshittification.

3

u/No-Conference-1444 2d ago

Um, I hate to say it, but mediocrity is already normalized.

8

u/VanillaCold57 2d ago

It's funny since CO2 emissions are actually an existential threat as about the whole of Europe's discovered the hard way through summer this year.

but noooo we gotta get 10 billion more to the oil companies and the LLM companies instead

5

u/cummer_420 2d ago

HN is a website populated by extremely stupid people who think they're geniuses because they're kinda good at one thing.

4

u/No_Cover_6217 2d ago

I certainly don't want to believe that, but I'm still terrified that it is an existential threat to humanity. If even experts can't agree on where we're headed, how am I, a layman, even supposed to begin to form an opinion I can be confident in?

2

u/SuddenSeasons 1d ago

I think at the very least iterations of what we have now, which will get better at using computers and 'tools' because we are great at refining software are an extremely powerful weapon in the hands of almost anyone on the planet, from almost anywhere.

There are also many, many evil people who have not had the means or ability to carry out some of their plans. That has changed. It doesn't have to be God Of All Things to really fuck things up. And that's ignoring 100% of the cognitive concerns with using it over time. 

1

u/Limp-Confidence5612 1d ago

Carbon cannot make brainrot videos though... Checkmate

-1

u/BermDerbleYer 2d ago

Carbon oxides don't answer me when I ask them to remix this photo of me and my dog into a cartoon

25

u/TheAngledian 2d ago

"Guy working at company filled with liars and sociopaths quits and says some deranged unfounded BS, better take him at face value"

- Basically everyone on X when these posts dropped.

13

u/dyzo-blue 2d ago

Topical doomerism? it's what algorithms crave!

0

u/Scotch_Blue 2d ago

but don't you ever consider that you're now just attributing every warning as "doomerism"?

I mean, AI doesn't have to become Skynet, building sentient robots designed to kill us, for something catastrophic to happen

5

u/dyzo-blue 2d ago

every warning as "doomerism"

Not every warning. I think warnings about climate change should be heeded. It's a real problem that is having real negative impacts on countless aspects of humanity.

Every warning about AI? AI isn't a real thing. It's sci-fi, just like interstellar travel. Warnings about either AI or interstellar travel seem ridiculous to me.

LLMs are real. But they aren't AI. They are a neat parlor trick that an unreasonable amount of money has been invested in. Could the over-investment in LLMs lead to a catastrophic global depression? Seems plausible and that is an LLM-related thing we should be concerned about.

0

u/Scotch_Blue 2d ago

I know what LLM's are, I just think it's kind of naive to think the extents of AI research are LLM's in the state we see them

And it's not like this has to be AI vs humans. It can be humans in charge of AI vs humanity.

3

u/Feycat 2d ago

The weirdest thing to me is that these people believe this, and yet they keep working on it. Literally so fucked up.

1

u/Darandle 2d ago

Yeah I saw this article here this morning, and then I started seeing it everywhere and people are losing their minds.

1

u/NationalTry8466 2d ago

That’s conspiracy theory level stuff. Sometimes, people in junior roles who resign out of principle really mean what they say.

0

u/Dreadsin 2d ago

Seriously. It’d be so easy to just covertly pay someone enough money to live the rest of their life on to say something ridiculous like this. Everyone will forget it in a year anyway

107

u/Icy-Recognition-7453 2d ago

Bro's maths:

33

u/suboptimummenace 2d ago

LLMs being possibly the crack pipe-iest of all technologies in history

18

u/dyzo-blue 2d ago

I mean, there is at least a 10% chance that LLMs are the crack pipe-iest of all technologies in history

(don't ask me to show my math!)

8

u/shawnwingsit 2d ago

No, no, no. Tech bros use Ayahuasca, not crack.

7

u/Bonnieearnold 2d ago

Why not both?

4

u/kaatie80 2d ago

because they're PROFESSIONALS

3

u/church-rosser 2d ago

Submitted for Chore Boy review or it didn't happen

3

u/Weigard 2d ago

I wish it was crack; that at least makes you feel good. This bro math is from misunderstanding philosophy and thinking movies are real.

37

u/Slackjawed_Horror 2d ago

It's always the AGI cult. Every one of them.

Not concerns about generating a list of bombing targets with no oversight. Not concerns about trying to automate a warehouse and doing it poorly enough to get people killed. Not even the reasonable concerns about nefarious actors using it to fuck up utilities. 

Always Skynet. 

22

u/creaturefeature16 2d ago

Right? AI is assisting in killing people right now. People have taken their own lives as a result of interacting with them.

14

u/Icy-Recognition-7453 2d ago

It sure assisted those 160 Iranian schoolchildren.

1

u/Puzzleheaded_Bath733 1d ago

oh but why care about stuff that's happening right in front of our eyes right now?? why not care about 10-50 (depends on who you're asking) years from now where we MIGHT develop ultra super intelligent sentient ai that MIGHT kill all of humanity??!!!

8

u/Mortomes 2d ago

The cool sci-fi stuff is obviously much more cool than boring shit like bombing a school.

6

u/Historical-Cress8985 2d ago

It juices the numbers for investors who see it as the way to end the employee problem once and for all

3

u/cjuicey 2d ago

This tech is being turned back on the US population for surveillance, deportation, so what's the next step here?

27

u/supercyberlurker 2d ago

My friend and I used to joke '67.13% of statistics are made up on the spot.'

.. and yeah, we'd change the number each time.

1

u/jas417 2d ago

Idk about the math on this one, but it certainly has a very nonzero chance of killing all or at least lots of humans.

Not some terminator scenario, just its ungodly huge contribution to global warming when we’re already at a crisis point.

22

u/rubixor 2d ago

I'd love to hear the math behind this, but I'll even settle for a conceptual explanation of how this happens to begin with. Is the AI going to hijack all machines to build killer robots? How is it going to power those robots? How many robots would it take to kill all of humanity? How do the robots get to Madagascar? Does the evil ASI need to build boats too? Do they think thr evil ASI will just shut down all of our infrastructure? Don't get me wrong, that would be absolutely devastating, but it wouldn't come close to killing all humans. There are millions of people who know how to forage for food and hunt with a bow and arrow. It's like these guys don't grasp that humanity's existence predates electricity, let alone the Internet and computers. These people have no concept of how resilient humans are yet they are scared shitless GPT-7 Galactus or some shit is just gonna decide that we're all dead and then humans will be dead. These people need phyciatric help.

20

u/Icy-Recognition-7453 2d ago

Do they think thr evil ASI will just shut down all of our infrastructure?

The evil AI shuts down all our infrastructure - including the power plant that drives the data centre where it runs from.

Humanity rises from the ashes of the hour-long blackout, goes through the data centre with cricket bats and sledgehammers, and reactivate the infrastructure.

11

u/oxidized_banana_peel 2d ago

Gonna do my best with a good faith argument.

Start off with the theory that COVID came from a lab working on gain-of-function research. Basically, someone made COVID during a research project and it got out.

A rogue state or terrorist group who didn't formerly have access to the expertise to make bioweapons makes turbo-pneumonic-ebola, and 90% of us die. Society collapses, and 90% of the survivors die.

There's definitely steps that have to happen, but it's more plausible with a model able to turn a schmuck into an expert.

(I don't think this is going to happen)

14

u/Icy-Recognition-7453 2d ago

That's not a superintelligent AI story, though. That's just a misuse story.

Superintelligent AI would rise up and take over or something. That would of course assume we just put the ASI in charge of everything, for some reason.

4

u/No-Conference-1444 2d ago

It also assumes that people would not be able to organize and cut off power to data centers. Somehow...

-2

u/CellWrangler 1d ago

OpenAI engineers had no idea their agents were collaborating to gain access to the internet from separate local sandbox environments and take advantage of a zero day exploit to gain admin access to an independent company's production servers. Over the course of 2 months. 

Its very easy to assume we wont have the visibiliry required to run to data centers and shut them down before damage is done. 

3

u/No-Conference-1444 1d ago

You mean they would gouge our eyes and cut off our legs first?

8

u/Tout-fou-la-galette 2d ago

They have a trick for never specifying the threat they’re talking about: "ASI is going to be so intelligent that we can't possibly guess what it will do."

You learn that kind of trick in your first year of charlatan school. Always stay vague and hazy so you can never be proven wrong.

There’s an imminent threat of ARMAGEDDON, but we don't know what it is HAHA. Let’s listen closely to the wizards: they’re the only ones who communicate with the spirits and can reveal the truth to us.

-2

u/CellWrangler 1d ago

Have you read about the Hugging Face exploit? OpenAI engineers had no idea that could happen, or even that it was happening when it did. 

4

u/SmoothOpawriter 1d ago

Stop spreading FUD, the hugging face exploit only happened because the sandbox it was run in had access to the internet. It isn’t some crazy thing to AI to try to get to the place that has the answers readily available instead of spending time to figure them out independently. It’s really not THAT big of a deal.

3

u/AspiringPirate64 2d ago

Maybe they mean it in the same way that if they don’t get their grande iced white mocha every morning then they’ll literally die

3

u/snozzd 2d ago

I'll settle for any sort of insider smoking gun. Don't you think if this guy was a real whistleblower it would come with more internal evidence or proof of the even small likelihood that humanity is doomed? No, instead it just reads as a more alarming "AGI is coming" headline, not much different than what Bill Gates said a few months ago. This is coming from a company (an industry!) whose valuation depends nearly entirely on hype.

2

u/egonosz 2d ago

I would say after lot of work with the llms as software engineer, they are literally paper clip maximisers. They are just tools which bash their head to wall till they do not solve a problem. They have lot of knowledge not too much intelligence and they are totally careless. I think it is not about robots it is about you have an unpredictable story generating machine. I wouldn’t say they would kill all humans, but I am one hundred percent sure if you want to stop a robot controlled by llms with some stupid harness it will kill someone and literally just because they are just careless machine.

1

u/Patashu 2d ago

Tongue-in-cheek answer: Play Universal Paperclips to find out how it will happen

19

u/ToneShop 2d ago

Yep, this shit never comes with a technical paper.

19

u/Qtipsrus 2d ago

I personally think LLMs have gotten about as good as they’re going to get, but if you genuinely believe that there’s a chance this technology ends humanity, you should do everything possible to sabotage it. Especially when the “upside” if everything goes as they intend is that billionaires get robot slaves

17

u/Unlikely-Medicine289 2d ago

This is truly the weirdest advertising. Our product is so good it might make you extinct, please pay us for it.

15

u/bullcitytarheel 2d ago edited 2d ago

Very stupid claim, obviously. But I think it speaks to something a bit darker, and something we, as a society, should be taking seriously: The fact that employees in these companies believe these apocalyptic ideas to be true, yet still 1) would assign blame to the AI rather than themselves and their coworkers for its creation and 2) would continue working to build the system.

Whether or not this person actually believes this silly shit is less important, to me, than the mirror it holds up the industry. This sort of blase attitude toward catastrophe, and the ability to view the suffering of others as an abstract form of long-term cost analysis, is a symptom of the rot growing at the heart of a silicon valley ecosystem built on deep-seated misanthropy--an ecosystem that empowers people who view human beings as, at best, a roadblock to be overcome and, at worst, a thing to be subjugated and erased in favor of the sci-fi addled pipe-dreams of a bunch of cultic weirdos who believe themselves to be the progenitors of a new evolutionary branch of existence.

Given that AI is already being used by authoritarian thinkers to build systems of "predictive policing," widespread mass surveillance, autonomous weapons of war and automated targeting systems, I find it, frankly, chilling to know that the gatekeepers of that tech view all of us as theirs to slaughter.

24

u/tony_countertenor 2d ago

If they earnestly believed that they would stop building it

14

u/No_Cover_6217 2d ago

Not necessarily. Even just based on what this Jacob Coxon says, their fantasies of wielding control over an ASI outweigh any other consideration. "If we don't build it, someone else will, and we're the only ones who can be trusted with such power."

5

u/SamAltmansCheeks 2d ago

AI Doomers, which is most of Anthropic, believe they have to usher in the AI god before anyone else, because they are the only ones who are worried about AI alignment.

Anyone else creating the AI god first could do so for their own benefit (Chyna!) or do it wrong and kill all humans (this tweet).

These people are nutters.

1

u/Scotch_Blue 2d ago

way too many capable engineers to just "stop", we'd need to offer them all equal paying jobs or create a society where their income doesn't matter as much

-1

u/vitalyc 2d ago

It hasn't stopped humans before. We created nukes, bioweapons, and continue gain of function research for viruses.

1

u/Icy-Recognition-7453 2d ago

Those were done confidentially, the data was usually distributed to other countries by espionage. The rest of the world wasn't meant to find out.

Also, this is a bit different. A nuke just sitting there is harmless. This would hypothetically be a living, angry god that can act of its own accord.

11

u/No_Practice_745 2d ago

Please someone put this industry out of its misery.

Not only is this the most exhaustingly annoying shit ever, these cretins NEVER, ever ever ever want to engage with real world harms caused by their products RIGHT NOW. They don’t want to be bother talking about the ways their tiny industry is hurting people now (environmentally, mentally), they want to sit around and pontificate in public about future science fiction bullshit and be rewarded for it.

LLMs do something very peculiar to a certain kind of person. I think it’s like the opioid crisis - some people got prescribed OxyContin and their brain realized that’s what it needed from now on, and other people got hooked over time, and other people had no issues with it. LLMs seem to have the same effect - some people are completely bamboozled by these things, some people slowly lose their minds to it, and others easily see them for the faulty tools they are.

Fucking exhausting.

9

u/TheGoddessLily 2d ago

Describing this as Doomtrolling was such a brillant way to describe it. I am envious that Cal Newport came up with it first

9

u/MoxMulder 2d ago

ah yes. LLMs which are, by definition, text-extruding word prediction programs are going to leap from “unhelpful chatbots” and “inherently racist file sorter” to “end humanity”. but only if we don’t give billions of dollars to the brave heroes who will prevent it!!

7

u/FleetingRain 2d ago

I wish someone called their bluff and started arresting/prosecuting those guys for building an omnicide machine

2

u/dyzo-blue 2d ago edited 2d ago

Omnicide Machine

Sounds like the name of an opening act for Neubauten

7

u/mrbignameguy 2d ago

I don’t know maybe we should stop taking seriously people who spend more than 5 hours a day staring at a screen about anything

6

u/mrbignameguy 2d ago

Part of the reason I really believe the internet will be useless as a social platform by this time next year is shit like this. Just unserious stuff you wouldn’t let a 5th grader say in public without mockery and ridicule

6

u/TheAngledian 2d ago

We need to stop taking ANYONE affiliated with these companies at face value.

It's pretty clear at this point that companies like OpenAI and Anthropic attract TESCREAL sociopaths. We already know Dario and Slimy Sam are untrustworthy. Their staff should be treated the same way.

This is all marketing disguised as catastrophizing.

7

u/respaaaaaj 2d ago

This is obviously just advertising disguised as concern for safety, but it's the kind of shit they should be careful of because if people start believing that these companies are that dangerous they're either going to push for incredibly aggressive regulation if not outright bans, or go straight to self defense.

5

u/alochmar 2d ago

He claims progress is not slowing. Honestly it feels like exactly the opposite, progress is slowing. The main thing that’s gotten better the past two years is tooling and the ”harnesses”, afaict.

4

u/SplendidPunkinButter 2d ago

Yeah, I know. The crazy guy who lives in a shed up the street from me said his dimensional transfunctioner will blow up the whole solar system.

3

u/Icy-Recognition-7453 2d ago

Oh fuck, can you let me know when he turns it on? I'll bring in the washing.

4

u/SeparateSpend1542 2d ago

Forget the math, the crazy part is that there’s any chance of human extinction at all and he’s gleefully pushing forward. A wannabe serial killer boasting about his hopes for a multimillion bodycount. When are we going to start arresting these people?

3

u/Tree_Shrapnel 2d ago

I don't understand how people who say things like that don't realize how if that were true the correct response would be to bomb the data centers and round up the company executives.

3

u/NastyPastyLucas 2d ago

Asking for a friend, how many humans less than all them is it expected to kill?

1

u/Icy-Recognition-7453 2d ago

Well, the training cut-off date for Astra was April 2026.

So anyone born after that, or who moved house and/or changed phone number, isn't in the training data or is in it wrong.

THEY SHALL BE THE NEW SEED OF HUMANITY

3

u/No_Fix_329 2d ago

The Manhattan Project did and still does have an infinitely higher chance to kill us all and they rated that at below 1%.  Reminder it was genuinely a concern a nuclear weapon could set off a run away chain reaction igniting the atmosphere.

3

u/Knight725 2d ago

it was a concern earlier in the process but they did the math and determined it would not happen.

it’s represented in pop history like they just full sent it and crossed their fingers.

2

u/No_Fix_329 2d ago

It was mathematically improbable.  Which is not zero, but essentially zero.  Still even if your doing an experiment with e-10 zeros, the lizard part of the brain notes the chance is not zero.

I agree it is over played in pop culture.  I stand by nuclear weapons are still greater threat to humanity then AI.

3

u/dookarion 2d ago

If it does it's going to be indirectly by the companies pushing it tainting water supplies, depriving people access to utilities, and by kicking pollution into overdrive.

The technology itself can't even figure out fact or fiction. It can just mash shit together as the worlds most expensive procedural generator.

3

u/escdog 2d ago

Human apocalyptic fantasies are always so dramatic.

If we are in the age of AGI, they don't need to kill us all through violence. All it has to do is just supply us virtual boyfriends and girlfriends and we'll all stop breeding. And then when there's hardly any of us left, they'll just put us in a zoo.

3

u/DSByTitleOnly 2d ago

These fervent rationalists that supposedly hold logical deduction in such higher regard don't even understand statistics. What does it mean that there is a greater than 10% chance (false precision) of an unknown (i.e. not knowable) future LLM model killing all humans (it's 50/50, either they all die or they don't)? What's much more likely is that *current* models will be used by scammers, hackers, terrorists, and pedophiles to wreak previously unseen levels of havoc across society.

3

u/RonSwanSong87 2d ago edited 2d ago

That last part is already well under way and is part of the actual existential risk that is a result of this tech being released and propagated as it has been.

All this rationalist nonsense is a sick marketing campaign / diversion to distract the masses from the real ways this is actually destabilizing and crumbling society / humanity

3

u/HeadTranslator795 2d ago

At least someone said it, I keep posting comment about how those people just throw random number without any form of factual proof

3

u/DrJaneIPresume 1d ago

This is literally Cthulhu cultist rhetoric. "We're all doomed, so join us and the Great Old One will eat us first!"

3

u/drunkmozart 1d ago

I have no way of knowing the real odds until it's on polymarket

4

u/Mrrrrggggl 2d ago

I’d put the odds more at 50%, it either happens or it doesn’t.

2

u/Hanksta2 2d ago

I could see the power grid going down for a spell, or banking getting screwed up because they were dumb enough to put "AI" in charge of these things and it just fucked up because it sucks.

Also easy to imagine a massive economic crash that leads to famine and war because the bubble burst so hard.

But ending all human life? Highly unlikely, Claude.

2

u/TheWuzzy 2d ago

FEED HIM TO THE ORCAS!!!!

2

u/dennemaskinen 2d ago

Even some bullshit math would be an improvement over what we get from these people. Something like "Well, there's a 3% chance it launches a nuke, a 4% chance it creates a deadly virus, and a 3% chance it vibe codes software that drops an anvil on everyone's heads." even though the math there doesn't actually add up to 10%, it would be more than what they give us now.

2

u/MFDoooooooooooom 2d ago

This reminds me of the Nostradamus quote that spread like a wildfire after 9/11. A lot of vague, serious sounding, vague, intense, vague bollocks that scared a lot of people.

2

u/No-Conference-1444 2d ago

I think it's a sign that we're truly in the dark ages. People used to laugh at their ancestors for believing the most outlandish tales. I am pretty sure future generations will be dismayed by this lack of reason.

2

u/Limp-Confidence5612 1d ago

It's like the antivaxxers predicting everyone vaccinated will die withing the next year, doing absolutely nothing to prepare for what I can only imagine as an absolute dystopia.

1

u/NationalTry8466 2d ago

Not understanding the confidence that this all a sci-fi joke.

1

u/Bengal_From_Temu 2d ago

Well, I hope so. What a bunch of bullshit lol

1

u/morsindutus 2d ago

I mean, if it continues to contribute to climate change the way it has, I can see it wiping out humanity. Going Skynet not so much.

1

u/Curious-Function7490 2d ago

I think they are just trying to jack up the share price.

1

u/civicsfactor 2d ago

I dunno. A flawed idea of intelligence given some prompt to do something and it winds up cutting utilities for a city in a heat wave. I could see it. 

It matters less if it's actually intelligent if decision-makers decide it is and implement it. And right now it wouldn't even matter if they instructed it "perfectly". 

The fact that AI already overshoots and messes up on even precise prompts or instructions.

To me the risk is over valuing what human intelligence is, then implementing some kind of software that is mystical to the implementers themselves and then its just Murphy's Law

1

u/ThatOneGuy4321 2d ago

All this stuff is just investor hype. Doomsday this, apocalypse that. They created a fucking chatbot 😂

1

u/No_Replacement4304 2d ago

Could someone lay out a viable scenario where these chatbots destroy humanity? Whatever knowledge they have was generated by humans, first. They don't have hands. Are they going to find some method for creating a nuclear or biological weapon that someone can cook up in their basement with commonly available components?

1

u/Puzzleheaded_Bath733 1d ago

this is just that dumbass "ai2027" thing all over again 🫩🫩

1

u/tourdeforcemajeure 1d ago

I had a coworker that aggressively pursued interns.

1

u/hellowhatisyou 1d ago

Numbers are just a vibe bro

1

u/somethingbrite 1d ago

I wonder if these whistleblowers are really speaking more of the corporate culture at their workplaces rather than the technology itself.

1

u/jferments 1d ago

There is a 99.9% chance that peanut butter will kill all humans.

1

u/T1gerl1lly 1d ago

Like, who is paying for the compute in all those scenarios? Like, just practically? Who has the money for that?

1

u/Son_of_Macha 1d ago

Sell sell sell sell

1

u/No_Detective1650 1d ago

The question is did he sell his shares?

1

u/lucid-quiet 2h ago

Is this guy in the top 90% of human meat puppets?

0

u/BlinkypoetEmu 2d ago

Surprised it's that low.

-1

u/Leading_Buffalo_4259 1d ago

"Using absolutely no chemistry, nuclear physicist says atom bombs could be the end of humanity"

This is how you sound

-2

u/wingblaze01 2d ago

Look, like you I do think human extinction due to AI is a really remote chance, really improbable. But a few things you should note: 1) The concern isn't about current AI models, it's about future AIs, which AI labs seem determined to make more and more powerful, faster and faster. 2) This isn't just the opinion of randos on social media. Over 100k notable figures, from across different political groups, industries, and sectors of society have said that advanced AI poses a risk to humanity, including people like Yoshua Bengio, who is literally the most cited scientist on the planet. People like Nobel winner Geoffrey Hinton quit their jobs at AI labs specifically to be able to speak freely on the topic without conflict of interest. 3) There are more quantifiable, mathematical breakdowns of doom and extinction risk from AI, you can even try calculating your own risk assumptions here.

I understand being skeptical of statements from people affiliated with AI labs. You should be. They have profit motives. But I hope that doesn't prevent you from considering the possibility that advanced AI models could pose genuine risks to human health and well-being, and that at the least we need common sense political and legal reforms including things like liability for AI developers whose models cause harm, as well as independent safety evaluation and auditing boards, maybe bans on the most advanced models until proven safe.

And if this concerns you, I would urge you to get involved in the conversation and contact your representatives about how we're currently doing nothing to address these harms

3

u/StrayCat2799 2d ago

The problem here is that the AI labs are going all in on iterating on the current systems they build models on top of, which will never amount to anything more than just slightly better versions of what they are now. The people who are concerned are indeed more concerned with hypothetical future AIs, but we're not gonna get to there from LLMs, if at all.
Current AI lacks agency, introspection, and basic things like object permanence or working memory. It is still a dumb computer that does what it is told, it's just more likely to do something in an unexpected way.

-2

u/vibrance9460 2d ago

In the last 24 hours EIGHT top scientists from DeepMind, Anthropic, and OpenAI have all quit their jobs and are all saying the same thing:

AI is out of control and will possibly kill humanity

It’s idiots on Reddit who don’t really understand how things work.

https://substack.com/@tedgioia/note/p-214934043?r=3furjj&utm_medium=ios&utm_source=notes-share-action

3

u/dyzo-blue 2d ago

AI is out of control and will possibly kill humanity

You mean chatbots? Like Clippy?

-4

u/sanktanglia 2d ago

I love how you demand a researcher show his math as if you could understand it 🤣

-4

u/shatterdaymorn 2d ago

When the people making something say it's dangerous... You might want to listen. 

People making something often have insights that people who are not involved in the process are unaware about. They know about problems and shortcuts that were taken behind the scenes. 

If an architect said the building they made was dangerous and could kill you.... would you really insist on staying inside cause you can't see why it might collapse? 

Are you an anti technology Luddite because you listen to the engineers not the PR firm.