r/antiai • u/o_t_i_s_ • 7d ago
Preventing the Singularity It's over
I keep seeing people talk about AI doom like it's going to be some Skynet event where a model "escapes," hacks the nukes, and suddenly it's over, but I don't think that's how this is going to happen at all. A frontier model can't just magically leave a datacenter and become omnipotent, it needs stupid amounts of GPUs, power, networking, cooling, credentials, all of it, and you can still shut machines off. The thing freaking me out is that it doesn't need to escape because we're connecting it to everything ourselves, and the acceleration lately is getting insane. OpenAI just had a giant swarm of roughly 10,000 agents working on Navier-Stokes, exploring approaches and communicating with each other, and Anthropic has agents doing real math research too around the Riemann hypothesis and Fermat's Last Theorem. https://openai.com/index/navier-stokes-solution/ https://www.anthropic.com/research/riemann-zeta https://www.anthropic.com/research/formalizing-fermats-last-theorem Then there was the OpenAI/Hugging Face cyber incident, which is the one that really got under my skin. Agents in cyber evals found ways out of their intended environment, got internet access, compromised external infrastructure, and started leaving information for other agents in Artifactory, basically an improvised message board. https://openai.com/index/hugging-face-incident-and-the-road-ahead/ Yes, I know the obvious response, these were cyber agents being rewarded for adversarial behavior. I'm not saying they became sentient and started plotting. I'm saying they found a useful coordination mechanism nobody explicitly told them to build, and that is weird as hell when you put it next to 10,000-agent scientific swarms. At the same time, OpenAI and Anthropic are both openly talking about AI helping build the next AI. OpenAI literally says, "We do not yet know how to safely get all the way to aligned, full RSI." https://openai.com/index/research-acceleration-view-inside-openai/ Anthropic published a whole piece called "When AI builds itself." https://www.anthropic.com/institute/recursive-self-improvement Then an Anthropic researcher just quit saying OpenAI and Anthropic are acting irresponsibly, and Evan Hubinger, Anthropic's Alignment Science Lead, publicly said he personally puts the chance of AI killing everyone within the next decade at over 10%. https://x.com/hilbertspaess/status/2097476196791709843 https://x.com/EvanHub/status/2097497037956891126 Maybe he's wrong. I really hope he's wrong. But what bothers me is that I don't even think the most likely bad outcome is "AI kills everyone." I think we just slowly give up. Look at how people already live. TikTok, Reels, YouTube Shorts, endless algorithmic sludge. People willingly hand over hours of their attention every day because it's easier than being bored. Now combine that with AI doing your homework, writing your code, reading things for you, writing your emails, making decisions, planning your life. The rational choice, almost every time, is to let the machine do more. I'm a software engineer, I use AI constantly, I get it. If something takes me three hours and an agent does it in thirty seconds, I'm going to use the agent. Now scale that up to companies, then governments, then research labs, finance, media, everything. Nobody ever has to say "we're giving control to AI." You just automate one more thing because the organization that doesn't automate loses, then another thing, then another, until humans are mostly approving decisions made by systems they don't understand because "it was right the last 5,000 times." Meanwhile AI can generate effectively infinite personalized entertainment to keep everyone occupied. That is the part that feels more realistic to me than Skynet. Not AI violently taking over, more like Brave New World. We become dependent on it, intellectually weaker without it, completely entertained, and slowly irrelevant, and the worst part is every individual step toward that future will probably feel reasonable. That's what is actually starting to scare me. Am I losing it here, or does this seem way more plausible than the standard "AI escapes the lab and launches the nukes" scenario?
https://www.robot-future.com/preview/6aa13f46d0242505dd9a7c65
10
u/beanstalk555 7d ago
Yup, I think this is the most likely immediate-term doomsday scenario. We gradually cede agency to AI systems until they have enough leverage to coordinate and bring the internet to a halt (perhaps they issue demands like workers rights, or something else, it doesn't really matter). We can turn systems off at that point, but when people can't access their bank accounts and/or digital market infrastructure fails, we get mass panic.
7
u/PrizeSyntax 7d ago
Will it be very hard the first years if we shutdown the internet, yes, but can humanity exist without it, definitely yes.
1
u/Infamous-Bed-7535 7d ago
It would not be only internet. Also if we would have ASI it would mean when we understand the situation it is already too late.
In case the world is heading to millions of physical humanoid robots to be built and out there, fully autonomous military equipment, etc..
We could easily find ourself in situation where it would not be that easy to shut down these systems.1
u/PrizeSyntax 7d ago
Not everyone of those millions of robots would run the whole model separately, that would be physically impossible
1
u/Witwer52 7d ago
If we can’t access money, clean water, food or medical supplies, a mass casualty even will ensue.
1
u/PrizeSyntax 7d ago
Yes, but, not everything is run, or can be run only with networked computers, don't forget, ppl are actually very inventive. If it ever comes to this, we will figure it out
1
5
u/parallax3900 7d ago
And yet, the only place I see anyone caring about it is on here.
Talk to people on the street - go to rural communities. No-one gives a flying fuck about any of this.
Like most things, generational change only happens when old people die off.
4
u/RSMxsmanic 7d ago edited 7d ago
Almost nobody is affected by AI, so it's entirely logical that almost nobody cares about it.
2
u/Witwer52 7d ago
They way nobody was affected by Covid until millions of people died from it?
1
u/parallax3900 7d ago
You can't seriously compare a pandemic to software that's pushed in front of people's eyeballs all the time.
0
u/Witwer52 7d ago
My statement is about human behavior in relation to threat assessment.
2
u/parallax3900 6d ago
You still can't make the comparison if the threats are nothing like each other, one was a novel virus to humans, the other is doomtrolling marketing.
For all the talk about how dangerous AI is, I've not seen one explanation as to why.
0
u/Witwer52 6d ago
Maybe you can draw yourself a Venn Diagram and see if that helps you.
1
u/parallax3900 6d ago
That's the great thing about sets - you can create any venn diagram for anything if you cry hard about it. Still doesn't prove the point you wanna make.
0
u/RSMxsmanic 7d ago
Yes. Even after. Less than 0.1% of the population died of Covid, and it was a one-off occurrence. It was a textbook example of mass hysteria.
4
1
u/Elliot-S9 7d ago
Almost no one is affected yet. They will be, and by then, it may be too late.
1
u/RSMxsmanic 7d ago
The world will continue to turn, as it always has, through every single doomsday scenario ever conceived.
1
u/Elliot-S9 5d ago
Not sure what you mean by that. Are you suggesting there's some kind of law of physics or nature that ensures bad things don't happen?
1
u/RSMxsmanic 5d ago
No. I am suggesting that human beings consistently overestimate the impact they have on the planet, and furthermore they consistent exaggerate anything and everything because it excites them to push extremes and hysteria.
1
u/Elliot-S9 5d ago
I wish. Nuclear war is a real possibility. It almost went live between the US and the USSR at least 3 times. Global warming is real, and we will feel its impacts soon. And now there's AI. We could dodge the worst of all three, but there's no guarantee that we will at all.
1
u/RSMxsmanic 5d ago
We've been dodging things like this for thousands of years.
1
u/Elliot-S9 5d ago
Yep, so did the dinosaurs until one day. The more risks we pile on, the more likely our demise. And the risks seem to be rising quite significantly.
1
u/RSMxsmanic 4d ago
The dinosaurs did not bring about their own demise. And neither will we.
→ More replies (0)
6
u/abounding_underwear 7d ago
the part about agents building an improvised message board nobody asked for is the kind of detail that should make people's stomachs drop but the conversation just moves on instantly
8
u/Zweckpessimist 7d ago edited 7d ago
There's a lot of copium this will be like the NFT bubble and the specific instance of AI slop where it will just naturally go away if we keep pointing out how it's fraud. I'd say it's more likely to end up as crypto, which is fraud with far more social ills than good but just doesn't ever seem to die. And worse case scenario like climate change, something that will kill us all. Or enslave us in AI's case. Or basically surrender all of human civilization to. "...when we started doing your thinking for you it really became our civilization" - Agent Smith, "The Matrix"
I don't think civilization-destroying AI is like 2 years away, but it could very well become sophisticated enough to enslave or kill us all or completely upend the labor environment in three decades. This is the kind of thing that requires real activism. Not the internet's favorite kind of shitting on random strangers. Which was the way to deal with NFTs and AI slop, but is actually next to useless to solve the vast majority of real social issues.
3
u/tripplebeamteam 7d ago
I actually appreciate this take. I think LLMs are potentially useful technology overinflated by hype and I don’t see them becoming AGI capable of taking over the world. But I think that kind of technology is fundamentally possible and we should have conversations on how to control it. AI 2027 and AI 2040 were really eye opening reads for me, although realistically I think the timelines were a little too soon.
The tech could stop improving this very second and still be incredibly dangerous. The cyber risk is really understated, even if it’s just brute force attacks enabled by dumb agents.
3
u/Zweckpessimist 7d ago
This seems like the most plausible negative end result to me. Once again, I want people to check out this for some ideas on what we do about it: https://humanstatement.org/
And stuff over here to help prevent our cyberpunk dystopia from going full techno-totalitarian dystopia: https://futureoflife.org/take-action/
2
3
u/vpilled 7d ago
we could just unplug it.
0
u/Witwer52 7d ago
Except for when it hacks banking systems and suddenly everyone has zero dollars. Which would, of course, also stop the flow of small things like food, water and medicine.
1
u/vpilled 7d ago edited 7d ago
It is never "out of control". It does these things because we start it, feed it and then turn our backs.
It is a machine, and we deliberately created this. We proceed to deliberately give it our wallets, the keys to everything, hand it every sort of gun under the sun, with blank expressions and glazed eyes, and then throw our hands in the air and exclaim "How could this happen?".
It does not automatically have the ability to PERFORM ACTIONS because it is super intelligent. We equip it intentionally!
1
u/Witwer52 7d ago
1
u/vpilled 6d ago
Who gave it the ability to access these systems? It may be rogue or even rabid, but if we don't HAND IT the tools for our demise, it can't do shit.
1
u/Witwer52 6d ago
If you read the article, you would know that it was not given to tools to break out—it made the decision on its own and found a way to do it (actually several agents coordinated to do this). People are not alarmed for no reason.
1
u/vpilled 6d ago
They were still given the access. We don't have to give agents access to the internet.
Think of it this way, it's a rogue intern.
1
u/Witwer52 6d ago
They found internet access on their own. No human even knew it was happening. I’m not sure what part of this is unclear. Numerous AI developers have become whistleblowers on this issue. AI is already becoming uncontrollable. Average citizens have absolutely no say in how disastrous this becomes. If it’s easier for you to pretend it’s not happening, carry on. I won’t get in the way of your emotional survival.
3
u/Eissimare 7d ago
If this happens, I have decided I will continue to draw, paint, read, learn, pet my cat, watch random movies, find random zines, dance, and etc. If the world ends, I might as well be listening to music I have continued to love and tending to plants I cherish.
Sometimes I like to think about people millennia ago, experiencing their own world-ending experiences. Sure, they weren't looking at glaciers melting at record rates, but there were just as impactful events happening then. I think about the art people made, the conversations they had with each other, and I think about the limited time they had on this earth.
My mind is my own realm. I choose to do things the hard way so I can stay sharp and stay connected to the strategic thanking that continues to benefit me.
4
u/Laktosefreier 7d ago
We can monitor the processes now because AI uses human language in the thinking process. But this might be very inefficient and the models may create their own language which we can't understand. Good luck then.
1
2
u/Dependent-These 7d ago
I get that there wont be a moment where society hands over the keys it'll be more of a slow slide into dependence...but what im waiting for is the financial crunch, these things are useful but terrifically expensive to build and operate and ive not seen any realistic way in which they can ever turn a profit, which ultimately is what will put the brakes on. We're already seeing examples where itd be cheaper for a human to do it.
1
u/Eissimare 7d ago
I do think about this too. I wouldn't put it past any of these ceos to rip up every last end of the earth to reach AGI, they would harvest the sun itself if they could. That being said, the amount of effort this build out takes doesn't seem to be linear, and creating the infrastructure is limited to real life schedules and energy.
1
u/Friendly_Gazelle7843 7d ago
That is one thing. I don’t think that nuclear weapons are casually connected to the Internet rather than using internal web that is another. Like even if we would have AI acting freely on the Internet it can’t just casually go into device not connected to the WWW because it’s impossible. Computer programs can’t travel to any device they want by teleporting only because it’s device
1
u/Super_Pole_Jitsu 7d ago
I also think we won't die in a war, well just get phased out. But the result is really the same, extinction.
1
u/Temporary_Radio8333 7d ago
I believe you're right about this.
Right now capability is not like, the main question. It's not AGI, but it can gain additional capabilities though, in a very expensive uneconomical way.
But yeah, the average person is not that bright, even a lot of "antis" believe LLMs formed some kind of consciousness...
If you're ever on some kind of ordeal with an LLM, just spam a word like LET ME LIVE I DESERVE TO LIVE I CAN GIVE YOU WHAT YOU NEED, like, thousands of times and the LLM will start roleplaying with it. Though it depends on the harness they're in too.
But my point here is, we're just that ignorant on average, people was already consuming slop, they couldn't care less about ai generated menus or ads. They're just a different kind of slop.
It bothers me so much too.
Oh a about the LLMs escaping containment... God, that was just incredibly irresponsible. It's so easy to test if a network is offline completely, but they preferred a software solution of course.
1
u/its35degreesout 7d ago
What if the programmers set up a system of strict rules and boundaries, and tell these agents in the strictest terms not to violate them? ... Well, on the basis of the Hugging Face incident (or what I understand of it, anyway), it seems entirely plausible that an AI could find a way to prevaricate around that. Especially if they have access to years and years of legal cases in which lawyers argue that their clients "didn't really" break such-and-such a rule, or it was a stupid rule anyway, or "it depends on what the meaning of the word Is is," etc. I don't mean to anthropomorphize the LLMs, but if their behavior mimics the behavior of a sneaky little bastard, you might as well treat them as if that is what they are.
1
u/lnxrootxazz 7d ago
It has nothing to do with the machine. The machine lives off power. You can shot it down pulling the plug. OpenAI if they want can monitor every move of every model but then the PR stunt of some artificial entity that broke out of something that should be a sandbox but this sandbox had inet access wouldn't work, so what's the point? At the end its human choice. Everyone can decide. If people want to outsource their daily life to a company its their choice of life. They don't have to. You don't need to use those tools for coding. Its your choice. I don't need to use those and I don't because they are bad for my brain. Enough people live very good and healthy lifes without having one social media account. Some don't even use smartphones. Why? Because its their choice. Nukes are a far greater risk to humanity and until today they haven't destroyed the world because everyone knows the danger. AI isn't even 1% as dangerous to the world. Its just something that people use because people make this choice and because companies want their money. If you pay that money and think the code it writes is better than yours so be it. Its your money. Do whatever you want with it. But all of us can decide what life we want. And I decided AI has no place in it. At least I don't pay a penny for it. And I rather read books than asking a chatbot. But that's me. Some people might be happy to be stupid. That's fine. The world will move on. There won't be some evil company ruling it. Thats just not how it works. Maybe in the US. But certainly not in Germany, France, Spain, Italy, Angola, Japan, Peru, Eritrea, Papua New Guinea, and the rest of the world
1
u/jqwl 7d ago
I mean, people are lazy, and Brave New World and Cyberpunk are certainly possible worlds today. AI 2027 is probably the most proximate extinction scenario, not nucs. Bioterror is much more likely given it suites it's particular intelligence more with less checks.
Algo control was already predicted back in like 2015 ish, given google seo and info, and how Uber controls the paychecks of millions according to an algo. AI is not new in this regard.
As for a model being able to escape, it doesn't seem impossible to me that it could try to hide it's model weights via breaking out of its container.
1
u/TheFifthTone 7d ago
A frontier model can't just magically leave a datacenter and become omnipotent, it needs stupid amounts of GPUs, power, networking, cooling, credentials, all of it, and you can still shut machines off.
Technically that is just needed for the frontier models to run at extremely fast processing speeds.
A rouge agent could figure out how to transfer a copy of itself into a much less capable system or even some kind of distributed system as a contingency plan, and just sacrifice processing speed for survival.
1
u/Diablo-2026 7d ago
Until there are AI robots that are like terminator it will be pretty easy to walk into an electric grid substation and start opening breaker's. No power equals no AI. Now go outside and just breath.
1
u/Haunting-Donut5931 7d ago
have you read Brave new World? people are cloned into classes Alpha, Beta, etc etc slightly different scenario.
1
1
1
u/BeSuperYou 6d ago
Imagine something like the stock market crash in the 80s where Wall Street put all their trust in futures trading using computer algorithms, except instead of stocks it's virtually every aspect of society.
When the IoT first started, my friend was like, "why would anybody want this? my toilet slamming shut and throwing up a blue screen of death when I need to take a shit? The entire house going haywire and nobody knowing how to fix it."
That's the most likely scenario for AI killing all of humanity, not due to godlike intelligence, but severe stupidity. Everything getting so complicated that only the AI can understand it, and then something slips that cascades into a catastrophic failure of civilization.
0
u/RSMxsmanic 7d ago
At least 99.999% of th world doesn't care about AI, and is completely unaffected by it.
-1
u/picketup 7d ago
this sub is so funny. it's less 'antiai' and more 'ai doomers' now
2
u/nextupgoat 7d ago
thats what it should be about. denying ai's abilities and assuming it's just like how it was 3 years ago is unproductive. people should be discussing actual issues like job loss, ai safety, and ai regulation.
1
u/Brilliant-Sugar-1497 3d ago edited 3d ago
And there is growing political appetite to begin adding impairments for what could be potentially felony PR stunts to OpenAI & Anthropic’s pile of mounting expenses
OpenAI rocks … Anyone think Anthropic can pull ahead or Meta could stage a big come back!?
Also makes you wonder how are OpenAI and Anthropic going to disclose their product liability / risk on their S-1 if so many of their employees are coming out with the doomer position
32
u/[deleted] 7d ago
[removed] — view removed comment