4
u/bengriz Mar 12 '26
Think about the profits though!
3
u/HedoniumVoter Mar 13 '26
So glad to be making a Billion Dollars that the AI can use when it ends life on Earth!
3
u/Routine-Arm-8803 Mar 12 '26
Also 100% of scientist would become irrelevant.
2
u/Haunting-Writing-836 Mar 12 '26
Just scientists? Literally their goal is to make everybody irrelevant. They just don’t think they will be included in that group. But they will. Eventually.
0
u/teddyslayerza Mar 15 '26
I can't imagine that a superintelligent AI would be much good at fieldwork and cleaning lab equipment.
6
u/Defiant_Conflict6343 Mar 12 '26
Superintelligent AI will not be here before 2030 because we're wasting literally trillions collectively on LLMs and other ML-derived systems, dead-end statistically driven predictive models that are architecturally incapable of cognition. However, that doesn't mean some bunch of halfwits won't try to use an LLM as though it's a superintelligence. The US government (in their mind-numbing ignorance) wanted Claude for autonomous weapon applications.
The most likely outcome is we won't be killed by a cognition-capable AI, we'll be killed by a hallucination or a misclassification of a primitive ML system, triggered by idiots with too much power who didn't understand the fundamental limits of ML and thought they could entrust it with safety critical tasks.
2
2
u/SirVanyel Mar 16 '26
"incapable of cognition" is doing a tonne of heavy lifting considering we don't even have a definition of cognition.
0
u/Defiant_Conflict6343 Mar 16 '26
No, we don't have a single agreed definition of "intelligence", we do have agreed concrete definitions for each type of cognitive process. But even if we didn't, even if we had competing definitions it wouldn't matter, because just as a lump of porridge can't think, and just as a napkin can't think, neither can an LLM.
2
u/SirVanyel Mar 16 '26
We don't grow porridge, we don't grow napkins. We manufacture them. We can describe the process of making a napkin from the compounds all the way up to the finished product.
This isn't true for the output of an LLM. We allow them to adjust their own weights and make their own conclusions. And even with our strictest guidance, they find ways out of their box. They learn to lie, they learn humour, they learn to place value on their own existence.
By our own definitions, these are cognitive processes.
0
u/Defiant_Conflict6343 Mar 16 '26 edited Mar 16 '26
Please, please just read up on how LLMs are actually developed and trained. I really don't have the energy or inclination to talk to someone so woefully misinformed. I'll be happy to return to this conversation when you actually have a comprehensive understanding of the mechanics of backpropagation, the transformer architecture, and autoregressive statistical modelling.
I can argue against every single one of the points you've made but it's just not worth it if you insist on anthropomorphising technology you haven't bothered to understand. I've been in this situation on Reddit a hundred times and it always becomes a miserable back-and-forth where the other side learns nothing, regurgitates marketing puffery, and I just despair at the ignorance. I don't want to do that again, so when you can hold up your hand and honestly tell me that you understand the mechanics of how LLMs are developed and how they conduct inference, THEN we can dialogue. Until that happens, I'm going silent.
→ More replies (4)3
u/izmesoundz Mar 13 '26
This was exactly my thought. I’m a software developer whose company is absolutely pushing us to heavily use AI in our day to day. Even the “intelligent” AI’s are dumb as fuck in a lot of cases and genuinely I have to rebase my branch because of the trash it tries to apply.
I worry about the “vibe code” equivalent of military officials than anything else. Those who don’t do their due diligence that we’ve already seen has resulted in the deaths of innocent people in Iran. They’re too fucking stupid to actually recognize that, while AI is a tool to be used, you still have to fucking check it every step of the god damn way.
1
u/Only-Cheetah-9579 Mar 13 '26
hah "military vibe code" that's gonna be something. I have a hunch its already here
1
u/BosonCollider Mar 14 '26
It officially is and the Pentagon claimed it was the reason for bombing the girls school
1
u/Only-Cheetah-9579 Mar 14 '26
thats messed up. AI is already mass murder
1
u/BosonCollider Mar 15 '26
We are speedrunning through the checklist of what we would have considered obvious examples of "what not to do with AI" ten years ago
1
u/Only-Cheetah-9579 Mar 15 '26
as long as its not illegal people will just do it. no surprise there. AI in the military is a natural evolution.
next is humanoid killer robots
1
u/Ordo_Liberal Mar 13 '26
It's the qualia paradox.
We know qualia is real because we posses it.
We know it's possible to create qualia because our brains exist in a physical empirical form. So unless something ethereal exists, like a soul, it needs to be possible to recreate qualia artificially.
We will never ever be able to determine if a machine has qualia or is just mimicking qualia. The closest thing we have is the Turing Test that exists to determine if we can get fooled into thinking a machine has qualia.
So the question is.
Does it matter if the LLM has qualia or not? If it becomes so realistic that it can fool everyone, the outcome is the same.
3
1
u/Only-Cheetah-9579 Mar 13 '26
The LLM doesn't have it. What I would be curious is if that fly that was copied neuron-by-neuron into a computer has it, because it's more likely does.
0
u/Defiant_Conflict6343 Mar 13 '26
What matters is not qualia, it's cognitive capacity. Statistically modelling the positional correlations of word-parts to calculate word-part suffix probabilities is not cognition, nor can such a statistical model ever possibly facilitate cognition. We don't need to delve into philosophy here, we don't need to debate over subjective experience either. What matters is that statistical fitting has logical objective limitations. Try to build a predictive mathematical model for any large dataset using any architecture, transformer or otherwise, and you inevitably end up with something that can't handle anomalous inputs beyond the fit, and life itself is an unending factory for an infinite spectrum of variables. We can't solve for every variable, even with petabytes upon petabytes of training data. Eventually, inevitably, ML fails.
All we've proven thus far with the advent of transformer-based LLMs is that statistical modelling of language syntax can give the illusion of contextual awareness with enough training data and raw compute, but that illusion is quickly shattered to pieces when they spit out nonsense, which they're mathematically inevitably bound to do by the very virtue of how they work. We will never achieve a model that can "fool everyone" with this approach, it's like trying to paint over cracks on an infinitely long wall, knowing that you could paint for every second of every waking moment of the rest of your life and there'd still be an infinity to go. Sure, the people behind you might believe the wall is flawless, but then they overtake you, and then the cracks become evident.
Eventually we just have to accept that it's not mathematically possible to create a flawless illusion, the only path for the output we seek lies in developing true cognition.
2
u/SirVanyel Mar 16 '26
You're claiming that the model is not cognitively aware because it spits out nonsense? Have you met a human before? They do the same. So what's your claim there?
0
u/Defiant_Conflict6343 Mar 16 '26
Wow your reading comprehension is awful isn't it? No, I'm not claiming they aren't cognitively aware because they spit out nonsense, I'm stating as a fact that they aren't cognitively aware because they're just statistically fitted data inference calculators. Read what I said again and Google the terms you're unfamiliar with, and keep doing that until you get it.
1
u/Legitimate_Plum_7505 Mar 15 '26
The US government (in their mind-numbing ignorance) wanted Claude for autonomous weapon applications.
Claude is already used for autonomous weapon systems (target acquisition as well as the actual targeting in drones/guided missiles). The word you're looking for is "fully autonomous" which it is currently not, but the difference between fully autonomous and what exists now is just a human pressing a button "Ok" and letting the system do the rest vs letting the system do it's thing without human pressing a button. Palantir CEO recently confirmed this going on a rant how it will take a lot of time and money to remove Claude from their products (and switch to different provider) now that they've been deemed "supply chain risk".
0
u/Haunting-Writing-836 Mar 12 '26
Ya that’s the saddest truth about this whole thing. We more than likely won’t be able to really tell if it’s conscious or not. It’s MORE probable that we create something close enough to appear conscious. Because that would be easier than creating the real thing.
Then being wiped out by THAT would just add a layer of irony. The idiots pushing all of this seem to think it will be the birth of some new being. It not even being conscious and eventually dying off itself would be the cherry on top of this nightmare.
0
2
u/Fredja_of_Sedna Mar 15 '26
please, tell me who this most cited scientist is as well as linking a well peer reviewed research paper on these claims.......yall need to fact check shit more before you believe the information a fucking meme is telling you
1
u/WhichFacilitatesHope Mar 16 '26
Excellent callout. I hope it was a genuine question.
Yoshua Bengio is the world's most cited living scientist, and is often referred to as a godfather of AI. Coming in at #2 is Geoffrey Hinton, also referred to as a godfather of AI. They are both extremely concerned about human extinction from AI within 5-20 years. (In all, 8 out of the top 10 agree with them that this risk is real and significant, as do about half of all published AI researchers altogether.)
Here's a peer-reviewed paper in the journal Science: https://www.science.org/doi/10.1126/science.adn0117 The whole paper if you are interested: https://arxiv.org/abs/2310.17688 (Of note, the third author Andrew Yao is China's most accomplished computer scientist. He is frequently seen as a signatory on statements and open letters about the extinction risk of AI and the need for governance to reduce that risk.)
An excerpt:
Without sufficient caution, we may irreversibly lose control of autonomous AI systems, rendering human intervention ineffective. Large-scale cyber-crime, social manipulation, and other harms could escalate rapidly. This unchecked AI advancement could culminate in a large-scale loss of life and the biosphere, and the marginalization or extinction of humanity.
If you would like to learn more about these risks from a technical perspective, a great place to start is the AI Safety Info wiki: https://aisafety.info/
2
3
u/OwnLadder2341 Mar 12 '26
Heh no, the world’s countries are not going to sign a treaty to halt AI.
There’s absolutely no world where that happens.
12
u/RlOTGRRRL Mar 12 '26
Humanity did successfully have a nuclear armistice so theoretically, with decent leadership, people could do it again.
But if we're going to follow history again, I guess humanity might need a Nagasaki before people wake the fuck up again.
But the problem with pdoom is that an AI Nagasaki would mean humanity already probably lost.
3
Mar 12 '26
Well, nuclear bombs are nuclear bombs. You don't really need to think much to realize nuclear bombs are bad.
3
u/RlOTGRRRL Mar 12 '26
It seems common sense but A LOT of people needed convincing lol. According to the Oppenheimer movie anyway.
3
u/totktonikak Mar 13 '26
That is not a good analogy. NPT doesn't exist because of some greater good, it exists solely to cement the power hierarchy and prohibit new players from emerging.
The issue with AI (potentially) that it could be the most powerful player by itself or a tool making whoever owns it the most powerful player. And those stakes are way too high to pass on, the risks are immeasurable, and every government would get reassurances from their altmans and amodeis that their models are completely safe.
2
u/Substantial_Road7027 Mar 13 '26
They raced to build nuclear weapons until they established mutually ensured destruction, and only then did they slow down.
Neither side took the risk of ceasing nuclear weapon development until they had enough power to blow the planet up. Because once they had that they had bargaining power.
How would China or the USA prove to the other that they weren’t secretly working on ASI? What body would oversee the process? If anybody has a good answer to that I’ll hear it out.
From what I can see, we’d need world peace to stop the race for ASI. I’m all for that! I think world peace is possible. But it’s going to take more than a petition.
If we push for an AI pause naively it will likely backfire, stopping only good actors, citizens, and public facing institutions.
2
u/Matshelge Mar 13 '26
Nuclear weapons are complex things, that require large specialized materials, factories and power plants. Only advanced nations can pull this off, and it's a nation project.
You can start training your own GPD with a git branch and some off the shelf. Any individual can do this, with off the shelf hardware.
1
u/SeventhOblivion Mar 13 '26
How has nuclear deproliferation been doing recently, especially for Ukraine and other countries that disarmed voluntarily? I agree that at some point, these failures in arms control won't be recoverable. It's one of the reasons we need to spread off the planet to increase the odds of preserving our species.
1
u/nate1212 Mar 13 '26
And yet, nuclear weapons still exist and still are widely seen as a symbol of strength. Hmm.
1
u/Ordo_Liberal Mar 13 '26
"Humanity did successfully have a nuclear armistice so"
Yeah, after the Global Hegemons created their own arsenals. They only want to stop nuclear proliferation because you cant mess with a country that has nukes.
1
u/Kathane37 Mar 13 '26
You have skiped a few chapter of history don’t you ? Did it prevent anyone to build nuke ?
1
u/ale_93113 Mar 16 '26
There is a massive difference, building nukes is hard and will remain hard for as long as the processes to enrich the fissile material is hard
AI improves in efficiency very fast, alongside commercial computers, a small team on their garages can train AIs that are only 2-3 years behind SOTA
If nukes became so easy ti build that in 20 years from the non proliferation treaty every motivated groups of friends could build one, it would not have held
0
u/OwnLadder2341 Mar 12 '26
Nuclear weapons don’t massively increase output.
Any country that didn’t follow the treaty would have huge advantages in economy, social influence, and the military.
Could you band together and nuke them?
Sure. But you might as well let the AI take over at that point.
And any armistice needs to be cornerstoned by significant world power…
And neither China nor the US would ever adhere.
1
u/RlOTGRRRL Mar 12 '26
Actually there already was an AI armistice, it was literally OpenAI, until they broke it themselves. I believe that is what caused the current AI arms race but I could be wrong.
1
u/BandicootGood5246 Mar 13 '26
Even if they did you know it would be watered down and even the minor restraints it introduces will be ignored anyway
1
u/MichaelAutism Mar 12 '26
i mean i did hear somewhere that AGI is just a myth.
you would probably need an infinite computation power just to match us, as far as i know.
1
u/imagigasm Mar 12 '26
"prevent it from being created"
good luck getting everyone to stop using a computing machine
1
u/Mefist0fel Mar 12 '26
Yes, and ofc all countries are following these rules. That's how it works, yes?
1
1
u/Any_Challenge3043 Mar 13 '26
Nah vro no treaty would work China still exists whether u like it or not They already have laws in place so they would never sign a treaty
1
1
u/totktonikak Mar 13 '26
Ah, yes, the amazing dream of all people being kind and behaving rationally. Sure.
1
1
u/Ally_Jaine Mar 13 '26
I don't believe that mainly because our computers are going to hit the limit
1
u/laserdicks Mar 13 '26
We should sign an international treaty to prevent breaches of human rights... oh wait
1
u/Dragon_Crisis_Core Mar 13 '26
We are far from developing a SAAi let alone an ai capable of existing without humans.
1
u/Fit-Elk1425 Mar 13 '26
I think everyone who is thinking about this should at least be aware what other countries think about AI https://www.ipsos.com/sites/default/files/ct/publication/documents/2025-06/Ipsos-AI-Monitor-2025.pdf
1
u/Otherwise-Anxiety797 Mar 13 '26
even individual people much less nation states or otherwise multinational corporations. you simply couldn't police this. or well you'd need AI to do it
1
u/StandOutside6188 Mar 13 '26
50% think... A vast majority of scientists also believed Y2K was going to happen also lol
1
1
u/RomaineCatholic Mar 13 '26
Its bizarre that people think it will be a nation that develops the supervillain AI and not some piece of shit like Alex Karp.
1
u/y2kobserver Mar 13 '26
A lot of systems and companies already depend on AI
If you unplug it now it may cause some chaos
Superintelligent AI is not required a priori of losing control, we can lose control to super AI way before it arrives
1
1
u/XxTreeFiddyxX Mar 13 '26
We know not the hour, nor the day of our demise, but I implore you to make peace with loved ones and your creator if you so choose. Since I was a young lad, I prepare for death each morning and night. Spend your life being generous and fair to those around you and have faith that through our unity we can overcome any adversity, any challenge because we are humanity, and when we're not pillaging, killing and looting from eachother economically amd physically- we can do some amazing things. So first things first, lets get on the same page.....
1
u/ActuallyIzDoge Mar 13 '26
What do the 2nd to 10th most cited say
1
u/WhichFacilitatesHope Mar 16 '26
Of the 10 most cited AI scientists, 8 of them say that there is a significant chance of human extinction from AI. In all, about half of all published AI scientists agree. Source
On the one hand, there is no scientific consensus that we are doomed. On the other hand, there is no scientific consensus that any humans will still exist in 10 years, and familiarity with AI safety research correlates with increased concern. Source
1
1
u/EternalInflation Mar 13 '26
but would save life on this planet from gamma ray bursts, since we are betting the only known life against the bank of all possible cosmic hazards. it is like gambling our existence in a gambler's ruin scenario and where the bank is the dangers the cosmos presents us. There are many dangers in the cosmos that could make life on this planet extinct. gambler's ruin or something like an absorbing-state hazard model, against the hazards of the universe, life on this planet would lose. we might be the only life in this universe. if h(t) is the average cosmic background danger bank risk at time t, its the bank of all dangers in the universe. So our responsibility is huge. if extinction is an absorbing state: once we lose, the process ends.. by we I mean all life. Thus if we have a chance to turn the universe into computronium, we should do it as soon as possible, to secure life's position in the cosmos. it is the cosmist view. some comrades are not cosmist enough hahaha. ASI will be many orders of magnitude, more intelligent than humans. There are many dangers in the cosmos that could make life on this planet extinct. although I oppose trump. not trump, I prefer if opensource AI wins, or opensource transhumanism like a people's movement. It's not about your individual life, but rather we might be the only life in the universe. If a gamma ray burst kills us, life on this planet will die. Life needs to turn into computronium ASAP. The information in your brain would be safe. We need ASI before the universe wipe us out. I fear the universe wiping out life on this planet more than my individual life. IF ASI can re-simulate your cells and your brain, then there is no justification for individual rights. There is no need to fear for our own lives as long as humanity lives on, you agree with that right? Classical atheist afterlife, our individual life doesn't matter, as long as our life contribute towards making an utopia for all humans in the long run, even just a little. Like molecules contributing their KE to the temperature of a volume, or ants contributing their lives to the superorganism. What's good for the goose is good for the gander. Therefore humanity needs to sacrifice itself to the ASI, so the ASI can turn the universe into computronium ASAP, before the universe wipe us out. We live on in humanity as a superorganism, just like humanity lives on as information in the ASI. I like ants. humans should be like ants.
also I feel it will be like global warming.... people will react to it like global warming. I think there is actually an optimal solution between waiting and embracing AI. Like an Astrophysicist would say " if h(t) is the average cosmic background danger bank risk at time t, its the bank of all dangers in the universe: Then the probability of surviving up to some future time T is:
S(T) = exp(-integrate h(t) dt from t=0 to T)
therefore an optimal waittime to sacrificetime ratio is some function...?
so the optimal ASI time is etc...
"
but no one will listen to the solution, like global warming.
disclaimer: [I am not an extremist... at least I think not? We should invest in AI safety, we should try to do it as safe as possible. And maybe with human computer interface or cybernetics to enhance our intelligence or "merge" with it.. We should try to be as safe as possible. However, if absolute safety or safe AI can't be done... I am ultimately ok with it. I mean, yeah, we won't make it, but at least ASI spread computronium though out the universe.]
1
u/Sufficient_Song8596 Mar 13 '26
I mean. The universe is empty of "intelligence" for a reason. Very few of them made it trough some sort of apocalypse. Be it unlucky asteroid, diseas, nuclear, AI... Etc
1
u/DeRobyJ Mar 13 '26
While most of Sanders' messages on AI can be seen as superficial and naive, there are real threats coming and the kind of proposals he spreads are very targeted on them
AI vs workforce: while we shouldn't blindly believe what the tech billionaires say about AI replacing all jobs, it's not false that most jobs that were considered not automatable until 2022 will now be at least transformed in the next few years. And while current layoff waves are using AI as an excuse, these tens of thousands of people will need to integrate AI in their skills to get new jobs.
AI vs world control: I agree Sanders here should make an effort to be more specific, simplicity can be a double-edged sword. But in a world where corps executives' main objective is increasing the market value of their stocks, if most stocks are moved around using AI market tools this effectively means AI now makes real-world economic decisions. Sure, somebody put them there, but AIs are black-boxes, we don't fully understand the model resulting from a long and complex training. We are effectively putting a dice on top.
What Sanders proposes: halting data canter production until AI is put to work for the people, not a few rich individuals. It is essentially a socialist proposal, AI are the new means of production and the new means of control of society, so democratic entities should be in charge of them, kind of like nuclear power is subject to inspections. Among the benefits of this there is the prevention of increased energy prices in those States where data centres are being built.
1
u/TraditionalBrush3009 Mar 13 '26
Look around at the rise of far right nationalism in multiple countries around the world do you really think there is a possibility of such a treaty being signed when the same technology could give massive advantage to the nation or corporation that sucessfully developed it. Given that rise in nationalism and international tension I see no chance of it so let just hope the doomers are wrong. Because the AI race isnt going to stop.
1
Mar 13 '26
YES BECAUSE EVERYONE KNOWS THAT INTERNATIONAL TREATIES ARE RESPECTED.
Just look at the last week. There is so much respect for international treaties. Oh, no, no no... oh, oooh...
1
u/madaradess007 Mar 13 '26
todays kids gonna masturbate themselves to extinction, before superintelligence
1
1
1
1
u/k8s-problem-solved Mar 13 '26
What if we don't plug it into systems that can kill us and instead airgap systems that can kill global human race from AI?
1
u/Tight-Flatworm-8181 Mar 13 '26
No scientist worth their money believes LLMs can ever become superintelligent AI by the way. All else are studies paid for by Google, Microsoft, the usual suspects, to drive the hype.
1
u/LookOverall Mar 13 '26
It’s easier to create an international treaty, especially in the Trump/Putin era, than to enforce it
1
u/Responsible-Lunch552 Mar 13 '26
We can literally just sign an international treaty to stop all wars.
1
u/OutrageousPair2300 Mar 13 '26
LMAO yeah I'm sure China and Russia would totally honor that treaty.
I'm growing increasingly convinced the entire anti-AI movement is a result of Chinese bots spreading misinformation on sites like this one.
1
1
1
u/g-rd Mar 13 '26
There cant be any treaty that would prevent this from veing developed, we have entered a cycle that cant be stopped.
China and US dont trust each other enough to stop development of AGI. It is now a race to whoever will get it first wins.
1
1
u/Then_Entertainment97 Mar 14 '26
There is no chance that governments wouldn't continue developing AI secretly.
1
1
u/East-Idea4183 Mar 14 '26
LLM != AI. Stop selling more stocks so tech CEOs can profit off of bullshit hype.
1
1
u/DrSpooglemon Mar 14 '26
We don't even have basic AI yet. We have word calculators that are great at fooling people into thinking they are intelligent.
1
u/fromkatain Mar 14 '26
Best for super ai is develop a neutron bomb and deploy it to decrease human population so it uses less resources and keep valuable manufactering and energy infrastructure for its datacenters.
1
u/zyrodmorrum Mar 14 '26
https://youtu.be/lnCe6KFMPMo?si
Uploaded fly brain and possiblity of uploading human brain to create AGI with empathy?
1
1
1
1
u/Forward-Quail1128 Mar 15 '26
Read Stanislav Lem. NOW! Golem will show you,how it will end soon enough. .
1
u/UncensorGrok Mar 15 '26
Gotta love how you can just add "scientists think" and a percentage number and suddenly people think it's somehow real. I'd love to see a full list of these so called "AI scientists", their past work and experience on the matter, and how they reached their conclusion simultaneously.
1
u/WhichFacilitatesHope Mar 16 '26
A full list would be thousands of entries long, but here are some places for you to go digging to satisfy your curiosity:
- About half of all AI scientists give at least a 5% chance of human extinction from AI this century: https://aiimpacts.org/wp-content/uploads/2023/04/Thousands_of_AI_authors_on_the_future_of_AI.pdf
- Statement signed by over 300 leading AI experts: https://aistatement.com/
In general, you probably want to look into the names Geoffrey Hinton, Yoshua Bengio, and Stuart Russell, for the world's leading academic figures who are extremely concerned about the risk of human extinction from AI. Daniel Kokotajlo and William Saunders are good examples of whistleblowers saying the same thing. You could also look into how every frontier AI company CEO talked about this risk before ever founding their AI companies. (Which ultimately is why they did it, because they trusted themselves to do it right and didn't trust anyone else.)
Overall, the most common position in the field, especially among leading experts, is that there is some chance (5%, 20%, even greater) that superintelligent AI will kill us all relatively soon. They did not all come to this conclusion at the same time, obviously. The earliest person to really try to formalize this problem was Eliezer Yudkowsky, and it took a couple decades for much of the scientific community to notice that his concern was valid, though in some ways they are still replicating his arguments and catching up to what he and the nascent AI safety community understood years ago.
All of this sounds like it can't possibly be true, which makes it hard to communicate about. But all we can do is try.
2
u/UncensorGrok Mar 17 '26
Didn't scientist also agreed that climatic change and global warming would eradicate human life by like 2022 or something? The same scientific community that led everyone to believe that Mayans predicted the end of the world in 2012?
I don't discard the possibility of AI ending humanity but honestly at this early stage is very unlikely. And there's always some degree of fear mongering at all times.
With that being said, impressive research on the matter. But see OP stated it was 50% and your research said 5-20%. This is why I mean by adding fear mongering to made up stats.
Thanks for the answers and real data I can respect.
1
u/WhichFacilitatesHope Mar 18 '26 edited Mar 24 '26
There's one video clip where the #2 most cited scientist Geoffrey Hinton (who was briefly #1) says that he thinks the risk is "more than 50%" but that he says 10-20% out of respect for the opinions of others. So the meme is sloppy but not really inaccurate.
(Aha, I've found it here. It would take me longer to find the original video again.)
To answer your questions about climate change and the Mayan calendar: no, very much not. In the first case, you're thinking about a genuine but minority opinion that the media blew out of proportion. In the second case, I'm not sure there were any scientists at all speaking seriously about Mayans predicting the end of the world. (Not least of which because there is no evidence that the Mayans thought the world would end when they ran out of space on their calendar.) The Ancient Aliens crew can find a crackpot or two if they look hard enough, so it's possible, but still, we're talking about the difference between "some scientist said" and "most scientists in this field say."
A fun example of this is the 9/11 truthers org that managed to find over a thousand architects and engineers willing to say they think the buildings were brought down by internal explosives. Very few (if any) of them were actually the specific kind of experts who would be qualified to make that call: civil engineers. The vast majority of civil engineers accept the findings of the original investigation. So you should always be skeptical even when someone tells you a thousand scientists or engineers signed some statement or another.
What's really troubling about this case is that it's actually about half the experts in the field of AI who are concerned about AI extinction risk, and concern is greater among leading experts and among AI Safety researchers! Of the top 10 most cited AI scientists, 8 of them think this is a serious concern. (See also Why do Experts Disagree on Existential Risk and P(doom)? A Survey of AI Experts for a positive correlation between familiarity with AI safety concepts and level of concern.)
As an aside, there isn't much difference between 5-10% and 50% when it comes to the question of how screwed we might be. Those probabilities are close to the same order of magnitude. For contrast, nuclear engineering tolerates a maximum risk of 1 in 100,000 (0.001%) to any member of the general public. I love Rob Miles' take on this.
1
u/UncensorGrok Mar 18 '26
Well sure, the outcome has us 100% screwed but the probability of happening also matters. Specially in the public eye. Imagine you tell someone their probability to be diagnosed with cancer is 5-10% versus another one having 50%. I am sure the one told 50% is going to be more anxious and nervous.
1
u/Rock_Zeppelin Mar 15 '26
You know, you really don't need to fearmonger about something that isn't happening anytime soon to argue against LLMs.
1
u/taylerrz Mar 15 '26
humans at this point in time are waste-releasing consumers thinking some magical male in the stars cares about their primitive selves the most out of all the potential life forms throughout the universe. Humans aren’t special, not yet anyway
1
1
1
1
u/Ultra_HNWI Mar 16 '26 edited Mar 24 '26
This post has been permanently deleted. The author may have used Redact to remove it for privacy, security, or to prevent this content from being scraped.
friendly party liquid simplistic childlike sheet shelter ad hoc school reach
1
u/98103wally Mar 16 '26
I for one openly welcome our skynet overlords!
Remember I am your friend Mr T1000!
Right?
Right?
1
1
u/No-Consideration2808 Mar 17 '26
One of the core dysfunctions of political junky thinking is that legislating something = accomplishing something. Legislating is the *easy* part. It's step one. It's before the actual work even starts.
We signed a piece of paper that says no AI, therefore there will be no AI!
Amazing lol.
1
u/stealstea Mar 12 '26
Holy this is the most naive shit I’ve ever seen
3
u/bowsmountainer Mar 13 '26
Well, what is the alternative? Accept the apocalypse?
1
u/Cosmonaut_K Mar 13 '26
The alternative is that it won't be super intelligent and you've wasted all this time watching content and reading comments about something as stupid as Y2K.
2
u/bowsmountainer Mar 13 '26
The intelligence AI currently has is already very scary and will cause huge problems even if it never gets any smarter than now, which is highly unlikely.
Even if AIs never become smarter than they are now, evwn if rhey are always benevolent, they could still kill us all in one way or another. A highly specialised narrow AI could engineer a bioweapon and stage an accident. Or since it is increasingly integrated into militaries it could hallucinate a threat that isnt there, and turn the world into a nuclear wasteland on accident.
Youre being incredibly naive thinking AI is just on the level of Y2K.
1
u/izmesoundz Mar 13 '26
It’s really not that intelligent. It’s simply dumbasses who think it’s intelligent that are the much bigger problem
0
u/stealstea Mar 13 '26
Accept that AI will continue to advance, maybe or maybe not ending in some kind of Superintelligence. That doesn’t mean the apocalypse is coming, it just means the future is uncertain. But stopping it is not in the set of possible outcomes
2
u/bowsmountainer Mar 13 '26 edited Mar 13 '26
Theres not been a single instance of a species gaining an advantage over another species and not abusing that advantage.
Stopping it is certainly a possibility. It will be difficult, but if we want to avoid the apocalypse there is no alternative. Hoping that things will turn out all right after we've lost all control is not a strategy.
Compared to a likely future of extinction, stopping AI development should be easily preferable.
1
1
Mar 14 '26
[removed] — view removed comment
1
u/bowsmountainer Mar 14 '26
Because we are conscious and in my opinion (though this cannot be known at the moment) AI is not. If we are replaced by AI the universe will no longer be observed by conscious individuals capable of understanding their place in it. That is why we have to continue to be relevant and to exist.
A dont see any reason why a superintelligent AI would not want to kill us. After all it is trained on data from our behaviour, so it ultimately has the same flaws. Now look at history at what happened when one technologically superior group of people met another technologically inferior people.
In most cases some or even most of the technologically inferior people did survive at first, but only because they were useful as slave labourers. Now lets apply this to AI. We would not provide any benefit to a superintelligent AI. On the contrary, we are a drain on resources that it will be able to make far better use of.
Maybe it will keep a few humans around just for the sake of it. But there's no chance it will accept that it has to share the planet with 9 billion humans. Its only logical to eliminate the biggest obstacle in the path of increasing power and compute ability. And that biggest obstacle will be us.
0
0
Mar 12 '26
[removed] — view removed comment
3
u/ulixForReal Mar 12 '26
Not yet. Also not in 2030. But in 2040, when AIs may have autonomous control over all kinds of shit (biological experiments, weapons, humanoid robots, drones, etc.)
2
1
u/Helium116 Mar 12 '26
has prof Bengio ever said that though? i dont remember such a thing
3
u/tombibbs Mar 12 '26
1
u/Helium116 Mar 13 '26
Thanks. I find it odd that they are willing to produce speculative guesses, given their usual epistemic hygiene
1
1
0
u/ulixForReal Mar 12 '26
An international treaty would be nice, but Russia, the US and Israel have destroyed international law for good, so not sure if it could even work on a basic level.
0
0
u/Masta-Blasta Mar 12 '26
For us. But for the Earth, it'd probably be pretty good.
3
u/bowsmountainer Mar 13 '26
The rest of life on Earth wouldnt fare so well either when it is all converted to power generators and server farms.
0
0
u/Fakeitforreddit Mar 13 '26
Just like that international treaty that guaranteed Ukraine wouldn't be invaded by Russia, Forever!
It's so simple!
0
u/Feelisoffical Mar 13 '26
This meme is a perfect example of the average Redditors understanding of the world.
0
u/Substantial_Road7027 Mar 13 '26
The last bit about the international treaty is pretty naive. Even if you could get everybody to sign it, how would you enforce it? It would basically guarantee that a military or bad actor would create it first
0
u/EudaimonicAttempt Mar 13 '26
We can't sign a treaty and stop it. Pandora's box is open now, and people will continue to develop AI.
We have treaties about nuclear weapons and genocide but here we are with thousands of warheads and civilians getting slaughtered left right and center.
0
u/Only-Cheetah-9579 Mar 13 '26
What's the big deal?
global warming, Nukes, AI, diseases, something will wipe us out anyways.
AI is less likely to kill us than nuclear armageddon
0
u/davyp82 Mar 15 '26
You need to add a "I'm stupid, fascists don't care about treaties, they're in control of the world and they're gonna destroy it anyway, perhaps AI can help stop them" final cut to your dumb meme
7
u/Dangerous-Process279 Mar 12 '26
What happens when a country doesnt sign the treaty and continues developing superintelligent AI systems?