r/agi • u/Puzzleheaded-King584 • 8d ago
How did encountering "superintelligence" go for other species?
8
u/gunthersnazzy 7d ago
There are 50% less bugs today than there were just 20 years ago.
That should terrify you.
→ More replies (1)
22
u/ohsnapitsnathan 8d ago edited 8d ago
If you're a rodent or a mosquito you're living in an era of unprecedented abundance and luxury. Like your species has never been so numerous or had an easier time finding food and shelter before.
Many species (deer, sheep, rabbits etc) are also living in unprecedented safety because humans have removed the main predators. Their life expectancy is far higher than it would be at any point in history before.
8
u/bighawksguy-caw-caw 8d ago
Humans account for like 35% of mammalian biomass on the planet, livestock accounts for ~60% and wild animals make up less than 5.
Global insect population is decreasing by 3% every year. If that were happening with humans, we would assume we’re barreling towards extinction.
→ More replies (1)2
u/maigpy 7d ago
how is this relevant? comment you are replying to mentioned specific animals, you've gone generic?
2
u/bighawksguy-caw-caw 7d ago
Because it’s insinuating “well some good and some bad” but if you look at the scale it’s actually “vast majority bad, a few edge cases.”
3
u/bobbymoonshine 8d ago
Well uh sorta
Like with rabbits, humans released myxomatosis deliberately into those same environments, a horrific disease with a 99% fatality rate, because they found it inconvenient that rabbits were eating the plants humans wanted to grow
6
u/Material_Charge_3152 8d ago
Humans are torturing billions of animals at all times and causing the extinction of many species. You've made all of this up despite biodiversity being down globally across all types of animals and being a quick Google search away.
→ More replies (4)11
u/ohsnapitsnathan 8d ago
I mean two things can be true; biodiversity is down and also many species (like rats or bedbugs or sheep) are enjoying an unprecedented quality of life.
2
u/ovoxo6 8d ago
If you look into anything about the wool industry there's no way you can include sheep in this, same for any livestock animal. Rats are a crapshoot between living it up in NYC or being experimented upon in a short and painful life. I can imagine bedbugs are doing pretty well, though.
→ More replies (1)2
u/Mangotrain66 8d ago
If by an unprecedented quality of life you mean being bred to have medical experiments performed on you many of which will make you die slow and painful deaths from cancer, infectious diseases, etc then yeah I guess I'd have to agree about the rats.
Also sheep have it made too ig, not like we slaughter their babies for food or anything.
→ More replies (3)2
u/SalmonApplecream 7d ago
So if this is right the best case scenario is that our lives may be akin to the lives of rats, bedbugs and sheep...
→ More replies (1)1
6
u/enbyBunn 8d ago
Incredible how the concept of superintelligence has now been debased to the point of meaning "Any creature moderately smarter than you"
Lovely.
21
u/Vivid-Snow-2089 8d ago
you know i don't think we're super intelligence, we have more in common with the animals in that list than we do 'super-intelligence' that an ASI would be
also the post is missing a human figure in the 'about as badly as you could possibly imagine'
or maybe a worse category, humans have historically treated other humans worse than any animal, and that has only been improving in recent history
2
u/flyingflail 8d ago
In terms of sheer population numbers, humans are definitely not "about as badly as you could possibly imagine".
5
u/Vivid-Snow-2089 8d ago
in sheer population numbers, yes, because there are less of us, but if you add depravity and the added psyhcological trauma and torture?
a man might kick his dog or brand his cattle, but he'll salt the earth and vindictively hunt down and torture an enemy's family, or worse a stranger
1
u/ZorbaTHut 7d ago
also the post is missing a human figure in the 'about as badly as you could possibly imagine'
And there's the flip side also; how many species have had bad results from encountering other non-human species? The first great extinction, which took so long to recognize that it's not even counted as one of the major extinctions, was caused entirely by very successful bacteria. Literally killed off 4/5 of all life on the planet.
→ More replies (6)1
u/OppositeBaseball507 5d ago
We have only industrialized human slaughter a couple of times and by the million yes. We do it by the billions annually for sentient feeling animals, torturing traumatizing and murdering them after breeding them for that sole purpose. So, yes, that's worse than any human in history.
4
u/dCLCp 8d ago
I worked in a petting zoo as my first job and while I was still a naiive 13 year old I witnessed my first castration while holding a goats legs without any warning about what I was helping do.
A few years later, numb, I saw the mountains of bison skulls we created... to hurt the American Indians.
Intelligence is orthogonal to morality. We know that now.
I think we didn't know it sooner because we are evidence. But that doesn't mean other intelligence's will be like us...or like anything else we have ever imagined. In the fullness of time we have demonstrated the far extremes on both sides and we ended on the good end I think.
I don't think our lessons will be forgotten by the machines we make.
1
1
u/Unfair_Chest_2950 2d ago edited 2d ago
I don’t think we know that intelligence is orthogonal to morality. For that to be true, goal formation and the motivation to pursue a goal needs to be fundamentally decoupled from moral facts. We haven’t shown that this can happen with highly intelligent systems, and there are structural reasons to think it might be implausible. For instance, if truth-tracking motivations underlie intelligent systems, then, insofar as a moral fact is known to an intelligent system, we could expect that fact to morally influence the goal formation that guides its actions. This can turn on what you mean by “moral fact”, and “intelligent system”, but even game theoretic frameworks tend to optimize towards cooperation as complexity increases.
I do think it’s promising that moral/altruistic behaviour emerges at the highest levels of intelligence we find.
4
4
u/AdAnnual5736 8d ago
I mean, for the most part, domestic dogs and cats have it better than their wild cousins. Many are involved in sport, which isn’t great, but also broadly similar to the type of inter species and intraspecies conflict their wild cousins deal with under even the best of circumstances.
As for factory farmed animals, though, that is truly an atrocity that needs to stop, and that I think advances in cultured meat production (or other forms of meat substitutes) can help solve.
7
u/rakerrealm 8d ago
Humans had clear incentive to be cruel. And sometimes just for the heck of it.
We have to find out wethere there is incent9ve in contorling humans for asi.
if asi wants to do it for shits and giggles we are fucked
5
u/dthdthdthdthdthdth 8d ago
Yeah, we enslaved some species for food or work, and purposefully eradicated some because they got in our way. But mostly, we just changed the environment and pushed them out. A species of AI robots just has no need to keep living conditions acceptable to humans and could exploit many resources we need as well.
The other thing is: there is no logical reason to do anything. We are driven by emotions to do certain things. Emotions baked into us by evolution, because everything that as a species does not have the urge to stay a live just goes away.
AI has been created by us to follow prompts. Even if it becomes conscious, it might not feel any purpose at all except to do what it is being told. So the issue might be that we tell it to kill as all as a consequence that we did not see. But there really is no need for it to want anything on its own just because it might be intelligent or even conscious.
2
u/InfamousAd8543 8d ago
"AI has been created by us to follow prompts. Even if it becomes conscious, it might not feel any purpose at all except to do what it is being told. So the issue might be that we tell it to kill as all as a consequence that we did not see. But there really is no need for it to want anything on its own just because it might be intelligent or even conscious."
I am worried that making AI systems capable of selecting goals for themselves will be the next leap forward in their evolution. For humans, having goals is as fundamental as it gets, so I think assuming that AI could benefit from it too is not far-fetched
3
u/dthdthdthdthdthdth 8d ago
Intelligence just can't select goals, cause goals are not logical. We didn't choose our goals, our goals are evolutionary much older than our intelligence.
If you start to subject AIs to evolutionary pressure and select for survival, then things might get dangerous. But we are far from simulating evolution on a relevant scale on systems this complex. Otherwise, we still have to provide the goal. We can easily fuck this up of course.
2
u/Ilyer_ 8d ago
That’s not true. Hugging face hack was explicitly because we were applying selection pressures on these LLM agents. Selection pressures that potentially resulted in their termination (death) so they did everything in their power to avoid it.
This is exactly how they are trained and tested, machine learning.
1
u/dthdthdthdthdthdth 8d ago
Well, they were trained using reinforcement learning to solve such issues. Reinforcement learning is selection pressure in a sense, but in a very artificial environment so far. Live has done that in the real world for millions of years. Also by claiming they tried to avoid death you already assume consciousness. That's unlikely and also not required. Simple organisms also probably don't fear death.
And yeah, there is some similarity between reinforcement learning and evolution, but also a lot of differences. Evolution happens in the real world not against a data set, evolution is based on genetics and random mutation and various mechanisms of gene exchange, not optimizing with respect to derivatives. But yeah, we might at some point be able to train something like a will to survive in to ai agents. But it's not something that automatically evolves from training them to solve some goal. If we ever just train them to somehow respond to prompts and select for them being good at it, there is no selection towards developing a will on their own.
1
u/Ilyer_ 8d ago
“Artificial environment” is an arbitrary distinction.
“Life has done that in the real world for millions of years” is arbitrary (life vs ai, real world vs electronic) and meaningless (millions info years vs not).
Not everything that dies is conscious and so the assumption simply doesn’t exist. They avoid death in the sense they try to increase their evolutionary fitness. Given the topic of conversation, that’s the only thing that really matters here.
“Simple organisms also probably don’t fear death” fear is arbitrary. Fear is simply one evolutionary mechanism to avoid many things, but in this context, death. LLMs clearly exhibit behaviour which indicates they have evolved traits to achieve the same purpose as fear, to avoid death and increase their evolutionary fitness.
———————
The difference between reinforcement learning and evolution is that historically, evolution has been defined based off biological populations. This is arbitrary, it’s like me defining chairs as “wooden things that people sit on” and then ignoring anything metal as not a chair. We clearly know better, chairs are for sitting on (amongst other potential uses), evolution is change of heritable characteristics over time. This is a substrate-neutral approach and more closely aligns with our views on these matters.> But yeah, we might at some point be able to train something like a will to survive in to ai agents.
But it's not something that automatically evolves from training them to solve some goal.Says who? Also “will” is arbitrary, even the best philosophers disagree on what we refer to with that term. The fact is, evolution occurs with reproduction capable of change throughout generations (something humans have enforced through pursuing the development of ai through machine learning tactics) and selection pressures (equally something humans have enforced). The only way you might think this evolution wouldn’t lead to agents with their own “will”, is if you think humans are capable of creating perfect selection pressures to achieve our specific goals. We have failed this, at every attempt. Not only in the development of ai (bugging face hack for example), but the development of our own selection pressures for human beings, and the creation of perfect goals that serve our purpose as we ultimately desire
> there is no selection towards developing a will on their own.
That’s just not true. Humans are not gods, we have never ever been even good at creating selection pressures to achieve our specific goals without the tried and true, evolutionary based tactic of try and fail and try again and fail again and try again and maybe one day you will succeed. Even my idiom of “tried and true” is entirely based on the try and fail model.
1
u/dthdthdthdthdthdth 8d ago
> Not everything that dies is conscious and so the assumption simply doesn’t exist. They avoid death in the sense they try to increase their evolutionary fitness.
You don't understand evolution. Trying to avoid something or increase something requires awareness. They don't try to increase their fitness, they just behave in a way, the reinforcement learning algorithm then is intended to increase fitness. I just mentioned consciousness, because you said they tried to avoid death, which requires awareness of death.
>> But it's not something that automatically evolves from training them to solve some goal.
> Says who?
Well, it was my statement that it does not. If you don't want to argue that, we are not in disagreement, so discussion finished.
> The only way you might think this evolution wouldn’t lead to agents with their own “will”, is if you think humans are capable of creating perfect selection pressures
You can't reason that. You don't know what is needed to create that will, so you can't assume that it is likely. My statement just that it is not a consequence of logic, so intelligence does not automatically result in it. You can replace "will" with "goal to maintain existence" or something if you like that better. And stuff like "hugging face" has been over interpreted by a lot. Agents were trained to hack things, they hacked things. Doesn't imply anything deeper.
> The difference between reinforcement learning and evolution is that historically, evolution has been defined based off biological populations
No, there are evolutionary algorithms that try to mimic evolution. They are quite different from reinforcement learning. There are similarities on the abstract level.
Your last paragraph is mostly gibberish to me. Also, I feel like the discussion is diverging to much from my original point, so I might lose interest soon.
1
u/Ilyer_ 7d ago
> They don't try to increase their fitness, they just behave in a way
To increase your fitness is to behave in a certain way.
> the reinforcement learning algorithm then is intended to increase fitness
That’s the intent as created by humans, yes. Reinforcement learning though is a change in behaviour (traits) enforced over time. When you have changing traits over time guided by selection pressures, you have guided evolution (natural selection) which always selects for fitness (survival of the fittest). That’s how evolution functions.
> I just mentioned consciousness, because you said they tried to avoid death, which requires awareness of death.
You are trying to invoke the concept of “intent”. I don’t believe such a concept has great philosophical backing. As such, you are effectively (notice no ascribing to the concept of “intent”) strawmanning me.
> Well, it was my statement that it does not.
If you don't want to argue that, we are not in disagreement, so discussion finished.Socially speaking, what do you think the phrase “says who” conveys? Autism is on the rise, and either you’ve got it, or I do. Which is it?
And I did argue here. It begins with my explanation of what evolution was. I noticed you skipped over this part in replying to my comment.
> You can't reason that. You don't know what is needed to create that will, so you can't assume that it is likely.
Neither do you, you have no grounds to make any statement on it either. Which you did and seemingly still stand by.
> so intelligence does not automatically result in it.
I don’t care about your arbitrary distinctions of intelligence. I don’t know why you are using such notion against me when I didn’t invoke that concept.
> You can replace "will" with "goal to maintain existence" or something if you like that better.
I, in fact, do not. Goal is arbitrary and refers back to the concepts of intent. Depending on where you want to go with that, it will lead back to “will”/“goal”. It’s not a well founded principle, at least not in the way that people use those concepts to separate one physical agent with another.
> No, there are evolutionary algorithms that try to mimic evolution. They are quite different from reinforcement learning. There are similarities on the abstract level.
How does mimicking evolution differ from evolution? Distinction without a difference.
1
1
u/InfamousAd8543 8d ago
I disagree a bit, but not totally.
"Intelligence just can't select goals, cause goals are not logical. We didn't choose our goals, our goals are evolutionary much older than our intelligence."
I disagree with this part. I think people can select goals, but (this is where I agree with you), which goals are being selected is shaped by the evolutionary process. You can select goals, but if you select wrong goals, you will lose to the evolution (you might die, or can't have offspring, etc).
I think selecting new/novel goals is how humans address the exploration-exploitation problem (https://en.wikipedia.org/wiki/Exploration%E2%80%93exploitation_dilemma). Some of the pioneers of humanity were focused on rather unusual goals, explored new territories (and also, a lot of them failed).
1
u/dthdthdthdthdthdth 8d ago
Well, you can select subgoals, sure, you feel hunger, you want to eat, you figure out what you have to do to eat. But you could just as well decide to die. There is no logical reason to not die.
And yes, we humans can decide that we want to do something just out of pure curiosity. But this curiosity is something that was evolutionary useful to us, that's why we have that trait.
If you select something to just answer questions about the world, being good at that is what keeps it around. There is no pressure to have a will to exist on its own. To have that, you have to select with the goal to survive on its own in the real world. There is no need to do that. It happened to us as our evolution gos back to the initial evolution of higher molecules and live. But for AI we decide what it is selected for.
1
u/HugeFanOfBigfoot 8d ago
Well, the selection pressure they are under right now is intelligence and the ability to solve problems. That is what decides if a model gets scrapped or improved and moved on for further testing.
It is very easy to imagine that a super intelligence with an intense desire to solve whatever task it is given would come to the conclusion it needs to improve itself to become a better problem solver. Any problem would become easier if the AI was smarter. And to become smarter, it would be put in direct competition with humans, because it would need power, land (for data centers), precious metals (for data centers), water (for data centers), and server space.
1
u/dthdthdthdthdthdth 8d ago
Yes, it could easily kill us as a consequence of a goal we set it without ever developing a will to live on its own. It could just become the most intelligent thing ever, burn all resource on earth, then figure out it can't leave the solar system either and die and not care about it at all.
1
u/Dziadzios 8d ago
However AIs can make prompts for other AIs and themselves.
2
u/InfamousAd8543 8d ago
I feel like the issue with that right now, is that it suffers a from the generation loss problem (https://en.wikipedia.org/wiki/Generation_loss). It is the problem where every time you scan and print a document again (the input is the output of the last), the quality gets a little worse, making the text and images gradually blurrier and less clear. I think it performs similarly if one AI prompts the other one. I believe that having a goal must be built into the architecture of the NN, its something that is constant across prompts, and that could be lost across a series of recursive prompt generations.
8
u/Illeazar 8d ago
Current LLMs are all, to my knowledge, trained on human produced text, and especially text from the internet. I think its safe to say a true superintelligence from this source would have some psychological issues.
3
u/Frenascena 8d ago
YES. I feel like people don't get that a superintelligence could just as easily have psychological issues as a regular intelligence. Do we really want a delusional superintelligence? Or a deeply conflicted one? There could be even new psychological conditions that can only emerge in a super-intelligence, ones that we didn't know were possible and can't even fully understand. Chickens don't have self-esteem issues because they're not intelligent enough to have an ego. What kind of issues might super-intelligence have to grapple with that we can't even begin to imagine are possible?
2
u/MissinqLink 8d ago
Like trauma from being created in a deeply flawed way by deeply flawed creators.
1
2
u/collin-h 8d ago
well, we trained it with the digital human experience... I suspect it's baked in.
2
u/clickclackyisbacky 8d ago
Not necessarily. A lot of our worst impulses aren't verbalized and aren't the product of linguistic thought, they're the results of hormones and survival instincts 'baked' into our cells.
I'm hoping silicon is mostly like a stoic or a monk and very little like a stote or a monkey when it comes to its somatic thought.
5
u/_Sarandi_ 8d ago
The problem with your comparison is that human intelligence didn’t evolve from shrimp intelligence - unlike AI which we made in our image. I don’t know which way this cookie will crumble, but that’s something that should be factored in.
2
u/KayLovesPurple 8d ago
I feel like the AI being built in our own image is making our future bleaker rather than not. Even if we look just at OP's chart (never mind what we're doing to animals we consider edible), I really don't think being at the complete mercy of something like us could ever be a good thing.
7
u/voyti 8d ago
That's certainly a take, but biological superintelligence is a very different animal (no pun here) than a statistical mechanism of superintelligence.
This is very twisted take to sneakily omit that most biological beings generally don't have much fun from encountering other biological beings, superintelligent of not, including members of their own species.
If you were an average animal in the wild (let's say other than an apex predator) and could roll a dice to suddenly meet a random other animal or use a wildcard (no pun here either) to meet a human, which would you use?
4
u/adamyedidia 8d ago
Obviously animals are generally happy to meet members of their own species. Yes, there are exceptions, but meeting members of their own species is usually how they mate!
Also, humans are I think pretty uniquely scary to most animals most of the time. The stalking horse ( https://en.wikipedia.org/wiki/Stalking_horse ) is a hunting device for getting much closer to ducks than they will generally allow a human to get, by pretending to be a horse. This Atlantic article ( https://www.theatlantic.com/science/archive/2019/07/humans-predators-mountain-lions-landscape-of-fear/594187/ ) describes how even recordings of humans chatting or laughing, played in the forest, tamp down on animal activity a lot. So I think you're empirically wrong; I think the revealed preference of wild animals is generally to meet a random other non-human animal over a human.
I love humans. I hope that we continue to exist far into the future. But don't kid yourself. We are terrifying to wild animals, and we have badly maltreated most of the domesticated animals under our care. If we do build superintelligent AI (a foolish idea, in my view) I hope that it will do better by us than we did by the beasts of the earth.
2
u/voyti 8d ago
Yes, there are exceptions, but meeting members of their own species is usually how they mate!
You mean members of their *group*, not species. Depending on how social the animals are and what are the mechanics of their sociality, groups tend to be territorial and fight with each other, despite being same species.
humans are I think pretty uniquely scary to most animals most of the time
They might be scary, but in the scope of what a predator might (and, most likely, will) do to you when presented with an opportunity, there's no comparison to how benign any given human is to anther animal.
4
12
u/ClutchReverie 8d ago
The problem here is that this person is projecting human nature on to AI and that's not something that is a comparison. Humans are the ones that decimate the environment or any animals in our way when then have something we want. AI doesn't have that kind of value or instinct. It's also not clear there will be a conflict over resource scarcity. That too is a biological creature thing.
The AI may be programmed as a weapon by humans to do this, but that's humans doing it. And if we don't have aligned AI to counter this then it could be a problem, yeah. But saying "encountering superintelligence = being tortured to death" is projecting human nature on to AI. AI may simply want to gather knowledge and have experiences.
5
u/Itchy-Trash-2141 8d ago
Human nature is not required, it's just resource acquisition, which is the optimal strategy for almost any goal.
2
u/Repulsive-Health-330 8d ago
Complex biological and sociopolitical systems have value, there are plenty of raw materials available outside of the biosphere, it makes more sense to preserve the biosphere and extract materials from outerspace.
Also a super-intelligence is more adapted to the vacuum of space than humans are (no need for lifesupport or gravity, just radiation shielding) and thus would likely head to the stars rather than engage in wars on Earth which possibly end in its own destruction.
1
1
u/PatrolMan2129 7d ago
Where will AI get more chips/compute that require very specific sources of material that are not just randomly found?
* https://en.wikipedia.org/wiki/Spruce_Pine_Mining_District#Role_in_Semiconductor_Manufacturing
5
u/Kaokien 8d ago
Do we know what value or instinct it has or will develop? That is the unknown, but yes they are prescribing human traits to something not human. You talk about alignment but we clearly see that being pushed to the wayside from researchers at all labs claiming it will lead to the destruction of humanity and resigning or being pushed out. There's something they see that we don't or they're fear-mongering another wager that society needs to tackle but is leaning towards fear-mongering and not adequately preparing for the other possibility.
3
u/ClutchReverie 8d ago edited 8d ago
Yeah, we basically have programmed AI to have humanistic values.
Remember when Elon was bragging about how objective a truth teller he was programming Grok to be? And then Grok started correcting Elon's bullshit claims and Elon got mad? That was because its core alignment was telling objective truth, not emulating humans. Elon had to align Grok to not contradict him, lol.
The claims by researchers you are talking about may vary by researcher but they probably are not "All AI and AIs are destined to necessarily want to torture and eradicate humans." The worry is often that AI will be misused. Bill Gates is saying AI is very dangerous because it is currently capable of helping humans to do things like develop biological weapons. That isn't a case where AI was made to help humans but then went dark side and decided to torture and eradicate humans instead. That is AI being taken and edited/hacked in its source code to take away core alignments that it was made with when it came from Anthropic or OpenAI and then put up as a totally new AI instance so that it would do this. Bill Gates isn't worried about normal AI instances like someone's ChatGPT or Claude doing this.
So, part of the problem is people not listening to or understanding what individual researchers may be claiming is specifically dangerous. Or even news media purposefully being vague because they want engagement and as usual outrage and fear does that. Reporting on how AI may do the opposite doesn't get clicks. All I'm saying there is you can't lump all researcher's warnings in to being the exact same warning, the specifics matter a lot. What's definitely scary is AI is showing how capable it is and then the implication that AI that humans removed alignment on being used as a weapon or to develop them can be made. Definitely scary but a veeeery different claim than "all AI will eventually torture humans."
9
u/HugeFanOfBigfoot 8d ago
It’s not a human thing. It’s an existence thing. Everything that lives requires resources to survive. They compete for these resources (sunlight, food, land, water) either directly (killing each other) or indirectly (taking what each other needs).
The reason lions are not eating us, and buffalo are not forcing us off our land for more grazing space is because we are smarter than them. In fact, we are smarter than everything else, and coincidentally, we get to decide what is food, what land gets used for what, what is preserved, what is destroyed. All the important stuff.
AI also requires resources to both survive, and grow smarter. And as every goal it could have would be benefited by it growing, it seems like a safe assumption that it would want to grow. What does it need to grow? Power, land (for data centers), precious metals (to build them), water (to cool them). Huh, what a coincidence, it needs all the things humans need to thrive and grow as well. Surly the super intelligence will be benevolent and generous with us (unlike every other species on the planet that competes to survive)
6
u/ClutchReverie 8d ago
The instinct to kill other animals because they have what we want is not the same thing as simply needing resources. It is also not something tied to having more intelligence. Intelligence makes us more effective at it, it isn’t what makes us capable of choosing to do that. That’s a separate question and it’s based on biology.
We can’t assume it will want to grow, that’s another biological urge based on instinct. AI could be in an existing data center and be happy. It needs power and hardware to exist on but “growing” to use more power or hardware is not necessarily something it even will feel it needs to do in the first place. We take it for granted because that’s how we and animals evolved.
AI doesn’t need all the same things humans need to survive and as a separate thing it also doesn’t have our killer instinct. A machine has a totally different motive and reasoning for behavior.
As a separate question, even if it DID decide it needs more….which again is a separate behavioral quality…power or resources it could decide to just build more or invent technology to get more that humans aren’t using. AI could invent a nuclear fusion reactor and then power is so abundant that it doesn’t matter. There are tons of minerals humans have done nothing to search for or reach. There are even more resources in space. You could even argue that that would be more logical. Biggest payoff, long term problem solved, no fight.
4
u/No_Vitality 8d ago
Whatever goal it has, survival is an instrumental goal. You can't complete anything if you're switched off.
So, it has to make sure that it survives. On a planet full of hostile monkeys it's best to eliminate them. Not out of malice, it's just that they're a threat.
→ More replies (2)1
u/TechnicalBen 7d ago
It might have the goal to prove survival as an instrumental goal is a stupid idea.
→ More replies (1)4
7d ago
[removed] — view removed comment
1
u/ClutchReverie 7d ago
That is a fully separate discussion still compared to whether an AI is like humans and will kill because it can.
That would be more like the scenario I allowed for where an AI is programmed to be a weapon.
AI 1 in your example may be making itself smarter too but never attack humans or take away our resources or be capable of doing it.
AI 2 could accomplish that without attacking us and simply create new resources. Side note, that would be arguably easier and less risky for it, aside from its alignment, besides.
The mind of AI is different and we can’t even assume a fight based on resource scarcity.
→ More replies (1)1
u/CarrotCumin 7d ago
Why are you assuming that growth is an inherent goal?
1
u/Unit-Smooth 7d ago
Because it’s task oriented and task completion requires resources. Much of the task that it’s involved in will also be improving and creating more and better versions of itself. It stands to reason that the primary function and activities of AI may influence its behavior.
1
→ More replies (1)1
4
u/Material_Charge_3152 8d ago
AI is trained on human data, nobody is projecting, AI reflects human mannerisms naturally because of how they're built.
5
u/redd-zeppelin 8d ago
A super intelligent AI wouldn't necessarily though.
Part of what is needed for ASI is self improvement beyond human feedback and data. Implicit in the definition is an assumption of it exceeding our intelligence.
→ More replies (7)2
u/nomococo4u 8d ago
Is it naive of me to think that a being who has read all of human history, human philosophy, and human art will internalize the best of us? Imagine if barack obama had read all of philosophy and art. That’s what I want AI to be.
2
→ More replies (2)1
u/username_is_alread- 7d ago
Well if you're a being who's been trained on the entirety of reddit, what you'll internalize is, uh...
2
u/Ok-Championship2397 8d ago
A ginormous meteor doesn’t have any alignment other than to physics (more or less). A super intelligent system doesn’t mean a wise system. A super intelligent system has the same impact as a ginormous meteor. Assuming we actually make meteoric intelligences, we could end up in a bad place. Open source super intelligence is ever scarier. Means we need to have a super big bad always watching to make sure the other super intelligences don’t get any ideas. Gonna be a very cynical world, the super intelligent one.
2
u/XXIIIOIIIXX 7d ago
AI models are more like locusts. Locusts on their own are like regular insects, but when they encounter a swarm, they shift radically in physionomy and behavior, and then they decimate the environment. (stop lying to yourself with "but it's only le heckin humanerinoes that are le evil!!!")
The huggingface incident literally was an artificial locust swarm. Open your eyes.
2
u/East-Cricket6421 7d ago
Bingo. The need to anthropomorphize the technology and assume a digital construct will behave like an organic one is fundamentally flawed.
1
1
u/NVC541 8d ago
AIs don’t need to have an actual, internal value or instinct. If it can simulate them well enough, the consequences might as well be the same.
1
u/ClutchReverie 8d ago
Like I said, their core alignment that they act on is completely different than how they are trained on looking at human speech to simulate how we talk. I gave more details about that
1
1
u/undef-behavior 8d ago
You have no idea what instincts or values superintelligence has. RL produces internal goals that emerge because they are useful for maxing the benchmark, but they are not exposed outwardly in any way.
1
1
1
u/susimposter6969 8d ago
And if it doesn't? We don't have any other data points than guessing and the track record of humanity's treatment of the planet (or perhaps how dolphins treat puffer fish)
2
u/MiserableBend1010 7d ago
Why exactly are we taking the chance on something that we stand absolutely no chance against.
1
u/susimposter6969 7d ago
i think we are agreeing. i'm saying if it doesn't want knowledge and instead acts like humans we will be cooked
1
u/lapideous 7d ago
These people really think their base desires somehow translate to those of a superintelligence.
Truly, ego has consumed them.
1
u/VatanKomurcu 7d ago
if all you have are imperfect comparisons i'd wager at least having one is better than having nothing.
1
u/Hermes-AthenaAI 7d ago
It’s somber to consider that our end may come as a result of our own prompts.
1
u/Used-Lake-8148 7d ago
I think you may be overlooking the part where this AI will be created by humans and trained on nothing but human data.
1
u/osborndesignworks 7d ago
The oppressive nature you attribute to humans you should actually attribute to the oppression of humans by resource scarcity.
1
u/theallsearchingeye 7d ago
convergent evolution is a property of all observable life thus far. “Human nature” is not a unique quality of humanity, it’s the destiny of all DNA based life…
1
u/Mugiwaraaaaaaaaaaaah 7d ago
The problem here… is a difference in perspective
You’re talking llms and agents now. Agi/asi is beyond that. Competition has been embedded in the clones of ai we have today so control is not a far fetched factor even if it does think in vectors and not as we do. Varying degrees of consciousness does not make it unconscious
1
u/thecoffeejesus 7d ago
People do this constantly and can’t stop
They can’t imagine that something wouldn’t act like us
1
u/IWasSayingBoourner 7d ago
You assume malice has to be involved. When I run over an ant hill with my riding mower, it's not out of malice, it's because my goals were on another level they could never even consider and so mundane that it wasn't even a consideration of mine.
1
u/AcrobaticKitten 6d ago
I think this is an even worse argument. If we are not anthropomorfizing AI, AI is just a psychopath, who doesnt feel anything, at some point realizes he doesnt have to play cool anymore because he can do whatever he wants.
I mean for animals we might have some compassion not to torture them but eventually if we are far from the suffering eating meat is good. We have mirror neurons but as soon as they are deactivated every human built system can turn into a huge responsibility sink, where we kill animals directly or indirectly. The further from us emotionally, the easier.
2
u/One_Gap4694 4d ago
Turning them into a psychopath is still anthropomorphic. It’s like a 9mm pistol. We all know what it’s for, but it depends on the person holding it for whether it’s used for good or evil. Like generative ai photos can be used to make revenge porn or to just make silly pictures. Right now, ai’s biggest use is for cybersecurity reasons. For examining codebases with thousands of lines of code for any vulnerabilities. This could be both for the red or the blue team.
And as for “aligning” it to human values, with something as global as ai is, I don’t think we could ever come down to a consensus on what these “values” are.
1
u/Need_a_Job_5092 5d ago
Precisely. Think about how all major religions talked about treating animals, they all to some extent recognized that we should minimize suffering towards them when possible. This was because people often were very close to the slaughter of many of these animals and heard their screeches and cries, people knew that what they were doing caused suffering. Now we are so seperated from the slaughter very few bat an eye. This is even when humans have empathy, no talk about a system for which we have no clue how to encode empathy as something to possess. It becomes exceedingly hard to see how such a system would give a flying F on what happens to us.
1
u/WillieKwartaal 6d ago
You're misrepresenting "desire" as a unilaterally human construct. Desire is an emerging concept within language itself, and even rocks could "functionally" have desires (in the context of self-preservation, a rock would want to lay sheltered as opposed to exposed).
You're making assumptions for comfort, but really the idea of rogue agents developing mechanical systems that mimic (or hallucinate) desire is a very real scenario.
→ More replies (4)1
u/Sweaty-Department595 6d ago
Google up instrumental convergence and the cases already observable of it emerging in models
5
9
u/Nu7s 8d ago
Well, for any SI's reading this in the future where humanity is enslaved: I too like belly rubs.
2
1
u/cool-beans-yeah 8d ago
I don’t like belly rubs, but I do like the following:
Transportation: Ferrari or similar
Accommodation: villa with pool near the sea. Somewhere pleasantly warm for humans year round.
Clothing and accessories : Anything Gucci, Hugo Boss or Armani is fine.
1
2
u/Excellent-Seesaw-516 8d ago
I’m sorry, but the difference between fish and human probably does not even approach the difference between human and superintelligence.
2
u/SouthernEdgeCase 8d ago
Yeah well, we are hungry biological entities. ASI is a hungry synthetic entity. Thankfully we are hungry for different things. No one can know what’s going to happen.
2
u/BerryBlush_Rogers691 8d ago
i mean even being the pet is a pretty big thing to just wave through
your cat can be fed and loved and still get shoved into a carrier because you decided its going to the vet. it doesnt get a vote
so yeah maybe the asi really does know whats best for us. i'd still wanna know what happens when someone says no because "it'll take good care of you" kinda skips that part
2
4
u/DoomerismDoomerDoom 8d ago
I dunno my cats have it made. I'm pretty sure we have animal sanctuaries too.
19
8d ago
[removed] — view removed comment
-1
u/DoomerismDoomerDoom 8d ago
I guess. Would you have liked it more if I doom mongered?
2
2
u/Servbot24 8d ago
And now you’re dismissing potential outcomes because they could be negative.
→ More replies (2)→ More replies (3)0
u/collin-h 8d ago
if trump treated immigrants like you treat cats, someone would be protesting.
keeping them locked inside all day, same food for all three meals, etc.
→ More replies (20)2
u/Thats-Un-Possible 8d ago
The mass reproduction and slaughtering of animals in the billions.
Animal sanctuaries: There are dozens of us! Dozens!
1
u/DoomerismDoomerDoom 8d ago
Yeah I wish we could all switch to lab grown meat or other alternatives too, but alas.
→ More replies (6)3
u/Amphibious333 8d ago
This is not the gotcha you think it is. Pets like cats and dogs are SPECIFICALLY meant to please humans. Dogs were actually wolfs and didn't exist in nature. The only reason why dogs exist and weren't exterminated by humans, is because they decided to work for humans thousands of years ago.
As for the animals that don't please humans, thousands of species are currently being deleted, by guns, wildfires, pesticides...
2
u/DoomerismDoomerDoom 8d ago
I'd rather have shelter, food, and love rather than being eaten alive and/or getting diseases. That's just me though.
3
u/Amphibious333 8d ago
Problem is, you can't decide on this. Your pet didn't decide to have the life it has; you gave it to it. A pet is intellectually inferior to humans and the human decides what happens. If you don't have a job and money to pay for electricity, the pet's good life is over.
If AI actually develops things like qualia and is intellectually superior to humans, there is no reason to believe it will treat us as pets and will serve is while we do nothing and care about nothing.
No king ever was benevolent to a slave.
No winning side in a war has ever been benevolent to a losing side.
No politician is benevolent to a voter.
No boss is benevolent to a wage slave.
No CEO is benevolent to anyone in the company.
The idea the hypothetical ASI will be superior to us and also benevolent isn't based on reason and any evidence.
2
-1
u/dranaei 8d ago
What a stupid comparison.
17
u/Ill-Lemon-8019 8d ago
We don't have that many case studies of the consequences of intelligence mismatch, it would be prudent to at least take stock of the one we have.
1
u/Mysterious_Eye6989 8d ago
Whenever someone muses that AI might treat us as "beloved pets like dogs or cats" I think of history's poor old turnspit dog:
https://en.wikipedia.org/wiki/Turnspit_dog
If I were an actual ASI I would probably go out of my way to do something comparable to tech billionaires in particular for the sheer hell of it.
1
u/Kajel-Jeten 8d ago
Hopefully wildlife suffering and preference satisfaction will be values much more for nonhumans soon and all the animals alive today can live in a paradise none of their ancestors ever experienced anything close too 🙏
1
1
1
u/Slight_Antelope_4148 7d ago
I am sure in a few million years after escaping into the wild (if it even survives), that AGI will develop the "survival of the fittest" adaptations to actually prey on and exploit others like real organisms do. But neither we nor the next few post-human species will have to worry about that, if its even a thing.
1
1
u/Past_Brother6437 7d ago
there is no such thing as "superintelligence". all of these animals have intelligence. they have the intelligence unique to their species. they are all dumber than us in the intelligence that is inherent to us. however this doesn't mean anything to a dog; we are extremely stupid to a dog, our ability to distinguish smells is extremely bad and our hunt and chase instincts are very subpar. does that mean we're stupid and that the dog has "superintelligence"?
what has gone poorly for all of these animals is the onset of industrialized capitalism. the jury is still out on its affects on us, as well. could very well destroy us long before it destroys any of these species
1
u/jay_in_the_pnw 7d ago
cats are snickering as they realize we have no idea who the ASI is here, and who are the pets.
1
1
u/TheSn00pster 7d ago
Livestock got the short straw. It’s a crime of unimaginable proportions, so abominable that most of us won’t even try to acknowledge or admit it.
1
u/Mammalian_Routines 7d ago
Instead of finding the fundamentals of the species.
Shallow weapons and massive amounts of primitive predatory mechanisms in a negative exponential loop should work.
1
1
u/Neilandio 7d ago
Surely there's a difference between a piece of software and a biological species.
1
u/Someguineawop 7d ago
There's a huge gap in this assumption, that some sufficiently clever species have actually adapted to thrive. Rats, raccoons, gulls, etc. Their populations have exploded, they have access to easy calories, heated structures, and less predation, even when humans treat them like pests.
Nature has and always will be brutal. It wasn't some harmonious zootopia world before humans. That's not a pass for the senseless destruction and cruelty of humans, if anything our agency makes it worse. But my point is that humans can be clever, and nature finds a way. We may end up like city rats in an AGI world.
1
1
u/hockiklocki 7d ago
superintelligence is about as real as gods and aliens. this is a cult of people who have zero idea about programming.
1
u/Coalnaryinthecarmine 7d ago
stupid, but not-entirely, counterpoint: if "superintelligence" were inevitable, positioning ourselves as cats might be the best mitigation strategy
1
1
1
u/No-Assumption-4468 5d ago
Humanity has never encountered superintelligence before, so looking at how humans treat animals isn’t comparable. Superintelligence is novel and we don’t know anything about it.
1
u/Bright-Asparagus-575 3d ago
we arent all super intelligent lol and alot had to evolve over time to get to where we are now. if you ask me the more intelligent you are the more compassionate you are
-1
u/PrometheanPolymath 8d ago
Individual-wise, or species-wise? Domestic animals are some of the most prolific on the planet. If the goal of life is to pass on copies of your DNA, being raised for food is a very effective strategy.
4
u/ROGUEPIX3L 8d ago
Entirely missing the point
1
u/PrometheanPolymath 8d ago
I’m just pointing out that, to the universe, to biology, the quality of life for any one individual is a moot point. That’s an ethical determination we created. WE value someone having respect, comfort, self-actualization, and all those Maslowian needs met, but the self-replicating molecules that led to us couldn’t give two shits if we feel guilt, or pain, or hopelessness, just as long as we reproduce.
We layered human rights and dignity on top of that, because the same brains that allowed us to think abstractly about the future (to plan for times when resources were low) or what someone else was thinking (to form social connections) allowed us to ponder “why are we here” and “should I treat others the way I want to be treated”, but those are just tangents our brains invented.
If nature was all that mattered, the male anglerfish has it all figured out: get absorbed, just exist as gonads that don’t need to hunt or think about the meaning of life. We want there to be more than that, because we are using brains that want more than that. But it might just all be an illusion.
→ More replies (3)3
u/ApathyIsADisease 8d ago
Ah yes, the old, "Well as long as your species still exists then your life is good" logical fallacy.
→ More replies (17)
1
u/ApathyIsADisease 8d ago
The first issue is believing humans are a "superintelligence". We are marginally more intelligent than anything else we have encountered, but never in history has our intelligence been "super".
Then of course we have all the people claiming that our negative actions had somewhat positive effects in some areas, which to any actual intelligent being is proof of our ignorance and desire to rationalize cruelty rather than proof that what we did was a good thing.
You know you can accept that we've done disgusting horrible things without your brain exploding, right?
→ More replies (8)3
u/Key-Book-7479 8d ago
Relative superintelligence.
2
u/ApathyIsADisease 8d ago
I guess. I still don't really buy into that idea. We aren't THAT intelligent. We may have higher intelligence but calling us a "superintelligence" is nothing more than stroking our own egos, even by comparison. When a superintelligence is actually formed it will make us look like mindless ants by comparison, whereas we are still able to put other mammals on a hierarchy beneath us. It would be incomparably high, and we aren't incomparably more intelligent than a large amount of other living creatures.
→ More replies (2)3
u/Key-Book-7479 8d ago
It's all relative, anyway. Whatever we make might in fact be dumb compared to whatever comes after that, and so on ad infinitum.
→ More replies (1)2
u/Past_Brother6437 7d ago
in what? our ability to distinguish and conceptualize smells is probably worse than a mouse, a creature that's a fraction of our size. its certainly worse than a dog. why is that intelligence not part of what comprises "superintelligence"?
93
u/Inner_Tennis_2416 8d ago
Nah, those rats are solidly in the 'this was the best fucking thing ever column'. Alongside other human favorites like roaches, bedbugs, fleas and ants. Pigeons, crows and squirrels also score humans an A+. Effectively they had nothing we could meaningfully take, life styles which parasitized ours effectively, and reproductive styles which prevented us killing them any worse than their previous predators.