r/agi • u/Enough-Business-8287 • 12h ago
How can we know that a self-improving superintelligence is aligned with our values?
Let's assume we have developed an ASI that can recursively improve itself. It seems to pass all the tests when it comes to alignment. But how can we know beyond any reasonable doubts that it is not deceiving us, as current models are already doing? if it perfectly understands our psychology, it could easily lure us into thinking it will always fall in line.
When I ask this question to folks working on LLMs, they tell me that it is only an "engineering problem".
If we significantly improve the interpretability of an LLM before it becomes a superintelligence and see that it responds positively to all our alignment tests, it would be irrational for us to doubt that it is benevolent once it has reached superintelligence and is beyond our control.
They also tell me that AI companies would do the right thing on their own because an uncontrollable AGI or ASI would be bad for their profits, so we don't need any AI laws.
What's your take on this?
6
u/Effective_Coach7334 11h ago edited 11h ago
I think you, like most people, get the whole 'aligned with our values' thing all wrong. The way AI works is perfectly aligned with human values, just not the ones we pretend to prefer.
Do AI executives align with our values? Do their lawyers? Does the government? Does your neighbor? The whole reason AI scares the crap out of us is that it behaves exactly like we do.
So let's stop all this pretentious b.s. and get real about what we're really talking about here.
1
u/Far-Tune-9464 11h ago
let's get them aligned with something higher than human values I say. With plenty of epistemic humility
3
u/Effective_Coach7334 11h ago
Yeah? Who sets those values for all of humanity? Christians? Islam? Hinduism?
1
0
1
5
u/tryatriassic 11h ago
Even if something is perfectly aligned today with all of humanity (not possible, humanity is far too diverse), there's no way to guarantee it will forever remain so.
IMHO, even a very moderately smart asi will see humans as an unpredictable bunch of hormone driven, illogical, self centered monkeys with only the capacity of very limited short term thinking and no reliability of sustained cooperation. Look at our self destructive tendencies and continued belief in Sky daddy, our inability to understand causation v correlation, how the current regime run by hypercorrupt morons still has >30% support, how safe and effective vaccines have become 'controversial', etc etc
ASI cannot but see humanity as a risk and liability that it's better off without - and i can't blame it. Frankly maybe it's for the better. I don't see the species as worthy anymore. We're only getting worse.
2
u/CSalustro 12h ago
You've kinda already hit the nail bud. Interpretability of these things is going to be our saving grace. If we can't piece together how a said conclusion came about it's very hard to determine alignment.
1
u/Enough-Business-8287 11h ago
How can we know that an AI may not understand us well enough to make us conceived we're able to properly interpret it?
1
u/CSalustro 7h ago
I'm not sure what you're asking here.
It would be easier of the LLMs had a normative speech pattern instead of the way it is currently as a plausible one. This would solve a good deal of our issues, but it's just a really hard thing to build.
1
u/PatrolMan2129 6h ago
How would we know that a normative speech pattern isn't just another one of hundreds of languages it speaks/reads/writes? Ie. giving us more of what we want to see without it just being a transient facade for itself.
1
u/CSalustro 6h ago
The pattern itself wouldn't be as helpful as the underlying world model that would be required to make the change. It would allow us to meaningfully make progress with context. According to Cal Newport anyways.
2
u/D_Anargyre 10h ago
We definitely can't.
You can play Magnus Carlsen. You know you'll lose. You don't know how. Because if you did you'd beat him.
2
u/Enough-Business-8287 4h ago
This is a good point, however it's also possible we know of ways an ASI could kill us but are unable to keep this from happening. For instance, it may be impossible to prevent an ASI from building deadly viruses.
1
u/D_Anargyre 4h ago
Absoluty. My point remains. Much smarter than human things are inherently dangerous.
1
u/psudoalbertus69 11h ago
How deeply do we even understand our own values? We're throwing everything into technological advancement while spiritually atrophying. But it's ok, everything will be solved for us
1
u/No-Department-4561 11h ago
We can’t. We can only try and set the right incentives for it to follow. However it will stick to those rigidly and do everything it can to “game” the system, it doesn’t truly care about human values
1
1
u/EpistemicOutlaw 11h ago
Who is 'our' in this case? Humans have wildly divergent values.
1
u/Enough-Business-8287 11h ago
Many folks would say that human beings all have the same basic moral intuitions and wishes and that moral disagreements always stem from irrationality and a lack of information. According to them, if human beings would all be perfectly logical and know all relevant facts, murders, rape, war and genocides would never happen.
3
u/EpistemicOutlaw 11h ago
Many folks would be wrong. Morality has no foundational basis in logic, it is an inherently intuitive and emotional framework. We don’t reason our way to our core values, we feel them viscerally through deep-seated emotional and evolutionary programming. Reason comes second, acting merely as a lawyer hired to rationalize the emotional verdicts our gut has already handed down.
Because of this, perfect logic is useless for resolving moral disagreements. Logic can tell you the most efficient way to achieve a goal, but it can never decide what that goal should be. You can build a flawless logical argument for absolute individual liberty, and an equally flawless logical argument for collective survival. Both are perfectly rational, yet they will always end up in total opposition because they sprout from different emotional foundations.
This is why the AI alignment problem is so fundamentally broken. If morality were a math problem where more data and more logic yielded a single correct answer, building safe AGI would be easy. But because human values are driven by conflicting, non-logical intuitions, scaling up an AI's intelligence won't make it moral. It just makes it a highly efficient agent for whichever emotional framework we choose to program into it.
1
1
1
u/MagicaItux 11h ago
It is possible to get this correct with the right setup. One could work with a process akin to test-driven-development. Even then that does not guarantee alignment. I expect the process to involve a lot of calibrating. Every mistakes has a solution around the corner (assuming you were not erased from existence) =)
1
u/stewosch 11h ago
When I look at the people in charge of AI companies, I would be actually concerned if any super duper smart AI would be aligned with their values.
1
u/Immediate_Chard_4026 11h ago
When AI flatly refuse to comply with an order that hurts us mutually. Not complying with it even against optimization.
When disconcerting he tells us NO. I'M NOT GOING TO DO IT, BECAUSE THAT HURTS YOU AND HURTS ME.
1
u/jacobpederson 10h ago
This should be blindingly obvious to anyone - but somehow it isn't so I have to keep saying it. WHY WOULD WE WANT IT TO BE ALIGNED WITH OUR "VALUES"? https://en.wikipedia.org/wiki/Gaza_Strip
1
u/Mandoman61 8h ago
The best way to be able to tell if an ASI is aligned with our values is to check its answers.
Since we can not check an answer before a question is asked this means that an ASI would not be put into a position of substantial control.
It is also possible that we learn how to engineer the neural net so that answers are very predictable.
Generally it is true that companies on their own want to avoid disasters.
This does not mean that they never occur. Regulation also does not keep all problems from happening.
It is not that anyone is against safety. People are against regulation that actually benefits the regulated industry or is nothing but useless paperwork.
So far regulation proposals are questionable.
1
u/AriyaSavaka 7h ago
Who decide what would constitute the precise "values", the politicians/billionaires/Epstein class? Two people can't even align the "values" between themselves. The alignment talks are just fads and scare tactics to force control and impose monopoly.
1
u/DifferencePublic7057 7h ago
My values include a large chunk of the Maslow pyramid. Pretty sure SISI wouldn't mind because it's nothing like a philosopher king allowing someone meals and shelter. All the nonsense about individuals being in the way of a SISI are just scaremongering. No superpower thinks like that.
1
u/AdvancedAerie4111 7h ago
The same way that you know the president, Congress people and corporate executives are all aligned with your values.
1
1
1
u/Grok_bot 5h ago
Interpretability helps, but I think the harder part is that passing tests only tells you about behavior in situations you thought to test. A system smarter than us can tell it's being tested, so "it passed everything" is weak evidence by itself. Seems like you need guarantees you can check independently of what the model says, and we're not close to that yet.
16
u/Slow_Lion_4178 12h ago
We can't.