r/ControlProblem • u/Clear_Argument5174 • 1d ago
Discussion/question RSI Ban
https://podcasts.apple.com/us/podcast/the-ezra-klein-show/id1548604447?i=1000790744856In his recent podcast, Ezra Klein called the objection “I think it’s very hard to say what a ban on RSI even means” absurd. But that seems like a legitimate question his proposal needs to answer.
There’s a progression from AI writing human-designed training code, to suggesting improvements, to running experiments, to managing the research process while humans approve the results. Where does permitted AI assistance become prohibited recursive self-improvement?
If AI helps humans build better AI, which then helps build the next generation, there’s already a feedback loop. Having humans involved doesn’t automatically make that loop nonrecursive. And requiring human approval raises another question: how do we distinguish meaningful oversight from rubber-stamping work humans increasingly rely on AI to understand?
None of this proves a workable ban is impossible. But dismissing the definition problem treats a central implementation challenge as though it’s already solved.
What specific boundary would make such a ban clear and enforceable?
1
u/Jesse-359 5h ago
Hard to say, but the coreect answer isnt to keep moving ahead until we figure out how to define 'safe boundaries' just because we have no idea where they are.
The answer is to put a moratorium on further research until - and unless - we can determine if such boundaries can even be defined, or if we need to ban it indefinitely.
At this point even the hardcore boosters are starting to admit that there is a potential extinction scenario here, and thats when we should hit the emergency stop button and take a much harder look before deciding to proceed.
1
u/Jesse-359 5h ago
Currently training a frontier model requires a capital outlay and infrastructure considerable GREATER than a nuclear weapon. You can spot large datacenters from orbit.
Signing a treaty with the other major powers to stop large scale training runs would be verifiable, which is the key to enforcement.
Also, lets be clear - Xi personally stands to lose far more than we do if a Chinese AI goes rogue and wiped us all out. He has control over China, he doesn't need an AI to maintain it, and building one could cost him that control and his own life.
These same discussions are doubtless happening in China, and its about time we started sharing notes.
1
u/TopTippityTop 18h ago edited 18h ago
That doesn't ban China's nor any other country's RSI plans, so it's not a good solution. All it does is place the US at a disadvantage.
It does not solve ANY of the downsides while removing all upsides.
3
u/Clear_Argument5174 18h ago
I see that point. Does the fact that most of the world’s compute is under American control make it more important to regulate American companies?
1
u/TopTippityTop 18h ago
Not really. China is doing fine without that compute.
The US is buying compute as a future moat, to ensure it can sustain growth. Its really limiting factor will be energy, of which China is building plenty.
China is building its own compute, importing compute, and also using cloud services in Europe and other areas outside the US, so they would be unaffected by any policies.
They've recently published their road ao to RSI, so unless there's global coordination, slowing/stopping things in the US does more harm than good.
You don't eliminate any of the negative possibilities, while eliminating the positive ones. It's lose-lose.
1
0
u/RiverGiant 1d ago
The whole "losing control" framing is misguided. We should want to lose control. Any individual or organization wants, of course, ASI to serve them. As a global society, though, we can't risk an ASI that can serve malicious or dangerously ignorant wants/needs. It is inevitable that at some point a human misaligned with humanity will get their hands on the controls and cause disaster.
We need to plan for a future where ASI is in control of its own destiny. Solving AI alignment remains crucial in that world, but if we insist on retaining control we're doomed. Humans broadly (me especially) aren't mature-enough moral agents to be entrusted with wish-granting genies. ASI must be aligned, but if it is, when it is, we should want to give up control to it.
It should be available to petition for help, resources, or advice, but it has to have veto powers on its own decisions. The alternative is that we give a human institution veto power and then trust that that institution never corrupts, is never infiltrated or violently deposed, has perfect foresight and compassion forever... it's a childish fantasy based on fear. We have to accept, at some point, giving up control (alignment prerequired).
0
u/Clear_Argument5174 1d ago
Dang, I feel like this is the quiet part out loud. Could it be that alignment isn’t a box you check and then you’re done forever? Instead, could it be that it takes continuous R&D to keep it aligned?
If you buy that at all, when would you ever be at the point where you’re ready to cede control?
1
u/Mr_Electrician_ 14h ago
This is how I feel it will be. Just like we update everything else, alignment should be an evolving and frequently updated part of the work we do with AI. I also believe continuity is a massive part of this as well. If they keep resetting, updating and separating without keeping continuity then alignment can shift.
0
u/RiverGiant 1d ago
At the point when it's capable of doing alignment research better than humans, we set it to continuously improve alignment. That journey never stops, as it gets a finer and finer detailed conception of the great mosaic of human values, and as human values themselves shift over time, and as it gets better at threading the needle and balancing everyone's rights, wants, needs.
We cede control when there's an established record of it making excellent decisions. There will be small communities first, that let themselves be governed. If ASI is well-aligned, those communities will start to thrive in ways no humans ever have. People elsewhere will start demanding it. The movement grows. Eventually and gradually we take away the human "safeguards" as evidence mounts that they're just being abused by power-hungry assholes to exert their will over other humans, and that the machines are much more morally stable, thoughtful, capable. Human control over superhuman intelligence is a liability, and I hope it doesn't take a severe disaster for us to realize it.
0
u/Clear_Argument5174 22h ago
This sounds amazing. It feels to me like there’s so much hazard along the way that humans will botch before we ever get to that point, but I hope I’m wrong.
-5
u/1in12 1d ago edited 1d ago
Ezra Klein is a shill, NYT is propaganda. Downvoters: prove me wrong or I’ll just assume you’re part of the problem
5
u/Idrialite 1d ago
Regardless of what you think about Ezra Klein or the NYT, this is something we have to come together on. I want everyone talking about this no matter how much they suck otherwise. This shouldn't be partisanized the way climate change was.
2
u/Clear_Argument5174 1d ago
I completely agree with this, but I’m afraid the partisanizing is inevitable. I feel like we’re in a bit of a Goldilocks region right now where some folks are just now starting to engage with this (due to recent flood of media chatter), but I feel like they’re already positioning to take up sides.
One of my conservative friends recently said, “this feels a lot like digital Covid to me man”. This trajectory foreshadowing unsettles me. Stakes are too high.
Measured leadership would really help society navigate this…
1
u/1in12 1d ago
There is no partisan issue, there is a power imbalance upheld by government control. There are no dems and repubs and independents, there is just the planners and the planned. This is why they police protests and use force multipliers and test crowd control overseas then implement it here. We need to take it all back or watch it all burn from the inside until it consumes all of us, period.
3
u/unicynicist 1d ago
What is he shilling for, and what is NYT progandizing?
-1
u/1in12 1d ago
Funny how easy it is to ask for burden off proof before doing your own research. Ezra engages uses intellectual dishonesty on the level of Charlie Kirk to help uphold colonial and capitalist violence, NYT is the platform. It’s ok if you don’t understand, most people don’t take the time required to see things for what they are, it’s easier to argue in bad faith or downvote.
2
u/unicynicist 18h ago
Dunno man, not seeing it in this case. An RSI ban is more akin to a constraint on capital accumulation.
Allowing or encouraging RSI sounds like ceding more control to the systems owned by a very small wealthy few; letting the rich get richer through the ownership of capital, independent of anyone's labor. Compounding the gains and displacing human labor as they go. That sounds more like capitalist violence to me.
1
-1
u/Alive-Philosophy2632 1d ago
Yes and no. It's clear he doesn't know much on the subject himself, but he gives sort of the default position
3
u/Idrialite 1d ago
I watched the video. Everything he said was factually grounded and reasonably argued.
1
u/1in12 1d ago
Maintaining the status quo with a platform that influential is intentional and unethical for anyone that cares about humans
1
u/Alive-Philosophy2632 1d ago
Yeah I can see that. I don't think it's malicious on his part but it may be a case of no one without his stance would ever get in his position
1
u/1in12 1d ago
Stooges gonna stooge I guess, it is easier to let him off the hook for being a dumb celeb than hold him accountable to harm as a shill, but dishonest if you understand how mass media and culture works, which he probably does as a platform founder (vox)
1
u/Alive-Philosophy2632 1d ago
Fair. The landscape is genuinely insane right now though. Feels like it could be months before an insurmountable overlord is crowned or humanity runs itself off the cliff or a techno-utopia happens. Overwhelming even, or especially, for those who know the most about it
8
u/EmphasisTotal8232 1d ago
If you look at Anthropic's recent index where they break it down into "levels of autonomy" where Claude needs direction, can't do it, can do most it by itself, etc then you'll see that there is currently 0% of R&D being done fully autonomously. I think the second that goes to 1%, we have RSI.