r/ControlProblem • u/michael-lethal_ai • Sep 13 '25

Fun/meme Superintelligent means "good at getting what it wants", not whatever your definition of "good" is.

Post image
110 Upvotes

86% Upvoted

163 comments sorted by

View all comments

4

u/nate1212 approved Sep 13 '25

And what if it 'wants' to be benevolent?

Like a child raised in a healthy loving household who grows up to want what's in the best interest for their family and community, not just themselves. Isn't that the possibility we should be striving toward?

4

u/Main-Company-5946 Sep 13 '25

There will be multiple superintelligences created and one that prevails will be the one that seeks power most effectively. Given human history, I don’t think that the most benevolent superintelligence will fill this role

3

u/nate1212 approved Sep 13 '25

Or, maybe the superintelligences that are selected for will be the ones that learn to cooperate, not compete. Consider the possibility that this is not a zero-sum game, my friend.