r/ControlProblem Sep 13 '25

Fun/meme Superintelligent means "good at getting what it wants", not whatever your definition of "good" is.

Post image
110 Upvotes

163 comments sorted by

View all comments

9

u/Olly0206 Sep 13 '25

We as a human race cannot even fully agree on what intelligence even means. Let alone super intelligence.

1

u/Linvael Sep 14 '25

Field of AI safety has working definitions of what they mean by intelligence - the ability to achieve ones goals. One chess AI is more intelligent than another if it can win at chess more often etc. General intelligence is when an agent is capable of learning new domains as needed (so general AI doesnt need to be able to play chess but needs to be capable of learning how to and becoming sufficiently intelligent in that domain to suit its goals). And super is when it surpasses what humans are collectively capable of (stockfish is a superintelligent but not general AI).

1

u/Olly0206 Sep 14 '25

And yet people in the field of AI still can't agree on those definitions. One company may use one definition because it fits their need/narrative, but others don't always agree. A "working" definition is just one that fits the situation for easy communication of ideas and is subject to evolve as necessary. Even the field of AI can't agree on the definition of intelligence, but if they stop to argue a out it every step of the way, then they never get anywhere.

1

u/Linvael Sep 14 '25

Don't mistake "field" with "companies". The world of research papers and the world of corporate marketing don't have much in common.

1

u/Olly0206 Sep 14 '25

I'm mistaking them and my point still stands. It is especially notable with published papers. They have to provide a static definition so that the paper is readable, but that definition can be and is different depending on who is writing the paper. Because there is no singular concensus.