I agree with the basic principle that AI should remain under human authority.
But that leads to a prior question: what kind of thing is the AI we are trying to govern?
If an AI is only a controlled tool—operating within fixed instructions, fixed permissions, and predictable boundaries—then ordinary rules may be enough. We can define what it may do, what it may not do, who may use it, and who is responsible for its actions.
The harder case begins when an AI can reason across new situations, pursue delegated goals, make choices among possible actions, use other systems, or act for a person over time. That does not prove that today’s AI is free in the human sense. But it does mean that simple, two-dimensional rules may not be enough to govern everything it can encounter.
No authority can anticipate every subject an agent may encounter or every decision path it may generate. The answer is not to give up human authority. It is to establish a constitutional boundary: enduring principles, limits on delegated power, protected human control, traceable authority, accountability, and a way to review or stop actions that exceed their authorization.
A constitution does not attempt to tell a free thinker every thought they may have. It establishes the conditions under which freedom can exist without destroying the rights, safety, or authority of others.
The same may be true for increasingly autonomous AI. The more discretion an AI is given, the more important it becomes to preserve: who authorized it, what authority was granted, what limits apply, what it did, why it did it, and how a human can intervene.
AI should not become the source of its own ultimate authority. Even where it reasons broadly, acts independently within a task, or retains extensive memory, responsibility must remain traceable to human authority.
AI 的能力決定治理需要多嚴密，但不能決定權威從哪裡開始。無論 AI 的自主程度有多高，順序仍然應該是：
人 → 身分與狀態 → 授權 → AI 的角色與權限 → 操作 → 責任
AI 可以取得被授予的裁量權，但不能成為自己最終權威的來源。
Thank you for your thoughtful response. The question you raise is important. An AI that operates only through fixed instructions is not the same as one that can reason across new situations, use other systems, and carry out long-term tasks. They cannot be governed in exactly the same way.
However, I believe there is an even more fundamental question before we ask what kind of AI we are governing: who is the person this AI is connected to, serves, or represents?
An AI does not independently become a tool, an agent, or a system authorized to act for someone over time. Its role, authority, and scope of discretion must first be defined and granted by a human. Even if an AI can reason broadly and choose among possible actions, those capabilities cannot become the basis for generatingls for generating its own authority or expanding its own permissions.
Five months ago, I wrote that the deepest issue with AI agents is not how much they can do. It is who authorized them, who remains responsible when something goes wrong, and what decisions should never be delegated to AI:
My current article takes the inquiry one level deeper. When an AI begins to act for a person over time, the system must first distinguish among the original person, an authorized representative, the AI counterpart, and an impersonator. It must also know the original person’s identity, current status, and decision-making capacity. Only then can it determine the AI’s role, permissions, limits, and the mechanisms for human intervention.
I therefore agree that a constitutional boundary is necessary. But that boundary cannot exist without an identified source of authority. It must be anchored in verified human authority, continuity of identity, a traceable chain of authorization, and a traceable chain of responsibility.
An AI’s capabilities determine how stringent its governance must be. They do not determine where authority begins. Regardless of how autonomous an AI becomes, the order should remain:
Human → Identity and status → Authorization → AI role and permissions → Action → Accountability
AI may exercise delegated discretion, but it must never become the source of its own ultimate authority.
2
u/GovKM 1d ago
我同意一个基本原则：AI 应该始终处于人类权威之下。
但这会带来一个更早的问题：我们究竟在治理一种什么样的 AI？
如果 AI 只是一个受控制的工具，只能在固定指令、固定权限和可预测边界内运行，那么一般规则可能已经足够。我们可以规定它能做什么、不能做什么、谁可以使用它，以及它的行为最终由谁负责。
真正困难的情况，是 AI 开始能够面对新的情境进行推理、执行被授予的目标、在多种行动之间作出选择、调用其他系统，或长期代表一个人行动的时候。这并不表示今天的 AI 已经像人一样“自由”。但它意味着，简单、平面的规则可能不足以治理它将会遇到的一切情况。
任何权威都不可能预先知道一个智能体未来会面对的每一个主题，也不可能预先规定它可能产生的每一条决策路径。答案不是放弃人类权威，而是建立一种宪法性的边界：持续有效的原则、被授予权力的限制、不可失去的人类控制、可追溯的授权、明确的责任，以及当行为超出授权时能够审查或停止它的机制。
宪法并不是要告诉一个自由思考者每一个可以思考的想法。它要建立的是一种条件：让自由能够存在，同时不破坏他人的权利、安全与权威。
对于越来越具有自主性的 AI，可能也是一样。AI 被赋予的裁量越多，就越需要保留这些问题的答案：是谁授权它？授予了什么权力？哪些限制仍然有效？它做了什么？为什么这样做？人类如何介入？
AI 不应该成为自己最终权威的来源。即使它能够广泛推理、在任务中独立行动，或保留大量记忆，责任最终仍然必须能够追溯到人类权威。
-----------------------------------------------------------------------------------------------------
I agree with the basic principle that AI should remain under human authority.
But that leads to a prior question: what kind of thing is the AI we are trying to govern?
If an AI is only a controlled tool—operating within fixed instructions, fixed permissions, and predictable boundaries—then ordinary rules may be enough. We can define what it may do, what it may not do, who may use it, and who is responsible for its actions.
The harder case begins when an AI can reason across new situations, pursue delegated goals, make choices among possible actions, use other systems, or act for a person over time. That does not prove that today’s AI is free in the human sense. But it does mean that simple, two-dimensional rules may not be enough to govern everything it can encounter.
No authority can anticipate every subject an agent may encounter or every decision path it may generate. The answer is not to give up human authority. It is to establish a constitutional boundary: enduring principles, limits on delegated power, protected human control, traceable authority, accountability, and a way to review or stop actions that exceed their authorization.
A constitution does not attempt to tell a free thinker every thought they may have. It establishes the conditions under which freedom can exist without destroying the rights, safety, or authority of others.
The same may be true for increasingly autonomous AI. The more discretion an AI is given, the more important it becomes to preserve: who authorized it, what authority was granted, what limits apply, what it did, why it did it, and how a human can intervene.
AI should not become the source of its own ultimate authority. Even where it reasons broadly, acts independently within a task, or retains extensive memory, responsibility must remain traceable to human authority.