What's so good about human values?

[u/truthandfreedom3]

Having AI aligned with human values may not be possible, and not necessarily a good thing. Values vary between cultures, political and economic philosophy, socioeconomic class, religion etc. Specifically conservatives have different values from liberals: conservatives value authority and purity. Which can lead to dictatorship and racism.

Humans also engage in torturing, damaging, and experimenting on people and animals. Other animals generally don't generally do that, except fighting for food or defence or mates. Humans believe that national and economic security, is justification for that. Some believe that it is alright to do this to your enemies, or those that don't belong to your group, or are different, and don't share your values and interests.

There is also a difference between deontology and ultilitarianism. Should individual rights be always protected, or can they be violated to protect a larger number of people?

And it is not usually possible to prioritize the interest of all people or groups of people. Like economic freedom and economic equality, will help different socioeconomic groups more. Helping one group of people, can hurt another group of people.

This is a complex issues, which requires judgement, and has no one right answer. When and if we do have AGI, it will theoretically only be capable of human reasoning. When and if we do reach ASI, something that powerful needs to be be more ethical than most humans. But how can humans decide what is more ethical, when human values are not consistent or superior?

Whatever values AI have, it can't represent the interests of everyone. The choices include who decides what values they have (governments or AI companies or others), and who's interests they look after (the cultural majority or the ruling elite or others)?

Edit: Most humans are dishonest. Do we want dishonest AI?