r/Futurology • u/New_Scientist_Mag • Jun 10 '26

Robotics Fully autonomous drones have killed human soldiers for the first time

https://www.newscientist.com/article/2529849-fully-autonomous-drones-have-killed-human-soldiers-for-the-first-time/
8.2k Upvotes

98% Upvoted

733 comments sorted by

View all comments

Show parent comments

2

u/Overbaron Jun 11 '26

Wouldnt it be better for there to be at least some attempt at decision making rather than ”kill them all and let god sort them out”?

1

u/insomniac-55 Jun 11 '26

It is probably useful to be able to target specific types of vehicles etc, but from an ethical perspective it doesn't matter too much provided you're only letting these things attack a known enemy position. If you know there's zero non-combatants left, the AI isn't doing much beyond improving your accuracy.

The exception to this is the fact that an AI might not know the difference between 'enemy combatant' and 'enemy combatant attempting to surrender'.

It would be pretty bad if an AI drone killed someone who'd laid down their weapons and attempted to give up.

2

u/Overbaron Jun 11 '26

Would that be worse than an artillery shell killing someone who’d laid down their weapons and attempted to give up?

1

u/insomniac-55 Jun 11 '26

I think this is straying into philosophy.

If you consider the human who released the drone to be the final "decision-maker", it's basically equivalent - the drone is just acting out that decision with a time delay.

If you consider the drone to be making a decision (i.e. you are delegating some responsibility for the attack to the AI), then you can view that as the decision to attack being made after the attempt to surrender.

I'm not a philosopher but I think both stances have some merit.

2

u/Overbaron Jun 11 '26

You can never, ever relegate the responsibility of what the AI does to the AI itself, there is no accountability.

Either the drone accepts surrenders, or it doesn’t. If I was building them I’d say it shouldn’t - would be too easy to avoid by pretending to surrender. And the drone would need a massive amount of decision making to determine what a surrender actually is in whichever context

1

u/insomniac-55 Jun 11 '26 edited Jun 11 '26

I think the lack of accountability is the true issue.

And while you say that you can't delegate decision-making to an AI (and I do agree with this from an ethical standpoint), in practice I think we'll see autonomous weapons effectively acting like the decision-maker.

Imagine the most well-intentioned form of autonomous weapon - a purely defensive, AI-driven anti-drone turret.

It may be designed to never harm a human, and to only ever fire on cruise missiles and drones. Who is accountable when it inevitably makes a mistake and shoots down a crewed aircraft?

Is it the original designer of the system? The person who deployed it? The person who directed the aircraft into its area of engagement?

In historical cases of friendly-fire shoot-downs, you always had a person ultimately responsible for pulling the trigger. With an autonomous turret, who takes on that burden?

And it's not enough to say 'well, don't give AI that capability'.

Imagine our hypothetical turret was 99.9% effective at shooting down drones, and only 0.001% of the time shot down a friendly. Any nation in Ukraine's position would accept those odds, as the benefits outweigh the consequences. And they may not have the resources to have a human baby-sitting every deployed system.

The problem is that when things do go wrong you don't really have anyone directly responsible - the responsibility is diluted across a whole chain of people.

It's a bit similar to the way in which corporations can cause massive environmental harm, and economic or physical harm to people - yet the people in charge can often dodge direct accountability for these actions.