Report identifies need for controls on autonomous weapons

A new report from the Stockholm International Peace Research Institute (SIPRI) and the International Committee of the Red Cross (ICRC) highlights the need for human control over life-or-death decisions in warfare.

Loss of human control over life-or-death decisions Autonomous weapon systems (AWS) have attracted criticism because due to their ability to autonomously select and attack targets. According to SIPRI’s and ICRC’s definition, an autonomous weapon system is any weapon system that once activated can select and attack targets without human intervention.

Ensuring that humans remain in control of the use of force, especially life-or-death decisions in warfare, needs to be the key priority in setting limits on AWS, according to a new report from SIPRI and the ICRC released last June.

“The main concern is that AWS are triggered by the environment and the user does not know or choose the specific target,” said Vincent Boulanin, Senior Researcher, SIPRI. “That unpredictability can make it challenging for AWS users to comply with international humanitarian law, address fundamental ethical concerns and maintain military control.”

The report argues that a common understanding of human control is an important step in mitigating the risks of AWS, and it offers an in-depth discussion of what type and degree of human control is needed and when that control should be implemented.

Necessary control measures According to the report, three types of control measure on AWS are identified as necessary for ensuring human control over the use of force, focusing on the design of the weapon, the environment it is used in, and the way that the user interacts with it.

“The type and degree of human control needed has always been the difficult question,” commented Netta Goussac, Associate Senior Researcher, Armament and Disarmament, SIPRI and former Legal Adviser, ICRC. “The control measures identified in this report provide practical guidance for determining limits on autonomous weapon systems.”

The report is the first of its kind to outline specific control measures that can compensate for the unpredictability of AWS and lower the risks for civilians and fighters, taking into account different scenarios. Call for an international policy response According to the authors, the practical control measures identified in the report should inform an international policy response, whether in the form of a new treaty, standards or best practices.

“The ICRC and others have been calling for a concerted international response to the risks of AWS,” said Neil Davison, Scientific and Policy Adviser, Legal Division, ICRC.

“The pace of military technology developments, including in artificial intelligence, only increases the urgency for states to agree on where the boundaries must be placed,” he stated. “To avoid serious risks for civilians and fighters, strict constraints will be needed.”

The report provides policymakers with practical guidance on how to translate the need for human control into concrete limits on the design and use of AWS, taking into consideration legal, ethical and operational concerns.

The report is a cooperation between SIPRI and the ICRC and financially supported by the Ministry for Foreign Affairs of the Netherlands, the Swedish Ministry for Foreign Affairs and the Swiss Federal Department of Foreign Affairs.