States have agreed on the principle that machines cannot be held accountable for violations of international humanitarian law (IHL), but how would accountability be ensured in practice? 

In this post, Vincent Boulanin and Marta Bo from the Stockholm International Peace Research Institute (SIPRI) argue that looking at how responsibility for IHL violations is currently ascribed under international law provides useful lessons for the regulation of AWS.

Read the full blog post here: https://blogs.icrc.org/law-and-policy/2023/03/02/three-lessons-autonomous-weapons-systems-ihl/

3 lessons on the regulation of AWS to ensure accountability for violations of IHL

Last week, states parties met for the first session of the Group of Governmental Experts (GGE) on lethal autonomous weapon systems (LAWS) in 2024. This debate featured the GGE’s most substantive discussion to date about bias under the topic “risk mitigation and confidence building”, including around a working paper dedicated to bias by Canada, Costa Rica, Germany, Ireland, Mexico, and Panama.

In this post, Dr. Ingvild Bode, Associate Professor at the Center for War Studies (University of Southern Denmark) argues that bias is as much a social as a technical problem and that addressing it therefore requires going beyond technical solutions. She holds that the risks of algorithmic bias need to receive more dedicated attention as the GGE’s work turns towards thinking around operationalisation. These arguments are based on the author’s presentation at the GGE side event “Fixing Gender Glitches in Military AI: Mitigating Unintended Biases and Tackling Risks” organised by UNIDIR on 6 March 2024.

Falling under the radar: algorithmic bias and military applications of AI

Efforts to address the ethical challenges posed by autonomous weapon systems are growing. In December 2023, the UN General Assembly passed a resolution requesting the Secretary-General to seek states’ views for addressing, among other issues, the ethical challenges posed by these systems, and in April 2024 Austria is hosting the conference ‘Humanity at the Crossroads: Autonomous Weapon Systems and the Challenge of Regulation’, with the aim of bringing ethical challenges to the fore. 

In this post, Alexander Blanchard, Senior Researcher at the Stockholm International Peace Research Institute (SIPRI), argues that while ethics has been a prominent part of international regulatory debate on autonomous weapon systems for years, the promise and potential of ethics for advancing that debate is still far from being realized. He holds that, for progress to happen, new efforts must be coupled with work to clarify the role of ethics for regulation.

The road less travelled: ethics in the international regulatory debate on autonomous weapon systems

For more than a decade, states have met at the UN in Geneva to discuss the governance of autonomous weapon systems (AWS). One pandemic, several real-world cases of artificial intelligence (AI) being used in targeting decisions, and numerous meetings later, there is a growing consensus among states that the challenges posed by AWS should be addressed through both prohibitions and restrictions, a so-called ‘two-tier’ approach. But while there is progress on the basic structure (i.e. two tiers), the actual content of these tiers is debated.

To help states elaborate on possible elements of a two-tiered approach to the governance of AWS, Laura Bruun from the Stockholm International Peace Research Institute (SIPRI) points to three lessons from past arms control negotiations that can be applied to the AWS debate: First, a prohibition does not need to be grounded in a clearly defined class of weapons, second, restrictions can be used to clarify what international humanitarian law (IHL) requires in the specific context of AWS, and third, if there is will (and a need), two-tiered instruments can be grounded in concerns beyond IHL.

Reinventing the wheel: 3 lessons the AWS debate can learn from existing arms control agreements

The desire to develop technological solutions to help militaries in their decision-making processes is not new. However, more recently, we have witnessed militaries incorporating increasingly complex forms of artificial intelligence-based decision support systems (AI DSS) in their decision-making process, including decisions on the use of force. The novelty of this development is that the process by which these AI DSS function challenges the human’s ability to exercise judgement in military decision-making processes. This potential erosion of human judgement raises several legal, humanitarian and ethical challenges and risks, especially in relation to military decisions that have a significant impact on people’s lives, their dignity, and their communities. It is in light of this development that we must urgently and in earnest discuss how these systems are used and their impact on people affected by armed conflict.

With this post, Wen Zhou, Legal Adviser with the International Committee of the Red Cross (ICRC), and Anna Rosalie Greipl, Researcher at the Geneva Academy of International Humanitarian Law and Human Rights, launch a new series on artificial intelligence (AI) in military decision-making. To start the discussion, they outline some of the challenges and risks, as well as the potential, that pertain to the use of AI DSS in preserving human judgement in legal determinations on the use of force. They also propose some measures and constraints regarding the design and use of AI DSS in these decision-making processes that can inform current and future debates on military AI governance, in order to ensure compliance with international humanitarian law (IHL) and support mitigating the risk of harm to people affected by those decisions.

Artificial intelligence in military decision-making: supporting humans, not replacing them

Over the past decade, discussions surrounding artificial intelligence (AI) in the military domain have largely focused on autonomous weapon systems. This is partially due to the ongoing debates of the Group of Governmental Experts on Lethal Autonomous Weapons Systems of the Convention on Certain Conventional Weapons. While autonomous weapon systems are indeed a pressing concern, the critical reality is that AI is hastily deployed to gather intelligence and, even more worrisome, to support militaries to select and engage targets.

As AI-based decision support systems (AI DSS) are increasingly used in contemporary battlefields, Jimena Sofía Viveros Álvarez, member of the United Nations Secretary General’s High-Level Advisory Body on AI, REAIM Commissioner and OECD.AI Expert, advocates against the reliance on these technologies in supporting the target identification, selection and engagement cycle as their risks and inefficacies are a permanent fact which cannot be ignored, for they actually risk exacerbating civilian suffering.

The risks and inefficacies of AI systems in military targeting support

The military decision-making process is facing a challenge by the increasing number of interconnected sensors capturing information on the battlefield. The abundance of information offers advantages for operational planning – if it can be processed and acted upon rapidly. This is where AI-assisted decision-support systems (DSS) enter the picture. They are meant to empower military commanders to make faster and more informed decisions, thus accelerating and improving the decision-making process. Although they are just meant to assist – and not replace – human decision-makers, they pose several ethical challenges which need to be addressed.

In this post, Matthias Klaus, who has a background in AI ethics, risk analysis and international security studies, explores the ethical challenges associated with a military AI application often overshadowed by the largely dominating concern about autonomous weapon systems (AWS). He highlights a number of ethical challenges associated specifically with DSS, which are often portrayed as bringing more objectivity, effectivity and efficiency to military decision-making. However, they could foster forms of bias, infringe upon human autonomy and dignity, and effectively undermine military moral responsibility by resulting in peer pressure and deskilling.

Transcending weapon systems: the ethical challenges of AI in military decision support systems

In the debate on how artificial intelligence (AI) will impact military strategy and decision-making, a key question is who makes better decisions — humans or machines? Advocates for a better leveraging of artificial intelligence point to heuristics and human error, arguing that new technologies can reduce civilian suffering through more precise targeting and greater legal compliance. The counter argument is that AI-enabled decision-making can be as bad, if not worse, than that done by humans, and that the scope for mistakes creates disproportionate risks. What these debates overlook is that it may not be possible for machines to replicate all dimensions of human decision-making. Moreover, we may not want them to. 

In this post, Erica Harper, Head of Research and Policy at the Geneva Academy of International Humanitarian Law and Human Rights, sets out the possible implications of AI-enabled military decision-making as this relates to the initiation of war, the waging of conflict, and peacebuilding. She highlights that while such use of AI may create positive externalities — including in terms of prevention and harm mitigation — the risks are profound. These include the potential for a new era of opportunistic warfare, a mainstreaming of violence desensitization and missed opportunities for peace. Such potential needs to be assessed in terms of the current state of multilateral fragility, and factored into AI policy-making at the regional and international levels.

Will AI fundamentally alter how wars are initiated, fought and concluded?

The International Committee of the Red Cross (ICRC) Humanitarian Law & Policy blog is a unique space for timely analysis and debate on international humanitarian law (IHL) issues and the policies that shape humanitarian action.

New tracks tagged #autonomousweapons