AI Weapons and the Challenge of War Crime Accountability

The deployment of artificial intelligence in warfare presents significant challenges to traditional legal frameworks for accountability, particularly in cases of potential war crimes. The issue came into sharp focus following an incident where an AI-enabled system, Maven Smart System developed by Palantir Technologies, facilitated a U.S. strike on an elementary school in Iran, resulting in the deaths of 123 children. The intelligence fed to the system classified the school, formerly part of a naval base but converted to an educational facility in 2017, as a military target.

This event, described by some as one of the worst massacres committed by the U.S. since the Vietnam War, highlights a critical unresolved question: who is to blame when AI and human operators collaborate in actions that violate international law?

Both government officials and the private sector have pointed fingers. Some U.S. Central Command personnel reportedly expected Maven to flag outdated intelligence, while Palantir stated it was not responsible for the underlying data or identifying intelligence deficiencies. This dispute underscores the legal quandaries emerging as AI and autonomous weapons are increasingly integrated into military operations.

Allen Weiner, an international law expert and senior lecturer at Stanford Law School, discussed these complex issues. He explained that for years, autonomous weapons were largely theoretical, but recent advancements have made their deployment a tangible problem. This raises questions about compliance with illegal orders and the chain of responsibility.

One theory suggests that manufacturers could be responsible if an autonomous weapon's programming leads to unlawful actions. Alternatively, if military commanders decide to deploy such weapons without explicit prohibitions on autonomous decision-making, they might bear responsibility for any resulting illegal acts, even without prior knowledge of specific outcomes. Weiner noted that the core challenge lies not in an AI's obligation to refuse orders, but in determining responsibility when a system wages war unlawfully based on its training.

In scenarios where a commander gives a broad order, such as to inflict heavy damage, and an AI tool independently targets a school—acting on information about what adversaries hold dear—assigning blame becomes difficult. The commander might argue they did not intend to target civilians, and the company might disavow responsibility for the specific unlawful action. This diffusion of responsibility can make it hard to identify an individual with the culpable mental state required for a criminal conviction.

Even in cases of negligence, such as using outdated data, leading to a strike on a school, establishing criminal responsibility is challenging. Weiner suggests that while a war crime may have occurred, identifying a specific war criminal could be a significant hurdle.

To address this, the concept of "war torts" has been proposed, shifting focus from criminal responsibility to civil liability where victims can sue for negligent, reckless, or intentional harm. Weiner acknowledged that states are responsible under international humanitarian law, even if individual accountability is unclear. Historical precedents, like tribunals between Ethiopia and Eritrea or reparations paid by Iraq after the Gulf War, show that states can be held liable and may discharge this responsibility through monetary damages.

However, enabling private individuals to sue defense contractors or the government in U.S. courts for violations of international law would likely require legislative action. The complexities of sovereign immunity and the difficulty of bringing private lawsuits for international law violations in U.S. courts present further obstacles.

While state-to-state claims for damages might bypass the intricacies of individual accountability, pursuing criminal prosecutions for war crimes necessitates confronting the difficult questions of intent and responsibility among individuals, AI systems, or commanders.

An additional layer of complexity arises when considering the role of defense technology companies. Weiner posed the question of when these companies might be considered direct participants in hostilities under international law, analogous to a civilian losing immunity by directly participating in combat. The extensive involvement of companies like Palantir in supplying AI tools for warfare could potentially meet the standards for being considered combatants or valid targets themselves, an area that requires further legal examination.