Military cannot rely on AI for strategy or judgment, study suggests

"All of the hard problems in AI really are judgment and data problems, and the interesting thing about that is when you start thinking about war, the hard problems are strategy and uncertainty, or what is well known as the fog of war," said Jon Lindsay, an associate professor in the School of Cybersecurity & Privacy and the Sam Nunn School of International Affairs. "You need human sense-making and to make moral, ethical, and intellectual decisions in an incredibly confusing, fraught, scary situation."

AI decision-making is based on four key components: data about a situation, interpretation of those data (or prediction), determining the best way to act in line with goals and values (or judgment), and action. Machine learning advancements have made predictions easier, which makes data and judgment even more valuable. Although AI can automate everything from commerce to transit, judgment is where humans must intervene, Lindsay and University of Toronto Professor Avi Goldfarb wrote in the paper, "Prediction and Judgment: Why Artificial Intelligence Increases the Importance of Humans in War," published in International Security.

Many policy makers assume human soldiers could be replaced with automated systems, ideally making militaries less dependent on human labor and more effective on the battlefield. This is called the substitution theory of AI, but Lindsay and Goldfarb state that AI should not be seen as a substitute, but rather a complement to existing human strategy.

"Machines are good at prediction, but they depend on data and judgment, and the most difficult problems in war are information and strategy," he said. "The conditions that make AI work in commerce are the conditions that are hardest to meet in a military environment because of its unpredictability."

An example Lindsay and Goldfarb highlight is the Rio Tinto mining company, which uses self-driving trucks to transport materials, reducing costs and risks to human drivers. There are abundant, predictable, and unbiased data traffic patterns and maps that require little human intervention unless there are road closures or obstacles.

War, however, usually lacks abundant unbiased data, and judgments about objectives and values are inherently controversial, but that doesn't mean it's impossible. The researchers argue AI would be best employed in bureaucratically stabilized environments on a task-by-task basis.

"All the excitement and the fear are about killer robots and lethal vehicles, but the worst case for military AI in practice is going to be the classically militaristic problems where you're really dependent on creativity and interpretation," Lindsay said. "But what we should be looking at is personnel systems, administration, logistics, and repairs."

There are also consequences to using AI for both the military and its adversaries, according to the researchers. If humans are the central element to deciding when to use AI in warfare, then military leadership structure and hierarchies could change based on the person in charge of designing and cleaning data systems and making policy decisions. This also means adversaries will aim to compromise both data and judgment since they would largely affect the trajectory of the war. Competing against AI may push adversaries to manipulate or disrupt data to make sound judgment even harder. In effect, human intervention will be even more necessary.

Yet this is just the start of the argument and innovations.

"If AI is automating prediction, that's making judgment and data really important," Lindsay said. "We've already automated a lot of military action with mechanized forces and precision weapons, then we automated data collection with intelligence satellites and sensors, and now we're automating prediction with AI. So, when are we going to automate judgment, or are there components of judgment cannot be automated?"

Until then, though, tactical and strategic decision making by humans continues to be the most important aspect of warfare.