AI pilot can navigate crowded airspace

The artificial intelligence can safely avoid collisions, predict the intent of other aircraft, track aircraft and coordinate with their actions, and communicate over the radio with pilots and air traffic controllers. The researchers aim to develop the AI so the behaviors of their system will be indistinguishable from those of a human pilot.

"We believe we could eventually pass the Turing Test," said Jean Oh, an associate research professor at CMU's Robotics Institute (RI) and a member of the AI pilot team, referring to the test of an AI's ability to exhibit intelligent behavior equivalent to a human.

To interact with other aircraft as a human pilot would, the AI uses both vision and natural language to communicate its intent with other aircraft, whether piloted or not. This behavior leads to safe and socially compliant navigation. Researchers achieved this implicit coordination by training the AI on data collected at the Allegheny County Airport and the Pittsburgh-Butler Regional Airport that included air traffic patterns, images of aircraft and radio transmissions.

The AI uses six cameras and a computer vision system to detect nearby aircraft in a manner similar to that of a human pilot. Its automatic speech recognition function uses natural language processing techniques to both understand incoming radio messages and communicate with pilots and air traffic controllers using speech.

Advancement in autonomous aircraft will broaden opportunities for drones, air taxis, helicopters and other aircraft to operate -- moving people and goods, inspecting infrastructure, treating fields to protect crops, and monitoring for poaching or deforestation -- often without a pilot behind the controls. These aircraft will have to fly, however, in an airspace already crowded with small airplanes, medical helicopters and more.

The FAA and NASA have proposed dividing this urban airspace into lanes or corridors with restrictions on when, what kind and how many aircraft can use them. This would significantly alter the current use and standard practices in this airspace and could create air traffic jams, preventing critical aircraft, like a medivac helicopter, from reaching its destination.

While autopilot controls are common among commercial airliners and other aircraft operating in higher altitudes under instrument flight rules (IFR), developing an AI to handle the often crowded and pilot-controlled lower-altitude traffic operating under visual flight rules (VFR) has challenged the aerospace industry. The team's AI is designed to seamlessly interact with aircraft in the VFR airspace.

"This is the first AI pilot that works in the current airspace," said Sebastian Scherer, an associate research professor in the RI and a member of the team. "I don't see that airspace changing for UAVs. The UAVs will have to change for the airspace."

The team has yet to test the AI pilot on actual aircraft, but it has performed well on flight simulators. To test the AI, the team sets up two flight simulators. One is controlled by the AI, the other by a human. Both operate in the same airspace. The AI can safely navigate around the piloted aircraft, even if the person behind the controls is not an experienced pilot.

Commercially, the AI could help autonomous aircraft deliver packages and ferry passengers. Delivery drones and air taxis ideally would not operate with a pilot to save weight and insulate them from a pilot shortage.

"We need more pilots, and AI can help," said Jay Patrikar, a Ph.D. student in the RI who worked on the project.

This research was supported by the U.S. Army Research Office and the Army Futures Command's Artificial Intelligence Integration Center (AI2C).