Delivery by drone: New algorithm lets drones monitor their own health during long package-delivery missions

To ensure safe, timely, and accurate delivery, drones would need to deal with a degree of uncertainty in responding to factors such as high winds, sensor measurement errors, or drops in fuel. But such "what-if" planning typically requires massive computation, which can be difficult to perform on the fly.

Now MIT researchers have come up with a two-pronged approach that significantly reduces the computation associated with lengthy delivery missions. The team first developed an algorithm that enables a drone to monitor aspects of its "health" in real time. With the algorithm, a drone can predict its fuel level and the condition of its propellers, cameras, and other sensors throughout a mission, and take proactive measures -- for example, rerouting to a charging station -- if needed.

The researchers also devised a method for a drone to efficiently compute its possible future locations offline, before it takes off. The method simplifies all potential routes a drone may take to reach a destination without colliding with obstacles.

In simulations involving multiple deliveries under various environmental conditions, the researchers found that their drones delivered as many packages as those that lacked health-monitoring algorithms -- but with far fewer failures or breakdowns.

"With something like package delivery, which needs to be done persistently over hours, you need to take into account the health of the system," says Ali-akbar Agha-mohammadi, a postdoc in MIT's Department of Aeronautics and Astronautics. "Interestingly, in our simulations, we found that, even in harsh environments, out of 100 drones, we only had a few failures."

Agha-mohammadi will present details of the group's approach in September at the IEEE/RSJ International Conference on Intelligent Robots and Systems, in Chicago. His co-authors are MIT graduate student Kemal Ure; Jonathan How, the Richard Cockburn Maclaurin Professor of Aeronautics and Astronautics; and John Vian of Boeing.

Tree of possibilities

Planning an autonomous vehicle's course often involves an approach called Markov Decision Process (MDP), a sequential decision-making framework that resembles a "tree" of possible actions. Each node along a tree can branch into several potential actions -- each of which, if taken, may result in even more possibilities. As Agha-mohammadi explains it, MDP is "the process of reasoning about the future" to determine the best sequence of policies to minimize risk.

MDP, he says, works reasonably well in environments with perfect measurements, where the result of one action will be observed perfectly. But in real-life scenarios, where there is uncertainty in measurements, such sequential reasoning is less reliable. For example, even if a command is given to turn 90 degrees, a strong wind may prevent that command from being carried out.

Instead, the researchers chose to work with a more general framework of Partially Observable Markov Decision Processes (POMDP). This approach generates a similar tree of possibilities, although each node represents a probability distribution, or the likelihood of a given outcome. Planning a vehicle's route over any length of time, therefore, can result in an exponential growth of probable outcomes, which can be a monumental task in computing.

Agha-mohammadi chose to simplify the problem by splitting the computation into two parts: vehicle-level planning, such as a vehicle's location at any given time; and mission-level, or health planning, such as the condition of a vehicle's propellers, cameras, and fuel levels.

For vehicle-level planning, he developed a computational approach to POMDP that essentially funnels multiple possible outcomes into a few most-likely outcomes.

"Imagine a huge tree of possibilities, and a large chunk of leaves collapses to one leaf, and you end up with maybe 10 leaves instead of a million leaves," Agha-mohammadi says. "Then you can … let this run offline for say, half an hour, and map a large environment, and accurately predict the collision and failure probabilities on different routes."

He says that planning out a vehicle's possible positions ahead of time frees up a significant amount of computational energy, which can then be spent on mission-level planning in real time. In this regard, he and his colleagues used POMDP to generate a tree of possible health outcomes, including fuel levels and the status of sensors and propellers.

Proactive delivery

The researchers combined the two computational approaches, and ran simulations in which drones were tasked with delivering multiple packages to different addresses under various wind conditions and with limited fuel. They found that drones operating under the two-pronged approach were more proactive in preserving their health, rerouting to a recharge station midmission to keep from running out of fuel. Even with these interruptions, the team found that these drones were able to deliver just as many packages as those that were programmed to simply make deliveries without considering health.

Going forward, the team plans to test the route-planning approach in actual experiments. The researchers have attached electromagnets to small drones, or quadrotors, enabling them to pick up and drop off small parcels. The team has also programmed the drones to land on custom-engineered recharge stations.

"We believe in the near future, in a lab setting, we can show what we're gaining with this framework by delivering as many packages as we can while preserving health," Agha-mohammadi says. "Not only the drone, but the package might be important, and if you fail, it could be a big loss."

This work was supported by Boeing.