New way to 'see' objects accelerates the future of self-driving cars

The laser sensors currently used to detect 3D objects in the paths of autonomous cars are bulky, ugly, expensive, energy-inefficient -- and highly accurate.

These Light Detection and Ranging (LiDAR) sensors are affixed to cars' roofs, where they increase wind drag, a particular disadvantage for electric cars. They can add around $10,000 to a car's cost. Despite their drawbacks, most experts have considered LiDAR sensors the only plausible way for self-driving vehicles to safely perceive pedestrians, cars and other hazards on the road.

Now, Cornell researchers have discovered that a simpler method, using two inexpensive cameras on either side of the windshield, can detect objects with nearly LiDAR's accuracy and at a fraction of the cost. The researchers found that analyzing the captured images from a bird's eye view rather than the more traditional frontal view more than tripled their accuracy, making stereo cameras a viable and low-cost alternative to LiDAR.

"One of the essential problems in self-driving cars is to identify objects around them -- obviously that's crucial for a car to navigate its environment," said Kilian Weinberger, associate professor of computer science and senior author of the paper, which will be presented at the 2019 Conference on Computer Vision and Pattern Recognition, June 15-21 in Long Beach, California.

"The common belief is that you couldn't make self-driving cars without LiDARs," Weinberger said. "We've shown, at least in principle, that it's possible."

The first author of the paper is Yan Wang, doctoral student in computer science.

Ultimately, Weinberger said, stereo cameras could potentially be used as the primary way of identifying objects in lower-cost cars, or as a backup method in higher-end cars that are also equipped with LiDAR.

The research was partly supported by grants from the National Science Foundation, the Office of Naval Research and the Bill and Melinda Gates Foundation.