Legged robots need more testing before real-world use

Led by a team of researchers at The Ohio State University, the study published recently in the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2022 describes a framework for testing and characterizing the safety of legged robots, machines that, unlike their wheeled counterparts, rely on mechanical limbs for movement. The study found that many current legged robotic models don't always act predictably in response to real-life situations, meaning it's hard to predict whether they'll fail -- or succeed -- at any given task that requires movement.

"Our work reveals that these robotic systems are complex and, more importantly, anti-intuitive," said Bowen Weng, a PhD student in electrical and computer engineering at Ohio State. "It means you can't rely on the robot's ability to know how to react in certain situations, so the completeness of the testing becomes even more important."

As mobile robots evolve to carry out more diverse and sophisticated tasks, many in the scientific community also note that the industry needs a set of universal safety testing regulations, especially as robots and other artificial intelligence have gradually begun to flow into our everyday lives. Legged robots especially, which are often made of metal and can run as fast as 20 mph, could quickly become safety hazards when expected to operate alongside humans in real and often unpredictable environments, said Weng.

"Testing is really about assessing risk, and our aim is to investigate how much risk robotics currently presents to users or customers while in a working condition," he said.

While there are currently some safety specifications in place for the deployment of legged robots, Weng noted that there isn't yet any common agreement on how to test them in the field.

This study develops the first data-driven, scenario-based safety testing framework of its kind for legged robots, said Weng.

"In the future, these robots might have the chance to live with human beings side-by-side, and will most likely be collaboratively produced by multiple international parties," he said. "So having safety and testing regulations in place is extremely important for the success of this kind of product."

The research, which was partly inspired by Weng's work as a vehicle safety researcher at the Transportation Research Center, which partners with the National Highway Traffic Safety Administration, takes advantage of sample-based machine learning algorithms to discern how simulated robots would fail during real-world testing.

Though various factors can be used to characterize a robot's overall safety performance, this study analyzed a set of conditions under which the robot would not fall over while actively navigating a new environment. And because many of the algorithms the team used stemmed from previous robotics experiments, they were able to design multiple scenarios for the simulations to run.

One trial focused on studying the robot's ability to move while performing tasks at different gaits, such as walking backward or stepping in place. In another, researchers tested whether the robot would take a tumble if it was periodically pushed with enough force to alter its direction.

The study showed that while one robot failed to stay upright for 3 out of 10 trials when asked to slightly speed up its gait, another could remain upright over 100 trials when pushed from its left side, but fell over during 5 out of 10 trials when the same force was applied to its right side.

Eventually, the researchers' framework could help certify the commercial deployment of legged robots and help establish a safety benchmark for robots created with different structures and properties, though Weng noted it'll be a while before it can be implemented.

"We believe this data-driven approach will help create an unbiased, more efficient way to make observations of robots in the conditions of a test environment," he said. "What we're working towards isn't immediate, but for researchers down the line."

Co-authors were Guillermo Castillo and Ayonga Hereid of Ohio State, and Wei Zhang of the Southern University of Science and Technology in Shenzhen, China. This work was supported by the National Science Foundation and the National Natural Science Foundation of China.