GymCam tracks exercises that wearable monitors can't

The vision-based system, called GymCam, detects repetitive motions. By doing so, Rushil Khurana and Karan Ahuja, both Ph.D. students in CMU's Human-Computer Interaction Institute (HCII), found that they could detect exercises in a gym. Moreover, they could recognize the type of exercise and reliably count repetitions.

"In a gym, the repetitive motion almost always is an exercise," said Mayank Goel, assistant professor in the HCII and Institute for Software Research. "If you are moving both your arms, you tend to move them together in time. However, if two people are exercising next to each other and performing the same exercise, they are usually not in sync, and we can tell the difference between them."

Because the system only needs motion information, the camera feed can be reduced to pixel-by-pixel changes and eliminate identifiable faces that would intrude on privacy.

Khurana said that reliance on motion information also addresses a problem for single-camera systems in a crowded gym environment -- the inability to see a person's whole body. Gym equipment or other people can often obscure the camera's view. GymCam, however, can detect exercise as long as its camera can see any body part moving repetitively.

Khurana and Ahuja will present their findings Thursday, Sept. 12, at the International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp 2019) in London.

Ahuja said smartwatches and other wearables do a reasonable job of tracking many cardio exercises and some strength-training exercises. But their effectiveness depends on where the wearables are worn. A smartwatch might sense a dumbbell lift, but is useless for leg presses. Moreover, it is hard for a watch to differentiate between several body motions. Instrumenting the exercise machines is an option, but an expensive one. A camera, however, is relatively cheap and provides spatial as well as motion information.

The system can also learn the location of types of exercise machines or certain exercise stations in a gym. It can then use an individual's location, in addition to their movements, to determine the exercise they're doing.

The researchers tested their algorithm in a crowded gym. But Goel said that the same algorithm works perfectly on a smartphone as well, so a person can use their phone to record and track their workouts at home. Some companies have already expressed interest in using the system for tracking in-home exercises.

The system also might have uses beyond physical exercise. Goel said the camera system, combined with smartwatches worn by individuals, might help people with visual disabilities navigate shopping malls, airports and other public spaces. Instead of using the person's face as their identity, the system will use their motion as their signature. It allows people to easily opt-out of being tracked or located.