Bioscientists use mixed-reality headset, custom software to measure vegetation in the field

Rice researchers set up aMicrosoft HoloLens as a mixed-reality sensor to feed VegSense, their application to measure understory vegetation, plant life that grows between the forest canopy and floor.

A proof-of-concept study by graduate student Daniel Gorczynski and bioscientistLydia Beaudrot shows VegSense could be a suitable alternative to traditional classical field measurements at a low cost.

Their study in Methods in Ecology and Evolution shows the hardware-software combination excels at quantifying relatively mature trees in the wild, which is one measure of a forest's overall health.

Gorczynski came up with the idea to try HoloLens, commonly marketed as a productivity tool for manufacturing, health care and education. He developed the open-source software for the device and noted that while the combination is less effective at picking up saplings and small branches, there's ample room for improvement.

Gorczynski said he was introduced to mixed-reality sensing while an undergraduate at Vanderbilt University and recognized its potential for biological studies. "It seemed sort of like a natural fit," he said. Gorczynski brought the idea to Beaudrot in 2019 shortly after his arrival at Rice.

The combination of stock hardware and custom software cost far less than systems based onlidar (for "light detection and ranging") most often used in three-dimensional field studies, said Gorczynski, who developed VegSense on a platform geared more toward 3D games and interactive experiences than hard science.

Field tests at Houston's Memorial Park showed that at least for mature trees, the smaller solution is just as good. In their case study, VegSense easily detected 48 of 50 such trees in the target area, a circle about 30 feet in diameter that Gorczynski walked, looking up, down and around to build the 3D database. ("Imagine an asterisk with a circle around it," he said, describing the data-capture pattern.)

"For this study, we wanted to be really deliberate in trying to replicate more traditional understory vegetation structure measurements," Gorczynski said. "We tried to get that level of detail."

What he sees as he scans the environment is a holograph-like grid pattern that tracks the surfaces of vegetation. "What's really cool about that is you can see what the scanner is picking up, but also the spots you missed," Gorczynski said. "The idea is to get the mesh to cover as much of the vegetation as possible because that's what gets you the best scan."

"The results were so nice that Dan quickly wrote it up for publication," Beaudrot said, noting that Gorczynski expanded his validation of the gear during a subsequent field trip to Tanzania, the focus of one of 15 tropical forests in a recent rainforest study by the Rice group.

"This device can facilitate a lot of great ecological research, particularly because it's so cost-effective," she said. "Collecting vegetation information on the forest floor right now is really hard to do without a lot of manual labor, or a really expensive lidar system."

"So this is a groundbreaking, cost-effective device," Beaudrot said. "It's not going to give you the same resolution data that lidar will, but this is just the first application. We hope making VegSense open-source to the ecological research community will spur all the potential ways it can be developed."

Northrop Grumman, Conservation International and Rice supported the research.

Video: https://youtu.be/dBHfxvhMChU