Extracting blood-induced color changes on the face for non-contact heart rate estimation

To address these challenges, a team of researchers from Japan have now developed a novel method that leverages the temporal characteristics of the blood pulse. Importantly, it builds on the ability of the pulse to exhibit quasi-periodic behavior, which distinguishes it from noise artifacts. The study was led by Dr. Yoshihiro Maeda, Junior Associate Professor, from the Department of Electrical Engineering at the Tokyo University of Science and is published in Volume 11 of IEEE Access journal on 9 June 2023. Professor Takayuki Hamamoto and Kosuke Kurihara from the Tokyo University of Science and Associate Professor Daisuke Sugimura from the Department of Computer Science, Tsuda University, were also a part of this study.

The proposed method utilizes dynamic mode decomposition (DMD), a technique that analyzes spatio-temporal structures in multi-dimensional time-series signals. It also employs adaptive selection of the optimal spatio-temporal structure based on medical knowledge of HR frequencies. "Our method, unlike previous applications of DMD, effectively models and extracts the BVP signal by incorporating physics-informed DMD in a time-delay coordinate system, taking into account the nonlinearity and quasi-periodicity of the BVP dynamics," explains Kosuke Kurihara, a Ph.D. student.

The proposed method relies solely on tracking time-series data from videos of a person's face, eliminating the need for any attached detectors on the person's body. In this method, the video time-series of the face, monitoring continuous changes, are converted into RGB time-series signals, which helps in extracting information of blood volume changes occurring beneath the skin. After effectively dealing with noise or misinformation that might creep into the data, the observed RGB signals are then converted to pulse wave information data.

Using the DMD method in a time-delayed coordinate system with conservative dynamics modeling, pulse waves containing major and accurate information can be extracted to estimate HR.

To demonstrate the efficacy of this method, the researchers used 67 facial videos from three publicly available datasets -- namely TokyoTech Remote PPG dataset, MR-NIRP dataset, and UBFC-RPPG dataset. The results of this method were then compared with other non-contact HR estimation methods, including DistancePPG, SparsePPG, SAMC, Hierarchical, and MTTS-CAN.

Interestingly, the proposed method adaptively selects the dynamic mode that contains the most pulse wave components, based on the knowledge of the typical range of pulse wave components. As a result, the method showed a 36.5% improvement in estimation accuracy compared to conventional methods, especially in scenes with ambient light fluctuations.

"This achievement is expected to play a significant role as a fundamental technology for vital monitoring systems in the medical and fitness fields. The breakthrough contactless method holds great potential for non-contact heart rate estimation in various applications, such as remote health monitoring and physiological assessments," concludes Dr. Maeda. The research findings provide new possibilities for enhancing healthcare technologies and improving overall patient comfort and well-being. Going ahead, further research will be needed to explore techniques that incorporate multispectral information, which can contribute to reducing noise and improving the accuracy of the method.

We wish Dr. Maeda and his team luck for their ongoing efforts towards addressing the remaining issues with this novel method.