Accurately estimating human physiological signals such as heart rate from video remains challenging in real world environments. Existing datasets and algorithms often rely on posed expressions or controlled lighting, which fail to represent authentic, spontaneous human behavior. Methods using fixed facial regions are highly sensitive to motion and expression changes, resulting in noisy and unreliable heart rate predictions. Moreover, the lack of comprehensive, multimodal datasets capturing spontaneous expressions limits the development and benchmarking of robust models for affective computing and remote physiological monitoring.
The MMSE HR dataset comprises 102 synchronized video and physiological recordings from 40 subjects, collected using standard video sensors and a Biopac Mp150 system. It captures spontaneous emotional expressions with ground truth heart rate and blood pressure signals. The invention also introduces a Self Adaptive Matrix Completion (SAMC) algorithm that dynamically identifies reliable facial sub regions and denoises chrominance signals for heart rate estimation. This joint optimization approach mitigates noise from movement and illumination changes, enabling robust, real time, non contact physiological monitoring under naturalistic conditions.
• Authentic multimodal dataset captures spontaneous emotional expressions and physiological signals
• Self Adaptive Matrix Completion algorithm enhances robustness to motion, illumination, and expression variability
• Non contact heart rate estimation enables fully remote physiological monitoring
• High quality synchronized recordings support development of advanced AI models
• Broad demographic diversity improves algorithm generalization and cross population accuracy
• Facilitates benchmarking and validation of emotion recognition and health monitoring systems
• Z. Zhang et al. Multimodal Spontaneous Emotion Corpus for Human Behavior Analysis 2016 IEEE Conference on Computer Vision and Pattern Recognition CVPR Las Vegas NV USA 2016 pp. 3438 to 3446 doi 10.1109/CVPR.2016.374
Algorithm and Dataset
This technology is available for licensing.
Relevant for telehealth companies, AI developers, automotive safety system providers, and emotion aware computing platforms seeking validated multimodal datasets and robust remote physiological monitoring algorithms.
Information available upon request.