MUMBAI, India, July 13 -- Intellectual Property India has published a patent application (202611039649 A) filed by Panjab University; and Iit Mandi Ihub And Hci Foundation on March 30, 2026, for A Standalone Device For Multimodal Multilingual Emotion Detection Using Speech, Facial Expressions, And Rppg.
Inventors include Dr. Nirmal Kaur; Dr. Naveen Aggarwal; Dr. Sarbjeet Singh; and Vinay Sharma.
The application for the patent was published on July 10, 2026, under issue no. 28/2026.
Abstract: The present invention relates to a standalone embedded device for real-time multimodal emotion detection. The device utilizes speech signals, facial expressions, and contactless physiological signals derived from remote photoplethysmography (rPPG) for emotion recognition. A microphone and camera capture audio and video inputs, respectively, while physiological signals are extracted from the same video stream without requiring physical sensors. All processing, including feature extraction, synchronization, classification, and output generation, is performed locally on an embedded processing unit without reliance on external servers or network connectivity. The system supports multilingual and code-mixed speech inputs, including Punjabi, Hindi, and English. The device is configured to maintain performance under variable conditions by utilizing available modalities when one modality is degraded. Detected emotional states are communicated in real time through an LED-based visualization interface, enabling intuitive interpretation. (Referenced figure-2)
Disclaimer: Curated by HT Syndication.
