Multimodal Mood Recognition for Assistive Scenarios

We illustrate a system performing multimodal human emotion detection from video input through the integration of audio emotional recognition, text emotional recognition, facial emotional recognition, and emotional recognition from a spectrogram. The outcomes of the four emotion recognition modalities are compared, and a final evaluation provides the most likely perceived emotion. The system has been devised to be easily implemented on cheap mini-computer based boards. It is conceived to be used as auxiliary tool in the field of telemedicine to remotely monitor the mood of patients and observe their healing process, which is closely related to their emotional condition.