Talking with Sentiment: Adaptive Expression Generation Behavior for Social Robots

This paper presents a neural-based approach for generating natural gesticulation movements for a humanoid robot enriched with other relevant social signals depending on sentiment processing. In par- ticular, we take into account some simple head postures, voice parame- ters, and eyes colors as expressiveness enhancing elements. A Generative Adversarial Network (GAN) allows the proposed system to extend the variability of basic gesticulation movements while avoiding repetitive and monotonous behavior. Using sentiment analysis on the text that will be pronounced by the robot, we derive a value for emotion valence and co- herently choose suitable parameters for the expressive elements. In this way, the robot has an adaptive expression generation during talking. Ex- periments validate the proposed approach by analyzing the contribution of all the factors to understand the naturalness perception of the robot behavior.