Interplay of rhythmic and discrete manipulation movements during development: a policy-search reinforcement-learning robot model

The flexibility of human motor behavior strongly relies on rhythmic and discrete movements. Developmental psychology has shown how these movements closely interplay during development, but the dynamics of that are largely unknown and we currently lack computational models suitable to investigate such interaction. This work initially presents an analysis of the problem from a computational and empirical perspective and then proposes a novel computational model to start to investigate it. The model is based on a movement primitive capable of producing both rhythmic and end-point discrete movements, and on a policy search reinforcement learning algorithm capable of mimicking trial-and-error learning processes underlying development and efficient enough to work on real robots. The model is tested with hand manipulation tasks ("touching," "tapping," and "rotating" an object). The results show how the system progressively shapes the initial rhythmic exploration into refined rhythmic or discrete movements depending on the task demand. The tests on the real robot also show how the system exploits the specific hand-object physical properties, some possibly shared with developing infants, to find effective solutions to the tasks. The results show that the model represents a useful tool to investigate the interplay of rhythmic and discrete movements during development.