A Web Of Things Context-Aware IoT System leveraging Q-learning

The Internet of Things and more recently the Web of Things are changing how we interact with devices. The possibilities and novel services they provide enables the users to perform automatic operations and to monitor data of interest. Although many operations are performed autonomously by devices, there is still the need for the user to understand the data provided, and to configure their own services according to it. In this work we explore the possibility for devices to autonomously organize and understand the effects of the actions on the scenario, and provide a better status of the system. We do so by presenting a novel architecture, and developing a Q-learning algorithm which learns from the different statuses in which the system is. Our results indicate that devices with no prior knowledge of each other may eventually collaborate to provide a novel service to the end user, without any human intervention, and eventually achieve a better system status.