Smartphone-based augmented reality for end-user creation of home automations

In the last few years, several end-user tools have been designed to help people who are not professional developers in programming their smart environments. However, such tools are often based on structured visual editors providing abstract representations of the available connected sensors and objects, which can be problematic for end users, and do not particularly encourage their participation. This work aims to make the end-user experience of creating everyday automations involving various types of connected sensors and objects more engaging by replacing extensive, static, structured and comprehensive abstract visual tools with more narrowed, relevant, context-sensitive, dynamic, augmented reality-based representations. We present a solution for this purpose that mobile users can exploit through their smartphone. End users can use the smartphone camera to frame the relevant sensor or object through the developed prototype, then get the current automations associated with it, edit their definition, create new ones as well as monitor the automations involving the whole current environment. We also report a first user test of the developed prototype deployed in a home equipped with connected sensors and objects, which yielded positive feedback.