A multimodal pervasive framework for ambient assisted living

The increasing need to access information everywhere and at any time leads us to believe that future user interfaces, through which users interact with pervasive computing systems, must address both device and modality independence. The pervasive computing paradigm sees almost every object in the everyday environment as a system able to communicate with users and other systems in their own languages. The interaction between users and systems is therefore typically multimodal. The main challenge of multimodal interaction, that is also the main topic of this paper, lies in developing a framework that is able to process information derived from whatever input modalities, giving these inputs an appropriate representation and integrating these individual representations into a joint semantic interpretation. A description of this multimodal pervasive framework will be given in this paper, along with some details of its application in Ambient Assisted Living and the usability test that was implemented to validate its effectiveness.