An approach for managing ambiguities in multimodal interaction

Multimodal systems support people with different needs and different features during the interaction process making it more easy and natural. However naturalness can usually produce ambiguous interpretation. This paper discusses ambiguities in multimodal interaction proposing in particular two different examples of ambiguities: a) multimodal ambiguities produced combining un-ambiguous modal information containing contrasting concepts in multimodal dialogue, b) ambiguities arising by the propagation at multimodal level of a modal input ambiguity. Moreover, the paper proposes a formal representation of multimodal inputs using the hybrid approach that combines the attribute multi-set grammar with the linear logic. This paper starts from discussing the main features of a multimodal system and the classes of cooperation between modalities. The attribute multi-set grammar combined with the linear logic enables to highlight the multidimensionality of multimodality and allows to formally describe ambiguities that can occur.