Improving Multimedia Digital Libraries Usability Applying NLP Sentence Similarity to Multimodal Sentences

his chapter describes multimodality as a means of augmenting information retrieval activities in multimedia digital libraries. Multimodal interaction systems combine visual information with voice, gestures, and other modalities to provide flexible and powerful dialogue approaches. The use of integrated multiple input modes enables users to benefit from the natural approach used in human communication, improving usability of the systems. However, natural interaction approaches may introduce interpretation problems as the systems' usability is directly proportional to users' satisfaction. To improve multimedia digital library usability users can express their queries by means of a multimodal sentence. The authors proposes a new approach to match a multimodal sentence with a template stored in a knowledge base to interpret the multimodal sentence and define the multimodal templates similarity.