Designing interactive experiences to explore artwork collections: A multimedia dialogue system supporting visits in museum exhibits

Speech and natural language processing have a central role in the implementation of systems designed to make the museum more reactive to users' inputs and to improve the overall interaction quality. In this paper, we present the design and implementation of a dialogue system to provide multimedia presentations for museum visits. A corpus of speech recordings in Italian was collected with a mobile application to obtain a reference set of possible ways for the users to express their intentions. On the basis of this corpus, a set of recurring syntactic patterns associated to device requests was extracted to let the dialogue system separate device commands from information queries. Disambiguation strategies depending on the context are also applied in presence of partial syntactic patterns. Information queries are answered by automatically assembling portions of semantically annotated texts and are synchronized with relevant multimedia resources. A case study on the '800 exhibit at the Capodimonte museum in Naples is presented.