Multimedia responses in natural language dialogues

Offering contents to a visitor in a natural and attractive way is one of the most interesting challenges in promoting cultural heritage. In this paper, we present an ongoing research about the design and development of interactive systems based on dialogues in natural language to assist a user during a visit to a cultural space. The responses of system contain multimedia elements and are generated by users' queries or following contextual updates associated to their position, so the system can take initiative in absence of explicit stimuli. The response of system results from a composition process that coherently synchronises media elements with a synthetic voice delivering the textual content. This way, the visitor receives an audio explanation commented by images. To implement this approach, a semantic archive containing the annotation of stories has been built. The formalism used for the annotation is CSWL (Cultural Stories Web Language), used to represent cultural stories through events. A case study on the '800 exhibit at the Capodimonte museum is presented to describe how the system was designed and deployed.