Towards Multimodal Capture, Annotation and Semantic Retrieval from Performing Arts

A well-annotated dance media is an essential part of a nation's identity, transcending cultural and language barriers. Many dance video archives suffer from tremendous problems concerning authoring and access, because of the multimodal nature of human communication and complex spatio-temporal relationships that exist between dancers. A multimodal dance document consists of video of dancers in space and time, their dance steps through gestures and emotions and accompanying song and music. This work presents the architecture of an annotation system capturing information directly through the use of sensors, comparing and interpreting them using a context and a user's model in order to annotate, index and access multi modal documents.