MultiSEM: A Mulsemedia Model for Supporting the Development of Authoring Tools – Douglas Paulo de Mattos

Abstract:
This paper proposes MultiSEM (Multimedia Sensory Effect Model), a conceptual model for representing multiple sensory effects in interactive multimedia applications. MultiSEM is an extension of SIMM (Simple Interactive Multimedia Model) and allows the spatiotemporal integration and synchronization of sensory effects with traditional media content. MultiSEM is designed in order to ease the development of mulsemedia authoring tools based on it. To do that, sensory effects are modeled as document nodes, using a similar abstraction as media nodes. The proposed model also aims at supporting the development of authoring tools to be used by those who do not have knowledge of mulsemedia authoring languages and models to create interactive mulsemedia applications. In order to demonstrate that the model can support the development of an authoring tool, MultiSEM is implemented in STEVE 2.0 (Spatio-temporal View Editor), which is an editor for authoring hypermedia applications with sensory effects. Compared with existing mulsemedia models, MultiSEM is the unique model that represents sensory effects as first-class entities and is designed to underlie mulsemedia authoring tools. Besides, an example of mulsemedia application is described and specified using MultiSEM entities. We also present how the application can be created using STEVE 2.0 with sensory effects.

Preparation of Media Object Presentation and Sensory Effect Rendering in Mulsemedia Applications – Marina Ivanov Pereira Josué

Abstract :
Nowadays multimedia applications are employed in different fields as entertainment, education, government services, health and e-commerce. Moreover, some applications incorporate sensory effects along with traditional multimedia content, stimulating other human senses beyond sight and hearing to convey information. Those applications are called mulsemedia applications, where the maintenance of synchronization among media objects and sensory effects is a key point for user quality of experience. In order to minimize delays or failures on content reproduction, this paper proposes a new operation on media objects and sensory effects that can be offered to authors of mulsemedia applications. This operation enables the preparation of media object presentation or the preparation of sensory effect rendering, considering device limitations on which the application will be executed. As proof of concept, our proposal is implemented on multimedia applications using the NCL language, through specification of a new NCL event type, named preparation event. Furthermore, a new NCL media property is presented, allowing the control of continuous media reproduction. Finally, three use cases are defined to demonstrate the use of the preparation event in multimedia and mulsemedia applications.