Fig 12
Source publication
Since the eighties, the rapid development in the world of information technology has made possible to create systems that interfere with the user in a harmonious manner. This is due to the emergence of a technology known as multimodal interaction. This technology allows the user to use natural modalities (speech, gesture, eye gaze, etc.) to interac...
Citations
... We present these challenges summary form only in this paper. For more details, please refer to [19], [20], and [21]. ...
Today, technology allows us to produce extensive multimodal systems which are totally under human control. These systems are equipped with multimodal interfaces, which enable more natural and more efficient interaction between man and machine. End users can take advantage of natural modalities (e.g. audio, eye gaze, speech, gestures, etc.) to communicate or exchange information with applications. In this work, we assume that a number of these modalities are available to the user. In this paper, we present a prototype of a multimodal architecture, and show how modality selection and fission algorithms are implemented in such a system. We use a pattern technique to divide a complex command into elementary subtasks and select suitable modalities for each of them. We integrate a context-based method using a Bayesian network to resolve ambiguous or uncertain situations.
... In our work, a modality pattern is composed of: 1) Problem composed of the components: Application, parameter, priority, combination, scenario and service and 2) Solution composed of the chosen modality. For more details concerning scenarios selectionand modalities selection, the readers can refer to [18]. ...
... This class describes the different scenarios. These scenarios saved in patterns form which are mainly composed of two parts problem and solution, as detailed in [18]. Figure 11 shows an example of pattern. ...
Journal of Software Engineering and Applications
Vol. 6 No. 7 (2013) , Article ID: 34132 , 18 pages DOI:10.4236/jsea.2013.67045 Modeling Rules Fission and Modality Selection Using OntologyAtef Zaguia1, Ahmad Wahbi1,2, Moeiz Miraoui1, Chakib Tadj1, Amar Ramdane-Cherif2
●Abstract
●Full-Text PDF
●Full-Text HTML
•Full-Text XML
●Full-Text ePUB
●Linked References
●How to Cite this Article
1MMS Laboratory, Université du Québec, École de Technologie Supérieure, Montreal, Canada; 2LISV Laboratory, Université de Versailles-Saint-Quentin-en-Yvelines, Paris, France.Email: atef.zaguia.1@ens.etsmtl.ca,moeiz.miraoui.1@ens.etsmtl.ca, ctadj@ele.etsmtl.ca, rca@prism.uvsq.frCopyright © 2013 Atef Zaguia et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.Received May 6th, 2013; revised June 7th, 2013; accepted June 15th, 2013Keywords: Multimodal system; Multimodal fission; Modality; Ontology; Interaction context; PatternABSTRACTResearchers in computer science and computer engineering devote a significant part of their efforts on communication and interaction between man and machine. Indeed, with the advent of multimedia and multimodal processing in real time, the computer is no longer considered only as a computational tool, but as a machine for processing, communication, collection and control. Many machines assist and support many activities in daily life. The main objective of this paper is to propose a new methodological solution by modeling an architecture that facilitates the work of multimodal system especially for a fission module. To realize such systems, we rely on ontology to integrate data semantically. Ontologies provide a structured vocabulary usedas support for data representation. This paper provides a better understanding of the fission system and multimodal interaction. We present our architecture and the description of the detection of optimal modalities. This is done by using an ontological model that contains different applicable scenarios and describes the environment where a multimodal system exists.
The current technological advancement has created the need to produce machines more powerful, easy to use and to meet the needs of users. To achieve this goal, multimodal systems have been developed to combine multiple modalities depending on the task, preferences and communicative intentions of the users.In this paper, we present a new approach to overcome uncertain or ambiguous knowledge during the fission process in multimodal system. This approach is context-based method using Bayesian network. We also present a modular and distributed architecture, which is very useful in multimodal systems.
