Figure 1 - uploaded by Eva Eriksson
Content may be subject to copyright.
Source publication
In this paper, we describe a new interaction technique for mobile devices named Mixed Interaction Space that uses the camera of the mobile device to track the position, size and rotation of a fixed-point. In this demonstration we will present a system that uses a hand-drawn circle, colored object or a person's face as a fixed-point to determine the...
Context in source publication
Context 1
... Interaction Spaces, Mobile Interaction, Computer Vision, Mobile Phones, Interaction Techniques, Face Tracking The majority of interaction with mobile devices falls into two categories: It is either done by using a keypad or by using a stylus. Each of these input techniques has its limitation and interaction with mobile devices can be cumbersome. The keypad is good at dialing numbers, can be used to write small text messages, but is generally not good at supporting games or pointing and dragging applications. The pen is good for pointing but requires the use of two hands. As mobile devices move from being mainly a phone or a notebook to becoming a multifunctional device there is a need for developing new interaction techniques for mobile devices. Speech interaction is coming to mobile devices, but is still limited to recognizing a small set of predefined commands. Accelerometer based interaction is another promising new interaction form, but instead of using movement we wanted to explore how the space the mobile device inhabit can be used as interaction space. For some time we have developed an interaction concept for mobile devices called Mixed Interaction Space (MIXIS) [2][3]. Where several of the other input techniques focus on interaction on the device, MIXIS uses the space surrounding the mobile device for its input. We track the location of the mobile device by using its built-in camera to detect a fixed-point in its surroundings and use this fixed-point to determine the position and rotation of the device in the 3D space. The position of the mobile phone in the space is thereby transformed into a 4 dimensional input vector. One of the advantages of MIXIS is that it uses the camera that most mobile devices are equipped with instead of additional or specialized external hardware. Several other projects have worked with input for mobile devices [4][5][7], but all of them have used two dimensional barcodes as their fixed-point and none of them have explored the possibilities of the interaction space spanned by the camera’s field of vision. By using the camera to track a fixed-point, a mixed reality interface is created. The interface is a combination of the physical space the device is moved in and the digital space where the movement is mapped to interactions in different applications. The mixed interaction space has the shape of an inverse pyramid. When the device is close to the fixed-point the space is small, and the further away from the fixed-point the device is moved the larger the space becomes. Moving the phone in the interaction space can be mapped to actions in the graphical user interface on the device or an action on a nearby device or display. Figure 1 shows the concepts. We have worked with a set of different fixed-points. In the first version we use a hand-drawn circle as a fixed-point. In a later version we use a coloured object and in the newest version, we use a person’s face as the fixed point and track it with a mobile device that has a camera facing towards the user e.g. the Nokia 6680. We have used two different algorithms for performing the actual tracking of the device. In the first version we use the Randomized Hough Circle detection algorithm [6], optimized to run on mobile devices, to find the location of the device. The algorithm is pretty robust and works with different lightning conditions, and the circle do not need to be perfect, a hand drawn circle works as well as a printed. The second algorithm we use is an optimized version of the CamShift algorithm developed at Intel [1]. This algorithm uses a color histogram to guide a mean shift algorithm and is able to track an object with a distinct color signature or the users face if the front camera on the mobile device in use. Especially the face tracking algorithm provides new possibilities since it does not require any external fixed-point. The fixed-point is always with the user, since being a part of the user, and is therefore highly suitable as interaction technique for mobile use. We have built a number of applications based on Mixed Interaction Space. ImageZoomViewer, DrawME and Layered Pie Menu are examples of applications that run on the phone. ImageZoomViewer is an application for panning and zooming large pictures by moving the device in the 3D space (figure 2). DrawME recognized a hand drawn symbol inside a circle and this icon can be used as a short cut for calling e.g. a phone number (only with the circle algorithm) and Layered pie menu present a pie menu in which the user browse through the different layers of the menu by moving the device closer or further away. An item can be selected by moving the device away from the center. BlueMix and BlueServer are two generic components for developing MIXIS applications to control interaction on remote computers with the mobile devices. BlueMix is a program for the mobile phone that connects to a nearby PC and streams the input data from the MIXIS algorithm to the computer. BlueServer is a PC application that communicates with one or more BlueMix application over Bluetooth. Other clients e.g. developed in Flash or C# can connect to a socket port and thereby communicate with the BlueServer. Some of the application we have developed with this setup are Drozo where a user can drag, zoom and rotate images on a large wall display, multi-user Pong where two users can play pong on a large screen controlled by MIXIS and a multi- user cursor system where up to seven users can control a cursor with their mobile devices. Based on the work with MIXIS we have showed that MIXIS is a novel concept for interacting with mobile devices that takes advantage of the mobile device being part of a physical world instead of being a fixed computer tied to a desktop. The goal of Mixis is not to replace the keypad or pen, but to provide an additional 4D input method that can be used in a wide set of applications where e.g. physical movement provides a better mapping than pressing a key or pointing at an icon. We are currently working on developing several new applications and tracking algorithms based on the concept and we also focus on making the system more generic so it is easier to plug it into new ...
Similar publications
Objective: Nomophobia is defined as the involuntary fear experienced by the individual when he/she cannot access his mobile device or cannot communicate on the mobile device. This study aimed to examine nomophobia and related factors in the first-and fourth-year students in the Department of Turkish Language and Literature and Western Languages and...
Delivering immersive, 3D experiences for human communication requires a method to obtain 360 degree photo-realistic avatars of humans. To make these experiences accessible to all, only commodity hardware, like mobile phone cameras, should be necessary to capture the data needed for avatar creation. For avatars to be rendered realistically from any...
La revolución tecnológica, ha traído para el desarrollo de la sociedad varios aspectos positivos, sin embargo, también ha generado diversos desafíos asociados a las denominadas tecnoadicciones, dentro de los cuales se cuenta la dependencia al teléfono móvil, un tipo de adicción tecnológica que limita el desarrollo integral de los jóvenes estudiante...
The vibrating of concrete is one of the most important procedures that directly determines the quality of construction projects. The concrete vibration quality in field construction is mainly judged by the experience of workers, lacking quantitative indicators, and necessary supervision. However, the lack of research about concrete vibration qualit...
High noise values and poor signal to noise ratio are traditionally associated to the low light imaging. Still, there are several other camera quality features which may suffer from low light environment. For example, what happens to the color accuracy and resolution or how the camera speed behaves in low light? Furthermore, how low light environmen...
Citations
... This idea was originally proposed by Fitzmaurice as spatially aware displays [4] [5]. More recent work includes peephole displays [18], mixed interaction spaces [10] [11], as well as camerabased interfaces for two-handed input [8] [9]. An important class of applications for mobile devices concerns data that has an inherently spatial layout, such as map-based applications, or data that is conveniently viewed and interacted with in a 2-D (or 3-D) arrangement, such as calendar applications. ...
Information navigation techniques for handheld devices sup- port interacting with large virtual spaces on small displays, for example finding targets on a large-scale map. Since only a small part of the virtual space can be shown on the screen at once, typical interfaces allow for scrolling and panning to reach o-screen content. Spatially aware handheld dis- plays sense their position and orientation in physical space in order to provide a corresponding view in virtual space. We implemented various one-handed navigation techniques for camera-tracked spatially aware displays. The techniques are compared in a series of abstract selection tasks that re- quire the investigation of dierent levels of detail. The tasks are relevant for interfaces that enable navigating large scale maps and finding contextual information on them. The re- sults show that halo is significantly faster than other tech- niques. In complex situations zoom and halo show compara- ble performance. Surprisingly, the combination of halo and zooming is detrimental to user performance.
... Visual tracking for physical mobile interaction has been described in [13, 14, 23]. Hansen et al. [14, 15] track hand-drawn circles, colored objects, and faces, in order to implement mixed interaction spaces: physical spaces in which digital interaction takes place. Hachet et al. [13] propose a camera-based interface for two-handed input. ...
We demonstrate that mobile phones can be used as an actively oriented handheld musical performance device. To achieve this we use a visual tracking system of a camera phone. Motion in the plane, relative to movable targets, rotation and distance to the plane can be used to drive MIDI enabled sound generation software or hardware. Mobile camera phones are widely available technology and we hope to find ways to make them into viable and widely used performance devices.
... We have re-implemented several of the applications from the original MIXIS with the face tracking technique and designed some novel applications. The MIXIS face tracking project is further described in [5]. ...
In this position paper I will try to address the workshop topic named: Interaction design aesthetics -the experimental practice. The focus of my recent research work has not been consciously directed towards aesthetics, but I wish to reflect more upon this in my future work. I believe I can contribute to the discussion in the field of aesthetic interaction through reflecting on my practical experience. I wish to participate in the workshop to join the discussions in the emerging field of aesthetic interaction both for my research work and for my teaching. Based on my research I wish to present aspects of interaction that I consider important when designing for the aesthetic experience.
... because the phone is being used close to the scanned surface, there is often shadow from the phone and users hand. Adaptive thresholding, thus, would be the better choice to achieve the greatest quality. A mobile phone system that is most like our proposed system is a Mixed Interaction Spaces. It allows a user to use a mobile phone as a 3D cursor.-Olesen, 2005). The technique is to use the space surrounding the mobile device. In particular, it uses the human face as a physical mark instead of printed or hand drawn mark. All space surround the mobile device is available for use. When the camera is moving, the space is changing, then, the change is converted as 4 dimensional vector as input. Th ...
With the development of hardware, mobile phone has become a feature-rich handheld device. Built-in camera and Bluetooth technology are supported in most current mobile phones. A real-time image processing experiment was conducted with a SonyEricsson P910i smartphone and a desktop computer. This thesis describes the design and implementation of a system which uses a mobile phone as a PC mouse. The movement of the mobile phone can be detected by analyzing the images captured by the onboard camera and the mouse cursor in the PC can be controlled by the movement of the phone.
In this paper we present three concepts that address movement-based interaction using video tracking. Based on our work with several movement-based projects we present four selected applications and use these applications to leverage our discussion and to describe our three main concepts space, relations, and feedback. We see these as central for describing and analyzing movement-based systems using video tracking and we show how these three concepts can be used to analyse other video tracking applications.
In this paper we present three concepts that address movement-based interaction using camera tracking. Based on our work with
several movement-based projects we present four selected applications, and use these applications to leverage our discussion,
and to describe our three main concepts space, relations, and feedback. We see these as central for describing and analysing movement-based systems using camera tracking and we show how these three
concepts can be used to analyse other camera tracking applications.
