Jani Mäntyjärvi

VTT Technical Research Centre of Finland | vtt

Introduction Current stress detection methods concentrate on identification of stress and non-stress states despite the existence of various stress types. The present study performs a more specific, explainable stress classification, which could provide valuable information on the physiological stress reactions. Methods Physiological responses wer...

Human cognitive capabilities are under constant pressure in the modern information society. Cognitive load detection would be beneficial in several applications of human–computer interaction, including attention management and user interface adaptation. However, current research into accurate and real-time biosignal-based cognitive load detection l...

Personality describes the average behaviour and responses of individuals across situations; but personality traits are often poor predictors of behaviour in specific situations. This is known as the "personality paradox". We evaluated the interrelations between various trait and state variables in participants' everyday lives. As state measures, w...

Acute stress and high workload are part of everyday work at safety critical fields (e.g. health care). Adaptive human computer interaction systems could support and guide a nurse or a doctor in these hectic situations. Seamless interaction between human and computer requires accurate cognitive state estimation of the person. Currently studies are m...

Stress has become a major health concern and there is a need to study and develop new digital means for real-time stress detection. Currently, majority of stress detection research is using population based approaches that lack the capability to adapt to individual differences. They also use supervised learning methods, requiring extensive labeling...

Personality describes the average behaviour and responses of individuals across situations; but personality traits are often poor predictors of behaviour in specific situations. This is known as the “personality paradox”.We evaluated the interrelations between various trait and state variables in participants’ everyday lives. As state measures, we...

Background: Excessive stress has a negative impact on many aspects of life for both individuals and societies, from studying and working to health and well-being. Each individual has their unique level of stress-proneness, and positive or negative outcomes of stress may be affected by it. Technology-aided interventions have potential efficacy in th...

Existing work in human activity detection classifies physical activities using a single fixed-length subset of a sensor signal. However, temporally consecutive subsets of a sensor signal are not utilized. This is not optimal for classifying physical activities (composite activities) that are composed of a temporal series of simpler activities (atom...

Stress has become an important health problem, but existing stress detectors are inconvenient in long-term real-life use because users either have to wear dedicated devices or expend notable interaction efforts in system adaptation to specifics of each person. Adaptation is necessary because individuals significantly differ in their perception of s...

Due to miscommunication in the terms of using the first dataset a correction to the list of authors is required: Added authors, Ilmari Määttänen² and Mikko Lindholm¹.

Connected vehicles and automated driving is currently an emerging area for product development. Connected vehicles are expected to increase traffic safety and efficiency. Standardization activities are focusing on safety related applications but the deployment of V2X communication will also open up new opportunities for various applications which c...

The provisioning of information and capabilities of a Cyber-Physical System (CPS) can be performed directly in the CPS or in external applications and services. A Cyber Physical Production System (CPPS) can be seen to be an example of an evolving CPS, where new elements such as machines can be added to produce desired products. Human interaction wi...

Stress is the major problem in the modern society and a reason for at least half of lost working days in European enterprises, but existing stress detectors are not sufficiently convenient for everyday use. One reason is that stress perception and stress manifestation vary a lot between individuals; hence, “one-fits-all-persons” stress detectors us...

A control system basing on the use of gestures and functioning especially in mobile terminals. The gesture control system is provided with a general purpose interface (320) with its commands for applications (310) to be controlled. The processing software (330) of the gesture signals includes a training program (331), the trained free-form gestures...

Interaction in smart environments should be adapted to the users’ preferences, e.g., utilising modalities appropriate for the situation. While manual customisation of a single application could be feasible, this approach would require too much user effort in the future, when a user interacts with numerous applications with different interfaces, suc...

The objective of the study was to investigate the validity of 3D-accelerometry-based Berg Balance Scale (BBS) score estimation. In particular, acceleration patterns of BBS tasks and gait were targets of analysis. Accelerations of the lower back were measured during execution of the BBS test and corridor-walking for 54 subjects, consisting of neurol...

People access applications and services through different devices depending on location and needs. During a single day, people can use a smartphone, tablet, PC and a TV, sequentially or simultaneously, depending on the context of use. Even within a single task, people often start with one device, such as a smartphone, and continue with another devi...

A mobile terminal includes a user interface specified at least in part by a user. The mobile terminal includes a user interface specification unit that operates in accordance with a programmed data processor to select, in cooperation with the user, at least one trigger and an action, which together form a rule; to automatically generate a directory...

So called "smart products" try to recognise user context and to deliver relevant information upon own initiative, e.g., to advise to buy a windscreen washing liquid or to stir an overheated meal. As variety of usage situations grow, it may become difficult for the users to configure interaction manually in every new case, e.g., to specify via which...

A number of web-based systems provide environmental information to support our ecological lifestyle. However, there exists a need for more accurate and automated online systems facilitating personalised ecological awareness. This paper presents a travel type detection system for an automatic mobile carbon emission calculator. The system is able to...

Cooking assistant is an application that needs to find a trade-off between providing efficient help to the users (e.g., reminding them to stir a meal if it is about to burn) and avoiding users' annoyance. This trade-off may vary in different contexts, such as cooking alone or in a group, cooking new or known recipe etc. The results of the user stud...

Proceedings of the 9th International Conference on Mobile and Ubiquitous Multimedia MUM'10 Mobile communication technology has become an integral part of our everyday lives. Yet, do we recognize the patterns how we communicate with each other? In this paper we describe our work on increasing users' self-awareness of their communication with their s...

Decision making through distributed voting can help automate routine-like collaborative tasks. In this paper we concentrate on how distributed voting strategies can be used for scheduling meetings in mobile and pervasive environments. Our work focuses on optimizing the meeting scheduling result for each participant in a mobile team by using user-sp...

The capability to present advanced graphics in the present mobile devices can be utilized to improve their usability and overall user experience. Mobile devices have limitations compared to PCs due to their inferior computing power and small screens, but a successful design of animated transitions can hide processing delays and make the user experi...

In mobile devices there exist several in-built sensor units and sources which provide data for context reasoning. More context sources can be attached via wireless network connections. Usually, the mobile devices and the context sources are battery powered and their computational and space resources are limited. This sets special requirements for t...

The next wave of mobile applications is at hand. Mobile phones, PDAs, cameras, music players, and gaming gadgets are creating a connected mobile ecosystem where it is possible to implement systems with significant embedded intelligence. Such advances will make it possible to move many functions of the current PC-centric applications to the mobile d...

The informational and instrumental portabilities of mobile devices have made the devices appropriable in various contexts and for various uses. This, then, leads us to ask---how does the always-on usage impact our day-to-day lives? Extensive investigations were carried out to uncover sociotechnical configurations, appropriations, and negotiations d...

This paper presents research findings on designing social features for mobile wellness applications. The focus is on opportunities to support and motivate wellness by utilizing and enhancing social interaction between users. New knowledge is created using a 'Research through Design' process. The process combines findings of the user studies perform...

Social networking service for mobile devices is presented and evaluated. The operating principle of the service is inspired by human-like cumulative gossiping. The usage of the service is browser based and implementation utilises standard ad hoc communication between smartphones. The cumulative gossiping protocol is built on the service level and o...

Context-awareness is one of the rising trends of future mobile technology, and due to advances in technology development, new application and service concepts are being developed and demonstrated in an ever-increasing manner. This workshop brings together researchers and practitioners working on humancomputer interaction (HCI) aspects of context-aw...

Mobile phones with integrated video cameras have become ubiquitous tools that people use both to document everyday surroundings and to express themselves artistically. In this paper we report the findings of a user study on user created mobile videos, where the actions of 11 active mobile video users were documented for 2 weeks, the collected mater...

NGMAST 2009 - 3rd International Conference on Next Generation Mobile Applications, Services and Technologies. Cardiff, Wales, UK, 16 - 18 Sept. 2009, 119 - 124 The mobile devices have a key role in aggregating information while people are on the go. Therefore it would be beneficial to use that potential also for sharing information collaboratively....

Lecture Notes in Computer Science. Volume 5726/2009. Proceedings of the International Conference on Human-Computer Interaction. INTERACT 2009., 866 - 869 Today’s mobile phones are also video cameras. People are using these ubiquitous cameras to document everyday surroundings as well as create more artistic videos. This paper examines emergent mobil...

Lecture Notes in Computer Science. Volume 5726/2009. Proceedings of the International Conference on Human-Computer Interaction. INTERACT 2009., 772 - 781 This paper explores how the user interface of a mobile device could support human perception and conception of changes in its environment. Animated transition effects may hold potential for visual...

Mobile devices have limitations compared to PCs due to their inferior computing power and small screens, but a successful design of animated transitions can hide processing delays and make the user experience smoother. In this paper, we describe the design of animated transitions and present a user study on how they are perceived.

The proliferation of portable devices has transformed our everyday practices, blurring second and third places. However, almost no research exists on how the perpetual possession of devices impacts how we escape them. In this paper, we explore the notion of non-use of portable devices. Drawing from the results of a multi-step qualitative study, we...

Proceedings of the 27th International Conference Extended Abstracts on Human Factors in Computing Systems CHI 2009

MUM 2009 - Proceedings of the 8th International Conference on Mobile and Ubiquitous Multimedia. Cambridge, UK, 22 - 25 Nov. 2009, Article number 1658562

Mobile terminal users have needs for sharing experiences and common interests in a context sensitive manner. However, due to the current division of creation, delivery and access functionality of multimedia to applications, much user effort is needed to communicate efficiently. In this paper an approach for a user interface for mobile terminals to...

In this paper, we present an environment for authoring pervasive multimodal user interfaces. It is composed of a set of XML-based languages, transformations among such languages, and an authoring tool. It provides designers with the possibility of designing interfaces for a wide set of platforms, which support various modalities. We describe how th...

The core of activity recognition in mobile wellness devices is a classification engine which maps observations from sensors to estimated classes. There exists a vast number of different classification algorithms that can be used for this purpose in the machine learning literature. Unfortunately, the computational and space requirements of these met...

In this paper, we present a user study of the heart rate monitors (HRM), which is a commonly used personal wellness technology. HRMs have been used over several years for adjusting the exercise level and in the advanced form, also for measuring the users' fitness level and its progress. The user study included interviews with the HRM users and a su...

Physical activity has a positive impact on people's well-being, and it may also decrease the occurrence of chronic diseases. Activity recognition with wearable sensors can provide feedback to the user about his/her lifestyle regarding physical activity and sports, and thus, promote a more active lifestyle. So far, activity recognition has mostly be...

This paper focuses to a problem domain where several imperfect context recognition devices with restricted resources are located in the same context. The main aim of the paper is to compare different collaborative decision making methods in this domain. In particular, the focus is on certain voting mechanisms and confidence evaluation metrics. Also...

Emerging ubiquitous environments raise the need to support multiple interaction modalities in diverse types of devices. Designing multimodal interfaces for ubiquitous environments using development tools creates challenges since target platforms support different resources and interfaces. Model-based approaches have been recognized as useful for ma...

This paper presents a new interaction technique -scan and tilt- aiming to enable a more natural interaction with mobile museum guides. Our work combines multiple modalities - gestures, physical selection, location, graphical and voice. In particular, physical selection is obtained by scanning RFID tags associated with the artworks, and tilt gesture...

Accelerometer-based gesture control is studied as a supplementary or an alternative interaction modality. Gesture commands freely trainable by the user can be used for controlling external devices with handheld wireless sensor unit. Two user studies are presented. The first study concerns finding gestures for controlling a design environment (Smart...

A context-based, device-centric tool prototype lets users easily customize smart phones, offering low learning costs and high usability. In contrast, we've developed a context-aware solution for existing mobile devices and their applications that people can use independent of external infrastructure. The solution is hence applicable in general mobi...

The need for authenticating users of ubiquitous mobile devices is becoming ever more critical with the increasing value of information stored in the devices and of services accessed via them. Passwords and conventional biometrics such as fingerprint recognition offer fairly reliable solutions to this problem, but these methods require explicit user...

Movement of a living object is measured by a movement sensor carried by one living object at a time for forming one measurement signal per one living object. The measurement signal is compared With a predetermined reference signal Which may be measured from a known living object in a similar Way. A recognition operation is performed based on the co...

Automatic classification of everyday activities can be used for promotion of health-enhancing physical activities and a healthier lifestyle. In this paper, methods used for classification of everyday activities like walking, running, and cycling are described. The aim of the study was to find out how to recognize activities, which sensors are usefu...

Visualization method is proposed as an additional feature for accelerometer-based gesture control. The motivation for visualization of gesture control is justified and the challenges related to visualization are presented. The gesture control is based on Hidden Markov Models. This paper describes basic concepts of the gesture visualization and stud...

Context-aware computing sets new challenges for interaction design. This paper introduces the special characteristics of context-aware mobile devices that should be taken into account in interaction and user interface design, and proposes design guidelines for context-aware mobile applications. The proposed guidelines are evaluated with two design...

Identifying users of portable devices from gait signals acquired with three-dimensional accelerometers was studied. Three approaches, correlation, frequency domain and data distribution statistics, were used. Test subjects (N=36) walked with fast, normal and slow walking speeds in enrolment and test sessions on separate days wearing the acceleromet...

Protecting portable devices is becoming more important, not only because of the value of the devices themselves, but for the value of the data in them and their capability for transactions, including m-commerce and m-banking. An unobtrusive and natural method for identifying the carrier of portable devices is presented. The method uses acceleration...

Context-aware applications are expected to become a remarkable application area within future mobile computing. As mobile phones form a natural tool for interaction between people, the influence of the current context on collaboration is desirable to take into account to enhance the efficiency and quality of the interaction. This paper presents rol...

Accelerometer-based gesture control is proposed as a complementary interaction modality for small handheld devices to enable a variety of multimedia applications. The motivation for experimenting with gesture interaction is justified by the personal and public domain prototype applications developed. The challenges related to developing user-depend...

The quickly emerging usage of multimedia messages offers a new approach for mobile collaboration between users and services by providing a mature and easily accessible technology. This article presents a model for combining multimedia messaging, location awareness of a mobile phone and phone applications, and presents a study of users' experiences...

Quickly emerging usage of multimedia messages offers new approach for mobile and collaboration between users and services by providing a mature and easy accessible technology. This paper investigated possibilities of location related messaging, where pictorial and textual information are combined. We present a model involving contextual information...

Accelerometer based gesture control is proposed as a complementary interaction modality for handheld devices. Predetermined gesture commands or freely trainable by the user can be used for controlling functions also in other devices. To support versatility of gesture commands in various types of personal device applications gestures should be custo...

Knowledge when the terminal is in the hand of the user is important information that can be exploited in mobile applications. We present a touch detection system for mobile terminals based on impedance measurements. Experiments for recognizing touch of various objects are presented. The results show that the system is capable of recognizing if the...

This position paper describes a motivation for a sensor signal data set for exploring context recognition of mobile devices. Test scenarios, collection and pre-processing of data are briefly described. The data set has already been used in various studies on context sharing, rule based context inference for context-aware user interfaces, and patter...

Wearable sensing can be applied for monitoring various diseases. Accelerometers distributed over a person's body convey important information about symptoms of motor system disorders, for example Parkinson's disease. By interpreting the measured data carefully it is possible to exploit obtained information in disease diagnosis, research and treatme...

An implementation of an unobtrusive five-point acceleration sensing wireless body area network (WBAN) with mobile device data logging capabilities is presented, along with practical experiences on performance. Results show good communications performance in laboratory conditions. Field test performance is weaker and the reasons for this are discuss...

This paper introduces an accelerometer-based online gesture recognition system. Recognition of gestures can be utilised as a part of a human computer interaction for mobile devices, e.g. cell phones, PDAs and remote controllers. Gestures are captured with a small wireless sensor-box that produces three dimensional acceleration signal. Acceleration...

Recognizing user context is important in making mobile devices simple to use. The device and the underlying mobile service can provide a personalized user interface that adapts to the usage situation. The device can infer parts of the context of the user from features extracted from on-board sensors that make measurements of acceleration, noise lev...

Context-aware devices are able to take advantage of fusing sensory and application specific information to provide proper information on a situation, for more flexible services, and adaptive user interfaces (UI). It is characteristic for handheld devices and their users that they are continuously moving in several simultaneous fuzzy contexts. The d...

An advanced software framework which simplifies the development of context-aware mobile applications by managing raw context information gained from multiple sources and enables higher-level context abstractions, is discussed. This framework allows recognization of semantic contexts in real time in the presence of uncertain, noisy, and rapidly chan...

In mobile computing, the efficient utilisation of the information gained from the sensors embedded in the devices is difficult. Instead of using raw measurement data application specifically, as currently is customary, higher abstraction level semantic descriptions of the situation, context, can be used to develop mobile applications that are more...

Handheld communication devices equipped with sensing capabilities can recognize some aspects of their context to enable novel applications. We seek to improve the reliability of context recognition through an analogy to human behavior. Where multiple devices are around, they can jointly negotiate on a suitable context and behave accordingly. We hav...

The user expectations for usability and personalization along with decreasing size of handheld devices challenge traditional keypad layout design. We have developed a method for on-line adaptation of a touch pad keyboard layout. The method starts from an original layout and monitors the usage of the keyboard by recording and analyzing the keystroke...

Online communities enabled by mobile communication have needs for sharing experiences and common interests in a context sensitive manner. However, due to the current division of creation, delivery and access functionality of multimedia to applications, much user effort is needed to communicate efficiently. This paper presents an approach for a user...

Recognizing the context of use is im- portant in making mobile devices simple to use. The device and the underlying mobile service can provide a personalized user interface that adapts to the usage sit- uation. The device can infer parts of the context of the user from features extracted from on-board measure- ments of acceleration, noise level, lu...

Context-aware computing is proposed as an enabling technology for adapting different functions in computer-aided devices. The development of context-awareness for mobile devices requires recognition and extraction of implicit context information from the usage situations and environment of a device. Context information is provided for applications...

In mobile computing, the efficient utilisation of the information gained from the sensors embedded in the devices is difficult. Instead of using raw measurement data application specifically, as currently is customary, higher abstraction level semantic descriptions of the situation, context, can be used to develop mobile applications that are more...

Mobile devices, together with their users, are constantly moving from one situation to another. To adapt applications to these changing contexts, the devices must have ways to recognize the contexts. There are various sources for context information: sensors, tags, positioning systems, to name a few. The raw signals from these sources are translate...

The progress in the field of human-computer interaction with hand held electronic devices, such as, personal digital assistants (PDAs) and mobile phones searches for new interaction techniques. Proximity sensing extends the concept of computer-human interaction beyond actual physical contact with a device. In this paper, a virtual keyboard implemen...

The representation of information based on symbol strings has been applied to the recognition of context. A framework for approaching the context recognition problem has been described and interpreted in terms of symbol string recognition. The symbol string clustering map (SCM) is introduced as an efficient algorithm for the unsupervised clustering...

Context-aware appliances are able to take advantage of fusing sensory and application specific information to provide proper information for situation, more flexible services, and adaptive user interfaces. Characteristic for mobile devices and their users is that they are continuously moving in several simultaneous fuzzy contexts. We present an app...

In this paper experiments with acceleration sensors are described for human activity recognition of a wearable device user. The use of principal component analysis and independent component analysis with a wavelet transform is tested for feature generation. Recognition of human activity is examined with a multilayer perceptron classifier. Best clas...

This paper presents an approach for analyzing hand held device usage situation (context) phenomena. The situation information under examination is multidimensional fuzzy feature information derived from multisensor measurements. The analysis is conducted using principal component analysis (PCA) and independent component analysis (ICA). PCA is used...

In this paper we introduce a novel approach to share context to enhance the social quality of remote mobile communication. We provide an analysis of how people start a conversation in situations where they meet physically, especially looking on the influence of the situation. Than this is compared to the way remote communication is initiated using...

Experiments with acceleration sensors is described for static and dynamic gesture recognition of a mobile device user. Static gestures are recognized with the self-organizing mapping scheme of Kohonen while a hidden Markov model is used for recognizing dynamic gestures. An experimental sensor box for the research of context-awareness is also explai...

In this paper, we briefly discuss our ongoing research on the morphing role of portable device ecologies in urban journeys. Specifically, we discuss spirituality and liminality, socialities that emerge from the hybridities of devices and people, and the inevitable intrusiveness caused by perpetual possession of devices.

This paper presents a new interaction technique -scan and tilt- aiming to enable a more natural interaction with mobile museum guides. Our work combines multiple modalities -gestures, physical selection, location, graphical and voice. Physical selection is obtained by scanning RFID tags associated with the artworks, and single handed tilt gestures...

The growing amount of interactive devices and displays around us opens up new possibilities for creating multi-device collaborative environments, enhancing task continuity and migration across devices. The challenges in current multi-device user interface migration environments include sub-optimal techniques for target selection and process initiat...