Figure 5 - uploaded by Jaakko Hakulinen
Content may be subject to copyright.
Source publication
Speech-based applications commonly come with web-based or printed manuals. Alternatively, the dialogue can be designed so that users should be able to start using the application on their own. We studied an alternative approach, an integrated tutor. The tutor participates in the interaction when new users learn to use a speech-based system. It teac...
Context in source publication
Context 1
... viability of the whole integrated tutoring concept de- pends heavily on the type of application that is the target of the tutoring. We have summarized some differences be- tween different types of guidance in speech-based systems in the table found in Figure 5. The evaluations presented here were mostly usability tests detecting the problems of the tutor design and opinions on the tutoring. ...
Citations
... However, human-human social interaction offers many variants besides the one of a subservient, female assistant. Recent work has already begun to broaden the roles IPAs can take in such a relationship, for example, as a friend [52,59,67], companion [43,57], coach [71], or tutor [27], depending on the specific requirements of different situations, such as commuting to work. While an assistant is good at performing tasks, a friend, for example, can be "something" to confide in. ...
... In line with this, recent work explores other roles of IPAs, such as friend [59,67], coach [71], or tutor [27]. Ringfort-Felner et al. [59], for example, designed a fictional, social in-car voice assistant, which did not provide access to the car's functionalities but acted as a virtual passenger with a focus on a meaningful conversation. ...
... According to [21] written in [20], tutored is context sensitive, the tutor takes the pedagogical initiative and guides the student through the system with a structured agenda, uses some explicit instructions and makes sure the user is learning. Benefits of the tutored approach listed by Jaako Hakkulinen and colleagues' include: ...
This paper presents the development of a flashcard based digital storytelling system. It is an online interactive application which allows users to describe their experience and express creativity in the form of a digital story. The system was build based on two important concepts that is tutored approach and flashcards; basically, to draw upon the benefits of both. Contrary to other forms of guided learning, tutored approach was applied as it aim to ensure learning to occur by providing an optimal balance between giving and withholding assistance. Conversely, flashcard which has been widely used as a tool to build up student competence in terminology identification, improve students' visual memory, association, and comprehension, was incorporated as an important tool for the design of the application for the user to create their story.
... These included corpus data (Derriks and Willems, 1998), focus groups (Cowan et al., 2017), user modelling (Schaffer et al., 2015), perceived sense of agency (Limerick et al., 2015), physiological data (Kallinen and Ravaja, 2005) and cognitive walkthroughs (Johnson and Coventry, 2001). One paper was not explicitly clear in the methods for evaluating the iterative design of a tutor system, though it is assumed some form of interview was used (Hakulinen et al., 2004). The frequency of each data collection method is included in Table 5. ...
... These papers analysed multiple versions of a system in developing it based on various types of testing and feedback, or looked at one version of a specific, bespoke system tailored to accomplish a specific task. Hakulinen et al. (2004) explored the design and iterative development of an integrated tutor for teaching new users to operate a speech-based email reading application, while monitoring their actions to they learn how to use it. The authors presented three iterations of the tutor and presented several findings for possible future design for similar systems, including being explicit in telling users what to do, teaching one thing at a time and giving users the opportunities to control the pacing of the system's tutoring. ...
Speech interfaces are growing in popularity. Through a review of 99 research papers this work maps the trends, themes, findings and methods of empirical research on speech interfaces in the field of human–computer interaction (HCI). We find that studies are usability/theory-focused or explore wider system experiences, evaluating Wizard of Oz, prototypes or developed systems. Measuring task and interaction was common, as was using self-report questionnaires to measure concepts like usability and user attitudes. A thematic analysis of the research found that speech HCI work focuses on nine key topics: system speech production, design insight, modality comparison, experiences with interactive voice response systems, assistive technology and accessibility, user speech production, using speech technology for development, peoples’ experiences with intelligent personal assistants and how user memory affects speech interface interaction. From these insights we identify gaps and challenges in speech research, notably taking into account technological advancements, the need to develop theories of speech interface interaction, grow critical mass in this domain, increase design work and expand research from single to multiple user interaction contexts so as to reflect current use contexts. We also highlight the need to improve measure reliability, validity and consistency, in the wild deployment and reduce barriers to building fully functional speech interfaces for research.
RESEARCH HIGHLIGHTS
Most papers focused on usability/theory-based or wider system experience research with a focus on Wizard of Oz and developed systems Questionnaires on usability and user attitudes often used but few were reliable or validated Thematic analysis showed nine primary research topics Challenges identified in theoretical approaches and design guidelines, engaging with technological advances, multiple user and in the wild contexts, critical research mass and barriers to building speech interfaces
... These included corpus data (Derriks & Willems, 1998), focus groups (Cowan et al., 2017), user modelling (Schaffer et al., 2015), perceived sense of agency (Limerick et al., 2015) and physiological data (Kallinen & Ravaja, 2005). One paper was not explicitly clear in the methods for evaluating the iterative design of a tutor system, though it is assumed some form of interview was used (Hakulinen, Turunen, Salonen, & Räihä, 2004). The frequency of each data collection method is included in Table 5. ...
... One paper developed a system designed for healthcare workers to interact with a voice agent for multiple functions and presented the results of its deployment in a user study (Sammon et al., 2006). Hakulinen et al. (2004) explored the design and iterative development of an integrated tutor for teaching new users to operate a speech-based email reading application. Finally, one paper had a different approach to design insight. ...
*Please follow the link on https://lmhclark.com/publications/ to find the published version of this article. The information on metrics in particular differs from this preprint version*
Speech interfaces are growing in popularity. Through a review of 68 research papers this work maps the trends, themes, findings and methods of empirical research on speech interfaces in HCI. We find that most studies are usability/theory-focused or explore wider system experiences, evaluating Wizard of Oz, prototypes, or developed systems by using self-report questionnaires to measure concepts like usability and user attitudes. A thematic analysis of the research found that speech HCI work focuses on nine key topics: system speech production, modality comparison, user speech production, assistive technology & accessibility, design insight, experiences with interactive voice response (IVR) systems, using speech technology for development, people's experiences with intelligent personal assistants (IPAs) and how user memory affects speech interface interaction. From these insights we identify gaps and challenges in speech research, notably the need to develop theories of speech interface interaction, grow critical mass in this domain, increase design work, and expand research from single to multiple user interaction contexts so as to reflect current use contexts. We also highlight the need to improve measure reliability, validity and consistency, in the wild deployment and reduce barriers to building fully functional speech interfaces for research. Author Keywords Speech interfaces; speech HCI; review; speech technology; voice user interfaces Research Highlights • Most papers focused on usability/theory-based or wider system experience research with a focus on Wizard of Oz and developed systems, though a lack of design work • Questionnaires on usability and user attitudes often used but few were reliable or validated • Thematic analysis showed nine primary research topics • Gaps in research critical mass, speech HCI theories, and multiple user contexts
... These included corpus data (Derriks & Willems, 1998), focus groups (Cowan et al., 2017), user modelling (Schaffer et al., 2015), perceived sense of agency (Limerick et al., 2015) and physiological data (Kallinen & Ravaja, 2005). One paper was not explicitly clear in the methods for evaluating the iterative design of a tutor system, though it is assumed some form of interview was used (Hakulinen, Turunen, Salonen, & Räihä, 2004). The frequency of each data collection method is included in Table 5. ...
... One paper developed a system designed for healthcare workers to interact with a voice agent for multiple functions and presented the results of its deployment in a user study (Sammon et al., 2006). Hakulinen et al. (2004) explored the design and iterative development of an integrated tutor for teaching new users to operate a speech-based email reading application. Finally, one paper had a different approach to design insight. ...
Speech interfaces are growing in popularity. Through a review of 68 research papers this work maps the trends, themes, findings and methods of empirical research on speech interfaces in HCI. We find that most studies are usability/theory-focused or explore wider system experiences, evaluating Wizard of Oz, prototypes, or developed systems by using self-report questionnaires to measure concepts like usability and user attitudes. A thematic analysis of the research found that speech HCI work focuses on nine key topics: system speech production, modality comparison, user speech production, assistive technology \& accessibility, design insight, experiences with interactive voice response (IVR) systems, using speech technology for development, people's experiences with intelligent personal assistants (IPAs) and how user memory affects speech interface interaction. From these insights we identify gaps and challenges in speech research, notably the need to develop theories of speech interface interaction, grow critical mass in this domain, increase design work, and expand research from single to multiple user interaction contexts so as to reflect current use contexts. We also highlight the need to improve measure reliability, validity and consistency, in the wild deployment and reduce barriers to building fully functional speech interfaces for research.
... Tutored approach is context sensitive in which the tutor takes the pedagogical initiative and guides the student through the system with a structured agenda, uses some explicit instructions and makes sure that the user is learning [20]. A benefit of the tutored approach listed by Jaako Hakkulinen and colleagues includes [21]: ? Learning occurs in a meaningful context. ...
Assistance giving or guided learning refers to a condition where assistance is needed to motivate and encourage student understanding on the idea of story creation. Tutored approach is one form of guided learning condition designed to create a balance between giving and withholding assistance. This condition then allows learners to get helps, hints, and feedback anytime they need it without disturbing their ability to build knowledge on they own. G-Flash, an authoring tool, was developed based on a flashcard-based guided digital storytelling design framework for digital storytelling which highlights illustrated flashcards and tutored condition as important features of framework. Both illustrated flashcards and 2D animated character were designed as assistance providers in this framework. A study on the story creation process was conducted to examine the roles and usefulness of the assistance giving in encouraging student understanding of the story to be created.
... One of the main reasons for representing the knowledge of an expert is to enable non-experts (either human or machine) to accomplish tasks currently performed by experts. Efforts to create external representations have been used to inform the development of tutoring systems [5,6,10]. In these cases, the goal is to have the nonexpert understand or embody enough of the expert knowledge to enable them to use and manipulate it themselves. ...
Tactical Access to Complex Technology through Interactive Communication (TACTIC) is a user interface prototype developed under
a technology effort designed to provide personnel deployed in the field with access to resources that typically require intercession
by stationary experts with extensive training and experience. We present rationale behind a computer system that judiciously
and beneficially exposes expert knowledge to novices, outline the domains in which such a system is applicable and describe
a prototype interface implementation in the domain of military expeditionary forces.
... We have found an actual software tutor in speech interfaces to be effective as well. Tutoring was delivered in between system prompts and enabled users to learn a speech interface with significantly fewer problems than with a web manual [2]. Incorporating text and graphics has potential benefits of tutoring in a speech interface. ...
We have used text, graphics and non-speech audio to tutor new users in a spoken dialogue system. The guidance is given by a soft- ware tutor, a software component that interactively tutors the user. Four dieren t variations of tutoring were implemented and experiences were collected from user tests in order to gain insights into these tutoring concepts. Real-time visualization of speech interaction with comic book style balloons and structured guidance were received well while various other methods received mixed acceptance.
... We have developed several versions of a software tutor to an e-mail system AthosMail [10], and a bus timetable system Busman. In the e-mail system a speech-only tutor was used successfully [11]. In the Busman application we used a multimodal tutoring. ...
Graphical elements have been found very useful when spoken dialogue systems are developed and demonstrated. However, most of the spoken dialogue systems are designed for speech- only interaction and are very hard to extent to contain graphi- cal elements. We introduce a general model to visualize speech interfaces. Based on the model we present an imple- mented visualization framework, and several example visu- alizations for demonstrations, debugging, and interactive tutoring of speech applications.
... The e-mail systems have been evaluated with several approaches. In these studies, we have found several interesting phenomena from the user diversity point of view; for example, males and females have quite different expectations and perceptions of the speech interface; 40 tutor-based guidance is a promising approach to making the systems accessible for different users; 41 and sighted and visually impaired users prefer different DTMF layouts. When we tested various DTMF layouts, some of which had been designed together with users who were visually impaired, sighted novice users found the layouts to be randomly assigned to the application functionality. ...
Speech can be an efficient and natural means for communication between humans and computers. The development of speech applications requires techniques, methodology, and development tools capable of flexible and adaptive interaction, taking into account the needs of different users and different environments. In this paper, we discuss how the needs of different user groups can be supported by using novel architectural solutions. We present the Jaspis speech application architecture, which introduces a new paradigm for adaptive applications and has been released as open-source software to assist in practical application development. To illustrate how the architecture supports adaptive interaction and accessibility, we present several applications that are based on the Jaspis architecture, including multilingual e-mail systems, timetable systems, and guidance systems.
