Figure 7 - uploaded by David Traum
Content may be subject to copyright.
Source publication
There is a great need in the Joint Forces to have human to human interpersonal training for skills such as negotiation, leadership, interviewing and cultural training. Virtual environments can be incredible training tools if used properly and used for the correct training application. Virtual environments have already been very successful in traini...
Similar publications
Purpose:
The purpose of this survey study is to compare the experiences of programs and applicants in the MedPhys Match (MPM) in the 2020-21 match cycle with experiences reported from previous match cycles. The 2020-21 match cycle was unique in that recruitment and interviewing were almost exclusively virtual during the COVID-19 pandemic.
Methods...
Citations
... Regarding RQ1 (What is the effect of VA's engagement in group discussions?), there is a general acknowledgment that VA's engaging behaviors contribute to fostering successful interactions with individual users [21,47,50]. However, this study prompts an inquiry into how these effects are transferable to group settings. ...
This study investigates how different virtual agent (VA) behaviors
influence subjects’ perceptions and group decision-making. Participants carried out experimental group discussions with a VA
exhibiting varying levels of engagement and affective behavior.
Engagement refers to the VA’s focus on the group task, whereas
affective behavior reflects the VA’s emotional state.
The findings revealed that VA’s engagements effectively captured participants’ attention even in the group setting and enhanced group synergy, thereby facilitating more in-depth discussion and producing better consensus.
On the other hand, VA’s affective behavior negatively affected the perceived social presence and trustworthiness.
Consequently, in the context of group discussion, participants preferred the engaged and non-affective VA to the non-engaged and affective VA.
The study provides valuable insights for improving the VA’s behavioral design as a team member for collaborative tasks.
... First, ECA's are endowed with the capacity to timely respond to users using nonverbal language such as positive nodding and smiling [28]. This type of responsive behavior is also known as non-verbal entrainment [29] and stimulates the human interlocutor while speaking. ...
Self-guided eHealth has the benefit of providing autonomy to patients. However, the autonomy comes with a cost; elevated attrition rates. Embodied Conversational Agents (‘robots on screen’), have technological capabilities to build rapport with eHealth users and to support them, but are costly to realize and their evidence is inconclusive. We investigated a novel and low-technological method to build rapport. eHealth users synchronized their speech with a monologue-style ECA, a method for which there exists evidence within the human-to-human communication domain. User experiences were investigated using predominantly qualitative methods. As our study results show, users are fairly positive about speaking synchronously with the ECA. However, the experimental task needs refinements. Users need to priorly hear, at least once, the pace of their artificial interlocutor in order to synchronize. Future studies can further examine the refined synchronous speech task and its potential for more widely accessible rapport-building ECA’s aiming for eHealth adherence.
... Virtual humans have been developed to enhance user interaction experience in virtual training environments [25,28]. Such systems integrate various features, including emotion modeling, speech recognition, speech generation, and non-verbal behaviors of virtual humans. ...
... Currently, there is no one-stop software for creating fast 3D scenarios. There has been a lot of research on animation and control of virtual humans [7], include several platforms [10]. ...
... Our results have significant methodological implications. Virtual human training applications are widely used in different fields, like military, psychology, and medicine [3,[17][18][19]35]. We hope that this work will serve as an indication to the virtual human research community to broadly consider the application of scaffolded ping-pong feedback in their training simulations. ...
Virtual human interactions are increasingly used for empathy skills training in healthcare by providing feedback during or after the interaction. The post-interview feedback consists of evaluation results of users' empathic responses and can be provided once without interfering with the interaction. However, this type of feedback is insufficient to engage trainees in obtaining a deeper understanding and insights into their learning. The scaffolded ping-pong feedback consists of a multi-round of descriptions explaining how to formulate desired empathic responses to induce users to explore an understanding of how to respond empathically. To increase the training effectiveness to enhance users' expressed empathy, we studied how to apply scaffolded ping-pong feedback in virtual human interactions to train users' empathy skills. In this paper, we studied how different forms of feedback impact users learning how to express empathy to screen-based virtual humans. To evaluate the training effectiveness, we collected 638 empathic responses from 27 clinician participants in the interaction with two virtual patients integrated with scaffolded ping-pong feedback. We compared them with 809 empathic responses from 25 clinician participants in the post-interview condition. The result shows that the scaffolded ping-pong feedback helped clinician participants to provide a higher percentage of medium-empathy level and a lower percentage of low-empathy level empathic responses than the post-interview feedback. The scaffolded ping-pong feedback was perceived as more difficult to use but did not affect the overall interaction experience with virtual patients. This work demonstrates the applicability of integrating ping-pong feedback to strengthen the training effectiveness of virtual human education interventions.
... In such systems, the virtual agent has the ability to answer users' questions about the tasks or assist them to complete their tasks. However, communication is more than just language and verbal communication and it includes non-verbal factors such as gaze, facial expression, and body language [22,29]. Gaze is widely applied in VR, including gaze modeling in verbal communication [8] and dyadic interaction [44], speech driven gaze [49], and social interaction in a market place [46]. ...
... Collaboration or Training with Virtual Agents. Various concepts have been proposed for collaboration with virtual agents [4,7,9,22,37,43]. For example, task-oriented agents are useful in training and educating humans in performing tasks [32]. ...
... In such applications the agent is 'the system'. In this paper, we consider the agent to be embedded in a broader simulation of a shared human-agent team task environment, in line with for instance research efforts on building virtual humans for training applications [17,18]. ...
In military training simulations there is an increasing need for synthetic agents as team members that are capable of replacing human role players. Human-agent interaction using spoken natural language is an important capability for such agents, especially when communication skills are part of training. Interaction technologies such as speech-to-text, text-to-speech, natural language understanding and dialogue management are maturing, though their use in human-in-the-loop simulations is scarce. In this paper we address design challenges for building conversational agents in the domain of military training. In this domain, agents often have to adhere to specific communication standards, protocols and use of jargon. We propose a data-driven design method to tailor conversational agents to a particular application domain, while minimizing human authoring. We demonstrate this method within an existing military training simulation capability and show that agents are able to effectively build shared situational awareness and coordinated their activities in a shared task environment.
... Likewise, virtual agents have implemented similar cues in order to emulate natural human behavior. These agents have been created for a number of different application areas, including teaching (Andrist, 2013;Baylor & Kim, 2008, 2009Noma et al., 2000;Rickel & Johnson, 1999, coaching (Anderson et al., 2013;Bergmann & Macedonia, 2013;Kang et al., 2008), healthcare (DeVault et al., 2014;Hirsh et al., 2009;Kang et al., 2012), military settings (Kenny et al., 2007;Lee et al., 2007;Traum et al., 2008), as general conversation partners (Buschmeier & Kopp, 2018;Gratch, Wang, Okhmatovskaia et al., 2007;Hartmann et al., 2006;Pelachaud, 2005a), and as virtual assistants Matsuyama et al., 2016;Theune, 2001). In our literature search, we identified papers describing agents that implement each of the NVC types detailed in the previous section. ...
... They then derived a set of rules specifying when an agent should use a specific gesture during its speech. Likewise, the agents by Kenny et al. (2007) also interact using similarly derived rules from the social sciences, behaving appropriately based on each agent's intent. ...
... In addition, researchers have also made attempts to merge languages, but have just created others in the process, as in the case of FML-APML (Mancini & Pelachaud, 2008). Out of all these markup languages, in our review, we saw that BML was the most adopted language (as part of the SAIBA architecture (Kopp et al., 2006)), with use in the Rapport Agent (Huang et al., 2011), SimSensei (DeVault et al., 2014, SARA (Matsuyama et al., 2016), and a number of behavior generation systems (Kenny et al., 2007;Kipp et al., 2010;Lee & Marsella, 2006). However, the other languages were only briefly mentioned in the literature, or limited to a few authors. ...
Virtual agents are systems that add a social dimension to computing, often featuring not only natural language input but also an embodiment or avatar. This allows them to take on a more social role and leverage the use of nonverbal communication (NVC). In humans, NVC is used for many purposes, including communicating intent, directing attention, and conveying emotion. As a result, researchers have developed agents that emulate these behaviors. However, challenges pervade the design and development of NVC in agents. Some articles reveal inconsistencies in the benefits of agent NVC; others show signs of difficulties in the process of analyzing and implementing behaviors. Thus, it is unclear what the specific outcomes and effects of incorporating NVC in agents and what outstanding challenges underlie development. This survey seeks to review the uses, outcomes, and development of NVC in virtual agents to identify challenges and themes to improve and motivate the design of future virtual agents.
... Participants' comments suggest that the more life-like the character, the more likely they are to take it seriously. Realistic appearance of virtual characters has also been shown to be effective in other scenarios such as negotiation, tactical questioning etc. [37]. ...
We present the design of an online social skills development interface for teenagers with autism spectrum disorder (ASD). The interface is intended to enable private conversation practice anywhere , anytime using a web-browser. Users converse informally with a virtual agent, receiving feedback on nonverbal cues in real-time, and summary feedback. The prototype was developed in consultation with an expert UX designer, two psychologists, and a pediatrician. Using the data from 47 individuals, feedback and dialogue generation were automated using a hidden Markov model and a schema-driven dialogue manager capable of handling multi-topic conversations. We conducted a study with nine high-functioning ASD teenagers. Through a thematic analysis of post-experiment interviews, identified several key design considerations, notably: 1) Users should be fully briefed at the outset about the purpose and limitations of the system, to avoid unrealistic expectations. 2) An interface should incorporate positive acknowledgment of behavior change. 3) Realistic appearance of a virtual agent and responsiveness are important in engaging users. 4) Conversation personalization, for instance in prompting laconic users for more input and reciprocal questions, would help the teenagers engage for longer terms and increase the system's utility. CCS CONCEPTS · Human-centered computing → Empirical studies in HCI. KEYWORDS Animated conversational agent design, social skills training, real-time feedback, autism intervention, nonverbal behavior, schema-based dialogue management ACM Reference Format:
... At present, cross-disciplinary research traversing computer graphics and animation, human computer interaction, cognitive science and psychology, has looked into how emerging and new media forms can be used to re-invent storytelling and audiences' experience of storytelling. Towards this end, embodied conversational agents (ECAs) or Virtual Humans (VHs), have been used with much success in virtual reality scenarios for training [9], entertainment and education, and particularly health education [9,17]purposes. Within ICH, ECAs have been used to populate 3D recreated virtual historical or archaeological sites such as ancient Pompeii [14] and Angkor Wat [5] where they are imbued with the appearance of the original inhabitants to increase realism for a more authentic user experience [13]. ...
... At present, cross-disciplinary research traversing computer graphics and animation, human computer interaction, cognitive science and psychology, has looked into how emerging and new media forms can be used to re-invent storytelling and audiences' experience of storytelling. Towards this end, embodied conversational agents (ECAs) or Virtual Humans (VHs), have been used with much success in virtual reality scenarios for training [9], entertainment and education, and particularly health education [9,17]purposes. Within ICH, ECAs have been used to populate 3D recreated virtual historical or archaeological sites such as ancient Pompeii [14] and Angkor Wat [5] where they are imbued with the appearance of the original inhabitants to increase realism for a more authentic user experience [13]. ...
Works of African orature, owing to their performative and participatory nature, are challenging to reproduce in written text, or as linear media in audio or video formats (animation or live action). This has created a research gap in virtual heritage research where emphasis has been placed on digitalization of tangible culture. In my dissertation, I examine the implication of bodily, expressive non-verbal characteristics of performance in traditional African oral storytelling to position their mediation in digital media formats using Embodied Conversational Agents (ECAs). Using the iconic Song of Lawino [2] as a research example for reenactment using ECAs, I illustrate that applying these performance elements and techniques can result in a new digitally embodied oral narrative that mimics the unique relationship in traditional African orature between a storyteller, the space in which the story is told and the audience.
