Malte Jung

Cornell University | CU · Information Science

We explore whether robots can positively influence conflict dynamics by repairing interpersonal violations that occur during a team-based problem-solving task. In a 2 (negative trigger: task-directed vs. personal attack) x 2 (repair: yes vs. no) between-subjects experiment (N = 57 teams, 114 participants), we studied the effect of a robot intervent...

Do teams show stable conflict interaction patterns that predict their performance hours, weeks, or even months in advance? Two studies demonstrate that two of the same patterns of emotional interaction dynamics that distinguish functional from dysfunctional marriages also distinguish high from low-performance design teams in the field, up to 6 mont...

Participating in interaction requires not only coordination on content and process, as previously proposed, but also on affect. The term affective grounding is introduced to refer to the coordination of affect in interaction with the purpose of building shared understanding about what behavior can be exhibited, and how behavior is interpreted emoti...

Autonomous robots are increasingly placed in contexts that require them to interact with groups of people rather than just a single individual. Interactions with groups of people introduce nuanced challenges for robots, since robots’ actions influence both individual group members and complex group dynamics. We review the unique roles robots can pl...

Drawing on ethnographic fieldwork in two different teaching hospitals that deployed the da Vinci surgical robot, this paper traces how the introduction of robotics reconfigures the sensory environment of surgery and how surgeons and their teams recalibrate their work in response. We explore the entangled and mutually supportive nature of sensing wi...

In this paper, we investigate how to elicit new perspectives in research-through-design (RtD) studies through annotated portfolios. Situating the usage in human-robot interaction (HRI), we used two robotic artefacts as a case study: we first created our own annotated portfolio and subsequently ran online workshops during which we asked HRI experts...

Successful entrainment during collaboration positively affects trust, willingness to collaborate, and likeability towards collaborators. In this paper, we present a mixed-method study to investigate characteristics of successful entrainment leading to pair and group-based synchronisation. Drawing inspiration from industrial settings, we designed a...

Robots’ proliferation throughout society offers many opportunities and conveniences. However, our ability to effectively employ these machines relies heavily on our perceptions of their competence. In six studies (N = 2,660), participants played a competitive game with a robot to learn about its capabilities. After the learning experience, we measu...

Driven by the vision of future responsive environments, where everyday surroundings can perceive human behaviors and respond through intelligent robotic actuation, we propose Wizard of Props (WoP): a human-centered design workflow for creating expressive, implicit, and meaningful interactions. This collaborative experience prototyping approach inte...

Machines powered by artificial intelligence increasingly permeate social networks with control over resources. However, machine allocation behavior might offer little benefit to human welfare over networks when it ignores the specific network mechanism of social exchange. Here, we perform an online experiment involving simple networks of humans (49...

Robotic appliances are continually being adopted into private homes. However, users have yet to exhibit the same acceptance towards domestic social robots. In this paper, we seek to bridge this issue by augmenting already-existing home appliances with capabilities mimicking social robots. We present a robotic toaster designed with animated movement...

Social distance, or perception of the other, is recognized as a dynamic dimension of an interaction, but yet to be widely explored or understood. Through CORAE, a novel web-based open-source tool for COntinuous Retrospective Affect Evaluation, we collected retrospective ratings of interpersonal perceptions between 12 participant dyads. In this work...

This paper introduces CORAE, a novel web-based open-source tool for COntinuous Retrospective Affect Evaluation, designed to capture continuous affect data about interpersonal perceptions in dyadic interactions. Grounded in behavioral ecology perspectives of emotion, this approach replaces valence as the relevant rating dimension with approach and w...

Artificial intelligence (AI) is already widely used in daily communication, but despite concerns about AI’s negative effects on society the social consequences of using it to communicate remain largely unexplored. We investigate the social consequences of one of the most pervasive AI applications, algorithmic response suggestions (“smart replies”),...

As AI-mediated communication (AI-MC) becomes more prevalent in everyday interactions, it becomes increasingly important to develop a rigorous understanding of its eects on interpersonal relationships and on society at large. Controlled experimental studies oer a key means of developing such an understanding, but various complexities make it dicult...

Horns and sirens are important tools for communicating on the road, which are still understudied in autonomous vehicles. While HRI has explored different ways in which robots could sound, we focus on the range of actions that a single sound can accomplish in interaction. In a Research through Design study involving autonomous shuttle buses in publi...

As autonomous vehicles (AV) become increasingly common on our roads, it is important for first responders-police officers, firefight-ers, and emergency medical services to learn new interaction protocols as they can no longer rely on those applied to human-driven vehicles. This study identifies critical pain points and concerns of first responders...

Despite the recognized need to prepare for a future of human-AI collaboration, the technical skills necessary to develop and deploy AI systems are considerable, making such research difficult to perform without specialized knowledge. To make human-AI collaboration research more accessible, we developed a novel experimental method that combines a st...

We develop data-driven models to predict when a robot should feed during social dining scenarios. Being able to eat independently with friends and family is considered one of the most memorable and important activities for people with mobility limitations. Robots can potentially help with this activity but robot-assisted feeding is a multi-faceted...

This paper presents a new approach for evaluating and controlling expressive humanoid robotic faces using open-source computer vision and machine learning methods. Existing research in Human-Robot Interaction lacks flexible and simple tools that are scalable for evaluating and controlling various robotic faces; thus, our goal is to demonstrate the...

Research in joint action focuses on the psychological, neurological, and physical mechanisms by which humans collaborate with other agents, and overlaps with several domains related to human-robot interaction. The development of artificial systems that can support or emulate the requisite aspects of joint action could lead to improved human-robot t...

In an era of rapid advances in artificial intelligence, the deployment of robots in organizations is accelerating. Further, robotic capabilities are expanding to serve a broader range of leadership behaviors related to task accomplishment and relationship support. Despite the increasing use of robots in various roles across different industries, re...

The increased use of algorithms to support decision making raises questions about whether people prefer algorithmic or human input when making decisions. Two streams of research on algorithm aversion and algorithm appreciation have yielded contradicting results. Our work attempts to reconcile these contradictory findings by focusing on the framings...

In this paper, we investigate how to elicit new perspectives in research-through-design (RtD) studies through annotated portfolios. Situating the usage in human-robot interaction (HRI), we used two robotic artefacts as a case study: we first created our own annotated portfolio and subsequently ran online workshops during which we asked HRI experts...

AI-Mediated Communication (AI-MC) is interpersonal communication that involves an artificially intelligent system that can modify, augment, or even generate content to achieve communicative and relational goals. AI-MC is increasingly involved in human communication and has the potential to impact core aspects of human communication, such as languag...

Artificial intelligence (AI) is now widely used to facilitate social interaction, but its impact on social relationships and communication is not well understood. We study the social consequences of one of the most pervasive AI applications: algorithmic response suggestions ("smart replies"). Two randomized experiments (n = 1036) provide evidence t...

Autonomous robots are increasingly placed in contexts that require them to interact with groups of people rather than just a single individual. Interactions with groups of people introduce nuanced challenges for robots, since robots? actions influence both individual group members and complex group dynamics. We review the unique roles robots can pl...

In recent years, there has been a surge of interest in mobile applications and devices for emotion regulation. Examples include biofeedback devices and mobile applications that help users manage their stress [1][2]. These technologies are often designed to help users reflect on their past or current emotions or prompt users to perform specific task...

Significance It is well-known that the presence of others drastically changes how we behave, yet a majority of social neuroscience studies are limited to single-person neuroimaging experiments. Using simultaneous imaging of multiple interacting brains (also known as hyperscanning), recent studies have started to examine the brain basis underlying s...

BACKGROUND Inhibitory control, or inhibition, is one of the humans' core executive functions. It contributes to our attention, performance, physical and mental wellbeing. Our inhibitory control is modulated by a variety of factors and therefore fluctuates over time. Being able to continuously and unobtrusively assess our inhibitory control will all...

Background Inhibitory control, or inhibition, is one of the core executive functions of humans. It contributes to our attention, performance, and physical and mental well-being. Our inhibitory control is modulated by various factors and therefore fluctuates over time. Being able to continuously and unobtrusively assess our inhibitory control and un...

Research on human-robot collaboration or human-robot teaming, has focused predominantly on understanding and enabling collaboration between a single robot and a single human. Extending human-robot collaboration research beyond the dyad, raises novel questions about how a robot should allocate resources among group members and about what the consequ...

The introduction of artificial teammates in the form of autonomous social robots, with fewer social abilities compared to humans, presents new challenges for human–robot team dynamics. A key characteristic of high performing human-only teams is their ability to establish, develop, and calibrate trust over long periods of time, making the establishm...

People are inherently playful, and playfulness matters not only when engaging in actual play but also in all other activities. Based on this, I propose using a ludic design approach as a means to broaden the design space for Human-Robot Interaction (HRI). In this paper, I discuss the application of ludic design in HRI and explore how ludic activiti...

How should a robot that collaborates with multiple people decide upon the distribution of resources (e.g. social attention, or parts needed for an assembly)? People are uniquely attuned to how resources are distributed. A decision to distribute more resources to one team member than another might be perceived as unfair with potentially detrimental...

Social robots are becoming increasingly influential in shaping the behavior of humans with whom they interact. Here, we examine how the actions of a social robot can influence human-to-human communication, and not just robot–human communication, using groups of three humans and one robot playing 30 rounds of a collaborative game ( n = 51 groups). W...

Conversational agents are increasingly becoming integrated into everyday technologies and can collect large amounts of data about users. As these agents mimic interpersonal interactions, we draw on communication privacy management theory to explore people's privacy expectations with conversational agents. We conducted a 3x3 factorial experiment in...

Drawing on ethnographic fieldwork in two different teaching hospitals deploying the da Vinci surgical robot, this paper traces how the introduction of robotics reconfigures the sensory environment of surgery and how surgeons and their teams recalibrate their work in response. We explore the entangled and mutually supportive nature of sensing within...

AI-mediated communication (AI-MC) represents a new paradigm where communication is augmented or generated by an intelligent system. As AI-MC becomes more prevalent, it is important to understand the effects that it has on human interactions and interpersonal relationships. Previous work tells us that in human interactions with intelligent systems,...

Humans have an extraordinary ability to interact and cooperate with others, which plays a pivotal role in societies at large. Despite its potential social and evolutionary significance, research on finding the neural correlates of collaboration has been limited partly due to restrictions on simultaneous neuroimaging of more than one participant (a....

Much work in robotics and operations research has focused on optimal resource distribution, where an agent dynamically decides how to sequentially distribute resources among different candidates. However, most work ignores the notion of fairness in candidate selection. In the case where a robot distributes resources to human team members, favoring...

In this paper, we draw attention to the social functions of emotional display in interaction. A review of HRI papers on emotion suggests that this perspective is rarely taken in the field, but that it is useful to account for the context-and culture-dependency of emotional expression. We show in two case studies that emotional display is expected t...

A person's emotional state can strongly influence their ability to achieve optimal task performance. Aiming to help individuals manage their feelings, different emotion regulation technologies have been proposed. However, despite the well-known influence that emotions have on task performance, no study to date has shown if an emotion regulation tec...

We describe a physical interactive system for human-robot collaborative design (HRCD) consisting of a tangible user interface (TUI) and a robotic arm that simultaneously manipulates the TUI with the human designer. In an observational study of 12 participants exploring a complex design problem together with the robot, we find that human designers h...

Fast-paced contemporary life full of planned interaction usually makes people miss out on wonderful moments. We here present BubbleBot, a speculative robot designed to support serendipity of interactions in public space. After observations in public spaces and embodied design workshops, we have designed BubbleBot to be a peripheral public-space rob...

Many of the problems we face are solved in small groups. Using decades of research from psychology, HRI research is increasingly trying to understand how robots impact the dynamics and outcomes of these small groups. Current work almost exclusively uses humanoid robots that take on the role of an active group participant to influence interpersonal...

Research on human-robot collaboration or human-robot teaming, has focused predominantly on understanding and enabling collaboration between a single robot and a single human. Extending human-robot collaboration research beyond the dyad, raises novel questions about how a robot should distribute resources among group members and about what the socia...

Virtual reality experiences via immersive optics and sound are becoming ubiquitous; there are several consumer systems (e.g., Oculus Rift and HTC Vive) now available with these capabilities. Other sensory experiences, such as that of touch remain elusive in this field. The most successful examples of haptic sensation (e.g., Nintendo 64's Rumble Pac...

This paper investigates how a teleoperated surgical robot reconfigures teamwork in the operating room by spatially redistributing team members. We report on findings from two years of fieldwork at two hospitals, including interviews and video data. We find that while in non-robotic cases team members huddle together, physically touching, introducti...

Wizard-of-Oz (WoZ) is a technique that is popular for prototyping and generating expressive movement behaviours in the field of Human-Robot Interaction (HRI). This technique often has people (usually experimenters or trained confederates) remotely pilot a robot using a control interface in order to simulate an artificially intelligent robot. Resear...

In this exploratory study, we examine how personification and interactivity may influence people's disclosures around sensitive topics, such as psychological stressors. Participants (N=441) shared a recent stressful experience with one of three agent interfaces: 1) a non-interactive, non-personified survey, 2) an interactive, non-personified chatbo...

https://dl.acm.org/citation.cfm?id=3208975

Interpersonal distance behaviors can vary significantly across countries and impact human social interaction. Do these cross-cultural differences play out when one of the interaction partners participates through a teleoperated robot? Emerging research shows that when being approached by a robot, people tend to hold similar cultural preferences as...

Foreign language students must learn to use language creatively to overcome knowledge gaps and keep readers or listeners interested. However, few tools exist to support practicing this skill. Therefore, we set out to explore design of storytelling games for practicing creative language use. Through an iterative design process, we identified narrati...

Emotions play a major role in how interpersonal conflicts unfold. Although several strategies and technological approaches have been proposed for emotion regulation, they often require conscious attention and effort. This often limits their efficacy in practice. In this paper, we propose a different approach inspired by self-perception theory: noti...

Despite a growing body of research about the design and use of conversational agents, existing work has almost exclusively focused on interactions between an agent and a human. Less is known about how an agent is perceived and used during human-human conversation. We compared conversations between dyads using AI-assisted and standard messaging apps...

A 2 (mediator: human, robot) x 3 (humor styles: affiliative, aggressive, self-defeating) factorial experiment was conducted to test participant perceptions of robot-enacted humor in conflict mitigation. Participants watched brief video vignettes of roommate conflict in which either a robot or human employed humor. Funniness ratings did not differ s...

The ability to constructively resolve interpersonal conflicts is a crucial set of social skills people need to effectively work and live well together. Is it possible to design social robots to support the early development of children's interpersonal conflict resolution skills? To investigate this question, 64 (32 pairs of) children ages 3-6 years...

Successful teams are characterized by high levels of trust between team members, allowing the team to learn from mistakes, take risks, and entertain diverse ideas. We investigated a robot's potential to shape trust within a team through the robot's expressions of vulnerability. We conducted a between-subjects experiment (N = 35 teams, 105 participa...

Telepresence robots provide remote team members with embodied presence, but whether this improves remote teammate participation, remote users' perceptions of team collaboration, or collocated members' perceptions of remote teammates is an open question. We conducted an experiment in which teams of two collocated members and one telepresent (remote)...

1 When people communicate, their messages convey affect alongside informational content. The affective dimension of messages is often unclear and open to multiple interpretations especially in an intercultural context. Thus, interlocutors may or may not achieve a state of affective grounding in which each person's affective behaviors are correctly...

Robots intended for social contexts are often designed with explicit humanlike attributes in order to facilitate their reception by (and communication with) people. However, observation of an “uncanny valley”—a phenomenon in which highly humanlike entities provoke aversion in human observers—has lead some to caution against this practice. Both of t...

How does a robot's sound shape our perception of it? We overlaid sound from high-end and low-end robot arms on videos of the high-end KUKA youBot desktop robotic arm moving a small block in functional (working in isolation) and social (interacting with a human) contexts. The low-end audio was sourced from an inexpensive OWI arm. Crowdsourced partic...

Robots intended for social contexts are often designed with explicit humanlike attributes in order to facilitate their reception by (and communication with) people. However, observation of an "uncanny valley" – a phenomenon in which highly humanlike entities provoke aversion in human observers – has lead some to caution against this practice. Both...

Previous studies indicate that the way we perceive our bodily signals, such as our heart rate, can influence how we feel. Inspired by these studies, we built EmotionCheck, which is a wearable device that can change users' perception of their heart rate through subtle vibrations on the wrist. The results of an experiment with 67 participants show th...

Over the last decade, the idea that robots could participate meaningfully in complex human contexts such as groups and organizations has developed from a promising vision into a reality. Robots now assist human collectives in simple tasks such as delivery through complex high-stakes tasks such as disaster response or surgery. Despite this dramatic...

Authentic foreign language videos are effective for developing pragmatic competence, or sensitivity to meanings expressed by tone and word choice, and the ability to effectively express these meanings. However, established methods for learning from foreign language videos are primarily text-based (e.g.captioning). Using text, learners do not practi...