Fig 2 - available via license: Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International
Content may be subject to copyright.
Source publication
This paper presents a tool for supporting operators in shared industrial workplaces where humans and robots coexist. The tool has been developed in the form of software application for wearable devices, such as smartwatches, and provides functionalities for direct interaction with the robot. Interfaces to audio commands, manual guidance application...
Similar publications
Global trends like urbanization, digitization, and climate change enforce and accelerate the change of personal mobility. New mobility services and growing transportation networks give travelers more choices. Combining traditional mobility services, like trains, with emerging services, like electric carsharing, enables fast, low-priced and environm...
Regardless of age, it is always important to detect deviations in long-term blood pressure from normal levels. Continuous monitoring of blood pressure throughout the day is even more important for elderly people with cardiovascular diseases or a high risk of stroke. The traditional cuff-based method for blood pressure measurements is not suitable f...
Here we report the lightweight mechanically flexible, thermally insulating rGO/polyimide nanocomposite foam with anisotropic microstructure. The effect of ration between graphene oxide and polyimide on the microstructure was systematically studied. By optimizing the ratio between GO and PI, the rGO/PI nanocomposite exhibits hierarchically aligned m...
There is currently strong demand for the development of advanced energy storage devices with inexpensive, flexibility, lightweight, and eco-friendly materials. Cellulose is considered as a suitable material that has the potential to meet the requirements of the advanced energy storage devices. Specifically, nanocellulose has been shown to be an env...
Citations
... Several approaches towards the efficient interaction of human and robotic resources in HRC scenarios have been presented in the literature. Existing approaches are based on Augmented Reality (AR) techniques [13][14][15], smartphones [16], tablets [17,18], smartwatches [19] and voice-based interfaces [20]. In this paper, special emphasis is given to voice-based approaches due to the fact that human operators are able to interact with the robot without needing to stop their actions and use their hands to interact with the required interfaces. ...
Quality inspection plays a vital role in current manufacturing practice since the need for reliable and customized products is high on the agenda of most industries. Under this scope, solutions enhancing human-robot collaboration such as voice-based interaction are at the forefront of efforts by modern industries towards embracing the latest digitalization trends. Current inspection activities are often based on the manual expertise of operators, which has been proven to be time-consuming. This paper presents a voice-enabled ROS2 framework towards enhancing the collaboration of robots and operators under quality inspection activities. A robust ROS2-based architecture is adopted towards supporting the orchestration of the process execution flow. Furthermore, a speech recognition application and a quality inspection solution are deployed and integrated to the overall system, showcasing its effectiveness under a case study deriving from the automotive industry. The benefits of this voice-enabled ROS2 framework are discussed and proposed as an alternative way of inspecting parts under human-robot collaborative environments. To measure the added value of the framework, a multi-round testing process took place with different parameters for the framework's modules, showcasing reduced cycle time for quality inspection processes, robust HRI using voice-based techniques and accurate inspection.
... This trend towards greater AGV adoption for transport processes signifies the growing coexistence between humans and robots within smart factories [3]. In addition, companies are proactively integrating smart devices to enhance worker productivity [4][5][6] promote well-being [6][7][8] and ensure safety [8][9][10]. These devices provide workers with instructions and insights relevant to their tasks. ...
The integration of Autonomous Guided Vehicles (AGVs) into smart factories is transforming modern manufacturing, creating coexistence between humans and robotic systems. In this evolving landscape, one critical aspect is the seamless coordination of AGVs and human workers within factory settings. To achieve this, our research presents a portable indoor localization system that utilizes ESP32 microcontrollers as compact access points. Using Wi-Fi Fine Time Measurement (FTM) with smartphones, the system estimates worker positions through multi-lateration techniques in conjunction with advanced filtering methods. This localization system serves as a pivotal bridge, ensuring that AGVs can interact with and respond to the movements of workers within warehouses. A field study in an actual warehouse environment validates the system's performance , demonstrating 1.13 m accuracy in lateral movements. Furthermore, its localization capabilities within specific warehouse areas showcase its potential to enhance order picking processes and optimize human-AGV interaction.
... The research areas within this project can be categorized into five main areas of focus. Firstly, they introduced an innovative MM presented in Figure 14 [50] and their application in assembly systems [98], and they enhanced the HRI and HRC by introducing wearable devices such as smartwatches [99] and by employing AR-based software [100,101]. The development of a digital world model [102] and DTs for designing and optimizing the design was investigated in [103,104]. ...
In the realm of Industry 4.0, diverse technologies such as AI, Cyber-Physical Systems, IoT, and advanced sensors converge to shape smarter future factories. Mobile manipulators (MMs) are pivotal, fostering flexibility, adaptability, and collaboration in industrial processes. On one hand, MMs offer a remarkable level of flexibility, adaptability, and collaboration in industrial processes, facilitating swift production line changes and efficiency enhancements. On the other hand, their integration into real manufacturing environments requires meticulous considerations, such as safety, human–robot interaction, and cybersecurity. This article delves into MMs’ essential role in achieving Industry 4.0’s automation and adaptability by integrating mobility with manipulation capabilities. The study reviews MMs’ industrial applications and integration into manufacturing systems. The most observed applications are logistics (49%) and manufacturing (33%). As Industry 4.0 advances, the paper emphasizes updating and aligning MMs with the smart factory concept by networks of sensors and the real-time analysis of them, especially for an enhanced human–robot interaction. Another objective is categorizing considerations for MMs’ utilization in Industry 4.0-aligned manufacturing. This review methodically covers a wide range of considerations and evaluates existing solutions. It shows a more comprehensive approach to understanding MMs in Industry 4.0 than previous works. Key focus areas encompass perception, data analysis, connectivity, human–robot interaction, safety, virtualization, and cybersecurity. By bringing together different aspects, this research emphasizes a more integrated view of the role and challenges of MMs in the Industry 4.0 paradigm and provides insights into aspects often overlooked. A detailed and synthetic analysis of existing knowledge was performed, and insights into their future path in Industry 4.0 environments were provided as part of the contributions of this paper. The article also appraises initiatives in these domains, along with a succinct technology readiness analysis. To sum up, this study highlights MMs’ pivotal role in Industry 4.0, encompassing their influence on adaptability, automation, and efficiency.
... The scientific theory supporting the impact of workload on performance is Wickens' multiple resource theory which states that humans have only a limited amount of cognitive resources to dedicate to a task, and when the task at hand exceeds the available resources it leads to inefficiency and deteriorated performance [74]. While prior studies have reported similar findings where AR UI has a negative impact on user performance, a few other studies have reported the efficacy of augmented reality tools in human-robot collaboration [12], [15], [17]. Although our results indicate that using the AR UI had a negative impact on travel distance and travel speed, it should be noted that all the participants successfully completed the task. ...
... Augmented reality (AR) has been adopted in many industries to improve efficiency, safety, and quality in various processes [8][9] [10]. One of the advantages of using AR is its ability to enhance safety. ...
This paper proposes a novel operation for controlling a mobile robot using a head-mounted device. Conventionally, robots are operated using computers or a joystick, which creates limitations in usability and flexibility because control equipment has to be carried by hand. This lack of flexibility may prevent workers from multitasking or carrying objects while operating the robot. To address this limitation, we propose a hands-free method to operate the mobile robot with a human gaze in an Augmented Reality (AR) environment. The proposed work is demonstrated using the HoloLens 2 to control the mobile robot, Robotnik Summit-XL, through the eye-gaze in AR. Stable speed control and navigation of the mobile robot were achieved through admittance control which was calculated using the gaze position. The experiment was conducted to compare the usability between the joystick and the proposed operation, and the results were validated through surveys (i.e., SUS, SEQ). The survey results from the participants after the experiments showed that the wearer of the HoloLens accurately operated the mobile robot in a collaborative manner. The results for both the joystick and the HoloLens were marked as easy to use with above-average usability. This suggests that the HoloLens can be used as a replacement for the joystick to allow hands-free robot operation and has the potential to increase the efficiency of human-robot collaboration in situations when hands-free controls are needed.
... An experimental study verified that the user determined where the robot was going to move using the interface more quickly and accurately than 2D display or no assisted visualization. Gkournelos et al. [14] developed an AR assisted system to support human operators in an industrial workplace shared with robots. The system allowed direct interaction with the robots via visual and audio communication in wearable devices. ...
... This technology can integrate virtual information in the operators workspace [35,42], helping them in assembly tasks [18,43,49], provide context-aware assistance [5], data visualization and interaction (acting as a Human-Machine Interface (HMI)) [16,40], indoor localization [60], maintenance applications [8,18,61], quality control [4,65], material management [16,51] or remote collaboration [7,39,66], by presenting additional layers of digital information on top of real-world environments [3,28,33,37,38,57]. Prior studies identify certain benefits of applying AR for technological industrialization, like increased work safety, effective learning and training, as well as more task effectiveness [10,12,31], as well as improved Human-Robot Interaction (HRI) [1,13,19,34]. ...
Augmented Reality (AR) is a pillar of the transition to Industry 4.0 and smart manufacturing. It can facilitate training, maintenance, assembly, quality control, remote collaboration and other tasks. AR has the potential to revolutionize the way information is accessed, used and exchanged, extending user's perception and improving their performance. This work proposes a Pervasive AR tool, created with partners from the industry sector, to support the training of logistics operators on industrial shop floors. A Human-Centered Design (HCD) methodology was used to identify operators difficulties, challenges, and define requirements. After initial meetings with stakeholders, two distinct methods were considered to configure and visualize AR content on the shop floor: Head-Mounted Display (HMD) and Handheld Device (HHD). A first (preliminary) user study with 26 participants was conducted to collect qualitative data regarding the use of AR in logistics, from individuals with different levels of expertise. The feedback obtained was used to improve the proposed AR application. A second user study was realized, in which 10 participants used different conditions to fulfill distinct logistics tasks: C1-paper; C2-HMD; C3-HHD. Results emphasize the potential of Pervasive AR in the operators' workspace, in particular for training of operators not familiar with the tasks. Condition C2 was preferred by all participants and considered more useful and efficient in supporting the operators activities on the shop floor.
... Thanks to the advances in the machine learning domain which helped to overcome the challenge of recognition accuracy, more recent approaches to voice-based robot control are focused on how to integrate wearable devices capable of speech recognition (e.g. Android smart watches (Gkournelos et al. 2018)) and how to interface voice-based control with existing robot application development environments (e.g. ABB Robot Studio (Pires and Azar 2018; Kumar et al. 2016)). ...
This paper introduces a novel voice-based programming approach and software framework for collaborative robots (cobots) based on the Web Speech API, which is now supported by most modern browsers. The framework targets human programmable interfaces and human-machine interfaces, which can be used by people with little or no programming experience. The framework follows a meta-programming approach by enabling users to program cobots by voice in addition to using a mouse, tablet, or keyboard. Upon a voice instruction, the framework automates the manual tasks required to manipulate the vendor-provided interfaces. The main advantages of this approach are simplified, guided programming, which only requires the knowledge of 5–10 voice instructions; increased programming speed compared to the manual approach; and the possibility of sharing programs as videos. The approach is generalized to other kinds of robots and robot programming tools using so-called meta-controllers, which leverage the power of graphical user interface automation tools and techniques.
... Verma et al. (2013) employed a smartphone microphone to record the sound generated by machines, which makes automatic fault diagnosis possible. Moreover, audio recognition can be applied to audio commands to control robot (Gkournelos et al., 2018) or vehicle (Correa et al., 2010;Saod et al., 2016) movements. ...
With the increasing market penetration of smart devices (smartphones, smartwatches, and tablets), various mobile applications (apps) have been developed to fulfill tasks in daily life. Recently, efforts have been made to develop apps to support human operators in industrial work. When apps installed on commercial devices are utilized, tasks that were formerly done purely manually or with the help of investment-intensive specific devices can be performed more efficiently and/or at a lower cost and with reduced errors. Despite their advantages, smart devices have limitations because embedded sensors (e.g., accelerometers) and components (e.g., cameras) are usually designed for nonindustrial use. Hence, validation experiments and case studies for industrial applications are needed to ensure the reliability of app usage. In this study, a systematic literature review was employed to identify the state of knowledge about the use of mobile apps in production and logistics management. The results show how apps can support human centricity based on the enabling technologies and components of smart devices. An outlook for future research and applications is provided, including the need for proper validation studies to ensure the diversity and reliability of apps and more research on psychosocial aspects of human-technology interaction.
... pipe maintenance, and exoskeletons: though all are discussed by Michalos, the former do not seem to participate in production processes, and the latter are not easy to conceptualise as collaborators, but only as wearable tools.4 See alsoMichalos et al. (2016),Makris et al. (2016),Gkournelos et al. (2018).Content courtesy of Springer Nature, terms of use apply. Rights reserved. ...
Automation does not always replace human labour altogether: there is an intermediate stage of human co-existence with machines, including robots, in a production process. Cobots are robots designed to participate at close quarters with humans in such a process. I shall discuss the possible role of cobots in facilitating the eventual total elimination of human operators from production in which co-bots are initially involved. This issue is complicated by another: cobots are often introduced to workplaces with the message (from managers) that they will not replace human operators but will rather assist human operators and make their jobs more interesting and responsible. If, in the process of learning to assist human operators, robots acquire the skills of human operators, then the promise of avoiding replacement can turn out to be false, and if a human operator loses his job, he has been harmed twice over: once by unemployment and once by deception. I shall suggest that this moral risk attends some cobots more than others.
