Fig 1 - available via license: Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International
Content may be subject to copyright.
Source publication
The increasing complexity of products and machines as well as short production cycles with small lot sizes present great challenges to production industry. Both, the programming of industrial robots in online mode using hand-held control devices or in offline mode using text-based programming requires specific knowledge of robotics and manufacturer...
Contexts in source publication
Context 1
... achieve the desired intuitive programming of robot programs we extended an existing cloud based robot control system. Figure 1 shows the new architecture of the robot control. ...
Context 2
... through it. OPC UA gives a great advantage in flexibility, scalability and migration. Any user client can access the robotic services that are published through the OPC UA Server and implement e.g. own data visualization layers. In this particular approach, the OPC UA Server itself is considered as an integral part of the cloud solution (see Fig. ...
Similar publications
The paper presents a methodology of creating a Hierarchical Petri Net modelling the activities of a multi-agent robotic system. The methodology follows the separation of concerns approach to the design of robot control software, thus five layers resulted, representing: the system composed of agents, agents’ subsystems, behaviours of subsystems, beh...
Citations
... Not only the robot programming but the verification of the planned trajectories is also essential. Wassermann et al. [133] developed a system for industrial robots' workspace simulation and program verification based on augmented reality. Specifically, the environment was constructed by 2D images and 3D point clouds in the AR system, and the user performed task-oriented level programming. ...
Robots are expanding from industrial applications to daily life, in areas such as medical robotics, rehabilitative robotics, social robotics, and mobile/aerial robotics systems. In recent years, augmented reality (AR) has been integrated into many robotic applications, including medical, industrial, human–robot interactions, and collaboration scenarios. In this work, AR for both medical and industrial robot applications is reviewed and summarized. For medical robot applications, we investigated the integration of AR in (1) preoperative and surgical task planning; (2) image-guided robotic surgery; (3) surgical training and simulation; and (4) telesurgery. AR for industrial scenarios is reviewed in (1) human–robot interactions and collaborations; (2) path planning and task allocation; (3) training and simulation; and (4) teleoperation control/assistance. In addition, the limitations and challenges are discussed. Overall, this article serves as a valuable resource for working in the field of AR and robotic research, offering insights into the recent state of the art and prospects for improvement.
... The interaction between mixed reality systems and robots is usually developed through Robot Operating System (ROS) due to Its versatility for node and topic management systems, allowing the combination of commercial software such as Unity or Unreal Engine with ROS development software (Wassermann et al., 2018). ...
In recent years, exploration tasks in disaster environments, victim localization and primary assistance have been the main focuses of Search and Rescue (SAR) Robotics. Developing new technologies in Mixed Reality (M-R) and legged robotics has taken a big step in developing robust field applications in the Robotics field. This article presents MR-RAS (Mixed-Reality for Robotic Assistance), which aims to assist rescuers and protect their integrity when exploring post-disaster areas (against collapse, electrical, and toxic risks) by facilitating the robot’s gesture guidance and allowing them to manage interest visual information of the environment. Thus, ARTU-R (A1 Rescue Tasks UPM Robot) quadruped robot has been equipped with a sensory system (lidar, thermal and RGB-D cameras) to validate this proof of concept. On the other hand, Human-Robot interaction is executed by using the Hololens glasses. This work’s main contribution is the implementation and evaluation of a Mixed-Reality system based on a ROS-Unity solution, capable of managing at a high level the guidance of a complex legged robot through different interest zones (defined by a Neural Network and a vision system) of a post-disaster environment. The robot’s main tasks at each point visited involve detecting victims through thermal, RGB imaging and neural networks and assisting victims with medical equipment. Tests have been carried out in scenarios that recreate the conditions of post-disaster environments (debris, simulation of victims, etc.). An average efficiency improvement of 48% has been obtained when using the immersive interface and a time optimization of 21.4% compared to conventional interfaces. The proposed method has proven to improve rescuers’ immersive experience of controlling a complex robotic system.
... Focus of attention information for each user can be visualized by the remote assistant during the pick-and-move task. Object collision avoidance is discussed by Wassermann et al. [27]. The monitor-based augmented reality application allows for the visualization of bounding boxes around real objects involved in a pick-and-place task. ...
The adoption of Digital Twin (DT) solutions for industrial purposes is increasing among small- and medium-sized enterprises and is already being integrated into many large-scale companies. As there is an increasing need for faster production and shortening of the learning curve for new emerging technologies, Virtual Reality (VR) interfaces for enterprise manufacturing DTs seem to be a good solution. Furthermore, with the emergence of Industry 5.0 (I5.0) paradigm, human operators will be increasingly integrated in the systems interfaces though advanced interactions, pervasive sensors, real time tracking and data acquisition. This scenario is especially relevant in collaborative automated systems where the introduction of immersive VR interfaces based on production cell DTs might provide a solution for the integration of the human factors in the modern industrial scenarios. This study presents experimental results of the comparison between users controlling a physical industrial robot system via a traditional teach pendant and a DT leveraging a VR user interface. The study group involves forty subjects including experts in robotics and VR as well as non-experts. An analysis of the data gathered in both the real and the virtual use case scenario is provided. The collected information includes time for performing a task with an industrial robot, stress level evaluation, physical and mental effort, and the human subjects’ perceptions of the physical and simulated robots. Additionally, operator gazes were tracked in the VR environment. In this study, VR interfaces in the DT representation are exploited to gather user centered metrics and validate efficiency and safety standards for modern collaborative industrial systems in I5.0. The goal is to evaluate how the operators perceive and respond to the virtual robot and user interface while interacting with them and detect if any degradation of user experience and task efficiency exists compared to the real robot interfaces. Results demonstrate that the use of DT VR interfaces is comparable to traditional tech pendants for the given task and might be a valuable substitute of physical interfaces. Despite improving the overall task performance and considering the higher stress levels detected while using the DT VR interface, further studies are necessary to provide a clearer validation of both interfaces and user impact assessment methods.
... Therefore, trained personnel are needed to program for the specific system. For this reason, AR robot programming can provide a more intuitive experience, making it easier to get started with robot teaching (Wassermann et al. 2018). Evlampev and Ostanin have developed as a basis a concept of path finding from a starting point to an end point with obstacle avoidance (Evlampev et al. 2019). ...
Programming of production systems depends on highly specialized personnel and proprietary development environments. This can hinder the evolution towards flexible production. In this paper we present a concept for an intuitive way of human-machine interaction by utilizing the augmented reality technology. This involves a method of programming a machine using an augmented reality headset. All desired machine operations can be predefined via a virtual user interface and by arbitrary placement of virtual working points. We further show how the referencing of the coordinate systems of the augmented reality environment and the robot can be accomplished. We prototypically apply the concept to a robotic arm with 6 degrees of freedom and employ a Microsoft HoloLens 2 optical head-mounted display to interact with robotic arm in the virtual world.
... For medium scale industries, the software used for teaching robots needs to be simple and non-time consuming as they do not have the resources to hire specialised personnel. To ease this programming of Industrial robots a research was carried out to develop a task-oriented control system that is cloud based with perception of the environment and has plausibility check functions [97]. ...
Since the first Industrial Revolution the trends in manufacturing have evolved a lot, from mechanical production to the era of smart manufacturing via technologies like Cyber Physical Systems, Internet of Things, Big Data, Cyber Security, Cloud Computing, Additive Manufacturing, Advanced robots, Modelling and Simulation and Augmented Virtual Reality. These technologies are enabling Interoperability and integration of various processes and departments in an organization because of the attribute of real-time inter-connectivity. Due to high inter-connectivity advantages like shorter development time, mass customization and modularity, configurability can be brought into existence. This will not only change the dynamics of the production lines but also add to the profit ratio of an organization by controlling over inventory via virtualization and predictive manufacturing. Due to such attributes of the Industry 4.0 paradigm, understanding them in depth is necessary. Hence, this paper aims to review many such characteristics, enablers, and main drivers of the Industry 4.0 paradigm and ultimately provides insight on the future scopes of each of the main pillars of Industry 4.0.
... AR applications (24% of them) can also be used for human robot collaboration. These applications are not only implemented in cases where humans are serving industrial robots but also for programming or trajectory defining as well; this results in the avoidance of collisions [76][77][78][79][80][81][82][83][84][85][86][87][88][89][90][91]. One of the significant advantages in this area is related to the safety of employees and equipment because these applications can show operators the location of unsafe areas [92]. ...
New technologies, such as cloud computing, the Internet of Things, wireless communications, etc., have already become part of our daily lives. This paper provides an insight into one of the new technologies, i.e., augmented reality (AR), as part of the manufacturing paradigm Industry 4.0 (I4.0). The aim of this paper is to contribute to the current state in the field of AR by assessing the main areas of the application of AR, the used devices and the tracking methods in support of the digitalization of the industry. Searches via Science Direct, Google Scholar and the Internet in general have resulted in the collection of a large number of papers. The examined works are classified according to several criteria and the most important data resulting from them are presented here. A comprehensive analysis of the literature has indicated the main areas of application of AR in I4.0 and, among these, those that stand out are maintenance, assembly and human robot collaboration. Finally, a roadmap for the application of AR in companies is proposed and the most promising future areas of research are listed.
... In particular, they state that MR can illuminate the gap between simulation and implementation, robots can be expanded with virtual capabilities or can interact with humans in shared space. Wassermann, Vick, and Krüger (2018) explored the industrial programming system with workspace analysis based on a depth camera data and point cloud processing. This approach allowed the authors to validate and verify the robot program before execution. ...
The paper presents a framework for interactive control of multi robots using mixed reality interfaces. The proposed approach can operate with both single and multi-robot systems, including interaction with industrial manipulators, mobile robots, and unmanned aerial vehicles (UAV). The human–robot interface allows both interactive and physical interaction with the robot. Particular attention is paid to a visualisation of the robots' intent during control. The presented framework has Robotic Operation System (ROS) as a core that can be connected to any robot and mixed reality interface. This allows us to make framework scalable and connect several different mixed reality devices and multiple heterogeneous robots. In the implementation part we connected our framework with Microsoft HoloLens, collaborative robot KUKA IIWA, mobile platform Plato, UAV designed on the basis of dji f450. The proposed framework was tested experimentally with real experimental setups mentioned above. The results showed the capabilities of the mixed reality system for interactive control of different robots' type.
... reasons. The task of finding an intuitive user-friendly interface for Human-Robot Interaction (HRI) that is able to suit different working environments for non-skilled operators is not simple [4][5][6]. ...
... Moreover, operators found difficulty in defining obstacles of irregular shapes compared with regular ones. 6. Conclusions and Future Work. ...
Robots are often used to replace humans in hazardous, uncomfortable and tedious works. However, due to the low repeatability and high versatility in High-Mix Low-Volume (HMLV) manufacturing, the cost of robots' programming is still significant. Programming methods based on getting in direct contact with robots, although intuitive, are not always allowed for safety concerns. Moreover, unstructured environments pose an additional challenge where a collision-free path including, not only the end-effector but also the entire robot links, is required. In this research, an intuitive semi-automatic offline robot programming method based on Augmented Reality (AR) and Stereo Vision (SV) system is proposed. The method has a simple user interface where the operator can intuitively program a remotely located robot in unstructured environments. The human-robot interface system is formulated by information fusion of stereo vision data, proposed AR algorithm, and human intuitiveness. Experiments are conducted to evaluate the proposed system. Candidate operators are given a task of programming a remotely located 6-DOF robot to accomplish a free-of-collision pick-and-place operation in unstructured environment. The results showed effectiveness in robotic task implementation.
... Finding a simple user interface for quicker, intuitive, and safer robot programming has been a challenging issue. Thus, there is an urgent demand to have a quicker, intuitive, and safer human-robot interface that uses the operator's common knowledge and addresses the drawbacks regarding the traditional programming methods to assist complex robots to solve unpredictable problems [5][6][7][8][9].Human-robot interface (HRI) can be defined as the interpretation and description of human intentions of the predicted task into a number of robot motions while taking the capabilities of the robot into consideration and the task requirements [10,11]. ...
... ;Wassermann et al. (2018), where the authors focused on the interaction with the industrial robots. Programming organizes by arranging virtual control points in the robot workspace and displaying the trajectory through Microsoft HoloLens. ...
The paper presents a mixed reality-based approach for interactive control of robotic manipulators and mobile platforms. In particular, we designed an interactive and understandable interface for human-robot interaction. The interface provides tools for robot path programming and visualizes it. Path visualization helps workers understand robot behavior, it is important for safety human-robot interaction. The paper presents an architecture of that system and the implementation for an industrial robot KUKA iiwa and mobile robot platform Plato. The main issue of a multi-platform system is related to the synchronization of coordinate frames for all elements. To deal with this problem we realized 3 setting options: manually, by a camera with markers, point clouds processing. We implemented our interface on Microsoft HoloLens and evaluated it on users.
