Fig 2 - available via license: Creative Commons Attribution 4.0 International
Content may be subject to copyright.
Source publication
Increased market demand for composite products and shortage of expert laminators is compelling the composite industry to explore ways to acquire layup skills from experts and transfer them to novices and eventually to machines. There is a lack of holistic methods in literature for capturing composite layup skills especially involving complex moulds...
Contexts in source publication
Context 1
... of a task that belong to the skill, rule and knowledge based levels. The data flow within the process follows the standard informatics data flow, namely, data input, data processing, data storage and data output. The resulting digitisation process comprises 6 sequential steps, namely Capture, Segment, Model, Extract, Decode, and Reproduce (Fig. ...
Context 2
... belonging to the laminator's upper body are recorded in the capture step of the framework. From these joint coordinates, several motion parameters can be obtained using vector computing. Examples of four different motion Fig. 10. Layup strategy adopted by the expert laminator. mechanics computed using skeletal coordinate data is shown in Fig. 12. This data helps in visualising the laminator's body posture sand orientations, glance angles and the positions of his hands with respect to the ply and the mould while per- forming critical hand layup ...
Citations
... Machine learning (ML) has also been used for the optimization of fabric bending rigidity in spun-lace production lines [47]. While digitization of the motions involved in composite layup has been studied [43], there has been limited work on analyzing the intricate motions performed by the skilled operators [17,12]. Therefore, a more detailed analysis of hand-intensive processes with an emphasis on the ergonomic risks posed to the operators is essential to improve safety and health outcomes in the manufacturing workplace. ...
Hand-intensive manufacturing processes, such as composite layup and textile draping, require significant human dexterity to accommodate task complexity. These strenuous hand motions often lead to musculoskeletal disorders and rehabilitation surgeries. We develop a data-driven ergonomic risk assessment system with a special focus on hand and finger activity to better identify and address ergonomic issues related to hand-intensive manufacturing processes. The system comprises a multi-modal sensor testbed to collect and synchronize operator upper body pose, hand pose and applied forces; a Biometric Assessment of Complete Hand (BACH) formulation to measure high-fidelity hand and finger risks; and industry-standard risk scores associated with upper body posture, RULA, and hand activity, HAL. Our findings demonstrate that BACH captures injurious activity with a higher granularity in comparison to the existing metrics. Machine learning models are also used to automate RULA and HAL scoring, and generalize well to unseen participants. Our assessment system, therefore, provides ergonomic interpretability of the manufacturing processes studied, and could be used to mitigate risks through minor workplace optimization and posture corrections.
... Depth cameras, also known as RGB-D sensors, have been employed in many different applications to capture data about the worker and the components. For example, Prabhu et al. [6] used depth cameras to supervise operators during composite layup tasks. Chen et al. [7] utilised an RGB-D sensor and deep learning algorithms to monitor the manufacture of a small gear reducer. ...
Currently, systems installed on large-scale aerospace structures are manually equipped by trained operators. To improve current methods, an automated system that ensures quality control and process adherence could be used. This work presents a mobile robot capable of autonomously inspecting aircraft systems and providing feedback to workers. The mobile robot can follow operators and localise the position of the inspection using a thermal camera and 2D lidars. While moving, a depth camera collects 3D data about the system being installed. The in-process monitoring algorithm uses this information to check if the system has been correctly installed. Finally, based on these measurements, indications are shown on a screen to provide feedback to the workers. The performance of this solution has been validated in a laboratory environment, replicating a trailing edge equipping task. During testing, the tracking and localisation systems have proven to be reliable. The in-process monitoring system was also found to provide accurate feedback to the operators. Overall, the results show that the solution is promising for industrial applications.
... After ML is performed on real data and the trained model is obtained, we can use data synthesis to generate data and apply a trained model to make a prediction. It is worth noting that Data Twin could model not just physical products, but also human behaviors, which are difficult to construct by Digital Twin [45]. ...
... Although manual lay-up is still the main approach to performing the task, exploring ways to obtain layup skills from experts is vital due to the increased market demand and lack of experienced operators. Hidden Markov models (HMMs) and three-dimensional Convolutional Neural Networks (CNN) are ML approaches used to recognize human activities and digitize behavior [45]. Modelling human-workpiece interactions constitutes the prediction model of the lay-up DTS, and includes monitoring the work of operators and transferring knowledge to novices. ...
Because of the complex production processes and technology-intensive operations that take place in the aerospace and defense industry, introducing Industry 4.0 into the manufacturing processes of aircraft composite materials is inevitable. Digital Twin and Cyber-Physical Systems in Industry 4.0 are key techniques to develop digital manufacturing. Since it is very difficult to create high-fidelity virtual models, the development of digital manufacturing for aircraft manufacturers is challenging. In this study, we provide a view from a data simulation perspective and adopt machine learning approaches to simplify the high-fidelity virtual models in Digital Twin. The novel concept is called Data Twin, and the deployable service to support the simulation is known as the Data Twin Service (DTS). Relying on the DTS, we also propose a microservice software architecture, Cyber-Physical Factory (CPF), to simulate the shop floor environment. Additionally, there are two war rooms in the CPF that can be used to establish a collaborative platform: one is the Physical War Room, used to integrate real data, and the other is the Cyber War Room for handling simulation data and the results of the CPF.
... A similar technology called Kinect has been used by Prabhu et al. [11] to capture human motion information in conjunction with machine learning algorithms. This was used to analyse the digitisation of skilled workers in composite laying operations. ...
High value manufacturing systems still require ergonomically intensive manual activities. Examples include the
aerospace industry where the fitting of pipes and wiring into confined spaces in aircraft wings is still a manual
operation. In these environments, workers are subjected to ergonomically awkward forces and postures for long
periods of time. This leads to musculoskeletal injuries that severely limit the output of a shopfloor leading to loss
of productivity. The use of tools such as wearable sensors could provide a way to track the ergonomics of workers
in real time. However, an information processing architecture is required in order to ensure that data is processed
in real time and in a manner that meaningful action points are retrieved for use by workers.
In this work, based on the Adaptive Control of Thought—Rational (ACT-R) cognitive framework, we propose a
Cognitive Architecture for Wearable Sensors (CAWES); a wearable sensor system and cognitive architecture that
is capable of taking data streams from multiple wearable sensors on a worker’s body and fusing them to enable
digitisation, tracking and analysis of human ergonomics in real time on a shopfloor. Furthermore, through tactile
feedback, the architecture is able to inform workers in real time when ergonomics rules are broken. The architecture
is validated through the use of an aerospace case study undertaken in laboratory conditions. The
results from the validation are encouraging and in the future, further tests will be performed in an actual working
environment.
... Subsequently, knowledge management also became a major focus of these new industrial practices. Many approaches have been proposed: casebased reasoning [184], graph-based-reasoning [185], enterprise modelling [186], design project memory [187], etc. Digitalisation has been a great mean capturing high skilled tasks in industry [188]. ...
Over the past 70 years, product design has undergone many important changes due to the impact of contemporary digital technologies (i.e. digital design).
To support digital design and information flow throughout the product lifecycle, the digital-driven technologies currently in use rely on the evolution of CAD and PLM systems to address new design and manufacturing challenges generated by the new era of 4.0 digital transformation.
This paper will discuss the past and present coevolution and shortcomings within industrial organisations, the digital technologies employed in the product development cycle and will illustrate the current challenges and future prospects of the digital thread for design.
... Some related studies attempt to understand the manual layup process and propose solutions for various usage scenarios [55,56]. Prabhu et al. provide a generic approach to gain knowledge for the industrial tasks which require manual operations (Fig. 2 right adopted from [57]). The Kinect is used to capture human motions, recognize objects, and track depth information. ...
Industry 4.0 has led to paradigm shifts and changes for planning and developing manufacturing processes. To successfully embrace this revolution and confront the numerous challenges, manufacturing enterprises have to cope with the need for technological advancement and provide sustainable training and education for their workforce. This great transformation affects not only the integration of the digital and physical environments but, most importantly, also the relationships between humans and manufacturing sites. In this paper, we take into account the human-in-the-loop for digitalization challenges and present a 3I (Intellect, Interaction, and Interface) aspect for factories to increase the adoption of smart technologies toward the smart manufacturing vision. The Intellect aspect aims to add knowledge to the manufacturing equipment. The Interaction aspect targets at the collaboration between humans and manufacturing equipment. The Interface aspect explores appropriate means for humans to exploit the intelligence of technologies for the communication with the manufacturing equipment. In addition to the concept-related propositions, we also demonstrate a set of selected application examples to illustrate how the proposed aspects can drive new growth opportunities for enterprises.
... Therefore, we investigated how and if sensors can be or have been used to support modelling. Our intention is to sup- (2014), Sun, Byrns, Cheng, Zheng, and Basu (2017), Kowalewski et al. (2016), Khan (2015), Li, Lu, Chan, and Skitmore (2015), Prabhu, Elkington, Crowley, Tiwari, and Ward (2017), Daponte, De Vito, Riccio, and Sementa (2014), Jang, Kim, Woo, and Wakefield (2014), Ahmmad, Ming, Fai, and Narayanan (2014) (2017) Record video Camera Demonstration Sanfilippo (2017) port apprentices in modelling the expert performance by providing rich multimodal representations of the expert performance. In an analysis of 78 studies, the authors have identified 17 studies that have exclusively used experts as models for training (Limbu, Jarodzka, Specht, & Klemke, 2018). ...
... Therefore, we investigated how and if sensors can be or have been used to support modelling. Our intention is to sup- (2014), Sun, Byrns, Cheng, Zheng, and Basu (2017), Kowalewski et al. (2016), Khan (2015), Li, Lu, Chan, and Skitmore (2015), Prabhu, Elkington, Crowley, Tiwari, and Ward (2017), Daponte, De Vito, Riccio, and Sementa (2014), Jang, Kim, Woo, and Wakefield (2014), Ahmmad, Ming, Fai, and Narayanan (2014) (2017) Record video Camera Demonstration Sanfilippo (2017) port apprentices in modelling the expert performance by providing rich multimodal representations of the expert performance. In an analysis of 78 studies, the authors have identified 17 studies that have exclusively used experts as models for training (Limbu, Jarodzka, Specht, & Klemke, 2018). ...
The chapter highlights the role of sensors for supporting seamless learning experiences. In the first part, the relation between sensor tracking of learning activities and research around real-time feedback in educational situations is introduced. The authors present an overview of the kinds of sensor data that have been used for educational purposes in the literature. Secondly, the authors introduce the link between sensor data and educational interventions, and especially the role of building expert models from real-world expert tracking. The third part of the paper illustrates how educational AR applications have used sensor data for different forms of learning support. The authors present 15 design patterns that have been implemented in different educational AR applications that build on our analysis of sensor tracking. For future AR applications, the authors propose that the use of sensors for building expert performance models is essential for a variety of educational interventions.
... Fominykh et al. provide an overview on existing approaches for capturing performance in the real world [6], with consideration also given to the tacit knowledge and the its role in learning new tasks. Wireless inertial sensor, depth camera [10][11][12][13][14][15] Contextualisation, In situ real time feedback, haptic hints Record of Force applied ...
The WEKIT.one prototype is a platform for immersive procedural training with wearable sensors and Augmented Reality. Focusing on capture and re-enactment of human expertise, this work looks at the unique affordances of suitable hard- and software technologies. The practical challenges of interpreting expertise, using suitable sensors for its capture and specifying the means to describe and display to the novice are of central significance here. We link affordances with hardware devices, discussing their alternatives, including Microsoft Hololens, Thalmic Labs MYO, Alex Posture sensor, MyndPlay EEG headband, and a heart rate sensor. Following the selection of sensors, we describe integration and communication requirements for the prototype. We close with thoughts on the wider possibilities for implementation and next steps.
This paper addresses the recent growth in publications on the topic of digitalization by systematically reviewing the literature dealing with this major technological and organizational transformation in the manufacturing sector. Our aim is to enhance the understanding of this phenomenon and provide a critical account of the state of the art of the research on industrial digitalization. After summarizing the different conceptualizations of digitalization and the stages of its evolution, we identify four thematic areas (technologies, impacts, enabling factors, and barriers) and create a meaningful segmentation of the existing publications along these dimensions. We then develop a future research agenda on industrial digitalization at both a conceptual and an empirical level, encompassing the following themes: (i) the need for a more nuanced conceptualization; (ii) the potential evolution of one or more digital technologies toward the category of “general-purpose technology”; (iii) the search for new contingency approaches; (iv) the development of accurate models of inter-firm collaboration among digital manufacturers; and (v) the elaboration of suitable solutions able to guarantee the compatibility of industrial digitalization with ethical and sustainability principles.
