FIGURE 1 - uploaded by Felix Grumbach
Content may be subject to copyright.
Source publication
The following interdisciplinary article presents a memetic algorithm with applying deep reinforcement learning (DRL) for solving practically oriented dual resource constrained flexible job shop scheduling problems (DRC-FJSSP). From research projects in industry, we recognize the need to consider flexible machines, flexible human workers, worker cap...
Contexts in source publication
Context 1
... framework provides a memetic algorithm integrating DRL and a DES to evaluate the schedules under the consideration of complex constraints. Figure 1 displays the main building blocks and their relationship within the scheduling dataflow. The memetic algorithm is based on a GA to control the overall optimization process and to provide a breadth-first search for assignment and sequencing decisions. ...
Context 2
... framework provides a memetic algorithm integrating DRL and a DES to evaluate the schedules under the consideration of complex constraints. Figure 1 displays the main building blocks and their relationship within the scheduling dataflow. The memetic algorithm is based on a GA to control the overall optimization process and to provide a breadth-first search for assignment and sequencing decisions. ...
Citations
... In recent years, RL has attracted more and more attention as an alternative approach for successfully solving scheduling problems [2], [3]. However, many approaches do not move beyond the academic context due to their abstraction from real-world requirements, as [4] and [5] point out. ...
Solving production scheduling problems is a difficult and indispensable task for manufacturers with a push-oriented planning approach. In this study, we tackle a novel production scheduling problem from a household appliance production at the company Miele & Cie. KG, namely a two-stage permutation flow shop scheduling problem (PFSSP) with a finite buffer and sequence-dependent setup efforts. The objective is to minimize idle times and setup efforts in lexicographic order. In extensive and realistic data, the identification of exact solutions is not possible due to the combinatorial complexity. Therefore, we developed a reinforcement learning (RL) approach based on the Proximal Policy Optimization (PPO) algorithm that integrates domain knowledge through reward shaping, action masking, and curriculum learning to solve this PFSSP. Benchmarking of our approach with a state-of-the-art genetic algorithm (GA) showed significant superiority. Our work thus provides a successful example of the applicability of RL in real-world production planning, demonstrating not only its practical utility but also showing the technical and methodological integration of the agent with a discrete event simulation (DES). We also conducted experiments to investigate the impact of individual algorithmic elements and a hyperparameter of the reward function on the overall solution. INDEX TERMS Reinforcement learning, production scheduling, permutation flow shop scheduling problem.
Current solutions for holistic and real-time planning of dynamic manufacturing processes are reaching their limits. This is particularly applicable to complex sociotechnical production environments with flexible material flows as well as undetermined events and fluctuations. Methods of optimization under uncertainty are very computationally intensive and crucial interactions with the real world are insufficiently considered. This lack of field synchronicity reduces the quality of production schedules, leads to manual efforts firefighting), and has a negative impact on the logistical performance. The present work is based on four journal articles that demonstrate novel methods and models for improving field-synchronous scheduling. Through the combination of instruments from operations research and machine learning, generic and predictive algorithms are developed to improve the efficiency and effectiveness of planning procedures. The findings suggest that regression models can replace computation-heavy stochastic simulations in obtaining robustness metrics. Additionally, using reinforcement learning, uncertainty-robust and realistic production schedules for human-centered manufacturing can be generated in a short time. For this purpose, discrete simulation models are used, which are data-driven initialized based on a general control logic. The algorithms can be integrated into a virtual factory, which serves as a digital representation of the real world and is the basis for smart and field-synchronous scheduling systems. In this context, a prototype distributed system for the planning of dynamic manufacturing processes can be presented, which is being tested by industry research partners and further developed in collaboration. Beyond the publications, further research needs can be derived. In order to ensure the transferability of the methods, they need to be evaluated in the context of additional and more comprehensive environments. From a scientific and practical perspective, it is a crucial challenge to develop holistic and proactive scheduling systems that orchestrate a comprehensive set of data-driven analysis and decision-making processes. In this regard, the presented methods and models need to be further developed and integrated into a generic overall concept. The work identifies four focus areas that future research should address in an interdisciplinary manner: (1) Generic simulation models, (2) Human-centered optimization, (3) Field-synchronous scheduling, and (4) System development and rollout.
