Figure - available via license: Creative Commons Attribution 4.0 International
Content may be subject to copyright.
Source publication
This paper presents a multi-agent Double Deep Q Network (DDQN) based on deep reinforcement learning for solving the transmission network expansion planning (TNEP) of a high-penetration renewable energy source (RES) system considering uncertainty. First, a K-means algorithm that enhances the extraction quality of variable wind and load power uncerta...
Contexts in source publication
Context 1
... system configuration is i9-9900K with 3.6 GHz, a memory of 32 GB, and graphics card of 2080Ti. DQN and PSO are used for contrast, and the parameters of multi-agent DDQN and DQN are listed in Table 4. The TNEP schemes of four methods are shown in Tables 5-8, and the transmission network structure is in Figure 8. Figure 7b, the cumulative energy of load 1, 3, and 4 modes are different, and the fluctuation characteristics of mode 2 are more unique. ...Context 2
... system configuration is i9-9900K with 3.6 GHz, a memory of 32 GB, and graphics card of 2080Ti. DQN and PSO are used for contrast, and the parameters of multi-agent DDQN and DQN are listed in Table 4. The TNEP schemes of four methods are shown in Tables 5-8, and the transmission network structure is in Figure 8. Tables 5-8 show that all four methods optimize the stability and economy of the transmission network by constructing lines. ...Citations
... A multi-agent double deep Q network (DDQN) based on deep learning for solving the TNEP problem with high penetration of renewable energy under uncertainty is proposed in [14]. An algorithm termed as "K-means" is used to enhance the extraction quality of the variable of load power and wind uncertain characteristics. ...
The electrical energy demand increase does evolve rapidly due to several socioeconomic factors such as industrialisation, population growth, urbanisation and, of course, the evolution of modern technologies in this 4th industrial revolution era. Such a rapid increase in energy demand introduces a huge challenge into the power system, which has paved way for network operators to seek alternative energy resources other than the conventional fossil fuel system. Hence, the penetration of renewable energy into the electricity supply mix has evolved rapidly in the past three decades. However, the grid system has to be well planned ahead to accommodate such an increase in energy demand in the long run. Transmission Network Expansion Planning (TNEP) is a well ordered and profitable expansion of power facilities that meets the expected electric energy demand with an allowable degree of reliability. This paper proposes a DC TNEP model that minimises the capital costs of additional transmission lines, network reinforcements, generator operation costs and the costs of renewable energy penetration, while satisfying the increase in demand. The problem is formulated as a mixed integer linear programming (MILP) problem. The developed model was tested in several IEEE test systems in multi-period scenarios. We also carried out a detailed derivation of the new non-negative variables in terms of the power flow magnitudes, the bus voltage phase angles and the lines’ phase angles for proper mixed integer variable decomposition techniques. Moreover, we intend to provide additional recommendations in terms of in which particular year (within a 20 year planning period) can the network operators install new line(s), new corridor(s) and/or additional generation capacity to the respective existing power networks. This is achieved by running incremental period simulations from the base year through the planning horizon. The results show the efficacy of the developed model in solving the TNEP problem with a reduced and acceptable computation time, even for large power grid system.
... In order to reduce the number of candidate scenarios that satisfy the probability threshold but have little difference in data distribution within the scenario matrix, so as to improve the representativeness of typical scenarios and reduce the complexity of association rules. So, in this paper, the Wasserstein distance metric is utilized to measure the degree of similarity between sets of candidate-typical scenes [26][27][28]. Finally, discard some redundant scenarios that are more similar even if they satisfy the probability threshold requirement. ...
Regarding the generation and integration of typical scenes of PV and loads in urban photovoltaic distribution networks, as well as the insufficient consideration of the spatiotemporal correlation between PV and loads, this paper proposes a typical scene extraction method based on local linear embedding, kernel density estimation, and a joint PV–load typical scene extraction method based on the FP-growth algorithm. Firstly, the daily operation matrices of PV and load are constructed by using the historical operation data of PV and load. Then, the typical scenes are extracted by the dimensionality reduction of local linear embedding and the kernel density estimation method. Finally, the strong association rules of PV–meteorological conditions and load–meteorological conditions are mined based on the FP-growth algorithm, respectively. The association of PV–load typical daily operation scenarios is completed using meteorological conditions as a link. This experiment involved one year of operation data of a distribution network containing PV in Qingyuan, Guangdong Province. The typical scene extraction joint method, Latin hypercube sampling method, and k-means clustering-based scene generation method proposed in this paper are used for comparison, respectively. The results show that compared to the other two scenario generation methods, the error between the typical scenario obtained by this method and the actual operating scenario of the distribution network is smaller. The extracted typical PV and load scenarios can better fit the actual PV and load operation scenarios, which have more reference value for the operation planning of actual distribution networks containing PV.
... A multi-agent Double Deep Q Network (DDQN) based on deep learning for solving the TNEP problem with high penetration of renewable energy under uncertainty is proposed in [16]. An algorithm termed as "K-means" is used to enhance the extraction quality of variable of load power and wind uncertain characteristics. ...
Electrical energy demand increase does evolve rapidly due to several socioeconomic factors such as industrialization, population growth, urbanization and of course the evolution of modern technologies in this 4th industrial revolution era. Such rapid increase in energy demand introduces a huge challenge in power system. Such has paved way for network operators to seek for alternative energy resources other than the conventional fossil fuel system. Hence, the penetration of renewable energy into the electricity supply mix has evolved rapidly in the past three decades. However, the grid system has to be well planned ahead to accommodate such increase in energy demand in the long run. Transmission Network Expansion Planning (TNEP) is a well ordered and profitable expansion of power facilities that meets the expected electric energy demand with an allowable degree of reliability. This paper proposes a TNEP model that minimises the network reinforcements, operational costs and costs of renewable energy penetrations, while satisfying the increase in demand. The problem is formulated as a mixed integer linear programming (MILP) problem. The developed model has been tested in several IEEE test systems in multi-period scenarios. The paper also carried out a detailed derivation of the new non-negative variables in terms of the power flow magnitudes, the bus voltage phase angles and the lines’ phase angles for proper mixed integer variables’ decomposition techniques. Moreover, this paper tends to provide additional recommendation in terms of which particular year (within 20 years of planning period) can the network operators install new line(s), new corridor(s) and/or additional generation capacity to the respective existing power networks. Such is achieved by running incremental periods simulations from base year through the planning horizon. The results show the efficacy of the developed model in solving the TNEP problem with a reduced and acceptable computation time even for large power grid system.
... The electric power system is not deterministic. Hence, the power system's uncertainties are yet another feature that must be considered in the TEP model [7]. Uncertain events, such as stochastic behavior of electrical loads and contingencies related to transmission lines and generation units, always occur randomly in an unexpected way [8,9]. ...
... Although the deterministic approach simplifies the problem, it fails to represent the random nature of the real power system and may produce unrealistic plans. In this regard, current research has focused on tackling this issue [7][8][9]. ...
... Equations (5) and (6), respectively, limit the active power balance at bus i and power flows through transmission lines. While the voltage angles at bus i, the capacity of clean and thermal stations, number of circuits in each corridor are restricted by (7)(8)(9)(10). The reliability constraints are represented in (11) to (13). ...
This chapter introduces a stochastic multi-period transmission expansion planning (SMTEP) model that considers the power system’s uncertainties and reliability constraints. Renewable generation sources (RGSs) are widely used in power systems. RGSs have a stochastic behavior that menaces the power system’s reliability and may result in partial or complete blackouts. The inclusion of N-1 security in SMTEP is also essential to ensure the continuity of electricity supply to loads under the worst conditions. The problem is formulated as a mixed-integer non-linear optimization problem. The whale optimization algorithm (WOA) is applied to solve the SMTEP problem. A reduction technique and an acceleration scheme are incorporated with the WOA to accelerate the convergence to the optimal solution and decrease computation time. The results of testing the WOA on a benchmark system and a realistic network show its efficiency and superiority, compared to other well-established algorithms, in terms of convergence time and quality of solutions. Further, case studies demonstrate the effectiveness of the suggested model in improving the power system’s reliability.
... Deep-learning-based models are also used in [17,18] to address expansion planning problems. In [17], a bi-directional long short-term memory network is applied to forecast the annual peak load of the power system as a means to address the generation expansion planning problem. ...
... The power plants lifetime and the carbon cost are considered as relevant factors in the designed problem formulation. In turn, [18] focuses on the problem of transmission network expansion planning, by analyzing load and wind power uncertainties. This goal is achieved through the application of a deep reinforcement learning model, namely a multi-agent double deep Q network. ...
Worldwide power and energy systems are changing significantly [...]
... This parameter is about detecting and governing the power demand level in energy markets. It captures various dimensions of "Energy Markets & Management" including using IML method for the management of decentralized optimal power flow [60], designing an interpretable DRL approach for transmission network expansion in wind power [61], power distribution systems' reliability, interpretability, and security [62], optimal multi-agent energy management for interconnected energy systems in the context of a co-trading market to promote fair commerce and to maintain the privacy of entities [63], management of energy pipeline infrastructure [64], and applying IML and collaborative game theory for market regression analysis and its use in energy forecasting [65]. ...
Traditional electrical power grids have long suffered from operational unreliability, instability, inflexibility, and inefficiency. Smart grids (or smart energy systems) continue to transform the energy sector with emerging technologies, renewable energy sources, and other trends. Artificial intelligence (AI) is being applied to smart energy systems to process massive and complex data in this sector and make smart and timely decisions. However, the lack of explainability and governability of AI is a major concern for stakeholders hindering a fast uptake of AI in the energy sector. This paper provides a review of AI explainability and governance in smart energy systems. We collect 3,568 relevant papers from the Scopus database, automatically discover 15 parameters or themes for AI governance in energy and elaborate the research landscape by reviewing over 100 papers and providing temporal progressions of the research. The methodology for discovering parameters or themes is based on "deep journalism", our data-driven deep learning-based big data analytics approach to automatically discover and analyse cross-sectional multi-perspective information to enable better decision-making and develop better instruments for governance. The findings show that research on AI explainability in energy systems is segmented and narrowly focussed on a few AI traits and energy system problems. This paper deepens our knowledge of AI governance in energy and is expected to help governments, industry, academics, energy prosumers, and other stakeholders to understand the landscape of AI in the energy sector, leading to better design, operations, utilisation, and risk management of energy systems.
... The essence of optimizing electricity and natural gas is to improve the utilization rate of renewable energy or new energy. Therefore, the development of a safe and stable multienergy complementary system with high renewable energy penetration is an important part of energy system reform, based on considering the policy and demand of renewable energy applications in the framework of the current social goal of "carbon neutrality" [26]. To fully tap into the advantages of MCSs in terms of high environmental protection and energy efficiency, renewable energy penetration and carbon transaction cost are identified as the key factors for system operational optimization. ...
Multi-energy complementary systems (MCSs) are complex multilevel systems. In the process of system planning, many aspects—such as power planning, investment cost, and environmental impact—should be considered. However, different decision makers tend to have different levels of control objectives, and the multilevel problems of the system need to be solved effectively with comprehensive judgment. Therefore, based on the terminal MCS energy structure model, the optimization method of MCS planning and operation coordination, considering the influence of planning and operation in the system’s life cycle, is studied in this paper. Consequently, the research on the collaborative optimization strategy of MCS construction and operation was carried out based on the bi-level multi-objective optimization theory in this paper. Considering the mutual restraint and correlation between system construction and operation in practical engineering, a bi-level optimization model for collaborative optimization of MCS construction and operation was constructed. To solve the model effectively, the existing non-dominated sorting genetic algorithm III (NSGA-III) was improved by the authors on the basis of previous research, which could enhance the global search ability and convergence speed of the algorithm. To effectively improve and strengthen the reliability of energy supply, and increase the comprehensive energy utilization of the system, the effects of carbon transaction cost and renewable energy penetration were considered in the optimization process. Based on an engineering example, the bi-level model was solved and analyzed. It should be noted that the optimization results of the model were verified to be applicable and effective by comparison with the single-level multi-objective programming optimization. The findings of this paper could provide theoretical reference and practical guidance for the planning and operation of MCSs, making them significant for social application.
... Uncertain events, such as load demand variability and line contingencies, may occur. The installation of distributed generation (DG) units, such as renewable energy sources (RESs), poses new uncertainties [5,6]. These could dramatically affect the power system's operation and weaken the grid's reliability (for instance, a rolling blackout may occur, similar to the blackouts in California in 2020 [7]). ...
... The short-circuit current constraint was embedded in the model without using the FCLs introduced by Teimourzadeh et al. [16] and Wang et al. [17]. Figure 8 shows that the short-circuit currents at buses 5,6,7,8,31,36,41,42, and 53 exceeded 6.05 p.u. ...
Today, generation and transmission expansion planning (G&TEP) to meet potential load growth is restricted by reliability constraints and the presence of uncertainties. This study proposes the reliability constrained planning method for integrated renewable energy sources and transmission expansion considering fault current limiter (FCL) placement and sizing and N-1 security. Moreover, an approach for dealing with uncertain events is adopted. The proposed planning model translates into a mixed-integer non-linear programming model, which is complex and not easy to solve. The problem was formulated as a tri-level problem, and a hybridization framework between meta-heuristic and mathematical optimization algorithms was introduced to avoid linearization errors and simplify the solution. For this reason, three meta-heuristic techniques were tested. The proposed methodology was conducted on the Egyptian West Delta system. The numerical results demonstrated the efficiency of integrating G&TEP and FCL allocation issues in improving power system reliability. Furthermore, the effectiveness of the hybridization algorithm in solving the suggested problem was validated by comparison with other optimization algorithms.
The efficient planning of electric power systems is essential to meet both the current and future energy demands. In this context, reinforcement learning (RL) has emerged as a promising tool for control problems modeled as Markov decision processes (MDPs). Recently, its application has been extended to the planning and operation of power systems. This study provides a systematic review of advances in the application of RL and deep reinforcement learning (DRL) in this field. The problems are classified into two main categories: Operation planning including optimal power flow (OPF), economic dispatch (ED), and unit commitment (UC) and expansion planning, focusing on transmission network expansion planning (TNEP) and distribution network expansion planning (DNEP). The theoretical foundations of RL and DRL are explored, followed by a detailed analysis of their implementation in each planning area. This includes the identification of learning algorithms, function approximators, action policies, agent types, performance metrics, reward functions, and pertinent case studies. Our review reveals that RL and DRL algorithms outperform conventional methods, especially in terms of efficiency in computational time. These results highlight the transformative potential of RL and DRL in addressing complex challenges within power systems.
