Figure - available from: Geophysical Research Letters
This content is subject to copyright. Terms and conditions apply.
Source publication
In this study, we propose an ensemble long short‐term memory (EnLSTM) network, which can be trained on a small data set and process sequential data. The EnLSTM is built by combining the ensemble neural network and the cascaded LSTM network to leverage their complementary strengths. Two perturbation methods are applied to resolve the issues of overc...
Citations
... The energy sector witnesses a proliferation of intelligent detection devices across various organizations, generating distributed data at the network edge. This data holds immense potential for training machine learning models to forecast future energy production [1,8,28,41] or consumption [6,12] and infer crucial unknown variables [5,4] collecting is economically burdensome. Accurate energy generation and consumption prediction is crucial for management, infrastructure planning and budgeting. ...
... Otherwise, fail to reject the null hypothesis. [1], [2], [3] [0], [1], [2], [3] #1 [5,6,10,19,20], [4,8,9,12,18], [7,15,16,21,23], [11,13,14,17,22] [4], [5], [6],… , [23] #2 [5,6,19,20], [4,8,9,12], [7,15,16,23], [11,13,17,22] [4], [5], [6], [7], [8], [9], [10], [11], [12], [13], [15], [16], [17], [21], [22], [23] #3 [5,19,20], [8,9,12], [15,16,23], [11,13,17] [5], [8], [9], [11], [12], [13], [15], [16], [17], [21], [22], [23] #4 [5,20], [9,12], [16,23], [11,17] [5], [9], [11], [12], [16], [17], [21], [23] #5 [5], [12], [16], [17] [5], [12], [16], [17] ...
... Otherwise, fail to reject the null hypothesis. [1], [2], [3] [0], [1], [2], [3] #1 [5,6,10,19,20], [4,8,9,12,18], [7,15,16,21,23], [11,13,14,17,22] [4], [5], [6],… , [23] #2 [5,6,19,20], [4,8,9,12], [7,15,16,23], [11,13,17,22] [4], [5], [6], [7], [8], [9], [10], [11], [12], [13], [15], [16], [17], [21], [22], [23] #3 [5,19,20], [8,9,12], [15,16,23], [11,13,17] [5], [8], [9], [11], [12], [13], [15], [16], [17], [21], [22], [23] #4 [5,20], [9,12], [16,23], [11,17] [5], [9], [11], [12], [16], [17], [21], [23] #5 [5], [12], [16], [17] [5], [12], [16], [17] ...
Machine learning models offer the capability to forecast future energy production or consumption and infer essential unknown variables from existing data. However, legal and policy constraints within specific energy sectors render the data sensitive, presenting technical hurdles in utilizing data from diverse sources. Therefore, we propose adopting a Swarm Learning (SL) scheme, which replaces the centralized server with a blockchain-based distributed network to address the security and privacy issues inherent in Federated Learning (FL)'s centralized architecture. Within this distributed Collaborative Learning framework, each participating organization governs nodes for inter-organizational communication. Devices from various organizations utilize smart contracts for parameter uploading and retrieval. Consensus mechanism ensures distributed consistency throughout the learning process, guarantees the transparent trustworthiness and immutability of parameters on-chain. The efficacy of the proposed framework is substantiated across three real-world energy series modeling scenarios with superior performance compared to Local Learning approaches, simultaneously emphasizing enhanced data security and privacy over Centralized Learning and FL method. Notably, as the number of data volume and the count of local epochs increases within a threshold, there is an improvement in model performance accompanied by a reduction in the variance of performance errors. Consequently, this leads to an increased stability and reliability in the outcomes produced by the model.
... In 2018, Akkurt et al. [33] built an automated process for rapid learning and enhanced analysis of rock physics to detect outliers in well logging data, evaluate inter-well similarity post-outlier removal using clustering information, and subsequently reconstruct well log curves. In 2020, Chen et al. [34] proposed an ensemble Long Short-Term Memory (EnLSTM) network capable of processing sequential data on a small dataset. By combining ensemble neural network and cascaded LSTM network, EnLSTM reduces costs and saves time. ...
The prediction of formation resistivity plays a crucial role in the evaluation of oil and gas reservoirs, identification and assessment of geothermal energy resources, groundwater detection and monitoring, and carbon capture and storage. However, traditional well logging techniques fail to measure accurate resistivity in cased boreholes, and the transient electromagnetic method for cased borehole resistivity logging encounters challenges of high-frequency disaster (the problem of inadequate learning by neural networks in high-frequency features) and noise interference, badly affecting accuracy. To address these challenges, frequency-aware framework and temporal anti-noise block are proposed to build frequency aware LSTM (FAL). The frequency-aware framework implements a dual-stream structure through wavelet transformation, allowing the neural network to simultaneously handle high-frequency and low-frequency flows of time-series data, thus avoiding high-frequency disaster. The temporal anti-noise block integrates multiple attention mechanisms and soft-threshold attention mechanisms, enabling the model to better distinguish noise from redundant features. Ablation experiments demonstrate that the frequency-aware framework and temporal anti-noise block contribute significantly to performance improvement. FAL achieves a 24.3% improvement in R2 over LSTM, reaching the highest value of 0.91 among all models. In robustness experiments, the impact of noise on FAL is approximately 1/8 of the baseline, confirming the noise resistance of FAL. The proposed FAL effectively reduces noise interference in predicting formation resistivity from cased transient electromagnetic well logging curves, better learns high-frequency features, and thereby enhances the prediction accuracy and noise resistance of the neural network model.
... Therefore, LSTM was used as the underlying neural network structure for transfer learning in this study. Chen et al [43] have introduced an innovative Integrated Long Short-Term Memory (EnLSTM) network that strategically merges Integrated Neural Networks (ENNs) and Cascaded Long Short-Term Memory (C-LSTM) networks. This integration effectively harnesses their complementary strengths, offering a compelling solution to confront the intricate challenges associated with small data problems, notably addressing issues like over-convergence and training failures. ...
Reservoir reconstruction, where parameter prediction plays a key role, constitutes an extremely important part in oil and gas reservoir exploration. With the mature development of artificial intelligence, parameter prediction methods are gradually shifting from previous petrophysical models to deep learning models, which bring about obvious improvements in terms of accuracy and efficiency. However, it is difficult to achieve large amount of data acquisition required for deep learning due to the cost of detection, technical difficulties, and the limitations of complex geological parameters. To address the data shortage problem, a transfer learning prediction model based on long short-term memory neural networks has been proposed, and the model structure has been determined by parameter search and optimization methods in this paper. The proposed approach transfers knowledge from historical data to enhance new well prediction by sharing some parameters in the neural network structure. Moreover, the practicality and effectiveness of this method was tested by comparison based on two block datasets. The results showed that this method could significantly improve the prediction accuracy of the reservoir parameters in the event of data shortage.
... Therefore, further research is necessary in the field. Drawing on previous method of predicting logging curve [23][24][25][26], the study develops three deep learning models for forecasting missing the curves of azimuthal EM LWD based on Long Short-Term Memory (LSTM) [27], Gated Recurrent Unit (GRU) [28] and UNET [29]. The result show that the deep learning method can accurately and efficiently predict the response of azimuthal EM LWD in both isotropic and anisotropic formations. ...
The data of azimuthal electromagnetic (EM) Logging-While-Drilling (LWD) tool is crucial for controlling and optimizing the trajectory of the wellbore, making it a key technology in geosteering. However, the measurement of the tool involves multiple frequencies, spaces, and sectors, leading to a significant volume of measured data that can’t be uploaded in real-time. Attempting to invert formation resistivity and boundaries based solely on the limited data that transmitted to the surface may not accurately reflect the true formation model. Therefore, this paper proposes a method for supplementing the measurement curves of the tool based on deep learning. The intelligent method can predict the missing logging information according to limited data and improve the utilization efficiency of logging data. Firstly, the database of azimuthal EM LWD is generated using various synthetic formation models and numerical forward modeling techniques, and the complete logging data is artificially separated into known logging data and missing logging data. Then, three deep learning models are established based on LSTM, GRU, and UNET networks respectively, and use the above sample database for training and testing them. The results demonstrate that missing curves of the tool’s measurement can be accurately and efficiently predicted using deep learning techniques. Finally, the original logging data and the complete logging data after supplementing are used for inverting the formation information. The result shows that the latter yields higher inversion accuracy. Moreover, the difference in inversion accuracy will grow as the complexity of the formation model increases after data supplementing. Therefore, the data supplement of azimuthal EM LWD by deep learning is very important for the accurate inversion of complex formation models.
... Besides the experimental studies above, more work is done by utilizing advanced machine-learning techniques, especially deep learning (DL). In addition, more practical issues in logs generation are considered to achieve better performances [18], [19], [20], [21], [22], [23]. Considering the reservoir sedimentary rhythm and logging response characteristics, Wang et al. [19] proposed a deep hybrid neural network (DHNN) for shear wave velocity, which combines traditional convolutional neural network (CNNs) and long short-term-memory neural networks (LSTMs). ...
... Coupled with CNNs establishing the complex mapping from logs to shear wave velocity, the deep convolutional GRU (DC-GRU) neural network performs better than the individual CNNs and LSTMs. Similarly, Chen and Zhang [22] proposed the ensemble LSTM (EnL-STM) for missing logs generation under the limit of data volume. These studies commonly establish a logs generation model by inputting the logs within a depth segment and outputting a segment as well, thereby naming depth segmentwise modeling strategy. ...
... The first group of methods for comparison is basic machine-learning models without domain adaptation. As done by the existing studies in logs generation, they have tested the effectiveness of RF, [16], support vector regression (SVR) [17], LSTM and its variants [20], [21], [22], simple multilayer perceptron (MLP) [15], but without considering the very practical non-iid issue. We use a fully convolutional neural network (FCN) with GMF as the backbone of LogRegX, so we add it to the first group as a comparison method. ...
Geophysical logging instruments continuously measure multiple geophysical properties of borehole rocks, thus providing a feasible way to fine borehole geology modelling. Since the missing problem of well logs is inevitable, it is essential to generate the missing logs by the available ones. Recently, a large body of interdisciplinary studies has demonstrated the effectiveness of applying machine learning to solve the missing logs generation problem, under which the training and testing datasets obey the independent and identical distribution (iid) assumption. This assumption, however, is not satisfied in the case of the cross-well missing logs generation task. A standard method to solve the non-iid issue is to map source and target data to a common feature space and then employ Mean Maximum Discrepancy (MMD) to measure domain differences. However, this method suffers from high computational complexity and poor feature explainability when dealing with logs generation tasks. To solve the above problems, we propose an explainable regression network for cross-well geophysical logs generation named LogRegX. LogRegX integrates single-well feature extraction, cross-well feature alignment, and missing logs prediction while maintaining the explainability of logging features. Specifically, LogRegX leverages the gating mechanism to fuse multi-scale logging features to capture the response characteristics of well logs. The learned source and target feature representations are subject to domain discrepancy constraints, measured by Random Fourier Feature transform induced MMD. Additionally, target-domain information retaining mechanism is introduced to maintain the structure of target data so that the transferred features are explainable. Experiments on real-world field data demonstrate the superiority and the explainability of LogRegX over the existing methods.
... It has been found that the accurate prediction of geomechanical parameters is a prerequisite for successful volumetric fracturing operation. The use of an ensemble long short-term memory (EnLSTM) network in well log generation [9] is one such method for predicting geomechanical parameters based on well logging interpretation. Researchers trained an interpretation model based on an available data set and combined an ensemble neural network and a cascaded LSTM network to improve the model accuracy in interpretation. ...
In order to accurately predict geomechanical parameters of oil-bearing reservoirs and influencing factors of volumetric fracturing, a new method of geomechanical parameter prediction combining seismic inversion, well logging interpretation and production data is proposed in this paper. Herein, we present a structure model, petrophysical model and geomechanical model. Moreover, a three-dimensional geomechanical model of a typical reservoir was established and corrected using history matching. On this basis, a typical well model was established, 11 influencing factors of volume fracturing including formation parameters and fracturing parameters were analyzed and their impact were ranked, and the oil recovery rate and the accumulated oil production before and after optimal fracturing were compared. The results show that with respect to formation parameters, reservoir thickness is the main influencing factor; interlayer thickness and stress difference are the secondary influencing factors; and formation permeability, Young’s modulus and Poisson’s ratio are the weak influencing factors. For a pilot well of a typical reservoir, the optimized fracture increased production by 7 tons/day relative to traditional fracturing. After one year of production, the method increased production by 4 tons/day relative to traditional fracturing, showing great potential in similar oil reservoirs.
... TgDLF used domain knowledge to obtain the trend of the load, and employed the model EnLSTM [19,20] to predict local fluctuations. TgDLF has achieved better results than pure knowledge-based models and pure datadriven models. ...
Electrical energy is essential in today's society. Accurate electrical load forecasting is beneficial for better scheduling of electricity generation and saving electrical energy. In this paper, we propose theory-guided deep-learning load forecasting 2.0 (TgDLF2.0) to solve this issue, which is an improved version of the theory-guided deep-learning framework for load forecasting via ensemble long short-term memory (TgDLF). TgDLF2.0 introduces the deep-learning model Transformer and transfer learning on the basis of dividing the electrical load into dimensionless trends and local fluctuations, which realizes the utilization of domain knowledge, captures the long-term dependency of the load series, and is more appropriate for realistic scenarios with scarce samples. Cross-validation experiments on different districts show that TgDLF2.0 is approximately 16% more accurate than TgDLF and saves more than half of the training time. TgDLF2.0 with 50% weather noise has the same accuracy as TgDLF without noise, which proves its robustness. We also preliminarily mine the interpretability of Transformer in TgDLF2.0, which may provide future potential for better theory guidance. Furthermore, experiments demonstrate that transfer learning can accelerate convergence of the model in half the number of training epochs and achieve better performance.
... For example, Zhu et al. and Angshuman Paul et al. [20,21] proposed models applicable to the prediction of image data. Chen et al. [22] proposed the use of an ensemble long short-term memory (EnLSTM) model for timeseries data. Although these models do lead to greater prediction accuracy for small sample sizes, they cannot fully overcome the inherent disadvantages of NN overfitting and weak generalizability [23], which means that such models do not train reliably with small datasets [20]. ...
Accurate prediction of the internal corrosion rates of oil and gas pipelines could be an effective way to prevent pipeline leaks. In this study, a proposed framework for predicting corrosion rates under a small sample of metal corrosion data in the laboratory was developed to provide a new perspective on how to solve the problem of pipeline corrosion under the condition of insufficient real samples. This approach employed the bagging algorithm to construct a strong learner by integrating several KNN learners. A total of 99 data were collected and split into training and test set with a 9:1 ratio. The training set was used to obtain the best hyperparameters by 10-fold cross-validation and grid search, and the test set was used to determine the performance of the model. The results showed that the Mean Absolute Error (MAE) of this framework is 28.06% of the traditional model and outperforms other ensemble methods. Therefore, the proposed framework is suitable for metal corrosion prediction under small sample conditions.
... However, big datasets are required for training such models. Deep learning algorithms trained using such dataset often perform poorly and tend to overfit (Feng et al., 2019;Chen & Zhang, 2020;Ozer et al., 2021). ...
Avocado (Persea americana) production is increasing in Kenya, with both small and largeholder farming for domestic and export markets. However, one of main challenges that limit production is infestation by insect pests, notably the oriential fruit fly Bactocera dorsalis and Ceratitis spp. fruit flies, which cause direct crop losses and are indirectly responsible for non-tariff trade barriers due to stringent export requirements. Data on weekly pest trap counts were collected between September 2017 and December 2020 within orchards in avocado plantations. Fuzzy neural network (FNN) were used to model the population dynamics of B. dorsalis and Ceratitis spp. Weekly pest counts, rainfall, average temperature, relative humidity and avocado plant physiological stages were used for predictive modeling in different orchards. The performance of the resulting models was evaluated using coefficient of determination (R²), mean absolute error (MAE), mean relative approximation error (MRAE) and root mean squared error (RMSE). FNN models achieved satisfactory results in predicting the dynamics of the pests in the orchards, with most of the models obtaining R² > 0.85. We demonstrated how FNN models can be used as predictive tools for managing and controlling fruit fly pest populations in these plantations, and how they may be suitable to predict fruit fly or other pests in similar cropping systems. Once the input variables are known, they can be loaded into the FNN models to predict field pest populations, and based on threshold values, allow for implementation of timely and adequate control measures such as the use of biopesticides.
... Long Short-Term Memory has been widely used on EEG signal classification and showed high accuracy when combined with other classification techniques [9,2]. Neural Networks usually work well on large datasets, yet Long Short-Term Memory can show high accuracy with small datasets [7,13,20,24]. ...
In this paper, we propose a new method to detect noise hindrances in Electroencephalographic (EEG) signals caused by mental distractions, which we named “daydreaming signals.” Our approach is based on sliding windows and aims to detect and locate these daydreaming signals to specific points in time. We expect to get cleaner data and, therefore, higher prediction accuracy in current available EEG datasets by removing these daydreaming signals. Beyond these improvements to existing data, this approach also has the potential to improve the quality of future data collection, as researchers can discover the pattern of daydreaming signals in trial rounds and deal with these signals accordingly.KeywordsDesign methods and techniquesMachine learningSupervised learningElectroencephalography (EEG)Sliding windowsEEG signal classification
