FIG 8 - uploaded by Ali Kashefi
Content may be subject to copyright.
![R 2 scores obtained (a) with the ReLU activation function [see Eq. (7)] in the last layer of the point-cloud neural network and (b) without using the input and feature transforms in the neural network architecture (see Fig. 4).](profile/Ali-Kashefi/publication/354908376/figure/fig4/AS:1073982056980481@1633068684075/R-2-scores-obtained-a-with-the-ReLU-activation-function-see-Eq-7-in-the-last-layer.png)
R 2 scores obtained (a) with the ReLU activation function [see Eq. (7)] in the last layer of the point-cloud neural network and (b) without using the input and feature transforms in the neural network architecture (see Fig. 4).
Source publication
We propose a novel deep learning framework for predicting the permeability of porous media from their digital images. Unlike convolutional neural networks, instead of feeding the whole image volume as inputs to the network, we model the boundary between solid matrix and pore spaces as point clouds and feed them as inputs to a neural network based o...
Contexts in source publication
Context 1
... to keep the permeability in the physical domain and use the ReLU activation function [see Eq. (7)] in the last layer. This option has been used by several researchers such as Hong and Liu 11 and Tembely et al. 14 We implement the latter option to compare these two strategies. The outcome of using the ReLU function [see Eq. (7)] is illustrated in Fig. 8(a). A comparison between Figs. 8(a) and 6(a) indicates a higher R 2 score for our current approach (i.e., using the sigmoid activation function). Note that the scatter in Figs. 6 and 8 is quantified by the R 2 ...
Context 2
... answer this question, we remove the input and feature transform blocks from the point-cloud neural network (see Fig. 4) to investigate its usefulness. Figure 8(b) shows the R 2 plot as a consequence of this modification. As can be observed in Fig. 8(b), the R 2 score is reduced to 0.915 27. ...
Context 3
... second contribution of the transforms to the computer vision application improves our results as well. To answer this question, we remove the input and feature transform blocks from the point-cloud neural network (see Fig. 4) to investigate its usefulness. Figure 8(b) shows the R 2 plot as a consequence of this modification. As can be observed in Fig. 8(b), the R 2 score is reduced to 0.915 27. Hence, we conclude that the existence of these two transforms increases the network ability for ...
Similar publications
Rigorous boresight calibration between light detection and ranging (LiDAR) and the camera is crucial for geometry and optical information fusion in earth observation and robotic applications. Although boresight parameters can be obtained through pre-calibration with artificial targets, unforeseen movement of sensors during data collection can lead...
Citations
... A prediction at an arbitrary time requires iteratively feeding past ML predictions into the model. Kashefi and Mukerji [20] used neural networks to predict the permeability from 2D or 3D point clouds of the boundary surface of porous media. These studies all had very specific applications in mind, and our work will build upon these methods to show how to approach a general hydrodynamics problem. ...
We are interested in the computational study of shock hydrodynamics, i.e. problems involving compressible solids, liquids, and gases that undergo large deformation. These problems are dynamic and nonlinear and can exhibit complex instabilities. Due to advances in high performance computing it is possible to parameterize a hydrodynamic problem and perform a computational study yielding $\mathcal{O}\left({\rm TB}\right)$ of simulation state data. We present an interactive machine learning tool that can be used to compress, browse, and interpolate these large simulation datasets. This tool allows computational scientists and researchers to quickly visualize "what-if" situations, perform sensitivity analyses, and optimize complex hydrodynamic experiments.
... Fluids flow in the granular aggregates can be investigated by experiments on the tests of transport parameters like permeability [4,5] which characterizes the hydrodynamic properties of the mass flow in the porous media [6]. The permeability can be measured by Darcy test which uses two constant pressure heads of a fluid to generate steady flow in the porous media and then the permeability can be calculated based on the flow rate, pressure gradient and sample sizes [5]. ...
... Fluids flow in the granular aggregates can be investigated by experiments on the tests of transport parameters like permeability [4,5] which characterizes the hydrodynamic properties of the mass flow in the porous media [6]. The permeability can be measured by Darcy test which uses two constant pressure heads of a fluid to generate steady flow in the porous media and then the permeability can be calculated based on the flow rate, pressure gradient and sample sizes [5]. For convenience, the falling-head method can also be used in the granular packing though improvements are needed for the high flow rate of larger-size aggregates packing [7]. ...
... The simulation mainly involves two steps, i.e., collision and streaming. After the simulation reaches to convergency, the permeability can be obtained by the Darcy's law [5,34]: ...
Properties of fluids flow in granular aggregates are important for the design of pervious infrastructures used to alleviate urban water-logging problems. Here in this work, five groups of aggregates packing with similar average porosities but varying particle sizes were scanned by a high-energy X-ray computed tomography (X-CT) facility. The structures of the packings were reconstructed. Porosities were calculated and compared with those measured by the volume and mass of infilled water in the packing. Then pore networks were extracted and analyzed. Simulations of fluids flow in the packings were performed by using a lattice Boltzmann method (LBM) with BGK (Bhatnagar-Gross-Krook) collision model in the pore-network domain of the packings. Results showed wall effect on the porosity of aggregates packing was significant and the influence increased with the aggregate sizes. In addition, Poisson law and power law can be used to fit the coordination number and coordination volume of the packing's pore network, respectively. Moreover, the mass flow rates of fluids in the aggregates were affected by the porosities. On the two-dimensional slices, the mass flow rate decreased when the slice porosity increased. But for the three-dimensional blocks, the average mass flow rate increased with the volume porosity. And the permeability of the aggregates packing showed correlating change trend with the average pore diameter and fitting parameters of coordination volumes, when the sizes of aggregates changed. Though the limitation of merging interfaces causing fluctuation and discontinuity on micro parameters of fluid flow existed, the methods and results here may provide knowledge and insights for numerical simulations and optimal design of aggregate-based materials.
... Finally, it is not straightforward to embed the complexity of varying degrees of confinement effects in the ML model. While much progress has been made in emulating transport in porous media using ML [6][7][8][9][10][11][12][13][14][15], models in the literature are not general enough to be used as a generic tool. ...
Modeling effective transport properties of 3D porous media, such as permeability, at multiple scales is challenging as a result of the combined complexity of the pore structures and fluid physics—in particular, confinement effects which vary across the nanoscale to the microscale. While numerical simulation is possible, the computational cost is prohibitive for realistic domains, which are large and complex. Although machine learning (ML) models have been proposed to circumvent simulation, none so far has simultaneously accounted for heterogeneous 3D structures, fluid confinement effects, and multiple simulation resolutions. By utilizing numerous computer science techniques to improve the scalability of training, we have for the first time developed a general flow model that accounts for the pore-structure and corresponding physical phenomena at scales from Angstrom to the micrometer. Using synthetic computational domains for training, our ML model exhibits strong performance (R ² = 0.9) when tested on extremely diverse real domains at multiple scales.
... In recent years, with the explosive growth of data and rapid development of computer hardware, deep learning algorithms, represented by neural networks, have made significant advances in areas such as computer vision and image processing . Deep learning has been widely applied in various applications of porous media research, such as permeability estimation (WU et al., 2018) (RABBANI et al., 2020 (ISHOLA and VILCáEZ, 2022) (Kashefi and Mukerji, 2021), porosity characterization (WU et al., 2018) (RABBANI et al., 2020) (ISHOLA and VILCáEZ, 2022), flow field prediction (Takbiri et al., 2022) (Takbiri-Borujeni et al., 2020), and colloidal deposition (Marcato et al., 2021), among others. Numerous studies have shown that data-driven deep learning methods have great potential for predicting flow parameters in porous media. ...
... Numerous studies have shown that data-driven deep learning methods have great potential for predicting flow parameters in porous media. Taking pure data-driven machine learning as an example, Keshefi et al. (Kashefi and Mukerji, 2021) utilized point cloud neural networks to obtain the geometric shapes of solid skeletons and pore boundaries, representing porous media as a set of points rather than complete porous media voxels. This approach enables the prediction of permeability and flow fields in porous media using point clouds. ...
... This requirement for extensive Graphics Processing Unit (GPU) memory not only demands resources but also affects training accuracy by constraining the batch size, a critical parameter for model optimization [25][26][27]. To address some of these limitations, PointNet architectures were introduced, which represent the microstructures with a point cloud to predict the permeability of porous media [28]. Despite its more economic use of GPU memory compared to CNNs, PointNet also has its limitations. ...
This study presents a Graph Neural Networks (GNNs)-based approach for predicting the effective elastic moduli of rocks from their digital CT-scan images. We use the Mapper algorithm to transform 3D digital rock images into graph datasets, encapsulating essential geometrical information. These graphs, after training, prove effective in predicting elastic moduli. Our GNN model shows robust predictive capabilities across various graph sizes derived from various subcube dimensions. Not only does it perform well on the test dataset, but it also maintains high prediction accuracy for unseen rocks and unexplored subcube sizes. Comparative analysis with Convolutional Neural Networks (CNNs) reveals the superior performance of GNNs in predicting unseen rock properties. Moreover, the graph representation of microstructures significantly reduces GPU memory requirements (compared to the grid representation for CNNs), enabling greater flexibility in the batch size selection. This work demonstrates the potential of GNN models in enhancing the prediction accuracy of rock properties and boosting the efficiency of digital rock analysis.
... 11 Additionally, the inherent susceptibility of soil structure to changes makes it challenging to obtain complete soil samples for permeability studies. 12 Consequently, some scholars aim to clarify the percolation mechanism 13 and establish the relationship between pore characteristics and permeability through theoretical analysis, 14 experiments, 15 simulation, 16 and artificial intelligence, 3,17 to facilitate the engineering practice. ...
The fluid transport in porous media is a critical property for oil and gas exploitation, construction engineering, and environmental protection. It is profoundly influenced by pore geometry and mineral properties. Currently, the Kozeny–Carman equation serves as the permeability prediction equation for porous media, established on the circular pores model. However, it fails to fully account for the impact of pore shape and mineral properties of the soil, leading to significant deviations between predicted and measured soil permeability results. In this paper, based on scanning electron microscope image and mercury intrusion porosimetry, the pores were divided into circular pores and narrow slit pores according to the ratios of pore area and circumference. Then, the quantitative expression of the two types of pores and their connectivity and tortuosity were given, and the circular and narrow slit composite pore model was used to describe the soil pore. Subsequently, the electrostatic potential of pore water was calculated by the Poisson–Boltzmann equation to consider the adsorption effect of minerals on pore water. Combined with the Navier–Stokes equation, the permeability prediction equation considering pore geometry, pore connectivity, and tortuosity and mineral properties was established. Finally, the experimental results illustrated that the theoretical prediction results were in good agreement with the experimental results. The proposed permeability prediction equation proves valuable for assessing and predicting the fluid transport in porous media.
... This requirement for extensive Graphics Processing Unit (GPU) memory not only demands resources but also affects training accuracy by constraining the batch size, a critical parameter for model optimization (Smith et al., 2017;He et al., 2019;Kandel and Castelli, 2020). To address some of these limitations, PointNet architectures were introduced, which represent the microstructures with a point cloud to predict the permeability of porous media (Kashefi and Mukerji, 2021). Despite its more economic use of GPU memory compared to CNNs, PointNet also has its limitations. ...
This study presents a Graph Neural Networks (GNNs)-based approach for predicting the effective elastic moduli of rocks from their digital CT-scan images. We use the Mapper algorithm to transform 3D digital rock images into graph datasets, encapsulating essential geometrical information. These graphs, after training, prove effective in predicting elastic moduli. Our GNN model shows robust predictive capabilities across various graph sizes derived from various sub-cube dimensions. Not only does it perform well on the test dataset, but it also maintains high prediction accuracy for unseen rocks and unexplored subcube sizes. Comparative analysis with Convolutional Neural Networks (CNNs) reveals the superior performance of GNNs in predicting unseen rock properties. Moreover, the graph representation of microstructures significantly reduces GPU memory requirements (compared to the grid representation for CNNs), enabling greater flexibility in the batch size selection. This work demonstrates the potential of GNN models in enhancing the prediction accuracy of rock properties and boosting the efficiency of digital rock analysis.
... Different from the digital models directly built by computer aided design (CAD) software, the raw geometric information of the heterogeneous geometries, which exist in natural structures including porous rock and soil, or the artificial structure including alloy material and concrete material, is obtained by camera such as CT scanner [5,6]. These digital models have voxel-based irregular shape (named as VI-shape for brevity) rather than the smooth shape analytically described by non-uniform rational B-splines (NURBS) function, leading to specific requirements for numerical methods to handle the voxel-based irregular structures [7][8][9]. ...
... Machine learning (ML), especially neural network (NN), has emerged as powerful computing algorithms that have gained widespread adoption in the engineering field in recent years [19][20][21][22], including properties prediction [9,23,24], data recovery [25][26][27], structure optimization [28][29][30], constitutive substitution model [31,32] and partial differential equations (PDEs) solver [33][34][35][36], etc. ML methods open up new possibilities for FEM to handle VI-shape, and existing ML-enhanced FEM methods [8,[37][38][39][40][41][42][43][44][45] provide inspiration for the establishment of finite elements with interpolation functions capable for VI-shape. One of the recent representative works is the multiscale model proposed by Yin et al. [41]. ...
... It captures the spatial distributions and properties of point clouds by applying a series of symmetric functions, and has been widely used in tasks such as object classification (Qi et al., 2017) and recognition (Wu et al., 2020) in three dimensional (3D) datasets. With respect to pore-scale modelling, a PointNetbased approach has been proposed to predict porous media properties from point cloud representations of porous media (Kashefi and Mukerji, 2021). In this approach, the authors extracted the point cloud coordinates of the rock surface from binary images and used them to train the PointNet architecture to regress the permeability of the input point clouds. ...
... We adopted the Adam optimizer (Kingma and Ba, 2014) and set the learning rate to 0.1, 0.05, and 0.025 times the initial learning rate when the epoch was 30, 60, and 90, respectively. In neural networks, the hyperparameter batch size is limited by GPU memory and thus affects their performance (Kandel and Castelli, 2020;Kashefi and Mukerji, 2021;Keskar et al., 2016). We set the batch size to 8, 16, 32, and 64 while keeping the rest of the hyperparameters unchanged to observe the effect of batch size on the performance of the two models for predicting the permeability of 3D porous media. ...
... We evaluated the performance of the models on the validation subset and the testing set by the coefficient of determination R 2 , the RMSE, and the median absolute relative error (MedARE), measures that are widely used in predicting the permeability of porous media (Elmorsy et al., 2022;Kashefi and Mukerji, 2021;Tang et al., 2022). The evaluation functions can be stated by the following expressions: ...
The direct acquisition of the permeability of porous media by digital images helps to enhance our understanding of and facilitate research into the problem of subsurface flow. A complex pore space makes the numerical simulation methods used to calculate the permeability quite time-consuming. Deep learning models represented by three-dimensional convolutional neural networks (3D CNNs), as a promising approach to improving efficiency, have made significant advances concerning predicting the permeability of porous media. However, 3D CNNs require significant computational resources due to their extensive parameters, which limit studies to small-sized porous media, and their generalization capabilities are insufficiently explored. To address these challenges, we propose a novel CNN-Transformer hybrid neural network, merging a 2D CNN with a self-attention mechanism. Additionally, we incorporate physical information into digital images, constructing a PhyCNN-Transformer model to reflect the impact of physical properties on permeability prediction. In terms of dataset preparation, we employ the publicly available DeePore porous media dataset with sample size of 2563 cubic voxels and labeled permeability calculated by pore network modelling (PNM). We compare the two transformer-based models with a 3D CNN in terms of parameter number, training efficiency, prediction performance, and generalization, and the results show significant improvement. By employing transfer learning, the well-trained transformer-based models proved capable of adapting to porous media with different sizes (achieving an R2 score of 0.9563 with 300 training samples), while the 3D CNN lacks this transferability.
