Content uploaded by Derek Groen
Author content
All content in this area was uploaded by Derek Groen on Feb 20, 2015
Content may be subject to copyright.
A preview of the PDF is not available
Lattice-Boltzmann methods in fluid dynamics: Turbulence and complex colloidal fluids
Figures
Figures - uploaded by Derek Groen
Author content
All figure content in this area was uploaded by Derek Groen
Content may be subject to copyright.
... lbmpy is a system for development of computational fluid dynamics codes based on the LBM. In this regard, it is comparable to other LBM frameworks such as OpenLB [22], [30], Palabos [27], [31], elbe [29], [12], LB3D [18], [34], [28] and HemeLB [19]. Some of these LB frameworks like Sailfish [32] and TCLB [36] also use metaprogramming techniques, mainly for portability to GPUs. ...
Lattice Boltzmann methods are a popular mesoscopic alternative to macroscopic computational fluid dynamics solvers. Many variants have been developed that vary in complexity, accuracy, and computational cost. Extensions are available to simulate multi-phase, multi-component, turbulent, or non-Newtonian flows. In this work we present lbmpy, a code generation package that supports a wide variety of different methods and provides a generic development environment for new schemes as well. A high-level domain-specific language allows the user to formulate, extend and test various lattice Boltzmann schemes. The method specification is represented in a symbolic intermediate representation. Transformations that operate on this intermediate representation optimize and parallelize the method, yielding highly efficient lattice Boltzmann compute kernels not only for single- and two-relaxation-time schemes but also for multi-relaxation-time, cumulant, and entropically stabilized methods. An integration into the HPC framework waLBerla makes massively parallel, distributed simulations possible, which is demonstrated through scaling experiments on the SuperMUC-NG supercomputing system
... The LBM has been applied to many disciplines of computational science and engineering, due to how well it can be scaled up on high performance computing (HPC) clusters. Recently developed massively parallel software are the Multiphysics WaLBerla (Feichtinger at al. 2011) able to model 10 12 nodes on Petascale computers, the non-uniform grid implementation reaching up to a trillion grid nodes illustrated in (Schornbaum & Rüde 2016), the widely used Palabos (Lagrava et al. 2012), OpenLB (Heuveline & Latt 2007), LB3D (Groen et al. 2011), all of them showing excellent performance on HPC. In this work, we illustrate an example of parallel implementation in 2-D, tested up to 1280 cores, only in Python, for pedagogical purposes and for specific applications in geosciences. ...
The Lattice Boltzmann Method (LBM) is a method to simulate fluid dynamics based on modelling distributions of particles moving and colliding on a lattice. The {\it Python} scripting language provides a clean programming paradigm to develop codes based on the Lattice Boltzmann Method, however in order to reach performance comparable to compiled languages, it needs to be carefully implemented, maximizing its vectorized tools, mostly integrated in the NumPy module. We present here the details of a Python implementation of a concise LBM code, with the purpose of offering a pedagogical tool for students and professionals in the geosciences who are approaching this technique for the first time. The first half of the paper focuses on how to vectorize a 2D LBM code and show how if carefully done, this allows performance close to a compiled code. In the second part of the paper, we use the vectorization described earlier to naturally write a parallel implementation using MPI and test both weak and hard scaling up to 1280 cores. One benchmark, Poiseuille flow, and two applications, one on sound wave propagation and another on fluid-flow through a simplified model of a rock matrix are finally shown.
... A priori static grid refinement in parallel simulations based on the LBM is studied in [Fre08,Has14b,Sch11]. Other popular simulation codes based on the LBM such as Palabos [Lag12,Palabos], OpenLB [Fie12,Heu07,OpenLB], LB3D [Gro11,LB3D], HemeLB [Gro13], HARVEY [Ran13], or LUDWIG [Des01] are currently at a state that they either do not support grid refinement, only support grid refinement for 2D problems, or have not yet demonstrated large-scale simulations on nonuniform grids. ...
Dynamically adapting the space discretization during the runtime of a simulation can contribute to significantly reduce the time to solution. This thesis presents a novel approach for block-structured adaptive mesh refinement (AMR) that is suitable for extreme-scale parallelism. A central component of this approach are data structures that are designed such that the size of the meta data of each process remains bounded independent of the total number of processes. Moreover, all stages of the AMR algorithm only use distributed algorithms, meaning no data replication or central resources such as a master process are required. For the dynamic load balancing in particular, the thesis proposes to exploit the hierarchical nature of the block-structured domain partitioning by creating a lightweight, temporary copy of the distributed core data structure. This copy, however, does not contain any simulation data, but only acts as a proxy that provides topological information about the domain partitioning into blocks. Ultimately, this approach enables inexpensive, diffusion-based dynamic load balancing schemes. All proposed algorithms operate on the blocks that result from the domain partitioning. This concept and its realization enable the storage of arbitrary data. In consequence, the resulting software framework can be used for different simulation methods, including mesh-based and meshfree methods. In this thesis, fluid simulations based on the lattice Boltzmann method (LBM) are presented. To that end, the thesis introduces a novel parallelization scheme for the LBM on nonuniform grids. Because of its applicability for massively parallel simulations, the LBM has increasingly become an alternative method for large-scale non-stationary flow simulations. Using such LBM-based flow simulations, the excellent performance and scalability of the AMR implementation are demonstrated for two architecturally different petascale supercomputers. Benchmarks on an IBM Blue Gene/Q system with a mesh containing 3.7 trillion unknowns distributed to 458,752 processes confirm the applicability for future extreme-scale parallel machines. On the same system, the implementation of the LBM on nonuniform grids demonstrates an absolute performance of close to a trillion lattice cell updates per second when executed with almost two million threads. On an Intel Xeon-based supercomputer, LBM-based simulations on nonuniform grids that span the entire machine are shown to be capable of reaching a performance of less than one millisecond per time step. With 65,536 processor cores, one complete AMR cycle, including adaptive refinement and dynamic load balancing, is demonstrated to be executed in half a second for a mesh that consists of a total of 13.8 billion cells.
... Flow of complex fluids in porous media and under shear was investigated[23]. The utilisation of substantial resources like the US TeraGrid and UK HPC facilities in ground breaking federation of national grid infrastructures[12][13][14]and the highest performance class of national supercomputers for capability computing[24,25], allowed scientists working with LB3D large-scale numerical investigation of rheological properties of the gyroid phase[26][27][28][29]. Later work explored the parameters of ternary amphiphilic fluids and their flow properties in detail[30][31][32][33][34]. ...
We introduce the lattice-Boltzmann code LB3D, version 7.1. Building on a parallel program and supporting tools which have enabled research utilising high performance computing resources for more than 16 years, LB3D version 7 provides a subset of the research code functionality as an open source project. Here, we describe the theoretical basis of the algorithm as well as computational aspects of the implementation. The software package is validated against simulations of meso-phases resulting from self-assembly in ternary fluid mixtures comprising immiscible and amphiphilic components such as water-oil-surfactant systems. The impact of the surfactant species on the dynamics of spinodal decomposition are tested and quantitative measurement of the permeability of a body centred cubic (BCC) model porous medium for a simple binary mixure is reported. Single-core performance and scaling behaviour of the code are reported for simulations on current supercomputer architectures.
... [25] introduces the MUSUBI software that relies on a linearized octree, [38] uses the Peano framework [7] that is based on a more generalized spacetree concept. To our best knowledge, other popular simulation codes based on the LBM such as Palabos [39,31], OpenLB [27,30], LB3D [33,21], HemeLB [22], HARVEY [43], or LUDWIG [12] are at a state that they either do not support grid refinement, only support grid refinement for 2D problems, or have not yet demonstrated large-scale simulations on non-uniform grids. ...
The lattice Boltzmann method exhibits excellent scalability on current
supercomputing systems and has thus increasingly become an alternative method
for large-scale non-stationary flow simulations, reaching up to a trillion grid
nodes. Additionally, grid refinement can lead to substantial savings in memory
and compute time. These saving, however, come at the cost of much more complex
data structures and algorithms. In particular, the interface between subdomains
with different grid sizes must receive special treatment. In this article, we
present parallel algorithms, distributed data structures, and communication
routines that extend the software framework waLBerla in order to support
large-scale, massively parallel lattice Boltzmann-based simulations on
non-uniform grids. Additionally, we evaluate the performance of our approach on
two current petascale supercomputers. On an IBM Blue Gene/Q system, the largest
weak scaling benchmarks with refined grids are executed with almost two million
threads, demonstrating not only near-perfect scalability but also an absolute
performance of close to a trillion lattice Boltzmann cell updates per second.
On an Intel-based system, strong scaling a simulation with refined grids and a
total of more than 8.5 million cells is demonstrated to reach a performance of
less than one millisecond per time step. This enables simulations with complex,
non-uniform grids and four million time steps per hour compute time.
... It has been ported to many supercomputers worldwide, where it has shown excellent scalability. Recently it has been shown to scale linearly on up to 262,144 cores on the European Blue Gene/P system Jugene [30]. We were able to confirm very good scaling of our ternary systems on the Opteron based XC2 as well (Fig. 2, left) In recent years, lb3d has been coupled to a molecular dynamics solver in order to simulate complex fluids containing particulate components, such as blood [31], or nano-particle stabilised emulsions and suspensions [32]. ...
The flow of fluid mixtures in complex geometries is of practical interest in many fields, ranging from oil recovery to freeze-dried food products. Due to its inherent locality the lattice Boltzmann method allows for straightforward implementation of complex boundaries and excellently scaling parallel computations. The widely applied Bhatnagar Gross Krook (BGK) scheme, used to model the contribution of particle collisions to the velocity field, does however suffer from limitiations in precision that become more prominent with increasing surface to volume ratio. To increase the accuracy of simulations of mixtures in porous media, we integrated a so-called multi relaxation time (MRT) collision scheme with a pseudo-potential method for fluids with multiple components. We describe some optimisation details of the implementation and present test results verifying the physical accuracy as well as benchmarks obtained on the XC2 Opteron cluster at the Scientific Supercomputing Centre Karlsruhe (SSCK).
Lattice Boltzmann methods are a popular mesoscopic alternative to classical computational fluid dynamics based on the macroscopic equations of continuum mechanics. Many variants of lattice Boltzmann methods have been developed that vary in complexity, accuracy, and computational cost. Extensions are available to simulate multi-phase, multi-component, turbulent, and non-Newtonian flows. In this work we present lbmpy, a code generation package that supports a wide variety of different lattice Boltzmann methods. Additionally, lbmpy provides a generic development environment for new schemes. A high-level domain-specific language allows the user to formulate, extend and test various lattice Boltzmann methods. In all cases, the lattice Boltzmann method can be specified in symbolic form. Transformations that operate on this symbolic representation yield highly efficient compute kernels. This is achieved by automatically parallelizing the methods, and by various application-specific automatized steps that optimize the resulting code. This pipeline of transformations can be applied to a wide range of lattice Boltzmann variants, including single- and two-relaxation-time schemes, multi-relaxation-time methods, as well as the more advanced cumulant methods, and entropically stabilized methods. lbmpy can be integrated into high-performance computing frameworks to enable massively parallel, distributed simulations. This is demonstrated using the waLBerla multiphysics package to conduct scaling experiments on the SuperMUC-NG supercomputing system on up to 147 456 compute cores.
Programming current supercomputers efficiently is a challenging task. Multiple levels of parallelism on the core, on the compute node, and between nodes need to be exploited to make full use of the system. Heterogeneous hardware architectures with accelerators further complicate the development process. waLBerla addresses these challenges by providing the user with highly efficient building blocks for developing simulations on block-structured grids. The block-structured domain partitioning is flexible enough to handle complex geometries, while the structured grid within each block allows for highly efficient implementations of stencil-based algorithms. We present several example applications realized with waLBerla, ranging from lattice Boltzmann methods to rigid particle simulations. Most importantly, these methods can be coupled together, enabling multiphysics simulations. The framework uses meta-programming techniques to generate highly efficient code for CPUs and GPUs from a symbolic method formulation.
In this article, we present a novel approach for block-structured adaptive mesh refinement (AMR) that is suitable for extreme-scale parallelism. All data structures are designed such that the size of the meta data in each distributed processor memory remains bounded independent of the processor number. In all stages of the AMR process, we use only distributed algorithms. No central resources such as a master process or replicated data are employed, so that an unlimited scalability can be achieved. For the dynamic load balancing in particular, we propose to exploit the hierarchical nature of the block-structured domain partitioning by creating a lightweight, temporary copy of the core data structure. This copy acts as a local and fully distributed proxy data structure. It does not contain simulation data, but only provides topological information about the domain partitioning into blocks. Ultimately, this approach enables an inexpensive, local, diffusion-based dynamic load balancing scheme. We demonstrate the excellent performance and the full scalability of our new AMR implementation for two architecturally different petascale supercomputers. Benchmarks on an IBM Blue Gene/Q system with a mesh containing 3.7 trillion unknowns distributed to 458,752 processes confirm the applicability for future extreme-scale parallel machines. The algorithms proposed in this article operate on blocks that result from the domain partitioning. This concept and its realization support the storage of arbitrary data. In consequence, the software framework can be used for different simulation methods, including mesh based and meshless methods. In this article, we demonstrate fluid simulations based on the lattice Boltzmann method.
Fluid-fluid interfaces appear in numerous systems of academic and industrial interest. Their dynamics is difficult to track since they are usually deformable and of not a priori known shape. Computer simulations pose an attractive way to gain insight into the physics of interfaces. In this report we restrict ourselves to two classes of interfaces and their simulation by means of numerical schemes coupled to the lattice Boltzmann method as a solver for the hydrodynamics of the problem. These are the immersed boundary method for the simulation of vesicles and capsules and the Shan-Chen pseudopotential approach for multi-component fluids in combination with a molecular dynamics algorithm for the simulation of nanoparticle stabilized emulsions. The advantage of these algorithms is their inherent locality allowing to develop highly scalable codes which can be used to harness the computational power of the currently largest available supercomputers.
The purpose of the present paper is to report on the first computational study of the dynamical and rheological response o a self-assembled diamond mesophase under Couette flow in a ternary mixture composed of oil, water and an amphiphilic species.
The amphiphilic diamond mesophase arises in a wide range of chemical and biological systems, and a knowledge of its rheologica response has important implications in materials science and biotechnological applications. The simulations reported her are performed using a kinetic lattice–Boltzmann method. Lyotropic liquid crystals exhibit characteristic rheological response in experiments that include shear-banding and a non-Newtonian flow curve as well as viscoelasticity under oscillatory shear.
Their behaviour under steady and oscillatory shear is correctly reproduced in our simulations. On cessation of shear, as th morphology returns to the diamond phase, the relaxation of the stress response follows a stretched-exponential form for lo initial strain rates.
Particle stabilized emulsions are ubiquitous in the food and cosmetics
industry, but our understanding of the influence of microscopic fluid-particle
and particle-particle interactions on the macroscopic rheology is still
limited. In this paper we present a simulation algorithm based on a
multicomponent lattice Boltzmann model to describe the solvents combined with a
molecular dynamics solver for the description of the solved particles. It is
shown that the model allows a wide variation of fluid properties and arbitrary
contact angles on the particle surfaces. We demonstrate its applicability by
studying the transition from a "bicontinuous interfacially jammed emulsion gel"
(bijel) to a "Pickering emulsion" in dependence on the contact angle, the
particle concentration, and the ratio of the solvents.
Human blood flow is a multiscale problem: in first approximation, blood is a dense suspension of plasma and deformable red cells. Physiological vessel diameters range from about one to thousands of cell radii. Current computational models either involve a homogeneous fluid and cannot track particulate effects or describe a relatively small number of cells with high resolution but are incapable to reach relevant time and length scales. Our approach is to simplify much further than existing particulate models. We combine well-established methods from other areas of physics in order to find the essential ingredients for a minimalist description that still recovers hemorheology. These ingredients are a lattice Boltzmann method describing rigid particle suspensions to account for hydrodynamic long-range interactions and-in order to describe the more complex short-range behavior of cells-anisotropic model potentials known from molecular-dynamics simulations. Paying detailedness, we achieve an efficient and scalable implementation which is crucial for our ultimate goal: establishing a link between the collective behavior of millions of cells and the macroscopic properties of blood in realistic flow situations. In this paper we present our model and demonstrate its applicability to conditions typical for the microvasculature.
The fractal invariant measure of chaotic strange attractors can be approximated systematically by the set of unstable n-periodic orbits of increasing n. Algorithms for extracting the periodic orbits from a chaotic time series and for calculating their stabilities are presented. With this information alone, important properties like the topological entropy and the Hausdorff dimension can be calculated.