No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
A finite-volume module for simulating global all-scale atmospheric flows
Abstract
The paper documents the development of a global nonhydrostatic finite-volume module designed to enhance an established spectral-transform based numerical weather prediction (NWP) model. The module adheres to NWP standards, with formulation of the governing equations based on the classical meteorological latitude-longitude spherical framework. In the horizontal, a bespoke unstructured mesh with finite-volumes built about the reduced Gaussian grid of the existing NWP model circumvents the notorious stiffness in the polar regions of the spherical framework. All dependent variables are co-located, accommodating both spectral-transform and grid-point solutions at the same physical locations. In the vertical, a uniform finite-difference discretisation facilitates the solution of intricate elliptic problems in thin spherical shells, while the pliancy of the physical vertical coordinate is delegated to generalised continuous transformations between computational and physical space. The newly developed module assumes the compressible Euler equations as default, but includes reduced soundproof PDEs as an option. Furthermore, it employs semi-implicit forward-in-time integrators of the governing PDE systems, akin to but more general than those used in the NWP model. The module shares the equal regions parallelisation scheme with the NWP model, with multiple layers of parallelism hybridising MPI tasks and OpenMP threads. The efficacy of the developed nonhydrostatic module is illustrated with benchmarks of idealised global weather.
... Schur complement). In particular, the SI approach is at the heart of the newly developed finite-volume module (FVM) of the Integrated Forecasting System (IFS) at the European Centre for Medium-Range Weather Forecasts (ECMWF) Smolarkiewicz et al., 2016Smolarkiewicz et al., , 2017Smolarkiewicz et al., , 2019. The FVM provides an alternative dynamical core to the spectral transform based IFS. ...
... With our choice of model, we aim to stay conceptionally as close as possible to the approach used by other MPDATA based fluid models (Kurowski et al., 2016;Prusa et al., 2008;Szmelter & Smolarkiewicz, 2010) and consequently the newly developed IFS-FVM atmospheric model Smolarkiewicz et al., 2016Smolarkiewicz et al., , 2019. Thus, we chose to use the shallow-water equations (SWE) on the sphere written in the form of generalized transport equations (Smolarkiewicz & Margolin, 1998;Szmelter & Smolarkiewicz, 2010): ...
... The second term however is unknown, as R n+1 depends (nonlinearly) both on Φ n+1 and Q n+1 . There are several alternative approaches to assure an easily invertable linear problem, involving outer iterations or explicit predictors, or both Smolarkiewicz et al., 2016Smolarkiewicz et al., , 2017Smolarkiewicz et al., , 2019. As our goal is to assess the role of mixed precision on the elliptic solver performance, we here select an ad hoc approach that complicates (rather than simplifies) the target elliptic solver. ...
Semi‐implicit (SI) time‐stepping schemes for atmosphere and ocean models require elliptic solvers that work efficiently on modern supercomputers. This paper reports our study of the potential computational savings when using mixed precision arithmetic in the elliptic solvers. Precision levels as low as half (16 bits) are used and a detailed evaluation of the impact of reduced precision on the solver convergence and the solution quality is performed. This study is conducted in the context of a novel SI shallow‐water model on the sphere, purposely designed to mimic numerical intricacies of modern all‐scale weather and climate (W&C) models. The governing algorithm of the shallow‐water model is based on the non‐oscillatory MPDATA methods for geophysical flows, whereas the resulting elliptic problem employs a strongly preconditioned non‐symmetric Krylov‐subspace Generalized Conjugated‐Residual (GCR) solver, proven in advanced atmospheric applications. The classical longitude/latitude grid is deliberately chosen to retain the stiffness of global W&C models. The analysis of the precision reduction is done on a software level, using an emulator, whereas the performance is measured on actual reduced precision hardware. The reduced‐precision experiments are conducted for established dynamical‐core test‐cases, like the Rossby‐Haurwitz wavenumber 4 and a zonal orographic flow. The study shows that selected key components of the elliptic solver, most prominently the preconditioning and the application of the linear operator, can be performed at the level of half precision. For these components, the use of half precision is found to yield a speed‐up of a factor 4 compared to double precision for a wide range of problem sizes.
... In the example of a numerical model, Borchert et al. [2019] introduced an upper-atmosphere extension to the ICON model [Zängl et al., 2015] that solves the full Coriolis term. Another deep-atmosphere numerical model with full Coriolis support has been published by Smolarkiewicz et al. [2016]. From hereon, we use the term "non-traditional setting" to refer to a setting where the full effect of the Coriolis force is considered. ...
... However, the incorporation of height-dependent dissipation into our model would render it intractable for analytic progress, and we argue that the free surface is a worthwhile alternative for this study. Furthermore, while non-reflecting upper boundaries may be physically unrealistic, they are common practice in numerical weather and climate models including deep-atmosphere non-traditional models [Smolarkiewicz et al., 2016, Borchert et al., 2019. Subsequent investigations in this paper will assume a free surface for the top boundary. ...
... Given the free surface top boundary, which is usually employed as a "sponge" layer in weather and climate models, one would encounter this instability in numerical simulations. This effect would be especially significant for the numerical simulations of the deep atmosphere, e.g. the papers by Borchert et al. [2019], Smolarkiewicz et al. [2016] as mentioned above, or tropical dynamics with the non-traditional setting. Therefore, the characterisation of the instability presented here is a worthwhile pursuit. ...
The traditional approximation neglects the cosine components of the Coriolis acceleration, and this approximation has been widely used in the study of geophysical phenomena. However, the justification of the traditional approximation is questionable under a few circumstances. In particular, dynamics with substantial vertical velocities or geophysical phenomena in the tropics have non-negligible cosine Coriolis terms. Such cases warrant investigations with the non-traditional setting, i.e., the full Coriolis acceleration. In this manuscript, we study the effect of the non-traditional setting on an isothermal, hydrostatic and compressible atmosphere assuming a meridionally homogeneous flow. Employing linear stability analysis, we show that, given appropriate boundary conditions, i.e. free-slip boundary at the surface and non-reflecting boundary at the top, the equatorial atmosphere at rest becomes unstable. Numerical experiments were conducted, and Rayleigh damping is used as a numerical approximation for the non-reflecting top boundary. Our two main results are as follows. 1) Experiments involving the full Coriolis terms exhibit exponentially growing instability while experiments subject to the same initial condition and the traditional approximation remain stable. 2) The experimental instability growth rate is close to the theoretical value. Despite the limitations of our investigations wherein only studies on the $f$-plane are conducted, and effects from the $\beta$-plane approximation are ignored, the presence of this instability may have physical and experimental implications for the non-traditional setting. A discussion of the limitations and implications of our study concludes our investigations. Specifically, the influence on numerical deep-atmosphere models and the validity of assuming meridionally homogeneous flow are discussed.
... This led to a substantial efficacy gain for the entire anelastic model and paved the way towards ADI preconditioners meeting needs of all-scale global atmospheric models [34]. Here we further the latter development by extending it on the generalized all-scale perturbation equations of [61] and other advancements originated in the Finite-Volume Module (FVM) [59,60,24] of the Integrated Forecasting System (IFS) at the European Centre for Medium Range Weather Forecasts (ECMWF). The aspiration of this work is to document a suite of effective and flexible preconditioners that rely on bespoke Thomas algorithms operating on collocated grids with periodic and Neumann boundaries. ...
... After [61] it is referred as GBIS, for "generalized buoyancy and implicit shear". The most established form [58,59,22,60,61,24], referred as REF for the "reference", solves the equations for perturbations of the potential temperature and pressure while retaining the full velocity components. From the two intermediate forms, the "implicit shear" (IS) extends REF the vector velocity perturbations, whereas the "generalized buoyancy" (GB) extends REF by maximizing the implicitness of the model integrators. ...
... Circumventing the stiffness crucially depends on maximizing the implicitness of the resulting linear problem. The outlined class of algorithms (6)-widely documented in the literature, see [58,59] and references therein-benefits both computational stability and accuracy of discrete integrations. ...
The paper documents a suite of preconditioners for Krylov-subspace solvers of elliptic boundary-value problems (BVPs) that underlie semi-implicit integrations of the nonhydrostatic equations governing the dynamics of all-scale atmospheric flows. Effective preconditioning of the linear operators inherent in the semi-implicit models lies at the heart of the state-of-the-art multiscale-flow simulation. This is especially evident in simulations of global weather and climate—posed on a thin spherical shell—where some form of direct tridiagonal inversion of the operator in the vertical is crucial to relax the often enormous stiffness of the problem. The documented preconditioners stem from the Richardson's (1910) idea of augmenting an elliptic BVP with a transient diffusion equation. Exploiting this idea for mixed explicit-implicit pseudo-time-stepping schemes leads to a broad suite of stationary-iteration solvers, including the many classical algorithms. Here, the high-performance all-scale EULAG model [J. Comput. Phys. 263 (2014) 185-205], with a flexible three-dimensional decomposition of MPI tasks, is furnished with the preconditioners akin to the classical alternating-direction-implicit (ADI) algorithms, generalized to optional permutations of parallel tridiagonal inversions. The utility of various options is found to be problem dependent, in terms of computational accuracy as well as efficiency. The main thrust of the work is on the long-range forecasts using large anisotropic grids. The relative efficiency and/or accuracy gains attainable with the developed preconditioners are illustrated for idealized scenarios representative of atmospheric flows from planetary to a single-cloud and laboratory scales. The key insight that best encapsulates the significance and novelty of the present work is that there is no single “super-preconditioner” that will perform best in all cases, yet the suite as a whole offers substantial gains in the model performance.
... Schur complement). In particular, the SI approach is at the heart of the newly-developed Finite-Volume Module (FVM) of the Integrated Forecasting System (IFS) at the European Centre for Medium-Range Weather Forecasts (ECMWF) [18,19,20,21]. The FVM provides an alternative dynamical core to the spectral transform based IFS. ...
... With our choice of model, we aim to stay conceptionally as close as possible to the approach used by other MPDATA based fluid models [35,23,36] and consequently the newly developed IFS-FVM atmospheric model [18,20,21]. Thus, we chose to use the shallow-water equations (SWE) on the sphere written in the form of generalized transport equations [37,23]: ...
... The second term is unknown, as R n+1 depends (nonlinearly) both on Φ n+1 and Q n+1 . There are several alternative approaches to assure an easily invertable linear problem, involving outer iterations or explicit predictors, or both [18,19,20,21]. As our goal is to assess the role of mixed precision on the elliptic solver performance, we here select an ad hock approach that complicates (rather than simplifies) the target elliptic solver. ...
Semi-implicit time-stepping schemes for atmosphere and ocean models require elliptic solvers that work efficiently on modern supercomputers. This paper reports our study of the potential computational savings when using mixed precision arithmetic in the elliptic solvers. The essential components of a representative elliptic solver are run at precision levels as low as half (16 bits), and accompanied with a detailed evaluation of the impact of reduced precision on the solver convergence and the solution quality. A detailed inquiry into reduced precision requires a model configuration that is meaningful but cheaper to run and easier to evaluate than full atmosphere/ocean models. This study is therefore conducted in the context of a novel semi-implicit shallow-water model on the sphere, purposely designed to mimic numerical intricacies of modern all-scale weather and climate (W&C) models with the numerical stability independent on celerity of all wave motions. The governing algorithm of the shallow-water model is based on the non-oscillatory MPDATA methods for geophysical flows, whereas the resulting elliptic problem employs a strongly preconditioned non-symmetric Krylov-subspace solver GCR, proven in advanced atmospheric applications. The classical longitude/latitude grid is deliberately chosen to retain the stiffness of global W&C models posed in thin spherical shells as well as to better understand the performance of reduced-precision arithmetic in the vicinity of grid singularities. Precision reduction is done on a software level, using an emulator. The reduced-precision experiments are conducted for established dynamical-core test-cases, like the Rossby-Haurwitz wave number 4 and a zonal orographic flow. The study shows that selected key components of the elliptic solver, most prominently the preconditioning, can be performed at the level of half precision.
... The uncertainties concerning the SISL integration based on the ST method with regard to emerging and future HPC architectures is one of the main reasons for ECMWF and its European partners to look into alternative nonhydro- static, all-scale global model formulations and discretization schemes to be incorporated in the IFS. With this objective in mind, the Finite-Volume Module of the IFS (henceforth IFS-FVM) is under development at ECMWF ( Smolarkiewicz et al., 2016Smolarkiewicz et al., , 2017Kühnlein and Smolarkiewicz, 2017). An important property of the finite-volume (FV) method ap- plied in IFS-FVM is a compact spatial discretization stencil in "grid-point" space, associated with a distributed-memory communication footprint that is predominantly local and per- formed using thin overlap regions with the nearest neigh- bours, in contrast to the non-local high-volume communi- cations required in IFS-ST. ...
... In both formulations, accu- rate and robust integration with large time steps is achieved by 3-D implicit representation of fast acoustic and buoy- ant modes supported by the fully compressible equations. Furthermore, fully implicit representation of slow rotational modes is another common feature of both IFS-FVM and IFS-ST (Temperton, 2011;Smolarkiewicz et al., 2016). Although implicit time stepping is predominantly associated with com- putational stability, there are indications for favourable bal- ance and accuracy in multi-scale flows ( Knoll et al., 2003;Dörnbrack et al., 2005;Wedi and Smolarkiewicz, 2006). ...
... IFS-FVM solves the deep-atmosphere 3 , nonhydrostatic, fully compressible equations with a generalized height-based terrain-following vertical coordinate. Numerical integration of the governing equations employs a centred two-time-level semi-implicit scheme that provides unconditional stability in 3-D with respect to the fast acoustic and buoyant modes, as well as slower rotational modes ( Smolarkiewicz et al., 2014Smolarkiewicz et al., , 2016). In Sect. ...
We present a nonhydrostatic finite-volume global atmospheric model formulation for numerical weather prediction with the Integrated Forecasting System (IFS) at ECMWF and compare it to the established operational spectral-transform formulation. The novel Finite-Volume Module of the IFS (henceforth IFS-FVM) integrates the fully compressible equations using semi-implicit time stepping and non-oscillatory forward-in-time (NFT) Eulerian advection, whereas the spectral-transform IFS solves the hydrostatic primitive equations (optionally the fully compressible equations) using a semi-implicit semi-Lagrangian scheme. The IFS-FVM complements the spectral-transform counterpart by means of the finite-volume discretization with a local low-volume communication footprint, fully conservative and monotone advective transport, all-scale deep-atmosphere fully compressible equations in a generalized height-based vertical coordinate, and flexible horizontal meshes. Nevertheless, both the finite-volume and spectral-transform formulations can share the same quasi-uniform horizontal grid with co-located arrangement of variables, geospherical longitude-latitude coordinates, and physics parameterizations, thereby facilitating their comparison, coexistence, and combination in the IFS.
We highlight the advanced semi-implicit NFT finite-volume integration of the fully compressible equations of IFS-FVM considering comprehensive moist-precipitating dynamics with coupling to the IFS cloud parameterization by means of a generic interface. These developments - including a new horizontal-vertical split NFT MPDATA advective transport scheme, variable time stepping, effective preconditioning of the elliptic Helmholtz solver in the semi-implicit scheme, and a computationally efficient implementation of the median-dual finite-volume approach - provide a basis for the efficacy of IFS-FVM and its application in global numerical weather prediction. Here, numerical experiments focus on relevant dry and moist-precipitating baroclinic instability at various resolutions. We show that the presented semi-implicit NFT finite-volume integration scheme on co-located meshes of IFS-FVM can provide highly competitive solution quality and computational performance to the proven semi-implicit semi-Lagrangian integration scheme of the spectral-transform IFS.
... In both formulations, accurate and robust integration with large time steps is achieved by 3D implicit rep-25 resentation of fast acoustic and buoyant modes supported by the fully compressible equations. Furthermore, fully implicit representation of slow rotational modes is another common feature of both IFS-FVM and IFS-ST (Temperton, 2011;Smolarkiewicz et al., 2016). Although implicit time stepping is predominantly associated with the computational stability, there are indications for favourable balance and accuracy in multi-scale flows (Knoll et al., 2003;Dörnbrack et al., 2005;Wedi and Smolarkiewicz, 2006). ...
... Building on the earlier works by Smolarkiewicz et al. (2014Smolarkiewicz et al. ( , 2016, the two-time-level numerical integrators of IFS-FVM 10 for (6) can be subsumed in the following template scheme ...
... which straightforwardly returns the updated density ρ n+1 d i . The O(δt 2 ) estimate for the advector (vG) n+1/2 in (10) is implemented here by linear extrapolation of the advective velocities from the previous time levels t n−1 and t n ; see Appendix A port of all other scalar variables (see Table 2), a common approach to enable mass-compatible and monotonic solutions; see Smolarkiewicz et al. (2016) and Kühnlein and Smolarkiewicz (2017) for discussion in the context of IFS-FVM. ...
We present a nonhydrostatic finite-volume global atmospheric model formulation for numerical weather prediction with the Integrated Forecasting System (IFS) at ECMWF, and compare it to the established operational spectral-transform formulation. The novel Finite-Volume Module of IFS (henceforth IFS-FVM) integrates the fully compressible equations using semi-implicit time stepping and non-oscillatory forward-in-time (NFT) Eulerian advection, whereas the spectral-transform IFS solves the hydrostatic primitive equations (optionally the fully compressible equations) using a semi-implicit semi-Lagrangian scheme. The IFS-FVM complements the spectral-transform counterpart by means of the finite-volume discretisation with a local communication footprint, fully conservative and monotone advective transport, all-scale deep-atmosphere fully compressible equations in a generalised height-based vertical coordinate, applicable on flexible meshes. Nevertheless, both the finite-volume and spectral-transform formulations can share the same quasi-uniform horizontal grid with co-located arrangement of variables, geospherical longitude-latitude coordinates, and physical parametrisations, thereby facilitating their comparison, coexistence and combination in IFS.
We highlight the advanced semi-implicit NFT finite-volume integration of the fully compressible equations of the novel IFS-FVM considering comprehensive moist-precipitating dynamics with coupling to the IFS cloud parametrisation by means of a generic interface. These developments – including a new horizontal-vertical split NFT MPDATA advective transport scheme, variable time stepping, effective preconditioning of the elliptic Helmholtz solver in the semi-implicit scheme, and a computationally efficient coding implementation – provide a basis for the efficacy of IFS-FVM and its application in global numerical weather prediction. Here, numerical experiments focus on relevant dry and moist-precipitating baroclinic instability at various resolutions. We show that the presented semi-implicit NFT finite-volume integration scheme on co-located meshes of IFS-FVM can provide highly competitive solution quality and computational performance to the proven semi-implicit semi-Lagrangian integration scheme of the spectral-transform IFS.
... We shall return to this point while concluding the paper. All proposed theoretical/numerical developments directly pertain to two advanced all-scale modelling systems, the Eulerian-Lagrangian (EULAG) research model for multiscale flows [20,30,31] and the Finite-Volume Module (IFS-FVM) [32,11,33] in the Integrated Forecasting System (IFS) of the European Centre for Medium-Range Weather Forecasts (ECMWF). ...
... The generalised PDEs of EULAG [31] and IFS-FVM [32] assume the compressible Euler/Navier-Stokes equations under gravity on a rotating sphere as default, but include reduced soundproof equations [15,5] as options. From the perspective of numerics, the design of the semi-implicit integrators in FVM follows the same path for the compressible and the soundproof system. ...
... Importantly, substituting (9) in (7b) and manipulating the terms-such as to separate the gravitational, inertial and dissipative forcings on the rhs-and then moving the first term on the rhs of (7b) to the lhs, using (6a) and moving du a /dt back to D(u a ) + θ θ a d a u a dt . (10) The latter form is already familiar from [31,32]-cf. eqs. ...
This paper presents a generalised perturbation form of the nonhydrostatic partial differential equations (PDEs) that govern dynamics of all-scale global atmospheric flows. There can be many alternative perturbation forms for any given system of the governing PDEs, depending on the assumed ambient state about which perturbations are taken and on subject preferences in the numerical model design. All such forms are mathematically equivalent, yet they have different implications for the design and accuracy of effective semi-implicit numerical integrators of the governing PDEs. Practical and relevant arguments are presented in favour of perturbation forms that maximise the degree of implicitness of the associated integrators. Two optional forms are implemented in the high-performance finite-volume module (IFS-FVM) for simulating global all-scale atmospheric flows Smolarkiewicz et al. (2016) [32]. Their relative performance is verified with a class of ambient states of reduced complexity. A series of numerical simulations of the planetary baroclinic instability assumes geostrophically balanced zonally uniform ambient flows with significant meridional and vertical shear and exemplifies accuracy gains enabled by the generalised perturbation approach. The newly developed semi-implicit integrators show the potential for numerically accurate separation of a background state from finite-amplitude perturbations of the global atmosphere and provide a base for further development.
... Simulations of high Reynolds number flows with the nonoscillatory MPDATA revealed the implicit large-eddy-simulation (ILES) property [20], subsequently studied in depth [21][22][23][24][25][26] and verified in diverse geo-and astrophysical applications [27][28][29][30][31][32]. 1 Recent advances comprise soundproof-time-step semi-implicit integration schemes for the compressible Euler equations of all-scale atmospheric dynamics based on the FD [34] and the FV [16] MPDATA formulations. The FD-MPDATA is the basis of the EULAG model [28,35] as well as an open-source free/libre solver library libmpdata++ 2 [7], while the FV-MPDATA is employed in the Finite-Volume Module (FVM) for global all-scale atmospheric flows [36,37]. ...
... To substantiate the significance of the new development beyond the passive tracer advection, we start with a synopsis of the MPDATA-based flow solvers, widely documented in the literature [28,35,34,36]. ...
... This may be virtually impossible in a paradigm of essentially two-time-level integrators. Moreover, the requirements such as the solution monotonicity [17], compatibility of scalar conservation laws with their Lagrangian forms [44,36,37], or compatibility of elliptic Poisson/Helmholtz operators with advection [34] may take precedence over the formal accuracy, for the sake of physical realisability and efficacy in complex simulations. Nevertheless, the increased accuracy of the homogeneous algorithm A i can benefit the overall accuracy of integrations, as evidenced by the subsequent examples. ...
... The Finite-Volume Module (FVM) of the Integrated Forecasting System (IFS) is currently under development at ECMWF ( Smolarkiewicz et al., 2016;Kühnlein and Smolarkiewicz, 2017;Smolarkiewicz et al., 2017). FVM solves the non-hydrostatic Euler equations on an octahedral reduced Gaussian grid (Sect. ...
... FVM solves the non-hydrostatic Euler equations on an octahedral reduced Gaussian grid (Sect. 3.6) with a height-based terrainfollowing vertical coordinate ( Szmelter and Smolarkiewicz, 2010;Smolarkiewicz et al., 2016). The horizontal spatial discretization uses the median-dual finite-volume approach, combined with a structured-grid finite-difference method in the vertical. ...
... As with the classical reduced Gaussian grid of Hortal and Simmons (1991), the octahedral reduced Gaussian grid ( Malardel et al., 2016;Smolarkiewicz et al., 2016) specifies the latitudes according to the roots of the Legendre polynomials. The two grids differ in the arrangement of the points along the latitudes, which follows a simple rule for the octahedral grid: starting with 20 points on the first latitude around the poles, 4 points are added with every latitude towards the Equator, whereby the spacing between points along the latitudes is uniform and there are no points at the Equator. ...
Atmospheric dynamical cores are a fundamental component of global atmospheric modeling systems and are responsible for capturing the dynamical behavior of the Earth's atmosphere via numerical integration of the Navier–Stokes equations. These systems have existed in one form or another for over half of a century, with the earliest discretizations having now evolved into a complex ecosystem of algorithms and computational strategies. In essence, no two dynamical cores are alike, and their individual successes suggest that no perfect model exists. To better understand modern dynamical cores, this paper aims to provide a comprehensive review of 11 non-hydrostatic dynamical cores, drawn from modeling centers and groups that participated in the 2016 Dynamical Core Model Intercomparison Project (DCMIP) workshop and summer school. This review includes a choice of model grid, variable placement, vertical coordinate, prognostic equations, temporal discretization, and the diffusion, stabilization, filters, and fixers employed by each system.
... • Support different spatial discretisation strategies, such as the spectral transform approach currently used at ECMWF [7], compactstencil finite volume methods [16][17][18], discontinuous spectral element methods [19][20][21] and possibly others, to solve the set of PDEs forming the dynamical core. ...
... The FVM of the IFS is developed as an alternative dynamical core module at ECMWF to supplement the spectral formulation currently employed in operational forecasting [16] and is built using Atlas. FVM integrates the compressible Euler equations in a geospherical framework [16,40]. ...
... The FVM of the IFS is developed as an alternative dynamical core module at ECMWF to supplement the spectral formulation currently employed in operational forecasting [16] and is built using Atlas. FVM integrates the compressible Euler equations in a geospherical framework [16,40]. The horizontal spatial discretisation is fully unstructured using the median-dual finite volume approach. ...
The algorithms underlying numerical weather prediction (NWP) and climate models that have been developed in the past few decades face an increasing challenge caused by the paradigm shift imposed by hardware vendors towards more energy-efficient devices. In order to provide a sustainable path to exascale High Performance Computing (HPC), applications become increasingly restricted by energy consumption. As a result, the emerging diverse and complex hardware solutions have a large impact on the programming models traditionally used in NWP software, triggering a rethink of design choices for future massively parallel software frameworks. In this paper, we present Atlas , a new software library that is currently being developed at the European Centre for Medium-Range Weather Forecasts (ECMWF), with the scope of handling data structures required for NWP applications in a flexible and massively parallel way. Atlas provides a versatile framework for the future development of efficient NWP and climate applications on emerging HPC architectures. The applications range from full Earth system models, to specific tools required for post-processing weather forecast products. The Atlas library thus constitutes a step towards affordable exascale high-performance simulations by providing the necessary abstractions that facilitate the application in heterogeneous HPC environments by promoting the co-design of NWP algorithms with the underlying hardware.
... Kühnlein and Smolarkiewicz, 2017;Smolarkiewicz et al., 2017). FVM solves the compressible Euler equations in a geospherical framework (Szmelter and Smolarkiewicz, 2010;Smolarkiewicz et al., 2016). A centered twotime-level semi-implicit integration scheme is employed with 3D implicit treatment of acoustic, buoyant, and rotational modes . ...
... As with the classical reduced Gaussian grid of Hortal and Simmons (1991), the octahedral reduced Gaussian grid (Malardel et al., 2016;Smolarkiewicz et al., 2016) specifies the latitudes according to the roots of the Legendre polynomials. The two 5 grids differ in the arrangement of the points along the latitudes, which follows a simple rule for the octahedral grid: starting with 20 points on the first latitude around the poles, four points are added with every latitude towards the equator, whereby the spacing between points along the latitudes is uniform and there are no points at the equator. ...
... Therefore, the model time step is identical for all processes and typically selected with regard to the stability of the advective transport scheme-i.e. the time step is continuously adapted according to a given maximum advective Courant number permitted by the MPDATA scheme. A comprehensive discussion of the integration scheme can be found in Smolarkiewicz et al. (2014Smolarkiewicz et al. ( , 2016 and Kühnlein and Smolarkiewicz (2017) for dry dynamics, whereas 20 in Kurowski et al. (2014) and Smolarkiewicz et al. (2017) for extension to moist-precipitating dynamics. Here, we provide a short outline of the solution procedure for the compressible Euler equations (10). ...
Atmospheric dynamical cores are a fundamental component of global atmospheric modeling systems, and are responsible for capturing the dynamical behavior of the Earth's atmosphere via numerical integration of the Naviér–Stokes equations. These systems have existed in one form or another for over half of a century, with the earliest discretizations having now evolved into a complex ecosystem of algorithms and computational strategies. In essence, no two dynamical cores are alike, and their individual successes suggest that no perfect model exists. To better understand modern dynamical cores, this paper aims to provide a comprehensive review of eleven dynamical cores, drawn from modeling centers and groups that participated in the 2016 Dynamical Core Model Intercomparison Project (DCMIP) workshop and summer school. This review includes choice of model grid, variable placement, vertical coordinate, prognostic equations, temporal discretization, and the diffusion, stabilization, filters and fixers employed by each system.
... Moreover, the advancement facilitates multiple error-compensative iterations of the finite-volume MPDATA. The presented scheme is the basis of a novel finite-volume module (FVM) for global all-scale atmospheric flows [33]. ...
... Here we present the discrete implementation of the advanced FV-MPDATA following our modified equation analysis of the previous section. We consider a hybrid computational mesh, unstructured in the horizontal and structured in the vertical, which is of particular relevance to global atmospheric modelling [33]. While the unstructured horizontal mesh is an effective way to achieve quasi-uniform resolution over the surface of the sphere and optionally local mesh adaptivity, the structured grid benefits direct preconditioning of elliptic operators in the stiff vertical direction, important for efficient integration of the governing equations with implicit time stepping. ...
... The numerical integral of the homogeneous generalised transport equation (6) with R Ψ ≡ 0 can be written in the functional form as [33] Ψ n+1 ...
An advancement of the unstructured-mesh finite-volume MPDATA (Multidimensional Positive Definite Advection Transport Algorithm) is presented that formulates the error compensative pseudo-velocity of the scheme to rely only on face-normal advective fluxes to the dual cells, in contrast to the full vector employed in previous implementations. This is essentially achieved by expressing the temporal truncation error underlying the pseudo velocity in a form consistent with the flux-divergence of the governing conservation law. The development is especially important for integrating fluid dynamics equations on non rectilinear meshes whenever face-normal advective mass fluxes are employed for transport compatible with mass continuity the latter being essential for flux-form schemes. In particular, the proposed formulation enables large-time-step semi-implicit finite-volume integration of the compressible Euler equations using MPDATA on arbitrary hybrid computational meshes. Furthermore, it facilitates multiple error-compensative iterations of the finite-volume MPDATA and improved overall accuracy. The advancement combines straightforwardly with earlier developments, such as the nonoscillatory option, the infinite gauge variant, and moving curvilinear meshes. A comprehensive description of the scheme is provided for a hybrid horizontally-unstructured vertically-structured computational mesh for efficient global atmospheric flow modelling. The proposed finite-volume MPDATA is verified using selected 3D global atmospheric benchmark simulations, representative of hydrostatic and non-hydrostatic flow regimes. Besides the added capabilities, the scheme retains fully the efficacy of established finite-volume MPDATA formulations.
... Moreover, the advancement facilitates multiple error-compensative iterations of the finite-volume MPDATA. The presented scheme is the basis of a novel finite-volume module (FVM) for global all-scale atmospheric flows [33]. ...
... Here we present the discrete implementation of the advanced FV-MPDATA following our modified equa- tion analysis of the previous section. We consider a hybrid computational mesh, unstructured in the hori- zontal and structured in the vertical, which is of particular relevance to global atmospheric modelling [33]. While the unstructured horizontal mesh is an effective way to achieve quasi-uniform resolution over the sur- face of the sphere and optionally local mesh adaptivity, the structured grid benefits direct preconditioning of elliptic operators in the stiff vertical direction, important for efficient integration of the governing equations with implicit time stepping. ...
... The numerical integral of the homogeneous generalised transport equation (6) with R Ψ ≡ 0 can be written in the functional form as [33] ...
An advancement of the unstructured-mesh finite-volume MPDATA (Multidimensional Positive Definite Advection Transport Algorithm) is presented that formulates the error-compensative pseudo-velocity of the scheme to rely only on face-normal advective fluxes to the dual cells, in contrast to the full vector employed in previous implementations. This is essentially achieved by expressing the temporal truncation error underlying the pseudo-velocity in a form consistent with the flux-divergence of the governing conservation law. The development is especially important for integrating fluid dynamics equations on non-rectilinear meshes whenever face-normal advective mass fluxes are employed for transport compatible with mass continuity—the latter being essential for flux-form schemes. In particular, the proposed formulation enables large-time-step semi-implicit finite-volume integration of the compressible Euler equations using MPDATA on arbitrary hybrid computational meshes. Furthermore, it facilitates multiple error-compensative iterations of the finite-volume MPDATA and improved overall accuracy. The advancement combines straightforwardly with earlier developments, such as the nonoscillatory option, the infinite-gauge variant, and moving curvilinear meshes. A comprehensive description of the scheme is provided for a hybrid horizontally-unstructured vertically-structured computational mesh for efficient global atmospheric flow modelling. The proposed finite-volume MPDATA is verified using selected 3D global atmospheric benchmark simulations, representative of hydrostatic and non-hydrostatic flow regimes. Besides the added capabilities, the scheme retains fully the efficacy of established finite-volume MPDATA formulations.
... In order for NWP applications to optimally exploit future computer hardware emerging over the next 20-30 years, a flexible and dynamic data structure framework, named Atlas is being developed to serve as a foundation for a wide variety of applications, ranging from the use within European NWP models and for the development of alternative dynamical core modules [4], to development of applications responsible for pre-and post-processing of exponentially growing output data. It is imperative for Atlas to remain flexible and maintainable since the development of NWP models typically takes a decade, and NWP models typically last much longer than that. ...
... As mentioned in Section 1, the NWP and climate models using the spectral transform method require global communication of large amounts of data. Realising the performance limitations of spectral transform methods at scale, a three-dimensional hybrid finite difference -finite volume module is developed [4], which only relies on nearest neighbor communication patterns as illustrated in Figure 4. In the horizontal direction an unstructured median-dual edge-based finite volume method is used [7], while the vertical direction is discretized with a structured finite-difference method preferred with the vertically shallow nature of the atmosphere compared to the radius of the earth. ...
... The simulation is three-dimensional and uses compressible non-hydrostatic dynamics. For a full description of the methodology, the reader is referred to [4]. The hybrid finite volume module is expected to perform well at extremescale due to local nearest neighbor communication patterns. ...
Numerical Weather Prediction (NWP) and climate simulations have been intimately connected with progress in supercomputing since the first numerical forecast was made about 65 years ago. The biggest challenge to state-of-the-art computational NWP arises today from its own software productivity shortfall. The application software at the heart of most NWP services is ill-equipped to efficiently adapt to the rapidly evolving heterogeneous hardware provided by the supercomput-ing industry. If this challenge is not addressed it will have dramatic negative consequences for weather and climate prediction and associated services. This article introduces Atlas, a flexible data structure framework developed at the European Centre for Medium-Range Weather Forecasts (ECMWF) to facilitate a variety of numerical discretisation schemes on heterogeneous architectures, as a necessary step towards affordable ex-ascale high-performance simulations of weather and climate. A newly developed hybrid MPI-OpenMP finite volume module built upon Atlas serves as a first demonstration of the parallel performance that can be achieved using Atlas' initial capabilities.
... In the example of a numerical model, Borchert et al. (2019) introduced an upper-atmosphere extension to the ICOsahedral Non-hydrostatic (ICON) model (Zängl et al. 2015) that includes the full Coriolis term. Another deep-atmosphere numerical model with full Coriolis support has been published by Smolarkiewicz et al. (2016). From here on, we use the term 'non-traditional setting' to refer to a setting where the full effect of the Coriolis force is considered. ...
... One would encounter this unstable mode in numerical simulations if there is energy flux from the atmospheric boundary layer. This effect would be especially significant for the numerical simulations of the deep atmosphere, e.g. the papers by Borchert et al. (2019) and Smolarkiewicz et al. (2016) as mentioned in the introduction, or in tropical dynamics with the non-traditional setting. Therefore, the characterisation of the unstable mode presented here is a worthwhile pursuit that may broaden our understanding of the simulation results from these numerical models. ...
The traditional approximation neglects the cosine components of the Coriolis acceleration, and this approximation has been widely used in the study of geophysical phenomena. However, the justification of the traditional approximation is questionable under a few circumstances. In particular, dynamics with substantial vertical velocities or geophysical phenomena in the tropics have non-negligible cosine Coriolis terms. Such cases warrant investigations with the non-traditional setting, i.e. the full Coriolis acceleration. In this manuscript, we study the effect of the non-traditional setting on an isothermal, hydrostatic and compressible atmosphere assuming a meridionally homogeneous flow. Employing linear stability analysis, we show that, given appropriate boundary conditions, i.e. a bottom boundary condition that allows for a vertical energy flux and non-reflecting boundary at the top, the atmosphere at rest becomes prone to a novel unstable mode. The validity of assuming a meridionally homogeneous flow is investigated via scale analysis. Numerical experiments were conducted, and Rayleigh damping was used as a numerical approximation for the non-reflecting top boundary. Our three main results are as follows: (i) experiments involving the full Coriolis terms exhibit an exponentially growing instability, yet experiments subjected to the traditional approximation remain stable; (ii) the experimental instability growth rate is close to the theoretical value; (iii) a perturbed version of the unstable mode arises even under sub-optimal bottom boundary conditions. Finally, we conclude our study by discussing the limitations, implications and remaining open questions. Specifically, the influence on numerical deep-atmosphere models and possible physical interpretations of the unstable mode are discussed.
... The concrete implementation of the Method concept uses the FunctionSpace and Field classes, both required to generate a concrete numerical method. Atlas currently provides a fvm::Method class, which contains everything required to construct mathematical operators using an edge-based finite volume scheme [15]. A concrete fvm::Nabla operator then implements the actual numerical algorithm using the fvm::Method. ...
... Left: general design of numerical operators. Right: Derivative, divergence, curl, and Laplacian implemented in the Nabla vector operator specific for a finite volume Method[15]. ...
This document is one of the deliverable reports created for the ESCAPE project. ESCAPE stands for Energy-efficient Scalable Algorithms for Weather Prediction at Exascale. The project develops world-class, extreme-scale computing capabilities for European operational numerical weather prediction and future climate models. This is done by identifying Weather & Climate dwarfs which are key patterns in terms of computation and communication (in the spirit of the Berkeley dwarfs). These dwarfs are then optimised for different hardware architectures (single and multi-node) and alternative algorithms are explored. Performance portability is addressed through the use of domain specific languages. In this deliverable report, we present Atlas, a new software library that is currently being developed at the European Centre for Medium-Range Weather Forecasts (ECMWF), with the scope of handling data structures required for NWP applications in a flexible and massively parallel way. Atlas provides a versatile framework for the future development of efficient NWP and climate applications on emerging HPC architectures. The applications range from full Earth system models, to specific tools required for post-processing weather forecast products. Atlas provides data structures for building various numerical strategies to solve equations on the sphere or limited area's on the sphere. These data structures may contain a distribution of points (grid) and, possibly, a composition of elements (mesh), required to implement the numerical operations required. Atlas can also represent a given field within a specific spatial projection. Atlas is capable of mapping fields between different grids as part of pre- and post-processing stages or as part of coupling processes whose respective fields are discretised on different grids or meshes.
... Yet, because of the complexity of the grid configuration for the spherical geometry [32,33], and the computational expense they imply, GCRMs require considerably more development, both in terms of their formulation and computational implementation, than would be implied by simply increasing the domain of pre-existing models. For this reason, and due to a lack of access to Tier-0 computational resources, GCRMs remain the remit of a rather limited number of groups, which include developments around: the Nonhydrostatic Icosahedral Atmospheric Model (NICAM) [1,34,35] in Japan, ICOsahedral Nonhydrostatic (ICON) [36,37] in Germany, the Model for Prediction Across Scales (MPAS) [38], Finite-Volume Dynamical Core on the Cubed Sphere (FV3) [39], the Goddard Earth Observing System Model, Version 5 (GEOS-5) [40], the global version of the System for Atmospheric Modeling (Global SAM) [41] in the USA, and the Integrated Forecast System (IFS) by the European Centre for Medium-Range Weather Forecasts (ECMWF) which is available as a spectral (IFS-ST [42]) and finite-volume model (IFS-FVM [43,44]). Before the first GCRMs listed above, there was considerable effort toward GCRMs, as described in earlier reviews, e.g., [34,45,46]. ...
... IFS-FVM is developed with approaches more similar to the other GCRMs in that it solves the compressible equations in physical space [43,139,140], on an A-Grid similar to NICAM. The equations are solved using semi-implicit time stepping. ...
Purpose of Review
Global cloud-resolving models (GCRMs) are a new type of atmospheric model which resolve nonhydrostatic accelerations globally with kilometer-scale resolution. This review explains what distinguishes GCRMs from other types of models, the problems they solve, and the questions their more commonplace use is raising.
Recent Findings
GCRMs require high-resolution discretization over the sphere but can differ in many other respects. They are beginning to be used as a main stream research tool. The first GCRM intercomparison studies are being coordinated, raising new questions as to how best to exploit their advantages.
Summary
GCRMs are designed to resolve the multiscale nature of moist convection in the global dynamics context, without using cumulus parameterization. Clouds are simulated with cloud microphysical schemes in ways more comparable to observations. Because they do not suffer from ambiguity arising from cumulus parameterization, as computational resources increase, GCRMs are the promise of a new generation of global weather and climate simulations.
... The time integration algorithm in FVM is built upon extensive earlier experience with the EULAG model and the MPDATA advection scheme. Through appropriate correction of a first-order upwind discretization, a system is constructed that encompasses transport and implicit dynamics in an elegant theoretical framework [37,42] and references therein. The approach, which in its default configuration relies on time extrapolation of advecting velocities and subtraction of reference states, also contains soundproof analytical systems as subcases and has shown excellent performance in integrating atmospheric flows at all scales without instabilities. ...
... Note that the implicit trapezoidal step (20) and, to a lesser extent the treatment of the P in (17), (18b), and (20d), closely resemble the EULAG/FVM forward-in-time discretization from [40,33,42,37,27]. ...
We introduce a second-order numerical scheme for compressible atmospheric motions at small to planetary scales. The collocated finite volume method treats the advection of mass, momentum, and mass-weighted potential temperature in conservation form while relying on Exner pressure for the pressure gradient term. It discretises the rotating compressible equations by evolving full variables rather than perturbations around a background state, and operates with time steps constrained by the advection speed only. Perturbation variables are only used as auxiliary quantities in the formulation of the elliptic problem. Borrowing ideas on forward-in-time differencing, the algorithm reframes the authors' previously proposed schemes into a sequence of implicit midpoint, advection, and implicit trapezoidal steps that allows for a time integration unconstrained by the internal gravity wave speed. Compared with existing approaches, results on a range of benchmarks of nonhydrostatic- and hydrostatic-scale dynamics are competitive. The test suite includes a new planetary-scale inertia-gravity wave test highlighting the properties of the scheme and its large time step capabilities. In the hydrostatic-scale cases the model is run in pseudo-incompressible and hydrostatic mode with simple switching within a uniform discretization framework. The differences with the compressible runs return expected relative magnitudes. By providing seamless access to soundproof and hydrostatic dynamics, the developments represent a necessary step towards an all-scale blended multimodel solver.
... The dwarf originates from the elliptic solver used in the semi-implicit time integration of IFS-FVM (Smolarkiewicz et al., 2016;Kühnlein et al., 2018). We employ the Generalised Conjugate Residual (GCR, Eisenstat et al., 1983) approach to solve 20 the following linear elliptic problem ...
... where ρ represents the fluid density. MPDATA schemes are at the basis of the Finite-Volume Module of IFS (henceforth IFS-FVM), a novel dynamical core formulation under development at ECMWF (Smolarkiewicz et al., 2016;Kühnlein et al., 2018). ...
In the simulation of complex multi-scale flow problems, such as those arising in weather and climate modelling, one of the biggest challenges is to satisfy operational requirements in terms of time-to-solution and energy-to-solution yet without compromising the accuracy and stability of the calculation. These competing factors require the development of state-of-the-art algorithms that can optimally exploit the targeted underlying hardware and efficiently deliver the extreme computational capabilities typically required in operational forecast production. These algorithms should (i) minimise the energy footprint along with the time required to produce a solution, (ii) maintain a satisfying level of accuracy, (iii) be numerically stable and resilient, in case of hardware or software failure.
The European Centre for Medium Range Weather Forecasts (ECMWF) is leading a project called ESCAPE (Energy-efficient SCalable Algorithms for weather Prediction on Exascale supercomputers) which is funded by Horizon 2020 (H2020) under initiative Future and Emerging Technologies in High Performance Computing (FET-HPC). The goal of the ESCAPE project is to develop a sustainable strategy to evolve weather and climate prediction models to next-generation computing technologies. The project partners incorporate the expertise of leading European regional forecasting consortia, university research, experienced high-performance computing centres and hardware vendors.
This paper presents an overview of results obtained in the ESCAPE project in which weather prediction have been broken down into smaller building blocks called dwarfs. The participating weather prediction models are: IFS (Integrated Forecasting System), ALARO – a combination of AROME (Application de la Recherche à l'Opérationnel a Meso-Echelle) and ALADIN (Aire Limitée Adaptation Dynamique Développement International) and COSMO-EULAG – a combination of COSMO (Consortium for Small-scale Modeling) and EULAG (Eulerian/semi-Lagrangian fluid solver). The dwarfs are analysed and optimised in terms of computing performance for different hardware architectures (mainly Intel Skylake CPUs, NVIDIA GPUs, Intel Xeon Phi). The ESCAPE project includes the development of new algorithms that are specifically designed for better energy efficiency and improved portability through domain specific languages. In addition, the modularity of the algorithmic framework, naturally allows testing different existing numerical approaches, and their interplay with the emerging heterogeneous hardware landscape. Throughout the paper, we will compare different numerical techniques to solve the main building blocks that constitute weather models, in terms of energy efficiency and performance, on a variety of computing technologies.
... Such high resolutions are still computationally unaffordable and too inefficient to meet demands of the limited time window for distributing global forecasts to the end users. The recent work [139] shows how the ability of simulating global all-scale atmospheric dynamics on unstructured meshes can complement established NWP models and aid the progress-this motivates the technical content of this paper. ...
... In contrast to the preceding sections that addressed nonhydrostatic performance of the NFTFV approach in local-area simulations, here we illustrate its capability for simulating essentially hydrostatic modes of global circulation. The illustration employs a high performance finite-volume module (FVM) [139], with NFTFV discretisation hybridising unstructured-mesh in the horizontal with the structured-grid in the vertical. The governing equations are formulated in the classical meteorological latitude-longitude spherical framework [109,150,138] frequently criticised for its notorious stiffness in the polar regions. ...
AIAA 2018-3497 Paper
The paper examines recent advancements in the class of Nonoscillatory Forward-in-Time
(NFT) schemes that exploit the implicit LES (ILES) properties of Multidimensional Positive
Definite Advection Transport Algorithm (MPDATA). The reported developments address both
global and limited area models spanning a range of atmospheric flows, from the hydrostatic
regime at planetary scale, down to mesoscale and microscale where flows are inherently non-
hydrostatic. All models operate on fully unstructured (and hybrid) meshes and utilize a median
dual mesh finite volume discretisation. High performance computations for global flows employ
a bespoke hybrid MPI-OpenMP approach and utilise the ATLAS library. Simulations across
scales—from a global baroclinic instability epitomising evolution of weather systems down
to stratified orographic flows rich in turbulent phenomena due to gravity-wave breaking in
dispersive media, verify the computational advancements and demonstrate the efficacy of
ILES both in regularizing large scale flows at the scale of the mesh resolution and taking a role
of a subgrid-scale turbulence model in simulation of turbulent flows in the LES regime.
... The IFS model takes one approach to the discretization of prognostic partial differential equations and their evolution in time defined on a grid. However, alternative discretizations are also realised with a finite-volume approach, which added dwarfs related to the advection scheme (Kühnlein and Smolarkiewicz, 2017) and a 3D grid-point based elliptic solver (Smolarkiewicz et al., 2016), explored with both structured and unstructured grids. Two further dwarfs have also been chosen as representatives 17 ...
... Some of these other approaches rely on nearest-neighbour communication patterns, and may even be applied to unstructured grids, leading to the requirement for different data structures to get maximum performance; potentially with different variants depending on the target hardware! The Atlas library (Deconinck et al., 2016Deconinck et al., , 2017) is an attempt to address these issues. It provides a general framework for representing model data structures on different grids and meshes relevant to NWP and climate simulation. ...
Weather and climate models are complex pieces of software which include many individual components, each of which is evolving under the pressure to exploit advances in computing to enhance some combination of a range of possible improvements (higher spatio/temporal resolution, increased fidelity in terms of resolved processes, more quantification of uncertainty etc). However, after many years of a relatively stable computing environment with little choice in processing architecture or programming paradigm (basically X86 processors using MPI for parallelism), the existing menu of processor choices includes significant diversity, and more is on the horizon. This computational diversity, coupled with ever increasing software complexity, leads to the very real possibility that weather and climate modelling will arrive at a chasm which will separate scientific aspiration from our ability to develop and/or rapidly adapt codes to the available hardware.
In this paper we review the hardware and software trends which are leading us towards this chasm, before describing current progress in addressing some of the tools which we may be able to use to bridge the chasm. This brief introduction to current tools and plans is followed by a discussion outlining the scientific requirements for quality model codes which have satisfactory performance and portability, while simultaneously supporting productive scientific evolution. We assert that the existing method of incremental model improvements employing small steps which adjust to the changing hardware environment is likely to be inadequate for crossing the chasm between aspiration and hardware at a satisfactory pace, in part because institutions cannot have all the relevant expertise in house. Instead, we outline a methodology based on large community efforts in engineering and standardisation, one which will depend on identifying a taxonomy of key activities – perhaps based on existing efforts to develop domain specific languages, identify common patterns in weather and climate codes, and develop community approaches to commonly needed tools, libraries etc – and then collaboratively building up those key components. Such a collaborative approach will depend on institutions, projects and individuals adopting new interdependencies and ways of working.
... Before closing this introduction about temporal discretization schemes, it is interesting to note the recent attention for the modified equation approach [131,102,103]. Characteristic for such a scheme is the increase in the order of accuracy by subtracting part of the truncation error from the original equation. ...
... A recent European example of the FV method in NWP can be found in [103]. ...
... Such high resolutions are still computationally unaffordable and too inefficient to meet demands of the limited time window for distributing global forecasts to the end users. The recent work [139] shows how the ability of simulating global all-scale atmospheric dynamics on unstructured meshes can complement established NWP models and aid the progress-this motivates the technical content of this paper. ...
... In contrast to the preceding sections that addressed nonhydrostatic performance of the NFTFV approach in local-area simulations, here we illustrate its capability for simulating essentially hydrostatic modes of global circulation. The illustration employs a high performance finite-volume module (FVM) [139], with NFTFV discretisation hybridising unstructured-mesh in the horizontal with the structured-grid in the vertical. The governing equations are formulated in the classical meteorological latitude-longitude spherical framework [109,150,138] frequently criticised for its notorious stiffness in the polar regions. ...
The advance of massively parallel computing in the nineteen nineties and beyond encouraged finer grid intervals in numerical weather-prediction models. This has improved resolution of weather systems and enhanced the accuracy of forecasts, while setting the trend for development of unified all-scale atmospheric models. This paper first outlines the historical background to a wide range of numerical methods advanced in the process. Next, the trend is illustrated with a technical review of a versatile nonoscillatory forward-in-time finite-volume (NFTFV) approach, proven effective in simulations of atmospheric flows from small-scale dynamics to global circulations and climate. The outlined approach exploits the synergy of two specific ingredients: the MPDATA methods for the simulation of fluid flows based on the sign-preserving properties of upstream differencing; and the flexible finite-volume median-dual unstructured-mesh discretisation of the spatial differential operators comprising PDEs of atmospheric dynamics. The paper consolidates the concepts leading to a family of generalised nonhydrostatic NFTFV flow solvers that include soundproof PDEs of incompressible Boussinesq, anelastic and pseudo-incompressible systems, common in large-eddy simulation of small- and meso-scale dynamics, as well as all-scale compressible Euler equations. Such a framework naturally extends predictive skills of large-eddy simulation to the global atmosphere, providing a bottom-up alternative to the reverse approach pursued in the weather-prediction models. Theoretical considerations are substantiated by calculations attesting to the versatility and efficacy of the NFTFV approach. Some prospective developments are also discussed.
... It still finds use in idealised oceanic box models as well as in some general circulation models. The finite-volume model developed at ECMWF is e.g. using the upwind scheme in its advection-transport algorithm making use of an iterative approach (Smolarkiewicz et al., 2016). When using an upwind scheme one should, however, realise that it is highly diffusive. ...
The purpose of this book is to provide an introduction to numerical modelling of the ocean and the atmosphere. It originates from courses given at Stockholm University and is intended to serve as a textbook for students in meteorology and oceanography with a background in mathematics and physics. Focus is on numerical schemes for the most commonly used equations in oceanography and meteorology as well as on the stability, precision and other properties of these schemes. Simple equations capturing the properties of the primitive equations employed in models of the ocean and atmosphere will be used. These model equations are solved numerically on a grid by discretisation, the derivatives of the differential equations being replaced by finite-difference approximations. The focus will be on the basic numerical methods used for oceanographic and atmospheric modelling. These models are based on the Navier-Stokes equations (including the Coriolis effect) and a tracer equation for heat in both the atmosphere and ocean and tracer equations for humidity and salt in the atmosphere and ocean, respectively. A coupled atmospheric and oceanic general circulation model represents the core part of an Earth System climate model. The book starts by presenting the most common types of partial differential equations and finite difference schemes used in meteorology and oceanography. Subsequently the limitations of these numerical schemes as regards stability, accuracy, presence of computational modes and accuracy the computationally determined phase speed are discussed. The shallow-water equations are discretised for different spatial grids and friction and diffusion terms are introduced. Hereafter implicit and semi-implicit schemes are discussed as well as the semi-Lagrangian technique. Coordinates for atmospheric as well as oceanic models are presented as well as a highly simplified 3D model. A brief description is given of how some atmospheric general circulation models use spectral methods as "horizontal coordinates". Finally, some "pen-and-paper" theoretical exercises and a number of GFD computer exercises are given.
... The Multidimensional Positive Definite Advection Transport Algorithm (MPDATA) is a 2D stencil over an unstructured grid 24 implemented using the scalable Atlas library for numerical weather prediction. 25 This code is used in production by the European Centre for Mid-range Weather Forecasting (ECMWF), where it is applied to a 6.5 million point unstructured grid. ...
As core counts in processors increases, it becomes harder to schedule and distribute work in a timely and scalable manner. This article enhances the scalability of parallel loop schedulers by specializing schedulers for fine‐grain loops. We propose a low‐overhead work distribution mechanism for a static scheduler that uses no atomic operations. We integrate our static scheduler with the Intel OpenMP and Cilkplus parallel task schedulers to build hybrid schedulers. Compiler support enables efficient reductions for Cilk, without changing the programming interface of Cilk reducers. Detailed, quantitative measurements demonstrate that our techniques achieve scalable performance on a 48‐core machine and the scheduling overhead is 43% lower than Intel OpenMP and 12.1× lower than Cilk. We demonstrate consistent performance improvements on a range of HPC and data analytics codes. Performance gains are more important as loops become finer‐grain and thread counts increase. We observe consistently 16%–30% speedup on 48 threads, with a peak of 2.8× speedup.
... While NFT codes operating on Cartesian meshes and exploiting the advantages of the distributed memory paradigm are well established, the successful implementations of NFT-FV codes for unstructured meshes on multi-node architectures are more recent. To date these efforts have focused on global models for the Earth's atmosphere with meshes that are quasi-uniform in the horizontal and prismatic in the vertical [45,20]. In this paper, we complement our earlier work with new NFT-FV model developments that provide capabilities for parallel simulations on fully unstructured, irregular meshes. ...
A numerical study of stably stratified flows past spheres at Reynolds numbers $Re=200$ and $Re=300$ is reported. In these flow regimes, a neutrally stratified laminar flow induces distinctly different near-wake features. However, the flow behaviour changes significantly as the stratification increases and suppresses the scale of vertical displacements of fluid parcels. Computations for a range of Froude numbers $Fr\in [0.1,\infty]$ show that as Froude number decreases, the flow patterns for both Reynolds numbers become similar. The representative simulations of the lee-wave instability at $Fr=0.625$ and the two-dimensional vortex shedding at $Fr=0.25$ regimes are illustrated for flows past single and tandem spheres, thereby providing further insight into the dynamics of stratified flows past bluff bodies. In particular, the reported study examines the relative influence of viscosity and stratification on the dividing streamline elevation, wake structure and flow separation. The solutions of the Navier-Stokes equations in the incompressible Boussinesq limit are obtained on unstructured meshes suitable for simulations involving multiple bodies. Computations are accomplished using the finite volume, non-oscillatory forward-in-time (NFT) Multidimensional Positive Definite Transport Algorithm (MPDATA) based solver. The impact and validity of the numerical approximations, especially for the cases exhibiting strong stratification, are also discussed. Qualitative and quantitative comparisons with available laboratory experiments and prior numerical studies confirm the validity of the numerical approach.
... We use an Eulerian, semi-implicit shallow-water model that is conceptionally similar to the Finite Volume Model (FVM-IFS) which is developed at the European Centre for Medium-Range Weather Forecasts as a new dynamical core for the Integrated Forecasting System (IFS; P. K. Smolarkiewicz et al. (2016Smolarkiewicz et al. ( , 2019; Kühnlein et al. (2019)). The shallow-water model is using the well-known MPDATA advection scheme (Prusa et al., 2008;Szmelter & Smolarkiewicz, 2010) and the shallow water equations on the sphere are discretised as defined in (P. ...
It is tested whether machine learning methods can be used for preconditioning to increase the performance of the linear solver -- the backbone of the semi-implicit, grid-point model approach for weather and climate models. Embedding the machine-learning method within the framework of a linear solver circumvents potential robustness issues that machine learning approaches are often criticized for, as the linear solver ensures that a sufficient, pre-set level of accuracy is reached. The approach does not require prior availability of a conventional preconditioner and is highly flexible regarding complexity and machine learning design choices. Several machine learning methods are used to learn the optimal preconditioner for a shallow-water model with semi-implicit timestepping that is conceptually similar to more complex atmosphere models. The machine-learning preconditioner is competitive with a conventional preconditioner and provides good results even if it is used outside of the dynamical range of the training dataset.
... By developing a hybrid and hierarchical methodology that continues to support the spectral transform technique, ECMWF has introduced added flexibility through the development of the IFS-FVM [97,65]. The IFS-FVM dynamical core incorporates finite-volume, semi-implicit integration methods for atmospheric dynamics that is suitable from planetary-to micro-scales. ...
The efficiency of the forecasting system on future high-performance computing and data handling systems is considered one of the key challenges for implementing ECMWF’s ambitious strategy. This was already recognised by ECMWF in 2013, and has led to the foundation of the Scalability Programme. The programme aims to address this challenge as a concerted action between the Centre and its Member States, but also draws in the computational science expertise available throughout Europe. This technical memorandum provides an overview of the status of the programme, highlights achievements from the first five years ranging from observational data pre-processing, data assimilation, forecast model design and output data post-processing, and defines the roadmap for the next five years towards a sustainable system that can operate on the expected range of hardware and software technologies. This point in time is crucial because the programme will have a strong focus on implementation and operational benefit in the next period.
... the MCore model by Ullrich and Jablonowski (2012), and the Finite Volume Model (FVM) of the Integrated Forecasting System (IFS) developed at the the European Centre for Medium-Range Weather Forecasts (ECMWF) (e.g., Smolarkiewicz et al., 2016). An overview of some of these models can be found in Ullrich et al. (2017). ...
How the upper-atmosphere branch of the circulation contributes to and interacts with the circulation of the middle and lower atmosphere is a research area with many open questions. Inertia-gravity waves, for instance, have moved in the focus of research as they are suspected to be key features in driving and shaping the circulation. Numerical atmospheric models are an important pillar for this research. We use the ICOsahedral Non-hydrostatic (ICON) general circulation model, which is a joint development of the Max Planck Institute for Meteorology (MPI-M) and the German Weather Service (DWD), and provides, e.g., local mass conservation, a flexible grid nesting option and a non-hydrostatic dynamical core formulated on an icosahedral-triangular grid. We extended ICON to the upper atmosphere and present here the two main components of this new configuration named UA-ICON: an extension of the dynamical core from shallow- to deep-atmosphere dynamics, and the implementation of an upper-atmosphere physics package. A series of test cases and climatological simulations show that UA-ICON performs satisfactorily and is in good agreement with the observed global atmospheric circulation.
... Using a half barrier in our scheduler reduces the scheduling delay further compared to a full barrier. Figure 2 shows the performance of our fine-grain scheduler on Multidimensional Positive Definite Advection Transport Algorithm (MPDATA) [3], from the European Centre for Mid-range Weather Forecasting, on a grid with 5568 points and 16399 edges. The speedup of MPDATA with OpenMP stagnates with increasing parallelism (Figure 2 (left)) whereas the fine-grain scheduler increases performance by up to 22% (Figure 2 (right)) over the off-the-shelf Intel OpenMP runtime. ...
This work proposes a low-overhead half-barrier pattern to schedule fine-grain parallel loops and considers its integration in the Intel OpenMP and Cilkplus schedulers. Experimental evaluation demonstrates that the scheduling overhead of our techniques is 43% lower than Intel OpenMP and 12.1x lower than Cilk. We observe 22% speedup on 48 threads, with a peak of 2.8x speedup.
... In [22] it is shown that SPP (poles excepted) is tough to match in solution quality and that only the SPP can exhibit perfect zonal symmetry. A novel global unstructured grid approach based upon a geospherical framework is given in [23,24]. This allows the grid parameterization to directly employ SPP over most of the globe with special treatment required only near the poles. ...
Coordinate singularities are sometimes encountered in computational problems. An important example involves global atmospheric models used for climate and weather prediction. Classical spherical coordinates can be used to parameterize the manifold – that is, generate a grid for the computational spherical shell domain. This particular parameterization offers significant benefits such as orthogonality and exact representation of curvature and connection (Christoffel) coefficients. But it also exhibits two polar singularities and at or near these points typical continuity/integral constraints on dependent fields and their derivatives are generally inadequate and lead to poor model performance and erroneous results. Other parameterizations have been developed that eliminate polar singularities, but problems of weaker singularities and enhanced grid noise compared to spherical coordinates (away from the poles) persist. In this study reparameterization invariance of geometric objects (scalars, vectors and the forms generated by their covariant derivatives) is utilized to generate asymptotic forms for dependent fields of interest valid in the neighborhood of a pole. The central concept is that such objects cannot be altered by the metric structure of a parameterization. The new boundary conditions enforce symmetries that are required for transformations of geometric objects. They are implemented in an implicit polar filter of a structured grid, nonhydrostatic global atmospheric model that is simulating idealized Held–Suarez flows. A series of test simulations using different configurations of the asymptotic boundary conditions are made, along with control simulations that use the default model numerics with no absorber, at three different grid sizes. Typically the test simulations are ∼20% faster in wall clock time than the control—resulting from a decrease in noise at the poles in all cases. In the control simulations adverse numerical effects from the polar singularity are observed to increase with grid resolution. In contrast, test simulations demonstrate robust polar behavior independent of grid resolution.
... More recently, forward-in-time finite volume (FTFV) integrators, that can be written in a congruent manner as the SL scheme, have also emerged [91] with applications in NWP and climate. Furthermore, vertical Lagrangian coordinates have been successfully applied in hydrostatic models [46]. ...
The continuous partial differential equations (PDEs) governing a given physical phenomenon, such as the Navier-Stokes equations describing the fluid motion, must be numerically discretised in space and time in order to obtain a solution otherwise not readily available in closed (i.e., analytic) form. While the overall numerical discretization plays an essential role in the algorithmic efficiency and physically-faithful representation of the solution, the time-integration strategy commonly is one of the main drivers in terms of cost-to-solution (e.g., time-or energy-to-solution), accuracy and numerical stability, thus constituting one of the key building blocks of the computational model. This is especially true in time-critical applications, including numerical weather prediction (NWP), climate simulations and engineering. This review provides a comprehensive overview of the existing and emerging time-integration (also referred to as time-stepping) practices used in the operational \textit{global} NWP and climate industry, where \textit{global} refers to weather and climate simulations performed on the \textit{entire globe}. While there are many flavors of time-integration strategies, in this review we focus on the most widely adopted in NWP and climate centres and we emphasise the reasons why such numerical solutions were adopted. This allows us to make some considerations on future trends in the field such as the need to balance accuracy in time with substantially enhanced time-to-solution and associated implications on energy consumption and running costs. In addition, the potential for the co-design of time-stepping algorithms and underlying high performance computing hardware, a keystone to accelerate the computational performance of future NWP and climate services, is also discussed in the context of the demanding operational requirements of the weather and climate industry.
... The edge-based algorithms can be used on any type of structured, unstructured, or hybrid mesh, and are computationally more efficient. As a result, edge-based data structures have been adopted in various engineering and environmental projects and applications (Lyra et al. 2004;Sun et al. 2010;Al Qubeissi 2013;Szmelter et al. 2015;Zhang 2015;Smolarkiewicz et al. 2016). In reservoir simulation, face-based data structures are normally used King et al. 2012;Lie 2014). ...
Flow diagnostics is a common way to rank and cluster ensembles of reservoir models depending on their approximate dynamic behavior before beginning full-physics reservoir simulation. Traditionally, they have been performed on corner-point grids inherent to geocellular models. The rapid-reservoir-modeling (RRM) concept aims at fast and intuitive prototyping of geologically realistic reservoir models. In RRM, complex reservoir heterogeneities are modeled as discrete volumes bounded by surfaces that are sketched in real time. The resulting reservoir models are discretized by use of fully unstructured tetrahedral meshes where the grid conforms to the reservoir geometry, hence preserving the original geological structures that have been modeled.
This paper presents a computationally efficient work flow for flow diagnostics on fully unstructured grids. The control-volume finite-element method (CVFEM) is used to solve the elliptic pressure equation. The flux field is a multipoint flux approximation (MPFA). A new tracing algorithm is developed on a reduced monotone acyclic graph for the hyperbolic transport equations of time of flight (TOF) and tracer distributions. An optimal reordering technique is used to deal with each control volume locally such that the hyperbolic equations can be computed in an efficient node-by-node manner. This reordering algorithm scales linearly with the number of unknowns.
The results of these computations allow us to estimate swept-reservoir volumes, injector/producer pairs, well-allocation factors, flow capacity, storage capacity, and dynamic Lorenz coefficients, which all help approximate the dynamic reservoir behavior. The total central-processing-unit (CPU) time, including grid generation and flow diagnostics, is typically a few seconds for meshes with O (100,000) unknowns. Such fast calculations provide, for the first time, real-time feedback in the dynamic reservoir behavior while models are prototyped.
... Edge-based data structure offers a convenient way of discretisation and can be employed on any type of structured, unstructured or hybrid meshes. It is preferred for efficient computation and has been adopted in various engineering and environmental projects and applications (Lyra et al., 2004;Sun et al., 2010;Al Qubeissi, 2013;Zhang, 2015;Szmelter et al., 2015;Smolarkiewicz et al., 2016). The optimal reordering technique of Natvig et al. (2007) is adopted to unstructured grids to reduce the computational cost by dealing with each control volume locally, and the edge-based CVFEM formulation proves to be ideal for the reordering process. ...
Flow-diagnostics are a common way to rank and cluster ensembles of reservoir models based on their approximate dynamic behaviour prior to commencing full-physics reservoir simulation. Traditionally, flow diagnostics are carried out on corner-point grids inherent to geocellular models.
The novel "Rapid Reservoir Modelling" (RRM) concept enables fast and intuitive prototyping and updating of reservoir models. In RRM, complex reservoir heterogeneities are modelled as discrete volumes bounded by surfaces that can be modified using simple sketching operations in real time. The resulting reservoir models are discretized using fully unstructured 3D meshes where the grid conforms to the reservoir geometry.
This paper presents a new and computationally efficient numerical scheme that enables flow diagnostic calculations on fully unstructured grids. Time-of-flight and steady-state tracer distributions are computed directly on the grid. The results of these computations allows us to estimate swept reservoir volumes, injector-producer pairs, well-allocation factors, flow capacity, storage capacity and dynamic Lorenz coefficients which all help approximate the dynamic reservoir behaviour.
We use the Control Volume Finite Element Method (CVFEM) to solve the elliptic pressure equation. A scalable matrix solver (SAMG) is used to invert the linear system. A new edge-based CVFEM is developed to solve hyperbolic transport equations for time-of-flight and tracer distributions. An optimal reordering technique is employed to deal with each control volume locally such that the hyperbolic equations can be computed in an efficient node-by-node manner. This reordering algorithm scales linearly with the number of unknowns.
The total CPU time, including grid generation and flow diagnostics, is typically below 3 seconds for grids with 50k unknowns. Such fast calculations provide, for the first time, real-time feedback on changes in the dynamic reservoir behaviour while the reservoir model is updated.
We explore the domain-specific Python library GT4Py (GridTools for Python) for implementing a representative physical parametrization scheme and the related tangent-linear and adjoint algorithms from the Integrated Forecasting System (IFS) of ECMWF. GT4Py encodes stencil operators in an abstract and hardware-agnostic fashion, thus enabling more concise, readable and maintainable scientific applications. The library achieves high performance by translating the application into targeted low-level coding implementations. Here, the main goal is to study the correctness and performance-portability of the Python rewrites with GT4Py against the reference Fortran code and a number of automatically and manually ported variants created by ECMWF. The present work is part of a larger cross-institutional effort to port weather and climate models to Python with GT4Py. The focus of the current work is the IFS prognostic cloud microphysics scheme, a core physical parametrization represented by a comprehensive code that takes a significant share of the total forecast model execution time. In order to verify GT4Py for Numerical Weather Prediction (NWP) systems, we put additional emphasis on the implementation and validation of the tangent-linear and adjoint model versions which are employed in data assimilation. We benchmark all prototype codes on three European supercomputers characterized by diverse GPU and CPU hardware, node designs, software stacks and compiler suites. Once the application is ported to Python with GT4Py, we find excellent portability, competitive performance, and robust execution in all tested scenarios including with reduced precision.
Banner clouds are clouds in the lee of steep mountains or sharp ridges on otherwise cloud-free days. Previous studies investigated various aspects of banner cloud formation in numerical simulations, most of which were based on idealized orography and a neutrally stratified ambient atmosphere. The present study extends these simulations in two important directions by (1) examining the impact of various types of orography ranging from an idealized pyramid to the realistic orography of Mount Matterhorn, and (2) accounting for an ambient atmosphere that turns from neutral to stably stratified below the mountain summit. Not surprisingly, realistic orography introduces asymmetries in the spanwise direction. At the same time, banner cloud occurrence remains associated with a coherent area of strong uplift, although this region does not have to be located exclusively in the lee of the mountain any longer. In the case of Mount Matterhorn with a westerly ambient flow, a large fraction of air parcels rises along the southern face of the mountain, before they reach the lee and are lifted into the banner cloud. The presence of a shallow boundary layer with its top below the mountain summit introduces more complex behavior compared to a neutrally stratified boundary layer; in particular, it introduces a dependence on wind speed, because strong wind is associated with strong turbulence that is able to raise the boundary layer height and, thus, facilitates the formation of a banner cloud.
The Yin‐He Global Spectral Model (YHGSM) is a dry‐mass conserving hydrostatic global spectral model, relying on spectral transforms to compute horizontal derivatives. We present an extension of YHGSM core named YHGSM‐FVM which uses a second‐order finite‐volume method (FVM) to compute the horizontal derivatives in grid‐point space instead of the spectral approach. With this approach, the computational efficiency of the spectral model is improved since part of the spectral transforms is superseded by FVM which only needs local data and the computational demand is lower. More importantly, YHGSM‐FVM is still a spectral model solving the Helmholtz equation directly in spectral space with a highly efficient semi‐implicit semi‐Lagrangian advection scheme. The comparisons between YHGSM‐FVM and YHGSM are conducted, and the results show that both models have comparable prediction skill, but YHGSM‐FVM outperforms YHGSM in computational efficiency.
This paper presents the semi-implicit compressible EULAG as a new dynamical core for convective-scale numerical weather prediction. The core is implemented within the infrastructure of the operational model of the Consortium for Small Scale Modeling (COSMO), forming the NWP COSMO-EULAG model (CE). This regional high-resolution implementation of the dynamical core complements its global implementation in the Finite-Volume Module of ECMWF’s Integrated Forecasting System. The paper documents the first operational-like application of the dynamical core for realistic weather forecasts. After discussing the formulation of the core and its coupling with the host model, the paper considers several high-resolution prognostic experiments over complex Alpine orography. Standard verification experiments examine the sensitivity of the CE forecast to the choice of the advection routine and assess the forecast skills against those of the default COSMO Runge-Kutta dynamical core at 2.2 km grid size showing a general improvement. The skills are also compared using satellite observations for a weak-flow convective Alpine weather case-study, showing favorable results. Additional validation of the new CE framework for partly convection-resolving forecasts using 1.1 km, 0.55 km, 0.22 km, and 0.1 km grids, designed to challenge its numerics and test the dynamics-physics coupling, demonstrates its high robustness in simulating multi-phase flows over complex mountain terrain, with slopes reaching 85 degrees, and the flow’s realistic representation.
A numerical study of stably stratified flows past spheres at Reynolds numbers Re=200 and Re=300 is reported. In these flow regimes, a neutrally stratified laminar flow induces distinctly different near-wake features. However, the flow behaviour changes significantly as the stratification increases and suppresses the scale of vertical displacements of fluid parcels. Computations for a range of Froude numbers Fr∈[0.1,∞] show that as Froude number decreases, the flow patterns for both Reynolds numbers become similar. The representative simulations of the lee-wave instability at Fr=0.625 and the two-dimensional vortex shedding at Fr=0.25 regimes are illustrated for flows past single and tandem spheres, thereby providing further insight into the dynamics of stratified flows past bluff bodies. In particular, the reported study examines the relative influence of viscosity and stratification on the dividing streamline elevation, wake structure and flow separation. The solutions of the Navier-Stokes equations in the incompressible Boussinesq limit are obtained on unstructured meshes suitable for simulations involving multiple bodies. Computations are accomplished using the finite volume, non-oscillatory forward-in-time (NFT) Multidimensional Positive Definite Transport Algorithm (MPDATA) based solver. The impact and validity of the numerical approximations, especially for the cases exhibiting strong stratification, are also discussed. Qualitative and quantitative comparisons with available laboratory experiments and prior numerical studies confirm the validity of the numerical approach.
How the upper-atmosphere branch of the circulation contributes to
and interacts with the circulation of the middle and lower atmosphere
is a research area with many open questions. Inertia–gravity waves, for instance,
have moved in the focus of research as they are suspected to be key features
in driving and shaping the circulation. Numerical atmospheric models are
an important pillar for this research. We use the ICOsahedral Non-hydrostatic
(ICON) general circulation model, which is a joint development
of the Max Planck Institute for Meteorology (MPI-M)
and the German Weather Service (DWD), and provides, e.g., local mass conservation,
a flexible grid nesting option, and a non-hydrostatic dynamical core
formulated on an icosahedral–triangular grid.
We extended ICON to the upper atmosphere and present here the two main components
of this new configuration named UA-ICON: an extension of the dynamical core from
shallow- to deep-atmosphere dynamics and the implementation of an upper-atmosphere
physics package.
A series of idealized test cases and climatological simulations is performed
in order to evaluate the upper-atmosphere extension of ICON.
This work proposes a low-overhead half-barrier pattern to schedule fine-grain parallel loops and considers its integration in the Intel OpenMP and Cilkplus schedulers. Experimental evaluation demonstrates that the scheduling overhead of our techniques is 43% lower than Intel OpenMP and 12.1x lower than Cilk. We observe 22% speedup on 48 threads, with a peak of 2.8x speedup.
A non-oscillatory forward-in-time (NFT) integrator is developed to provide solutions of the Navier–Stokes equations for incompressible flows. Simulations of flows past a sphere are chosen as a benchmark representative of a class of engineering flows past obstacles. The methodology is further extended to moderate Reynolds number, stably stratified flows under gravity, for Froude numbers that typify the characteristic regimes of natural flows past distinct isolated features of topography in weather and climate models. The key elements of the proposed method consist of the Multidimensional Positive Definite Advection Transport Algorithm (MPDATA) and a robust non-symmetric Krylov-subspace elliptic solver. The solutions employ a finite volume spatial discretisation on unstructured and hybrid meshes and benefit from a collocated arrangement of all flow variables while being inherently stable. The development includes the implementation of viscous terms with the detached-eddy simulation (DES) approach employed for turbulent flows. Results demonstrate that the proposed methodology enables direct comparisons of the numerical solutions with corresponding laboratory studies of viscous and stratified flows while illustrating accuracy, robustness and flexibility of the NFT schemes. The presented simulations also offer a better insight into stably stratified flows past a sphere.
In this study, we identify the key message passing interface (MPI) operations
required in atmospheric modelling; then, we use a skeleton program and a
simulation framework (based on SST/macro simulation package) to simulate
these MPI operations (transposition, halo exchange, and
allreduce), with the perspective of future exascale machines in
mind. The experimental results show that the choice of the collective
algorithm has a great impact on the performance of communications; in
particular, we find that the generalized ring-k algorithm for the alltoallv
operation and the generalized recursive-k algorithm for the allreduce
operation perform the best. In addition, we observe that the impacts of
interconnect topologies and routing algorithms on the performance and
scalability of transpositions, halo exchange, and allreduce operations are
significant. However, the routing algorithm has a negligible impact on the
performance of allreduce operations because of its small message size. It is
impossible to infinitely grow bandwidth and reduce latency due to hardware
limitations. Thus, congestion may occur and limit the continuous improvement
of the performance of communications. The experiments show that the
performance of communications can be improved when congestion is mitigated by
a proper configuration of the topology and routing algorithm, which uniformly
distribute the congestion over the interconnect network to avoid the hotspots
and bottlenecks caused by congestion. It is generally believed that the
transpositions seriously limit the scalability of the spectral models. The
experiments show that the communication time of the transposition is larger
than those of the wide halo exchange for the semi-Lagrangian method and the
allreduce in the generalized conjugate residual (GCR) iterative solver for
the semi-implicit method below 2×105 MPI processes. The
transposition whose communication time decreases quickly with increasing
number of MPI processes demonstrates strong scalability in the case of very
large grids and moderate latencies. The halo exchange whose communication
time decreases more slowly than that of transposition with increasing number
of MPI processes reveals its weak scalability. In contrast, the allreduce
whose communication time increases with increasing number of MPI processes
does not scale well. From this point of view, the scalability of spectral
models could still be acceptable. Therefore it seems to be premature to
conclude that the scalability of the grid-point models is better than that of
spectral models at the exascale, unless innovative methods are exploited to
mitigate the problem of the scalability presented in the grid-point models.
Weather and climate models are complex pieces of software which include many individual components, each of which is evolving under pressure to exploit advances in computing to enhance some combination of a range of possible improvements (higher spatio-temporal resolution, increased fidelity in terms of resolved processes, more quantification of uncertainty, etc.). However, after many years of a relatively stable computing environment with little choice in processing architecture or programming paradigm (basically X86 processors using MPI for parallelism), the existing menu of processor choices includes significant diversity, and more is on the horizon. This computational diversity, coupled with ever increasing software complexity, leads to the very real possibility that weather and climate modelling will arrive at a chasm which will separate scientific aspiration from our ability to develop and/or rapidly adapt codes to the available hardware.
In this paper we review the hardware and software trends which are leading us towards this chasm, before describing current progress in addressing some of the tools which we may be able to use to bridge the chasm. This brief introduction to current tools and plans is followed by a discussion outlining the scientific requirements for quality model codes which have satisfactory performance and portability, while simultaneously supporting productive scientific evolution. We assert that the existing method of incremental model improvements employing small steps which adjust to the changing hardware environment is likely to be inadequate for crossing the chasm between aspiration and hardware at a satisfactory pace, in part because institutions cannot have all the relevant expertise in house. Instead, we outline a methodology based on large community efforts in engineering and standardisation, which will depend on identifying a taxonomy of key activities – perhaps based on existing efforts to develop domain-specific languages, identify common patterns in weather and climate codes, and develop community approaches to commonly needed tools and libraries – and then collaboratively building up those key components. Such a collaborative approach will depend on institutions, projects, and individuals adopting new interdependencies and ways of working.
A new grid system on a sphere is proposed that allows for straightforward implementation of both spherical-harmonics-based spectral methods and gridpoint-based multigrid methods. The latitudinal gridpoints in the new grid are equidistant and spectral transforms in the latitudinal direction are performed using Clenshaw-Curtis quadrature. The spectral transforms with this new grid and quadrature are shown to be exact within the machine precision provided that the grid truncation is such that there are at least 2N + 1 latitudinal gridpoints for the total truncation wavenumber of N. The new grid and quadrature is implemented and tested on a shallow-water equations model and the hydrostatic dry dynamical core of the global NWP model JMA-GSM. The integration results obtained with the new quadrature are shown to be almost identical to those obtained with the conventional Gaussian quadrature on Gaussian grid. Only minor code changes are required to adapt any Gaussian-based spectral models to employ the proposed quadrature.
*** This is a preprint version of the manuscript published in QJRMS as https://doi.org/10.1002/qj.3282 ***
A new grid system on a sphere is proposed that allows for straightforward implementation of both spherical-harmonics-based spectral methods and gridpoint-based multigrid methods. The latitudinal grid-points in the new grid are equidistant and spectral transforms in the latitudinal direction are performed using Clenshaw-Curtis quadrature. The spectral transforms with this new grid and quadrature are shown to be exact within the machine precision provided that the grid trun-cation is such that there are at least 2N + 1 latitudinal gridpoints for the total truncation wavenumber of N. The new grid and quadrature is implemented and tested on a shallow-water equations model and the hydrostatic dry dynamical core of the global NWP model JMA-GSM. The integration results obtained with the new quadrature are shown to be almost identical to those obtained with the conventional Gaussian quadrature on Gaussian grid.
Cite as: arXiv:1711.11296 [math.NA]
(or arXiv:1711.11296v1 [math.NA] for this version)
Members in ensemble forecasts differ due to the representations of initial uncertainties and model uncertainties. The inclusion of stochastic schemes to represent model uncertainties has improved the probabilistic skill of the ECMWF ensemble by increasing reliability and reducing the error of the ensemble mean. Recent progress, challenges and future directions regarding stochastic representations of model uncertainties at ECMWF are described in this paper. The coming years are likely to see a further increase in the use of ensemble methods in forecasts and assimilation. This will put increasing demands on the methods used to perturb the forecast model. An area that is receiving a greater attention than 5 to 10 years ago is the physical consistency of the perturbations. Other areas where future efforts will be directed are the expansion of uncertainty representations to the dynamical core and to other components of the Earth system as well as the overall computational efficiency of representing model uncertainty.
With the objective to develop and maintain one of the most advanced and flexible modelling infrastructures in Europe for operational, global NWP applications, recent advances and future challenges are described. A particular challenge arises from the need to achieve computationally and energy efficient solutions for operating global, complex, high-resolution, ensemble based systems on high-performance computers so that they will remain affordable given tight operational schedules. In particular, the rising cost of quantifying uncertainty needs to be addressed. This paper presents the current status and steps taken towards increasing the model realism and complexity to improve forecasts with a sustainable modelling infrastructure. The progress to date includes the flexibility to explore unstructured horizontal discretizations, the addition of a new, powerful 3D solver for elliptic problems arising from the implicit discretization of the non-hydrostatic system, an option for inherently conserving, monotone, multi-tracer transport, and developments towards a flexible vertical coordinate formulation. Future scientific priorities are to combine the strengths of the newly developed, principally autonomous non-hydrostatic finite-volume module FVM with the hydrostatic semi-Lagrangian spectral transform options of the IFS, to review the vertical discretization, and to carefully address physics-dynamics as well as Earth-system component coupling.
This paper accompanies the first release of libmpdata++, a C++
library implementing the multi-dimensional positive-definite
advection transport algorithm (MPDATA) on regular structured grid.
The library offers basic
numerical solvers for systems of generalised transport equations.
The solvers are forward-in-time, conservative and non-linearly
stable. The libmpdata++ library covers the basic
second-order-accurate formulation of MPDATA, its third-order
variant, the infinite-gauge option for variable-sign fields and
a flux-corrected transport extension to guarantee non-oscillatory
solutions. The library is equipped with a non-symmetric variational
elliptic solver for implicit evaluation of pressure gradient terms.
All solvers offer parallelisation through domain decomposition using
shared-memory parallelisation.
The paper describes the library programming interface, and serves as
a user guide. Supported options are illustrated with benchmarks
discussed in the MPDATA literature. Benchmark descriptions include
code snippets as well as quantitative representations of simulation
results. Examples of applications include homogeneous transport in
one, two and three dimensions in Cartesian and spherical domains;
a shallow-water system compared with analytical solution (originally
derived for a 2-D case); and a buoyant convection problem in an
incompressible Boussinesq fluid with interfacial instability. All
the examples are implemented out of the library tree. Regardless of
the differences in the problem dimensionality, right-hand-side
terms, boundary conditions and parallelisation approach, all the
examples use the same unmodified library, which is a key goal of
libmpdata++ design. The design, based on the principle of
separation of concerns, prioritises the user and developer
productivity. The libmpdata++ library is implemented in
C++, making use of the Blitz++ multi-dimensional array
containers, and is released as free/libre and open-source software.
This paper accompanies the first release of libmpdata++, a C++ library implementing the multi-dimensional positive-definite advection transport algorithm (MPDATA) on regular structured grid. The library offers basic numerical solvers for systems of generalised transport equations. The solvers are forward-in-time, conservative and non-linearly stable. The libmpdata++ library covers the basic second-order-accurate formulation of MPDATA, its third-order variant, the infinite-gauge option for variable-sign fields and a flux-corrected transport extension to guarantee non-oscillatory solutions. The library is equipped with a non-symmetric variational elliptic solver for implicit evaluation of pressure gradient terms. All solvers offer parallelisation through domain decomposition using shared-memory parallelisation.
The paper describes the library programming interface, and serves as a user guide. Supported options are illustrated with benchmarks discussed in the MPDATA literature. Benchmark descriptions include code snippets as well as quantitative representations of simulation results. Examples of applications include homogeneous transport in one, two and three dimensions in Cartesian and spherical domains; a shallow-water system compared with analytical solution (originally derived for a 2-D case); and a buoyant convection problem in an incompressible Boussinesq fluid with interfacial instability. All the examples are implemented out of the library tree. Regardless of the differences in the problem dimensionality, right-hand-side terms, boundary conditions and parallelisation approach, all the examples use the same unmodified library, which is a key goal of libmpdata++ design. The design, based on the principle of separation of concerns, prioritises the user and developer productivity. The libmpdata++ library is implemented in C++, making use of the Blitz++ multi-dimensional array containers, and is released as free/libre and open-source software.
The steady path of doubling the global horizontal resolution approximately every 8 years in numerical weather prediction (NWP) at the European Centre for Medium Range Weather Forecasts may be substan- tially altered with emerging novel computing architectures. It coincides with the need to appropriately address and determine forecast uncertainty with increasing resolution, in particular, when convective-scale motions start to be resolved. Blunt increases in the model resolution will quickly become unaffordable and may not lead to improved NWP forecasts. Consequently, there is a need to accordingly adjust proven numerical techniques. An informed decision on the modelling strategy for harnessing exascale, massively parallel computing power thus also requires a deeper understanding of the sensitivity to uncertainty-for each part of the model-and ultimately a deeper understanding of multi-scale interactions in the atmosphere and their numerical realization in ultra-high-resolution NWP and climate simulations. This paper explores opportunities for substantial increases in the forecast efficiency by judicious adjustment of the formal accuracy or relative resolution in the spectral and physical space. One path is to reduce the formal accuracy by which the spectral transforms are computed. The other pathway explores the importance of the ratio used for the horizontal resolution in gridpoint space versus wavenumbers in spectral space. This is relevant for both high-resolution simulations as well as ensemble-based uncertainty estimation.
EULAG (Eulerian/semi-Lagrangian fluid solver) is an established computational model for simulating thermofluid flows across a wide range of scales and physical scenarios. It is noteworthy for its non-oscillatory integration algorithms, robust elliptic solver, and generalized coordinate formulation enabling grid adaptivity technology. In this paper we highlight the key model ingredients, demonstrate its capabilities with a select subset of recent applications, and show its performance both in terms of accuracy and scalability on massively parallel processor architectures. A comprehensive list of references is provided to facilitate more detailed study.
A second-order accurate, forward-in-time, finite-difference scheme for the advection equation with arbitrary forcing is extended to an arbitrary curvilinear system of coordinates. A compact scheme that is consistent with Eulerian and Lagrangian formulations for fluids is obtained using a rigorous truncation-error analysis. Particular attention is given to alternative approximations to the advective velocity in the transport flux. An extrapolation consistent with the governing equations of motion is used to achieve second-order accuracy of the forward-in-time approximation. The approximation is derived from concepts based on Runge-Kutta methods for ordinary differential equations.
The recursive zonal equal area (EQ) sphere partitioning algorithm is a practical algorithm for partitioning higher dimensional spheres into regions of equal area and small diameter. This paper describes the partition algorithm and its implementation in Matlab, provides numerical results and gives a sketch of the proof of the bounds on the diameter of regions. A companion paper [13] gives details of the proof.
¶The nonhydrostatic Meso model developed at NCEP (Janjic et al, 2001) is based on a new approach. Namely, a hydrostatic NWP
model using mass based vertical coordinate has been extended to include the nonhydrostatic motions. In this way favorable
features of the hydrostatic formulation have been preserved. This procedure did not require any linearization or approximation.
The nonhydrostatic dynamics has been introduced through an add-on module. The nonhydrostatic module can be turned on and off,
so that easy comparison can be made of hydrostatic and nonhydrostatic solutions.
Here, the basic philosophy behind the discretization methods applied in the model, and not covered by Janjic et al (2001),
is discussed, and the latest developments are reviewed. The forecast examples shown indicate that significant differences
between hydrostatic and nonhydrostatic forecasts may develop even at relatively coarse resolution of 8 km. Possible future
developments are considered.
A semi-implicit edge-based unstructured-mesh model is developed that integrates nonhydrostatic soundproof equations, inclusive
of anelastic and pseudo-incompressible systems of partial differential equations. The model builds on nonoscillatory forward-in-time
MPDATA approach using finite-volume discretization and unstructured meshes with arbitrarily shaped cells. Implicit treatment
of gravity waves benefits both accuracy and stability of the model. The unstructured-mesh solutions are compared to equivalent
structured-grid results for intricate, multiscale internal-wave phenomenon of a non-Boussinesq amplification and breaking
of deep stratospheric gravity waves. The departures of the anelastic and pseudoincompressible results are quantified in reference
to a recent asymptotic theory [Achatz et al. 2010, J. Fluid Mech., 663, 120–147)].
Key wordsunstructured mesh models–nonoscillatory forward-in-time schemes–atmospheric models–soundproof equations–mountain waves
The evolution of global atmospheric model dynamical cores from the first developments in the early 1960s to present day is reviewed. Numerical methods for atmospheric models are not straightforward because of the so-called pole problem. The early approaches include methods based on composite meshes, on quasi-homogeneous grids such as spherical geodesic and cubed sphere, on reduced grids, and on a latitude-longitude grid with short time steps near the pole, none of which were entirely successful. This resulted in the dominance of the spectral transform method after it was introduced. Semi-Lagrangian semi-implicit methods were developed which yielded significant computational savings and became dominant in Numerical Weather Prediction. The need for improved physical properties in climate modeling led to developments in shape preserving and conservative methods. Today the numerical methods development community is extremely active with emphasis placed on methods with desirable physical properties, especially conservation and shape preservation, while retaining the accuracy and efficiency gained in the past. Much of the development is based on quasi-uniform grids. Although the need for better physical properties is emphasized in this paper, another driving force is the need to develop schemes which are capable of running efficiently on computers with thousands of processors and distributed memory. Test cases for dynamical core evaluation are also briefly reviewed. These range from well defined deterministic tests to longer term statistical tests with both idealized forcing and complete parameterization packages but simple geometries. Finally some aspects of coupling dynamical cores to parameterization suites are discussed.
A new type of ultra-high resolution atmospheric global circulation model is developed. The new model is designed to perform “cloud resolving simulations” by directly calculating deep convection and meso-scale circulations, which play key roles not only in the tropical circulations but in the global circulations of the atmosphere. Since cores of deep convection have a few km in horizontal size, they have not directly been resolved by existing atmospheric general circulation models (AGCMs). In order to drastically enhance horizontal resolution, a new framework of a global atmospheric model is required; we adopted nonhydrostatic governing equations and icosahedral grids to the new model, and call it Nonhydrostatic ICosahedral Atmospheric Model (NICAM). In this article, we review governing equations and numerical techniques employed, and present the results from the unique 3.5-km mesh global experiments—with O(109) computational nodes—using realistic topography and land/ocean surface thermal forcing. The results show realistic behaviors of multi-scale convective systems in the tropics, which have not been captured by AGCMs. We also argue future perspective of the roles of the new model in the next generation atmospheric sciences.
A benchmark calculation is proposed for evaluating the dynamical cores of atmospheric general circulation models (GCMs) independently of the physical parameterizations. The test focuses on the long-term statistical properties of a fully developed general circulation; thus, it is particularly appropriate for intercomparing the dynamics used in climate models. To illustrate the use of this benchmark, two very different atmospheric dynamical cores--one spectral, one finite difference--are compared. It is found that the long-term statistics produced by the two models are very similar. Selected results from these calculations are presented to initiate the intercomparison.
Prediction of the Earth's climate andweather is difficult in large part because of the ubiquity of turbulence in the atmosphere and oceans. Geophysical flows evince fluid motions ranging from dissipation scales as small as a fraction of a millimeter to planetary scales of thousands of kilometers. The span in time scales (from a fraction of a second to many years) is equally large. Turbulence in the atmosphere and the oceans is generated by heating and by boundary stresses – just as in engineering flows. However, geophysical flows are further complicated by planetary rotation and density–temperature stratification, which lead to phenomena not commonly found in engineering applications. In particular, rotating stratified fluids can support a variety of inertia-gravity and planetary waves. When the amplitude of such a wave becomes sufficiently large (i.e., comparable to the wavelength), the wave can break, generating a localized burst of turbulence. If one could see the phenomena that occur internally in geophysical flows at any scale, one would be reminded of familiar pictures of white water in a mountain stream or of breaking surf on a beach. The multiphase thermodynamics of atmosphere and oceans – due to ubiquity of water substance and salt, respectively – adds complexity of its own. Because of the enormous range of scales, direct numerical simulation (DNS) of the Earth's weather and climate is far beyond the reach of current computational technology. Consequently, all numerical simulations truncate the range of resolved scales to one that is tractable on contemporary computational machines. However, retaining the physicality of simulation necessitates modeling the contribution of truncated scales to the resolved range.
ABSTRACT: Today the European Centre for Medium Range Weather Forecasts (ECMWF) runs a 16 km global T1279 operational weather forecast model using 1536 cores of an IBM Power7. Following the historical evolution in resolution upgrades, the ECMWF could expect to be running a 2.5 km global forecast model by 2030 on an exascale system that should be available and hopefully affordable by then. To achieve this would require the Integrated Forecasting System (IFS) to run efficiently on about 1000 times the number of cores it uses today. In a step towards this goal, the ECMWF have demonstrated the IFS running a 10 km global model efficiently on over 40,000 cores of HECToR a Cray XE6 at the Edinburgh Parallel Computing Centre. However, getting to over a million cores remains a formidable challenge, and many scalability improvements have yet to be implemented. The ECMWF is exploring the use of Fortran2008 coarrays; in particular, it is possibly the first time that coarrays have been used in a world-leading production application within the context of OpenMP parallel regions. The purpose of these optimisations is primarily to allow the overlap of computation and communication, and further, in the semi-Lagrangian advection scheme, to reduce the volume of data communicated. The importance of this research is such that if these and other planned developments are successful, the IFS model may continue to use the spectral transform method to 2030 and beyond on an exascale-sized system. The current status of the coarray scalability developments within the IFS are described together with a brief outline of future developments.
Anelastic and compressible solutions are compared for two moist deep convection benchmarks, a two-dimensional thermal rising in a saturated moist-neutral deep atmosphere, and a three-dimensional supercell formation. In the anelastic model, the pressure applied in the moist thermodynamics comes from either the environmental hydrostatically balanced pressure profile in the standard anelastic model or is combined with nonhydrostatic perturbations from the elliptic pressure solver in the generalized anelastic model. The compressible model applies either an explicit acoustic-mode-resolving scheme requiring short time steps or a novel implicit scheme allowing time steps as large as those used in the anelastic model. The consistency of the unified numerical framework facilitates direct comparisons of results obtained with anelastic and compressible models.
The anelastic and compressible rising thermal solutions agree not only with each other but also with the previously published compressible benchmark solution based on the comprehensive representation of moist dynamics and thermodynamics. In contrast to earlier works focusing on the formulation of moist thermodynamics, the compatibility of the initial conditions is emphasized and its impact on the benchmark solutions is documented. The anelastic and compressible supercell solutions agree well for various versions of anelastic and compressible models even for cloud updrafts reaching 15% of the speed of sound. The nonhydrostatic pressure perturbations turn out to have a negligible impact on the moist dynamics. Numerical and physical details of the simulations, such as the advection scheme, spatial and temporal resolution, or parameters of the subgrid-scale turbulence, have a more significant effect on the solutions than the particular equation system applied.
Since the mid-90s IFS has used a 2-dimensional scheme for partitioning grid point space to MPI tasks. While this scheme has served ECMWF well there has nevertheless been some areas of concern, namely, communication overheads for IFS reduced grids at the poles to support the Semi-Lagrangian scheme; and the halo requirements needed to support the interpolation of fields between model and radiation grids.
These issues have been addressed by the implementation of a new partitioning scheme called EQ_REGIONS which is characterised by an increasing number of partitions in bands from the poles to the equator. The number of bands and the number of partitions in each particular band are derived so as to provide partitions of equal area and small 'diameter'. The EQ_REGIONS algorithm used in IFS is based on the work of Paul Leopardi, School of Mathematics, University of New South Wales, Sydney, Australia.
The numerical simulation of turbulent flows is a subject of great practical importance to scientists and engineers. The difficulty in achieving predictive simulations is perhaps best illustrated by the wide range of approaches that have been developed and are still being used by the turbulence modeling community. In this book the authors describe one of these approaches, Implicit Large Eddy Simulation (ILES). ILES is a relatively new approach that combines generality and computational efficiency with documented success in many areas of complex fluid flow. This book synthesizes the current understanding of the theoretical basis of the ILES methodology and reviews its accomplishments. ILES pioneers and lead researchers combine here their experience to present the first comprehensive description of the methodology. This book should be of fundamental interest to graduate students, basic research scientists, as well as professionals involved in the design and analysis of complex turbulent flows.
A three-dimensional semi-implicit edge-based unstructured-mesh model is
developed that integrates nonhydrostatic anelastic equations, suitable
for simulation of small-to-mesoscale atmospheric flows. The model builds
on nonoscillatory forward-in-time MPDATA approach using finite-volume
discretization and admitting unstructured meshes with arbitrarily shaped
cells. The numerical advancements are evaluated with canonical
simulations of convective planetary boundary layer and strongly (stably)
stratified orographic flows, epitomizing diverse aspects of highly
nonlinear nonhydrostatic dynamics. The unstructured-mesh solutions are
compared to equivalent results generated with an established
structured-grid model and observation.
A numerical framework is developed for consistent integrations of the soundproof and fully compressible nonhydrostatic equations of motion for all-scale atmospheric flows; i.e., low Mach number, high Reynolds number, rotating stratified flows under gravity. The reduced anelastic and pseudo-incompressible soundproof equations and the fully compressible Euler equations are combined into a common form of conservation laws for mass, momentum and entropy that facilitates the design of a sole principal algorithm for its integration, with minimal alterations for accommodating each special case. The development extends a proven numerical framework for integrating the soundproof equations. It relies on non-oscillatory forward-in-time transport methods, applied consistently to all dependent variables of the system at hand, and with buoyant and rotational modes of motion treated implicitly in the integration. When the fully compressible equations are solved, the framework admits congruent schemes with explicit or implicit representation of acoustic modes, so the former can provide a reference for the latter. The consistency of the framework minimises the numerical differences between the soundproof and compressible integrators, thus admitting conclusive comparisons between compressible and soundproof solutions, unobscured by algorithmic disparities. For the large-time-step implicit schemes, technical differences between the soundproof and compressible integrators reduce to the selection of either a prescribed or a numerically prognosed density, and extension of the generalised Poisson solver to a bespoke Helmholtz solver. The numerical advancements and merits of the approach are illustrated with canonical simulations of planetary baroclinic instability, an archetype of global weather, and the breaking of a deep stratospheric gravity waves, an example of nonhydrostatic mesoscale dynamics.
Following previous work on an inherently mass-conserving semi-implicit (SI) semi-Lagrangian (SL) discretization of the two-dimensional (2D) shallow-water equations and 2D vertical slice equations, that approach is here extended to the 3D deep-atmosphere, non-hydrostatic global equations. As with the reduced-dimension versions of this model, an advantage of the approach is that it preserves the same basic structure as a standard, non-mass-conserving, SISL version of the model. Additionally, the model is simply switchable to hydrostatic and/or shallow-atmosphere forms. It is also designed to allow simple switching between various geometries (Cartesian, spherical, spheroidal). The resulting mass-conserving model is applied to a standard set of test problems for such models in spherical geometry and compared with results from the standard SISL version of the model. This article is published with the permission of the Controller of HMSO and the Queen's Printer for Scotland.
[1] For several decades, jets and fronts have been known from observations to be significant sources of internal gravity waves in the atmosphere. Motivations to investigate these waves have included their impact on tropospheric convection, their contribution to local mixing and turbulence in the upper-troposphere, their vertical propagation into the middle atmosphere and the forcing of its global circulation. While many different studies have consistently highlighted jet exit regions as a favored locus for intense gravity waves, the mechanisms responsible for their emission had long remained elusive: one reason is the complexity of the environment in which the waves appear, another is that the waves constitute small deviations from the balanced dynamics of the flow generating them, i.e., they arise beyond our fundamental understanding of jets and fronts based on approximations that filter out gravity waves. Over the past two decades, the pressing need for improving parameterizations of non-orographic gravity waves in climate models that include a stratosphere has stimulated renewed investigations. The purpose of this review is to presents current knowledge and understanding on gravity waves near jets and fronts from observations, theory and modeling, and to discuss challenges for progress in coming years.
Since April 2007, the numerical weather prediction model, COSMO (Consortium for Small Scale Modelling), has been used operationally in a convection-permitting configuration, named COSMO-DE, at the Deutscher Wetterdienst (DWD; German weather service). Here the authors discuss the model changes that were necessary for the convective scale, and report on the experience from the first years of operational application of the model. For COSMO-DE the ability of the numerical solver to treat small-scale structures has been improved by using a Runge-Kutta method, which allows for the use of higher-order upwind advection schemes. The one-moment cloud microphysics parameterization has been extended by a graupel class, and adaptations for describing evaporation of rain and stratiform precipitation processes were made. Comparisons with a much more sophisticated two-moment scheme showed only minor differences in most cases with the exception of strong squall-line situations. Whereas the deep convection parameterization was switched off completely, small-scale shallow convection was still parameterized by the appropriate part of the Tiedtke scheme. During the first year of operational use, convective events in synoptically driven situations were satisfactorily simulated. Also the daily cycles of summertime 10-m wind and 1-h precipitation sums were well captured. However, it became evident that the boundary layer description had to be adapted to enhance convection initiation in airmass convection situations. Here the asymptotic Blackadar length scale l(infinity) had proven to be a sensitive parameter.
After six years of scientific, technical developments and meteorological validation, the Application of Research to Operations at Mesoscale (AROME-France) convective-scale model became operational at Mé téo-France at the end of 2008. This paper presents the main characteristics of this new numerical weather prediction system: the nonhydrostatic dynamical model core, detailed moist physics, and the associated three-dimensional variational data assimilation (3D-Var) scheme. Dynamics options settings and variables are explained. The physical parameterizations are depicted as well as their mutual interactions. The scale-specific features of the 3D-Var scheme are shown. The performance of the forecast model is evaluated using objective scores and case studies that highlight its benefits and weaknesses.
We propose a method for computing effective numerical eddy viscosity acting in dissipative numerical schemes used in monotonically integrated large eddy simulations of turbulence. The method is evaluated on an example of a specific nonoscillatory finite volume scheme MPDATA developed for simulations of geophysical flows.
Very high resolution spectral transform models are believed to become prohibitively expensive, due to the relative increase in computational cost of the Legendre transforms compared to the gridpoint computations. This article describes the implementation of a practical fast spherical harmonics transform into the Integrated Forecasting System (IFS) at ECMWF. Details of the accuracy of the computations, of the parallelisation and memory use are discussed. Results are presented that demonstrate the cost-effectiveness and accuracy of the fast spherical harmonics transform, successfully mitigating the concern about the disproportionally growing computational cost. Using the new transforms, the first T7999 global weather forecast (equivalent to approximately 2.5km horizontal grid size) using a spectral transform model has been produced.
EULAG is an established high-performance computational model for simulating fluid flows across a wide range of scales and physical scenarios [Prusa et al., Comput. Fluids 37 (2008) 1193]. Historically driven by interests in simulating weather and climate processes, the numerics of EULAG are unique, owing to a synergistic blend of non-oscillatory forward-in-time MPDATA methods, robust elliptic solver, and generalized coordinate formulation enabling grid adaptivity. In this paper the numerical apparatus of an ideal magnetohydrodynamic (MHD) extension of the EULAG model is discussed, the robust workings of which have been recently revealed in global large-eddy simulations of solar magneto-convection producing solar-like magnetic cycles and dynamo action [Ghizaru et al., ApJL 715 (2010) L133; Racine et al., ApJ 735 (2011) 46]. Here, a specialized nonoscillatory forward-in-time scheme for integrating ideal anelastic MHD equations is presented in detail, and illustrated with an abstract example of magnetized three-dimensional flow in time-dependent geometry for a weak, moderate and strong magnetic field. An analysis of the model performance reveals that multiple solutions of elliptic problems do not have to imply proportionally larger computational expense.
This paper describes an unstructured/hybrid mesh framework providing a robust environment for multiscale atmospheric modeling. The framework builds on nonoscillatory forward-in-time MPDATA solvers using finite volume edge-based discretization, and admits meshes with arbitrarily shaped cells. The numerical formulation is equally applicable to global and limited area models. Theoretical considerations are supported with canonical examples of slab-symmetric, nonhydrostatic orographic problems in weakly and strongly stratified flow regimes and three-dimensional hydrostatic analogues of the strongly stratified case on a slowly and rapidly rotating sphere.
A benchmark calculation is proposed for evaluating the dynamical cores of atmospheric general circulation models independently of the physical parameterizations. The test focuses on the long-term statistical properties of a fully developed general circulation; thus, it is particularly appropriate for intercomparing the dynamics used in climate models. To illustrate the use of this benchmark, two very different atmospheric dynamical cores - one spectral, one finite difference - are compared. It is found that the long-term statistics produced by the two models are very similar. Selected results from these calculations are presented to initiate the intercomparison. -Authors
We present new results from a joint ocean-acoustic modeling study of solitary wave generation in the Strait of Messina, their propagation in the Tyrrhenian Sea and subsequent shoaling in the Gulf of Gioia. The nonhydrostatic 3D EULAG model is used for the oceanographic predictions. The simulations are initialized with measured temperature and salinity profiles from an October 1995 survey of the Messina region, and forced with semidiurnal tidal magnitudes predicted by a barotropic tidal model. Parameter sensitivity studies are performed. The predicted solitary wave trains are compared with CTD chain measurements. The model results and data are examined through a wavelet analysis. The wavelengths are tracked by the spines (maximum intensity for each wavelength) at various times. From the slope of the variations, phase speeds are derived as a function of wavelength. For the parameters extracted from CTD measurements and existing tidal conditions, phase speed distribution for wavelengths ranging from about 0.6m to 1.6km are obtained. The model predicted phase speed magnitudes range from 0.85ms−1 to 0.93ms−1. The phase speeds derived from data range from 0.77ms−1 to 0.88ms−1. The model predicted phase speed versus wavelength distribution has similar trends to the phase speed versus wavelength distribution derived from data. The shoaling of the solitary waves in the Gulf of Gioia is studied. Calculations of the acoustical field are conducted, along the solitary wave propagation path, with the parabolic (PE) acoustical model.
The classical terrain-following coordinate transformation of Gal-Chen and Somerville has been extended to a broad class of time-dependent vertical domains. We provide explicit formulae for the associated transformation coefficients which are readily applicable to numerical implementations. The proposed extension facilitates modeling of undulating boundaries in various areas of computational fluid dynamics. The implementation is discussed in the context of a nonhydrostatic anelastic model for simulations of atmospheric and oceanic flows. The theoretical development is illustrated with numerical simulations of idealized flows. We also discuss an example of a practical application which incorporates a long-wave-approximation for a finite-amplitude free-surface upper boundary, directly relevant to ocean models.
Numerical integration of the compressible nonhydrostatic equations using semi-implicit techniques is complicated by the need to solve a Helmholtz equation at each time step. The authors present an accurate and efficient technique for solving the Helmholtz equation using a conjugate-residual (CR) algorithm that is accelerated by ADI preconditioned. These preconditioned CR solvers possess four distinct advantages over most other solvers that have been used with the Helmholtz equations that arise in compressible nonhydrostatic semi-implicit atmospheric models: the preconditioned CR methods 1) can solve Helmholtz equations containing variable coefficients, alleviating the need to prescribe a reference state in order to simplify the elliptic problem; 2) transparently include the cross-derivative terms arising from terrain transformations; 3) are efficient and accurate for nonhydrostatic models used across a broad range of scales, from cloud scales to synoptic-global scales; and 4) are easy to formulate and program. These features of the CR solver allow semi-implicit formulations that are unconstrained by the form of the Helmholtz equations, and the authors propose a formulation that is more consistent than those most often used in that it includes implicit treatment of all terms associated with the pressure gradients and divergence. This formulation is stable for nonhydrostatic-scale simulations involving steep terrain, whereas the more common semi-implicit formulation is not. The ADI preconditioners are presented for use in simulations of both hydrostatic and nonhydrostatic scale flows. These simulations demonstrate the efficiency and accuracy of the preconditioned CR method and the overall stability of the model formulation. The simulations also suggest a general convergence criteria for the iterative algorithm in terms of the solution divergence.
A scale analysis valid for deep moist convection is carried out. The
approximate equations of motion are anelastic with the time scale set by
the Brunt- Väisälä frequency. A new assumption is that
the base state potential temperature is a slowly varying function of the
vertical coordinate. It is this assumption that eliminates the energetic
inconsistency discussed by Wilhelmson and Ogura (1972) for a
non-isentropic base state. Another key result is that the dynamic
pressure is an order of magnitude smaller than the first-order
temperature and potential temperature. In agreement with observations,
the kinetic energy is found to be an order of magnitude smaller than the
first-order thermodynamic energy.A set of six numerical simulations
representing moderately deep moist convection is carried out. The base
state is an idealized maritime tropical sounding with no vertical wind
shear. The first calculation (Run A) shows the growth and dissipation of
a typical shower cloud. The remaining calculations have small changes in
either initial conditions or model equations from Run A. These
calculations indicate the sensitivity of the present model to different
approximations and give additional evidence for the validity of the
scale analysis.
A brief review of the scale analysis of Lipps and Hemler is given
without any reference to the parameters G and B. The resulting anelastic
equations conserve energy, in contrast to the modified anelastic set of
equations analyzed by Durran. In addition, the present equations give an
accurate solution for the frequency of gravity waves in an isothermal
atmosphere. The present anelastic equations have these characteristics
in common with the pseudo-incompressible equations introduced by
Durran.The equations obtained from the scale analysis are appropriate
for numerical integration of deep convection. The associated Poisson
equation can be solved using standard procedures. For the
pseudo-incompressible set of equations, the Poisson equation is more
difficult to solve.
The validity of the hydrostatic approximation is examined for use in
predicting the dynamics of topographically generated atmospheric gravity
waves (lee waves) propagating in an atmosphere with realistic wind
shear. To isolate nonhydrostatic effects, linear, analytic solutions
derived both with and without the hydrostatic assumption are compared.
The atmospheric profiles of wind and stability are chosen both to render
the governing equations analytically tractable and be representative of
typical atmospheric conditions. Two atmospheric models are considered:
1) a troposphere-only model in which the wind increases linearly with
height and the stability is constant and 2) a troposphere-stratosphere
model, which retains the important effect of the vertical wind shear in
the troposphere and adds the essential feature of a stability jump at
the tropopause. The nonhydrostatic trapping effect of wind shear on
gravity wave modes is clearly illustrated in the troposphere-only
atmospheric model. In the troposphere-stratosphere model the vertical
wind shear partially traps nonhydrostatic waves in the troposphere,
which leak energy into the stratosphere; this effect is completely
eliminated in the hydrostatic solution. Solutions for both hydrostatic
and nonhydrostatic cases are examined for a range of tropospheric
Richardson numbers and tropopause depths. Results show that the
hydrostatic approximation radically alters the character of the gravity
wave reflection and transmission through the tropopause, as well as both
the magnitude and distribution of the momentum flux in the troposphere
and stratosphere. Of particular importance is the downstream shift of
momentum flux by the nonhydrostatic component, which can lead to
misinterpretation of momentum flux measurements in both aircraft data
and numerical models. It is found that the nonhydrostatic component is
significant in this strongly sheared environment, even when the mountain
is broad. Thus, even for relatively large-scale topographic forcing, the
hydrostatic assumption may not be justified for gravity wave
calculations.
Scale analysis suggests that use of this "pseudo-incompressible equation' is justified if the Langrangian time scale of the disturbance is large compared with the time scale for sound wave propagation and the perturbation pressure is small compared to the vertically varying mean-state pressure. The mass-balance in the "pseudo-incompressible approximation' accounts for those density perturbations associated (through the equation of state) with perturbations in the temperature field. Density fluctuations associated with perturbations in the pressure field are neglected. The pseudo-incompressible equation is identical to the anelastic continuity equation when the mean stratification is adiabatic. The pseudo-incompressible approximation yields a system of equations suitable for use in nonhydrostatic numerical models. It also permits the diagnostic calculation of the vertical velocity in adiabatic flow, and might also be used to compute the net heating rate in a diabatic flow from extremely accurate observations of the three-dimensional velocity field and very coarse resolution (single sounding) thermodynamic data. -from Author
At the Canadian Meteorological Center (CMC), we are currently developing the future global forecasting Yin-Yang model. In the horizontal we use spherical coordinates on the overset Yin-Yang grid, while in the vertical we use a log-hydrostatic-pressure coordinate on the Charney–Phillips grid. The parametrization of physical processes is kept the same as in the current Global Environmental Multiscale (GEM) operational model. The Yin-Yang global forecast is performed by considering a domain decomposition (a two-way coupling method) between two limited-area models (LAMs) discretized on the two panels of the Yin-Yang grid and using the same time step. Each panel of the Yin-Yang grid system is extended by a static halo region and uses the same fully implicit semi-Lagrangian method as in the GEM operational model to solve its own dynamic core. The spatial and time discretizations are implemented independently on each quasi-uniform latitude–longitude subgrid. The static halo region plays the same role as the piloting region in limited-area modelling. Since the two subgrids of the Yin-Yang grid do not match, the update of the variables in the pilot region is done by cubic Lagrange interpolation. For our model validation, we ran 42 winter and 42 summer cases using analysis from 2008–2009 and we compared five-day forecast results against observations. No noise is seen in the overlap regions during the simulations. Preliminary results presented in this article are encouraging and demonstrate that in comparison with observations the new Yin-Yang system performs as well as the GEM global model. © 2011 Crown in the right of Canada. Published by John Wiley & Sons Ltd.
A deterministic initial‐value test case for dry dynamical cores of atmospheric general‐circulation models is presented that assesses the evolution of an idealized baroclinic wave in the northern hemisphere. The initial zonal state is quasi‐realistic and completely defined by analytic expressions which are a steady‐state solution of the adiabatic inviscid primitive equations with pressure‐based vertical coordinates. A two‐component test strategy first evaluates the ability of the discrete approximations to maintain the steady‐state solution. Then an overlaid perturbation is introduced which triggers the growth of a baroclinic disturbance over the course of several days.
The test is applied to four very different dynamical cores at varying horizontal and vertical resolutions. In particular, the NASA/NCAR Finite Volume dynamics package, the National Center for Atmospheric Research spectral transform Eulerian and the semi‐Lagrangian dynamical cores of the Community Atmosphere Model CAM3 are evaluated. In addition, the icosahedral finite‐difference model GME of the German Weather Service is tested. These hydrostatic dynamical cores represent a broad range of numerical approaches and, at very high resolutions, provide independent reference solutions. The paper discusses the convergence‐with‐resolution characteristics of the schemes and evaluates the uncertainty of the high‐resolution reference solutions. Copyright © 2006 Royal Meteorological Society
A three-time-level semi-Lagrangian global spectral model was introduced operationally at the European Centre for Medium-Range Weather Forecasts in 1991. This paper first documents some later refinements to the three-time-level scheme, and then describes its conversion to a two-time-level scheme. Experimental results are presented to show that the two-time-level scheme maintains the accuracy of its three-time-level predecessor, while being considerably more computationally efficient.
In multidimensional positive definite advection transport algorithm (MPDATA) the leading error as well as the first- and second-order solutions are known explicitly by design. This property is employed to construct refinement indicators for mesh adaptivity. Recent progress with the edge-based formulation of MPDATA facilitates the use of the method in an unstructured-mesh environment. In particular, the edge-based data structure allows for flow solvers to operate on arbitrary hybrid meshes, thereby lending itself to implementations of various mesh adaptivity techniques. A novel unstructured-mesh nonoscillatory forward-in-time (NFT) solver for compressible Euler equations is used to illustrate the benefits of adaptive remeshing as well as mesh movement and enrichment for the efficacy of MPDATA-based flow solvers. Validation against benchmark test cases demonstrates robustness and accuracy of the approach. Copyright © 2005 John Wiley & Sons, Ltd.
Multidimensional positive definite advection transport algorithm (MPDATA) was proposed in the early eighties as a simple positive-definite advection scheme with small implicit diffusion, for evaluating the advection of water-substance constituents in atmospheric cloud models. Over the two decades, MPDATA evolved from an advection scheme into a class of generalized transport algorithms that expand beyond advective transport to alternate PDEs and complete fluid models with a wide range of underlying governing equations. Recently, MPDATA has attracted attention in the context of several mutually-beneficial developments such as (i) quantification of MPDATA implicit turbulence modelling capability in the spirit of monotonically integrated large eddy simulations (MILES), (ii) extensions to flow solvers cast in generalized time-dependent curvilinear coordinates, and (iii) unstructured-grid formulations. The aim of this paper is to assist the special issue on MPDATA methods for fluids by providing an up to date comprehensive review of the approach, including the underlying concepts, principles of implementation, and guidance to the accumulated literature. Copyright © 2005 John Wiley & Sons, Ltd.
With the emergence of non-hydrostatic global dynamical cores, an alternative testing strategy is proposed, where the planetary radius is suitably reduced to capture non-hydrostatic phenomena without incurring the computational cost of actual simulations of weather and climate at non-hydrostatic resolution. The procedure is simple and tests various aspects of the discretized hydrostatic and non-hydrostatic equations in the same setting on a sphere. Furthermore, it facilitates verification against Cartesian-domain analytic solutions and against large-eddy simulation (LES) benchmarks available for limited-area models. The proposed framework is illustrated with examples of inertia–gravity wave dynamics in linear and nonlinear regimes, including flows past idealized mountains, stratified shear flows and critical layers. Finally, an intercomparison of the Held–Suarez climate variability for reduced-size planets is presented, which provides a path for future investigations on the dynamics of convective boundary layers on a sphere. This assesses the ability to adequately capture interactions of large-scale dynamics with intermittent turbulent structures, an important aspect of future weather and climate predictions. Copyright © 2009 Royal Meteorological Society
Drawing from the results of theoretical studies about the behaviour of constant-coefficients semi-implicit schemes, the dynamical kernel of the Aladin–NH spectral limited-area numerical weather prediction (NWP) model has been modified in order to allow for a stable and efficient integration of the fully elastic Euler equations. The resulting dynamical kernel offers the possibility to use semi-Lagrangian transport schemes together with two-time-level discretizations at kilometric scales for NWP purposes. The main characteristics of the adiabatic part of the model formulation and the space and time discretization are described in this article. In order to illustrate the dependence of the results on adjustable parameters of the dynamical kernel, some real-case dynamical-adaptation forecasts performed with a basic physical parameterization package are presented. The results obtained with this model in real-case experiments fully confirm the conclusions drawn in previous numerical analysis studies. The good quality of the results is found to be compatible with a routine exploitation in a NWP framework. The Aladin–NH dynamical kernel has been used in the operational NWP ‘AROME’ model since December 2008 at the kilometric scale, with an appropriate physical parameterization package and data assimilation system. Copyright © 2010 Royal Meteorological Society and Crown Copyright.
An arbitrary finite-volume approach is developed for discretising partial differential equations governing fluid flows on the sphere. Unconventionally for unstructured-mesh global models, the governing equations are cast in the anholonomic geospherical framework established in computational meteorology. The resulting discretisation retains proven properties of the geospherical formulation, while it offers the flexibility of unstructured meshes in enabling irregular spatial resolution. The latter allows for a global enhancement of the spatial resolution away from the polar regions as well as for a local mesh refinement. A class of non-oscillatory forward-in-time edge-based solvers is developed and applied to numerical examples of three-dimensional hydrostatic flows, including shallow-water benchmarks, on a rotating sphere.
We present an advancement in the evolution of MPDATA (multidimensional positive definite advection transport algorithm). Over the last two decades, MPDATA has proven successful in applications using single-block structured cuboidal meshes (viz. Cartesian meshes), while employing homeomorphic mappings to accommodate time-dependent curvilinear domains. Motivated by the strengths of the Cartesian-mesh MPDATA, we develop a new formulation in an arbitrary finite-volume framework with a fully unstructured polyhedral hybrid mesh. In MPDATA, as in any Taylor-series based integration method for PDE, the choice of data structure has a pronounced impact on the technical details of the algorithm. Aiming at a broad range of applications with a large number of control-volume cells, we select a general, compact and computationally efficient, edge-based data structure. This facilitates the use of MPDATA for problems involving complex geometries and/or inhomogeneous anisotropic flows where mesh adaptivity is advantageous. In this paper, we describe the theory and implementation of the basic finite-volume MPDATA, and document extensions important for applications: a fully monotone scheme, diffusion scheme, and generalization to complete flow solvers. Theoretical discussions are illustrated with benchmark results in two and three spatial dimensions.
We have developed an adaptive grid-refinement approach for simulating geophysical flows on scales from micro to planetary. Our model is nonoscillatory forward-in-time (NFT), nonhydrostatic, and anelastic. The major focus in this effort to date has been the design of a generalized mathematical framework for the implementation of deformable coordinates and its efficient numerical coding in a generic Eulerian/semi-Lagrangian NFT format. The key prerequisite of the adaptive grid is a time-dependent coordinate transformation, implemented rigorously throughout the governing equations of the model. The transformation enables mesh refinement indirectly via dynamic change of the metric coefficients, while retaining advantages of Cartesian mesh calculations (speed, low memory requirements, and accuracy) conducted fully in the computational domain. Diverse test results presented in this paper – simulations of a traveling stratospheric inertio-gravity-wave packet (with numerically advected dense-mesh region) and an idealized climate of the Earth (with analytically prescribed adaptive transformations) – demonstrate the potential and the efficacy of the new deformable grid model for tracing targeted flow features and dynamically adjusting to prescribed undulations of model boundaries.
A class of high-resolution schemes established in integration of anelastic equations is extended to fully compressible flows, and documented for unsteady (and steady) problems through a span of Mach numbers from zero to supersonic. The schemes stem from iterated upwind technology of the multidimensional positive definite advection transport algorithm (MPDATA). The derived algorithms employ standard and modified forms of the equations of gas dynamics for conservation of mass, momentum and either total or internal energy as well as potential temperature. Numerical examples from elementary wave propagation, through computational aerodynamics benchmarks, to atmospheric small- and large-amplitude acoustics with intricate wave-flow interactions verify the approach for both structured and unstructured meshes, and demonstrate its flexibility and robustness.
The integrated Forecasting System (IFS) of the European Centre for Medium-range Weather Forecasts (ECMWF) is a spectral weather forecasting model, which daily produces weather forecasts on up to 16 processors of a CRAY C90. This paper describes the shared-memory implementation of the code and the subsequent development that has been carried out in order to generate a parallel version, suitable for a scalable distributed-memory architecture with many processors. Performance results presented for several vector and parallel systems indicate that the parallelization effort has been successful in achieving good performance and high efficiency.