Content uploaded by Michael Tsyrulnikov
Author content
All content in this area was uploaded by Michael Tsyrulnikov on Sep 18, 2016
Content may be subject to copyright.
Consortium
for
Small-Scale Modelling
Technical Report No. 29
A Stochastic Pattern Generator
for ensemble applications
by
M. Tsyrulnikov and D. Gayfulin
July 2016
Deutscher Wetterdienst
MeteoSwiss
Ufficio Generale Spazio Aereo e Meteorologia
EΘNIKH METEΩPOΛOΓIKH ΥΠHPEΣIA
Instytucie Meteorogii i Gospodarki Wodnej
Administratia Nationala de Meteorologie
ROSHYDROMET
Agenzia Regionale per la Protezione Ambientale del Piemonte
Agenzia Regionale per la Protezione Ambientale dell’Emilia-Romagna
Centro Italiano Ricerche Aerospaziali
Amt f¨ur GeoInformationswesen der Bundeswehr
www.cosmo-model.org
Editor: Massimo Milelli, ARPA Piemonte
A Stochastic Pattern Generator
for ensemble applications
M. Tsyrulnikov and D. Gayfulin
Hydrometeorological Center of Russia
11-13 B. Predtechensky Lane
123242 Moscow
Russia
Contents 2
Contents
1 Abstract 5
2 Introduction 5
2.1 Stochastic dynamic prediction . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.2 Modelerrors .................................... 5
2.3 Ensembleprediction ................................ 6
2.4 Practical model error modeling . . . . . . . . . . . . . . . . . . . . . . . . . . 6
3 SPG: Requirements 8
4 The proposed solution 8
5 Tentative first-order SPG model 9
5.1 Physical-spacemodel................................ 9
5.2 Spectral-spacemodel................................ 9
5.3 Stationary spectral-space statistics . . . . . . . . . . . . . . . . . . . . . . . . 10
5.4 Physical-space statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
5.5 Imposing the SPG requirements . . . . . . . . . . . . . . . . . . . . . . . . . . 12
5.5.1 “Proportionality of scales” implies that q=1
2.............. 12
5.5.2 For Var ξ(t, s) to be finite, α(t, s) needs to be a red noise in space . . . 13
5.5.3 Implications for the SPG design . . . . . . . . . . . . . . . . . . . . . 13
6 Higher-order in time model 13
6.1 Motivation and formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
6.2 Stationary spectral-space statistics . . . . . . . . . . . . . . . . . . . . . . . . 14
6.3 Finite-variance criterion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
6.4 Isotropyinspace-time ............................... 15
6.4.1 Spatialisotropy............................... 15
6.4.2 Spatio-temporal spectra . . . . . . . . . . . . . . . . . . . . . . . . . . 16
6.4.3 Continuity of realizations of ξin space-time . . . . . . . . . . . . . . . 17
6.5 Spatio-temporal covariances: the Mat´ern class . . . . . . . . . . . . . . . . . . 17
6.5.1 Spatial correlation functions . . . . . . . . . . . . . . . . . . . . . . . . 18
6.5.2 Temporal correlation functions . . . . . . . . . . . . . . . . . . . . . . 18
6.5.3 Spatio-temporal correlations . . . . . . . . . . . . . . . . . . . . . . . . 19
2
Contents 3
6.6 The final formulation of the SPG model . . . . . . . . . . . . . . . . . . . . . 19
7 Time discrete solver for the third-order in time SPG model 21
7.1 Thespectralsolver................................. 21
7.2 Correction of spectral variances . . . . . . . . . . . . . . . . . . . . . . . . . . 21
7.3 “Warm start”: ensuring stationarity from the beginning of time integration . 22
7.4 Computationalefficiency.............................. 22
7.4.1 Making the time step ∆tdependent on the spatial wavenumber k. . . 22
7.4.2 Introduction of a coarse grid in spectral space . . . . . . . . . . . . . . 23
7.4.3 Numerical acceleration: results . . . . . . . . . . . . . . . . . . . . . . 24
7.5 Examples of the SPG fields . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
8 Application to the COSMO model 25
9 Discussion 28
9.1 Physical-space or spectral-space SPG solver? . . . . . . . . . . . . . . . . . . 28
9.2 ExtensionsoftheSPG............................... 28
10 Conclusions 29
10.1Summary ...................................... 29
10.2Applications..................................... 30
Appendix A Illustration of the “proportionality of scales” property 31
Appendix B Spatio-temporal structure of the driving 4-D noise 33
B.1 Whitenoise..................................... 33
B.2 Spectrum of the white noise on Td........................ 33
B.3 Space-integrated spatio-temporal white noise on Td×R............ 34
B.4 Spatial spectrum of a spatio-temporal white noise . . . . . . . . . . . . . . . . 34
B.5 Spectral decomposition of a white in time and colored in space noise . . . . . 35
B.6 Discretization of the spectral processes ˜αk(t)intime .............. 35
Appendix C Physical-space approximation of the operator √1−λ2∆36
Appendix D Stationary statistics of a higher-order OSDE 39
Appendix E Smoothness of sample paths of the spatial Mat´ern random field
for different ν41
3
Contents 4
Appendix F Stationary statistics of a time discrete higher-order OSDE 43
F.1 First-order numerical scheme . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
F.2 Third-order numerical scheme . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
4
COSMO Technical Report No. 29 5
1 Abstract
A generator of spatio-temporal pseudo-random Gaussian fields that satisfy the “propor-
tionality of scales” property (Tsyroulnikov 2001) is presented. The generator is based on a
third-order in time stochastic differential equation with a pseudo-differential spatial operator
defined on a limited area 2D or 3D domain in the Cartesian coordinate system. The gen-
erated pseudo-random fields are homogeneous and isotropic in space-time. The correlation
functions in any spatio-temporal direction belong to the Mat´ern class. The spatio-temporal
correlations are non-separable. A spectral-space numerical solver is implemented and accel-
erated exploiting properties of real-world geophysical fields, in particular, smoothness of their
spatial spectra. The generator is designed to simulate additive or multiplicative, or other
spatio-temporal perturbations that represent uncertainties in numerical prediction models
in geophysics. The generator is tested with the COSMO model as a source of additive
spatio-temporal perturbations to the forecast model fields.
2 Introduction
2.1 Stochastic dynamic prediction
Since the works of [1] and [2], we know that accounting for the uncertainty in the initial
forecast fields can improve weather (and other geophysical) predictions. Assigning a prob-
ability distribution for the truth at the start of the forecast (instead of using deterministic
initial data) and attempting to advance this distribution in time according to the dynamic
(forecast) model is called stochastic dynamic prediction.
The advantage of the stochastic dynamic prediction paradigm is twofold. First, the resulting
forecast probability distribution provides a valuable measure of the uncertainty in the pre-
diction, leading to probabilistic forecasting and flow-dependent background-error statistics
in data assimilation. Second, for a nonlinear physical model, switching from the determin-
istic forecast to the mean of the forecast probability distribution improves the mean-square
accuracy of the prediction, i.e. can improve the deterministic forecasting.
2.2 Model errors
Since [3], we realize that not only uncertainties in the initial data (analysis errors) matter,
forecast model (including boundary conditions) imperfections also play an important role.
Simulation of model errors is the subject of this study, so we define them now. Let the
forecast model be of the form dx
dt=F(x),(1)
where tis time, xis the vector that represents the (discretized) state of the system, and
Fis the model (forecast) operator. The imperfection of the model Eq.(1) means that the
(appropriately discretized) truth does not exactly satisfy this equation. The discrepancy is
called the model error [e.g. 4]:
ξt=F(xt)−dxt
dt.(2)
The true model error ξtis normally unknown. In order to include model errors in the
stochastic dynamic prediction paradigm, one models ξt(t) as a random process,ξ(t), or, in
COSMO Technical Report No. 29 6
other words, as a spatio-temporal random field ξ(t, s) (where sis the spatial vector). The
probability distribution of ξ(t) is assumed to be known.
Rearranging the terms in Eq.(2), and replacing the unknown ξtwith its stochastic counter-
part ξ, we realize that the resulting model of truth is the stochastic dynamic equation
dx
dt=F(x)−ξ.(3)
Thus, the extended stochastic dynamic prediction (or modeling) paradigm requires two in-
put probability distributions (that of initial errors and that of model errors) and aims to
transform them to the output (forecast) probability distribution.
2.3 Ensemble prediction
Stochastic dynamic modeling of complex geophysical systems is hampered by their high
dimensionality and non-linearity. The output probability distribution appears to be in most
cases analytically intractable. An affordable approximate solution is provided by the Monte-
Carlo method called in geophysics ensemble prediction.
In ensemble prediction, the input uncertainties (i.e. initial and model errors) are represented
by simulated pseudo-random draws from the respective probability distributions. A rela-
tively small affordable number of these draws are fed to the forecast model giving rise to an
ensemble of predictions (forecasts). Members of this ensemble (called ensemble forecasts) are
solutions to Eq.(3) with ξreplaced with simulated pseudo-random draws from the model-
error probability distribution. The ensemble forecasts start from the initial data perturbed
according to the probability distribution of initial errors.
If initial and model errors are sampled from the correct respective distributions, then the fore-
cast ensemble members are draws from the correct probability distribution of the truth given
all available external data (initial and boundary conditions). This mathematically justifies
the ensemble prediction principle. From the practical perspective, members of the forecast
ensemble can be interpreted as “potential truths” consistent with all available information.
In what follows, we concentrate on the model error field ξ(t, s). We briefly review existing
models for ξ(t, s) and then present our stochastic pattern generator, whose goal is to simulate
pseudo-random draws of ξ(t, s) from a meaningful and flexible distribution.
2.4 Practical model error modeling
In meteorology, our knowledge of the actual model error probability distribution is scarce.
Justified stochastic model-error models are still to be devised and verified. In the authors’
opinion, the best way to stochastically represent spatio-temporal forecast-model-error fields
is to treat each error source separately, so that, say, each physical parametrization is ac-
companied with a spatio-temporal stochastic model of its uncertainty. Or, even better, to
completely switch from deterministic physical parameterizations to stochastic ones. There is
a growing number of such developments [see 5, for a review], but the problem is so complex
that we cannot expect it to be solved in the near future. Its solution is further hampered
by the fact that the existing meteorological observations are too scarce and too inaccurate
for model errors to be objectively identified by comparison with measurement data with
satisfactory accuracy [6].
COSMO Technical Report No. 29 7
As a result, in meteorology ad-hoc model-error models are in wide use. The existing ap-
proaches include generating pseudo-random additive or multiplicative perturbations of the
right-hand sides of the model equations [e.g. 7, 8] in the course of forecast. These two
model-error modeling techniques as well as Stochastic Kinetic Energy Backscatter schemes
[9] require a pseudo-random spatio-temporal field as a stochastic input. Stochastic parame-
terization schemes can also demand such fields [see e.g. 10].
The simplest non-constant pseudo-random field is the white noise, i.e. the uncorrelated in
space and time random field. The white noise is the default forcing in stochastic differen-
tial equations [e.g. 11, 12]. Its advantage is the complete absence of any spatio-temporal
structure, it is a pristine source of stochasticity. But in model-error modeling, this lack of
structure precludes its direct use as an additive or multiplicative perturbation field because
model errors are related to the weather pattern and so should be correlated (dependent)
both in space and time. Tsyrulnikov [13] showed in a simulation study that model errors
can exhibit complicated spatio-temporal behavior.
A correlated pseudo-random spatio-temporal field can be easily computed by generating
independent random numbers at points of a coarse spatio-temporal grid and then assigning
each of them to all model grid points within the respective coarse-grid cell [8]. As a result,
the model-grid field becomes correlated in space and time. The decorrelation space and time
scales are, obviously, defined by the respective coarse-grid spacings. This determines the
choice of the coarse grid, e.g. in [8] the spatial grid spacing was about 1000 km and the
temporal one 6 hours). This technique is extremely simple but it suffers from two flaws.
First, the resulting model-grid field appears to be discontinuous and inhomogeneous. Second,
the spatio-temporal structure of the field is not scale dependent, that is, the resulting tem-
poral length scales do not depend on the respective spatial scales. In reality, longer spatial
scales “live longer” than shorter spatial scales, which “die out” quicker. This ‘proportion-
ality of scales’ is widespread in geophysical fields [see 14, and references therein] and other
media, [e.g. 15, p.129], so we believe this property should be represented by model-error
models. Note also that the “proportionality of scales” is a special case of the non-separability
of spatio-temporal covariances. For a critique of simplistic separable space-time covariance
models, see [16, 17, 18] and Appendix A in this report.
Another popular pseudo-random field generation technique in space and time employs a
spectral transform in space and then imposes independent temporal auto-regressions for the
coefficients of the spectral expansion [19, 9, 20, 21, 22]. This technique is more general and
produces homogeneous fields, but the above implementations use the same time scale for
all spatial wavenumbers so that there are still no space-time interactions in the generated
spatio-temporal fields.
In this report, we propose and test a spatio-temporal Stochastic (pseudo-random) Pattern
Generator (SPG) that accounts for the above “proportionality of scales” and imposes mean-
ingful space-time interactions. The SPG operates on a limited-area domain. It is based on
a (spectral-space) solution to a stochastic partial differential equation, more precisely, to a
stochastic differential equation in time with a pseudo-differential spatial operator. In what
follows, we present the technique, examine properties of the resulting spatio-temporal fields in
2D and 3D spatial domains, and describe the numerical scheme. We start with a first-order in
time SPG model. Then we show that this model needs to be modified in order to meet all the
criteria we impose. Eventually, we end up with a third-order in time model. The technique is
implemented as a Fortran program freely available from https://github.com/gayfulin/SPG.
COSMO Technical Report No. 29 8
3 SPG: Requirements
The general requirements are:
•The SPG should produce stationary in time and homogeneous and isotropic in space
Gaussian pseudo-random fields ξ(t, s) in 3D and 2D spatial domains.
•The SPG should be fast enough so that it does not significantly slow down the forecast
model computations.
•Variance as well as spatial and temporal length scales of ξ(t, s) are to be tunable.
We also impose more specific requirements:
1. The random field ξ(t, s) should have finite variance and continuous realizations (sample
paths).
2. The spatio-temporal covariances should obey the “proportionality of scales” principle:
larger (shorter) spatial scales should be associated with larger (shorter) temporal scales
[14].
3. The SPG ansatz should be flexible enough to allow for practicable solutions in both
physical space and spectral space.
4 The proposed solution
We select the general class of linear evolutionary stochastic partial differential equations
(SPDE) as a starting point in the development of the SPG. This choice is motivated by the
flexibility of this class of spatio-temporal models [e.g. 23]. In particular, for an SPDE, it is
relatively easy to introduce inhomogeneity in space and time as well as local anisotropy—
either by changing coefficients of the spatial operator or by changing local properties of the
driving noise. One can also produce non-Gaussian fields by making the random forcing
non-Gaussian [e.g. 24, 25]. Physical-space discretizations of SPDEs lead to sparse matrices,
which give rise to fast numerical algorithms. If an SPDE has constant coefficients, then it
can be efficiently solved using spatial spectral-space expansions.
In this study, we develop the SPG that relies on a spatio-temporal stochastic model with
constant coefficients so that both physical-space and spectral-space solvers can be employed.
To facilitate the spectral-space solution, the general strategy is to define the SPG model on
a standardized spatial domain. The operational pseudo-random fields are then produced by
mapping the generated fields from the standardized domain to the forecast-model domain.
In 3D, the standardized spatial domain is chosen to be triply periodic: the three-dimensional
(3D) unit torus. In 2D, the standardized domain is the 2D unit torus. The 3D and 2D cases
are distinguished by the dimensionality d= 2 or d= 3 in what follows. To simplify the
presentation, the default dimensionality will be d= 3.
COSMO Technical Report No. 29 9
5 Tentative first-order SPG model
5.1 Physical-space model
The random field in question ξ(t, s) is a function of the time coordinate tand the space
vector s:= (x, y, z), where (x, y, z) are the three spatial coordinates. Each of the spatial
coordinates belongs to the the unit circle S1, so that sis on the unit torus T3≡S1×S1×S1
(T2in the 2D case).
We start with the simplest general form of a first-order Markov model:
∂ξ(t, s)
∂t +A ξ (t, s) = α(t, s),(4)
where Ais the spatial linear operator to be specified and αis the driving noise postulated
to be homogeneous in space and white in time.
The SPG is required to be fast, so we choose Ato be a differential operator (because, as we
noted, in this case a physical-space discretization of Agives rise to a very sparse matrix).
Further, since we wish ξ(t, s) to be homogeneous and isotropic in space, we define Ato be a
polynomial of the negated spatial Laplacian:
A:= P(−∆) :=
q
j=0
cj(−∆)j,(5)
where P(x) is the polynomial and qits degree (a positive integer). We will refer to qas
the spatial order of the SPG model. Note that the negation of the Laplacian is convenient
because (−∆) is a non-negative definite operator.
The coefficients cjare selected to ensure that the spatial operator cj(−∆)jhas only
positive eigen-values not close enough to 0, which can be achieved if all cj≥0 and c0>0;
this guarantees stability of the SPG.
The model Eq.(5) appears to be too rich for the purposes of the SPG at the moment, so in
what follows we employ an even more reduced (but still quite flexible) form
A=P(−∆) := µ(1 −λ2∆)q,(6)
where µand λare positive real parameters.
So, we start with the following SPG equation:
∂ξ(t, s)
∂t +µ(1 −λ2∆)qξ(t, s) = α(t, s).(7)
5.2 Spectral-space model
On the torus Td, a Fourier series is an expansion in the basis functions ei(k,s)≡ei(mx+ny+lz),
where the wavevector kis the triple of integer wavenumbers, k:= (m, n, l). We perform the
Fourier decomposition for both α(t, s) and ξ(t, s),
α(t, s) =
k∈Zd
˜αk(t)ei(k,s)(8)
COSMO Technical Report No. 29 10
and
ξ(t, s) =
k∈Zd
˜
ξk(t)ei(k,s)(9)
(where Zdenotes the set of integer numbers) and substitute these expansions into Eq.(7).
Noting that the application of P(−∆) to ei(k,s)returns P(k2) ei(k,s), recalling that P(−∆)
is defined by Eq.(6), and using orthogonality of the basis functions, we obtain that Eq.(7)
decouples into the set of ordinary stochastic differential equations [OSDE, e.g. 11, 12] in
time:
d˜
ξk
dt+ak˜
ξk(t) = ˜αk(t),(10)
where
ak:= µ(1 + λ2k2)q.(11)
From the postulated homogeneity of αin space, its spectral-space coefficients ˜αk(t) are
mutually independent. This is well known for random fields on Rd(where spectra are
continuous), see e.g. Chapter 2 in [26] or section 8 in [27], and can be directly verified
in our case of the fields on the torus (where spectra are discrete). Therefore, for different
wavevectors k, the resulting spectral-space equations, Eqs.(10)–(11), are probabilistically
completely independent from each other. This greatly simplifies the solution of the SPG
equations because instead of handling the complicated SPDE Eq.(7) we have to solve a
number of independent simple OSDEs Eq.(10).
Further, from the postulated whiteness of αin time, all ˜αk(t) are white in time random
processes (see Appendix B.4). So, we may write
˜αk(t) = σkΩk(t),(12)
where Ωk(t) are independent standard white noises, i.e. derivatives of independent standard
Wiener processes Wk(t) such that
Ωk(t)dt= dWk(t) (13)
and σkare intensities of the white-noise processes. In space, σ2
kis proportional to the spatial
spectrum of the driving noise α(t, s), see Eq.(8), Eq.(12), and Appendix B. If σk=const,
then α(t, s) is white both in space and time, otherwise α(t, s) is a colored in space and white
in time noise.
Thus, the first-order SPG model reduces to a series of OSDEs
d˜
ξk+µ(1 + λ2k2)q˜
ξkdt=σkdWk.(14)
For practical purposes the series is truncated, so that k≡(m, n, l) is limited: |m|< mmax,
|n|< nmax, and |l|< lmax , where mmax,nmax, and lmax are the truncation limits. If not
otherwise stated, all the truncation limits are the same and denoted by nmax.
5.3 Stationary spectral-space statistics
Equation (14) is a first-order OSDE with constant coefficients sometimes called the Langevin
equation, see e.g. [12] or Example 4.12 in [11]. Its generic form is
dη+aη dt=σdW, (15)
COSMO Technical Report No. 29 11
where η(t) is the random process in question, aand σare constants, and W(t) is the standard
Wiener process. The solution to Eq.(15) is known as the Ornstein-Uhlenbeck random process,
whose stationary (steady-state) temporal covariance function is
Bη(t) = σ2
2ae−a|t|(16)
[e.g. 11, Example 4.12]. From Eq.(16), it is clear that ahas the meaning of the inverse
temporal length scale τ:= 1/a.
Now, consider the stationary covariance function of the elementary random process ˜
ξk(t),
E˜
ξk(t0)·˜
ξk(t0+t) = bk·Ck(t),(17)
where bkis the variance and Ck(t) the correlation function. According to Eq.(9), ˜
ξkis the
spatial spectral component of the random field in question ξ(t, s). Therefore bk=Var ˜
ξkis
called the spatial spectrum of ξ(t, s). From Eqs.(14) and (16), we have
bk=σ2
k
2µ(1 + λ2k2)q(18)
and
Ck(t) = e−|t|
τk,(19)
where
τk:= 1
ak
=1
µ(1 + λ2k2)q(20)
is the temporal length scale associated with the spatial wavevector k.
Note that by the spectrum (e.g. bk), we always mean the modal spectrum, i.e. the variance
associated with a single basis function (a single wavevector k); the modal spectrum is not
to be confused with the variance (or energy) spectrum.
5.4 Physical-space statistics
In the stationary regime (i.e. after an initial transient period has passed), the above inde-
pendence of the spectral random processes ˜
ξk(t) (see section 5.2) implies that the random
field ξ(t, s) is spatio-temporally homogeneous, i.e. invariant under shifts in space and time:
Eξ(t, s)·ξ(t+ ∆t, s+ ∆s) = B(∆t, ∆s),(21)
where
B(t, s) =
k
bkCk(t) ei(k,s).(22)
In particular, the spatial covariance function is
B(s) = B(t= 0,s) =
k
bkei(k,s),(23)
where it is seen that the spatial spectrum bkis the Fourier transform of the spatial covariance
function B(s). Finally, the variance is
Var ξ=B(t= 0,s=0) =
k
bk.(24)
COSMO Technical Report No. 29 12
5.5 Imposing the SPG requirements
In section 3, we have formulated requirements 1–3 the SPG model should satisfy. In more
specific terms, they imply the following three conditions.
Requirement 1 states, in particular, that
Var ξ < ∞.(25)
Next, the more precise formulation of requirement 2 states that that for large k, the temporal
length scale τkshould be inversely proportional to k:
τk∼1
kas k→ ∞.(26)
Finally, requirement 3 entails that the driving noise α(t, s) should be white not only in time
but also in space:
σk=σ=const.(27)
This is because the physical-space simulation of the discretized white noise is cheap (since
its grid-point values are just independent zero-mean Gaussian pseudo-random variables),
whereas to simulate a non-white noise requires building a model for the noise and solving its
equations, which is normally expensive (and complicates the design of the SPG).
In the rest of this section we show that these three conditions cannot be simultaneously
satisfied for the first-order SPG model Eq.(7).
5.5.1 “Proportionality of scales” implies that q=1
2
We start with condition Eq.(26). Substituting Eq.(20) into Eq.(26) yields
(1 + λ2k2)q∼kas k → ∞ ⇔ q=1
2.(28)
Note that here and elsewhere, boxed equations are the ones that present the key aspects of
our final SPG model.
With q=1
2, the model’s spatial operator Abecomes (see Eq.(6))
A=µ(1 −λ2∆)1
2≡µ1−λ2∆.(29)
This is a pseudo-differential operator [e.g. 28] with the symbol
a(k) := µ1 + λ2k2,(30)
so that the action of Aon the test function φ(s) is defined as follows. First, we Fourier
transform φ(s) getting {˜φk}. Then, ∀k∈Zd, we multiply ˜φkby the symbol a(k). Finally,
we perform the backward Fourier transform of {a(k) ˜φk}retrieving the result, the function
(Aφ)(s).
Obviously, there is no problem with the above fractional negated and shifted Laplacian in
spectral space (as its action on test functions is well defined, see the previous paragraph).
Importantly, the pseudo-differential operator Aappears to have nice properties in physical
space, too. Specifically, Acan be approximated by a discrete-in-space operator which is
COSMO Technical Report No. 29 13
avery sparse matrix, see Appendix C. So, in both spectral space and physical space, the
resulting operator Awith the fractional degree q=1
2is numerically tractable.
The spectral-space SPG model Eq.(14) becomes
d˜
ξk+µ1 + λ2k2˜
ξkdt=σkdWk.(31)
The resulting spatial spectrum is
bk=σ2
k
2µ√1 + λ2k2.(32)
The SPG model becomes
∂ξ(t, s)
∂t +µ1−λ2∆·ξ(t, s) = α(t, s).(33)
Below, we will see that the choice q=1
2implies that besides the “proportionality of scales”,
the generated spatio-temporal field has other nice properties.
5.5.2 For Var ξ(t, s)to be finite, α(t, s)needs to be a red noise in space
Next, we turn to condition Eq.(25) in section 5.5. Substituting Eq.(32) into Eq.(24) yields
Var ξ=1
2µσ2
k
√1 + λ2k2.(34)
Assuming that σkis smoothly varying, we may approximate the sum in this equation with
the integral (where σ(k) = σ(k) for integer wavenumbers), getting
Var ξ≈Rd
σ2
k
√1 + λ2k2dk∝R
σ2
k
√1 + λ2k2kd−1dk. (35)
To check the convergence of the integral in Eq.(35), we examine the k→ ∞ limit (where
√1 + λ2k2∼k). As we know, the integral of this kind converges if the integrand decays
faster than 1
k1+ϵwith some ϵ > 0. This is the case whenever
σ2
k∼1
kd−1+ϵas k→ ∞.(36)
So, to satisfy the requirement Var ξ < ∞, the spectrum σ2
kneeds to be a decaying function of
its argument k. This clearly contradicts to white-in-space-driving-noise condition Eq.(27).
5.5.3 Implications for the SPG design
As we have just seen, the conditions Eqs.(25)–(27) cannot be met by the first-order SPG
model. So, the model Eq.(7) is to be somehow changed. The solution is to increase the
temporal order of the stochastic model.
6 Higher-order in time model
6.1 Motivation and formulation
Equation (36) shows that with the first order SPG model, we could met both the the finite-
variance condition Eq.(25) and the proportionality-of-scales condition Eq.(26) if we specified
COSMO Technical Report No. 29 14
ared in space (i.e. with the decaying spectrum) driving noise α. The red noise can be
obtained by the application of a linear integral operator with the decaying symbol to the
white noise. The problem here is that with the quite rapidly decaying red-noise spectrum
Eq.(36), the support of the physical-space integral operator will be rather large, resulting in
a computationally expensive numerical scheme.
The idea is, instead of introducing of an expensive integral operator with a decaying symbol
to the right-hand side of the model, Eqs.(4) or (7) or (33), to introduce a differential operator
(with a growing symbol) to the left-hand side of the SPG equation. The simplest way of
doing so is to raise the operator ∂/∂t +A, which already acts in our model on ξin the l.h.s.
of Eq.(4), to a power. This implies that the temporal order of the SPG model increases:
∂
∂t +µ1−λ2∆p
ξ(t, s) = α(t, s),(37)
where pis the temporal order of the modified SPG model (a positive integer) and αis white
both in time and in space. In spectral space, the model Eq.(37) becomes, obviously,
d
dt+µ1 + λ2k2p˜
ξk(t) = σΩk(t),(38)
where, we recall, Ωk(t) are mutually independent standard white noises.
The resulting higher-order SPG model satisfies condition Eq.(27) by construction. Now,
we show that the model Eq.(37) can be defined to satisfy the two remaining conditions,
Eqs.(25)–(26).
6.2 Stationary spectral-space statistics
For each k, Eq.(38) is a pth-order in time OSDE. In Appendix D, we examine properties the
generic pth-order OSDE, specifically, the steady-state statistics of its solution. Using Table
2 in Appendix D, we can write down the stationary variance bkand the temporal correlation
function Ck(t) of the solution to Eq.(38), the process ˜
ξk(t):
bk∝σ2
µ2p−1(1 + λ2k2)p−1
2
(39)
and
Ck(t) = 1 + |t|
τk
+r2|t|2
τ2
k
+· ·· +rp−1|t|p−1
τp−1
ke−|t|
τk.(40)
Here r2, . . . , rp−1are real numbers (given for p= 1,2,3 in Table 2) and τkis the temporal
length scale associated with the spatial wavevector k:
τk=1
µ√1 + λ2k2.(41)
Specifically, for the temporal order p= 2, we have
bk|p=2 =σ2
4µ3(1 + λ2k2)3
2
(42)
and
Ck(t)|p=2 =1 + |t|
τke−|t|
τk.(43)
COSMO Technical Report No. 29 15
For the temporal order p= 3, we have
bk|p=3 =3σ2
16µ5(1 + λ2k2)5
2
(44)
and
Ck(t)|p=3 =1 + |t|
τk
+1
3|t|2
τ2
ke−|t|
τk.(45)
From Eq.(41), it is seen that the “proportionality of scales” condition Eq.(26) is indeed
satisfied because τkis indeed inversely proportional to kfor large k. In order to achieve the
desired dependency of τknot only on k(which we already have from Eq.(41)), but also on
λ(the greater is λthe greater should be τk), we parameterize µas
µ=U
λ,(46)
where U > 0 is the velocity-dimensioned tuning parameter. With this parameterization,
λaffects both the spatial length scale of ξ(due to Eq.(39)) and the temporal length scale
(thanks to Eq.(41)). In contrast, Uaffects only the temporal length scale.
6.3 Finite-variance criterion
Substituting bkfrom Eq.(39) into Eq.(24), approximating the sum over the wavevectors by
the integral (as in Eq.(35)), and exploiting the isotropy of the integrand yields
Var ξ≈const ·∞
0
σ2
(1 + λ2k2)p−1
2
kd−1dk, (47)
so that Var ξ < ∞, i.e. the finite-variance condition Eq.(25) is met, whenever
p > d+ 1
2.(48)
Thus, in the higher-order-in-time model Eq.(37) we can rely on the white in space and time
driving noise α(t, s)provided that the temporal order is large enough: in 2D, it is required
that p≥2 whilst in 3D, we have to set up p≥3.
6.4 Isotropy in space-time
In this subsection, we show that, remarkably, q=1
2is the unique spatial order for which the
field ξ(t, s) appears to be isotropic is space-time. In particular, the shape of the correlation
function is the same in any spatial or temporal or any other direction in the spatio-temporal
domain Td×R.
6.4.1 Spatial isotropy
First, we note that spatial isotropy of the random field ξis invariance of its covariance
function B(s) under rotations. If we were in Rdrather than on Td, isotropy of B(s) = B(s),
where s:= |s|, would be equivalent to isotropy of its Fourier transform (spectrum) b(k),
so that the latter would be dependent only on the modulus of the wavevector k, i.e. the
COSMO Technical Report No. 29 16
total wavenumber k:= |k|=√m2+n2+l2. On the torus, spectra are discrete, i.e. m, n, l
take only integer values, so, strictly speaking, b(k) cannot be isotropic there. To avoid this
technical difficulty, we resort to the device used in sections 5.5.2 and 6.3, the approximation
of a sum over the wavevectors by the integral.
Specifically, we assume that b(k) is smooth enough (which is tantamount to the assumption
that B(s) decays on length scales much smaller than the domain’s extents) for the validity
of the approximation
B(s) =
k∈Zd
bkei(k,s)≈Rd
b(k) ei(k,s)dk,(49)
where b(k) is a smooth function of the real vector argument ksuch that ∀k∈Zd,b(k) = bk.
From Eq.(49), it is obvious that B(s) is indeed approximately invariant under rotations
because so is b(k), see Eq.(39).
In the theoretical analysis in this section, we will rely on the approximation Eq.(49) and
thus assume that the “spectral grid” is dense enough for the spatial spectra to be treated as
continuous ones.
6.4.2 Spatio-temporal spectra
Consider the OSDE Eq.(38) in the stationary regime. Following [27, section 8], the stationary
random process can be spectrally represented as the stochastic integral
˜
ξk(t) = R
eiωtZk(dω),(50)
where ωis the angular frequency (temporal wavenumber) and Zis the orthogonal stochastic
measure such that
E|Zk(dω)|2=bk(ω) dω, (51)
where bk(ω) is the spectral density of the process ˜
ξk(t) and, at the same time, the spatio-
temporal spectrum of the field ξ. In the spectral expansion of the driving white noise Ωk(t)
(see Eq.(38)),
Ωk(t) = R
eiωtZΩk(dω),(52)
we have E|ZΩk(dω)|2=const ·dωbecause the white noise has constant spectral density.
Next, we substitute Eqs.(50) and (52) into Eq.(38), getting
(iω+µ1 + λ2k2)pZk(dω) = ZΩk(dω).(53)
In this equation, taking expectation of the squared modulus of both sides yields
[(ω2+µ2(1 + λ2k2)]pbk(ω) = const,(54)
whence, recalling that µ=U/λ and introducing the scaled angular frequency ω′:= ω/U, we
finally obtain
bk(ω′)≡bK∝1
(λ−2+ (ω′)2+k2)p=1
(λ−2+K2)p,(55)
where
K= ( ω
U,k)≡(ω
U, m, n, l) (56)
is the spatio-temporal wavevector.
COSMO Technical Report No. 29 17
From Eq.(55), one can see that with the scaled frequency (note that the change ω→ω/U
corresponds to the change of the time coordinate t→t·U), the spatio-temporal spectrum
bk(ω′) = bKbecomes isotropic in space-time. This implies that the correlation function
of ξin isotropic in space-time as well (with the scaled time coordinate). Note that this
remarkable property can be achieved only with the spatial order q=1
2.
6.4.3 Continuity of realizations of ξin space-time
The functional form of the spatio-temporal spectrum Eq.(55) together with the constraint
Eq.(48) imply that the conditions of Theorem 3.4.3 in [26] are satisfied, so that spatio-
temporal sample paths of the random field ξare almost surely continuous, as we demanded
in section 3, see requirement 1.
6.5 Spatio-temporal covariances: the Mat´ern class
The spatio-temporal field satisfying the p-th order SPG model Eq.(37) has the spatio-
temporal correlation function belonging to the so-called Mat´ern class of covariance functions
[e.g. 29, 30]. To see this, we denote
ν:= p−d+ 1
2>0,(57)
where positivity follows from Eq.(48). Then Eq.(55) rewrites as
bK∝1
(λ−2+K2)ν+d+1
2
.(58)
Note that here d+ 1 is the dimensionality of space-time. Equation (58) indeed presents
the spectrum of the Mat´ern family of correlation functions, see e.g. Eq.(32) in [29]. The
respective isotropic correlation function is given by the equation that precedes Eq.(32) in
[29] or by Eq.(1) in [30]:
B(r)∝(r/λ)νKν(r/λ),(59)
where ris the distance (in our case, the spatio-temporal distance r=(U t)2+s2, with s
being the spatial distance) and Kνis the MacDonald function (the modified Bessel function
of the second kind).
The Mat´ern family is often recommended for use in spatial analysis due to its notable flexibil-
ity with only two free parameters: νand λ, see e.g. [29] and [30]. Specifically, λcontrols the
length scale, whereas ν > 0 determines the degree of smoothness (the higher νthe smoother
sample paths of the random field, for illustration see Appendix E).
Table 1 lists the resulting spatial correlation functions for several combinations of dand p
[see 30, for details].
With the fixed d, the larger pcorresponds, according to Eq.(57), to the larger νand so to
the smoother in space and time field ξ. This allows us to change the degree of smoothness
of the generated field by changing the temporal order of the SPG model.
From the constraint Eq.(48), the minimal temporal order pthat can be used in both 2D
and 3D is equal to 3. This value p= 3 will be used by default in what follows and in the
current SPG computer program.
COSMO Technical Report No. 29 18
Table 1: The spatial correlation functions B(s)for some plausible combinations
of the dimensionality dand the temporal order p
d p ν =p−d+1
2B(s)
2 2 1
2e−s
λ
2 3 3
2(1 + s
λ) e−s
λ
2 4 5
2(1 + s
λ+1
3s
λ2) e−s
λ
3 3 1 s
λK1(s
λ)
0 500 1000 1500
0.0 0.2 0.4 0.6 0.8 1.0
distance
crf
lambda=40 km
lambda=80 km
lambda=125 km
lambda=200 km
Spatial correlation functions. d=2, p=3
0 500 1000 1500
0.0 0.2 0.4 0.6 0.8 1.0
distance
crf
lambda=40 km
lambda=80 km
lambda=125 km
lambda=200 km
Spatial correlation functions. d=3, p=3
Figure 1: The spatial correlation functions for p= 3 in 2D (the left panel) and 3D (the right
panel)—for four spatial length scales indicated in the legend.
6.5.1 Spatial correlation functions
The above spatio-temporal isotropy (see section 6.4.2 and Eq.(59)) means, in particular, that
the spatial correlations are also isotropic. Figure 1 presents the spatial correlation functions
calculated for different length scales in 2D and 3D following Eq.(59). To make the plots more
accessible, it is arbitrarily assumed that the extent of the standardized spatial domain (the
torus) in each dimension equals 3000 km, so that the distance is measured in kilometers.
From Fig.1, one can notice, first, that the actual length scale is indeed well controlled by
the parameter λ. Second, it is seen that in 2D (the left panel), where, according to Eq.(57),
ν=3
2, the correlation functions are somewhat smoother at the origin than in 3D (the right
panel), where ν= 1. This is consistent with the above statement that the greater νthe
smoother the field. But in general, the 2D and 3D spatial correlation functions are quite
similar.
6.5.2 Temporal correlation functions
Equation (59) shows that the spatial and temporal correlations have the same shape. The
latter feature is very nice because atmospheric spectra are known to be similar in the spatial
and in the temporal domain, e.g. the well-known “-5/3” spectral slope law is observed both
in space and time, see e.g. [31, section 23]. So, our SPG does reproduce this observed in the
nature similarity of spatial and temporal correlations.
Figure 2 shows the temporal correlation functions for the truncated (with nmax = 90) spatial
spectral series Eq.(22). It is evident that the temporal length scale is well controlled by
COSMO Technical Report No. 29 19
0 5 10 15 20
0.0 0.2 0.4 0.6 0.8 1.0
time_hours
crf
U=10 m/s
U=15 m/s
U=25 m/s
U=50 m/s
Temporal correlation functions. d=3, p=3
Figure 2: The temporal correlation functions in 3D for the four values of Uindicated in the
legend and λ= 125 km.
the parameter U. Comparing Fig.2 with Fig.1(right), one can observe that the spatial
and temporal correlations indeed have the same shape; the effect of the spatial spectral
truncation, which can cause a difference in the shapes, is barely visible.
6.5.3 Spatio-temporal correlations
Here, we explore the 3D spatio-temporal correlations calculated using Eq.(22) with the max-
imal wavenumbers in all three dimensions equal to nmax = 90, λ= 125 km, and U= 20
m/s. Figure 3 presents the spatial correlation functions for four time lags. Figure 4 displays
the spatio-temporal correlation function.
From both Fig.3 and Fig.4, one can see the noticeable non-separability of the spatio-temporal
covariances. The larger the time lag, the broader the spatial correlations. Note that this is
consistent with the behavior of the spatio-temporal covariances found by [their Fig.8 16] in
real-world wind speed data.
6.6 The final formulation of the SPG model
1. The temporal order of the SPG model is p= 3 .
2. The driving noise α(t, s) is white both in time and space, so that the intensities of the
spectral-space driving noises ˜αk(t) are constant (and equal to σ). The intensity of the
spatio-temporal white noise αis (2π)d/2σ.
The resulting SPG model is
∂
∂t +U
λ1−λ2∆3
ξ(t, s) = α(t, s).(60)
COSMO Technical Report No. 29 20
0 500 1000 1500
0.0 0.2 0.4 0.6 0.8 1.0
distance
crf
dt=0 h
dt=3 h
dt=6 h
dt=12 h
Spatial time−lagged crfs. d=3, p=3
Figure 3: The spatial correlations in 3D for the four time lags indicated in the legend and
U= 20 m/s.
Time
distance
cov
Spatio−temporal covariances
Ranges: t=0...12 h, r=0...750 km
0.1
0.2
0.3
0.4
0.5
0.6
Figure 4: The spatio-temporal covariances.
COSMO Technical Report No. 29 21
In spectral space, each spectral coefficient ˜
ξk(t) satisfies the equation
d
dt+U
λ1 + λ2k23˜
ξk(t) = ˜αk(t) = σΩk(t),(61)
where Ωk(t) are mutually independent complex standard white noise processes.
7 Time discrete solver for the third-order in time SPG model
In physical space, our final evolutionary model Eq.(60) can be discretized using the approx-
imation of the operator √1−λ2∆ proposed in Appendix C. The respective physical-space
solver looks feasible but we do not examine it in this study. Below, we present our basic
spectral-space technique. From this point on, we will consider only the spectral SPG.
7.1 The spectral solver
To numerically integrate the SPG model equations in spectral space, we discretize Eq.(61) us-
ing an implicit scheme. The model operator ( d
dt+ak)3(where, we recall, ak:= U
λ√1 + λ2k2)
is discretized by replacing the time derivative d
dtwith the backward finite difference I −B
∆t,
where ∆tis the time step, Iis the identity operator, and Bis the backshift operator. The
r.h.s. of Eq.(61) (the white noise) is discretized following Appendix B, Eq.(89). As a result,
we obtain the time discrete evolution equation
ˆ
ξk(i) = 1
κ33κ2ˆ
ξk(i−1) −3κˆ
ξk(i−2) + ˆ
ξk(i−3) + σ∆t5
2ζkt,(62)
where i= 0,1,2, . . . denotes the discrete time instance, κ:= 1 + ak∆t, and ζkt∼CN(0,1)
are independent complex standard Gaussian pseudo-random variables (for their definition,
see Appendix B.6). Note that the solution of the time-discrete Eq.(62) is denoted by the
hat, ˆ
ξk(i), in order to distinguish it from the solution of the time-continuous Eq.(61), which
is denoted by the tilde, ˜
ξk(t).
It can be shown that the numerical stability of the scheme Eq.(62) is guaranteed whenever
κ>1, which is always the case because ak>0, see Eq.(11).
Note that the derivation of the numerical scheme for a higher-order (i.e. with p > 3) SPG
model is straightforward: one should just raise the difference operator I−B
∆tto a power higher
than 3.
7.2 Correction of spectral variances
Because of discretization errors, the time discrete scheme Eq.(62) gives rise to the steady-
state spectral variances ˆ
bk:= Var ˆ
ξk(i), which are different from the “theoretical” ones, bk,
given in Eq.(42) or Eq.(44). The idea is to correct ˆ
ξk(i) so that their steady-state variances
coincide with bk. To this end, we derive ˆ
bkfrom Eq.(62), see Eq.(104) in Appendix F, and
then, knowing the “theoretical” bk, we introduce the correction coefficients, bk/ˆ
bk, to be
applied to ˆ
ξk(i). As a result of this correction, Var ˆ
ξk(i) becomes, obviously, equal to the
desired spectral variances bk. This simple device ensures that for any time step, the spatial
spectrum and thus the spatial covariances are perfect. But the temporal correlations do
depend on the time step, this aspect is discussed below in section 7.4.1.
COSMO Technical Report No. 29 22
7.3 “Warm start”: ensuring stationarity from the beginning of time inte-
gration
To start the numerical integration of the third-order scheme Eq.(62) (for any wavevector k),
we obviously need three initial conditions. If the integration is the continuation of a previous
run, then we just take values of ˆ
ξk(i) at the three last time instances ifrom that previous
run; this ensures “continuity” of the resulting trajectory. If we start a new integration, we
have to somehow generate values of ˆ
ξk(i) at i= 1,2,3, let us denote them here as the vector
ξini := (ˆ
ξk(1),ˆ
ξk(2),ˆ
ξk(3))⊤. Simplistic choices like specifying zero initial conditions give
rise to a substantial initial transient period, which distorts the statistics of the generated
field in the short time range.
In order to have the steady-state regime right from the beginning of the time integration and
thus avoid the initial transient period altogether, we simulate ξini as a pseudo-random draw
from the multivariate Gaussian distribution with zero mean and the steady-state covariance
matrix of ˆ
ξk(i). In Appendix F, we derive the components of this 3 ×3 matrix, namely, its
diagonal elements (all equal to the steady-state variance), see Eq.(104), and the lag-1 and
lag-2 covariances, see Eq.(105).
7.4 Computational efficiency
In this subsection, we describe two techniques that allow us to significantly decrease the
computational cost of running the spectral SPG.
7.4.1 Making the time step ∆tdependent on the spatial wavenumber k
For an ordinary differential equation, the accuracy of a finite-difference scheme depends on
the time step. More precisely, it depends on the ratio of the time step ∆tto the temporal
length scale τof the process in question. For high accuracy, ∆t≪τis needed.
In our problem, τkdecays with the total wavenumber k, see Eq.(41). This implies that for
higher k, smaller time steps are needed. To maintain the accuracy across the wavenumber
spectrum, we choose the time step to be a portion of the time scale:
(∆t)k:= γτk.(63)
The less γ, the more accurate and, at the same time, more time consuming the numerical
integration scheme.
We note that in the atmospheric spectra, small scales have, normally, much less variance
(energy) than large scales. But with the constant γ, the computational time would be,
on the contrary, spent predominantly on high wavenumbers (because the latter require a
smaller time step on the one hand and are much more abundant in 3D or 2D on the other
hand). So, to save computer time whilst ensuring reasonable overall (i.e. for the whole range
of wavenumbers) accuracy, we specify γto be wavenumber dependent (growing with the
wavenumber) in the following ad-hoc way:
γk:= γmin + (γmax −γmin)k
kmax 2
,(64)
where γmin and γmax are tunable parameters, k=√m2+n2+l2, and kmax := max k.
COSMO Technical Report No. 29 23
Note that the choice of γdepends on the shape of the temporal correlation function. For
an OSDE of the type defined by Eq.(91) and τdefined to be equal to a−1, the higher is the
order p, the slower is the decay of the temporal correlation function for the same τand thus
the larger is to be γ.
The choice of the “optimal” γmin and γmax is discussed just below in section 7.4.2.
7.4.2 Introduction of a coarse grid in spectral space
Here we propose another technique to reduce the computational cost of the spectral solver.
The technique exploits the smoothness of the SPG spectrum bkEq.(44). This smoothness
allows us to introduce a coarse grid in spectral space and perform the integration of the time
discrete spectral OSDEs Eq.(62) only for those wavevectors that belong to the coarse grid.
The spectral coefficients ˆ
ξk(i) are then interpolated from the coarse grid to the dense (full)
grid in spectral space.
The latter interpolation would introduce correlations between different spectral coefficients
ˆ
ξk(i), which would destroy the spatial homogeneity. In order to avoid this, we employ a device
used to generate so-called surrogate time series [32, section 2.4.1]. At each t, we multiply
the interpolated ˆ
ξk(i) by eiθk, where θkare independent random phases, i.e. independent for
different krandom variables uniformly distributed on the segment [0,2π]. It can be easily
seen that this multiplication removes any correlation between the spectral coefficients.
Note also that the random phase rotation does not destroy the Gaussianity because ˆ
ξk(i) are
complex circularly-symmetric random variables with uniformly distributed and independent
of |ˆ
ξk(i)|arguments (phases), see e.g. [33, section A.1.3].
To preserve the temporal correlations of the field ξ(t, s), we keep the set of θkconstant during
the SPG-model time integration.
The exact spectrum bkafter the trilinear (bilinear in 2D) interpolation of ˆ
ξk(i) from the
coarse to the full spectral grid is imposed in a way similar to that described in section 7.2
as follows. At any time instance when we wish to compute the physical space field, for each
kon the full spectral grid, the linearly interpolated value ˇ
ξkis a linear combination of the
closest coarse-grid points kj:
ˇ
ξk=
2d
j=1
wjˆ
ξkj(65)
whereˇdenotes the interpolated value and wjis the interpolation weight (note that the set
of the closest coarse-grid points kjdepends, obviously, on k). In Eq.(65), the coarse-grid
variances Var ˆ
ξkj=bkjare known for all kjfrom the spectrum {bk}, see Eqs.(42) or (44).
Therefore, we can find Var ˇ
ξk=jw2
jbkj. Besides, we know which variance ˇ
ξkshould have
on the fine grid, namely bk. So, we normalize ˇ
ξkby multiplying it by bk/(Var ˇ
ξk), thus
imposing the exact spatial spectrum for all k.
Technically, the 3D coarse spectral grid is the direct product of three 1D grids. Any of the
(non-uniform) 1D coarse grids is specified as follows. The jth coarse grid point is located at
the fine-grid wavenumber nj, which equals jfor |j| ≤ n0(where n0is an integer) and the
closest integer to n0(1 + ε)|n|−n0for |j|> n0. Here, εis a tunable small positive number.
In the below numerical experiments, the coarse-grid parameters were n0= 20 and ε= 0.2,
which resulted in the following positive 1D coarse-grid points: 1 2 3 . .. 19 20 24 29 35 42
50 60 72 86 103 124 150 (if not otherwise stated, the 1D grid extent was 300 points and,
correspondingly, the maximal wavenumber was 150).
COSMO Technical Report No. 29 24
Figure 5: The theoretical and estimated temporal correlations and CPU times for the 2D
SPG. The legend indicates the range γmin–γmax and whether the coarse grid was used. The
respective spectral-space computation time (per one hour of time integration on one CPU)
is also indicated in the legend. The SPG setup: λ= 85 km, U= 12 m/s.
7.4.3 Numerical acceleration: results
As the two above acceleration techniques guarantee that the spatial spectrum is always
precise, we test how these techniques impact the temporal correlations (and what is the
speedup).
In the numerical experiments, the introduction of the coarse spectral grid impacted the
temporal correlations to a lesser extent than an increase in the time-step parameter γ. So,
we examined the role of the two parameters γmin and γmax of the acceleration technique
described in section 7.4.1, and the impact of the presence or absence of the coarse spectral
grid introduced in section 7.4.2.
Figure 5 shows the temporal correlations functions for the different setups (indicated in the
legend) of the 2D SPG. The respective CPU times for the spectral-space computations (on
one CPU per one hour of model integration) are also shown in the legend. The discrete
backward Fourier transforms were performed every hour of lead time.
From Fig.5, one can see that the combined effect of both numerical acceleration techniques
(the green curve) was dramatic: the speedup was about 80 times as compared to the non-
accelerated scheme (i.e. without the coarse grid and with constant γ= 0.1, the yellowish
curve). The contributions of the two above numerical acceleration techniques to this speedup
were comparable in magnitude (not shown). Most importantly, this big speedup was achieved
at the very little cost: the temporal correlation length scale was distorted, as a result of
approximations caused by the two acceleration techniques, by only some 5 percent w.r.t. the
non-accelerated scheme and some 10 percent w.r.t. the theoretical model (the red curve).
COSMO Technical Report No. 29 25
Figure 6: 2D (x-y) cross-section of a spatial SPG field.
Note that the speedup was for the spectral-space computations only, i.e. it did not include
the cost of the discrete backward Fourier transform.
In summary, the 2D-in-space SPG took only 1 second on one CPU to perform the spectral-
space model integration for as long as 100 hours of lead time. The respective cost of the
backward Fourier transform performed every hour was about 4 seconds. The total cost was
thus 5 seconds.
The same computations performed for the 3D-in-space grid with 300 ×300 ×64 points took
60 seconds on one CPU for 100 hours of spectral-space model integration and 110 seconds
for the hourly backward Fourier transform. The speedup of the spectral-space computations
for the 3D scheme due to the two acceleration techniques was about 50 times.
7.5 Examples of the SPG fields
Figure 6 shows a “horizontal” x-ycross-section and Fig.7 a spatio-temporal x-tcross-section
of a simulated pseudo-random field ξ(t, x, y). Note that in each spatial direction, there were
300 grid points, whilst only 256 contiguous points are shown in the Figures. This is done in
the SPG for practical purposes in order to avoid correlations between the opposite sides of
the spatial domain, which would be spurious in real-world applications.
8 Application to the COSMO model
The SPG was embedded into the Fortran code of the limited-area meteorological non-
hydrostatic model COSMO [34]. Within COSMO, the SPG code was parallelized using
COSMO Technical Report No. 29 26
Figure 7: 2D (abscissa is x, ordinata is t) spatio-temporal cross-section of an SPG field.
MPI. The SPG was used to generate additive perturbations of the model’s right-hand sides.
Figure 8 displays results of a numerical experiment, in which independent SPG perturbation
fields for temperature and horizontal wind were simulated and added to the model fields
every 15 minutes of lead time. The standard deviations of the SPG perturbations were
specified as 0.005 K for temperature and 0.01 m/s for each of the two mutually orthogonal
wind components. Besides, pressure perturbations were computed (and added to the model
pressure 3D field) from the SPG temperature perturbation using the hydrostatic equation
with the zero pressure perturbation at the model’s top level.
The first generated by the SPG meridional wind perturbation field on the model level 25
(about 3 km above the ground) is displayed in Fig.8(top). The forecast (i.e. accumulated
over time) meridional wind perturbation field after 3 hours of model integration is shown in
Fig.8(bottom). Comparing the top and bottom panels of Fig.8 suggests that the perturbation
field becomes more large-scale and, at the same time, less smooth over time. Specifically,
the large-scale structure of the perturbation field becomes more large scale. This is to be
expected because the time integration reduces (filters out) small-scale-in-time components
of the spatio-temporal field and so, from the “proportionality of scales”, filters out smaller
scales also in space.
Besides, an opposite effect, namely appearance of a kind of fine structure can be seen in the
forecast perturbation fields, see Fig.8(bottom). In particular, in the forecast perturbation
field, there are many very localized features, which presumably develop due to a non-linear
interaction with convection and effects related to steep orography. Finally, we note that
in contrast to the “input” SPG model-error perturbation field, the “output” forecast per-
turbation field is clearly seen to be flow dependent (as desired in ensemble prediction and
ensemble data assimilation schemes).
Figure 9 displays the growth of the meridional wind RMS forecast perturbation at the same
model level 25 in response to the above SPG model-error perturbations. One can see that the
magnitude of the forecast perturbation grows monotonically and “regularly” (as expected).
Finally, we note that at the time of writing, the numerical acceleration techniques described
in section 7.4 were implemented only in the stand-alone version of the SPG. Without those
accelerators, the cost of running the SPG within COSMO was about 0.8% of the total
COSMO model wall-clock time per generated SPG field.
COSMO Technical Report No. 29 27
Figure 8: The top panel: The first SPG V-wind perturbation added to the COSMO field at
the lead time 15 minutes. The bottom panel: the COSMO 3-h forecast V-wind perturbation
field in response to the additive SPG perturbations of temperature, pressure, and both wind
components added every 15 minutes of lead time.
COSMO Technical Report No. 29 28
Figure 9: The COSMO forecast RMS V-wind perturbation in response to the SPG forcing.
9 Discussion
9.1 Physical-space or spectral-space SPG solver?
In this study, we have investigated both the spectral-space and the physical-space approx-
imations of the SPG spatio-temporal model. We have found that both approaches can be
used to build a practical SPG scheme. We have selected the spectral-space technique. Here,
we briefly compare both approaches.
Advantages of the spectral-space technique are the following.
•Simplicity of realization. If the SPG model has constant coefficients, then the compli-
cated SPG equation decouples into a series of simple OSDEs.
•Straightforward accommodation of non-local-in-physical-space spatial operators.
Advantages of the physical-space approach are:
•The relative ease of introduction of inhomogeneous and anisotropic capabilities to the
SPG.
•The SPG solver can be implemented in domains with complex boundaries.
•Better suitability for an efficient implementation on massively parallel computers.
9.2 Extensions of the SPG
The proposed SPG technique can be extended in the future along the following lines.
•Development of a physical-space solver.
COSMO Technical Report No. 29 29
•Introduction of advection to the SPG model.
•Introduction of spatial inhomogeneity/anisotropy and non-stationarity.
•Introduction of non-Gaussianity. This can be done either by applying a nonlinear
transform to the output SPG fields, or by introducing a non-Gaussian driving noise
[as in 24, 25]. The former approach is simpler but the latter allows for much richer
deviations from Gaussianity, including the multi-dimensional aspect.
•Going beyond additive and multiplicative perturbations for highly non-Gaussian vari-
ables like humidity and cloud fields.
•Simulation of several mutually correlated pseudo-random fields.
•Making the temporal order pa user defined variable. As noted above, the larger pthe
smoother the generated field.
10 Conclusions
10.1 Summary
•The proposed Stochastic Pattern Generator (SPG) produces pseudo-random spatio-
temporal Gaussian fields on 2D and 3D spatial domains.
•The SPG model is defined on a standardized domain in space, specifically, on the unit
2D or 3D torus. Fields on a limited-area geophysical domain in question are obtained
by mapping from the standardized domain.
•The SPG is based on a linear third-order in time stochastic model driven by the white
in space and time Gaussian noise.
•The spatial operator of the stochastic model is built to ensure that solutions to the
SPG model, i.e. the generated pseudo-random fields satisfy the “proportionality of
scales” property: large-scale (small-scale) in space field components have large (small)
temporal length scales.
•Besides the “proportionality of scales”, the generated fields possess a number of other
nice properties:
–The spatio-temporal realizations are (almost surely) continuous.
–With the appropriately scaled time coordinate, the spatio-temporal fields are
isotropic in space-time.
–The spatial and temporal correlation functions belong to the Mat´ern class.
–The spatial and temporal correlations have the same shape.
•It is shown that the spatial operator of the SPG model can be effectively discretized
both in physical space and spectral space.
•The basic SPG solver is spectral-space based.
•Two techniques to accelerate the spectral-space computations are proposed and im-
plemented. The first technique selects the time step of the spectral-space numerical
integration scheme to be dependent on the wavenumber, so that the discretization er-
ror is smaller for more energetic larger spatial scales and is allowed to be larger for
COSMO Technical Report No. 29 30
less energetic smaller scales. The second technique introduces a coarse grid in spectral
space. The combined speedup for spectral-space computations from both techniques
is as large as 50–80 times.
•The SPG is embedded in the COSMO model as a source of additive model-error pertur-
bations. The numerical experiments show that the forecast perturbations in response
to the SPG model-error perturbations behave as expected.
•For a peer reviewed publication on the SPG, see [35].
10.2 Applications
Potential applications of the SPG include ensemble prediction and ensemble data assimilation
in meteorology, oceanography, hydrology, and other areas.
The SPG can be used to generate spatio-temporal perturbations of the model fields (in the
additive or multiplicative or other mode), and of the boundary conditions.
Acknowledgements
The SPG has been developed as part of the Priority Project KENDA (Kilometer scale
Ensemble Data Assimilation) of COSMO. We have used the discrete fast Fourier package
fft991 developed by C. Temperton at ECMWF in 1978. Christoph Schraff and Michael
Baldauf kindly helped us to interpret the appearance of the small-scale noise in the COSMO
forecast perturbation fields.
COSMO Technical Report No. 29 31
A Illustration of the “proportionality of scales” property
Figure 10 shows a realization of the spatio-temporal field with non-separable correlations
that satisfy the “proportionality of scales” property (the top panel) and a realization of the
field with separable spatio-temporal correlations (the bottom panel).
The non-separable spatio-temporal correlation function is defined as B(x, t) = exp(−r/L),
where r:= x2+ (Ut)2,U= 10 m/s, L= 200 km, and the domain size in the xdirection
is 3000 km. The separable correlation function is B(x, t) = exp(−|x|/L)·exp(−|Ut|/L). So,
both separable and non-separable fields have exactly the same spatial correlation functions
and the same temporal correlation functions.
Note that both the separability and the exponential temporal correlation function is what
the scale-independent first-order auto-regression used in [19, 9, 20, 21, 22] implies.
Comparing the two panels of Fig.10, one can see that the two fields are quite different. In the
non-separable case, Fig.10(top), large spatial structures indeed tend to live longer than small
structures, as it is expected from the “proportionality of scales” principle. In contrast, in
the case with separable space-time correlations, Fig.10(bottom), the “longevity” of a spatial
pattern is rather independent of its size (which is unphysical). Besides, in the non-separable
case, a kind of spatio-temporal “organization” is evident, which is absent in the separable
case. Finally, the field with non-separable correlations exhibits a sort of spatio-temporal
isotropy, again, not visible in the separable case.
COSMO Technical Report No. 29 32
0.0 0.2 0.4 0.6 0.8 1.0
0.0 0.2 0.4 0.6 0.8 1.0
PS
x
t
−3
−2
−1
0
1
2
3
0.0 0.2 0.4 0.6 0.8 1.0
0.0 0.2 0.4 0.6 0.8 1.0
Sep
x
t
−3
−2
−1
0
1
2
3
Figure 10: Simulated spatio-temporal fields. Top: With non-separable space-time correla-
tions satisfying the “proportionality of scales” principle. Bottom: With separable space-time
correlations.
COSMO Technical Report No. 29 33
B Spatio-temporal structure of the driving 4-D noise
Here, we recall the general definition of the white noise, define the spatial spectrum of the
white noise on the d-dimensional unit torus, and find its spatial spectral decomposition in
the spatio-temporal case. Then we introduce a colored in space and white in time noise,
and find it spatial spectrum. Finally, we define the time discrete complex-valued white-noise
process.
B.1 White noise
By definition, see e.g. [36, section 1.1.3] or [37, section 3.1.4], the standard white noise Ω(x)
defined on a manifold Dis a generalized random field that acts on a test function φ(x) (where
x∈D) as follows:
(Ω, φ) := φ(x) Ψ(dx),(66)
where Ψ is the Gaussian orthogonal stochastic measure such that for any Borel set A, Ψ(A)
is a (complex, in general) Gaussian random variable with EΨ(A) = 0 and E|Ψ(A)|2=|A|,
where |.|denotes the Lebesgue measure.
The equivalent definitions of the standard white noise are
E|(Ω, φ)|2:= |φ(x)|2dx,(67)
and
E(Ω, φ)·(Ω, ψ) := φ(x)ψ(x) dx,(68)
where ψis another test function. Thus, we have defined of the standard white noise. By the
general Gaussian white noise, we mean a multiple of the standard white noise.
B.2 Spectrum of the white noise on Td
The formal Fourier transform of the spatial white noise Ω(s) (where s∈T3),
˜
Ωk=1
(2π)dTd
Ω(s) e−i(k,s)ds,(69)
can be rigorously justified as the action of the white noise Ω on the test function
χ(s) := 1
(2π)de−i(k,s).(70)
Then, the spatial spectrum of Ω(s) is
bk:= E|˜
Ωk|2≡E|(Ω, χ)|2=Td|χ(s)|2ds=1
(2π)d.(71)
Here, the third equality is due to Eq.(67). We stress that it is the modal spectrum that
is constant for the white noise (not the variance spectrum), see also remark at the end of
section 5.3.
COSMO Technical Report No. 29 34
B.3 Space-integrated spatio-temporal white noise on Td×R
Let us consider the spatio-temporal white noise Ω = Ω(t, s), where t∈Ris time and s∈Td
the spatial coordinate vector. Take a spatial test function c(s) and define the temporal
process Ω1(t) formally as
Ω1(t) := Td
Ω(t, s)c(s) ds,(72)
so that it acts on a test function in the temporal domain, φ(t), as
(Ω1, φ) := R
Ω1(t)φ(t) dt=RTd
Ω(t, s)c(s)φ(t) dsdt. (73)
Here, we note that the latter double integral is nothing other than the result of action of the
original white noise Ω(t, s) on the spatio-temporal test function c(s)·φ(t). This enables us
to mathematically rigorously define Ω1(t) as the generalized random process that, with the
fixed c(s), acts on the test function φ(t) as follows:
(Ω1(t), φ(t)) := (Ω(t, s), c(s)φ(t)).(74)
Now, using the definition Eq.(67) of the white noise Ω(t, s), we have
E|(Ω(t, s), c(s)φ(t))|2=TdR|c(s)|2|φ(t)|2dsdt=Td|c(s)|2dsR|φ(t)|2dt. (75)
Since we have fixed c(s), we observe that
σ2:= |c(s)|2ds(76)
is a constant such that
E|(Ω1(t), φ(t))|2=σ2R|φ(t)|2dt. (77)
Comparing this equation with one of the definitions of the standard white noise, Eq.(67), we
recognize Ω1(t) as a general Gaussian white noise in time, i.e. the standard temporal white
noise multiplied by σ. We call σthe intensity of the white noise.
B.4 Spatial spectrum of a spatio-temporal white noise
Now, we are in a position to derive the spatial spectrum of the standard spatio-temporal
white noise Ω(t, s). In the formal Fourier decomposition
Ω(t, s) =
k
˜
Ωk(t) ei(k,s),(78)
the elementary temporal processes ˜
Ωk(t) can be shown to be white noises in time. Indeed,
again formally, we have
˜
Ωk(t) = 1
(2π)dTd
Ω(t, s) e−i(k,s)ds.(79)
Here, we recognize an expression of the kind given by Eq.(72) with c(s) := e−i(k,s)/(2π)d.
Therefore, from Eq.(77), ˜
Ωk(t) is a temporal white noise with the intensity σΩ
ksquared equal
to
(σΩ
k)2=|c(s)|2ds=1
(2π)2dTd|e−i(k,s)|2ds=1
(2π)d.(80)
COSMO Technical Report No. 29 35
In addition, using Eq.(68), it is easy to show that ˜
Ωk(t) and ˜
Ωk′(t) are mutually orthogonal
for k=k′.
To summarize, ˜
Ωk(t) are mutually orthogonal white-in-time noises, all with equal intensities
σΩ
k= (2π)−d/2:
˜
Ωk(t) = 1
(2π)d/2Ωk(t),(81)
where Ωk(t) are the standard white noises.
B.5 Spectral decomposition of a white in time and colored in space noise
In order to introduce a white in time and colored in space noise, let us convolve the spatio-
temporal white noise Ω(t, s) with a smoothing kernel in space u(s), getting
α(t, s) := Td
u(s−r) Ω(t, r) dr.(82)
In this equation, the stochastic integral is defined, for any tand s, following Eq.(74) with
c(r) := u(s−r). Fourier transforming u(s),
u(s) =
k
˜ukei(k,s),(83)
and, in space, α(t, s),
α(t, s) =
k
˜αk(t) ei(k,s),(84)
we easily obtain that the elementary spectral processes ˜αk(t) are independent white noises
in time with the intensities squared
σ2
k= (2π)d|˜uk|2,(85)
so that the stochastic differential ˜αk(t)dtis
˜αk(t) dt=σkdWk(t).(86)
Equivalently,
˜αk(t) = σkΩk(t).(87)
B.6 Discretization of the spectral processes ˜αk(t)in time
Being white noises, ˜αk(t) have infinite variances. They become ordinary random processes
if, e.g., we discretize them in time. With the time step ∆t, we define the discretized process
ˆαk(tj) at the time instance tjby replacing, in Eq.(86), dtwith ∆tand dWkwith ∆Wk:
ˆαk(tj) ∆t:= σk∆Wk(t).(88)
As E|∆Wk(t)|2= ∆t, we obtain
ˆαk(tj) = σk
√∆t·ζkj,(89)
where ζkjare independent complex standard Gaussian random variables CN (0,1). The
latter is defined as a complex random variable whose real and imaginary parts are mutually
uncorrelated zero-mean random variables with variances equal to 1/2. CN (0,1) is sometimes
referred to as circularly symmetric complex Gaussian (normal) random variable [e.g. 33].
Equation (89) shows that the spatial spectrum of the time discrete driving noise is σ2
k/∆t.
COSMO Technical Report No. 29 36
C Physical-space approximation of the operator √1−λ2∆
As we have discussed in section 5.5.1, the fractional power (square root) of the negated and
shifted Laplacian operator, L:= √1−λ2∆, is defined as the pseudo-differential operator
with the symbol ˜
l(k) := √1 + λ2k2. In the literature, one can find approaches to discretiza-
tion of fractional powers of elliptic operators, e.g. finite elements were used in the spatial
context in [38].
Here, we propose a simple technique to build a spatial discretization scheme that approxi-
mates the operator √1−λ2∆ in the sense that the symbol of the approximating operator is
close to √1 + λ2k2.
To this end, we do the following.
1. Perform the backward Fourier transform of the symbol ˜
l(k), getting the function l(s).
As multiplication in Fourier space by ˜
l(k) is equivalent to convolution in physical space
with l(s), we obtain that for any test function φ(s),
(Lφ)(s) = T3
l(s−r)φ(r) dr.(90)
The crucial moment here is that the kernel function l(s) appears to be oscillating while
rapidly decreasing in modulus as |s|increases (see below). This enables its efficient
approximation with a compact-support (truncated) function.
2. With the discretization on the grid with npoints in each of the ddimensions on the
torus Td, the kernel function l(s) is represented by the set of its grid-point values l(si),
where s= (s1, . . . , sd), i= (i1, . . . , id), and si= (s1(i1), . . . , sd(id)). If l(s) appears to
be rapidly decreasing away from s=0, we truncate the l(si) function by limiting its
support near the origin, thus getting the function ltrunc(si). E.g. in 3D, the support
of ltrunc(si) consists of the grid points i= (i1, i2, i3) that simultaneously satisfy the
following constraints: |i1| ≤ J,|i2| ≤ J, and |i3| ≤ J, where Jis the spatial order of
the scheme. Below, we present results with J= 1 (3 grid points in the support of the
truncated kernel function in each dimension) and J= 3 (7 grid points in the support
in each dimension).
3. Fourier transform ltrunc(s) back to the spectral space, getting the approximated symbol
˜
ltrunc(k).
4. Compare ˜
l(k) with ˜
ltrunc(k) and conclude whether a parsimonious (that is, with a very
small J) approximation is viable.
Now, we present the results. We found that for d= 1, d= 2, and d= 3, the goodness of fit
was similar, so we examine the 3D case below.
We selected the grid of n= 2 ·nmax = 256 points in each of the three dimensions. We
specified the spatial non-dimensional length scale λto be much greater than the mesh size
h= 2π/n and much less than the domain’s extents, 2π. Specifically, we chose λ= 1/n1,
where n1:= √nmax. (The results were not much sensitive to changes in n1within the whole
wavenumber range on the grid.)
Figure 11 displays the resulting kernel function l(s) for positive s(note that l(s) is an even
function of the scalar distance s). One can see the remarkably fast decay of |l(s)|with the
growing s. Consequently, a stencil with just a few points in each dimension can be expected
to work well.
COSMO Technical Report No. 29 37
Figure 11: The kernel function
Figure 12 shows the exact and approximated symbols for the stencil that contains 3 grid
points in each dimension (the left panel) and the stencil that contains 7 grid points in each
dimension (the right panel). (The 5-point scheme worked not much better than the 3-point
one and so its performance is not shown.)
From Fig.12, one can see that the 3-point scheme’s performance is rather mediocre, whereas
the 7-point scheme works very well (in terms of the reproduction of the operator’s symbol).
Finally, we verified that the symbol ˜
ltrunc(k) of the discrete operator for J= 3,5,7 was every-
where positive, which guarantees that the operator is positive definite and so the discretized
SPG model should be stable.
To summarize, the operator √1−λ2∆ can be approximated with parsimonious physical-
space discretization schemes. For simulation of uncertainty in meteorology, where precise
0 20 40 60 80 100 120
1 2 3 4 5
wavenumber + 1
symbol
Exact (solid), 3−point fin−diff (circles)
0 20 40 60 80 100 120
1 2 3 4 5 6
wavenumber + 1
symbol
Exact (solid), 7−point fin−diff (circles)
Figure 12: Goodness of fit of the symbol ˜
l(k) (solid curve) by ˜
ltrunc(k) (circles). Left: the
3-point stencil in each dimension. Right: the 7-point stencil
COSMO Technical Report No. 29 38
error statistics is not available, the simplest 3-point (in each direction) scheme seems most
appropriate and computationally attractive. For more demanding applications, the 7-point
scheme can be more appropriate.
COSMO Technical Report No. 29 39
D Stationary statistics of a higher-order OSDE
We examine the OSDE Eq.(38) in its generic form:
d
dt+ap
η(t) = σΩ(t),(91)
where σand aare the positive numbers, pis the positive integer, and Ω(t) is the standard
white noise (see Appendix B).
The goal here is to find the variance and the correlation function of η(t) in the stationary
regime. The technique is to reduce the p-th order OSDE to a system of first-order OSDEs.
To simplify the exposition, we consider the third-order OSDE (p= 3) and rewrite Eq.(91)
as d
dt+a d
dt+a d
dt+aη(t)=σΩ(t).(92)
Here, by η1we denote the term in brackets,
d
dt+aη=: η1(93)
and by η2the term in braces,
d
dt+aη1=: η2,(94)
so that the original equation Eq.(91) can be rewritten as
d
dt+aη2=σΩ.(95)
In Eqs.(93)–(95), the last equation is the familiar first-order OSDE forced by the white
noise, whereas the other equations are not forced by the white noise. Generalizing the
above construction, Eqs.(92)–(95), to the arbitrary p > 0, we form the following first-order
vector-matrix OSDE (a system of first-order OSDEs):
dη+Aηdt=ΣΩdt, (96)
where η:= (η, η1, . . . , ηp−2, ηp−1), Ω:= (0,0, . . . , 0,Ω) , and the design of the matrices A
and Σis obvious (not shown).
With Eq.(96) in hand, we derive a differential equation for the covariance matrix P:= Eηη∗,
where ∗denotes transpose complex conjugate [see e.g. 11, example 4.16]. First, we compute
the increment of P:
∆P=E(η+ dη)(η+ dη)∗−Eηη∗=Eηdη∗+Edηη∗+Edηdη∗.(97)
Then, using Eq.(96) and the fact that E|Ωdt|2=E|dW|2= dt, we obtain the differential of
Pfrom Eq.(97):
dP=−APdt−PA∗dt+ΣΣ∗dt. (98)
In the stationary regime dP= 0, so the equation for the stationary covariance matrix is
AP +PA∗=ΣΣ∗.(99)
Next, we look at the first diagonal entry of the resulting stationary covariance matrix P,
which represents the required Var η(because ηis defined above to be the first entry of the
COSMO Technical Report No. 29 40
vector η). Dropping tedious derivations, we present in Table 2 (second row) the formulas
for the temporal orders p= 1, p= 2, p= 3, and for the general p.
Finally, we derive the temporal correlation function for the pth-order OSDE. To this end,
we multiply Eq.(91) by η(s) with s < t and take expectation. Since ais non-stochastic,
we may interchange the expectation and the differential operator d
dt+ap, getting the pth-
order ordinary differential equation for the temporal covariance function, whose solutions for
different pare presented in row 3 of Table 2.
Table 2: The variances Var ηand correlation functions Cη(t)of the stationary
solution to Eq.(91) for different temporal orders p
p1 2 3 Arbitrary p
Var ησ2
2a
σ2
4a33σ2
16a5∝σ2
a2p−1
Cη(t) e−a|t|(1 + a|t|) e−a|t|(1 + a|t|+a2t2
3) e−a|t|∝Rp−1(a|t|)·e−a|t|
Here Rp−1(x) is a polynomial of order p−1.
COSMO Technical Report No. 29 41
E Smoothness of sample paths of the spatial Mat´ern random
field for different ν
Here, we show how sample paths (realizations) of the Mat´ern random field with the smooth-
ness parameter νlook. Specifically, in Fig.13, we present three plots with 1D cross-sections of
randomly chosen realizations of the Mat´ern random field for the following three values of ν:
1/2, 3/2, and 5/2. The spatial length scale parameter λis selected in each of the three cases
in such a way that the spatial correlation function intersects the 0.7 level at approximately
the same distance (we denote this distance by L0.7): L0.7= 500 km. Again, as in section
6.5 and Appendix A, we assume, for convenience, that the extent of the spatial domain in
each coordinate direction is 3000 km (rather than 2π). For comparison, we also display a
realization with L0.7= 1500 km for ν= 1/2 (the bottom panel of Fig.13).
One can see that, indeed, the larger ν, the smoother the realizations—in the sense that they
have less small-scale “noise”. By contrast, increasing the length scale λ(compare the top
and bottom panels of Fig.13) makes the large-scale pattern smoother but does not remove
the smallest scales. So, the large-scale behavior is determined by the length scale λ, whereas
the degree of small-scale smoothness/roughness depends predominantly on the smoothness
parameter ν.
COSMO Technical Report No. 29 42
0 100 200 300 400 500 600 700
−3−2−1 0 1 2
Sample path. crf= exp
Index
Xe[, 3]
0 100 200 300 400 500 600 700
−2−1 0 1 2 3
Sample path. crf= soar
Index
Xe[, 1]
0 100 200 300 400 500 600 700
−2−1 0 1 2
Sample path. crf= toar
Index
Xe[, 3]
0 100 200 300 400 500 600 700
−3−2−1 0 1
Sample path. crf= exp
Index
Xe[, 3]
Figure 13: Sample paths for various νand L0.7. From the top to the bottom:
[ν=1
2,L0.7= 500], [ν=3
2,L0.7= 500], [ν=5
2,L0.7= 500], [ν=1
2,L0.7= 1500]
COSMO Technical Report No. 29 43
F Stationary statistics of a time discrete higher-order OSDE
Here, to simplify the exposition, we first examine the simplest first-order (i.e. with p= 1)
OSDE and then give the results for the third-order OSDE used in the current version of the
SPG.
F.1 First-order numerical scheme
Discretization of the Langevin Eq.(15) by an implicit scheme yields
ηi−ηi−1+aηi∆t=σ∆Wi,(100)
so that
ηi=ηi−1+σ∆Wi
1 + a∆t.(101)
In the stationary regime, Var ηi=Var ηi−1, whence, bearing in mind that Var ∆Wi= ∆tand
∆Wiis independent on the values of ηfor all time moments up to and including the moment
i−1, we apply the variance operator to both sides of Eq.(101) and obtain the stationary
variance
V(∆t) := lim
i→∞ Var ηi=σ2
2a+ (g∆t)2.(102)
Note that, as ∆t→0, V(∆t) tends to the continuous-time variance σ2
2a, see Eq.(16).
F.2 Third-order numerical scheme
Consider the continuous-time OSDE, Eq.(91), with p= 3. The implicit scheme Eq.(62) we
use to numerically solve it is reproduced here as
ηi=1
κ33κ2ηi−1−3κηi−2+ηi−3+σ(∆t)2∆Wi,(103)
where κ:= 1+a∆t. Here, the goal is to find the stationary variance V:= limi→∞ Var ηialong
with lag-1 and lag-2 stationary covariances, c1:= limi→∞ Eηiηi−1and c2:= limi→∞ Eηiηi−2,
respectively. To reach this goal, we build three linear algebraic equations for the three
unknowns, V,c1, and c2. The first equation is obtained by applying the variance operator to
both sides of Eq.(103). The second and third equations are obtained by multiplying Eq.(103)
by ηi−1and ηi−2, respectively, and applying the expectation operator to both sides of the
resulting equations. Omitting the derivations, we write down the results:
V=κ4+ 4κ2+ 1
(κ2−1)5(∆t)5σ2.(104)
c1=3κ(κ2+ 1)
(κ2−1)5(∆t)5σ2, c2=6κ2
(κ2−1)5(∆t)5σ2.(105)
As in the first-order case, one can see that as ∆t→0, Vtends to the continuous-time
variance 3
16
σ2
a5, see Table 2 in Appendix D.
COSMO Technical Report No. 29 44
References
[1] E.S. Epstein. Stochastic dynamic prediction. Tellus, 21(6):739–759, 1969.
[2] B.V. Tatarsky. The use of dynamic equations for a probabilistic forecasting of the
pressure fields. Izv. Akad. Nauk SSSR, FAO, 5:293–297, 1969.
[3] E.J. Pitcher. Application of stochastic dynamic prediction to real data. J. Atmos. Sci.,
34(1):3–21, 1977.
[4] D. Orrell, L. Smith, J. Barkmeijer, and T.N. Palmer. Model error in weather forecasting.
Nonlin. Proc. Geophys., 8(6):357–371, 2001.
[5] J. Berner, U. Achatz, L. Batte, A. De La Camara, D. Crommelin, H. Christensen,
M. Colangeli, S. Dolaptchiev, C.L. Franzke, P. Friederichs, et al. Stochastic pa-
rameterization: Towards a new view of weather and climate models. arXiv preprint
arXiv:1510.08682, 2015.
[6] M. Tsyrulnikov and V. Gorin. Are atmospheric-model tendency errors perceivable from
routine observations? COSMO Newsletter No. 13, pages 3–18, 2013.
[7] P.L. Houtekamer, H.L. Mitchell, and X. Deng. Model error representation in an opera-
tional ensemble Kalman filter. Mon. Wea. Rev., 137(7):2126–2143, 2009.
[8] R. Buizza, M. Miller, and T.N. Palmer. Stochastic representation of model uncertainties
in the ECMWF ensemble prediction system. Q. J. R. Meterol. Soc., 125(560):2887–2908,
1999.
[9] J. Berner, G.J. Shutts, M. Leutbecher, and T.N. Palmer. A spectral stochastic ki-
netic energy backscatter scheme and its impact on flow-dependent predictability in the
ECMWF ensemble prediction system. J. Atmos. Sci., 66(3):603–626, 2009.
[10] L. Bengtsson, M. Steinheimer, P. Bechtold, and J.F. Geleyn. A stochastic parametriza-
tion for deep convection using cellular automata. Q. J. R. Meterol. Soc., 139(675):
1533–1543, 2013.
[11] A.H. Jazwinski. Stochastic processes and filtering theory. Academic Press, 1970.
[12] L. Arnold. Stochastic differential equations. Wiley, 1974.
[13] M. Tsyrulnikov. Stochastic modelling of model errors: A simulation study. Q. J. R.
Meterol. Soc., 131(613):3345–3371, 2005.
[14] M. Tsyroulnikov. Proportionality of scales: An isotropy-like property of geophysical
fields. Q. J. R. Meterol. Soc., 127(578):2741–2760, 2001.
[15] N. Meunier and J. Zhao. Observations of photospheric dynamics and magnetic fields:
from large-scale to small-scale flows. Space Sci. Rev., 144(1-4):127–149, 2009.
[16] N. Cressie and H.C. Huang. Classes of nonseparable, spatio-temporal stationary covari-
ance functions. J. Amer. Statist. Assoc., 94(448):1330–1339, 1999.
[17] M.L. Stein. Space–time covariance functions. J. Amer. Statist. Assoc., 100(469):310–
321, 2005.
[18] T. Gneiting, M.G. Genton, and P. Guttorp. Geostatistical space-time models, station-
arity, separability, and full symmetry. Monographs Statist. Appl. Probab., 107:151, 2006.
COSMO Technical Report No. 29 45
[19] X. Li, M. Charron, L. Spacek, and G. Candille. A regional ensemble prediction system
based on moist targeted singular vectors and stochastic parameter perturbations. Mon.
Wea. Rev., 136(2):443–462, 2008.
[20] N.E. Bowler, A. Arribas, S.E. Beare, K.R. Mylne, and G.J. Shutts. The local ETKF
and SKEB: Upgrades to the MOGREPS short-range ensemble prediction system. Q. J.
R. Meterol. Soc., 135(640):767–776, 2009.
[21] T.N. Palmer, R. Buizza, F. Doblas-Reyes, T. Jung, M. Leutbecher, G.J. Shutts,
M. Steinheimer, and A. Weisheimer. Stochastic parametrization and model uncertainty.
ECMWF Tech. Memo. n. 598, ECMWF, Shinfield Park, 42 pp., 2009.
[22] Bouttier F., Vi´e B., O. Nuissier, and L. Raynaud. Impact of stochastic physics in a
convection-permitting ensemble. Mon. Wea. Rev., 140(11):3706–3721, 2012.
[23] F. Lindgren, H. Rue, and J. Lindstr¨om. An explicit link between Gaussian fields and
Gaussian Markov random fields: the stochastic partial differential equation approach.
J. Roy. Statist. Soc. B, 73(4):423–498, 2011.
[24] S. ˚
Aberg and K. Podg´orski. A class of non-Gaussian second order random fields. Ex-
tremes, 14(2):187–222, 2011.
[25] J. Wallin and D. Bolin. Geostatistical modelling using non-Gaussian Mat´ern fields.
Scand. J. Statist., 42(3):872–890, 2015.
[26] R.J. Adler. The geometry of random fields. Wiley, 1981.
[27] A.M. Yaglom. Correlation theory of stationary and related random functions, Volume
1: Basic results. Springer Verlag, 1987.
[28] M.A. Shubin. Pseudodifferential operators and spectral theory. Springer, 1987.
[29] M.L. Stein. Interpolation of spatial data: some theory for kriging. Springer, New York,
1999.
[30] P. Guttorp and T. Gneiting. Studies in the history of probability and statistics XLIX.
On the Matern correlation family. Biometrika, 93(4):989–995, 2006.
[31] A.S. Monin and A.M. Yaglom. Statistical fluid mechanics, Volume II: Mechanics of
turbulence. Courier Corp., 2013.
[32] J. Theiler, S. Eubank, A. Longtin, B. Galdrikian, and J.D. Farmer. Testing for nonlin-
earity in time series: the method of surrogate data. Physica D, 58(1-4):77–94, 1992.
[33] D. Tse and P. Viswanath. Fundamentals of wireless communication. Cambridge Uni-
versity Press, 2005.
[34] M. Baldauf, A. Seifert, J. F¨orstner, D. Majewski, M. Raschendorfer, and T. Reinhardt.
Operational convective-scale numerical weather prediction with the COSMO model:
description and sensitivities. Mon. Wea. Rev., 139(12):3887–3905, 2011.
[35] M. Tsyrulnikov and D. Gayfulin. A limited-area spatio-temporal stochastic pattern
generator for ensemble prediction and ensemble data assimilation. Meteorol. Zeitschrift
(under review), 2016.
[36] Y.A. Rozanov. Markov random fields. Springer, 1982.
COSMO Technical Report No. 29 46
[37] H.H. Kuo. White noise theory. In “Handbook of stochastic analysis and applications”
Kannan D. and Lakshmikantham V. (Eds.), pages 107–158, 2001.
[38] D. Simpson, F. Lindgren, and H. Rue. Think continuous: Markovian Gaussian models
in spatial statistics. Spatial Statist., 1:16–29, 2012.
COSMO Technical Report No. 29 47
List of COSMO Newsletters and Technical Reports
(available for download from the COSMO Website: www.cosmo-model.org)
COSMO Newsletters
No. 1: February 2001.
No. 2: February 2002.
No. 3: February 2003.
No. 4: February 2004.
No. 5: April 2005.
No. 6: July 2006.
No. 7: April 2008; Proceedings from the 8th COSMO General Meeting in Bucharest, 2006.
No. 8: September 2008; Proceedings from the 9th COSMO General Meeting in Athens, 2007.
No. 9: December 2008.
No. 10: March 2010.
No. 11: April 2011.
No. 12: April 2012.
No. 13: April 2013.
No. 14: April 2014.
No. 15: July 2015.
No. 16: July 2016.
COSMO Technical Reports
No. 1: Dmitrii Mironov and Matthias Raschendorfer (2001):
Evaluation of Empirical Parameters of the New LM Surface-Layer Parameterization
Scheme. Results from Numerical Experiments Including the Soil Moisture Analysis.
No. 2: Reinhold Schrodin and Erdmann Heise (2001):
The Multi-Layer Version of the DWD Soil Model TERRA LM.
No. 3: G¨unther Doms (2001):
A Scheme for Monotonic Numerical Diffusion in the LM.
No. 4: Hans-Joachim Herzog, Ursula Schubert, Gerd Vogel, Adelheid Fiedler and Roswitha
Kirchner (2002):
LLM ¯ the High-Resolving Nonhydrostatic Simulation Model in the DWD-Project LIT-
FASS.
Part I: Modelling Technique and Simulation Method.
COSMO Technical Report No. 29 48
No. 5: Jean-Marie Bettems (2002):
EUCOS Impact Study Using the Limited-Area Non-Hydrostatic NWP Model in Oper-
ational Use at MeteoSwiss.
No. 6: Heinz-Werner Bitzer and J¨urgen Steppeler (2004):
Documentation of the Z-Coordinate Dynamical Core of LM.
No. 7: Hans-Joachim Herzog, Almut Gassmann (2005):
Lorenz- and Charney-Phillips vertical grid experimentation using a compressible non-
hydrostatic toy-model relevant to the fast-mode part of the ’Lokal-Modell’.
No. 8: Chiara Marsigli, Andrea Montani, Tiziana Paccagnella, Davide Sacchetti, Andr´e Walser,
Marco Arpagaus, Thomas Schumann (2005):
Evaluation of the Performance of the COSMO-LEPS System.
No. 9: Erdmann Heise, Bodo Ritter, Reinhold Schrodin (2006):
Operational Implementation of the Multilayer Soil Model.
No. 10: M.D. Tsyrulnikov (2007):
Is the particle filtering approach appropriate for meso-scale data assimilation ?
No. 11: Dmitrii V. Mironov (2008):
Parameterization of Lakes in Numerical Weather Prediction. Description of a Lake
Model.
No. 12: Adriano Raspanti (2009):
COSMO Priority Project ”VERification System Unified Survey” (VERSUS): Final Re-
port.
No. 13: Chiara Marsigli (2009):
COSMO Priority Project ”Short Range Ensemble Prediction System” (SREPS): Final
Report.
No. 14: Michael Baldauf (2009):
COSMO Priority Project ”Further Developments of the Runge-Kutta Time Integration
Scheme” (RK): Final Report.
No. 15: Silke Dierer (2009):
COSMO Priority Project ”Tackle deficiencies in quantitative precipitation forecast”
(QPF): Final Report.
No. 16: Pierre Eckert (2009):
COSMO Priority Project ”INTERP”: Final Report.
No. 17: D. Leuenberger, M. Stoll and A. Roches (2010):
Description of some convective indices implemented in the COSMO model.
No. 18: Daniel Leuenberger (2010):
Statistical analysis of high-resolution COSMO Ensemble forecasts in view of Data As-
similation.
No. 19: A. Montani, D. Cesari, C. Marsigli, T. Paccagnella (2010):
Seven years of activity in the field of mesoscale ensemble forecasting by the COSMO–
LEPS system: main achievements and open challenges.
No. 20: A. Roches, O. Fuhrer (2012):
Tracer module in the COSMO model.
COSMO Technical Report No. 29 49
No. 21: Michael Baldauf (2013):
A new fast-waves solver for the Runge-Kutta dynamical core.
No. 22: C. Marsigli, T. Diomede, A. Montani, T. Paccagnella, P. Louka, F. Gofa, A. Corigliano
(2013):
The CONSENS Priority Project.
No. 23: M. Baldauf, O. Fuhrer, M. J. Kurowski, G. de Morsier, M. M¨ullner, Z. P. Piotrowski,
B. Rosa, P. L. Vitagliano, D. W´ojcik, M. Ziemia´nski (2013):
The COSMO Priority Project ’Conservative Dynamical Core’ Final Report.
No. 24: A. K. Miltenberger, A. Roches, S. Pfahl, H. Wernli (2014):
Online Trajectory Module in COSMO: a short user guide.
No. 25: P. Khain, I. Carmona, A. Voudouri, E. Avgoustoglou, J.-M. Bettems, F. Grazzini
(2015):
The Proof of the Parameters Calibration Method: CALMO Progress Report.
No. 26: D. Mironov, E. Machulskaya, B. Szintai, M. Raschendorfer, V. Perov, M. Chumakov,
E. Avgoustoglou (2015):
The COSMO Priority Project ’UTCS’ Final Report.
No. 27: J-M. Bettems (2015):
The COSMO Priority Project ’COLOBOC’: Final Report.
No. 28: Ulrich Blahak (2016):
RADAR MIE LM and RADAR MIELIB - Calculation of Radar Reflectivity from Model
Output.
COSMO Technical Report No. 29 50
COSMO Technical Reports
Issues of the COSMO Technical Reports series are published by the COnsortium for Small-
scale MOdelling at non-regular intervals. COSMO is a European group for numerical weather
prediction with participating meteorological services from Germany (DWD, AWGeophys),
Greece (HNMS), Italy (USAM, ARPA-SIMC, ARPA Piemonte), Switzerland (MeteoSwiss),
Poland (IMGW), Romania (NMA) and Russia (RHM). The general goal is to develop, im-
prove and maintain a non-hydrostatic limited area modelling system to be used for both
operational and research applications by the members of COSMO. This system is initially
based on the COSMO-Model (previously known as LM) of DWD with its corresponding data
assimilation system.
The Technical Reports are intended
•for scientific contributions and a documentation of research activities,
•to present and discuss results obtained from the model system,
•to present and discuss verification results and interpretation methods,
•for a documentation of technical changes to the model system,
•to give an overview of new components of the model system.
The purpose of these reports is to communicate results, changes and progress related to the
LM model system relatively fast within the COSMO consortium, and also to inform other
NWP groups on our current research activities. In this way the discussion on a specific
topic can be stimulated at an early stage. In order to publish a report very soon after the
completion of the manuscript, we have decided to omit a thorough reviewing procedure and
only a rough check is done by the editors and a third reviewer. We apologize for typographical
and other errors or inconsistencies which may still be present.
At present, the Technical Reports are available for download from the COSMO web site
(www.cosmo-model.org). If required, the member meteorological centres can produce hard-
copies by their own for distribution within their service. All members of the consortium will
be informed about new issues by email.
For any comments and questions, please contact the editor:
Massimo Milelli
Massimo.Milelli@arpa.piemonte.it
