Handling nonlinearities and uncertainties of fed-batch cultivations with difference of convex functions tube MPC

Abstract

Bioprocesses are often characterized by nonlinear and uncertain dynamics. This poses particular challenges in the context of model predictive control (MPC). Several approaches have been proposed to solve this problem, such as robust or stochastic MPC, but they can be computationally expensive when the system is nonlinear. Recent advances in optimal control theory have shown that concepts from convex optimization, tube-based MPC, and difference of convex functions (DC) enable stable and robust online process control. The approach is based on systematic DC decompositions of the dynamics and successive linearizations around feasible trajectories. By convexity, the linearization errors can be bounded tightly and treated as bounded disturbances in a robust tube-based MPC framework. However, finding the DC composition can be a difficult task. To overcome this problem, we used a neural network with special convex structure to learn the dynamics in DC form and express the uncertainty sets using simplices to maximize the product formation rate of a cultivation with uncertain substrate concentration in the feed. The results show that this is a promising approach for computationally tractable data-driven robust MPC of bioprocesses.

Full text

HANDLING NONLINEARITIES AND UNCERTAINTIES OF
FED-BATCH CULTIVATIONS WITH DIFFERENCE OF CONVEX
FUNCTIONS TUBE MPC
A PREPRINT
Niels Krausch
Technische Universität Berlin
Bioprocess engineering
ACK24, Ackerstr. 76, 13355 Berlin, Germany
n.krausch@tu-berlin.de
Martin Doff-Sotta
University of Oxford
Department of Engineering Science
Parks Road, Oxford, UK
martin.doff-sotta@eng.ox.ac.uk
Mark Cannon*
University of Oxford
Department of Engineering Science
Parks Road, Oxford, UK
mark.cannon@eng.ox.ac.uk
Peter Neubauer
Technische Universität Berlin
Bioprocess engineering
ACK24, Ackerstr. 76, 13355 Berlin, Germany
peter.neubauer@tu-berlin.de
Mariano Nicolas Cruz Bournazou
Technische Universität Berlin
Bioprocess engineering
ACK24, Ackerstr. 76, 13355 Berlin, Germany
mariano.n.cruzbournazou@tu-berlin.de
December 7, 2023
ABS TRAC T
Bioprocesses are often characterized by nonlinear and uncertain dynamics. This poses particular
challenges in the context of model predictive control (MPC). Several approaches have been proposed
to solve this problem, such as robust or stochastic MPC, but they can be computationally expensive
when the system is nonlinear. Recent advances in optimal control theory have shown that concepts
from convex optimization, tube-based MPC, and difference of convex functions (DC) enable stable
and robust online process control. The approach is based on systematic DC decompositions of the
dynamics and successive linearizations around feasible trajectories. By convexity, the linearization
errors can be bounded tightly and treated as bounded disturbances in a robust tube-based MPC
framework. However, finding the DC composition can be a difficult task. To overcome this problem,
we used a neural network with special convex structure to learn the dynamics in DC form and express
the uncertainty sets using simplices to maximize the product formation rate of a cultivation with
uncertain substrate concentration in the feed. The results show that this is a promising approach for
computationally tractable data-driven robust MPC of bioprocesses.
Keywords Robust tube MPC ·Data-driven control ·Convex optimization ·Bioprocesses
∗Corresponding author

DC-TMPC for cultivations PREPRINT
1 Introduction
1.1 Rapid bioprocess development
The accelerating demand for cost-effective production of biologic drugs and sustainable biomaterials intensifies the
need for rapid bioprocess development. This is particularly true in the early project stages, characterized by limited
process information and a broad spectrum of potential optimal conditions. Advanced control approaches like MPC
coupled with online parameter estimation of the model have proven to be successful even when incomplete process
information is available (Krausch et al., 2022), but have been restricted to relatively stable process conditions. For
example, Kager, Tuveri, Ulonska, Kroll, and Herwig (2020) were able to increase total product formation in a fungal
process, but their approach is limited to the nominal case. Mowbray, Petsagkourakis, Del Rio Chanona, and Zhang
(2022) used Neural Networks (NN) to deal with uncertainties but required heavy offline training.
1.2 Tube-based MPC with difference of convex functions
A popular approach in advanced control to deal with uncertain dynamic systems is tube-based MPC (TMPC). TMPC has
been mainly applied to linear systems because nonlinear robust MPC requires online solution of nonconvex optimization
problems, which can be computationally expensive. A common strategy for applying TMPC to nonlinear systems is
to treat the nonlinearity as bounded disturbances of the system and perform successive linear approximations around
predicted trajectories. These approaches, nevertheless, rely on conservative estimates of the linearization error and can
lead to poor performance (Yu, Maier, Chen, & Allgöwer, 2013). Recent studies have shown that tighter bounds on the
linearization error can be achieved if the problem can be expressed as a difference of convex functions (Doff-Sotta
& Cannon, 2022). This is based on the observation that the necessarily convex linearization error is maximum at the
boundary of the set on which it is evaluated. Tight bounds can thus be derived and treated as disturbances in a robust
TMPC framework. Moreover, the DC structure of the dynamics is attractive as it results in a sequence of convex
programs that can be solved with predictable computational effort. Even though any twice continuously differentiable
function can be expressed in DC form, finding such functions can be a difficult task. To solve this problem, we have
harnessed an NN by restricting the kernel weights to non-negative values and used a convex activation function (ReLU)
leading to a so-called input-convex NN (ICNN) (Amos, Xu, & Kolter, 2017). Two ICNNs can thus be stacked and
their output subtracted to learn the dynamics of the function in DC form (Sankaranarayanan & Rengaswamy, 2022).
Moreover, in the context of TMPC, the parameterization of the tube plays an important role in the computational
complexity of the optimization problem. Doff-Sotta and Cannon (2022) propose state tube cross sections parameterized
by elementwise bounds, yielding
2nx+1
(with
nx
the number of states) inequality constraints and causing a significant
computational burden for large number of states. In this regard, using simplex tubes is a computationally efficient
alternative with only
nx+ 1
inequality constraints. Hence, this contribution describes a TMPC algorithm leveraging a
NN for learning the dynamics in DC form, implementing a simplex tube and optimizing product formation in a case
study of a fed-batch bioreactor for the production of penicillin.
2 Modelling and DC approximation with neural networks
Let us consider a perfectly mixed isothermal fed-batch bioreactor, a popular case study example from Srinivasan,
Bonvin, Visser, and Palanki (2003). The model states are the cell concentration X [
g L−1
], product concentration P
[g L−1], substrate concentration S [g L−1] and volume V [L]. The input is the feed flow rate F [L h−1] of S. The inlet
substrate concentration Si∈[180,220] g L−1is an uncertain parameter. The dynamics of the system are given by
˙
X=µ(S)X−F
VX
˙
S=−µ(S)X
YX/S
−vX
YP/S
+F
V(Si−S)
˙
P=vX −F
VP
˙
V=F
(1)
where
µ(S) = µmax S
S+KS+S2/Ki
and
µmax
denotes the maximal growth rate (0.02
h−1
),
KS
the affinity constant
of the cells towards the substrate (0.05
g L−1
),
Ki
an inhibition constant which inhibits growth at high substrate
concentrations (5
g L−1
), v the production rate (0.004
L h−1
),
YX/S
the yield coefficient of biomass per substrate (0.5
gX/gS
) and
YP/S
the yield coefficient of product per substrate (1.2
gP/gS
). The initial conditions are
X(0) = 1 g L−1
,
2

DC-TMPC for cultivations PREPRINT
S(0) = 0.5 g L−1,P(0) = 0 g L−1and V(0) = 120 L.
An NN framework was used to approximate the nonconvex dynamics as a difference of convex functions by
subtracting the outputs of two ICNN subnetworks. An ICNN with
L
layers is characterized by a parameter set
θ= Θ1:L−1,Φ0:L−1, b0:L−1and input-output map given by zL=f(y;θ), defined ∀l∈ {0, . . . , L −1}by
zl+1 =σ(Θlzl+ Φlx+bl)(2)
where
y
is the input,
zl
is the layer activation,
Θl
are positively constrained kernel weights
({Θl}ij )≥0∀l∈
{1, ..., L −1}
,
Φl
are input passthrough weights,
bl
are bias and
σ(·)
is a convex activation function (ReLU). Each
layer of an ICNN thus consists in the composition of a convex function with a nondecreasing convex function, which
implies that
zl+1 =f(y;θ)
is convex with respect to
y
. Choosing
zl+1 = ˙x
and
y= (x, u)
, where
x= (X, S, P, V )
and
u=F
are the state and input of (1), two ICNN whose outputs are subtracted can be trained simultaneously to learn
the nonconvex dynamics in (1) as a difference of (elementwise) convex functions f1,f2:
˙x=f1(x, u)−f2(x, u)(3)
The two ICNNs each consist of a single input layer, two hidden layers with 64 nodes each and an output layer. The
network was implemented in Keras and trained over 10 epochs with the RMSProp optimizer on 100,000 random
samples of (1), which were divided into 80% training and 20% validation sets. Convexity of the models was evaluated
by checking the numerical Hessian matrix of the functions for positive semidefiniteness, i.e.
∇2fi(x, u;θ)⪰0,∀x∈
Rnx,∀i={1,2}
. Figure 1 depicts a 3D projection of the DC decomposition for fixed values of the states and input.
As illustrated, the NN was able to obtain a good fit (MAE: 0.016) for the predictions of the ODEs (blue dots and blue
surface), and the DC form of the decomposition is apparent (orange and green surfaces).
Figure 1: DC decomposition. Depicted are the results from the actual model (blue dots), the results from the DC
decomposition
f=f1−f2
(blue plane) and the respective DC part convex functions
f1
(orange) and
f2
(green) at
a given product concentration, volume and feed rate for two states. Left: Biomass
˙
X
Right: Substrate
˙
S
. Each in
dependence of different concentrations of Xand S.
3 DC-TMPC framework with simplices
Doff-Sotta and Cannon (2022) proposed a robust TMPC algorithm based on successive linearisation for DC systems.
The so-called DC-TMPC algorithm capitalises on the idea that the successive linearisation steps yield necessarily convex
linearisation error functions that can be bounded tightly and treated as disturbances by a robust MPC scheme. We
extend that approach to nonconvex systems learned in DC form and consider a state tube parameterized by simplices to
reduce computational burden. The system in DC form in (3) is discretized and successively linearized around previously
computed predicted trajectories
x◦
k
,
u◦
k
with state and input perturbations
sk=xk−x◦
k
and
vk=uk−u◦
k
. As per the
TMPC paradigm,
vk
is parameterized by a two degree of freedom control law
vk=Kksk+ck
where
Kk
is a feedback
gain and
ck
is a feedforward control sequence computed at every time step. The sequence of sets
Sk∀sk,∀k
, defines the
cross sections of an uncertainty tube in which the system trajectories lie under all realisations of the uncertainty and
whose dynamics are given by
sk+1 = (Φ1,k −Φ2,k)sk+ (B1,k −B2,k )ck+g1(sk, ckx◦
k, u◦
k)−g2(sk, ckx◦
k, u◦
k)(4)
3

DC-TMPC for cultivations PREPRINT
where for
i= 1,2
,
gi=fi(x◦
k+sk, u◦
k+Kksk+ck)−fi(x◦
k, u◦
k)−Φi,ksk−Bi,k ck
are the (necessarily convex)
linearization errors of
fi, A(i, k) = ∂fi
∂x (x◦
k, u◦
k)
,
Bi,k =∂fi
∂u (x◦
k, u◦
k)
and
Φi,k =Ai,k +Bi,kKk
. While the approach
in Doff-Sotta and Cannon (2022) was to parameterize the tube with elementwise bounds, resulting in an exponential
increase of the inequality constraints, we consider here parameterizations of Skin terms of simplices
Q(sk)≤αk, Q =−I
1T(5)
where
I∈Rnx×nx
is the identity matrix,
1∈Rnx×1
is a vector of ones. The vector
αk∈R(nx+1)×1
is an optimization
variable. Consequently, the state perturbation dynamics can now be expressed as
nx+ 1
inequalities as follows,
combining (4) and (5)
max
s∈V(Sk)(−Φ1,k s−B1,k ck+f2(x◦
k+s, u◦
k+Kks+ck)−f2(x◦
k, u◦
k)) ≤[αk+1]1:nx
max
s∈V(Sk)1T(f1(x◦
k+s, u◦
k+Kks+ck)−f1(x◦
k, u◦
k)−Φ2,ks−B2,k ck)≤[αk+1 ]nx+1
(6)
where the simplex vertices are
V(Sk) = {−[αk]1:nx,−[αk]1:nx+e1σk,...,−[αk]1:nx+enσk}
,
σk= [αk]nx+1 +
1T[αk]1:nx
, and
e1, . . . , en
are the standard basis vectors of
R(nx)
. To obtain (6), we exploited the convexity of
gi
to obtain a tight lower bound on
αk
. Moreover, we note that (6) are convex inequalities by convexity of
fi
and that
each maximum operation can be reduced to a discrete search over the vertices
V(Sk)
since the maximum of a convex
function on a polytope occurs at one of the vertices.
We design a TMPC controller to optimize the feedforward sequence
ck
and tube sets
Sk
subject to (6) and
xk∈ X ⊂
Rnx,uk∈ U ⊂ Rnu,∀k. At each iteration we solve
min
c,α
N−1
X
k=0
max
s∈V(Sk)||x◦
k+s−xr||2
Q+ max
s∈V(Sk)||u◦
k+Kks−ur||2
R
s.t.
∀k∈ {0, . . . , N −1},∀s∈ V(Sk) : max
s∈V(Sk)(−Φ1,k s−B1,k ck+f2(x◦
k+s, u◦
k+Kks+ck)−f2(x◦
k, u◦
k)) ≤[αk+1]1:nx
max
s∈V(Sk)1T(f1(x◦
k+s, u◦
k+Kks+ck)−f1(x◦
k, u◦
k)−Φ2,ks−B2,k ck)≤[αk+1 ]nx+1
x◦
k+s∈ X , u0
k+Kks+ck∈ U, α0= 0
(7)
with a shrinking horizon
N
. The solution from (7) is used to update the state and input guess trajectories
(x◦, u◦)
at
next iteration with
s0←0
u◦
k←u◦
k+ck+Kksk
sk+1 ←f(x◦
k, u◦
k)−x◦
k+1
xk+1 ←f(x◦
k, u◦
k)
(8)
We run (7) and (8) repeatedly until
PN−1
k=0 ||ck||2< ϵtol
or a maximum number of iterations is reached. The control
input is then implemented at time
n
by
u[n] = u◦
0
. At time
n+ 1
, we set
x◦
0=x[n+ 1]
and the guess trajectory is
updated by
u◦
k←u◦
k+1
x◦
k+1 ←f(x◦
k, u◦
k)
uN−1←K(x◦
N−1−xr) + ur
x◦
N←f(x◦
N−1, u◦
N−1)
(9)
4

DC-TMPC for cultivations PREPRINT
4 Results and discussion
The proposed control algorithm was simulated on the batch reactor problem over a shrinking horizon of 20 h with a step
size of 1 h using CVXPY and solver MOSEK. As shown in Figure 2, the controller was able to maximize the product
concentration with parametric uncertainty of the substrate concentration in the feed, demonstrating the applicability
of this approach to complex nonlinear systems with monod-type nonlinear substrate affinity and substrate inhibition,
making the search for an optimal feed rate non-trivial. The presented DC-TMPC algorithm outperforms nominal MPC
approaches for this case study (Lucia & Engell, 2013), by considering the uneven substrate concentration in the feed by
augmenting the NN with the uncertain parameter, considering that the worst case scenario occurs at the vertices of the
parameter set. Further tuning is however necessary, to find an optimal trade-off between substrate concentration in the
reactor to avoid overfeeding (Pimentel, Benavides, Dewasme, Coutinho, & Wouwer, 2015).
Figure 2: Results from the DC-TMPC optimization.
5 Conclusion
In this study, we show that successive linearization robust tube MPC can be an adequate tool to optimize a bioprocess
under parametric uncertainty. Our approach was to decompose the nonconvex dynamics as a difference of convex
functions (DC) using a neural network with convex structure and treat the necessarily convex linearization errors as
bounded disturbances. Crucially, by convexity, these bounds are tight and the resulting controller is less conservative
than classical TMPC based on successive linearization. This approach is relatively new and has so far only been applied
to problems that already exist in DC form. Moreover, by using tubes parameterized with simplex sets, the computational
effort could be significantly reduced, making it attractive for real-time optimization. Future work will incorporate more
complex models and test it in a real-world bioprocess with online optimization of production in a fast-growing E. coli
strain.
Acknowledgments We gratefully acknowledge the financial support of the German Federal Ministry of Education and
Research (BMBF) (project no. 01DD20002A – KIWI Biolab) and the EPSRC (UKRI) Doctoral Prize scheme (grant
reference number EP/W524311/1).
References
Amos, B., Xu, L., & Kolter, J. Z. (2017). Input convex neural networks. In D. Precup & Y. W. Teh (Eds.), Proceedings
of the 34th international conference on machine learning (Vol. 70, pp. 146–155). PMLR. Retrieved from
https://proceedings.mlr.press/v70/amos17b.html
Doff-Sotta, M., & Cannon, M. (2022). Difference of convex functions in robust tube nonlinear mpc. In 2022 ieee 61st
conference on decision and control (cdc) (pp. 3044–3050). IEEE. doi: doi:10.1109/CDC51059.2022.9993390
Kager, J., Tuveri, A., Ulonska, S., Kroll, P., & Herwig, C. (2020). Experimental verification and comparison of
model predictive, pid and model inversion control in a penicillium chrysogenum fed-batch process. Process
Biochemistry,90, 1–11. doi: doi:10.1016/j.procbio.2019.11.023
5

DC-TMPC for cultivations PREPRINT
Krausch, N., Kim, J. W., Barz, T., Lucia, S., Groß, S., Huber, M. C., .. . Cruz Bournazou, M. N. (2022). High-throughput
screening of optimal process conditions using model predictive control. Biotechnology and bioengineering,
119(12), 3584–3595. doi: doi:10.1002/bit.28236
Lucia, S., & Engell, S. (2013). Robust nonlinear model predictive control of a batch bioreactor using multi-
stage stochastic programming. In 2013 european control conference (ecc) (pp. 4124–4129). IEEE. doi:
doi:10.23919/ECC.2013.6669521
Mowbray, M. R., Petsagkourakis, P., Del Rio Chanona, E. A., & Zhang, D. (2022). Safe chance constrained
reinforcement learning for batch process control. Computers & Chemical Engineering,157, 107630. doi:
doi:10.1016/j.compchemeng.2021.107630
Pimentel, G. A., Benavides, M., Dewasme, L., Coutinho, D., & Wouwer, A. V. (2015). An observer-based robust control
strategy for overflow metabolism cultures in fed-batch bioreactors. IFAC-PapersOnLine,48(8), 1081–1086. doi:
doi:10.1016/j.ifacol.2015.09.112
Sankaranarayanan, P., & Rengaswamy, R. (2022). Cdinn – convex difference neural networks. Neurocomputing,495,
153–168. doi: doi:10.1016/j.neucom.2022.01.024
Srinivasan, B., Bonvin, D., Visser, E., & Palanki, S. (2003). Dynamic optimization of batch processes. Computers &
Chemical Engineering,27(1), 27–44. doi: doi:10.1016/S0098-1354(02)00117-5
Yu, S., Maier, C., Chen, H., & Allgöwer, F. (2013). Tube mpc scheme based on robust control invariant
set with application to lipschitz nonlinear systems. Systems & Control Letters,62(2), 194–200. doi:
doi:10.1016/j.sysconle.2012.11.004
6