Content uploaded by Sen Jia
Author content
All content in this area was uploaded by Sen Jia on Jul 31, 2023
Content may be subject to copyright.
IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. XX, NO. XX, XXXX 2022 1
Equilibrated Zeroth-Order Unrolled Deep
Network for Parallel MR Imaging
Zhuo-Xu Cui, Sen Jia, Jing Cheng, Qingyong Zhu, Yuanyuan Liu, Kankan Zhao, Ziwen Ke, Wenqi Huang,
Haifeng Wang, Senior Member, IEEE, Yanjie Zhu, Leslie Ying, Senior Member, IEEE, Dong Liang, Senior
Member, IEEE
Abstract—In recent times, model-driven deep learning
has evolved an iterative algorithm into a cascade network
by replacing the regularizer’s first-order information, such
as the (sub)gradient or proximal operator, with a network
module. This approach offers greater explainability and pre-
dictability compared to typical data-driven networks. How-
ever, in theory, there is no assurance that a functional reg-
ularizer exists whose first-order information matches the
substituted network module. This implies that the unrolled
network output may not align with the regularization mod-
els. Furthermore, there are few established theories that
guarantee global convergence and robustness (regularity)
of unrolled networks under practical assumptions. To ad-
dress this gap, we propose a safeguarded methodology for
network unrolling. Specifically, for parallel MR imaging, we
unroll a zeroth-order algorithm, where the network module
serves as a regularizer itself, allowing the network output to
be covered by a regularization model. Additionally, inspired
by deep equilibrium models, we conduct the unrolled net-
work before backpropagation to converge to a fixed point
and then demonstrate that it can tightly approximate the ac-
tual MR image. We also prove that the proposed network is
robust against noisy interferences if the measurement data
contain noise. Finally, numerical experiments indicate that
This work was supported in part by the National Natural Science
Foundation of China (U21A6005, 62125111, 12026603, 62206273,
61771463, 81830056, U1805261, 81971611, 61871373, 81729003,
81901736); National Key R&D Program of China (2020YFA0712202,
2017YFC0108802 and 2017YFC0112903); Natural Science Foundation
of Guangdong Province (2018A0303130132); Shenzhen Key Labora-
tory of Ultrasound Imaging and Therapy (ZDSYS20180206180631473);
Shenzhen Peacock Plan Team Program (KQTD20180413181834876);
Innovation and Technology Commission of the government of Hong
Kong SAR (MRP/001/18X); Strategic Priority Research Program of
Chinese Academy of Sciences (XDB25000000).
Corresponding author:dong.liang@siat.ac.cn
Z.-X. Cui and S. Jia contributed equally to this work
Z.-X. Cui, Q. Zhu, K. Zhao and D. Liang are with Research Center
for Medical AI, Shenzhen Institutes of Advanced Technology, Chinese
Academy of Sciences, Shenzhen, China.
J. Cheng, S. Jia, H. Wang, Y. Zhu and D. Liang are with Paul C.
Lauterbur Research Center for Biomedical Imaging, Shenzhen Institutes
of Advanced Technology, Chinese Academy of Sciences, Shenzhen,
China.
Z. Ke is with Institute for Medical Imaging Technology, School of
Biomedical Engineering, Shanghai Jiao Tong University, Shanghai,
China.
W. Huang is Technical University of Munich, Munich, Germany.
Y. Liu is with National Innovation Center for Advanced Medical De-
vices, Shenzhen, China.
L. Ying is with the Department of Biomedical Engineering and the
Department of Electrical Engineering, The State University of New York,
Buffalo, NY 14260 USA
D. Liang is with Pazhou Lab, Guangzhou, China
the proposed network consistently outperforms state-of-
the-art MRI reconstruction methods, including traditional
regularization and unrolled deep learning techniques.
Index Terms—Deep equilibrium models, unrolling, paral-
lel MR imaging, inverse problem, convergence, robustness.
I. INTRODUCTION
MRI is widely used in routine clinical practice due to its
non-invasiveness, non-ionizing radiation, and superior
visualization of soft-tissue contrast. However, its slow data ac-
quisition speed has long been a challenge. Shortening imaging
time has become a research focus. Specifically, reconstructing
high-quality MR images or full-sampled k-space data from
undersampled k-space data is a direct and effective approach
to improve imaging speed [1]. Over the past 20 years, one
of the most successful technical methods has been parallel
imaging (PI), which interpolates undersampled k-space data
by leveraging multi-coil information [2], [3]. Another highly
effective method, compressed sensing (CS), was introduced in
2006 [4], [5], enabling the complete recovery of an under-
sampled sparse signal. CS-MRI has been successfully applied
to accelerated MRI [6], as has PI, and both methods can be
modeled as regularization models [7]–[10], which typically
require a relatively long time to find a high-quality solution.
Recently, inspired by the tremendous success of deep learn-
ing (DL), many studies have applied DL to MR reconstruc-
tion (termed DL-MRI) and achieved significant performance
gains [11]–[16]. DL-MRI adaptively captures priors in a data-
driven manner from training data and performs superfast
online reconstruction with the aid of offline training. Early
DL-MRI work relied mainly on learning mappings between
undersampled k-space data (or zero-filling images) and fully
sampled k-space data (or high-quality images) [11], [17].
Although this approach yields excellent results, the reconstruc-
tion results may be uncertain due to their separation from
the model. Another line of development began with [18],
which started with a regularization model and reset the first-
order information of the regularizer as learnable to unroll
corresponding algorithms into deep networks [19]–[25]. Since
the architecture of unrolled deep networks (UDNs) is driven
by regularization models, it appears more explainable and
predictable compared to data-driven networks. A significant
This article has been accepted for publication in IEEE Transactions on Medical Imaging. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/TMI.2023.3293826
© 2023 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See https://www.ieee.org/publications/rights/index.html for more information.
Authorized licensed use limited to: Shenzhen Institute of Advanced Technology CAS. Downloaded on July 13,2023 at 01:50:40 UTC from IEEE Xplore. Restrictions apply.
2 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. XX, NO. XX, XXXX 2022
number of experiments have confirmed its competitiveness in
reconstructed image quality.
However, UDNs still face several theoretical problems that
require further investigation. Firstly, there is the issue of
consistency. The first-order information (i.e., (sub)gradient or
proximal operator) of the unrolled network module cannot be
guaranteed to match any functional regularizer. This means
that the output of the UDN cannot entirely embody the ex-
plainable and predictable nature of the regularization models.
Secondly, there is the problem of convergence. Although it
has been demonstrated in [26] that a first-order UDN with
a nonexpansive constraint is guaranteed to converge to a
fixed point, there is no theoretical proof that this fixed point
corresponds to the solution of the regularization models or a
precise approximation of the real MR image. Thirdly, there
is the issue of robustness. Current UDNs lack theoretical
guarantees of being robust to noisy measurements. In fact,
a recent study [27] discovered that existing DL-MRI methods,
including UDNs, generally result in unstable reconstruction,
which seriously limits the clinical application of DL methods.
A. Contributions
Motivated by the issues mentioned above, this study aims
to propose a safeguarded UDN approach for MR image
reconstruction. Specifically, the paper’s main contributions are
as follows:
1) Firstly, we unroll a zeroth-order algorithm (i.e., projec-
tion over convex sets (POCS)) whose network module
represents a PI regularizer. This ensures that the network
output is consistent with a PI regularization model,
allowing the proposed zeroth-order UDN to inherit the
explainable and predictable nature of a PI regularization
model.
2) Inspired by the idea of deep equilibrium (DEQ) models,
we carry out the zeroth-order UDN to converge to a
solution of the PI regularization model before backprop-
agation. Under certain conditions, we also show that the
convergent solution is a tight complexity guarantee that
approximates the true MR image (or full-sampled k-
space data).
3) Furthermore, based on the network architecture and
the nonexpansive constraint in DEQ, we prove that
the proposed zeroth-order UDN is robust against noisy
interference. Thus, we can guarantee that the proposed
method is immune to Gaussian noise interference in k-
space measurement data.
4) Numerical results on two MR datasets demonstrate that
the proposed zeroth-order UDN significantly outper-
forms the traditional PI method and the state-of-the-art
first-order UDN in terms of image reconstruction quality,
robustness against noisy interference, and distribution
shifts (i.e., training and testing on different sampling
patterns).
The remainder of the paper is organized as follows. Section
II provides some notations and preliminaries. Section III
discusses the equilibrated zeroth-order UDN for the k-space
TABLE I
MEANING OF MATHEMATICAL SYMBOLS IN BACKGROUND AND
METHODS.
Symbol Symbol Meaning
xMR image
b
xk-space data, b
x=FFT(x)
Mundersampling operator
yundersampled k-space data, y=Mb
x
b
xkkth iteration
b
x∞fixed point of iteration b
xk
b
x†real solution of inverse problem y=Mb
x
b
xnnth column of b
x
Cnonempty closed and convex set of CN
PCprojection on C,PC(x) = arg miny∈C∥x−y∥
Ωsubset of indices, Ω⊆ {1,...,N}
QΩsampling operator on Ω
Iidentity mapping
Φϕmultilayer CNN module with parameters ϕ
PI model. Section IV discusses the corresponding theoreti-
cal guarantees. The implementation details are presented in
Section V. Experiments performed on several datasets are
presented in Section VI. A discussion is presented in Section
VII. Section VIII provides some concluding remarks. All
proofs are presented in Appendix.
II. NOTATI ONS
To eliminate ambiguity, matrices and vectors are all repre-
sented by bold lowercase letters. x,y. In addition, xi(xi,j)
refers to the i-th column ((i, j)th entry) of matrix x. The
superscript Ton a matrix denotes the transpose, and Hdenotes
the adjoint operator. A variety of norms on matrices will be
discussed. The spectral norm of a matrix xis denoted by
∥x∥. The Euclidean inner product between two matrices is
⟨x,y⟩=Trace(xHy), and the corresponding Euclidean norm,
termed the Frobenius norm, is denoted as ∥x∥Fwhich is
derived as ∥x∥F:= ⟨x,x⟩. For vectors, ∥·∥denotes the
ℓ2norm.
We say an operator T:CN1→CN2is L-Lipschitz, if it
holds
∥T(x)−T(y)∥ ≤ L∥x−y∥
for any x,y∈CN1. We say the operator Tis nonexpansive if
it is 1-Lipschitz. In lay terms, any two points, (x,y), passing
through some nonexpansive map will not expand the distance
between them. In addition, please see Table I for the specific
meaning of other symbols in the paper.
III. REL ATED WO RK AN D METHODS
In this section, we first briefly review some related work
and then introduce the proposed zeroth-order UDN in detail.
A. Related Work
1) Classical CS-PI methods:Mathematically, in the context
of multicoil acquisition, a PI model can be transformed from
a single linear equation into a redundant linear equation set.
The original SENSE algorithm aims to solve these redundant
equations to reconstruct an MR image [28]. With the advent
of compressed sensing (CS), the SENSE model has been
This article has been accepted for publication in IEEE Transactions on Medical Imaging. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/TMI.2023.3293826
© 2023 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See https://www.ieee.org/publications/rights/index.html for more information.
Authorized licensed use limited to: Shenzhen Institute of Advanced Technology CAS. Downloaded on July 13,2023 at 01:50:40 UTC from IEEE Xplore. Restrictions apply.
ZHUO-XU CUI: EQUILIBRATED ZEROTH-ORDER UNROLLED DEEP NETWORK FOR PARALLEL MR IMAGING 3
extended to a sparsity-regularized form [29]. Alternatively, a
PI model can be formulated in k-space as an interpolation
procedure, which assumes that the values of k-space data
within each channel are predictable within a neighborhood.
Prominent examples of k-space PI models include GRAPPA
[30] and SPIRIT [31]. Compared to a SENSE-based PI model,
ak-space PI model is easier to implement without the need
to estimate coil sensitivity.
2) Unrolled Deep Networks (UDNs):A UDN starts with
an iterative algorithm architecture for CS-MRI or PI R-
regularized problems and releases ∂R or ProxRand free
parameters (including regularization parameter, step size, etc.)
as a learnable convolutional neural network module, i.e.,
CNNi(·)at the ith layer, to derive a UDN with a finite number
of layers. A UDN may be trained using backpropagation,
resulting in CNNithat are learned from reference MR images.
In this way, a UDN can be naturally interpreted as a learnable
regularization method [13]–[15], [19]–[23], [32]. However,
conversely, there is no guaranteed functional regularizer R
such that the following equation holds:
CNNi(x)∈∂R(x)(or ProxR(x)), i = 1, . . . , T
where Tdenotes the number of layers of the UDN. Therefore,
there is no definite R-regularized model that covers the output
of a UDN. Although a UDN is not theoretically perfect at
present, many scholars believe that it presents a potential
approach to break the limits of analytic regularization methods
for solving inverse problems [12], [33].
3) Deep Equilibrium (DEQ) Models:In recursive networks,
including UDNs, it has been observed that as the network
layers become deeper, the network expressibility becomes
stronger. Naturally, one may wonder what happens when
the number of layers approaches infinity. However, due to
memory limitations, it is impractical to train a network with an
arbitrarily large number of layers. Fortunately, recent research
on DEQ has shown that these limitations can be modeled
using a fixed (equilibrium) point equation [34], [35]. In brief,
a DEQ model first executes recursive networks to converge to
a fixed point before backpropagating, which is equivalent to
running an infinite depth network. Then, the backpropagation
can be analytically computed through only this fixed point,
such that memory usage does not increase as the network depth
increases.
DEQ not only allows infinite depth networks but also sheds
light on the convergence of recursive networks. Based on
monotone operators, [36] introduced efficient solvers for find-
ing fixed points with guaranteed stable convergence. In par-
ticular, when a recursive network is designed in an unrolling
manner, [26], [37] showed that a first-order UDN under a DEQ
framework (called a DEQ-UDN) is guaranteed to converge to
a fixed point. However, there is no theoretical guarantee that
this fixed point is the solution to the regularization models or
a tight approximation of the true solution
B. Forward Model
In MRI, a forward model of parallel k-space data acquisition
can be formulated as
y=Mb
x(1)
b
x,y∈CN×Nc,Nc≥1denotes the number of channels,
b
xis the full-sampled k-space data, ith column of which
denotes the data acquired by the ith coil, yis an undersampled
measurement and Mdenotes the sampling pattern. Particu-
larly, Mb
x= [QΩ(b
x1),...,QΩ(b
xNc)], where b
xidenotes ith
column of b
xand QΩdenotes the sampling operator on a
subset of indices Ω⊆ {1, . . . , N }. Our task is to interpolate
the missing values of yas accurately as possible. In practice,
because of MR hardware limitations, the measurement data
are usually mixed with noisy interference, i.e.,
yδ=y+n(2)
where n∈CN×Ncdenotes the noise and δdenotes the
noisy intensity, i.e., δ:= ∥n∥. Then, interpolation methods
are required to be robust against noisy interferences.
C. Zeroth-Order UDN for the k-Space PI Model
As discussed above, because there is no need to estimate coil
sensitivities, the k-space PI model has received much attention
from industry, among which SPIRiT is the most prominent
example. SPIRiT considers that every point in the grid can
be linearly predicted by its entire neighborhood in all coils.
Given this assumption, any k-space data b
xiat the ith coil
can be represented by data from other coils with kernel wi,n.
Then, the k-space PI regularization model is:
min
b
x∈CN×Nc
R(b
x) :=
Nc
X
i=1
b
xi−
Nc
X
n=1 b
xn⊗wi,n
2
s.t. Mb
x=y.
(3)
where Rs the so-called self-consistency regularizer. Algorith-
mically, the POCS is an effective zeroth-order algorithm for
solving problem (3), which carries out the following updates:
b
xk+1
2
i=
Nc
X
n=1 b
xk
n⊗wi,n
b
xk+1 =PC(b
xk+1
2)
where b
xk= [b
xk
1,...,b
xk
Nc]and C:= {b
x∈Cd|Mb
x=y}.
Looking closely at the above iterations, we can see that the
POCS algorithm only calls the zeroth-order information of
regularizer Rand does not call higher-order information.
Leveraging the idea of a UDN, we release the linear con-
volution kernel (1-convolutional layer) in Ras a learnable
multilayer CNN module Φϕwith parameters ϕand train it
in an end-to-end fashion. In particular, the recursion of the
unrolled POCS algorithm is:
(b
xk+1
2= Φϕ(b
xk)
b
xk+1 =PC(b
xk+1
2).(4)
Specifically, the unrolled POCS (4) can be summed up as a
zeroth-order algorithm for solving the following generalized
PI regularization model:
(min
b
x∈CN×Nc
R(b
x) := ∥b
x−Φϕ(b
x)∥2
F
s.t. Mb
x=y.
(5)
This article has been accepted for publication in IEEE Transactions on Medical Imaging. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/TMI.2023.3293826
© 2023 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See https://www.ieee.org/publications/rights/index.html for more information.
Authorized licensed use limited to: Shenzhen Institute of Advanced Technology CAS. Downloaded on July 13,2023 at 01:50:40 UTC from IEEE Xplore. Restrictions apply.
4 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. XX, NO. XX, XXXX 2022
That is, the output of (4) can completely inherit the explainable
and predictable nature of model (5). (5) is a generalization of
the SPIRiT model (3), of which the linear predictable prior
is extended to nonlinear predictability. If Φϕis chosen as one
convolutional layer, (5) reduces to (3) absolutely.
The network architecture of Φϕis depicted in Figure 1
(a). To avoid gradient disappearance, we adopt a residual
architecture. Different from the general residual network, we
use the paradigm of (β−α)·I+α·CNN(·), (0≤α≤β, β < 1)
to ensure the nonexpansive property, which is crucial for its
convergence, where CNN(·)represents the cascaded convolu-
tional neural network part in Figure 1. Moreover, [38], [39]
have shown that MR images exhibit self-redundancies in the
image domain. We designed another network architecture de-
picted in Figure 1 (b). The nonexpansive residual architecture
above exploits the image domain redundancies, and the resid-
ual architecture below exploits the k-space self-consistency
and complementarity. The additional image domain residual
architecture does not affect the theoretical properties of the
unrolled POCS (4).
Fig. 1. Schematic diagram of the network architecture of the unrolled
POCS algorithm: (a) k-space unrolled POCS: the self-consistency Φϕ
is generalized by a five-layer nonexpansive residual network in k-space,
and PCadopts the general orthogonal projection; (b) k-space and
image domain hybrid unrolled POCS: the self-consistency Φϕis linearly
composed of two five-layer nonexpansive residual networks in k-space
and image domain, respectively.
D. Equilibrated Zeroth-Order UDN for the k-Space PI
Model
Recent work on DEQ showed that the convergence of re-
cursive networks can be modeled using a fixed point equation.
Following this, we propose to train and test the unrolled
POCS algorithm under the DEQ framework. First, before
backpropagation, we execute the unrolled POCS algorithm (4)
to converge to a fixed point, i.e., b
x∞=PC(Φϕ(b
x∞)). Note
that b
x∞is a mapping associated with the parameter ϕand
the measurement y, i.e., b
x∞:= b
x∞(ϕ, y). For convenience,
we omit the parameter and the measurement (ϕ, y)for a fixed
point b
x∞(ϕ, y)if not otherwise stated. To make the conver-
gent fixed point b
x∞approximate the true full-sampled k-space
data b
xas tight as possible, we minimize the loss between b
x∞
and b
x, i.e., ℓ(b
x∞(ϕ, y),b
x). According to the chain rule, the
calculation of the partial derivative of ℓ(b
x∞(ϕ, y),b
x)at ϕis:
∂ℓ(b
x∞(ϕ, y),b
x)
∂ϕ =∂b
x∞(ϕ, y)T
∂ϕ ·∂ℓ(b
x∞(ϕ, y),b
x)
∂b
x∞(ϕ, y).
On the other hand, by the denifition of fixed point, i.e., b
x∞=
PC(Φϕ(b
x∞)), we have
∂b
x∞
∂ϕ =∂PC(Φϕ(b
x∞))
∂ϕ +∂PC(Φϕ(b
x∞))
∂b
x∞·∂b
x∞
∂ϕ .
Then
∂b
x∞
∂ϕ =I − ∂PC(Φϕ(b
x∞))
∂b
x∞−1∂PC(Φϕ(b
x∞))
∂ϕ .
Combining the above equations, we can obtain
∂ℓ(b
x∞,b
x)
∂ϕ =∂PC(Φϕ(b
x∞))T
∂ϕ I − ∂PC(Φϕ(b
x∞))
∂b
x∞−T∂ℓ
∂b
x∞.
(6)
From the above formula, it can be found that it only calculates
the partial differential in b
x∞, and has nothing to do with {b
xk}.
It means that the backpropagation for ϕcan be calculated on
fixed point b
x∞directly, regardless of how many iterations are
carried out, so that the memory does not increase even if the
number of layers increases to infinity.
The training process for the DEQ-POCS is depicted in
Algorithm 1. Suppose that the training data are sampled from
a certain distribution πb
x×y, and the network parameter ϕ0is
initialized with normal distribution, b
x0is initialized as y. The
algorithms are executed for Kepochs. For each epoch, we
randomly shuffled the order of the data and then executed the
algorithm over the dataset with ergodicity. At each iteration,
we first carry out the unrolled POCS algorithm (4) to find a
fixed point. Then we update the network parameters ϕby a
certain optimizer (using ADAM optimizer in the subsequent
experiments) to the loss function ℓ(b
x∞(ϕ, y),b
x). In particular,
in Algorithm 1, we take the square of Frobenius-norm as
the loss function. When a training iteration is complete, the
algorithm outputs the self-consistency term ΦϕKM . To find the
fixed point faster, we can use the Anderson algorithm [40] to
accelerate the unrolled POCS algorithm (4).
The testing process for the DEQ-POCS network is depicted
in Algorithm 2. Insert the trained ΦϕKM into the unrolled
POCS algorithm (4) and execute it to converge to a fixed point,
which acts as the algorithm output.
IV. THEORETICAL RES ULTS
In this section, we first show the convergence analysis for
the proposed DEQ-POCS network in a case of noise-free
measurement data. Then, we prove that the proposed DEQ-
POCS network is robust against noisy interference when the
measurement data contain noise.
A. Convergence
Before proving our main result, we suppose that the learned
self-consistency terms Φϕi,i∈ {1, . . . , KM }in Algorithm 1
satisfy the following assumption:
This article has been accepted for publication in IEEE Transactions on Medical Imaging. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/TMI.2023.3293826
© 2023 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See https://www.ieee.org/publications/rights/index.html for more information.
Authorized licensed use limited to: Shenzhen Institute of Advanced Technology CAS. Downloaded on July 13,2023 at 01:50:40 UTC from IEEE Xplore. Restrictions apply.
ZHUO-XU CUI: EQUILIBRATED ZEROTH-ORDER UNROLLED DEEP NETWORK FOR PARALLEL MR IMAGING 5
Algorithm 1 Training DEQ-POCS Network.
1: Input: training samples {(b
x,y)}M
m=1 ∼πb
x×y;
2: Initialize: ϕ0,b
x0;
3: for k= 0,1, . . . , K do
4: n= 0;
5: for each randomly sampled (b
x,y)from training sam-
ples do
6: Carry out (4) to find a fixed point:
b
x∞=P{b
x|Mb
x=y}ΦϕkM+n(b
x∞);
7: ϕkM +n+1 =Optimizer(ℓ(b
x∞,b
x); ϕkM +n);
8: n=n+ 1;
9: end for
10: end for
11: Output: ΦϕKM .
Algorithm 2 Testing DEQ-POCS Network.
1: Input: testing samples y∼πy,ΦϕKM ;
2: Initialize: b
x0;
3: Carry out (4) to find a fixed point:
b
x∞=P{b
x|Mb
x=y}ΦϕKM (b
x∞);
4: Output: b
x∞.
Assumption 1: The learned self-consistency term Φϕi,i∈
{1, . . . , KM }in Algorithm 1 is L-Lipschitz continuous with
0<L<1.
Remark 4.1: In Figure 1, given the added spectral normal-
ization, it is known that the Lipschitz constant of the network
module CNN(·)is less than or equal to 1. The coupled residual
structure (β−α)·I +α·CNN(·), (0≤α≤β, β < 1) ensures
that the Lipschitz constant of the self-consistency term Φϕiis
less than or equal to β(<1).
Theorem 4.1: Suppose that Assumption 1 holds. The un-
rolled POCS (4) in Algorithms 1 and 2 converges to a fixed
point globally.
The proof is shown in Appendix A.
Remark 4.2: Although some studies have given a conver-
gence proof of a UDN under some conditions, such as,
the Kurdyka-Łojasiewicz condition [13], [14], asymptotically
nonexpansive condition [32], and uniform decrease condition
[41], these conditions are difficult to verify in practice and the
corresponding theory requires that the number of iterations
tends to be infinite and that the UDN usually unrolls only a
few layers.
Theorem 4.1 proves that the unrolled POCS (4) will con-
verge to a fixed point. Next, we investigate the properties of
this fixed point in the testing algorithm.
Remark 4.3: According to the unrolled POCS (4) in Algo-
rithm 2, the fixed point of convergence is known to satisfy
both the data consistency Mb
x=yand the self-consistency
b
x= ΦϕKM (b
x). If we assume that the intersection of the
solution space with the self-consistent space has only one real
solution b
x†, i.e., {b
x|Mb
x=y}∩{b
x|b
x= ΦϕKM (b
x)}={b
x†},
Fig. 2. Loss function graph for training the DEQ-POCS network on the
brain data.
then the fixed point of convergence is this real solution, i.e.,
b
x∞=b
x†.
On the other hand, considering that the assumption in the
above remark does not hold, we will statistically analyze the
approximation error between the output of Algorithm 2 and the
real solution. Before giving the result, we assume the learned
ΦϕKM meets the following condition:
Assumption 2: In Algorithm 1, {(b
xm,ym)}M
m=1 represents
the training samples. Let b
x∞
m:= b
x∞(ϕKM ,ym), and the loss
function ℓis taken to be the Frobenius-norm. Then, there exists
a constant ϵsuch that 1
MPM
m=1 ∥b
x∞
m−b
xm∥F≤ϵ.
Remark 4.4: Although the loss function is generally non-
convex and nonsmooth to network parameters, some studies
[42], [43] empirically indicate that the value of loss usually
attenuates to close to 0 as the training progresses, when
the depth of the network is deep enough. In theory, the
simple stochastic gradient descent algorithm can find global
minimizers (0-value loss points) for the network training under
certain conditions [44]. In particular, Figure 2 demonstrates
that the loss function values drop to the order of 1e-3 when
training our DEQ-POCS network.
Based on Assumptions 1 and 2, we have the following
result:
Proposition 4.1: Suppose that Assumptions 1 and 2 hold.
Then, there exists a constant B > 0such that the convergent
solution b
x∞of Algorithm 2 satisfies:
∥b
x∞−b
x†∥F≤1 + 1
√Mλ+ϵ(7)
with a probability at least 1−4 exp −λ2
2B2, where b
x†denote
the real solution (k-space data) of inverse problem Mb
x=y.
The proof is shown in Appendix B. At this point, we have
given a statistical error analysis for the output of Algorithm
2.
B. Robustness
In practice, due to the hardware limitations of MR systems,
the measurement data are usually mixed with some Gaussian
noise interference. Because of the ill-posedness of the accel-
erated MR reconstruction problem, a small amount of noise in
the measurement may seriously interfere with the interpolation
accuracy. Thus, the robustness (regularity) property is very
This article has been accepted for publication in IEEE Transactions on Medical Imaging. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/TMI.2023.3293826
© 2023 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See https://www.ieee.org/publications/rights/index.html for more information.
Authorized licensed use limited to: Shenzhen Institute of Advanced Technology CAS. Downloaded on July 13,2023 at 01:50:40 UTC from IEEE Xplore. Restrictions apply.
6 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. XX, NO. XX, XXXX 2022
important in designing MR reconstruction methods. In this
study, the proposed DEQ-POCS network (Algorithm 2) has
the following property:
Theorem 4.2: Suppose that Assumption 1 holds. When the
measurement contains noise, i.e., yδ=y+nwith noisy
level δ:= ∥n∥F, the convergent solution b
x∞(ϕKM ,yδ)of
Algorithm 2 satisfies:
∥b
x∞(ϕKM ,yδ)−b
x∞(ϕKM ,y)∥F≤δ
1−L
where b
x∞(ϕKM ,y)denotes the convergent solution of Algo-
rithm 2 with clean measurement y
The proof is shown in Appendix C. From the above theorem,
we can see that a δ-level noisy interference in the measurement
at most causes δ/(1 −L)-level error in the interpolated result
compared to the noisy-free case.
V. IMPLEMENTATION
The evaluation was performed on two multichannel k-space
data with various k-space trajectories. The details of the k-
space data are as follows:
A. Data Acquisition
1) Knee data:First, we tested our proposed method on knee
MRI data 1. The raw data were acquired from a 3T Siemens
scanner. The number of coils was 15 and the 2D Cartesian
turbo spin echo (TSE) protocol was used. The parameters for
data acquisition are as follows: the repetition time (TR) was
2800ms, the echo time (TE) was 22ms, the matrix size was
768 ×770 ×1and the field of view (FOV) was 280 ×280.7×
4.5mm3. Particularly, the readout oversampling was removed
by transforming the k-space to image, and cropping the center
384 ×384 region. Fully sampled multichannel knee images
of nine volunteers were collected, of which data from seven
subjects (including 227 slices) were used for training, while
the data from the remaining two subjects (including 64 slices)
were used for testing.
2) Human brain data:To verify the generalization of the
proposed method, we tested it on human brain MRI data 2,
which was collected by [39]. These MRI data were acquired
using a 3D T2 fast spin echo with an extended echo train
acquisition (CUBE) sequence with Cartesian readouts using a
12-channel head coil. The matrix dimensions were 256×232×
208 with 1 mm isotropic resolution. The training data contain
360 slices k-space data from four subjects and the testing data
contain 164 slices k-space data from two subjects. Each slice
has a spatial dimension of 256 ×232.
3) Sampling patterns:Four different types of undersampling
patterns were considered. We train the proposed method on
1-D and 2-D radom patterns and test it on 1-D and 2-D
regular patterns to verify the robustness on distribution shifts.
A visualization of these sampling patterns is depicted in Figure
3.
1http://mridata.org/
2https://drive.google.com/file/d/
1qp-l9kJbRfQU1W5wCjOQZi7I3T6jwA37/view?usp=sharing
Fig. 3. Various sampling patterns: (a) 1-D random undersampling at
R= 4, (b) 2-D random undersampling at R= 6, (c) 1-D regular
undersampling at R= 3, and (d) 2-D regular undersampling at R= 6.
B. Network Architecture and Training
A schematic diagram of the self-consisting module Φϕ
architecture is illustrated in Figure 1. To find the fixed point
faster, we use the Anderson algorithm to accelerate the un-
rolled POCS algorithm (4), whose code is available on this
page 3.
The ADAM [45] optimizer with is chosen for Algorithm
1β1= 0.9, β2= 0.999 with respect to the ℓ2-norm loss in
k-space. The size of the mini batch is 1, and the number of
epochs is 500. The learning rate is set to 10−4. The labels
for the network were the fully sampled k-space data. The
input data for the network was the regridded downsampled
k-space data from 1-D and 2-D random trajectories. The
details of the downsampling procedure are discussed above.
Without specific instructions, we train the network separately
for different trajectories. The models were implemented on
an Ubuntu 20.04 operating system equipped with an NVIDIA
A100 Tensor Core (GPU, 80 GB memory) in the open PyTorch
1.1.0 framework [46] with CUDA 11.3 and CUDNN support.
C. Performance Evaluation
In this study, the quantitative evaluations were all calcu-
lated on the image domain. The image is derived using an
inverse Fourier transform followed by an elementwise square-
root of sum-of-the squares (SSoS) operation, i.e. z[n] =
(PNc
i=1 |xi[n]|2)1
2, where z[n]denotes the n-th element of
image z, and xi[n]denotes the n-th element of the ith coil
image xi. For quantitative evaluation, the peak signal-to-noise
ratio (PSNR), normalized mean square error (NMSE) value
and structural similarity (SSIM) index [47] were adopted.
VI. EXPERIMENTATION RE SULTS
A. Comparative Studies
In this section, we evaluate the effectiveness of our pro-
posed DEQ-POCS approach using two architectures: the k-
space architecture and the hybrid architecture illustrated in
Figure 1. These architectures are referred to as K-DEQ-POCS
and H-DEQ-POCS, respectively. We conducted a series of
extensive comparative experiments on knee and brain datasets
to demonstrate the superiority of our methods. In particular,
we compared the traditional k-space PI method (SPIRiT-POCS
[31]) and SOTA k-space first-order UDN, i.e., Deep-SLR [23],
3http://implicit-layers- tutorial.org/
This article has been accepted for publication in IEEE Transactions on Medical Imaging. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/TMI.2023.3293826
© 2023 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See https://www.ieee.org/publications/rights/index.html for more information.
Authorized licensed use limited to: Shenzhen Institute of Advanced Technology CAS. Downloaded on July 13,2023 at 01:50:40 UTC from IEEE Xplore. Restrictions apply.
ZHUO-XU CUI: EQUILIBRATED ZEROTH-ORDER UNROLLED DEEP NETWORK FOR PARALLEL MR IMAGING 7
based on k-space and hybrid architectures, dubbed K-Deep-
SLR and H-Deep-SLR. In particular, for a fair comparison, we
added spectral normalization operations to their convolutional
layers and developed a PyTorch-based implementation based
on their publicly available TensorFlow codes4. In addition, we
also compared the POCS (4) unrolling network (without the
DEQ convergence constraint and with the normalized residual
structure degenerated to the ordinary residual structure in
Figure 1), dubbed Deep-POCS, as an ablation experiment.
B. Experiments Without Noisy Interference
In this section, we present the results of our experiments
on knee data using our proposed K-DEQ-POCS and H-DEQ-
POCS networks, as well as several comparative algorithms, all
of which were tested without additional noisy interference. The
knee data was acquired using a 1-D random trajectory with
an acceleration factor of 4. We first investigated the effect of
the number of convolutional layers in the network module, as
shown in Figure 1, on the reconstruction results. Specifically,
we compared the performance of our proposed methods and
the comparison methods using 2 to 5 convolutional layers,
and the results are summarized in Table II. To better visualize
the performance of the various methods as the number of
convolution layers varies, we plotted their PSNR graphs in
Figure 4. Overall, all methods showed an improvement in
performance as the number of convolutional layers increased.
However, it is important to note that the main comparison
method, H-Deep-SLR, was particularly sensitive to the num-
ber of convolutional layers, exhibiting a relatively significant
degradation in performance as the number of layers decreased.
In contrast, the proposed H-DEQ-POCS method achieved
satisfactory performance even with only two convolutional
layers in the network module.
Fig. 4. Quantitative metric (PSNR) graphs of various methods across
different convolutional layers.
Figure 5 illustrates the reconstruction results obtained using
various methods with five convolutional layers. As depicted
in Figure 5, the reconstructed images obtained using single
k-space architecture networks, including the K-Deep-SLR, K-
DEQ-POCS, and SPIRiT-POCS algorithms, exhibit an aliasing
pattern. However, for networks that utilize hybrid architectures
with both k-space and image domains, the quality of the
4https://github.com/anikpram/Deep-SLR
reconstructed images is significantly improved. Nonetheless,
upon examining the error view presented in Figure 5, it
becomes apparent that the reconstructed images produced by
H-Deep-SLR and the ablation method H-Deep-POCS still
exhibit a slight aliasing pattern, whereas our proposed H-DEQ-
POCS method produces images that are less affected by such
distortions.
Figure 6 presents the reconstruction results of brain data
using various methods (with five convolutional layers) under
the 2-D random trajectory with an acceleration factor of 6. It
is evident from the results that our proposed method, K-DEQ-
POCS, outperforms K-Deep-SLR and SPIRiT-POCS, with
respect to reducing the impact of noisy artifacts. For networks
utilizing hybrid architectures, both H-DEQ-POCS, H-Deep-
POCS, and H-Deep SLR demonstrate good performance.
Upon closer inspection of the error view, it is apparent that
H-DEQ-POCS yields the highest reconstruction accuracy.
The competitive quantitative results of the above methods
are shown in Tables II and III. Our method consistently
outperforms the other comparative methods for knee and brain
data. Therefore, as characterized by visual and quantitative
evaluations, the above experiments confirm the competitive-
ness of our method in the case with no noisy interference.
Finally, we compare the memory occupied and the training
and testing time of the proposed and compared methods.
The corresponding data are presented in Tables II and III.
The proposed method occupies more memory and takes more
time to train and test than these comparative methods due
to the need to find the fixed point of (4) and the need to
solve the inverse of the operator when back-propagating (6).
However, the proposed method does not take up significantly
more memory, training, and testing time than the comparison
methods and is still in an acceptable range. It is our future
work to speed up and reduce memory.
C. Interference on Measurement
In real-world scenarios, MR systems often generate noisy
measurements due to magnetic field inhomogeneity and hard-
ware limitations. To ensure the robustness of our method in
the presence of noise, we evaluated its performance on various
measurement data with additional δintensity noise, denoted as
yδ:= y+n. Note that the conventional technique of SPIRiT-
POCS is often effective in mitigating noise amplification
through early stopping. However, the focus of this study is
on exploring the inherent robustness of algorithms or networks
against measurement interference. To ensure a fair comparison,
we removed the early stopping criterion of SPIRiT-POCS and
investigated the impact of noise interference on the iterative
algorithm.
We conducted experiments to compare different methods
and measure their robustness to noise, with Table IV present-
ing the quantitative metrics of each method’s reconstruction re-
sults at varying noise levels. The noise intensity is indicated by
the percentage s%of the norm of the added noise to the norm
of the measurement, which is expressed as ∥n∥F/∥y|F=s%.
Figure 7 visually displays the trend of the quantitative metrics
(PSNR) with increasing noise interference. As evidenced by
This article has been accepted for publication in IEEE Transactions on Medical Imaging. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/TMI.2023.3293826
© 2023 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See https://www.ieee.org/publications/rights/index.html for more information.
Authorized licensed use limited to: Shenzhen Institute of Advanced Technology CAS. Downloaded on July 13,2023 at 01:50:40 UTC from IEEE Xplore. Restrictions apply.
8 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. XX, NO. XX, XXXX 2022
Fig. 5. Reconstruction results under 1-D random undersampling at R= 4. The values in the corner are each slice’s NMSE/PSNR/SSIM values.
The second and third rows illustrate the enlarged and error views, respectively. The grayscale of the reconstructed images and the error images’
color bar are on the figure’s right.
Fig. 6. Reconstruction results under 2-D random undersampling at R= 6. The values in the corner are each slice’s NMSE/PSNR/SSIM values.
The third row illustrates the error views. The grayscale of the reconstructed images and the error images’ color bar are on the figure’s right.
Table IV and Figure 7, the comparison methods H-Deep-
SLR and H-Deep-POCS exhibit significant degradation as
the noise scale increases. In contrast, the proposed method
outperforms not only H-Deep-SLR and H-Deep-POCS but
also traditional SPIRiT-POCS for all levels of noise. These
results demonstrate that the proposed method is comparable
to traditional approaches and significantly outperforms other
deep learning methods regarding measurement robustness.
Figure 8 shows the reconstruction results of the various
methods when the knee measurement data contains 5% noise
and the brain data measurement contains 1% noise. The H-
Deep-SLR and H-Deep-POCS methods amplify the noise
severely, resulting in distorted image contrast. On the knee
data, SPIRiT-POCS also amplifies the noise significantly. It is
worth mentioning that the reconstruction results of the pro-
posed method are less degraded under noise interference. The
experimental result verifies the validity of Theorem 4.2. We
can conclude that our approach ensures both the competitive
performance of deep learning methods and the robustness as
traditional algorithms.
D. Interference on Initial Input
Ideally, the selection of the initial input has little effect
on the final solution for convergent algorithms. To verify
the convergence of our method, in this experiment, we test
various methods against interferences on the initial input, i.e.,
b
xδ,0:= b
x0+n.
Table V presents the quantitative metrics for each method’s
reconstruction results for different interference levels in the
initial input. To provide a more visual representation of the
trend, Figure 9 displays the variation of the quantitative
metrics (PSNR) with increasing noisy interference. As shown
in Table V and Figure 9, the proposed method and traditional
SPIRiT-POCS exhibit almost no impact from the initial value
This article has been accepted for publication in IEEE Transactions on Medical Imaging. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/TMI.2023.3293826
© 2023 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See https://www.ieee.org/publications/rights/index.html for more information.
Authorized licensed use limited to: Shenzhen Institute of Advanced Technology CAS. Downloaded on July 13,2023 at 01:50:40 UTC from IEEE Xplore. Restrictions apply.
ZHUO-XU CUI: EQUILIBRATED ZEROTH-ORDER UNROLLED DEEP NETWORK FOR PARALLEL MR IMAGING 9
TABLE II
QUANTITATIVE COMPARISON FOR VARIOUS METHODS ON THE KNEE DATASET WITH DIFFERENT NUMBERS OF CONVOLUTIONAL LAYERS.
Datasets Quantitative Evaluation
& Methods NMSE PSNR(dB) SSIM Training
Memory(MiB)
Training Time(s)
(Per-Iteration)
Testing Time(s)
(Per-Iteration)
SPIRiT-POCS 0.0071±0.0055 34.12±2.53 0.88±0.02 / / /
Knee
(2-layer)
K-Deep-SLR 0.0084±0.0067 33.52±2.85 0.91±0.02 3443 0.039745 0.036569
K-DEQ-POCS 0.0085±0.0008 33.70±2.97 0.90±0.02 5799 0.142495 0.112768
H-Deep-SLR 0.0081±0.0031 34.12±2.03 0.90±0.02 4331 0.069045 0.061285
H-Deep-POCS 0.0063±0.0040 34.40±2.16 0.91±0.02 4227 0.075663 0.068965
H-DEQ-POCS 0.0049±0.0032 35.54±2.15 0.91±0.02 9663 0.321472 0.189644
Knee
(3-layer)
K-Deep-SLR 0.0082±0.0063 33.60±2.54 0.91±0.02 3857 0.048642 0.042967
K-DEQ-POCS 0.0080±0.0064 33.73±2.77 0.91±0.02 7153 0.197435 0.139478
H-Deep-SLR 0.0056±0.0020 34.67±1.74 0.91±0.02 5131 0.095966 0.064989
H-Deep-POCS 0.0056±0.0042 35.10±2.24 0.91±0.02 4971 0.099915 0.068965
H-DEQ-POCS 0.0049±0.0031 35.55±2.14 0.91±0.02 12027 0.369497 0.226315
Knee
(4-layer)
K-Deep-SLR 0.0078±0.0059 33.79±2.53 0.91±0.02 4261 0.071136 0.046637
K-DEQ-POCS 0.0077±0.0060 33.82±2.65 0.91±0.02 8269 0.242690 0.167353
H-Deep-SLR 0.0048±0.0032 35.50±1.75 0.91±0.02 5857 0.129240 0.077666
H-Deep-POCS 0.0050±0.0039 35.52±2.06 0.91±0.02 5697 0.131619 0.075314
H-DEQ-POCS 0.0044±0.0030 35.95±2.04 0.91±0.02 14311 0.457825 0.266956
Knee
(5-layer)
K-Deep-SLR 0.0081±0.0073 33.76±2.78 0.91±0.02 4665 0.097492 0.050363
K-DEQ-POCS 0.0078±0.0066 33.92±2.96 0.91±0.02 9419 0.331496 0.196750
H-Deep-SLR 0.0044±0.0022 35.87±1.80 0.91±0.02 6581 0.148643 0.085613
H-Deep-POCS 0.0043±0.0033 36.09±2.02 0.91±0.02 6421 0.160634 0.086382
H-DEQ-POCS 0.0043±0.0034 36.14±1.89 0.91±0.02 16599 0.580238 0.305063
TABLE III
QUANTITATIVE COMPARISON FOR VARIOUS METHODS ON THE BR AI N DATASE T.
Datasets Quantitative Evaluation
& Methods NMSE PSNR(dB) SSIM Training
Memory(MiB)
Training Time(s)
(Per-Iteration)
Testing Time(s)
(Per-Iteration)
Brain
(5-layer)
SPIRiT-POCS 0.0182±0.0079 34.64±2.46 0.91±0.03 / / /
K-Deep-SLR 0.0124±0.0131 37.33±1.92 0.88±0.09 2989 0.061975 0.032305
K-DEQ-POCS 0.0101±0.0118 38.57±2.26 0.89±0.08 4983 0.262126 0.131107
H-Deep-SLR 0.0058±0.0072 40.48±1.35 0.90±0.05 3765 0.105029 0.053262
H-Deep-POCS 0.0059±0.0069 40.49±1.53 0.91±0.06 3793 0.114285 0.053262
H-DEQ-POCS 0.0058±0.0069 40.61±1.52 0.92±0.05 8005 0.538723 0.291794
TABLE IV
QUANTITATIVE COMPARISON FOR VARIOUS METHODS ON THE KNEE
AND BRAIN DATA UNDER DIFFERENT SCALES OF INTERFERENCE ON
MEASUREMENT.
Datasets Quantitative Evaluation
& Methods NMSE PSNR(dB) SSIM
Knee
(1% noise)
SPIRiT-POCS 0.0089±0.0047 32.81±2.03 0.81±0.02
H-Deep-SLR 0.0050±0.0033 35.42±3.24 0.91±0.02
H-Deep-POCS 0.0056±0.0042 35.19±2.61 0.91±0.02
H-DEQ-POCS 0.0044±0.0037 36.12±1.99 0.91±0.02
Knee
(5% noise)
SPIRiT-POCS 0.0594±0.0103 24.23±1.74 0.53±0.09
H-Deep-SLR 0.0407±0.0086 25.88±1.24 0.61±0.05
H-Deep-POCS 0.1119±0.0227 21.49±1.05 0.46±0.05
H-DEQ-POCS 0.0112±0.0034 31.61±2.00 0.78±0.05
Knee
(10% noise)
SPIRiT-POCS 0.1567±0.0220 20.00±1.59 0.43±0.11
H-Deep-SLR 0.1872±0.0389 19.26±1.01 0.37±0.05
H-Deep-POCS 0.4193±0.0945 15.77±0.90 0.25±0.04
H-DEQ-POCS 0.0411±0.0048 25.79±1.41 0.54±0.06
Brain
(1% noise)
SPIRiT-POCS 0.2557±0.3084 26.26±4.99 0.77±0.08
H-Deep-SLR 0.3422±0.0727 21.71±3.09 0.38±0.04
H-Deep-POCS 0.0246±0.0086 33.20±1.50 0.58±0.06
H-DEQ-POCS 0.0132±0.0068 36.02±1.75 0.85±0.03
Brain
(5% noise)
SPIRiT-POCS 0.5915±0.2471 19.60±2.08 0.52±0.15
H-Deep-SLR 0.8497±0.0325 17.65±2.31 0.28±0.05
H-Deep-POCS 0.5273±0.1945 19.97±1.47 0.18±0.04
H-DEQ-POCS 0.1692±0.0189 24.68±1.93 0.31±0.05
Brain
(10% noise)
SPIRiT-POCS 0.7126±0.1876 18.56±1.84 0.48±0.18
H-Deep-SLR 0.8883±0.0218 17.46±2.23 0.29±0.05
H-Deep-POCS 1.3622±0.7336 16.12±1.33 0.09±0.02
H-DEQ-POCS 0.3809±0.0973 21.25±1.46 0.19±0.04
TABLE V
QUANTITATIVE COMPARISON FOR VARIOUS METHODS ON THE KNEE
AND BRAIN DATA UNDER DIFFERENT SCALES OF INTERFERENCE ON
INITIAL INPUT.
Datasets Quantitative Evaluation
& Methods NMSE PSNR(dB) SSIM
Knee
(1% noise)
SPIRiT-POCS 0.0071±0.0055 34.12±2.53 0.88±0.02
H-Deep-SLR 0.0045 ±0.0021 35.71±1.81 0.91±0.02
H-Deep-POCS 0.0049±0.0037 35.60±2.18 0.90±0.02
H-DEQ-POCS 0.0043±0.0034 36.14±1.89 0.90±0.02
Knee
(10% noise)
SPIRiT-POCS 0.0079±0.0064 33.72±2.59 0.88±0.02
H-Deep-SLR 0.0060±0.0023 34.40±1.90 0.89±0.03
H-Deep-POCS 0.0063±0.0042 34.58±2.49 0.90±0.03
H-DEQ-POCS 0.0043±0.0034 36.14±1.90 0.90±0.02
Knee
(50% noise)
SPIRiT-POCS 0.0098±0.0059 32.58±2.39 0.82±0.02
H-Deep-SLR 0.1081±0.0183 21.62±1.10 0.40±0.05
H-Deep-POCS 0.1080±0.0179 21.62±1.06 0.40±0.05
H-DEQ-POCS 0.0042±0.0031 36.15±1.86 0.90±0.02
Brain
(1% noise)
SPIRiT-POCS 0.0182±0.0079 34.64±2.46 0.91±0.03
H-Deep-SLR 0.0072±0.0074 39.26±1.33 0.88±0.05
H-Deep-POCS 0.0062±0.0070 40.19±1.47 0.90±0.06
H-DEQ-POCS 0.0058±0.0069 40.61±1.52 0.92±0.05
Brain
(10% noise)
SPIRiT-POCS 0.0183±0.0080 34.62±2.46 0.91±0.03
H-Deep-SLR 0.0806±0.0167 27.95±1.64 0.40±0.06
H-Deep-POCS 0.0635±0.0157 29.02±1.64 0.47±0.06
H-DEQ-POCS 0.0058±0.0069 40.56±1.51 0.92±0.05
Brain
(50% noise)
SPIRiT-POCS 0.0206±0.0091 34.09±2.46 0.89±0.03
H-Deep-SLR 0.6903±0.0213 18.55±2.21 0.21±0.03
H-Deep-POCS 0.8699±0.4516 18.00±1.23 0.11±0.03
H-DEQ-POCS 0.0092±0.0070 37.86±1.61 0.90±0.05
This article has been accepted for publication in IEEE Transactions on Medical Imaging. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/TMI.2023.3293826
© 2023 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See https://www.ieee.org/publications/rights/index.html for more information.
Authorized licensed use limited to: Shenzhen Institute of Advanced Technology CAS. Downloaded on July 13,2023 at 01:50:40 UTC from IEEE Xplore. Restrictions apply.
10 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. XX, NO. XX, XXXX 2022
Fig. 7. Quantitative analysis of PSNR metric performance across
different levels of measurement interference in various methods
Fig. 8. The first and third rows illustrate the reconstruction results from
noisy measurement data yδ:= y+nwhich contain ∥n∥F/∥y∥F=
5% Gaussian noise on knee measurement and ∥n∥F/∥y∥F= 1%
Gaussian noise on brain measurement, respectively. The second and
fourth rows illustrate the error views. The values in the corner are
the NMSE/PSNR/SSIM values of each slice. The grayscale of the
reconstructed images and the error images’ color bar are on the figure’s
right.
Fig. 9. Quantitative analysis of PSNR metric performance across
different levels of initial input interference in various methods.
perturbation, while the performance of the comparison meth-
ods, H-Deep-SLR and H-Deep-POCS, degrades sharply with
increasing initial input interference.
Fig. 10. The first and third rows illustrate the reconstruction results from
noisy initial input b
xδ,0:= b
x0+nwhich contain ∥n∥F/∥b
x0∥F= 50%
Gaussian noise on knee data and ∥n∥F/∥b
x0∥F= 10% Gaussian
noise on brain data, respectively. The second and fourth rows illustrate
the error views. The values in the corner are the NMSE/PSNR/SSIM
values of each slice. The grayscale of the reconstructed images and the
error images’ color bar are on the figure’s right.
The reconstruction results of various methods are presented
in Figure 8 for knee and brain data, respectively, when the
initial input contains 10% noise. As shown in Table V and
Figure 9, the performance of the quantification results is
consistent with the reconstruction results. Notably, H-Deep-
SLR and H-Deep-POCS heavily amplify the noise in the
initial input, while the proposed method and SPIRiT-POCS
are almost unaffected by the noise in the initial input. This
experimental result confirms the convergence of the proposed
method and suggests that the blank UDN (without DEQ)
generally fails to reach convergence.
E. Sampling Pattern Shifts
Ideally, if the UDN is consistent with the regularization
models, the network module will learn the regularizer or its
first-order information independently of the sampling trajec-
tories. This means that we can train the UDN under one
sampling trajectory and subsequently shift it to another trajec-
tory. To verify the consistency of our method, we tested the
performance of the UDN on different trajectories. Specifically,
we trained the UDN on both 4-fold 1-D and 6-fold 2-D random
trajectories and then shifted it to 3-fold 1-D and 6-fold 2-D
regular trajectories, respectively.
Figure 11 depicts the reconstruction outcomes of the UDN
shifting technique trained on 1-D regular undersampling with
an acceleration factor of 3 for knee data and 2-D regular
undersampling with an acceleration factor of 6 for brain data,
This article has been accepted for publication in IEEE Transactions on Medical Imaging. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/TMI.2023.3293826
© 2023 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See https://www.ieee.org/publications/rights/index.html for more information.
Authorized licensed use limited to: Shenzhen Institute of Advanced Technology CAS. Downloaded on July 13,2023 at 01:50:40 UTC from IEEE Xplore. Restrictions apply.
ZHUO-XU CUI: EQUILIBRATED ZEROTH-ORDER UNROLLED DEEP NETWORK FOR PARALLEL MR IMAGING 11
Fig. 11. Reconstruction results under 1-D regular undersampling at
R= 3 on knee data and 2-D regular undersampling at R= 6 on
brain data. The values in the corner are each slice’s NMSE/PSNR/SSIM
values. The second and third rows illustrate the enlarged and error
views, respectively. The grayscale of the reconstructed images and the
error images’ color bar are on the figure’s right.
TABLE VI
QUANTITATIVE COMPARISON FOR VARIOUS METHODS ON THE KNEE
AN D BR AIN DATASET.
Datasets Quantitative Evaluation
& Methods NMSE PSNR(dB) SSIM
Knee
(5-layer)
H-Deep-SLR 0.0073±0.0042 33.96±2.88 0.93±0.01
H-Deep-POCS 0.0032±0.0012 37.09±1.68 0.94±0.01
H-DEQ-POCS 0.0024±0.0009 38.38±1.29 0.94±0.01
Brain
(5-layer)
H-Deep-SLR 0.0071±0.0070 39.34±1.34 0.90±0.04
H-Deep-POCS 0.0076±0.0092 39.57±1.68 0.90±0.06
H-DEQ-POCS 0.0062±0.0080 40.38±1.53 0.93±0.05
respectively. As shown in Figure 11, the proposed approach ex-
hibits remarkable suppression of aliasing patterns on the knee
data and noise artifacts on the brain data. Table VI presents
the quantitative metrics, which consistently demonstrate that
the proposed method outperforms the comparison methods,
namely H-Deep-SLR and H-Deep-POCS.
After reviewing Figures 5 and 6 and Tables II and III, it can
be observed that the difference in performance between the
proposed method and the comparison methods is negligible
before the mask shift. However, following the mask shift,
the proposed method significantly surpasses the comparison
methods. This result strongly corroborates the consistency of
the proposed approach.
VII. DISCUSSION
In this study, we introduce a novel zeroth-order UDN)that
solves the k-space PI regularization problem in an equili-
brated manner. We refer to this method as the DEQ-POCS
network. Theoretical analysis demonstrates its consistency,
convergence, and robustness. These findings are supported
by comparative experiments on image reconstruction quality,
robustness to noise interference, and robustness to distribution
shifts. Our approach has the potential to be applied to a wider
range of applications, although further improvements are still
possible.
A. Extension When Coil Sensitivity is Available
The success of the proposed model presented in this paper
heavily depends on the network module Φϕ, which plays a cru-
cial role in learning self-consistency for multi-channel k-space
data. Importantly, previous studies have demonstrated that self-
consistency also applies to single-channel k-space data [23].
As a result, the proposed model can be readily adapted to
accommodate scenarios where coil sensitivity information is
available. In particular, problem (3) can be reduced to
(min
b
x∈CN×Nc
R(b
x) := ∥b
x−b
x⊗w∥2
s.t. MFFT(Sx) = y
where Sis the coil sensitivity. In general, the above problem
can be solved by projected gradient descent (PGD) algorithm:
(xk+1
2=xk−SHFFT−1MH(MFFT(Sxk)−y)
b
xk+1 =b
xk+1
2⊗w
which can be unrolled as:
xk+1
2=xk−SHFFT−1MH(MFFT(Sxk)−y)
b
xk+1 = Φϕb
xk+1
2.
Following the same network architecture and training proce-
dure of H-DEQ-POCS, we obtain the H-DEQ-PGD network.
Fig. 12. Reconstruction results under 1-D random undersampling at
R= 4. The values in the corner are each slice’s NMSE/PSNR/SSIM
values. The second row illustrates the error views. The grayscale of the
reconstructed images and the error images’ color bar are on the figure’s
right.
To validate the effectiveness of our H-DEQ-PGD algorithm,
we conducted a comparative analysis with ISTA-Net+. The
reconstruction outcomes of both techniques under 1-D random
4-fold undersampling are presented in Figure 12. Although
both methods visually restored the image effectively, our
This article has been accepted for publication in IEEE Transactions on Medical Imaging. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/TMI.2023.3293826
© 2023 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See https://www.ieee.org/publications/rights/index.html for more information.
Authorized licensed use limited to: Shenzhen Institute of Advanced Technology CAS. Downloaded on July 13,2023 at 01:50:40 UTC from IEEE Xplore. Restrictions apply.
12 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. XX, NO. XX, XXXX 2022
TABLE VII
QUANTITATIVE COMPARISON FOR VARIOUS METHODS ON THE KNEE
DATASET.
Datasets Quantitative Evaluation
& Methods NMSE PSNR(dB) SSIM
Knee
(5-layer)
ISTA-Net+ 0.0030±0.0018 37.67±2.42 0.94±0.01
H-DEQ-PGD 0.0028 ±0.0017 38.07±2.53 0.94±0.02
proposed approach exhibited marginally better reconstruction
accuracy. Furthermore, the quantitative metrics, as illustrated
in Table VII, reinforce the competitiveness of our proposed
method.
B. Improvement on Loss Function
In this paper, we employ the Frobenius-norm as the loss
function, which assumes that the k-space data follows a sub-
Gaussian distribution by default. However, this assumption
may not be realistic, and some recent works suggest that
the Wasserstein distance may be a better choice for the
loss function [48], [49]. The Wasserstein distance measures
the distance between distributions, specifically the distance
between the distribution πb
xto which the real k-space data
belongs and the distribution πb
x∞(ϕ,y)to which the fixed points
belong. Therefore, in Algorithm 1, we use the Wasserstein
distance as the loss function, which is defined as:
min
ϕW1(πb
x∞(ϕ,y), πb
x)
= min
ϕmax
f∈Lip1Zf(z)dπb
x∞(ϕ,y)−Zf(z)dπb
x
= min
ϕmax
f∈Lip1Zf(b
x∞(ϕ, y))dπy−Zf(z)dπb
x
Here, W1represents the 1-Wasserstein distance, and we em-
ploy a pyramid discriminator representation fas described
in [49]. In Algorithm 1, we train the network Φϕwhile
alternately maximizing the discriminator f. By introducing
the Wasserstein loss, our proposed method, W-DEQ-POCS,
is coupled with WGAN. The reconstruction results on ran-
dom 4-fold undersampled knee data presented in Figure 13
show that W-DEQ-POCS outperforms the original H-DEQ-
POCS in reconstructing image details, which aligns with the
original goal of modeling the Wasserstein loss. The advanced
Wasserstein loss is able to prevent blurring of details caused
by the Gaussian assumption. Furthermore, the SSIM metric in
Table VIII, which is more sensitive to image details, provides
further evidence of W-DEQ-POCS’s superior performance in
reconstructing image details.
TABLE VIII
QUANTITATIVE COMPARISON FOR VARIOUS METHODS ON THE KNEE
DATASET.
Datasets Quantitative Evaluation
& Methods NMSE PSNR(dB) SSIM
Knee
(5-layer)
H-DEQ-POCS 0.0043±0.0034 36.14±1.89 0.90±0.02
W-DEQ-POCS 0.0051±0.0035 35.46±2.53 0.92±0.01
A-DEQ-POCS 0.0037±0.0016 36.51±1.74 0.94±0.01
Fig. 13. Reconstruction results under 1-D random undersampling at
R= 4. The values in the corner are each slice’s NMSE/PSNR/SSIM
values. The second row illustrates the error views. The grayscale of the
reconstructed images and the error images’ color bar are on the figure’s
right.
C. Improvement on Network Architecture
On the one hand, this paper utilizes CNNs to characterize
the self-consistency term Φϕin order to ensure the consistency
between the unrolled network and SPIRiT model. However,
it is worth noting that there has been a recent surge in
the development of interpretable transformer models that are
based on attention mechanisms [50]. Therefore, exploring the
possibility of replacing the CNN with a new high-performance
network module to characterize Φϕcould be a promising
direction for enhancing the proposed model.
To this end, we follow the approach outlined in [51] and
attempt to replace the CNN module in Figure 1 with an
attention mechanism-guided UNet network. We denote this
new model as A-DEQ-POCS. The results of our experiments,
as shown in Figure 13 and Table VIII, reveal that the A-DEQ-
POCS model outperforms the original H-DEQ-POCS model
in terms of image details reconstruction, particularly in regard
to the SSIM metric which is sensitive to image details. This
improvement can be attributed to the attention mechanism’s
ability to focus on high-frequency components such as image
details.
D. Future Work
There are several areas for improvement that we plan to
explore in our future work.
First, our proposed method is currently only applicable
to the k-space interpolation model and cannot be used to
solve general image-domain inverse problems. Additionally,
Proposition 4.1 can only provide an approximation condition
between the network output solution and the true solution if
the assumptions in Remark 4.3 are not met, and it cannot
guarantee complete reconstruction. To address this, we plan to
leverage complete recovery conditions such as the null space
property (NSP) and restricted isometry property (RIP) inspired
by CS or matrix completion theory to develop DL-based MR
reconstruction methods with complete recovery properties.
This article has been accepted for publication in IEEE Transactions on Medical Imaging. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/TMI.2023.3293826
© 2023 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See https://www.ieee.org/publications/rights/index.html for more information.
Authorized licensed use limited to: Shenzhen Institute of Advanced Technology CAS. Downloaded on July 13,2023 at 01:50:40 UTC from IEEE Xplore. Restrictions apply.
ZHUO-XU CUI: EQUILIBRATED ZEROTH-ORDER UNROLLED DEEP NETWORK FOR PARALLEL MR IMAGING 13
In Theorem 4.2, we have proven that the DEQ-POCS
network is a regularization method. However, when the mea-
surement data contain noise, iterative methods tend to exhibit
a semiconvergence property, as illustrated in some literature
[52], [53]. This means that the iteration becomes close to
the sought solution at the beginning of the iteration but
then moves far away as the iteration progresses. To address
this, a suitable termination criterion is essential for iterative
regularization methods, but it can only be implemented if the
noise intensity is known in advance. Recent works suggest that
a Bayesian approach can avoid early termination and provide
reconstruction uncertainty [52], [53]. Therefore, we plan to
redesign our DEQ-POCS network with a Bayesian framework
in the future to improve its performance.
In Theorem 4.2, we have proven that the DEQ-POCS
network is a regularization method. However, when the mea-
surement data contain noise, iterative methods tend to exhibit
a semiconvergence property, as illustrated in some literature
[52], [53]. This means that the iteration becomes close to
the sought solution at the beginning of the iteration but
then moves far away as the iteration progresses. To address
this, a suitable termination criterion is essential for iterative
regularization methods, but it can only be implemented if the
noise intensity is known in advance. Recent works suggest that
a Bayesian approach can avoid early termination and provide
reconstruction uncertainty [52], [53]. Therefore, we plan to
redesign our DEQ-POCS network with a Bayesian framework
in the future to improve its performance.
VIII. CONCLUSION
In this paper, we proposed a novel approach for the k-
space PI regularization problem, called the DEQ-POCS net-
work, which is a zeroth-order UDN based on the generalized
POCS algorithm. Unlike the first-order UDN, the DEQ-POCS
network inherits the explainable and predictable nature of the
k-space of a generalized PI regularization model. Theoretical
analysis revealed that under certain conditions, the DEQ-
POCS network is guaranteed to converge to a fixed point that
approximates the true full-sampled k-space data. Moreover, we
demonstrated that the proposed method is robust against noisy
measurements. Experimental results showed that the DEQ-
POCS network outperforms existing state-of-the-art k-space
first-order UDNs and traditional methods. Our proposed ap-
proach has the potential to become a powerful framework for
parallel MR imaging, and we believe that further development
of this kind of method may lead to even greater gains in the
future.
APPENDIX
A. Proof of Theorem 4.1
Define C={b
x∈Cd|Mb
x=y}, we have
∥b
xk+1 −b
xk∥F
=∥PC(Φϕi(b
xk)) − PC(Φϕi(b
xk−1))∥F
=∥(I − M)(Φϕi(b
xk)−Φϕi(b
xk−1)) + y−y∥F
≤∥(I − M)∥∥Φϕi(b
xk)−Φϕi(b
xk−1)∥F
≤L∥b
xk−b
xk−1∥F
the first equality is due to the definition of PCand the last
inequality is due to 0⪯ I − M ⪯ I and the L-Lipschitz
continuity of Φϕi. Through recursion on above inequality, we
have
∥b
xk+1 −b
xk∥F≤Lk∥b
x1−b
x0∥F.
Summing the above inequality from 0 to ∞, we have
∞
X
k=1 ∥b
xk+1 −b
xk∥F≤1
1−L∥b
x1−b
x0∥F≤+∞
which means that the {b
xk}is a Cauchy sequence. Then, it
converges to a fixed point globally. The proof is completed.
B. Proof of Proposition 4.1
For the fixed point of unrolled POCS algorithm
with measurement ymand self-consistency ΦϕKM ,
i.e., b
x∞
m=P{b
x|Mb
x=ym}ΦϕKM (b
x∞
m),m∈
{1, . . . , M }, we define a sequence of random variables
Xm:= b
x∞
m−b
xm−Eπb
x×y[b
x∞−b
x]M
m=1, where
(b
x×y)∼πb
x×yand b
x∞=P{b
x|Mb
x=y}ΦϕKM (b
x∞).
Since (b
xm,ym)is sampled in distribution πb
x×y, we have
E[Xm] = 0. Since b
x∞
mis the fixed point, it has to be bounded.
Then, there exists a constant B > 0such that ∥Xm∥ ≤ B.
By the Hoeffding’s inequality (please see Theorem 7.20 of
literature [54] for detail), it holds
P"1
M
M
X
m=1{b
x∞
m−b
xm−Eπb
x×y[b
x∞−b
x]}
F≥t1
M#
≥P"∥Eπb
x×y[b
x∞−b
x]∥F≥t1
M+1
M
M
X
m=1
(b
x∞
m−b
xm)
F#
≥P"∥Eπb
x×y[b
x∞−b
x]∥F≥t1
M+1
M
M
X
m=1 ∥b
x∞
m−b
xm∥F#
≥P∥Eπb
x×y[b
x∞−b
x]∥F≥t1
M+ϵ(8)
with probability at most 2 exp −t2
1
2MB 2, where the first in-
equality is due to the triangle inequality of Frobenius norm,
the second inequality is due to the convexity of Frobenius
norm and the last inequality is due to the Assumption 2. Using
the the Hoeffding’s inequality again, for the output b
x∞of
Algorithm 2, it holds
P∥b
x∞−b
x†−Eπb
x×y[b
x∞−b
x]∥F≥t2
≥P∥b
x∞−b
x†∥F≥t2+∥Eπb
x×y[b
x∞−b
x]∥F(9)
with probability at most 2 exp −t2
2
2B2. Combining inequalities
(8) and (9) together, the result is yielded.
C. Proof of Theorem 4.2
Let {b
xk}denote the unrolled POCS iteration with noise-free
measurement, i.e., b
xk+1 =P{b
x|Mb
x=y}(ΦϕKM (b
xk)) and Let
{b
xδ,k}denote the unrolled POCS iteration with noisy mea-
surement, i.e., b
xδ,k+1 =P{b
x|Mb
x=yδ}(ΦϕKM (b
xδ,k)). Then,
This article has been accepted for publication in IEEE Transactions on Medical Imaging. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/TMI.2023.3293826
© 2023 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See https://www.ieee.org/publications/rights/index.html for more information.
Authorized licensed use limited to: Shenzhen Institute of Advanced Technology CAS. Downloaded on July 13,2023 at 01:50:40 UTC from IEEE Xplore. Restrictions apply.
14 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. XX, NO. XX, XXXX 2022
we have
∥b
xδ,k+1 −b
xk+1∥F
=∥P{b
x|Mb
x=yδ}(ΦϕKM (b
xδ,k)) − P{b
x|Mb
x=y}(ΦϕKM (b
xk))∥F
=∥(I − M)(ΦϕK M (b
xδ,k)−ΦϕK M (b
xk)) + yδ−y∥F
≤∥(I − M)∥∥ΦϕK M (b
xδ,k)−ΦϕK M (b
xk)∥F+∥yδ−y∥F
≤L∥b
xδ,k −b
xk∥F+δ
the first equality is due to the definition of projection operator
and the last inequality is due to 0⪯ I − M ⪯ I and the
L-Lipschitz continuity of ΦϕKM . Through recursion on above
inequality, we have
∥b
xδ,∞−b
x∞∥F≤
∞
X
k=1
Lk−1δ=δ
1−L
where b
xδ,∞and b
x∞denote the fixed points of iterations
{b
xδ,k}and {b
xk}respectively. The proof is completed.
REFERENCES
[1] Z.-P. Liang, F. Boada, R. Constable, E. Haacke, P. Lauterbur, and
M. Smith, “Constrained reconstruction methods in mr imaging,” Rev
Magn Reson Med, vol. 4, no. 2, pp. 67–185, 1992.
[2] D. K. Sodickson and W. J. Manning, “Simultaneous acquisition of spatial
harmonics (smash): Fast imaging with radiofrequency coil arrays,”
Magnetic Resonance in Medicine, vol. 38, no. 4, pp. 591–603, 1997.
[3] K. P. Pruessmann, M. Weiger, M. B. Scheidegger, and P. Boesiger,
“Sense: Sensitivity encoding for fast mri,” Magnetic Resonance in
Medicine, vol. 42, no. 5, pp. 952–962, 1999.
[4] E. J. Cand`
es, J. Romberg, and T. Tao, “Robust uncertainty principles: ex-
act signal reconstruction from highly incomplete frequency information,”
IEEE Transactions on Information Theory, vol. 52, no. 2, pp. 489–509,
2006.
[5] D. L. Donoho, “Compressed sensing,” IEEE Transactions on information
theory, vol. 52, no. 4, pp. 1289–1306, 2006.
[6] M. Lustig, D. L. Donoho, and J. M. Pauly, “Sparse MRI: The application
of compressed sensing for rapid mr imaging,” Magnetic Resonance in
Medicine, vol. 58, no. 6, pp. 1182–1195, 2007.
[7] D. Liang, B. Liu, J. Wang, and L. Ying, “Accelerating sense using
compressed sensing,” Magnetic Resonance in Medicine, vol. 62, no. 6,
pp. 1574–1584, 2009.
[8] H. She, R.-R. Chen, D. Liang, E. V. R. DiBella, and L. Ying, “Sparse
blip: Blind iterative parallel imaging reconstruction using compressed
sensing,” Magnetic Resonance in Medicine, vol. 71, no. 2, pp. 645–660,
2014.
[9] S. Wang, X. Peng, P. Dong, L. Ying, D. Dagan Feng, and D. Liang,
“Parallel imaging via sparse representation over a learned dictionary,”
in 2015 IEEE 12th International Symposium on Biomedical Imaging
(ISBI), pp. 687–690, 2015.
[10] S. Wang, S. Tan, Y. Gao, Q. Liu, L. Ying, T. Xiao, Y. Liu, X. Liu,
H. Zheng, and D. Liang, “Learning joint-sparse codes for calibration-free
parallel mr imaging,” IEEE Transactions on Medical Imaging, vol. 37,
no. 1, pp. 251–261, 2018.
[11] S. Wang, Z. Su, L. Ying, X. Peng, S. Zhu, F. Liang, D. Feng, and
D. Liang, “Accelerating magnetic resonance imaging via deep learning,”
in 2016 IEEE 13th International Symposium on Biomedical Imaging
(ISBI), pp. 514–517, 2016.
[12] D. Liang, J. Cheng, Z. Ke, and L. Ying, “Deep magnetic resonance
image reconstruction: Inverse problems meet neural networks,” IEEE
Signal Processing Magazine, vol. 37, no. 1, pp. 141–151, 2020.
[13] W. Huang, Z. Ke, Z.-X. Cui, J. Cheng, and D. Liang, “Deep low-rank
plus sparse network for dynamic mr imaging,” Medical Image Analysis,
p. 102190, 2021.
[14] J. Cheng, Z.-X. Cui, W. Huang, Z. Ke, L. Ying, H. Wang, Y. Zhu, and
D. Liang, “Learning data consistency and its application to dynamic
mr imaging,” IEEE Transactions on Medical Imaging, vol. 40, no. 11,
pp. 3140–3153, 2021.
[15] Z. Ke, W. Huang, Z.-X. Cui, J. Cheng, S. Jia, H. Wang, X. Liu, H. Zheng,
L. Ying, Y. Zhu, and D. Liang, “Learned low-rank priors in dynamic
mr imaging,” IEEE Transactions on Medical Imaging, pp. 1–1, 2021.
[16] Z. Ke, Z.-X. Cui, W. Huang, J. Cheng, S. Jia, L. Ying, Y. Zhu, and
D. Liang, “Deep manifold learning for dynamic mr imaging,” IEEE
Transactions on Computational Imaging, pp. 1–1, 2021.
[17] B. Zhu, J. Z. Liu, S. F. Cauley, B. R. Rosen, and M. S. Rosen,
“Image reconstruction by domain-transform manifold learning,” Nature,
vol. 555, no. 7697, pp. 1476–4687, 2018.
[18] K. Gregor and Y. LeCun, “Learning fast approximations of sparse cod-
ing,” in Proceedings of International Conference on Machine Learning,
pp. 399–406, 2010.
[19] Y. Yang, J. Sun, H. Li, and Z. Xu, “Deep ADMM-Net for compressive
sensing MRI,” in Advances in Neural Information Processing Systems,
2016.
[20] J. Zhang and B. Ghanem, “Ista-net: Interpretable optimization-inspired
deep network for image compressive sensing,” in Proceedings of the
IEEE Conference on Computer Vision and Pattern Recognition (CVPR),
June 2018.
[21] J. Adler and O. ¨
Oktem, “Learned primal-dual reconstruction,” IEEE
Transactions on Medical Imaging, vol. 37, no. 6, pp. 1322–1332, 2018.
[22] K. Hammernik, T. Klatzer, E. Kobler, M. P. Recht, D. K. Sodickson,
T. Pock, and F. Knoll, “Learning a variational network for reconstruction
of accelerated mri data,” Magnetic Resonance in Medicine, vol. 79, no. 6,
pp. 3055–3071, 2018.
[23] A. Pramanik, H. K. Aggarwal, and M. Jacob, “Deep generalization
of structured low-rank algorithms (deep-slr),” IEEE Transactions on
Medical Imaging, vol. 39, no. 12, pp. 4186–4197, 2020.
[24] T. Lu, X. Zhang, Y. Huang, D. Guo, F. Huang, Q. Xu, Y. Hu, L. Ou-
Yang, J. Lin, Z. Yan, and X. Qu, “pfista-sense-resnet for parallel mri
reconstruction,” Journal of Magnetic Resonance, vol. 318, p. 106790,
2020.
[25] Y. Zhou, C. Qian, Y. Guo, Z. Wang, J. Wang, B. Qu, D. Guo, Y. You, and
X. Qu, “Xcloud-pfista: A medical intelligence cloud for accelerated mri,”
in 2021 43rd Annual International Conference of the IEEE Engineering
in Medicine & Biology Society (EMBC), pp. 3289–3292, 2021.
[26] D. Gilton, G. Ongie, and R. Willett, “Deep equilibrium architectures
for inverse problems in imaging,” arXiv preprint arXiv:2102.07944v1,
2021.
[27] V. Antun, F. Renna, C. Poon, B. Adcock, and A. C. Hansen, “On
instabilities of deep learning in image reconstruction and the potential
costs of ai,” Proceedings of the National Academy of Sciences, vol. 117,
no. 48, pp. 30088–30095, 2020.
[28] K. P. Pruessmann, M. Weiger, M. B. Scheidegger, and P. Boesiger,
“Sense: sensitivity encoding for fast mri,” Magnetic Resonance in
Medicine: An Official Journal of the International Society for Magnetic
Resonance in Medicine, vol. 42, no. 5, pp. 952–962, 1999.
[29] F. Knoll, C. Clason, K. Bredies, M. Uecker, and R. Stollberger, “Par-
allel imaging with nonlinear reconstruction using variational penalties,”
Magnetic Resonance in Medicine, vol. 67, no. 1, pp. 34–41, 2012.
[30] M. A. Griswold, P. M. Jakob, R. M. Heidemann, M. Nittka, V. Jellus,
J. Wang, B. Kiefer, and A. Haase, “Generalized autocalibrating par-
tially parallel acquisitions (grappa),” Magnetic Resonance in Medicine,
vol. 47, no. 6, pp. 1202–1210, 2002.
[31] M. Lustig and J. M. Pauly, “Spirit: Iterative self-consistent parallel
imaging reconstruction from arbitrary k-space,” Magnetic Resonance in
Medicine, vol. 64, no. 2, pp. 457–471, 2010.
[32] I. Y. Chun, Z. Huang, H. Lim, and J. Fessler, “Momentum-net: Fast
and convergent iterative neural network for inverse problems,” IEEE
Transactions on Pattern Analysis and Machine Intelligence, pp. 1–1,
2020.
[33] T. Chen, X. Chen, W. Chen, Z. Wang, H. Heaton, J. Liu, and W. Yin,
“Learning to optimize: A primer and a benchmark,” arXiv preprint
arXiv:2103.12828v2, 2021.
[34] S. Bai, J. Z. Kolter, and V. Koltun, “Deep equilibrium models,” in
Advances in Neural Information Processing Systems, 2019.
[35] S. Bai, V. Koltun, and J. Z. Kolter, “Multiscale deep equilibrium
models,” in Advances in Neural Information Processing Systems, 2020.
[36] E. Winston and J. Z. Kolter, “Monotone operator equilibrium networks,”
2020.
[37] H. Heaton, S. W. Fung, A. Gibali, and W. Yin, “Feasibility-based fixed
point networks,” arXiv preprint arXiv:2104.14090v1, 2021.
[38] T. Eo, Y. Jun, T. Kim, J. Jang, H.-J. Lee, and D. Hwang, “Kiki-net: cross-
domain convolutional neural networks for reconstructing undersampled
magnetic resonance images,” Magnetic Resonance in Medicine, vol. 80,
no. 5, pp. 2188–2201, 2018.
[39] H. K. Aggarwal, M. P. Mani, and M. Jacob, “Modl: Model-based
deep learning architecture for inverse problems,” IEEE Transactions on
Medical Imaging, vol. 38, no. 2, pp. 394–405, 2019.
This article has been accepted for publication in IEEE Transactions on Medical Imaging. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/TMI.2023.3293826
© 2023 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See https://www.ieee.org/publications/rights/index.html for more information.
Authorized licensed use limited to: Shenzhen Institute of Advanced Technology CAS. Downloaded on July 13,2023 at 01:50:40 UTC from IEEE Xplore. Restrictions apply.
ZHUO-XU CUI: EQUILIBRATED ZEROTH-ORDER UNROLLED DEEP NETWORK FOR PARALLEL MR IMAGING 15
[40] H. F. Walker and P. Ni, “Anderson acceleration for fixed-point iterations,”
SIAM Journal on Numerical Analysis, vol. 49, no. 4, pp. 1715–1735,
2011.
[41] R. Liu, S. Cheng, Y. He, X. Fan, Z. Lin, and Z. Luo, “On the
convergence of learning-based iterative methods for nonconvex inverse
problems,” IEEE Transactions on Pattern Analysis and Machine Intel-
ligence, vol. 42, no. 12, pp. 3027–3039, 2020.
[42] Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature, vol. 521,
pp. 436– 444, 2015.
[43] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image
recognition,” in Proceedings of the IEEE Conference on Computer Vision
and Pattern Recognition (CVPR), 2016.
[44] Z. Allen-Zhu, Y. Li, and Z. Song, “A convergence theory for deep learn-
ing via over-parameterization,” in Proceedings of the 36th International
Conference on Machine Learning, pp. 242–252, 2019.
[45] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,”
in Proceedings of the International Conference on Learning Represen-
tations, 2014.
[46] A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan,
T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. K¨
opf,
E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner,
L. Fang, J. Bai, and S. Chintala, “Pytorch: An imperative style, high-
performance deep learning library,” 2019.
[47] Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image
quality assessment: from error visibility to structural similarity,” IEEE
Transactions on Image Processing, vol. 13, no. 4, pp. 600–612, 2004.
[48] I. Gulrajani, F. Ahmed, M. Arjovsky, V. Dumoulin, and A. C. Courville,
“Improved training of wasserstein gans,” in Advances in Neural In-
formation Processing Systems (I. Guyon, U. V. Luxburg, S. Bengio,
H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, eds.), vol. 30,
2017.
[49] B. Sim, G. Oh, J. Kim, C. Jung, and J. C. Ye, “Optimal transport driven
cyclegan for unsupervised learning in inverse problems,” SIAM Journal
on Imaging Sciences, vol. 13, no. 4, pp. 2281–2306, 2020.
[50] K. Han, Y. Wang, H. Chen, X. Chen, J. Guo, Z. Liu, Y. Tang, A. Xiao,
C. Xu, Y. Xu, Z. Yang, Y. Zhang, and D. Tao, “A survey on vision
transformer,” IEEE Transactions on Pattern Analysis and Machine
Intelligence, vol. 45, no. 1, pp. 87–110, 2023.
[51] J. Lee, J. Gu, and J. C. Ye, “Unsupervised ct metal artifact learning using
attention-guided β-cyclegan,” IEEE Transactions on Medical Imaging,
vol. 40, no. 12, pp. 3932–3944, 2021.
[52] Z. Cheng, M. Gadelha, S. Maji, and D. Sheldon, “A bayesian perspective
on the deep image prior,” in Proceedings of the IEEE/CVF Conference
on Computer Vision and Pattern Recognition (CVPR), 2019.
[53] M.-H. Laves, M. T¨
olle, and T. Ortmaier, “Uncertainty estimation in med-
ical image denoising with bayesian deep image prior,” in Uncertainty for
Safe Utilization of Machine Learning in Medical Imaging, and Graphs
in Biomedical Image Analysis, pp. 81–96, 2020.
[54] S. Foucart and H. Rauhut, A Mathematical Introduction to Compressive
Sensing. New York: Birkh ¨
auser Basel, 2013.
This article has been accepted for publication in IEEE Transactions on Medical Imaging. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/TMI.2023.3293826
© 2023 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See https://www.ieee.org/publications/rights/index.html for more information.
Authorized licensed use limited to: Shenzhen Institute of Advanced Technology CAS. Downloaded on July 13,2023 at 01:50:40 UTC from IEEE Xplore. Restrictions apply.
