Precoding Design for Multi-Group MIMO-NOMA Scheme With SIC Residual Analysis

Abstract

This paper presents a precoder design for multiple-input multiple-output (MIMO) uplink communications with a multi-group non-orthogonal multiple access (NOMA) scheme. In this scheme, QR-decomposition-based detection is used to decode user signals of each group under imperfect successive interference cancellation (SIC). In the uplink, transmitted signals are encoded with the constellation of a symmetric quadrature amplitude modulation (QAM), and SIC residual errors are generated from symbol demodulation. The statistics of the SIC errors are derived, with which expressions of approximate SIC-residual-power coefficient and signal-to-interference-plus-noise ratio (SINR) are developed. Consequently, the effect of the SIC errors can be explicitly addressed in the precoder design for minimizing the total transmit power subject to users' symbol error probability (SEP) constraints. A suboptimal design algorithm with efficient “logarithmic” golden-section search is developed. Simulation results show the effectiveness of the design criterion and algorithm, and validate the detection error analysis. Unlike the conventional fractional-SIC-error model that suffers from underestimation/overestimation of SIC errors and infeasibility, our SIC error model can provide accurate effect of the SIC errors in the SINR analysis and enable the proposed method to give design solutions to satisfy the SEP constraints.

Full text

1
Precoding Design for Multi-Group MIMO-NOMA
Scheme With SIC Residual Analysis
Rongbin Zhang, Huajiao Liu, Shu-Hung Leung, Yue Zhang, Weijun Tang, Hong Wang, Zhen Luo, and Sumit Roy
Abstract—This paper presents a precoder design for multiple-
input multiple-output (MIMO) uplink communications with a
multi-group non-orthogonal multiple access (NOMA) scheme.
In this scheme, QR-decomposition-based detection is used to
decode user signals of each group under imperfect successive
interference cancellation (SIC). In the uplink, transmitted signals
are encoded with the constellation of a symmetric quadrature
amplitude modulation (QAM), and SIC residual errors are
generated from symbol demodulation. The statistics of the SIC
errors are derived, with which expressions of approximate SIC-
residual-power coefficient and signal-to-interference-plus-noise
ratio (SINR) are developed. Consequently, the effect of the SIC
errors can be explicitly addressed in the precoder design for
minimizing the total transmit power subject to users’ symbol er-
ror probability (SEP) constraints. A suboptimal design algorithm
with efficient “logarithmic” golden-section search is developed.
Simulation results show the effectiveness of the design criterion
and algorithm, and validate the detection error analysis. Unlike
the conventional fractional-SIC-error model that suffers from
underestimation/overestimation of SIC errors and infeasibility,
our SIC error model can provide accurate effect of the SIC
errors in the SINR analysis and enable the proposed method to
give design solutions to satisfy the SEP constraints.
Index Terms—MIMO, NOMA, precoder design, imperfect SIC,
SIC residual analysis.
Copyright (c) 2015 IEEE. Personal use of this material is permitted.
However, permission to use this material for any other purposes must be
obtained from the IEEE by sending a request to pubspermissions@ieee.org.
Manuscript received January 28, 2022; revised July 9 and September 24,
2022; accepted November 22, 2022. (Corresponding author: Rongbin Zhang.)
This work was supported in part by the Guangdong Basic and Applied
Basic Research Foundation under Grant 2021A1515111053, in part by the
National Natural Science Foundation of China under Grants 62171237 and
62201336, in part by the Guangdong Natural Science Foundation under
Grant 208184073052, in part by the STU Scientific Research Initiation Grant
(SRIG) of Shantou University under Grants NTF21024 and NTF21039, in
part by the Project of Chongqing Natural Science Foundation under Grant
CSTB2022NSCQ-MSX0990, in part by the Science and Technology Research
Project of Chongqing Education Commission under Grant KJQN202000612,
and in part by the Venture and Innovation Support Program for Chongqing
Overseas Returnees under Grant cx2020070.
R. Zhang, H. Liu, and Y. Zhang are with the Department of Electronic
and Information Engineering and Guangdong Provincial Key Laboratory of
Digital Signal and Image Processing, Shantou University, Shantou 515063,
China (e-mail: {rbzhang; 22hjliu; yuezhang}@stu.edu.cn).
S.-H. Leung is with the Department of Electrical Engineering and State
Key Laboratory of Terahertz and Millimeter Waves, City University of Hong
Kong, Hong Kong (e-mail: eeeugshl@friends.cityu.edu.hk).
W. Tang is with UNISOC Technologies Co., Ltd, Shenzhen 518063, China
(e-mail: tangwj_scut@163.com).
H. Wang is with the School of Communication and Information Engineer-
ing, Nanjing University of Posts and Telecommunications, Nanjing 210003,
China (e-mail: wanghong@njupt.edu.cn).
Z. Luo is with the School of Communication and Information Engineering,
Chongqing University of Posts and Telecommunications, Chongqing 400065,
China (e-mail: luozhenluozhen@gmail.com).
S. Roy is with the Department of Electrical and Computer Engineering,
University of Washington, Seattle, Washington, USA (e-mail: sroy@uw.edu).
I. INTRODUCTION
The ever-increasing demand of high system capacity, re-
liable quality of service (QoS), and low processing latency
has been calling for new techniques in wireless commu-
nications, especially in the fifth-generation and beyond, to
support massive connectivity with stringent QoS requirements
[1]. Among existing promising candidates for performance
enhancement, multiple-input multiple-output (MIMO) with
precoding techniques has been widely used to provide diversity
and/or multiplexing gains via spatial degrees of freedom [2]. In
addition, the achievable rate region can be improved by power-
domain non-orthogonal multiple access (NOMA), which al-
lows multiple users to transmit data simultaneously in the same
frequency band at the same time. With the use of superposition
coding (SC) and successive interference cancellation (SIC),
NOMA has superior spectral efficiency over the conventional
orthogonal multiple access [3].
In the literature, single-antenna NOMA systems have been
intensively studied for both downlink and uplink communica-
tions, including system designs of user clustering with power
control [4], [5] and performance analysis of ergodic data rate,
outage probability, and diversity order [6], [7]. To enhance
the system performance, combining MIMO techniques and
NOMA is an attractive approach and has become a hot
research topic [8], [9].
Among existing transceiver designs for MIMO-NOMA sys-
tems, SIC and zero-forcing (ZF) detection are commonly used
to respectively mitigate the intra- and inter-cluster interfer-
ences [10], [11]. For doing so, the channels of the clustered
users should be highly correlated, and thus, the system per-
formance is sensitive to the correlatedness of channel real-
izations. In addition, some works are based on mathematical
optimization [12]–[14], most of which employ the successive
convex approximation algorithm to develop iterative design
procedures. Moreover, signal alignment techniques have been
applied in the precoder design for MIMO-NOMA systems
[15]–[18], where the effective channels of users in the same
cluster are projected into the same direction so that the inter-
cluster interferences can be fully cancelled by ZF process-
ing at the base station (BS). However, this approach has
shortcomings of degraded power efficiency partly because the
transceivers are not jointly designed, and partly because the
cluster formation lacks of flexibility to cope with various
system configurations.
In [19], [20], we proposed an uplink MIMO-NOMA scheme
with multi-group detection and developed precoder designs
to minimize the total transmit power of the system. In the
This article has been accepted for publication in IEEE Transactions on Vehicular Technology. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/TVT.2022.3225195
© 2022 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See https://www.ieee.org/publications/rights/index.html for more information.
Authorized licensed use limited to: Nanjing Univ of Post & Telecommunications. Downloaded on December 02,2022 at 08:07:19 UTC from IEEE Xplore. Restrictions apply.

2
scheme, users are divided into groups in accordance with
their locations for detection at the BS. The inter- and intra-
group interferences are cancelled by SIC and transceivers,
respectively. By utilizing the design freedom in the detection
at the BS, the scheme can work under more practical antenna
configurations and outperform upon existing cluster-based
MIMO-NOMA schemes in terms of transmit power efficiency
[20].
In the aforementioned works, however, perfect SIC is as-
sumed. Although such an assumption can provide the potential
performance gain of NOMA, practical issues such as SIC
residual and error propagation are not explicitly addressed. It is
of vital importance to ensure the reliability of NOMA designs
by considering the effect of SIC residuals explicitly [21]–
[23]. In practice, however, the imperfectness of SIC is deter-
mined by various factors including modulation/demodulation
schemes, signal-to-interference-plus-noise ratio (SINR) level,
channel state information (CSI) accuracy, hardware related
issues and so on, making it difficult to model the imperfect
SIC in a unified analytical fashion [24]. It is noted that there
are research works [24]–[28] simply introducing a so-called
fractional-SIC-error parameter 𝜖fto roughly account for all
sources of SIC errors. However, the 𝜖fis preset without
analytical justification for system designs and/or performance
analysis. As a result, this conventional error model suffers
from infeasibility and underestimation/overestimation of SIC
errors.
Unlike our previous works [19], [20] that use minimum
mean-square-error or ZF detections for mitigating the intra-
group interference, this paper introduces QR-decomposition-
based detection that is shown to provide improved transmit
power efficiency [29]. In addition, different from [19] that
assumes perfect SIC and [20] that adopts the conventional
fractional-SIC-error model to study the impact of imperfect
SIC, we particularly consider the cases where SIC residual
errors are generated from symbol demodulation. The main
contributions are summarized as follows.
1) In order to investigate the effects of SIC residuals on
the multiuser detection and multi-group NOMA, we consider
practical modulation and symbol demodulation to model the
SIC residuals and evaluate their statistics for facilitating the
precoder design.
2) With the statistical expression of the SIC residuals ex-
plicitly obtained from the detection errors in the demodulation,
we derive an approximate SIC-residual-power coefficient and
obtain an approximate SINR. The effect of the SIC residuals
is included in the design problem of minimizing the total
transmit power subject to users’ symbol error probability
(SEP) requirements.
3) We develop a suboptimal design algorithm with efficient
“logarithmic” golden-section search to obtain optimized user
precoders. Simulation results show the effectiveness of the
derived design criterion and the developed design algorithm,
which validate the detection error analysis.
4) As the mutual impact between the instantaneous SINR
and SIC error is explicitly addressed in the SINR analysis, the
proposed SIC error model enables the proposed design method
to provide precoder design solutions to satisfy the pre-specified
SEP requirements, thus, overcoming the shortcomings of the
conventional fractional-SIC-error model.
Notations: Upper and lower bold letters denote matrices and
column vectors, respectively. A−1stands for the inverse of
matrix A.A1
2denotes the principal square root of the positive
semi-definite matrix A=A1
2A𝐻
2.[A]𝑖 𝑗 denotes the (𝑖, 𝑗)th
element of A.I𝑁denotes the 𝑁×𝑁identity matrix. ∥a∥is
the Frobenius norm of adenoting its length. E[·] is taking the
expectation of the argument. ℜ{·} and ℑ{·} return the real
and imaginary parts of the argument, respectively.
II. SY ST EM MO DE L AN D DET ECTION ERROR ANALYS IS
A. Signal Model
We consider the uplink of a single-cell MIMO network,
where 𝑁urandomly scattered users are sharing the same time
slot and frequency band. According to their distances to the
BS, the users are divided into 𝐺groups with 𝐾=𝑁u
𝐺users
per group for detection at the BS 1. The user groups are
labeled in such an order that group 1is the nearest one to
the BS, while group 𝐺is the farthest one, as shown in Fig.
1. The BS and each user are equipped with 𝑁r(≥ 𝐾)and
𝑁tantennas, respectively 2. Each user transmits a single data
stream. Assume that all received user signals are synchronized
and accurate CSI is available at the BS 3. With a precoder
d(𝑔)
𝑚=√𝑝𝑔,𝑚 u(𝑔)
𝑚∈C𝑁t×1at user (𝑔, 𝑚 )for 𝑔=1,··· , 𝐺
and 𝑚=1,··· , 𝐾 , the received signal at the BS is expressed
as
y=𝐺
𝑔=1𝐾
𝑚=1H(𝑔)
𝑚u(𝑔)
𝑚√𝑝𝑔,𝑚 𝑠𝑔,𝑚 +z,(1)
where 𝑠𝑔,𝑚 , 𝑝𝑔 ,𝑚 , and u(𝑔)
𝑚are the unit-power signal symbol,
transmit power, and unit-norm beamforming vector of user
(𝑔, 𝑚), respectively; H(𝑔)
𝑚∈C𝑁r×𝑁tis the channel matrix
from user (𝑔, 𝑚)to the BS; and z∈C𝑁r×1with mean vector
1It is worth mentioning that the proposed multi-group scheme, precoder
design procedure, and system analysis can be generalized to cases where the
user groups have different numbers of users. However, for ease of exposition,
we assume that all groups have equal number of users, i.e., 𝑁u=𝐺𝐾 users
are divided into 𝐺equal-sized groups for detection at the BS.
2How to determine the optimal value of 𝐺, or equivalently 𝐾, is also
an important issue. Although it remains open at this stage, we suggest that
it should be preferable to schedule users whose large-scale pathlosses are
not too distinct from each other into the same group. In other words, the
users within one group have similar mean channel gains, implying that they
are with a similar distance to the BS [30]. Therefore, the overall effective
channel gains of different user groups should be as distinct as possible, which
will help exploit the potential performance enhancement of NOMA. Besides,
from the perspective of saving the computation cost, the optimal group size 𝐾
for minimizing the overall computational complexity of the proposed design
algorithm is in the order of 𝐾≈ ⌊𝑁r/𝑁t⌋, as will be discussed in the final
Remark in Sec. III-E. Lastly, the proposed scheme can work if 𝐾≤𝑁r,
implying an upper bound of feasible 𝐾.
3In fact, perfect CSI may not be practically available because of channel
estimation error, quantization error, and feedback delay. It is noted that
consideration of imperfect CSI has been widely studied in recent research
works, e.g., [31]–[34], most of which investigate the system design based
on the statistical/Gaussian-type CSI model or the bounded CSI error model.
Importantly, these works propose various methods to deal with outage-
related probabilistic constraints. However, in these works, perfect SIC is
commonly assumed to facilitate tractable design methods. By noting this,
a main difference between the these works and our work is that they consider
imperfect CSI but perfect SIC, while we consider imperfect SIC (with practical
modulation scheme in particular) but perfect CSI. How to characterize the
aggregated impact of imperfect SIC and imperfect CSI would be a challenging
extension in the future study.
This article has been accepted for publication in IEEE Transactions on Vehicular Technology. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/TVT.2022.3225195
© 2022 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See https://www.ieee.org/publications/rights/index.html for more information.
Authorized licensed use limited to: Nanjing Univ of Post & Telecommunications. Downloaded on December 02,2022 at 08:07:19 UTC from IEEE Xplore. Restrictions apply.

3
E[z]=0𝑁r×1and covariance matrix E[zz𝐻]=Czdenotes the
overall co-channel interference from interferers plus channel
noise [15]. The channel matrix H(𝑔)
𝑚for user (𝑔, 𝑚)located
at the distance 𝑙𝑔,𝑚 from the BS is H(𝑔)
𝑚=𝑙−𝛼
2
𝑔,𝑚 H(𝑔)
w,𝑚 ,
where 𝛼is the path-loss exponent, and the elements of H(𝑔)
w,𝑚
are independent and identically distributed (i.i.d.) complex
Gaussian random variables with zero mean and unit variance.
In group 𝑔, the 𝐾users are labeled in such an order that user
(𝑔, 1)is the nearest one to the BS, while user (𝑔, 𝐾)is the
farthest one.
Assume that all signal symbols {𝑠𝑔, 𝑚 ∀𝑔, 𝑚}are encoded
with an 𝑁I×𝑁Qrectangular quadrature amplitude modulation
(QAM) constellation with zero mean and unit power, i.e.,
E[𝑠𝑔,𝑚 ]=0,E[|𝑠𝑔 ,𝑚 |2]=1∀𝑔, 𝑚. The adopted signal
constellation is denoted by
S=𝛼𝛼=(𝑛I+𝑛Qi)𝑑;𝑛I∈ {±1,±3,· ·· ,±(𝑁I−1)}
𝑛Q∈ {±1,±3,· ·· ,±(𝑁Q−1)} ,
(2)
where i=√−1is the imaginary number, and the cardinality
of Sis |S| =𝑁I𝑁Q. The constellation Sis symmetric in the
sense that for any symbol 𝛼∈ S, we can always find 𝛼∗,−𝛼,
and −𝛼∗in S. The amplitude 𝑑for 𝛼=(𝑛I+𝑛Qi)𝑑is set as
𝑑=3/(𝑁2
I+𝑁2
Q−2)(3)
to ensure unit power [35]. The users’ signal symbols are
transmitted independently with equal probability.
Considering group 𝑔, the received signal y𝑔for detection is
expressed as
y𝑔=y−𝑔−1
𝑗=1F𝑗P
1
2
𝑗˘
s𝑗=F𝑔P
1
2
𝑔s𝑔+z𝑔,(4)
where ˘
s𝑗=[˘𝑠𝑗, 1,··· ,˘𝑠𝑗 ,𝐾 ]𝑇contains the decision symbols of
the 𝐾users of group 𝑗,F𝑔is defined as
F𝑔=H(𝑔)
1u(𝑔)
1,··· ,H(𝑔)
𝐾u(𝑔)
𝐾,(5)
P𝑔=diag(𝑝𝑔,1,··· , 𝑝𝑔, 𝐾 ),s𝑔=[𝑠𝑔,1,· · · , 𝑠𝑔, 𝐾 ]𝑇, and z𝑔
is the interference-plus-noise that consists of the interference
from farther groups, SIC-residual interference from nearer
groups, and noise, which is expressed as
z𝑔=𝐺
𝑙=𝑔+1F𝑙P
1
2
𝑙s𝑙+𝑔−1
𝑗=1F𝑗P
1
2
𝑗e𝑗+z(6)
with the SIC-residual-error vector of group 𝑗,e𝑗, expressed as
e𝑗=s𝑗−˘
s𝑗.(7)
According to Lemma 1(to be discussed at the end of this
section), E[e𝑗]=0𝐾×1holds for all 𝑗. Hence, by noting that
E[s𝑔]=0𝐾×1∀𝑔and E[z]=0𝑁r×1as previously assumed, the
mean of z𝑔is 0𝑁r×1. Assuming that the transmitted symbols of
all users and interferers are independent, the covariance matrix
of z𝑔is thus expressed as
C𝑔=𝐺
𝑙=𝑔+1F𝑙P𝑙F𝐻
𝑙+𝑔−1
𝑗=1F𝑗P
1
2
𝑗Cdet, 𝑗 P
1
2
𝑗F𝐻
𝑗+Cz,(8)
where Cdet,𝑔 is the covariance matrix of the post-detection
symbol error (referred to as detection-error covariance matrix
henceforth) of group 𝑔given by
Cdet,𝑔 =E[e𝑔e𝐻
𝑔]=E(s𝑔−˘
s𝑔)(s𝑔−˘
s𝑔)𝐻.(9)







Fig. 1. System model of the multi-group MIMO-NOMA scheme.
Note that the formulation of Cdet,𝑔 is related to the detection
rule. In Sec. II-B, we elaborate how the detection for each
group is performed. Then, we derive Cdet,𝑔 ∀𝑔in Sec. II-C.
B. Detection Rule
The detection for group 𝑔signals is performed as follows.
By employing a whitening filter C−1
2
𝑔on y𝑔, the whitened
signal is
y𝑔,w=C−1
2
𝑔F𝑔P
1
2
𝑔s𝑔+z𝑔,w,(10)
where z𝑔,w=C−1
2
𝑔z𝑔whose mean is E[z𝑔,w]=C−1
2
𝑔E[z𝑔]=
0𝑁r×1and covariance matrix is I𝑁r. We assume that z𝑔,wis a
circularly symmetric complex (CSC) Gaussian vector 4, i.e.,
z𝑔,w∼ CN0𝑁r×1,I𝑁r. Let
C−1
2
𝑔F𝑔=Q𝑔R𝑔(11)
be the QR decomposition of C−1
2
𝑔F𝑔, where Q𝑔∈C𝑁r×𝐾is
unitary, giving Q𝐻
𝑔Q𝑔=I𝐾, while R𝑔∈C𝐾×𝐾is a lower
triangular matrix. Note that the diagonal elements of R𝑔can
be made to be positive, while the elements strictly below its
main diagonal are in general complex.
Applying Q𝐻
𝑔to the whitened signal y𝑔,wyields
ˆ
s𝑔=Q𝐻
𝑔y𝑔,w=R𝑔P
1
2
𝑔s𝑔+𝜼𝑔,(12)
where 𝜼𝑔=Q𝐻
𝑔z𝑔,w∼ CN(0𝐾×1,I𝐾). Let the 𝑚th elements
of ˆ
s𝑔and 𝜼𝑔be denoted by ˆ𝑠𝑔,𝑚 and 𝜂𝑔, 𝑚, respectively.
Since R𝑔is lower triangular, the multiuser detection for
group 𝑔can be carried out from user (𝑔, 1)to user (𝑔, 𝐾 )
sequentially. The decision statistic for user (𝑔, 𝑚)is given by
¯𝑠𝑔,𝑚 =√𝑝𝑔 ,𝑚𝑟𝑔,𝑚𝑚 𝑠𝑔, 𝑚 +𝜂𝑔, 𝑚,eff ,(13)
where 𝑟𝑔,𝑚 𝑗 is the (𝑚, 𝑗 )th element of R𝑔, and 𝜂𝑔,𝑚,eff is the
effective noise containing the white noise 𝜂𝑔,𝑚 and intra-group
interference cancelation residuals, expressed as
𝜂𝑔,𝑚 ,eff =𝜂𝑔, 𝑚 +𝑚−1
𝑗=1√𝑝𝑔, 𝑗 𝑟𝑔 ,𝑚 𝑗 (𝑠𝑔 , 𝑗 −˘𝑠𝑔, 𝑗 ).(14)
As widely used in multiuser detection with decision feedback
[36], [37], for given ¯𝑠𝑔 ,𝑚 in (13), the symbol decision ˘𝑠𝑔,𝑚 ,
4Here, we consider practical finite-alphabet signalling. The transmitted
signal from each user is taken from a finite and discrete alphabet. Strictly
speaking, all the transmitted symbols are not Gaussian distributed. However,
for ease of analysis, we assume that the interference and SIC residual are
Gaussian. The assumption precision will be verified in the simulations.
This article has been accepted for publication in IEEE Transactions on Vehicular Technology. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/TVT.2022.3225195
© 2022 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See https://www.ieee.org/publications/rights/index.html for more information.
Authorized licensed use limited to: Nanjing Univ of Post & Telecommunications. Downloaded on December 02,2022 at 08:07:19 UTC from IEEE Xplore. Restrictions apply.

4
based on minimum Euclidean distance criterion by projecting
¯𝑠𝑔,𝑚 to the nearest constellation point, is computed as
˘𝑠𝑔,𝑚 =arg min
𝛼∈S ¯𝑠𝑔,𝑚 −√𝑝𝑔 ,𝑚𝑟𝑔 ,𝑚𝑚 𝛼.(15)
In the following lemma, it is shown that the mean of an
SIC residual is zero.
Lemma 1. For the signal model and detection rule adopted
in the above, E[s𝑔−˘
s𝑔]=0𝐾×1holds for all 𝑔=1,··· , 𝐺.
Proof: Please refer to Appendix A.
C. Detection-Error Covariance Matrix
In this subsection, the derivation of the detection-error
covariance matrices Cdet,𝑔 ∀𝑔is presented. Once Cdet, 𝑔 ∀𝑔
are derived, the covariance matrices C𝑔∀𝑔in (8) can be
obtained accordingly.
By definition, the (𝑚, 𝑛)th element of Cdet,𝑔 in (9), denoted
by 𝑐𝑔,𝑚 𝑛, is calculated as
𝑐𝑔,𝑚 𝑛 =E[(𝑠𝑔, 𝑚 −˘𝑠𝑔,𝑚)(𝑠𝑔,𝑛 −˘𝑠𝑔 ,𝑛 )∗].(16)
Noting that 𝑐𝑔, 𝑚𝑛 =𝑐∗
𝑔,𝑛 𝑚, we simply focus on the case where
𝑚≥𝑛.
By using (14), the decision statistic ¯𝑠𝑔,𝑚 of user (𝑔, 𝑚)in
(13) can be written as
¯𝑠𝑔,𝑚 =√𝑝𝑔 ,𝑚𝑟𝑔,𝑚𝑚 𝑠𝑔, 𝑚 +𝜂𝑔, 𝑚 +𝑑𝑚−1
𝑗=1√𝑝𝑔, 𝑗 𝑟𝑔 ,𝑚 𝑗 Δ𝑔, 𝑗 ,
(17)
where Δ𝑔,𝑚 =𝑠𝑔 ,𝑚 −˘𝑠𝑔,𝑚
𝑑∈ SΔ=𝛽𝛽=𝛼−˘𝛼
𝑑∀𝛼, ˘𝛼∈ S
and 𝜂𝑔,𝑚 ∼ CN(0,1)∀𝑔, 𝑚. For 𝑚 > 𝑛 (i.e., considering the
elements strictly below the main diagonal), the 𝑐𝑔,𝑚𝑛 in (16)
can be calculated as
𝑐𝑔,𝑚 𝑛 =𝑑2E[Δ𝑔,𝑚 Δ∗
𝑔,𝑛 ]=𝑑2𝛽𝑚, 𝛽𝑛∈SΔ
𝛽𝑚𝛽∗
𝑛𝑃𝑔,𝑚 ,𝑛, 𝛽𝑚,𝛽𝑛,
(18)
where 𝑃𝑔,𝑚 ,𝑛, 𝛽′, 𝛽′′ =Pr{Δ𝑔 ,𝑚 =𝛽′,Δ𝑔, 𝑛 =𝛽′′}is expressed in
(19) at the top of next page. For 𝑚=𝑛(i.e., for the elements
on the main diagonal), 𝑐𝑔, 𝑚𝑛 is calculated as
𝑐𝑔,𝑚 𝑚 =𝑑2E[Δ𝑔,𝑚 Δ∗
𝑔,𝑚 ]=𝑑2𝛽𝑚∈SΔ|𝛽𝑚|2𝑃𝑔,𝑚,𝛽𝑚,(20)
where 𝑃𝑔,𝑚 ,𝛽′=Pr{Δ𝑔,𝑚 =𝛽′}is expressed in (21) at the top
of next page.
The summands of (19) and (21) can be obtained in the
same way by invoking the chain rule of conditional probability,
the law of total probability, and the Gaussian Q-function
𝑄(𝑢)=1
√2𝜋∫+∞
𝑢e−1
2𝑥2d𝑥in order to obtain the pairwise
detection probabilities [35]. In specific, taking 𝑃𝑔, 𝑚,𝛽′in (21)
for example, each of its summands can be manipulated as
(22)-(25) at the top of next page, where in (22) we assume
Pr{Δ𝑔, 1=𝛽1Δ𝑔, 𝑗 =𝛽𝑗for 𝑗 < 1}=Pr{Δ𝑔,1=𝛽1}for
notation consistency, in (24) we use the fact that 𝑠𝑔, 𝑞 is
independent of Δ𝑔, 𝑗 =𝛽𝑗for 𝑗 < 𝑞, and (25) is based on
Pr{𝑠𝑔, 𝑞 =𝛼}=1
𝑁I𝑁Q∀𝛼∈ S. Each summand Pr{˘𝑠𝑔, 𝑞 =
˘𝛼𝑠𝑔,𝑞 =𝛼;𝑟0,Δ0, 𝜎2}in (25) can be easily obtained by
considering the generic decision statistic ¯𝑠𝑔,𝑞 =𝑟0𝑠𝑔 ,𝑞 +𝜂+Δ0
with 𝜂∼ CN(0, 𝜎2)in (A.1) and noting that each rectangular
QAM signal can be viewed as the superposition of two
independent pulse amplitude modulation (PAM) signals on
the in-phase and quadrature carriers since no colored noise
is involved [35]. In doing so, by computing each probability
term involved in (19) and (21), 𝑐𝑔,𝑚𝑛 can be obtained.
It is worth mentioning that, when performing the calculation
from user (𝑔, 1)to user (𝑔, 𝐾 )sequentially, it is possible to
utilize intermediate results to avoid redundant calculation. By
using the property (A.2), we can save computation further. In
Appendix B, we give the details of an efficient procedure so
developed for calculating the Cdet,𝑔 elements by adopting the
2×2QAM for modulation as an example.
Remark: In the system model, rectangular symmetric QAM
is assumed for ease of illustration. The method can be gen-
eralized to different modulation schemes with arbitrary bias
and/or rotations. However, when applying to other modulation
schemes, we need to re-examine the mean of the SIC residual
error vector if the symmetric constellation condition does not
hold. Moreover, for the detection-error covariance analysis,
each summand (i.e., pair-wise probability) in (25) should be re-
calculated. Simply saying, when applying to other modulation
schemes, the analysis is similar but the properties may be
varied in accordance with the constellation structures.
III. PRECODER DESIGN FOR THE MULTI -G ROUP
MIMO-NOMA SCHEME
A. Problem Formulation
We consider the total transmit power minimization subject
to users’ QoS requirements described as
Porig0 : min
{𝑝𝑔,𝑚 ,u(𝑔)
𝑚∀𝑔,𝑚 }𝐺
𝑔=1𝐾
𝑚=1𝑝𝑔,𝑚 (26a)
s.t.SEP𝑔,𝑚 ≤SEPth,𝑔, 𝑚,
u(𝑔)
𝑚
=1∀𝑔, 𝑚, (26b)
where SEPth,𝑔,𝑚 and SEP𝑔, 𝑚 are the pre-specified symbol error
probability (SEP) requirement and the achieved SEP of user
(𝑔, 𝑚), respectively. For simplicity, we assume SEPth,𝑔, 𝑚 =
SEPth ∀𝑔, 𝑚.
Remark: For uplink transmissions, individual maximum
transmit power constraints are important in practice since
mobile users have limited transmit power budget. In the
formulated problem Porig0, we choose the total transmit power
as the cost function without considering individual transmit
power constraints. This approach is still meaningful because
we focus on finding a system-wise spectral-efficient way of
satisfying pre-specified SEP thresholds. Precoder designs for
considering additional power constraints and possible user
selection are, however, beyond the scope of this work. When
individual maximum transmit power constraints are considered
in future studies, advanced user scheduling mechanism aiming
to avoid infeasibility by serving fewer users could be needed
for solving such a new problem.
For 𝑁I×𝑁Qrectangular QAM schemes, the theoretical SEP
is a function of SINR expressed as [38]
SEP =ℎSINR=𝜔1·𝑄√SINR−𝜔2·𝑄2√SINR,(27)
where 𝜔1=2(𝑁I−1)
𝑁I+2(𝑁Q−1)
𝑁Qand 𝜔2=4(𝑁I−1)(𝑁Q−1)
𝑁I𝑁Q. Recall
that the decision statistic of user (𝑔, 𝑚)is given by
¯𝑠𝑔,𝑚 =√𝑝𝑔 ,𝑚𝑟𝑔,𝑚𝑚 𝑠𝑔, 𝑚 +𝜂𝑔, 𝑚 +𝑑𝑚−1
𝑗=1√𝑝𝑔, 𝑗 𝑟𝑔 ,𝑚 𝑗 Δ𝑔 , 𝑗 .
(28)
This article has been accepted for publication in IEEE Transactions on Vehicular Technology. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/TVT.2022.3225195
© 2022 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See https://www.ieee.org/publications/rights/index.html for more information.
Authorized licensed use limited to: Nanjing Univ of Post & Telecommunications. Downloaded on December 02,2022 at 08:07:19 UTC from IEEE Xplore. Restrictions apply.

5
𝑃𝑔,𝑚 ,𝑛, 𝛽′, 𝛽′′ =PrΔ𝑔,𝑚 =𝛽′,Δ𝑔 ,𝑛 =𝛽′′=𝛽𝑗∈SΔfor 𝑗<𝑚, 𝑗 ≠𝑛PrΔ𝑔, 𝑚 =𝛽′,Δ𝑔, 𝑛 =𝛽′′,Δ𝑔, 𝑗 =𝛽𝑗for 𝑗 < 𝑚, 𝑗 ≠𝑛(19)
𝑃𝑔,𝑚 ,𝛽′=PrΔ𝑔, 𝑚 =𝛽′=𝛽𝑗∈SΔfor 𝑗<𝑚 PrΔ𝑔,𝑚 =𝛽′,Δ𝑔 , 𝑗 =𝛽𝑗for 𝑗 < 𝑚(21)
PrΔ𝑔 ,𝑚 =𝛽𝑚,Δ𝑔, 𝑗 =𝛽𝑗for 𝑗 < 𝑚
=Ö𝑚
𝑞=1PrΔ𝑔,𝑞 =𝛽𝑞Δ𝑔, 𝑗 =𝛽𝑗for 𝑗 < 𝑞 (22)
=Ö𝑚
𝑞=1𝛼, ˘𝛼∈S;𝛼−˘𝛼=𝛽𝑞𝑑Pr˘𝑠𝑔 ,𝑞 =˘𝛼𝑠𝑔, 𝑞 =𝛼, Δ𝑔, 𝑗 =𝛽𝑗for 𝑗 < 𝑞 Pr𝑠𝑔,𝑞 =𝛼Δ𝑔 , 𝑗 =𝛽𝑗for 𝑗 < 𝑞  (23)
=Ö𝑚
𝑞=1𝛼, ˘𝛼∈S;𝛼−˘𝛼=𝛽𝑞𝑑Pr˘𝑠𝑔 ,𝑞 =˘𝛼𝑠𝑔, 𝑞 =𝛼, Δ𝑔, 𝑗 =𝛽𝑗for 𝑗 < 𝑞 Pr𝑠𝑔,𝑞 =𝛼 (24)
=1
𝑁I𝑁QÖ𝑚
𝑞=1𝛼, ˘𝛼∈S;𝛼−˘𝛼=𝛽𝑞𝑑Pr˘𝑠𝑔 ,𝑞 =˘𝛼𝑠𝑔, 𝑞 =𝛼;𝑟0=√𝑝𝑔,𝑞 𝑟𝑔, 𝑞𝑞 ,Δ0=𝑑𝑞−1
𝑗=1√𝑝𝑔, 𝑗 𝑟𝑔 ,𝑞 𝑗 𝛽𝑗, 𝜎2=1(25)
Assume that 𝜂𝑔,𝑚 is independent of {Δ𝑔 , 𝑗 ∀𝑗 < 𝑚 }and 𝑠𝑔, 𝑚.
The SINR of user (𝑔, 𝑚)is calculated as
SINR𝑔,𝑚 =𝑝𝑔,𝑚 𝑟2
𝑔,𝑚 𝑚
1+𝑑2E|Í𝑚−1
𝑗=1√𝑝𝑔, 𝑗 𝑟𝑔 ,𝑚 𝑗 Δ𝑔, 𝑗 |2.(29)
The original problem Porig0in (26) can be equivalently rewrit-
ten as
Porig1 : min
{𝑝𝑔,𝑚 ,u(𝑔)
𝑚∀𝑔,𝑚 }𝐺
𝑔=1𝐾
𝑚=1𝑝𝑔,𝑚 (30a)
s.t. ℎSINR𝑔,𝑚≤SEPth ,
u(𝑔)
𝑚
=1∀𝑔, 𝑚. (30b)
Unfortunately, the problem Porig1is highly complicated
to solve. First, the SEP function of SINR𝑔,𝑚 ,ℎ(SINR𝑔, 𝑚),
in the QoS constraint in (30b) is highly non-convex, as
defined in (27). Moreover, the SINR of each user, SINR𝑔,𝑚 ,
is an extremely complicated function of the design variables
𝑝𝑔,𝑚 ,u(𝑔)
𝑚∀𝑔, 𝑚. Specifically, as seen in (29) when calculat-
ing SINR𝑔,𝑚, the denominator is taking the expectation over
𝑟𝑔,𝑚 𝑗 ’s and Δ𝑔, 𝑗 ’s. Here, 𝑟𝑔,𝑚 𝑗 is the (𝑚, 𝑗 )th element of R𝑔,
while Δ𝑔, 𝑗 =𝑠𝑔,𝑗 −˘𝑠𝑔, 𝑗
𝑑depends on the symbol demodulation.
Neither 𝑟𝑔,𝑚 𝑗 nor Δ𝑔, 𝑗 has a closed-form expression, not to
mention the denominator of SINR𝑔,𝑚 in (29).
Simply saying, the SIC residuals result in the couplings
between the users’ SINR expressions. In addition, the non-
convexity of the function ℎ(𝑥)and the discrete nature of
the considered finite-alphabet signal constellations make the
problem more involved. More importantly, the elements of
the detection-error covariance matrices Cdet,𝑔 ∀𝑔have no
closed-form expressions. Thus, the original problem has very
complicated structure.
In what follows, we first derive an approximation for
the user SINRs (yielding an approximate problem) in next
subsection and then develop a suboptimal design scheme to
solve the approximate problem in subsequent subsections.
Remark: The analysis of detection error in Sec. II-C is
essential for rigorously deriving the theoretical expressions
of user SINRs, although they are not used in the following
subsections (since we consider an approximate problem in-
stead due to the extreme difficulty of dealing with the original
problem directly). In addition, in Sec. IV-F, we conduct experi-
ments to evaluate the effectiveness of the adopted approximate
SINRs as the design criteria, in which the results of Sec.
II-C are employed. In specific, for each random realization of
H(𝑔)
𝑚∀𝑔, 𝑚, we perform the developed algorithm and obtain
the suboptimal user transmit powers and beamforming vectors
𝑝𝑔,𝑚 ,u(𝑔)
𝑚∀𝑔, 𝑚. With the obtained 𝑝𝑔,𝑚 ,u(𝑔)
𝑚∀𝑔, 𝑚, we
calculate the analyzed SINRs of all users by using the results
of Sec. II-C, which are then mapped to the corresponding
analyzed SEPs and compared with the designed SEP and
simulated SEP. In a word, the results of Sec. II-C are not used
in the suboptimal design procedure, but they are essential for
the theoretical analysis and performance evaluation.
B. Approximate Design Problem
To facilitate a tractable design, we assume that in the
denominator of (29) the weighted sum of the cross-correlations
between {Δ𝑔, 𝑗 ∀𝑗 < 𝑚 }is sufficiently small in comparison to
the auto-correlation terms. As such, the SINR𝑔,𝑚 in (29) can
be approximated as
SINR𝑔,𝑚 ≈𝑝𝑔,𝑚 𝑟2
𝑔,𝑚 𝑚
1+𝑑2Í𝑚−1
𝑗=1𝑝𝑔, 𝑗 |𝑟𝑔, 𝑚 𝑗 |2E[|Δ𝑔, 𝑗 |2].(31)
The E[|Δ𝑔, 𝑗 |2]in the denominator of (31) is calculated as
E[|Δ𝑔, 𝑗 |2]=𝛽∈SΔ|𝛽|2Pr{Δ𝑔, 𝑗 =𝛽}
=
𝛽∈SΔ|𝛽|2
𝛼, ˘𝛼∈S;𝛼−˘𝛼=𝛽 𝑑
Pr{˘𝑠𝑔, 𝑗 =˘𝛼, 𝑠𝑔, 𝑗 =𝛼}
=1
𝑁I𝑁Q
𝛽∈SΔ
𝛼, ˘𝛼∈S;𝛼−˘𝛼=𝛽 𝑑 |𝛽|2Pr{˘𝑠𝑔, 𝑗 =˘𝛼|𝑠𝑔, 𝑗 =𝛼}
≈1
𝑁I𝑁Q
𝛽∈SΔ
𝛼, ˘𝛼∈S;𝛼−˘𝛼=𝛽 𝑑 |𝛽|2𝑃˘𝛼 , 𝛼,SINR𝑔 , 𝑗 (32)
Δ
=˜𝜖𝑔, 𝑗 ,(33)
where in (32)Pr{˘𝑠𝑔, 𝑗 =˘𝛼|𝑠𝑔, 𝑗 =𝛼}is computed as
𝑃˘𝛼 , 𝛼,SINR𝑔 , 𝑗 , the pairwise detection probability of the event
that when 𝛼is transmitted while ˘𝛼is detected. In calculating
This article has been accepted for publication in IEEE Transactions on Vehicular Technology. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/TVT.2022.3225195
© 2022 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See https://www.ieee.org/publications/rights/index.html for more information.
Authorized licensed use limited to: Nanjing Univ of Post & Telecommunications. Downloaded on December 02,2022 at 08:07:19 UTC from IEEE Xplore. Restrictions apply.

6
𝑃˘𝛼 , 𝛼,SINR𝑔 , 𝑗 , the detection is based on the minimum Euclidean
distance criterion with the decision statistic ¯𝑠=SINRg,j𝑠+𝜂
where 𝑠∈ S and 𝜂∼ CN (0,1). Again, 𝑃˘𝛼, 𝛼 ,SINRg,jcan be
obtained by noting that each rectangular QAM signal can be
viewed as the superposition of two independent PAM signals
modulating the in-phase and quadrature carriers as no colored
noise is involved [35]. Besides, the ˜𝜖𝑔, 𝑗 defined in (33) is
called the SIC-residual-power coefficient of user 𝑗of group
𝑔. Note that ˜𝜖𝑔, 𝑗 is a function of the instantaneous SINR of
user (𝑔, 𝑗 ), i.e., SINR𝑔, 𝑗 .
Substituting (33) into (31) and then denoting the right-
hand-side of (31) by 
SINR𝑔,𝑚,SINR𝑔, 𝑚 is approximated by

SINR𝑔,𝑚 as
SINR𝑔,𝑚 ≈𝑝𝑔,𝑚 𝑟2
𝑔,𝑚 𝑚
1+𝑑2Í𝑚−1
𝑗=1𝑝𝑔, 𝑗 |𝑟𝑔, 𝑚 𝑗 |2˜𝜖𝑔, 𝑗
Δ
=
SINR𝑔,𝑚.(34)
Remark: In the above approximation, the loss of accu-
racy is 𝑑2Í𝑚−1
𝑗=1Í𝑚−1
𝑖=1,𝑖≠𝑗√𝑝𝑔 , 𝑗 √𝑝𝑔,𝑖𝑟𝑔 ,𝑚 𝑗 𝑟∗
𝑔,𝑚𝑖 E[Δ𝑔 , 𝑗 Δ∗
𝑔,𝑖 ], or
equivalently, Í𝑚−1
𝑗=1Í𝑚−1
𝑖=1,𝑖≠𝑗√𝑝𝑔 , 𝑗 √𝑝𝑔, 𝑖𝑟𝑔 ,𝑚 𝑗 𝑟∗
𝑔,𝑚𝑖 [Cdet ,𝑔 ]𝑗𝑖 in
the denominator of SINR𝑔,𝑚. Importantly, according to the
simulation results in Sec. IV-F, where we map the analytical
and approximate SINRs to the analyzed and designed SEPs,
respectively, we see that the loss of accuracy is minor. This
implies that the above assumption is quite reasonable.
Moreover, the adopted approximation can save a lot of
computations for obtaining the off-diagonal elements of Cdet,𝑔 ,
i.e., [Cdet,𝑔 ]𝑗𝑖 ∀𝑗≠𝑖. This is because, as derived in Sec.
II-C, each summand of [Cdet, 𝑔]𝑗𝑖 is expressed in terms of
summing up all possible joint probability terms, leading to
high computation load. In contrast, obtaining an approximation
for the diagonal elements of Cdet,𝑔 , i.e., [Cdet,𝑔 ]𝑗 𝑗 ∀𝑗, or
equivalently, E[|Δ𝑔, 𝑗 |2] ∀𝑗, can be quite computationally
efficient.
Remark: We can simplify the calculation of ˜𝜖𝑔, 𝑗 in (33)
with slight accuracy loss by noting that the detection error is
dominated by those neighboring constellation points adjacent
to the transmitted one if the system operates with low SEP.
Specifically, the calculation can be simplified as
˜𝜖𝑔, 𝑗 ≥1
𝑁I𝑁Q
𝛽∈SΔ;|𝛽|=2|𝛽|2
𝛼, ˘𝛼∈S;𝛼−˘𝛼=𝛽 𝑑
𝑃˘𝛼 , 𝛼,SINR𝑔 , 𝑗
=4
𝑁I𝑁Q
𝛽∈SΔ;|𝛽|=2
𝛼, ˘𝛼∈S;𝛼−˘𝛼=𝛽 𝑑
𝑃˘𝛼 , 𝛼,SINR𝑔 , 𝑗 ,(35)
yielding a lower bound for ˜𝜖𝑔, 𝑗 .
Next, we express the approximate 
SINR𝑔,𝑚 in (34) as a
function of u(𝑔)
𝑚and 𝑝𝑔,𝑚 explicitly for developing precoder
design procedures in subsequent subsections.
Consider the precoder design of group 𝑔for given C𝑔.
Recall that the post-filtering signal of group 𝑔is given
by ˆ
s𝑔=R𝑔P
1
2
𝑔s𝑔+𝜼𝑔in (12), where R𝑔with positive
diagonals is the lower triangular R-matrix of the QR de-
composition of C−1
2
𝑔F𝑔=C−1
2
𝑔H(𝑔)
1u(𝑔)
1,··· ,H(𝑔)
𝐾u(𝑔)
𝐾. Let
˜
H(𝑔)
𝑚=C−1
2
𝑔H(𝑔)
𝑚for notation convenience. Applying the
Gram-Schmidt process with normalization [39], the orthonor-
mal basis Q𝑔=[q(𝑔)
1,··· ,q(𝑔)
𝐾]for the subspace spanned
by the 𝐾columns of C−1
2
𝑔F𝑔=˜
H(𝑔)
1u(𝑔)
1,··· ,˜
H(𝑔)
𝐾u(𝑔)
𝐾is
constructed as
q(𝑔)
𝑚=v(𝑔)
𝑚∥v(𝑔)
𝑚∥(36)
for 𝑚=𝐾−1,··· ,1, where
v(𝑔)
𝑚=I𝑁r−𝐾
𝑙=𝑚+1q(𝑔)
𝑙q(𝑔)𝐻
𝑙˜
H(𝑔)
𝑚u(𝑔)
𝑚,(37)
while q(𝑔)
𝐾=˜
H(𝑔)
𝐾u(𝑔)
𝐾/∥ ˜
H(𝑔)
𝐾u(𝑔)
𝐾∥. The (𝑚, 𝑛)th element of
R𝑔, i.e., 𝑟𝑔, 𝑚𝑛 =q(𝑔)𝐻
𝑚˜
H(𝑔)
𝑛u(𝑔)
𝑛, is thus calculated as
𝑟𝑔,𝑚 𝑛 =u(𝑔)𝐻
𝑚˜
H(𝑔)𝐻
𝑚I𝑁r−Í𝐾
𝑙=𝑚+1q(𝑔)
𝑙q(𝑔)𝐻
𝑙˜
H(𝑔)
𝑛u(𝑔)
𝑛

I𝑁r−Í𝐾
𝑙=𝑚+1q(𝑔)
𝑙q(𝑔)𝐻
𝑙˜
H(𝑔)
𝑚u(𝑔)
𝑚
(38)
for 𝑚≥𝑛, while 𝑟𝑔, 𝑚𝑛 =0for 𝑚 < 𝑛. Using (38), the
approximate 
SINR𝑔,𝑚 in (34) can be expressed as

SINR𝑔,𝑚 =𝑝𝑔,𝑚 u(𝑔)𝐻
𝑚A(𝑔)
𝑚𝑚u(𝑔)
𝑚2
u(𝑔)𝐻
𝑚B(𝑔)
𝑚u(𝑔)
𝑚
,(39)
where
A(𝑔)
𝑚 𝑗 =˜
H(𝑔)𝐻
𝑚I𝑁r−𝐾
𝑙=𝑚+1q(𝑔)
𝑙q(𝑔)𝐻
𝑙˜
H(𝑔)
𝑗,(40)
B(𝑔)
𝑚=A(𝑔)
𝑚𝑚 +𝑑2𝑚−1
𝑗=1𝑝𝑔, 𝑗 ˜𝜖𝑔, 𝑗 A(𝑔)
𝑚 𝑗 u(𝑔)
𝑗u(𝑔)𝐻
𝑗A(𝑔)𝐻
𝑚 𝑗 .(41)
Now the design becomes an approximate problem Pappr0as
Pappr0 : min
{𝑝𝑔,𝑚 ,u(𝑔)
𝑚∀𝑔,𝑚 }
˜𝑝total,appr =𝐺
𝑔=1𝐾
𝑚=1𝑝𝑔,𝑚 (42a)
s.t. ℎ
SINR𝑔,𝑚≤SEPth ,
u(𝑔)
𝑚
=1∀𝑔, 𝑚. (42b)
In this problem, we approximate C𝑔as C𝑔≈Í𝐺
𝑙=𝑔+1F𝑙P𝑙F𝐻
𝑙+
𝑑2Í𝑔−1
𝑗=1F𝑗P
1
2
𝑗diag(˜𝜖𝑔,1,··· ,˜𝜖𝑔,𝐾 )P
1
2
𝑗F𝐻
𝑗+Cz∀𝑔based on
(31)-(33). Here, each ˜𝜖𝑔, 𝑗 is calculated in (33) by replacing
SINR𝑔, 𝑗 with 
SINR𝑔, 𝑗 . In what follows, we focus on solving
the approximate problem Pappr0in (42) instead of the original
problem Porig1in (30).
C. An Approximate Problem With Equality Constraints
Unfortunately, the problem Pappr0is still highly non-convex.
The main difficulties are the nonlinear relations between the
precoding vectors of inter- and intra-groups. In this subsection,
we develop a suboptimal design for solving Pappr0.
Let us start with considering a generic approximate problem
where the inequality constraints of Pappr0are replaced by
equality ones. Let 𝛤th =ℎ−1(SEPth), where the function SEP =
ℎ(SINR)is defined in (27). Note that the function 𝑦=ℎ(𝑥)
is monotonic 5, indicating that higher SINR gives lower SEP.
5The monotonicity of the function SEP =ℎ(SINR)is justified as follows.
Recall that the considered 𝑁I×𝑁Qrectangular QAM can be viewed as the
superposition of an 𝑁I-ary PAM and an 𝑁Q-ary PAM. The SEPs of the in-
phase 𝑁IPAM and the quadrature 𝑁QPAM in AWGN are given by 𝑃I=
2(1−1
𝑁I)𝑄(√SINR)and 𝑃Q=2(1−1
𝑁Q)𝑄(√SINR), respectively [38].
Then the probability of correct symbol reception for the 𝑁I×𝑁Qrectangular
QAM is (1−𝑃I)(1−𝑃Q), and the SEP is SEP =1− (1−𝑃I)(1−𝑃Q). It
can be easily verified that with larger SINR, 𝑄(√SINR)decreases, 𝑃Iand
𝑃Qdecrease, and thus SEP decreases. Therefore, the function 𝑦=ℎ(𝑥)is
monotonically decreasing.
This article has been accepted for publication in IEEE Transactions on Vehicular Technology. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/TVT.2022.3225195
© 2022 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See https://www.ieee.org/publications/rights/index.html for more information.
Authorized licensed use limited to: Nanjing Univ of Post & Telecommunications. Downloaded on December 02,2022 at 08:07:19 UTC from IEEE Xplore. Restrictions apply.

7
Thus, for given SEPth, the 𝛤th can be easily obtained by using
existing root-finding solvers, e.g., the bisection method. With
the obtained 𝛤th, the corresponding approximate SIC-residual-
power coefficient ˜𝜖𝑔, 𝑗 =˜𝜖∀𝑔, 𝑗 in (33) can be calculated by
setting SINR𝑔, 𝑗 =𝛤th ∀𝑔, 𝑗 in (32).
Using equality constraints (i.e., SEPth is strictly met for
every user), the deign problem Pappr,eq0is described as
Pappr,eq0 : ˜𝑝total,appr,eq SEPth =min
{𝑝𝑔,𝑚 ,u(𝑔)
𝑚}
𝐺

𝑔=1
𝐾

𝑚=1
𝑝𝑔,𝑚(43a)
s.t.
SINR𝑔,𝑚 =𝛤th,
u(𝑔)
𝑚
=1∀𝑔, 𝑚,(43b)
C𝑔=
𝐺

𝑙=𝑔+1
F𝑙P𝑙F𝐻
𝑙+𝑑2·˜𝜖SEPth·
𝑔−1

𝑗=1
F𝑗P𝑗F𝐻
𝑗+Cz∀𝑔,(43c)
where 
SINR𝑔,𝑚 is given by (39). We denote the yet-to-be-
minimized objective of Pappr,eq0by ˜𝑝total,appr,eq SEPth in (43a)
and treat ˜𝑝total,appr,eq as a function of SEPth . The ˜𝜖SEPthin
(43c) emphasizes that ˜𝜖can be determined for given SEPth.
For perfect SIC, the design can be performed from groups
𝐺to 1sequentially to achieve the optimal solution. However,
in the presence of SIC residuals, Pappr,eq0gets complicated
and is not easy to solve for an optimal solution. Nevertheless,
the group-by-group design for perfect SIC can be used to find
a feasible solution, if any, for imperfect SIC.
Specifically, we can start with the precoders obtained under
perfect SIC (or randomly generated precoders) and then iterate
group by group taking into account the SIC residuals in the
matrices C𝑔∀𝑔as well as in the individual SINRs. In this
way, there are outer- and inner-iterations. The process of
inner iterations for designing the precoders of the users in a
group is referred to as inner cycle, while the process of outer
iterations for the design from groups 𝐺to 1is outer cycle. If
the iterations converge, the convergent result is adopted as the
final solution. In case the iterative design for using different
initializations cannot converge, we conservatively report that
the problem Pappr,eq0is infeasible leading to ˜𝑝total,appr,eq =∞.
It is worth mentioning that the precoder design problem can
have converged solution if the SEPth is getting sufficiently
small for which the resulting SIC-residual-power coefficients
are small.
Specifically, for group 𝑔, the design is formulated as
Pappr,eq,𝑔 0 : min
{𝑝𝑔,𝑚 ,u(𝑔)
𝑚∀𝑚}𝐾
𝑚=1𝑝𝑔,𝑚 (44a)
s.t.
SINR𝑔,𝑚 =𝛤th,
u(𝑔)
𝑚
=1∀𝑚. (44b)
In the proposed suboptimal design procedure, the user pre-
coders d(𝑔)
𝑚=√𝑝𝑔,𝑚 u(𝑔)
𝑚for 𝑚=𝐾, ·· · ,1of group 𝑔are
updated iteratively. At each iteration, the precoder of a user is
optimized while the others are fixed. Particularly, the update
of d(𝑔)
𝑚=√𝑝𝑔,𝑚 u(𝑔)
𝑚at the 𝑛th inner-iteration, expressed as
d(𝑔)
𝑚,𝑛 =𝑓d(𝑔)
1,𝑛−1,·· · ,d(𝑔)
𝑚−1,𝑛−1,d(𝑔)
𝑚+1,𝑛 ,··· ,d(𝑔)
𝐾 ,𝑛 ,(45)
is performed according to the equality QoS constraint. Ac-
cording to (39), for power minimization, the update of u(𝑔)
𝑚at
the 𝑛th inner-iteration can be computed as
u(𝑔)
𝑚,𝑛 =arg max
∥x∥=1x𝐻A(𝑔)
𝑚𝑚x2x𝐻B(𝑔)
𝑚x.(46)
TABLE I
INN ER-O UT ER IT ERAT IVE P ROC EDU RE F OR SO LVIN G TH E
EQU ALI TY-CONSTRAINED PROBLEM Pappr,eq0I N (43).
1. Initialization.
1.1 Set 𝑡=0;
1.2 Set 𝑝𝑔 ,𝑚, 𝑡 =𝑝𝑔,𝑚,pSIC ,u(𝑔)
𝑚,𝑡 =u(𝑔)
𝑚,pSIC ∀𝑔, 𝑚 (the perfect
SIC solution as trivial initialization) or 𝑝𝑔 ,𝑚,𝑡 =𝑝𝑔 ,𝑚,random ,
u(𝑔)
𝑚,𝑡 =u(𝑔)
𝑚,random ∀𝑔, 𝑚 (random initialization);
2. Iteration. % Outer-iteration: Completion from groups 𝐺to 1
2.1 Set 𝑡=𝑡+1;
2.2 Precoder design group by group:
2.2.1 For 𝑔=𝐺: 1
2.2.1.1 If 𝑔=𝐺, then set C𝑔,𝑡 =C𝑔 ,𝑡 −1; otherwise, set
C𝑔,𝑡 =C𝑔+1, 𝑡 +F𝑔+1,𝑡 P𝑔+1,𝑡 F𝐻
𝑔+1,𝑡 −𝑑2˜𝜖F𝑔,𝑡 −1P𝑔 ,𝑡 −1F𝐻
𝑔,𝑡 −1;
2.2.1.2 Set 𝑛=0;% Inner-iteration for a group: Initialization
Set 𝑝𝑔,𝑚, 𝑡, 𝑛 =𝑝𝑔,𝑚, 𝑡 −1and u(𝑔)
𝑚,𝑡 ,𝑛 =u(𝑔)
𝑚,𝑡 −1∀𝑚;
2.2.1.3 Set 𝑛=𝑛+1;% Inner-iteration for a group: Iteration
2.2.1.4 For 𝑚=𝐾: 1
Compute A(𝑔)
𝑚𝑚 and B(𝑔)
𝑚by (40) and (41);
Update u(𝑔)
𝑚,𝑡 ,𝑛 by (46) and 𝑝𝑔,𝑚, 𝑡, 𝑛 by (47);
End
2.2.1.5 If diverged, then report divergence and end the design.
If converged, then go to Step 2.2.1.6;
otherwise, go back to Step 2.2.1.3;
% Inner-iteration for a group: Convergence checking
2.2.1.6 Compute F𝑔,𝑡 and P𝑔 ,𝑡 ;
End % End of Step 2.2.1 For 𝑔=𝐺: 1
2.2.2 Update C𝐺,𝑡 as C𝐺 ,𝑡 =𝑑2˜𝜖Í𝐺−1
𝑗=1F𝑗, 𝑡 P𝑗,𝑡 F𝐻
𝑗, 𝑡 +Cz;
2.3 Convergence checking for outer-iteration.
If diverged, then report divergence and end the design.
If convergence criterion is not met, then go back to Step 2.1;
otherwise, check feasibility of convergent result and end.
Such a single-user beamforming update problem can be solved
by our developed algorithms in [20], [29]. After solving u(𝑔)
𝑚,𝑛 ,
𝑝𝑔,𝑚 at the 𝑛th inner-iteration is updated as
𝑝𝑔,𝑚 ,𝑛 =𝛤thu(𝑔)𝐻
𝑚,𝑛 B(𝑔)
𝑚u(𝑔)
𝑚,𝑛 u(𝑔)𝐻
𝑚,𝑛 A(𝑔)
𝑚𝑚u(𝑔)
𝑚,𝑛 2.(47)
In Table I, we summarize the proposed iterative procedure
for solving the approximate problem Pappr,eq0with equality
constraints in (43).
D. A Suboptimal Algorithm for Solving the Approximate De-
sign Problem With Inequality Constraints
In this subsection, we solve the approximate problem Pappr0
with inequality constraints in (42). Note that we have con-
sidered the approximate problem Pappr,eq0with equality con-
straints in (43), which can be solved by the design procedure
summarized in Table I.
Here, we treat the yet-to-be-minimized objective ˜𝑝total,appr,eq
of Pappr,eq0as a single-variable function of SEPth , denoted by
˜𝑝total,appr ,eq (SEPth), as seen in (43a). Importantly, by inspect-
ing the minimized objectives for a large number of random
realizations of H(𝑔)
𝑚∀𝑔, 𝑚(some of which will be shown
in Sec. IV-C for using the 2×2QAM for modulation), it is
very likely that the single-variable function ˜𝑝total,appr,eq (𝑥)is
either monotonically decreasing or unimodal having a single
minimizer in its feasible region. In particular, for configu-
rations with small (𝐺 , 𝐾)and/or large (𝑁r, 𝑁t), it is more
likely that ˜𝑝total,appr,eq (𝑥)is monotonically decreasing; while
for configurations with large (𝐺, 𝐾)and/or small (𝑁r, 𝑁t), it
This article has been accepted for publication in IEEE Transactions on Vehicular Technology. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/TVT.2022.3225195
© 2022 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See https://www.ieee.org/publications/rights/index.html for more information.
Authorized licensed use limited to: Nanjing Univ of Post & Telecommunications. Downloaded on December 02,2022 at 08:07:19 UTC from IEEE Xplore. Restrictions apply.

8
is more likely that ˜𝑝total,appr ,eq (𝑥)is unimodal. This behavior
can be understood as follows 6. Note that the increasing rate
of ˜𝜖is much larger than the decreasing rate of 𝛤th as SEPth
increases, as will be seen in Fig. 2in Sec. IV-A. Hence, for
configurations with large (𝐺, 𝐾)and/or small (𝑁r, 𝑁t), it is
very likely that ˜𝑝total,appr,eq may not decrease with SEPth in
the high SEPth region provided that ˜𝑝total,appr,eq is in the state
of increasing with SEPth.
Based on this observation, a suboptimal greedy algorithm
can be developed to solve Pappr0by solving Pappr,eq0. First,
we find the minimizer SEP∗of the function ˜𝑝total,appr,eq (𝑥)by
using existing iterative bracketing methods, e.g., the conven-
tional golden-section search [40]. Second, if SEP∗is smaller
than SEPth, then we can adopt ˜𝑝total,appr,eq SEP∗as the (sub-
optimal) solution to Pappr0, which gives resultant SEPs lower
than the required ones but with the lowest power consumption;
otherwise, we adopt ˜𝑝total,appr,eq SEPth .
Specifically, for using iterative bracketing methods to find
the minimizer of ˜𝑝total,appr ,eq (𝑥), we start with an interval
[𝑥L, 𝑥U]. The lower limit 𝑥Lcan be set sufficiently smaller
than SEPth and close to zero, while the upper limit 𝑥Ucan
be set sufficiently larger than SEPth and close to one. Then,
two intermediate points 𝑥1and 𝑥2are somehow chosen, giving
𝑥L< 𝑥1< 𝑥2< 𝑥U. Four possible results can occur:
Case 1: If ˜𝑝total,appr ,eq (𝑥1)=∞and ˜𝑝total,appr,eq (𝑥2)=∞,
then set 𝑥U=𝑥1and use [𝑥L, 𝑥U]for the next round.
Case 2: If ˜𝑝total,appr ,eq (𝑥1)<∞and ˜𝑝total,appr,eq (𝑥2)=∞,
then set 𝑥U=𝑥2and use [𝑥L, 𝑥U]for the next round.
Case 3: If ˜𝑝total,appr ,eq (𝑥1)<˜𝑝total,appr,eq (𝑥2)<∞, then set
𝑥U=𝑥2and use [𝑥L, 𝑥U]for the next round.
Case 4: If ˜𝑝total,appr ,eq (𝑥2)<˜𝑝total,appr,eq (𝑥1)<∞, then set
𝑥L=𝑥1and use [𝑥L, 𝑥U]for the next round.
Importantly, we can use a “logarithmic” golden ratio instead
of the conventional “linear” golden ratio to make the iterative
search more efficient. In specific, in the conventional golden-
section search method, the two intermediate points 𝑥1and 𝑥2
are respectively chosen as 𝑥1=𝑥U−𝑣(𝑥U−𝑥L), 𝑥2=𝑥U+𝑥L−
𝑥1=𝑥L+𝑣(𝑥U−𝑥L), where 𝑣=√5−1
2. However, by inspecting
Fig. 5in Sec. IV-C, where the plots use a logarithmic scale
for the x-axis while linear for the y-axis, it is foreseeable that
most computation time will be spent on testing new bounds
located very close to one but far from the minimizer SEP∗
which can be much closer to zero than to one. To improve
the searching efficiency, we use a wiser choice of the two
intermediate points 𝑥1and 𝑥2based on
log(𝑥2)−log(𝑥L)=𝑣log(𝑥U)−log (𝑥L), 𝑥1𝑥2=𝑥U𝑥L.(48)
This rule is referred to as “logarithmic” golden-section search.
Solving (48), we have
𝑥1=𝑥U(𝑥U/𝑥L)−𝑣, 𝑥2=𝑥L(𝑥U/𝑥L)𝑣.(49)
Simulations have validated that the “logarithmic” golden-
section search rule outperforms upon the conventional “linear”
one with respect to computational efficiency.
6It should be pointed out that although this property is not justified
analytically, it helps develop an efficient algorithm to find suboptimal solution
with satisfactory SEP performance as validated by simulations.
E. Complexity Analysis
The computational complexity of the proposed design al-
gorithm in Sec. III-D for solving Pappr0in (42) is briefly
discussed. The calculation cost mainly includes two parts.
The first part is the “function evaluation” [40] of
˜𝑝total,appr ,eq (𝑥), i.e., solving the equality-constrained approxi-
mate problem Pappr,eq0in (43) by using the iterative algorithm
in Table I.
Basically, the iterative design takes a few outer-iterations to
converge, or we can recognize divergence and terminate the
evaluation after running in a few outer-iterations. Following
the complexity estimation in [20], the overall computational
complexity of a single outer cycle of the proposed inner-
outer iterative algorithm summarized in Table Ifor solving
the problem Pappr,eq0is in the order of 𝑄outer ≈ O𝐺𝑁3
r+
𝑁u𝑁t𝑁2
r+𝐾 𝑁u𝑁2
t𝑁r+𝑁inner 𝑁u¯
𝑄, where 𝑁inner denotes the
number of cycles for completing Step 2.2.1.4 of Table Ifor 𝐾
users of a group, and ¯
𝑄denotes the computational complex-
ity of solving the single-user beamforming update problem
[20]. Let 𝑁outer denote the number of outer cycles. Thus,
the overall complexity of solving Pappr,eq0is 𝑄inner−outer ≈
O𝑁outer (𝐺𝑁3
r+𝑁u𝑁t𝑁2
r+𝐾 𝑁u𝑁2
t𝑁r+𝑁inner 𝑁u¯
𝑄).
Importantly, we can come up with an efficient way of
recognizing divergence when implementing the inner-outer
iterative algorithm. Denote the objective of Pappr,eq0at the
𝑡th outer-iteration by ˜𝑝total,appr ,eq,𝑡 . Define Δ𝑡=˜𝑝total,appr,eq,𝑡 −
˜𝑝total,appr ,eq,𝑡 −1. During the implementation, after 𝑡gets larger
than a pre-specified number (say, after 𝑡≥20), we can
terminate the algorithm and report divergence if Δ𝑡>Δ𝑡−1>
··· >Δ𝑡−𝑁with a pre-specified 𝑁(say, 𝑁=3or even larger).
The second part is how many times the “function evalu-
ation” is called. This depends on how efficient the interval
containing the minimizer of ˜𝑝total,appr,eq (𝑥)is reduced in the
adopted iterative bracketing method. For using the “logarith-
mic” golden-section search, the interval with logarithmic scale
is reduced by a factor of √5−1
2≈0.618 per round.
To sum up, the overall computational complexity of
the developed suboptimal algorithm for solving Pappr0in
(42) is in the order of 𝑄ipSIC ≈ O𝑁eval𝑄inner−outer =
O𝑁eval𝑁outer (𝐺 𝑁 3
r+𝑁u𝑁t𝑁2
r+𝐾 𝑁u𝑁2
t𝑁r+𝑁inner 𝑁u¯
𝑄).
Remark: Given 𝑁r, 𝑁t, 𝑁u, we can express 𝑄ipSIC as
function of 𝐾given by 𝑄ipSIC ≈ O𝑁eval𝑁outer 𝑁u𝑁3
r𝐾−1+
𝑁2
t𝑁r𝐾+𝑁t𝑁2
r+𝑁inner ¯
𝑄. The optimal 𝐾for minimizing
𝑄ipSIC is 𝐾∗≈ [𝑁3
r/(𝑁2
t𝑁r)]1/2=𝑁r/𝑁t.
IV. SIMULATION RESULTS
We evaluate the proposed design method by numerical
simulations. In the simulation, 𝑁u=𝐺 𝐾 users were randomly
located within a ring with radii 1m (to avoid singularity when
the distance to the BS is smaller than 1m) and 500 m. The
path loss exponent was 3.5. The system bandwidth was 20
MHz. Assuming that the thermal noise power density was
−174 dBm/Hz, the noise power 𝜎2
nwas set as 𝜎2
n=−174
dBm/Hz ×20 MHz =−101 dBm. The covariance matrix Cz
was set as 𝜎2
nI𝑁r.
This article has been accepted for publication in IEEE Transactions on Vehicular Technology. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/TVT.2022.3225195
© 2022 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See https://www.ieee.org/publications/rights/index.html for more information.
Authorized licensed use limited to: Nanjing Univ of Post & Telecommunications. Downloaded on December 02,2022 at 08:07:19 UTC from IEEE Xplore. Restrictions apply.

9
10-10 10-8 10-6 10-4 10-2
0
5
10
15
20
25
30
35
40
45
10-10 10-8 10-6 10-4 10-2
0
0.05
0.1
0.15
0.2
0.25
Fig. 2. Demo of the SIC residual analysis for using 2×2,4×2, and 4×4
QAMs: Given SEPth, compute 𝛤th that gives SEPth =𝜔1·𝑄√𝛤th−𝜔2·
𝑄2√𝛤th, as seen in Fig. 2(a), and then calculate ˜𝜖and its lower bound
based on 𝛤th by (33) and (35), respectively, as seen in Fig. 2(b).
A. Approximate SIC-Residual-Power Coefficient
First, we show results of the minimum SINR threshold 𝛤th
and the derived approximate SIC-residual-power coefficient ˜𝜖
versus SEPth. We adopt three modulation schemes, namely,
2×2,4×2, and 4×4QAMs, for illustration. Using (3),
we have 𝑑=1
√2,1
√6,and 1
√10 , respectively. With SEPth =
𝜔1·𝑄√𝛤th−𝜔2·𝑄2√𝛤th in (27), 𝛤th can be obtained
by using the bisection method for given SEPth for each
modulation scheme. ˜𝜖and its lower bound are respectively
computed by (33) and (35) with SINR𝑔, 𝑚 =𝛤th. We plot 𝛤th
and ˜𝜖 𝑑2versus SEPth in Figs. 2(a) and (b), respectively. Since
the Q-function can be approximated as (upper bounded by) an
exponential function for large 𝛤th, the 𝛤th decreases linearly
with the logarithm of SEPth as seen in Fig. 2(a). For Fig. 2(b),
the detection error probability is roughly an exponential-like
increasing function of SEPth or a decreasing function of 𝛤th.
As expected, the lower bound for ˜𝜖is tight when the system
operates with low SEPth as seen in Fig. 2(b).
B. Convergence Behaviors of the Developed Algorithm for
Solving the Equality-Constrained Approximate Problem
In this subsection, we plot convergence curves of the total
transmit power ˜𝑝total,appr,eq,𝑡 =Í𝐺
𝑔=1Í𝐾
𝑚=1𝑝𝑔,𝑚 ,𝑡 versus outer-
iteration index 𝑡obtained by using the inner-outer iterative
precoder design algorithm, which is summarized in Table I, to
solve the equality-constrained approximate problem Pappr,eq0
in (43) for different SEPth targets with a random realization
of H(𝑔)
𝑚∀𝑔, 𝑚. We set 𝐾=3, 𝑁t=2, 𝑁r=8, and plot the
˜𝑝total,appr ,eq,𝑡 for 𝐺=4and 𝐺=5in Figs. 3and 4, respectively.
The 2×2QAM was adopted for modulation. The approximate
SIC-residual-power coefficient ˜𝜖was obtained by (33) for each
SEPth and shown in the legends.
In the experiments, we adopted the perfect SIC solution
as initialization for the inner-outer iterative algorithm. Under
perfect SIC condition, i.e., ˘𝑠𝑔, 𝑚 =𝑠𝑔, 𝑚 ∀𝑔, 𝑚, the matrix C𝑔in
(8) and the 𝜂𝑔,𝑚,eff in (14) reduce to C𝑔=Í𝐺
𝑙=𝑔+1F𝑙P𝑙F𝐻
𝑙+Cz
and 𝜂𝑔,𝑚 ,eff =𝜂𝑔, 𝑚, respectively. Hence, the SINR𝑔,𝑚 in (29)
2 4 6 8 10 12 14
10-2
10-1
Fig. 3. Convergence of the precoder design in Table Iusing 2×2QAM:
˜𝑝total,appr,eq,𝑡 =Í𝐺
𝑔=1Í𝐾
𝑚=1𝑝𝑔,𝑚, 𝑡 versus outer-iteration index 𝑡for a
random realization of H(𝑔)
𝑚∀𝑔, 𝑚for different SEPth (whose corresponding
˜𝜖is given in the legend) with 𝐺=4, 𝐾 =3, 𝑁t=2, 𝑁r=8.
2 4 6 8 10 12 14
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
2 4 6 8 10 12 14
10-1
100
101
102
Fig. 4. Convergence of the precoder design in Table Iusing 2×2QAM:
˜𝑝total,appr,eq,𝑡 versus 𝑡for a random H(𝑔)
𝑚∀𝑔, 𝑚for different SEPth with
𝐺=5, 𝐾 =3, 𝑁t=2, 𝑁r=8; the convergent cases are in Fig. 4(a) while the
divergent cases in Fig. 4(b).
reduces to SINR𝑔, 𝑚 =𝑝𝑔,𝑚 𝑟2
𝑔,𝑚 𝑚. Consequently, the users’
transmit powers are the lowest at the outer-iteration step 𝑡=0,
while they are increased as the proposed iterative algorithm
continues because the SIC residuals are taken into account,
requiring higher individual transmit power to compensate
the higher interference. As a result, the total-transmit-power
curve, ˜𝑝total,appr,eq ,𝑡 versus 𝑡, is monotonically increasing for
𝑡=0,1,2,··· . If all the users’ QoS requirements can be
satisfied simultaneously, then the ˜𝑝total,appr,eq, 𝑡 curve converges
eventually. Otherwise, the ˜𝑝total,appr,eq, 𝑡 curve diverges.
In Fig. 3for 𝐺=4, the precoder design converges for
all the considered SEPth’s and gives convergent optimized
precoders with 
SINR𝑔,𝑚 =𝛤th =ℎ−1(SEPth) ∀𝑔, 𝑚. It is
observed that the converged ˜𝑝total,appr,eq is decreased as SEPth
increases (i.e., less stringent SEP target). For Fig. 4(a) with
𝐺=5, the total-transmit-power curves are convergent for
small and moderate SEPth up to 10−2. However, the converged
˜𝑝total,appr ,eq is increased quite substantially as SEPth increases
in the moderate SEPth region. For Fig. 4(b) with 𝐺=5, where
This article has been accepted for publication in IEEE Transactions on Vehicular Technology. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/TVT.2022.3225195
© 2022 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See https://www.ieee.org/publications/rights/index.html for more information.
Authorized licensed use limited to: Nanjing Univ of Post & Telecommunications. Downloaded on December 02,2022 at 08:07:19 UTC from IEEE Xplore. Restrictions apply.

10
10-12 10-10 10-8 10-6 10-4 10-2
-12
-10
-8
-6
-4
-2
0
2
4
6
8
10-12 10-10 10-8 10-6 10-4 10-2
-2
0
2
4
6
8
10
12
14
16
18
10-12 10-10 10-8 10-6 10-4 10-2
5
10
15
20
25
30
35
10-12 10-10 10-8 10-6 10-4 10-2
18
20
22
24
26
28
30
32
34
36
38
Fig. 5. ˜𝑝total,appr,eq of Pappr,eq 0in (43) versus SEPth for different random
realizations of H(𝑔)
𝑚∀𝑔, 𝑚with 𝐾=3, 𝑁t=2, 𝑁r=8,and (a) 𝐺=2, (b)
𝐺=4, (c) 𝐺=6, (d) 𝐺=8.
SEPth >10−2, the precoder design fails to give convergent
solutions. In a word, the converged ˜𝑝total,appr,eq first decreases
and then increases as SEPth varies from the low region to the
moderate region in Fig. 4(a), and the design procedure fails to
give convergent results for larger SEPth as seen in Fig. 4(b).
Besides, the level of SEPth for convergent design is decreased
as 𝐺increases.
Qualitatively, the relations of SEPth to 𝛤th and ˜𝜖in Fig.
2are useful to understand these behaviors. When SEPth is
sufficiently small, say SEPth ≤10−3.5,˜𝜖is close to zero.
Under this condition, the system seems to operate like per-
fect SIC. In this case, the iterative design converges, and
the convergence of ˜𝑝total,appr ,eq is dominated by 𝛤th. Hence,
˜𝑝total,appr ,eq is decreased as SEPth increases (i.e., 𝛤th decreases)
for SEPth ≤10−3.5. When SEPth >10−3.5, the value of
˜𝜖grows, and it causes the transmit power being increased
considerably especially for 𝐺=5resulting in the total transmit
power larger than those for SEPth ≤10−3.5. Due to the
rapid growing of ˜𝜖with SEPth , the effect of ˜𝜖on the power
increment becomes more significant for larger SEPth, leading
the iterative procedure to fail to give convergent designs for
SEPth ≥10−1.5.
C. Demonstration of Convergent and Divergent Regions for
Solving the Equality-Constrained Approximate Problem
In Fig. 5, we plot the objective ˜𝑝total,appr,eq of the equality-
constrained problem Pappr,eq0versus SEPth for different ran-
dom realizations of H(𝑔)
𝑚∀𝑔, 𝑚with (𝑁t, 𝑁r, 𝐺, 𝐾 )given in
the caption. The 2×2QAM was used for modulation. For
each random realization, the iterative design in Table Iwas
carried out for 𝑁init =20 random initial values, out of which
the convergent solution that consumes the lowest total transmit
power was adopted as the final result and treated as a near-
optimal solution. If all the 𝑁init runs diverge, then ˜𝑝total ,appr,eq
is close to ∞and not be plotted in the figure. We observe that
˜𝑝total,appr ,eq tends to be monotonically decreasing in the whole
range of SEPth for the configurations with small (𝐺 , 𝐾), while
10-12 10-10 10-8 10-6 10-4 10-2
5
10
15
20
25
30
10-12 10-10 10-8 10-6 10-4 10-2
20
22
24
26
28
30
32
34
36
38
40
10-12 10-10 10-8 10-6 10-4 10-2
36
38
40
42
44
46
48
50
52
54
56
10-12 10-10 10-8 10-6 10-4 10-2
10
15
20
25
30
35
Fig. 6. ˜𝑝total,appr ,eq of the equality-constrained Pappr,eq0(dash curves) and
˜𝑝total,appr of the inequality-constrained Pappr0(solid curves) versus SEPth for
different random realizations of H(𝑔)
𝑚∀𝑔, 𝑚with (a) 𝑁t=2, 𝑁r=8, 𝐺 =
6, 𝐾 =3, (b) 𝑁t=2, 𝑁r=8, 𝐺 =8, 𝐾 =3, (c) 𝑁t=2, 𝑁r=8, 𝐺 =8, 𝐾 =
4, and (d) 𝑁t=2, 𝑁r=12, 𝐺 =8, 𝐾 =4.
it tends to decrease for small SEPth, then increase for larger
SEPth, and eventually grow up to infinity in a divergent region
for the configurations with large (𝐺, 𝐾). The observation
coincides with those behaviors in Figs. 3and 4.
D. Effectiveness of the Suboptimal Algorithm for Solving the
Approximate Design Problem
We examine the effectiveness of the developed greedy al-
gorithm for solving the inequality-constrained problem Pappr0
in (42). In Fig. 6, we plot the objective ˜𝑝total,appr of Pappr0
(solid curves) versus SEPth for different random realizations
of H(𝑔)
𝑚∀𝑔, 𝑚with configurations (𝑁t, 𝑁r, 𝐺 , 𝐾)given in
the caption. For each random realization, the ˜𝑝total,appr was
obtained by using the proposed greedy algorithm with “log-
arithmic” golden-section search developed in Sec. III-D. The
near-optimal ˜𝑝total,appr,eq of the equality-constrained Pappr,eq0
in (43) chosen from 𝑁init =20 local optima is also plotted
(dash curves), which has been examined in Fig. 5. Again, 2×2
QAM was used for modulation.
Given SEPth, the proposed algorithm first finds the mini-
mizer SEP∗of ˜𝑝total,appr ,eq (𝑥)and then determines ˜𝑝total,appr
by comparing SEPth and SEP∗. If SEPth ≥SEP∗, set
˜𝑝total,appr =˜𝑝total,appr,eq (SEP∗); otherwise, set ˜𝑝total,appr =
˜𝑝total,appr ,eq (SEPth). This means that, when SEPth ≥SEP∗, the
algorithm will give the design solution yielding SEP∗for each
user in order to minimize the total transmit power consumption
and also to meet the QoS requirement; when SEPth <SEP∗,
however, the two curves, ˜𝑝total,appr and ˜𝑝total,appr,eq versus
SEPth, should coincide with each other. These properties are
validated by the results in Figs. 6(a) to (d).
E. Total Transmit Power Versus SEP Threshold
We examine the total transmit power ˜𝑝total,appr for given SEP
threshold, SEPth. The 2×2QAM was used to determine the
This article has been accepted for publication in IEEE Transactions on Vehicular Technology. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/TVT.2022.3225195
© 2022 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See https://www.ieee.org/publications/rights/index.html for more information.
Authorized licensed use limited to: Nanjing Univ of Post & Telecommunications. Downloaded on December 02,2022 at 08:07:19 UTC from IEEE Xplore. Restrictions apply.

11
10-8 10-6 10-4 10-2
-10
-5
0
5
10
15
10-8 10-6 10-4 10-2
-10
-5
0
5
10
15
10-8 10-6 10-4 10-2
-10
-5
0
5
10
15
10-8 10-6 10-4 10-2
-10
-5
0
5
10
15
Fig. 7. ˜𝑝total,appr of Pappr0versus SEPth for 𝐺=2,3,4,5with (a) 𝑁t=
3, 𝑁r=16, 𝐾 =4, (b) 𝑁t=3, 𝑁r=12, 𝐾 =4, (c) 𝑁t=3, 𝑁r=16, 𝐾 =3,
and (d) 𝑁t=3, 𝑁r=12, 𝐾 =3.
approximate SIC-residual-power coefficient ˜𝜖. For each con-
figuration of (𝐺 , 𝐾, 𝑁t, 𝑁r,SEPth ), the ˜𝑝total,appr was averaged
over 1000 random realizations of H(𝑔)
𝑚∀𝑔, 𝑚.
Fig. 7plots the averaged ˜𝑝total ,appr versus SEPth using
the developed precoder design with the greedy algorithm for
different 𝐺with fixed (𝐾, 𝑁t, 𝑁r)as specified in the caption.
The total transmit power ˜𝑝total,appr increases with 𝐺and 𝐾(due
to increased inter- and intra-group interferences as well as SIC
residuals) while decreases with 𝑁r(due to additional spatial
degrees of freedom for improving the diversity gains of users
by properly designing the user precoders). In addition, we see
that for 𝐺=5in Fig. 7(b), the ˜𝑝total,appr curve stops decreasing
and saturates in the large SEPth region. This coincides with the
observations from Fig. 6that the total transmit power of using
the developed method with the greedy algorithm for solving
the approximate problem Pappr0tends to saturate in the large
SEPth region leading to SEP∗<SEPth for the configurations
with large (𝐺, 𝐾)and/or small (𝑁t, 𝑁r).
Recall that we use the large-scale pathloss as the inter- and
intra-group SIC ordering criterion. Despite its sub-optimality,
the impact of pathloss, which varies on a much shorter
time scale, is more stable and dominant as compared with
instantaneous fading. Thus, the ranking of user-to-BS distances
can be used as an approximation of the ranked received-signal-
powers for determining the SIC order. Note that solving the
optimal SIC order is a complicated task since the design
problem becomes extremely non-convex. One may have to
conduct exhaustive search by testing all possible SIC orderings
to obtain the optimal solution. However, it should be noted
that the number of possible SIC orderings for given 𝑁uusers,
denoted by 𝑄𝑁u, is as large as 𝑄𝑁u=𝑁u!=Î𝑁u
𝑛=1𝑛. Thus,
the computational complexity of exhaustive search is rather
unacceptable even if 𝑁uis not so large.
In Fig. 8, we compare the adopted trivial scheme’s per-
formance with the optimal one. We plot the total transmit
power ˜𝑝total,appr (averaged over 100 random realizations of
H(𝑔)
𝑚∀𝑔, 𝑚) versus SEPth with various configurations of
10-12 10-10 10-8 10-6 10-4 10-2
25
30
35
40
45
50
Fig. 8. ˜𝑝total,appr of Pappr0versus SEPth for using the trivial distance-
based SIC ordering and the exhaustive search over all possible (𝐺𝐾 )!SIC
orderings.
(𝑁r, 𝑁t, 𝐺, 𝐾). The 2×2QAM was used for modulation. As la-
belled in the legend, two schemes were considered, namely, the
adopted trivial distance-based SIC ordering and the optimum
by exhaustive search over all possible SIC orderings. Note
that the times of solving the inequality-constrained problem
Pappr0in the exhaustive search is 𝑄𝑁u=𝑁u!=Î𝐺𝐾
𝑛=1𝑛.
We can see that the adopted simple scheme with fixed inter-
and intra-group SIC decoding order can give near-optimal
performance on average. Note that the computational complex-
ity is significantly reduced as compared with the exhaustive
search. Though not surprisingly, this observation verifies the
adopted simple scheme’s efficiency in terms of transmit power
consumption and computational cost.
F. Designed, Analyzed, and Simulated SEPs
We examine the SEP performance of the developed precoder
design algorithm. The 2×2QAM was used. For each ran-
dom realization of H(𝑔)
𝑚∀𝑔, 𝑚, we performed the developed
algorithm with “logarithmic” golden-section search, yielding
{𝑝𝑔,𝑚 ,u(𝑔)
𝑚∀𝑔, 𝑚}. The obtained {𝑝𝑔, 𝑚,u(𝑔)
𝑚∀𝑔, 𝑚}gives ap-
proximate SINRs 
SINR𝑔,𝑚 ∀𝑔, 𝑚such that 
SINR𝑔,𝑚 ≥
𝛤th ∀𝑔, 𝑚 rather than the analyzed SINR𝑔 ,𝑚 ≥𝛤th ∀𝑔, 𝑚. We
mapped each designed 
SINR𝑔,𝑚 to its corresponding designed
SEP in terms of ℎ
SINR𝑔,𝑚as given in (27). Then, the
analyzed SINR, SINR𝑔,𝑚, of each user was calculated by
(29) with the obtained {𝑝𝑔, 𝑚,u(𝑔)
𝑚∀𝑔, 𝑚}and the detection-
error covariance matrices Cdet,𝑔 ∀𝑔that were computed
by the method described in Sec. II-C. Similarly, the ana-
lyzed SEP was computed as ℎ(SINR𝑔, 𝑚). With the obtained
{𝑝𝑔,𝑚 ,u(𝑔)
𝑚∀𝑔, 𝑚}, we performed uplink transmission simula-
tion and evaluated simulated SEPs for the 𝑁u=𝐺𝐾 users by
transmitting random 2×2QAM symbols and performing the
multi-group detection.
In Fig. 9, we plot the designed, analyzed, and simulated
SEPs (all of which were averaged over the 𝑁u=𝐺𝐾 users)
versus SEPth with (𝐺, 𝐾 , 𝑁t, 𝑁r)given in the caption. In
the experiments, the results were averaged over 100 random
This article has been accepted for publication in IEEE Transactions on Vehicular Technology. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/TVT.2022.3225195
© 2022 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See https://www.ieee.org/publications/rights/index.html for more information.
Authorized licensed use limited to: Nanjing Univ of Post & Telecommunications. Downloaded on December 02,2022 at 08:07:19 UTC from IEEE Xplore. Restrictions apply.

12
10-5 10-4 10-3 10-2 10-1
10-5
10-4
10-3
10-2
10-1
10-5 10-4 10-3 10-2 10-1
10-5
10-4
10-3
10-2
10-1
10-5 10-4 10-3 10-2 10-1
10-5
10-4
10-3
10-2
10-1
10-5 10-4 10-3 10-2 10-1
10-5
10-4
10-3
10-2
10-1
Fig. 9. Designed, analyzed, and simulated SEPs versus SEPth with 𝐾=
3, 𝑁t=2, 𝑁r=8,and (a) 𝐺=2, (b) 𝐺=3, (c) 𝐺=4, (d) 𝐺=5.
10-10 10-8 10-6 10-4 10-2
10-10
10-8
10-6
10-4
10-2
10-10 10-8 10-6 10-4 10-2
10-10
10-8
10-6
10-4
10-2
0.06 0.07 0.08 0.09 0.1
0.06
0.07
0.08
0.09
0.1
Fig. 10. Designed and analyzed SEPs versus SEPth with 𝐾=4, 𝑁t=2, 𝐺 =
2,3,4,5,6,7,8,and (a) 𝑁r=8, (b) 𝑁r=9.
realizations of H(𝑔)
𝑚∀𝑔, 𝑚. In all the subfigures except
Fig. 9(d), for relatively larger ratio 𝑁t𝑁r
𝐺𝐾 , the designed SEP
curves turn out to be very close to the desired straight line
𝑦=𝑥. This coincides with the results of the experiments in
Figs. 5and 6that the ˜𝑝total,appr,eq of the equality-constrained
problem Pappr,eq0tends to be monotonically decreasing in the
whole range of SEPth for configurations with small (𝐺 , 𝐾)
and/or large (𝑁t, 𝑁r), giving SEP∗=SEPth. In Fig. 9(d) with
relatively smaller ratio 𝑁t𝑁r
𝐺𝐾 , the designed SEP curve behaves
like the straight line 𝑦=𝑥in low SEPth regime and becomes
a horizontal line in moderate-to-high SEPth regime. This
coincides with the experiments in Figs. 5and 6that ˜𝑝total,appr,eq
tends to saturate in moderate-to-high SEPth regime for large
(𝐺, 𝐾 )and/or small (𝑁t, 𝑁r), leading to SEP∗<SEPth.
Importantly, we see that the analyzed SEP is almost the
same as the designed SEP. This indicates that the difference
between the approximate 
SINR𝑔,𝑚 and the analyzed SINR𝑔,𝑚
is neglected over averaging. In addition, the simulated SEP is
shown to be lower than or, at worst, slightly larger than the
pre-specified SEP threshold SEPth for all the cases. This means
that, although the design is based on the approximate SINR,
the user precoders obtained by the proposed design approach
can achieve an SEP performance lower than or near to a given
Fig. 11. 𝑝total versus QoS for the proposed schemes and the conventional
signal alignment MIMO-NOMA scheme with 𝑁t=4, 𝑁r=6and (a) 𝑁clu =
3, (b) 𝑁clu =4, (c) 𝑁clu =5(𝑁u=2𝑁clu =6,8,10, correspondingly).
SEP threshold. In other words, the derived approximate 
SINR
of the proposed scheme is an effective SNR expression to
design optimized precoders to satisfy SEP requirements.
In Fig. 10, we plot the designed and analyzed SEPs in a
wider range of SEPth for 𝐺=2,3,4,5,6,7,8with 𝐾=4, 𝑁t=
2, and (a) 𝑁r=8, (b) 𝑁r=9. In the figure, we see again that
the analyzed SEP is almost the same as the corresponding
designed SEP. In addition, the designed SEP curves are close
to the straight line 𝑦=𝑥in the whole range of SEPth for small
𝐺(similar to Fig. 9), while they become a horizontal line in
moderate-to-large SEPth region for large 𝐺with the turning
point getting closer to zero as 𝐺getting larger. The reason is
that, with larger (𝐺, 𝐾)and/or smaller (𝑁t, 𝑁r), the minimizer
SEP∗of ˜𝑝total,appr ,eq (𝑥)for problem Pappr,eq0becomes closer
to zero (at the cost of consuming larger total transmit power).
For the same reason, by comparing Figs. 10(a) and (b), where
the configurations are the same except 𝑁r(𝑁rin Fig. 10(a) is
smaller than that of Fig. 10(b)), we see that the turning point
in Fig. 10(a) is closer to zero than that in Fig. 10(b).
G. Comparison With Other Design Schemes
We adopt the widely used signal-alignment-MIMO-NOMA
(SA-MIMO-NOMA) scheme in [15]–[18] and our previously
proposed ZF-based multi-group MIMO-NOMA scheme in
[20] as benchmarks. For the sake of comparison, 𝑁u=2𝑁clu
users are divided into 𝑁clu clusters of two users for the
SA-MIMO-NOMA scheme, while the users are divided into
two groups of 𝑁clu users for the proposed ZF- and QR-
decomposition-based schemes. Following the assumption in
[15]–[18], [20], perfect SIC is considered. The three design
schemes are labelled as follows:
i) Prop., ZF: the proposed scheme for using ZF detection;
ii) Prop., QR: the proposed scheme for using QR-based
detection; and
iii) Signal Alignment: the SA-MIMO-NOMA scheme.
Fig. 11 plots the total transmit power 𝑝total versus QoS
requirement for the three design schemes with parameters
This article has been accepted for publication in IEEE Transactions on Vehicular Technology. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/TVT.2022.3225195
© 2022 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See https://www.ieee.org/publications/rights/index.html for more information.
Authorized licensed use limited to: Nanjing Univ of Post & Telecommunications. Downloaded on December 02,2022 at 08:07:19 UTC from IEEE Xplore. Restrictions apply.

13
10-6 10-5 10-4 10-3 10-2 10-1
0
0.2
0.4
0.6
0.8
1
10-6 10-5 10-4 10-3 10-2 10-1
0
0.2
0.4
0.6
0.8
1
10-6 10-5 10-4 10-3 10-2 10-1
0
0.2
0.4
0.6
0.8
1
10-6 10-5 10-4 10-3 10-2 10-1
0
0.2
0.4
0.6
0.8
1
Fig. 12. Outage probability of Pfixed0versus SEPth for the conventional fixed-
𝜖fmodel assuming 𝜖f=0.001,0.002,0.004,0.008 with 𝐾=4, 𝑁t=2, 𝑁r=
12, and (a) 𝐺=3, (b) 𝐺=4, (c) 𝐺=5, (d) 𝐺=6.
10-6 10-5 10-4 10-3 10-2 10-1
-10
-5
0
5
10
15
20
25
30
35
10-6 10-5 10-4 10-3 10-2 10-1
-10
-5
0
5
10
15
20
25
30
35
10-6 10-5 10-4 10-3 10-2 10-1
-10
-5
0
5
10
15
20
25
30
35
10-6 10-5 10-4 10-3 10-2 10-1
-10
-5
0
5
10
15
20
25
30
35
Fig. 13. ˜𝑝total,appr of Pappr 0for the proposed SIC error model and ˜𝑝total,fixed
(convergent cases) of Pfixed0for the conventional fixed-𝜖fmodel versus SEPth
assuming 𝜖f=0.001,0.002,0.004,0.008,0.016 with 𝐾=4, 𝑁t=2, 𝑁r=
12, and (a) 𝐺=3, (b) 𝐺=4, (c) 𝐺=5, (d) 𝐺=6.
given in the caption. It is seen that the proposed schemes
consume less transmit power than the SA-MIMO-NOMA
benchmark for various configurations. The precoder design
of the SA-MIMO-NOMA scheme focuses on projecting the
effective channels of the paired users into the same direction,
leading to limited exploitation of the spatial design freedom. In
contrast, in the proposed schemes, user precoders are designed
by taking the equalizers’ property into account. Meanwhile,
when 𝑁clu gets closer to 𝑁r, the performance superiority of
the ZF-based scheme over the SA-MIMO-NOMA scheme
reduces. This is because the intra-group-inter-user correlation
cannot be well mitigated by the linear ZF detection for such
large group size. Importantly, we see that the QR-based multi-
group scheme always outperforms upon the ZF-based one.
This is because the QR-based scheme uses not only inter-
group SIC but also intra-group SIC. The rigorous proof of the
superiority of the QR-based detection over the ZF detection
can be found in [29, Appendix B.1].
H. Outage Probability for Using the Conventional Fractional-
SIC-Error Model
In this subsection, we examine the conventional fractional-
SIC-error model that has been widely adopted in the literature
to roughly account for all sources of SIC errors [24]–[28].
In the model, a fractional-SIC-error parameter 𝜖fis prede-
fined and fixed, while its value is preset without quantitative
justification prior to system design and/or performance anal-
ysis. Note that the key difference between our SIC-residual
formulation and this conventional model is that the impact of
instantaneous SINR on the SIC error is included in our model.
Let us denote the design problem for the conventional model
by Pfixed0, whose objective function is denoted by ˜𝑝total,fixed .
We can employ the previously developed inner-outer iterative
algorithm to solve this problem 7. We plot a pessimistic outage
probability 𝑃out =𝑁out
𝑁total versus SEPth using the 2×2QAM for
modulation in Fig. 12 with configuration parameters given in
the caption, where 𝑁total is the number of random realizations
of H(𝑔)
𝑚∀𝑔, 𝑚, and 𝑁out is the number of realizations that
the inner-outer iterative procedure cannot converge.
From the figure, we see that the conventional fixed-𝜖f
model leads to severe infeasibility issue and may fail to give
convergent solutions under certain conditions, namely, large
𝜖fand/or large 𝐺. In contrast, our SIC-residual formulation
always provides a precoder design to satisfy a given SEP re-
quirement by employing the developed suboptimal algorithm.
The key of success is that the mutual impacts between the
users’ instantaneous SINRs and the SIC errors are considered
in our formulation and design, while the fractional-SIC-error
parameter 𝜖fis fixed and unrelated to the instantaneous SINRs
in the conventional model.
I. Total Transmit Powers for Different SIC Error Models
We compare the total transmit powers ˜𝑝total,appr of Pappr0for
using the proposed SIC error model and ˜𝑝total,fixed of Pfixed0
for using the conventional fixed-𝜖fmodel. We plot ˜𝑝total,appr
and ˜𝑝total,fixed versus SEPth for using the 2×2QAM in Fig. 13
with parameters given in the caption. The solid curves are the
results for using the proposed model (“˜𝑝total,appr , Prop. model”
in the legend), while the other curves with different markers
correspond to the results for using the conventional model
with fixed 𝜖f=0.001,0.002,0.004,0.008,0.016 (“˜𝑝total,fixed ,
Conv. model”). If the problem Pfixed0does not have a feasible
design for any channel realizations for a given SEPth, then
7The precoder design problem Pfixed0can be formulated as: ˜𝑝total,fixed =
min Í𝐺
𝑔=1Í𝐾
𝑚=1𝑝𝑔,𝑚 , s.t. ∥u(𝑔)
𝑚∥=1,𝑝𝑔,𝑚 (u(𝑔)𝐻
𝑚A(𝑔)
𝑚𝑚u(𝑔)
𝑚)2
u(𝑔)𝐻
𝑚B(𝑔)
𝑚u(𝑔)
𝑚
=𝛤th ∀𝑔, 𝑚,
where A(𝑔)
𝑚 𝑗 =H(𝑔)𝐻
𝑚C−𝐻/2
𝑔I𝑁r−Í𝐾
𝑙=𝑚+1q(𝑔)
𝑙q(𝑔)𝐻
𝑙C−1/2
𝑔H(𝑔)𝐻
𝑗
and B(𝑔)
𝑚=A(𝑔)
𝑚𝑚 +𝜖fÍ𝑚−1
𝑗=1𝑝𝑔, 𝑗 A(𝑔)
𝑚 𝑗 u(𝑔)
𝑗u(𝑔)𝐻
𝑗A(𝑔)𝐻
𝑚 𝑗 with C𝑔=
Í𝐺
𝑙=𝑔+1F𝑙P𝑙F𝐻
𝑙+𝜖fÍ𝑔−1
𝑗=1F𝑗P𝑗F𝐻
𝑗+Cz. Similar to the problem Pappr,eq0
in (43), Pfixed0suffers from infeasibility. Again, if the inner-outer iterative
algorithm converges, then the convergent result is adopted as the final solution.
If it fails to converge for different initial values, then we conservatively report
that Pfixed0is infeasible with ˜𝑝total,fixed → ∞.
This article has been accepted for publication in IEEE Transactions on Vehicular Technology. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/TVT.2022.3225195
© 2022 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See https://www.ieee.org/publications/rights/index.html for more information.
Authorized licensed use limited to: Nanjing Univ of Post & Telecommunications. Downloaded on December 02,2022 at 08:07:19 UTC from IEEE Xplore. Restrictions apply.

14
10-6 10-5 10-4 10-3 10-2 10-1
10-5
10-4
10-3
10-2
10-1
Fig. 14. ˜𝜖 𝑑2versus SEPth using the 2×2QAM for modulation.
no result is marked for that SEP threshold. Due to the
situation that the conventional fixed-𝜖fmodel is so vulnerable
to infeasibility, the ˜𝑝total ,fixed’s were computed by averaging
over the convergent cases only.
From Fig. 13, we see that the proposed model provides
lower transmit power than that of the conventional model
over a large range of SEPth. Only when SEPth is large does
the ˜𝑝total,appr become higher than the ˜𝑝total,fixed, where the
˜𝑝total,fixed is taking the advantage of only counting convergent
results. To explain this, we use the results of the derived
approximate SIC-residual-power coefficient ˜𝜖 𝑑2shown in Fig.
2(b). In Fig. 14, we plot ˜𝜖 𝑑 2versus SEPth on a log-log scale
for ease of visual inspection. Note that 𝑑=1
√2is used for
the 2×2QAM according to (3). Let us take the particular
case of (𝑁t=2, 𝑁r=12, 𝐺 =6, 𝐾 =4, 𝜖f=0.002)in
Fig. 13(d) for example. From Fig. 14, we learn that ˜𝜖 𝑑 2
is larger than 0.002 when SEPth >10−3and vice versa.
This implies that the conventional model with 𝜖f=0.002
underestimates (resp., overestimates) the demodulation error
when SEP is higher (resp., lower) than 10−3. Hence, one can
expect that the ˜𝑝total,appr and ˜𝑝total,fixed curves intersect roughly
at SEPth =10−3for 𝜖f=0.002, as they do in Fig. 13(d). The
other intersection points between the ˜𝑝total,appr and ˜𝑝total,fixed
curves in Fig. 13 can be explained similarly.
As a final remark, the conventional fixed-𝜖fmodel lacks of
capability to consider the detection error in the demodulation,
thus, it not only has infeasibility problem, but also encounters
underestimation/overestimation of SIC errors.
V. CONCLUSION
In this paper, our works on the previously proposed multi-
group MIMO-NOMA scheme under perfect SIC are extended
to the imperfect SIC case. In the uplink, the transmitted
user signals are encoded with the constellation of a gen-
eral symmetric rectangular QAM, and SIC residual errors
are generated from symbol demodulation. With this error
generation, statistics of the SIC residuals are analyzed. The
statistics are shown to be useful to derive the expressions of
approximate SIC-residual-power coefficient and approximate
SINR. These expressions can facilitate the precoder design to
include the effect of the SIC residuals explicitly. A suboptimal
precoder design algorithm with efficient “logarithmic” golden-
section search is developed to minimize the total transmit
power subject to SEP constraints. Simulation results show the
effectiveness of the derived design expressions and the devel-
oped design algorithm in terms of transmit power, SEP, and
feasibility, which validate the detection error analysis. Unlike
the conventional fractional-SIC-error model that suffers from
infeasibility and underestimation/overestimation of residual
errors, the proposed SIC error model, which can incorporate
the effects of SIC residuals in the precoder design, enables the
developed precoder design to provide feasible solutions for a
wide range of SEPs.
APPENDIX A
PROO F OF LEMMA 1
Proof: Step 1. For discussion convenience, let us consider
a generic decision statistic expressed as
¯𝑠=𝑟0𝑠+𝜂+Δ0,(A.1)
where 𝑟0>0is a real constant, 𝜂∼ CN(0, 𝜎2),Δ0is a
complex constant, and 𝑠is drawn from Sin (2) with equal
probability. The decision ˘𝑠is made as ˘𝑠=arg min𝛼∈S |¯𝑠−𝑟0𝛼|
conditioned on Δ0. Let Δ = 𝑠−˘𝑠
𝑑∈ SΔ=𝛽𝛽=𝛼−˘𝛼
𝑑∀𝛼, ˘𝛼∈
S. By symmetry, we have
PrΔ = 𝛽Δ0=𝛽0=PrΔ = −𝛽Δ0=−𝛽0.(A.2)
Step 2. Let Δ𝑔,𝑚 =𝑠𝑔 ,𝑚 −˘𝑠𝑔, 𝑚
𝑑∈ SΔ∀𝑔, 𝑚. In this step, we
prove that the statement H𝑚in (A.3) holds for 𝑚=1,·· · , 𝐾 .
H𝑚:𝜋𝛽𝑚,···,𝛽1=𝜋−𝛽𝑚,··· ,−𝛽1∀𝛽𝑚,· · · , 𝛽1∈ SΔ,(A.3)
where 𝜋𝛽𝑘,···,𝛽1=PrΔ1,𝑘 =𝛽𝑘,· · · ,Δ1,1=𝛽1.
For 𝑚=1, the decision statistic for user (1,1)is ¯𝑠1,1=
√𝑝1,1𝑟1,11𝑠1,1+𝜂1,1with Δ0=0. The statement H𝑚holds
for 𝑚=1, i.e., 𝜋𝛽1=𝜋−𝛽1∀𝛽1∈ SΔ, because of the
symmetry of the constellation and the probability density
function (PDF) of the noise 𝜂1,1. We assume that H𝑘is true
for some positive integer 𝑘. For the decision statistic of user
(1, 𝑘 +1)given by ¯𝑠1, 𝑘+1=√𝑝1, 𝑘+1𝑟1,(𝑘+1) (𝑘+1)𝑠1, 𝑘+1+𝜂1, 𝑘+1+
𝑑Í𝑘
𝑗=1√𝑝1, 𝑗 𝑟1,(𝑘+1)𝑗Δ1, 𝑗 with 𝜂1,𝑘+1∼ CN0,1, we have
𝜋𝛽𝑘+1,···,𝛽1=𝜋𝛽𝑘+1|𝛽𝑘,··· , 𝛽1·𝜋𝛽𝑘,···, 𝛽1
=𝜋𝛽𝑘+1|𝛽𝑘,···,𝛽1·𝜋−𝛽𝑘,··· ,−𝛽1
=𝜋−𝛽𝑘+1|−𝛽𝑘,···,−𝛽1·𝜋−𝛽𝑘,···,−𝛽1(A.4)
=𝜋−𝛽𝑘+1,···,−𝛽1,(A.5)
where 𝜋𝛽𝑘+1|𝛽𝑘,···,𝛽1=PrΔ1,𝑘 +1=𝛽𝑘+1Δ1,𝑘 =𝛽𝑘,· · · ,Δ1,1=
𝛽1and (A.4) holds based on (A.2). The result in (A.5) shows
that H𝑚is also true for 𝑚=𝑘+1. Hence, by mathematical
induction, the statement H𝑚holds for 𝑚=1,··· , 𝐾 .
Step 3. In this step, we base on (A.2) and (A.3) to show that
PrΔ1, 𝑚 =𝛽𝑚=PrΔ1, 𝑚 =−𝛽𝑚holds for all 𝑚and 𝛽𝑚∈
This article has been accepted for publication in IEEE Transactions on Vehicular Technology. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/TVT.2022.3225195
© 2022 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See https://www.ieee.org/publications/rights/index.html for more information.
Authorized licensed use limited to: Nanjing Univ of Post & Telecommunications. Downloaded on December 02,2022 at 08:07:19 UTC from IEEE Xplore. Restrictions apply.

15
SΔ. Considering the decision statistic ¯𝑠1,𝑚 =√𝑝1,𝑚𝑟1, 𝑚𝑚 𝑠1,𝑚 +
𝜂1,𝑚 +𝑑Í𝑚−1
𝑗=1√𝑝1, 𝑗 𝑟1, 𝑚 𝑗 Δ1, 𝑗 for user (1, 𝑚), we have
PrΔ1, 𝑚 =𝛽𝑚
=𝛽𝑚−1,···,𝛽1∈SΔ𝜋𝛽𝑚|𝛽𝑚−1,···, 𝛽1·𝜋𝛽𝑚−1,···,𝛽1
=𝛽𝑚−1,···,𝛽1∈SΔ𝜋𝛽𝑚|𝛽𝑚−1,···, 𝛽1·𝜋−𝛽𝑚−1,···,−𝛽1
=𝛽𝑚−1,···,𝛽1∈SΔ𝜋−𝛽𝑚|−𝛽𝑚−1,··· ,−𝛽1·𝜋−𝛽𝑚−1,··· ,−𝛽1
(A.6)
=PrΔ1, 𝑚 =−𝛽𝑚,(A.7)
where (A.6) holds due to the fact that we can always find −𝛽
in SΔfor any 𝛽∈ SΔ.
Step 4. Using (A.7), we are now ready to show that E[𝑠1,𝑚−
˘𝑠1,𝑚 ]=0holds. We have
E[𝑠1,𝑚 −˘𝑠1, 𝑚]=𝑑·E[Δ1, 𝑚]
=𝑑𝛽𝑚∈SΔ
𝛽𝑚Pr{Δ1,𝑚 =𝛽𝑚}
=𝑑𝛽𝑚∈SΔ(−𝛽𝑚)Pr{Δ1,𝑚 =−𝛽𝑚}
=𝑑𝛽𝑚∈SΔ(−𝛽𝑚)Pr{Δ1,𝑚 =𝛽𝑚}
=−E[𝑠1,𝑚 −˘𝑠1, 𝑚],(A.8)
implying E[𝑠1,𝑚 −˘𝑠1, 𝑚]=0∀𝑚and thus E[s1−˘
s1]=0𝐾×1.
Step 5. For 𝑔 > 1, the post-filtered signal ˆ
s𝑔=R𝑔s𝑔+𝜼𝑔
shares the same structure as ˆ
s1. Therefore, the above analysis
for 𝑔=1can be applied, leading to E[s𝑔−˘
s𝑔]=0𝐾×1∀𝑔.
The proof of Lemma 1is thus completed.
APPENDIX B
DERIVATION OF DETECTION-ERROR COVARIANCE
MATR IC ES F OR U SI NG T HE 2×2QAM
This appendix is to develop an efficient procedure for
calculating the elements of detection-error covariance matrices
{Cdet,𝑔 ∀𝑔}for using the 2×2QAM for modulation.
Let the 2×2QAM symbol set be denoted by S2×2=
{𝛼1, 𝛼2, 𝛼3, 𝛼4}with 𝛼1=(1+i)𝑑, 𝛼2=(1−i)𝑑, 𝛼3=(−1−i)𝑑,
and 𝛼4=(−1+i)𝑑, where 𝑑=1
√2. Define Δ𝑔,𝑚 =𝑠𝑔 ,𝑚 −˘𝑠𝑔, 𝑚
𝑑∈
S2×2,Δ={0,±2,±2i,±2±2i}∀𝑔, 𝑚. Note that |S2×2|=4and
|S2×2,Δ|=9. For the general decision statistic ¯𝑠=𝑟0𝑠+𝜂+Δ0
with 𝑟0>0, 𝜂 ∼ CN(0, 𝜎2), and 𝑠∈ S2×2, the probabilities
Pr{˘𝑠=𝛼𝑖𝑠=𝛼𝑗}for 𝑖, 𝑗 ∈ {1,2,3,4}can be expressed as
Pr{˘𝑠=𝛼1𝑠=𝛼𝑗}=𝑄(𝑐1)𝑄(𝑐2),
Pr{˘𝑠=𝛼2𝑠=𝛼𝑗}=𝑄(𝑐1)[1−𝑄(𝑐2)],
Pr{˘𝑠=𝛼3𝑠=𝛼𝑗}=[1−𝑄(𝑐1)][1−𝑄(𝑐2)],
Pr{˘𝑠=𝛼4𝑠=𝛼𝑗}=[1−𝑄(𝑐1)]𝑄(𝑐2)(B.1)
where 𝑐1=−√2ℜ{𝑟0𝛼𝑗+Δ0}/𝜎and 𝑐2=−√2ℑ{𝑟0𝛼𝑗+
Δ0}/𝜎. For given 𝛽𝑚, ..., 𝛽1∈ S2×2,Δ, one can calculate
Pr{Δ𝑔, 𝑚 =𝛽𝑚Δ𝑔,𝑚 −1=𝛽𝑚−1,··· ,Δ𝑔 ,1=𝛽1}as (B.2) at
the top of next page, where each summand in (B.2) can be
obtained by using (B.1).
Next, we list several equations, which are useful for saving
computation later. For the general decision statistic in (A.1),
it is straightforward to extend the property (A.2) as
Pr{Δ = 𝛽Δ0=𝛽0}=Pr{Δ = −𝛽Δ0=−𝛽0},(B.3)
Pr{Δ = 𝛽Δ0=𝛽0}=Pr{Δ = 𝛽∗Δ0=𝛽∗
0},(B.4)
Pr{Δ = 𝛽Δ0=𝛽0}=Pr{Δ = −𝛽∗Δ0=−𝛽∗
0}.(B.5)
The justifications for (B.4) and (B.5) are similar to that of
(B.3), and we omit the proof for brevity.
Without loss of generality, we derive [Cdet,𝑔 ]𝑚𝑛 for 𝑚 > 𝑛,
which is given by (18) and (19). By inspecting (19), the re-
maining task is to calculate all Pr{Δ𝑔, 𝑚 =𝛽𝑚,··· ,Δ𝑔 ,1=𝛽1}
for 𝑚=1,··· , 𝐾 and 𝛽𝑚,·· · , 𝛽1∈ S2×2,Δ. We will perform
the calculation from Pr{Δ𝑔, 1=𝛽1}for all 𝛽1∈ S2×2,Δto
Pr{Δ𝑔, 𝑗 =𝛽𝑗for 𝑗≤𝐾}for all 𝛽𝐾,·· · , 𝛽1∈ S2×2,Δ, during
which we can make use of the intermediate results.
i. Compute Pr{Δ𝑔, 1=𝛽1}for all 𝛽1∈ S2×2,Δ.
We start from computing Pr{Δ𝑔, 1=𝛽1}for all 𝛽1∈ S2×2,Δ.
Importantly, since Pr{Δ𝑔,1=𝛽1}=Pr{Δ𝑔,1=𝛽∗
1}=Pr{Δ𝑔, 1=
−𝛽1}=Pr{Δ𝑔, 1=−𝛽∗
1}holds as can be seen by setting
Δ0in (B.3)-(B.5), we only need to compute four probability
terms, namely, Pr{Δ𝑔,1=0},Pr{Δ𝑔,1=2},Pr{Δ𝑔, 1=2i}, and
Pr{Δ𝑔, 1=2+2i}, while the rest can be read out without
calculation by making use of (B.3)-(B.5).
ii. Compute Pr{Δ𝑔, 𝑗 =𝛽𝑗for 𝑗≤𝑚}for all 𝛽𝑚,·· · , 𝛽1∈
S2×2,Δand 𝑚=2,·· · , 𝐾 .
Similar to Step i, we calculate Pr{Δ𝑔, 𝑚 =𝛽𝑚Δ𝑔, 𝑗 =
𝛽𝑗for 𝑗 < 𝑚 }Pr{Δ𝑔, 𝑗 =𝛽𝑗for 𝑗 < 𝑚}. Since Pr{Δ𝑔, 𝑗 =
𝛽𝑗for 𝑗 < 𝑚 }’s have been obtained, we only need to find
Pr{Δ𝑔, 𝑚 =𝛽𝑚Δ𝑔, 𝑗 =𝛽𝑗for 𝑗 < 𝑚}for all 𝛽𝑚,· · · , 𝛽1∈
S2×2,Δ. Again, by using the properties (B.3)-(B.5), we only
need to compute 4× |S2×2,Δ|𝑚−1=4×9𝑚−1probability
terms, namely, Pr{Δ𝑔,𝑚 =0Δ𝑔, 𝑗 =𝛽𝑗for 𝑗 < 𝑚},Pr{Δ𝑔, 𝑚 =
2Δ𝑔, 𝑗 =𝛽𝑗for 𝑗 < 𝑚},Pr{Δ𝑔,𝑚 =2iΔ𝑔 , 𝑗 =𝛽𝑗for 𝑗 <
𝑚}, and Pr{Δ𝑔, 𝑚 =2+2iΔ𝑔, 𝑗 =𝛽𝑗for 𝑗 < 𝑚}for all
𝛽𝑚−1,··· , 𝛽1∈ S2×2,Δ, and then directly read out the rest
ones.
After obtaining Pr{Δ𝑔, 𝑗 =𝛽𝑗for 𝑗≤𝑚}for all
𝛽𝑚,··· , 𝛽1∈ S2×2,Δand 𝑚=1,·· · , 𝐾,[Cdet,𝑔 ]𝑚𝑛 can be
calculated by (18). Regarding the computational complexity
for constructing Cdet,𝑔 , the times of invoking the Gaussian Q-
function (in order for obtaining the probability terms) of the
developed efficient procedure is in the order of O4+4×9+
· · · + 4×9𝐾−1≈ O(9𝐾).
REFERENCES
[1] M. Shafi et al., “5G: A tutorial overview of standards, trials, challenges,
deployment, and practice,” IEEE J. Sel. Areas Commun., vol. 35, no. 6,
pp. 1201–1221, Jun. 2017.
[2] M. Vu and A. Paulraj, “MIMO wireless linear precoding,” IEEE Signal
Process. Mag., vol. 24, no. 5, pp. 86–105, Sep. 2007.
[3] Z. Chen, Z. Ding, X. Dai, and R. Zhang, “An optimization perspective of
the superiority of NOMA compared to conventional OMA,” IEEE Trans.
Signal Process., vol. 65, no. 19, pp. 5191–5202, Oct. 2017.
[4] L. Lei, D. Yuan, C. K. Ho, and S. Sun, “Power and channel allocation
for non-orthogonal multiple access in 5G systems: Tractability and
computation,” IEEE Trans. Wireless Commun., vol. 15, no. 12, pp. 8580–
8594, Dec. 2016.
This article has been accepted for publication in IEEE Transactions on Vehicular Technology. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/TVT.2022.3225195
© 2022 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See https://www.ieee.org/publications/rights/index.html for more information.
Authorized licensed use limited to: Nanjing Univ of Post & Telecommunications. Downloaded on December 02,2022 at 08:07:19 UTC from IEEE Xplore. Restrictions apply.

16
Pr{Δ𝑔, 𝑚 =𝛽𝑚Δ𝑔,𝑚 −1=𝛽𝑚−1,··· ,Δ𝑔 ,1=𝛽1}
=𝛼, ˘𝛼∈S2×2,Δ;𝛼−˘𝛼=𝛽𝑚𝑑Pr{˘𝑠𝑔 ,𝑚 =˘𝛼𝑠𝑔,𝑚 =𝛼;Δ𝑔, 𝑗 =𝛽𝑗for 𝑗 < 𝑚 }Pr{𝑠𝑔 ,𝑚 =𝛼Δ𝑔, 𝑗 =𝛽𝑗for 𝑗 < 𝑚}
=1
𝑁I𝑁Q𝛼, ˘𝛼∈S2×2,Δ;𝛼−˘𝛼=𝛽𝑚𝑑Pr{˘𝑠𝑔 ,𝑚 =˘𝛼𝑠𝑔,𝑚 =𝛼;Δ𝑔, 𝑗 =𝛽𝑗for 𝑗 < 𝑚 }
=1
𝑁I𝑁Q𝛼, ˘𝛼∈S2×2,Δ;𝛼−˘𝛼=𝛽𝑚𝑑Pr ˘𝑠𝑔, 𝑚 =˘𝛼𝑠𝑔,𝑚 =𝛼;𝑟0=√𝑝𝑔 ,𝑚𝑟𝑔,𝑚𝑚 ,Δ0=𝑑𝑚−1
𝑗=1√𝑝𝑔, 𝑗 𝑟𝑔 ,𝑚 𝑗 𝛽𝑗, 𝜎2=1(B.2)
[5] G. Lv, X. Li, R. Shang, P. Xue, and Y. Jin, “Dynamic resource alloca-
tion for uplink non-orthogonal multiple access systems,” IET Commun.,
vol. 12, pp. 649–655, Apr. 2018.
[6] Z. Ding, Z. Yang, P. Fan, and H. V. Poor, “On the performance of non-
orthogonal multiple access in 5G systems with randomly deployed users,”
IEEE Signal Process. Lett., vol. 21, no. 12, pp. 1501–1505, Dec. 2014.
[7] H. Tabassum, E. Hossain, and J. Hossain, “Modeling and analysis of
uplink non-orthogonal multiple access in large-scale cellular networks
using Poisson cluster processes,” IEEE Trans. Commun., vol. 65, no. 8,
pp. 3555–3570, Aug. 2017.
[8] Y. Liu et al., “Multiple-antenna-assisted non-orthogonal multiple access,”
IEEE Wireless Commun., vol. 25, no. 2, pp. 17–23, Apr. 2018.
[9] H. Wang, C. Liu, Z. Shi, Y. Fu, and R. Song, “Power minimization for
uplink RIS-assisted CoMP-NOMA networks with GSIC,” IEEE Trans.
Commun., vol. 70, no. 7, pp. 4559–4573, Jul. 2022.
[10] J. Cui, Z. Ding, and P. Fan, “Power minimization strategies in downlink
MIMO-NOMA systems,” in Proc. IEEE Int. Conf. Commun. (ICC), May
2017, pp. 1–6.
[11] X. Sun et al., “Joint beamforming and power allocation in downlink
NOMA multiuser MIMO networks,” IEEE Trans. Wireless Commun.,
vol. 17, no. 8, pp. 5367–5381, Aug. 2018.
[12] M. F. Hanif, Z. Ding, T. Ratnarajah, and G. K. Karagiannidis, “A
minorization-maximization method for optimizing sum rate in the down-
link of non-orthogonal multiple access systems,” IEEE Trans. Signal
Process., vol. 64, no. 1, pp. 76–88, Jan. 2016.
[13] V. Nguyen, H. D. Tuan, T. Q. Duong, H. V. Poor, and O. Shin, “Precoder
design for signal superposition in MIMO-NOMA multicell networks,”
IEEE J. Sel. Areas Commun., vol. 35, no. 12, pp. 2681–2695, Dec. 2017.
[14] M. Jiang, Y. Li, Q. Zhang, Q. Li, and J. Qin, “MIMO beamforming
design in nonorthogonal multiple access downlink interference channels,”
IEEE Trans. Veh. Technol., vol. 67, no. 8, pp. 6951–6959, Aug. 2018.
[15] Z. Ding, R. Schober, and H. V. Poor, “A general MIMO framework for
NOMA downlink and uplink transmission based on signal alignment,”
IEEE Trans. Wireless Commun., vol. 15, no. 6, pp. 4438–4454, Jun. 2016.
[16] M. Zeng, A. Yadav, O. A. Dobre, G. I. Tsiropoulos, and H. V. Poor,
“On the sum rate of MIMO-NOMA and MIMO-OMA systems,” IEEE
Wireless Commun. Lett., vol. 6, no. 4, pp. 534–537, Aug. 2017.
[17] M. Rihan, L. Huang, and P. Zhang, “Joint interference alignment and
power allocation for NOMA-based multi-user MIMO systems,” EURASIP
J. Wireless Commun. Netw., vol. 2018, no. 1, p. 217, Sep. 2018.
[18] C. Xiao et al., “Downlink MIMO-NOMA for ultra-reliable low-latency
communications,” IEEE J. Sel. Areas Commun., vol. 37, no. 4, pp. 780–
794, Feb. 2019.
[19] H. Wang, R. Zhang, R. Song, and S.-H. Leung, “A novel power
minimization precoding scheme for MIMO-NOMA uplink systems,”
IEEE Wireless Commun. Lett., vol. 22, no. 5, pp. 1106–1109, May 2018.
[20] R. Zhang, S.-H. Leung, H. Wang, W. Tang, and Z. Luo, “Power
minimization precoder design for uplink MIMO systems with multi-
group NOMA scheme,” IEEE Trans. Veh. Technol., vol. 70, no. 10, p.
10553–10569, Oct. 2021.
[21] Q. He, Y. Hu, and A. Schmeink, “Closed-form symbol error rate
expressions for non-orthogonal multiple access systems,” IEEE Trans.
Veh. Technol., vol. 68, no. 7, pp. 6775–6789, Jul. 2019.
[22] F. Kara and H. Kaya, “BER performances of downlink and uplink
NOMA in the presence of SIC errors over fading channels,” IET Com-
mun., vol. 12, no. 15, pp. 1834–1844, Sep. 2018.
[23] J.-M. Kang, I.-M. Kim, and C.-J. Chun, “Deep learning-based MIMO-
NOMA with imperfect SIC decoding,” IEEE Syst. J., vol. 14, no. 3, pp.
3414–3417, Sep. 2020.
[24] X. Chen, R. Jia, and D. W. K. Ng, “On the design of massive
non-orthogonal multiple access with imperfect successive interference
cancellation,” IEEE Trans. Commun., vol. 67, no. 3, pp. 2539–2551, Mar.
2019.
[25] X. Chen, Z. Zhang, C. Zhong, R. Jia, and D. W. K. Ng, “Fully non-
orthogonal communication for massive access,” IEEE Trans. Commun.,
vol. 66, no. 4, pp. 1717–1731, Apr. 2018.
[26] I. A. Mahady, E. Bedeer, S. Ikki, and H. Yanikomeroglu, “Sum-rate
maximization of NOMA systems under imperfect successive interference
cancellation,” IEEE Commun. Lett., vol. 23, no. 3, pp. 474–477, Mar.
2019.
[27] M. Zeng, W. Hao, O. A. Dobre, Z. Ding, and H. V. Poor, “Power
minimization for multi-cell uplink NOMA with imperfect SIC,” IEEE
Wireless Commun. Lett., vol. 9, no. 12, pp. 2030–2034, Dec. 2020.
[28] H. Zhang et al., “Energy efficient user clustering, hybrid precoding and
power optimization in terahertz MIMO-NOMA systems,” IEEE J. Sel.
Areas Commun., vol. 38, no. 9, pp. 2074–2085, Sep. 2020.
[29] R. Zhang, Precoder Design for MIMO Systems With User Cooperation
and Non-Orthogonal Multiple Access. PhD Thesis, City University of
Hong Kong, Hong Kong, 2020.
[30] C. Chen, W. Cai, X. Cheng, L. Yang, and Y. Jin, “Low complexity
beamforming and user selection schemes for 5G MIMO-NOMA systems,”
IEEE J. Sel. Areas Commun., vol. 35, no. 12, pp. 2708–2722, Dec. 2017.
[31] J. Cui, Z. Ding, and P. Fan, “Outage probability constrained MIMO-
NOMA designs under imperfect CSI,” IEEE Trans. Wireless Commun.,
vol. 17, no. 12, pp. 8239–8255, Dec. 2018.
[32] M. R. Zamani, M. Eslami, M. Khorramizadeh, and Z. Ding, “Energy-
efficient power allocation for NOMA with imperfect CSI,” IEEE Trans.
Veh. Technol., vol. 68, no. 1, pp. 1009–1013, Jan. 2019.
[33] Z. Li, M. Xia, M. Wen, and Y.-C. Wu, “Massive access in secure NOMA
under imperfect CSI: Security guaranteed sum-rate maximization with
first-order algorithm,” IEEE J. Sel. Areas Commun., vol. 39, no. 4, pp.
998–1014, Apr. 2021.
[34] Y. Xu, R. Q. Hu, and G. Li, “Robust energy-efficient maximization for
cognitive NOMA networks under channel uncertainties,” IEEE Internet
Things J., vol. 7, no. 9, pp. 8318–8330, Sep. 2020.
[35] K. Cho and D. Yoon, “On the general BER expression of one- and
two-dimensional amplitude modulations,” IEEE Trans. Commun., vol. 50,
no. 7, pp. 1074–1080, Jul. 2002.
[36] D. Shin, H. Cho, and K. Yang, “General closed-form expressions for
the SER and BER of decision-feedback detection with error propagation
over MIMO fading channels,” IEEE Trans. Veh. Technol., vol. 61, no. 2,
pp. 882–889, Feb. 2012.
[37] J. Choi, “Nulling and cancellation detector for MIMO and its application
to multistage receiver for coded signals: Performance and optimization,”
IEEE Trans. Wireless Commun., vol. 5, no. 5, pp. 1207–1216, May 2006.
[38] N. C. Beaulieu, “A useful integral for wireless communication theory
and its application to rectangular signaling constellation error rates,” IEEE
Trans. Commun., vol. 54, no. 5, pp. 802–805, May 2006.
[39] D. C. Lay, Linear Algebra and Its Applications. Boston, MA, U.S.A.:
Pearson Education, Inc., 2015.
[40] S. C. Chapra, Applied Numerical Methods With MATLAB for Engineers
and Scientists. New York, U.S.A.: The McGraw-Hill Companies, Inc.,
2012.
This article has been accepted for publication in IEEE Transactions on Vehicular Technology. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/TVT.2022.3225195
© 2022 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See https://www.ieee.org/publications/rights/index.html for more information.
Authorized licensed use limited to: Nanjing Univ of Post & Telecommunications. Downloaded on December 02,2022 at 08:07:19 UTC from IEEE Xplore. Restrictions apply.