ArticlePDF Available

Multi-Objective GAN-Based Adversarial Attack Technique for Modulation Classifiers

Authors:

Abstract and Figures

Deep learning is increasingly being used for many tasks in wireless communications, such as modulation classification. However, it has been shown to be vulnerable to adversarial attacks, which introduce specially crafted imperceptible perturbations, inducing models to make mistakes. This letter proposes an input-agnostic adversarial attack technique that is based on generative adversarial networks (GANs) and multi-task loss. Our results show that our technique reduces the accuracy of a modulation classifier more than a jamming attack and other adversarial attack techniques. Furthermore, it generates adversarial samples at least 335 times faster than the other techniques evaluated, which raises serious concerns about using deep learning-based modulation classifiers.
Content may be subject to copyright.
IEEE COMMUNICATIONS LETTERS, VOL. 26, NO. 7, JULY 2022 1583
Multi-Objective GAN-Based Adversarial Attack
Technique for Modulation Classifiers
Paulo Freitas de Araujo-Filho , Georges Kaddoum ,Senior Member, IEEE, Mohamed Naili,
Emmanuel Thepie Fapi , and Zhongwen Zhu ,Senior Member, IEEE
Abstract Deep learning is increasingly being used for many
tasks in wireless communications, such as modulation classifi-
cation. However, it has been shown to be vulnerable to adver-
sarial attacks, which introduce specially crafted imperceptible
perturbations, inducing models to make mistakes. This letter
proposes an input-agnostic adversarial attack technique that is
based on generative adversarial networks (GANs) and multi-task
loss. Our results show that our technique reduces the accuracy
of a modulation classifier more than a jamming attack and
other adversarial attack techniques. Furthermore, it generates
adversarial samples at least 335 times faster than the other
techniques evaluated, which raises serious concerns about using
deep learning-based modulation classifiers.
Index Terms—Adversarial attacks, wireless security, modula-
tion classification, deep learning, generative adversarial networks.
I. INTRODUCTION
DUE to its success in the most diverse fields, deep learn-
ing has been increasingly investigated and adopted in
wireless communications. It has been recently used for chan-
nel encoding and decoding [1], resource allocation [2], [3],
and automatic modulation classification (AMC) [4], [5]. More
specifically, deep learning-based modulation classifiers have
been replacing traditional AMC techniques because they
achieve better classification performance without requiring
manual feature engineering [6]–[8].
However, deep learning models have been shown to be
vulnerable to adversarial attacks, which puts into question the
security and reliability of wireless communication systems
that rely on such models [6], [9]–[12]. Adversarial attacks
introduce specially crafted imperceptible perturbations that
Manuscript received 22 February 2022; revised 24 March 2022; accepted
10 April 2022. Date of publication 13 April 2022; date of current version
12 July 2022. This work was funded and supported by Ericsson - Global
Artificial Intelligence Accelerator (GAIA) in Montreal, Mitacs Accelerate
Fellowship, the Tier 2 Canada Research Chair on the Next Generations of
Wireless IoT Networks, Fonds de recherche du Québec B2X Scholarship,
and CAPES. The associate editor coordinating the review of this letter
and approving it for publication was M. Wen. (Corresponding author:
Paulo Freitas De Araujo-Filho.)
Paulo Freitas de Araujo-Filho is with the Electrical Engineering Department,
École de Technologie Supérieure (ÉTS), University of Quebec, Montreal,
QC H3C 1K3, Canada, and also with the Centro de Informática, Universidade
Federal de Pernambuco (UFPE), Recife 50670-901, Brazil (e-mail: paulo.
freitas-de-araujo-filho.1@ens.etsmtl.ca).
Georges Kaddoum is with the Electrical Engineering Department,
École de Technologie Supérieure (ÉTS), University of Quebec, Montreal,
QC H3C 1K3, Canada (e-mail: georges.kaddoum@etsmtl.ca).
Mohamed Naili, Emmanuel Thepie Fapi, and Zhongwen Zhu are with
Ericsson GAIA Montreal, Montreal, QC H4S 0B6, Canada (e-mail:
mohamed.naili@ericsson.com; emmanuel.thepie.fapi@ericsson.com;
zhongwen.zhu@ericsson.com).
Digital Object Identifier 10.1109/LCOMM.2022.3167368
cause wrong classification results. Thus, they can force a
deep learning-based modulation classifier on a receiver to
misidentify the modulation mode used so that a signal is not
correctly demodulated and the communication compromised.
Adversarial attacks can be classified as white or black-
box attacks, depending on the knowledge they require from
their target models. White-box attacks require a complete
knowledge of the classifier’s model, such as training data,
architecture, learning algorithms, and hyper-parameters [13].
Black-box attacks, on the other hand, assume a more fea-
sible scenario in which the attacker has access to only the
model’s output [13]. Furthermore, the authors of [14] define
three more restrictive and realistic black-box threat models:
query-limited, partial-information, and decision-based. The
query-limited scenario considers that attackers have access to
only a limited number of the model’s outputs. The partial-
information scenario considers that attackers have access
to only the probabilities of some of the model’s classes.
Finally, the decision-based scenario considers that attackers
have access to only the model’s decision, i.e., the class to
which it assigns a given data sample.
Although existing adversarial attacks pose risks to the use
of deep learning in wireless communications, they require a
complete knowledge about the target model [7], [15] or take
too long to craft adversarial perturbations [11], [16], [17].
In this letter, we propose a novel input-agnostic decision-based
adversarial attack technique that reduces the accuracy of mod-
ulation classifiers more and crafts perturbations significantly
faster than existing techniques. Our technique is necessary
for assessing the risks of using deep learning-based AMC
in the more realistic scenario of decision-based black-box
attacks. Moreover, it can significantly contribute to developing
classifiers that are robust against adversarial attacks. The main
contributions of our work are as follows: First, we combine
generative adversarial networks (GANs) [18] and multi-task
loss [19] to generate adversarial samples, by simultaneously
optimizing their ability to cause wrong classifications and not
being perceived. Second, we reduce the accuracy of modula-
tion classifiers more and craft adversarial samples in a shorter
time than existing techniques while following the decision-
based black-box scenario. Third, we propose an input-agnostic
adversarial attack technique that does not depend on the
original samples to craft perturbations. It allows adversarial
perturbations to be prepared in advance, further reducing the
time for executing the adversarial attack. Finally, our work
verifies that modulation classifiers are at an increased risk and
urgently need to be enhanced against adversarial attacks.
1558-2558 © 2022 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See https://www.ieee.org/publications/rights/index.html for more information.
Authorized licensed use limited to: Ericsson. Downloaded on January 31,2023 at 12:17:36 UTC from IEEE Xplore. Restrictions apply.
1584 IEEE COMMUNICATIONS LETTERS, VOL. 26, NO. 7, JULY 2022
II. RELATED WORKS
Although adversarial attacks were initially explored in com-
puter vision applications, they have recently been investigated
for wireless communication applications, such as AMC. The
authors of [7] and [15] evaluate the robustness of a modulation
classifier against four white-box adversarial attack techniques:
fast gradient sign method (FGSM), projected gradient descent
(PGD), basic iterative method (BIM), and momentum iterative
method (MIM). The works show that the classifier’s accuracy
is significantly compromised. However, they do not measure
the extent of the perturbation or the time it takes to craft
adversarial samples. The work in [10] extends the white-
box techniques FGSM, momentum iterative fast gradient sign
method (MI-FGSM), and PGD to a power allocation applica-
tion. It shows that adversarial attacks also pose a significant
risk to regression-based applications, such as power allocation.
Several other works focus on black-box attacks, as they are
more realistic for not requiring complete knowledge about the
model [13]. The authors of [16] propose a boundary attack
technique that requires access to only the classifier’s decision.
It relies on a probabilistic distribution to iteratively craft
adversarial samples and reduce their distance to the original
sample. Although it compromises the accuracy of classifiers,
it takes more than a minute to craft a single adversarial sample.
The authors of [17] propose an iterative algorithm to produce
universal perturbations and show that state-of-the-art image
classification neural networks are highly vulnerable. However,
it takes more than 20 seconds to craft each adversarial sample.
The authors of [11] propose an algorithm to craft adversarial
attacks that is shown to require significantly less power than
conventional jamming attacks to compromise the performance
of a modulation classifier. Although the algorithm reduces the
craft time of adversarial perturbations, it still requires hundreds
of milliseconds to craft each adversarial sample.
III. ADVERSARIAL ATTACKS FORMULATION
Although deep learning models may be trained with a
large amount of data, it is impractical to train them to cover
all possible input feature vectors. As a result, the decision
boundary found by a trained model may differ from the real
one. The discrepancy creates room for a trained model to make
mistakes [7]. Adversarial attacks craft perturbations to corrupt
data samples so that they fall within that discrepancy area
and are misclassified by a trained model. However, this is not
a trivial task as the perturbations must be large enough to
cause misclassifications but small enough to not be perceived.
Therefore, given a sample x, the goal of an adversarial attacker
is to find a perturbation δand construct an adversarial sample
xadv =x+δwhile satisfying
min ||xadv x|| (1)
and
f(xadv )=f(x),(2)
where ||·|| represents a chosen distance metric, ρis the
maximum imperceptible perturbation according to that metric,
and fis the trained classifier target of the attack.
IV. PROPOSED ADVERSARIAL ATTACK TECHNIQUE
In our work, we consider that our proposed adversarial
attack technique is deployed as a malicious software on
Fig. 1. Our attack model considers the adversarial attacker as malicious
software on the wireless receiver.
software-defined wireless receivers, an essential piece of mod-
ern wireless communication and 5/6G. Although injecting such
malicious software is out of the scope of our work, it may be
done by infecting software-defined radios with malware [20].
The malware can send samples to the receiver’s modulation
classifier and has access to its decisions. It intercepts incoming
signals, craft perturbations δ, add the perturbations to original
samples to form adversarial samples xadv =x+δ, and forward
adversarial samples xadv to the modulation classifier. Thus,
the receiver’s modulation classifier fidentifies the modulation
mode of xas f(xadv ).Sincef(xadv)=f(x), the signal is
not correctly demodulated, and the communication is compro-
mised. Figure 1 shows our attack model. The analog-to-digital
converter (ADC) forwards clean samples to the modulation
classifier, but they are tampered by the adversarial attacker.
We propose a novel multi-objective adversarial attack tech-
nique by combining a GAN and multi-task loss. GANs
estimate generative models by simultaneously training two
competing neural networks: generator and discriminator [21].
The generator learns the probabilistic distribution of training
data, and the discriminator learns how to distinguish between
real data and data produced by the generator. We train a
GAN so that its generator produces adversarial perturbations
δ=G(z)from random latent vectors zand its discriminator
learns to distinguish between clean samples xand adversarial
samples xadv =x+G(z). We adopt the Wasserstein GAN
(WGAN), which minimizes the Wasserstein distance between
two probability distributions. It is easier to train than the
original GAN, and does not suffer from the gradient vanishing
problem [22], [23]. Although other GAN formulations, such as
WGAN Gradient Penalty (WGAN-GP) [24], try to overcome
WGAN’s difficulty in enforcing the Lipschitz constant, the
work in [25] shows that WGAN-GP does not necessarily
outperform WGAN. In future work, we will evaluate our
technique with other GAN formulations, such as WGAN-GP.
The WGAN discriminator estimates the Wasserstein dis-
tance by maximizing the difference between average critic
score on real and fake samples. Besides, since we want
the generator to produce perturbations rather than adversarial
samples, fake samples are designated as x+G(z)instead
of G(z). Thus, we minimize the discriminator loss given by
LD=D(x+G(z)) D(x). On the other hand, the WGAN
generator has the opposite goal of maximizing the average
critic score on fake samples. Hence, we minimize the generator
loss given by LG=D(x+G(z)).However,suchaLGonly
accounts for minimizing the difference between xand xadv,
which corresponds to the condition of equation (1). It does
not consider the condition of equation (2), which is to ensure
that xand xadv are assigned to different classes.
To ensure that our GAN considers the conditions of both
equation (1) and equation (2), we modify the generator’s loss
to simultaneously optimize two objective functions that are
Authorized licensed use limited to: Ericsson. Downloaded on January 31,2023 at 12:17:36 UTC from IEEE Xplore. Restrictions apply.
FREITAS DE ARAUJO-FILHO et al.: MULTI-OBJECTIVE GAN-BASED ADVERSARIAL ATTACK TECHNIQUE 1585
Fig. 2. Our proposed training model.
given by LG1and LG2.LG1represents the task of minimizing
the difference between xand xadv and is given by the original
generator loss, hence LG1=D(x+G(z)).LG2represents
the task of ensuring that xand xadv are assigned to different
classes. It is given by the cross entropy loss between the
class fassigns to xadv and the label of x, hence LG2=
CE(f(x+G(z)),y),whereCE stands for the cross entropy
loss largely adopted in classification problems and yis the
label of x. During training, our technique leverages its access
to the classifier’s decisions to simultaneously optimize its
ability to cause wrong classifications and not being perceived.
While most works that simultaneously learn multiple tasks
manually tune a weighted sum of losses, we leverage the multi-
task loss proposed in [19]. That work uses aleatoric uncer-
tainty, which is a quantity that stays constant for all input data
and varies between different tasks, to simultaneously optimize
any two losses by optimally balancing their contributions as
L=1
2σ2
1
L1+1
2σ2
2
L2+logσ1σ2,(3)
where L1and L2are any two losses, and σ1and σ2are
learnable weights automatically tuned when training a neural
network. Thus, while we train the GAN discriminator with
LD=D(x+G(z)) D(x),(4)
we combine LG1and LG2with equation (3), where L1=LG1
and L2=LG2, so that our generator loss becomes
LG=D(x+G(z))
2σ2
1
+CE(f(x+G(z)),y)
2σ2
2
+logσ1σ2.
(5)
Figure 2 shows the training model, and Algorithm 1 shows the
execution steps of our proposed adversarial attack technique.
Algorithm 1 Proposed Adversarial Attack Technique
1: Train a GAN according to equations (4) and (5)
2: for Each incoming sample xdo
3: Compute G(z)
4: Construct the adversarial sample xadv =x+G(z)
5: end for
V. METHODOLOGY AND EXPERIMENTAL EVA L UA T I O N
We use the RADIOML 2016.10A dataset and VT-CNN2
modulation classifier designed by DeepSiG and publicly avail-
able in [4] and [26] to evaluate our proposed adversarial
attack technique. The dataset is constructed by modulating and
exposing signals to an additive white Gaussian noise (AWGN)
Fig. 3. VT-CNN2 neural network architecture.
Fig. 4. GAN generator architecture.
channel that includes sampling rate offset, random process
of center frequency offset, multipath, and fading effects,
as described in [4] and [26]. Since our technique crafts
adversarial samples on receivers, it is not subject to channel
effects. In future work, we will consider them to enhance our
proposed technique so that it sends adversarial samples over
the air.
After modulation and channel modeling, the signals are
normalized and packaged into 220,000 samples of in-phase
and quadrature components with length 128, each associated
with a modulation scheme and a signal-to-noise ratio (SNR).
SNR is a measure of a signal’s strength. It is the ratio between
the power of the signal and of the background noise, i.e.,
SNR[dB]=10log(
Psignal
Pnoise ),wherePis the signal power.
Eleven different modulation schemes (eight digital and three
analog) are possible: 8PSK, BPSK, QPSK, QAM16, QAM64,
CPFSK, GFSK, PAM4, WBFM, AM-DSB, and AM-SSB.
Twenty different SNRs, ranging from 20 dB to 18 dB in
steps of 2 dB, are possible. Twenty percent of the samples are
reserved as a testing set to measure the VT-CNN2 modulation
classifier’s accuracy on clean and adversarial samples.
The VT-CNN2 modulation classifier relies on deep con-
volutional neural networks and classifies samples among the
eleven modulation schemes in the dataset. Figure 3 shows
VT-CNN2’s architecture. Although the softmax layer gives the
probability of membership for each class, we consider the clas-
sifier’s output to be only its final decision, i.e., the modulation
class that has the highest probability. Thus, f(x+G(z)) is the
predicted label of one of the modulation schemes considered.
Finally, Figures 4 and 5 show the GAN’s generator and
discriminator architectures. They were optimized using the
Optuna framework [27], which automatically searches for the
optimal hyper-parameters, and the early stopping mechanism
to avoid overfitting. Table I shows the hyper-parameter values
used in the GAN after tuning. All experiments were conducted
using an AMD Ryzen Threadripper 1920X 12-core 2.2GHz
processor with 64GB of RAM and an NVIDIA GeForce RTX
2080 in a Pytorch environment.
Authorized licensed use limited to: Ericsson. Downloaded on January 31,2023 at 12:17:36 UTC from IEEE Xplore. Restrictions apply.
1586 IEEE COMMUNICATIONS LETTERS, VOL. 26, NO. 7, JULY 2022
Fig. 5. GAN discriminator architecture.
TAB L E I
HYPER-PARAMETERSVALUES
VI. RESULTS AND DISCUSSION
As previously mentioned, the goal of adversarial attacks
is to introduce imperceptible perturbations capable of reduc-
ing the accuracy of a modulation classifier. Therefore,
we evaluated our proposed attack technique by measuring
the VT-CNN2’s accuracy on clean and adversarial samples,
and the perturbation-to-noise ratio (PNR). PNR measures the
ratio between the perturbation and noise power levels so
that PNR
[dB]=10log(
Pperturbation
Pnoise ),wherePis the signal
power. The larger the PNR, the larger the perturbation is
in comparison to the noise, becoming more distinguishable
and more likely to be detected. Perturbations are considered
imperceptible when they are in the same order as or below the
noise level, i.e., PNR <0dB.
Figure 6 shows the VT-CNN2’s accuracy versus PNR for
SNRs of 10, 0, and 10 dB. Without attacks, the classifier
achieves different accuracy depending on the SNR because
larger noises make it harder for the classifier to achieve correct
results. Under our proposed adversarial attack, the classifier’s
accuracy is significantly reduced in all cases. At 0 dB PNR,
our technique reduces the accuracy by 37% for 10 dB SNR,
56% for 0 dB SNR, and 7% for 10 dB SNR. Our technique
reduces the accuracy more for 0 dB than for 10 dB SNR
because, for signals with the same strength, larger SNRs mean
lower noise levels so that it is more challenging to produce
imperceptible perturbations that still significantly compromise
the accuracy. However, although the noise at 10 dB SNR is
the highest, allowing our technique to produce larger pertur-
bations, the accuracy reduction is not as significant as at 0 dB
SNRor10dBSNR.Iff(x+G(z)) in equation (5) gives too
many wrong results regardless of the adversarial perturbation
G(z), it is harder for our technique to find what perturbation
would reduce the classifier’s accuracy the most. Thus, the
fact that our technique relies on the classifier’s decisions
to train the GAN diminishes its capacity to produce wrong
classifications when the classifier’s accuracy is low. Since the
classifier’s accuracy is around only 22% at 10 dB SNR, the
adversarial perturbations that our proposed technique crafts are
less effective. Nevertheless, our proposed adversarial attack
technique still significantly reduces the classifier’s accuracy.
Fig. 6. Modulation classifier’s accuracy versus PNR with and without our
proposed adversarial attack technique.
Fig. 7. Waveform comparison of a 8PSK signal with SNR=10 dB before
(clean sample) and after (adversarial sample) our proposed adversarial attack.
We further examine the influence of perturbations on signal
waveforms. We verify that the signal waveform after per-
turbation (adversarial sample) is consistent with the original
waveform (clean sample), i.e., amplitude, frequency, and phase
do not significantly change. Thus, while our technique’s per-
turbations mislead the classifier, they are not easily recognized
by human eyes. Figure 7 illustrates the time domain waveform
of an 8PSK signal before and after the perturbation is intro-
duced. Similar results were achieved for the other modulation
schemes considered, such that clean and adversarial sam-
ples always have very similar waveforms without significant
changes in their amplitude, frequency, and phase.
Moreover, we compare our results to those of a jamming
attack, which adds Gaussian noise to signals, and two other
adversarial attack techniques: those proposed in [17] and [11].
Figure 8 shows the VT-CNN2’s accuracy on clean samples
and adversarial samples produced by the jamming attack
and the three adversarial attack techniques evaluated for
SNR =10 dB. Perturbations introduced by adversarial attacks
are specially crafted to reduce the classifier’s accuracy the
most while not being perceived. Thus, our technique and the
techniques from [17] and [11] are significantly more harmful
than attacks that introduce random noises, such as the jamming
attack. Moreover, our proposed attack technique is the one that
reduces the accuracy the most.
Finally, we evaluate how long it takes for each technique to
craft adversarial samples. Table II shows the mean execution
Authorized licensed use limited to: Ericsson. Downloaded on January 31,2023 at 12:17:36 UTC from IEEE Xplore. Restrictions apply.
FREITAS DE ARAUJO-FILHO et al.: MULTI-OBJECTIVE GAN-BASED ADVERSARIAL ATTACK TECHNIQUE 1587
Fig. 8. Modulation classifier’s accuracy versus PNR without and subject to
different adversarial attack techniques.
TAB L E II
MEAN EXECUTION TIME FOR CRAFTING ADVER SARIAL SAMPLES
time for crafting adversarial samples. Our proposed technique
achieves significantly shorter times than the other two tech-
niques by crafting adversarial samples in less than 0.7 ms.
Thus, it is more than 335 times faster than the second-fastest
attack technique. Techniques that take too long to craft pertur-
bations might be too late so that the signals they aim to perturb
have already been correctly demodulated. Thus, such a time
reduction is essential to compromise fast modulation classifiers
and is a great advantage of our technique. Moreover, since
our technique is input-agnostic, it can prepare perturbations
in advance and just add them to incoming signals. Therefore,
our proposed technique represents a severe risk to using deep
learning-based modulation classifiers.
VII. CONCLUSION
In this letter, we verified that deep learning is exposed
to security risks that must be considered despite its advan-
tages. Our results showed that it is possible to quickly craft
small imperceptible perturbations that completely compromise
modulation classifiers’ accuracy and hence wireless receivers’
performance. Therefore, it is urgently necessary to enhance
deep learning-based modulation classifiers’ robustness against
adversarial attacks. As future work, we will evaluate the use
of other GAN formulations, such as WGAN-GP, modify our
attack model to consider adversarial attacks transmitted over
the air, and investigate adversarial attack defense strategies.
REFERENCES
[1] F. Liang, C. Shen, and F. Wu, An iterative BP-CNN architecture for
channel decoding,” IEEE J. Sel. Topics Signal Process., vol. 12, no. 1,
pp. 144–159, Feb. 2018.
[2] L. Sanguinetti, A. Zappone, and M. Debbah, “Deep learning power
allocation in massive MIMO,” in Proc. 52nd Asilomar Conf. Signals,
Syst., Comput., Oct. 2018, pp. 1257–1261.
[3] H. Sun, X. Chen, Q. Shi, M. Hong, X. Fu, and N. D. Sidiropoulos,
“Learning to optimize: Training deep neural networks for wireless
resource management,” in Proc. IEEE 18th Int. Workshop Signal
Process. Adv. Wireless Commun. (SPAWC), Jul. 2017, pp. 1–6.
[4] T. J. O’Shea, J. Corgan, and T. C. Clancy, “Convolutional radio modula-
tion recognition networks,” in Proc. Int. Conf. Eng. Appl. Neural Netw.
Aberdeen, U.K., Springer, 2016, pp. 213–226.
[5] T. J. O’Shea, T. Roy, and T. C. Clancy, “Over-the-air deep learning
based radio signal classification,” IEEE J. Sel. Topics Signal Process.,
vol. 12, no. 1, pp. 168–179, Feb. 2018.
[6] B. Flowers, R. M. Buehrer, and W. C. Headley, “Evaluating adversarial
evasion attacks in the context of wireless communications,” IEEE Trans.
Inf. Forensics Security, vol. 15, pp. 1102–1113, 2020.
[7] Y. Lin, H. Zhao, X. Ma, Y. Tu, and M. Wang, Adversarial attacks
in modulation recognition with convolutional neural networks,” IEEE
Trans. Rel., vol. 70, no. 1, pp. 389–401, Mar. 2021.
[8] R. Sahay, C. G. Brinton, and D. J. Love, “A deep ensemble-based wire-
less receiver architecture for mitigating adversarial attacks in automatic
modulation classification,” 2021, arXiv:2104.03494.
[9] Y. Lin, H. Zhao, Y. Tu, S. Mao, and Z. Dou, “Threats of adversarial
attacks in DNN-based modulation recognition,” in Proc. IEEE Conf.
Comput. Commun. (INFOCOM), Jul. 2020, pp. 2469–2478.
[10] B. R. Manoj, M. Sadeghi, and E. G. Larsson, “Adversarial attacks on
deep learning based power allocation in a massive MIMO network,”
2021, arXiv:2101.12090.
[11] M. Sadeghi and E. G. Larsson, “Adversarial attacks on deep-learning
based radio signal classification,” IEEE Wireless Commun. Lett.,vol.8,
no. 1, pp. 213–216, Feb. 2019.
[12] O. Ibitoye, R. Abou-Khamis, A. Matrawy, and M. Omair Shafiq, “The
threat of adversarial attacks on machine learning in network security—A
survey, 2019, arXiv:1911.02621.
[13] X. Yuan, P. He, Q. Zhu, and X. Li, “Adversarial examples: Attacks
and defenses for deep learning,” IEEE Trans. Neural Netw. Learn. Syst.,
vol. 30, no. 9, pp. 2805–2824, Sep. 2019.
[14] A. Ilyas, L. Engstrom, A. Athalye, and J. Lin, “Black-box adversarial
attacks with limited queries and information,” in Proc. Int. Conf. Mach.
Learn., 2018, pp. 2137–2146.
[15] H. Zhao, Y. Lin, S. Gao, and S. Yu, “Evaluating and improving
adversarial attacks on DNN-based modulation recognition,” in Proc.
IEEE Global Commun. Conf. (GLOBECOM), Dec. 2020, pp. 1–5.
[16] W. Brendel, J. Rauber, and M. Bethge, “Decision-based adversarial
attacks: Reliable attacks against black-box machine learning models,”
2017, arXiv:1712.04248.
[17] S.-M. Moosavi-Dezfooli, A. Fawzi, O. Fawzi, and P. Frossard, “Univer-
sal adversarial perturbations,” in Proc. IEEE Conf. Comput. Vis. Pattern
Recognit. (CVPR), Jul. 2017, pp. 1765–1773.
[18] I. Goodfellow et al., “Generative adversarial nets,” in Proc. Adv. Neural
Inf. Process. Syst., vol. 27, 2014, pp. 1–9.
[19] R. Cipolla, Y. Gal, and A. Kendall, “Multi-task learning using
uncertainty to weigh losses for scene geometry and semantics,” in
Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Jun. 2018,
pp. 7482–7491.
[20] K. Li et al., “Security mechanisms to defend against new attacks on
software-defined radio,” in Proc. Int. Conf. Comput., Netw. Commun.
(ICNC), Mar. 2018, pp. 537–541.
[21] P. F. de Araujo-Filho, G. Kaddoum, D. R. Campelo, A. G. Santos,
D. Macêdo, and C. Zanchettin, “Intrusion detection for cyber–physical
systems using generative adversarial networks in fog environment,”
IEEE Internet Things J., vol. 8, no. 8, pp. 6247–6256, Sep. 2021.
[22] M. Arjovsky, S. Chintala, and L. Bottou, “Wasserstein generative adver-
sarial networks,” in Proc. Int. Conf. Mach. Learn., 2017, pp. 214–223.
[23] A. Creswell, T. White, V. Dumoulin, K. Arulkumaran, B. Sengupta, and
A. A. Bharath, “Generative adversarial networks: An overview, IEEE
Signal Process., vol. 35, no. 1, pp. 53–65, Jan. 2017.
[24] I. Gulrajani, F. Ahmed, M. Arjovsky, V. Dumoulin, and A. Courville,
“Improved training of Wasserstein GANs, 2017, arXiv:1704.00028.
[25] M. Lucic, K. Kurach, M. Michalski, S. Gelly, and O. Bousquet, “Are
GANs created equal? A large-scale study, 2017, arXiv:1711.10337.
[26] T. J. O’Shea and N. West, “Radio machine learning dataset generation
with GNU radio,” in Proc. 6th GNU Radio Conf., 2016, pp. 1–6.
[27] T. Akiba, S. Sano, T. Yanase, T. Ohta, and M. Koyama, “Optuna: A
next-generation hyperparameter optimization framework,” in Proc. 25th
ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining, Jul. 2019,
pp. 2623–2631.
Authorized licensed use limited to: Ericsson. Downloaded on January 31,2023 at 12:17:36 UTC from IEEE Xplore. Restrictions apply.
... Similarly, the work presented in [75] shows that adversarial attacks dramatically raise the bit error rate (BER) of DNN-based detectors, which are trained to recover the payload bits straight from received symbols. Furthermore, GANs can also be adapted to generate fake RF fingerprints to carry out wireless signal spoofing [76,77] or attack modulation classifiers in wireless communications [78][79][80]. Adversarial attacks can achieve an astounding attack success rate of over 90% in CSI-based Wi-Fi fingerprinting, with small perturbation-to-signal ratios [81]. ...
Article
Full-text available
The Internet of Things (IoT) paradigm and the advanced wireless technologies of 5G and beyond are expected to enable diverse applications such as autonomous driving, industrial automation, and smart cities. These applications bring together a vast and diverse IoT device population that occupy radio frequency spectrum. Such a large number of wireless devices expose previously unheard-of threat surfaces in addition to the bandwidth shortage and throughput issues. Device identification is crucial in such scenarios not only to authenticate and authorize nodes, but also to employ different network services. One of the promising solutions for device identification is the use of radio frequency (RF) fingerprinting. Recently, wireless device identification using RF fingerprinting along with machine learning and deep learning technologies showed outstanding results in the recent contemporary domains. This paper presents a systematic literature review of RF fingerprinting identification of wireless devices by presenting the results as a graphical and tabular representation of statistical data obtained. Only experimental research papers were considered of over 130 journals and international conference papers that have been classified and evaluated from the year 2010 till date. This survey focuses on exploring the commonly used RF fingerprinting approaches, feature extraction and filtration techniques, and classification algorithm used in the device identification. Finally, open issues and challenges along with future directions have presented which were discovered during the process of analyzing the literature.
... For example, Sadeghi et al. [6] investigated adversarial attacks in DL-based radio modulation classification tasks and demonstrated that adversarial attacks are superior to classical jamming attacks. Following this, Freitas de Araujo-Filho et al. [10] crafts adversarial attacks by generative adversarial networks (GANs), Kim et al. [7] developed channel-aware adversarial attacks against DL-based wireless signal classifiers. Moreover, Sadeghi et al. [11] evaluated the performance of physical black-box adversarial attacks against end-to-end autoencoder communication systems. ...
Article
Full-text available
Most existing adversarial attack methods generally rely on ideal assumptions, which is unreasonable for practical applications. In this paper, a practical threat model which utilizes adversarial attacks for anti-eavesdropping is proposed and a physical intra-class universal adversarial perturbation (IC-UAP) crafting method against DL-based wireless signal classifiers is then presented. First, an IC-UAP algorithm is proposed based on the threat model to craft a stronger UAP attack against the samples in a given class from a batch of samples in the class. Then, we develop a physical attack algorithm based on the IC-UAP method, in which perturbations are optimized under random shifting to enhance the robustness of IC-UAPs against the unsynchronization between adversarial attacks and attacked signals. Finally, the numerical results corroborate the effectiveness of the proposed approach based on the benchmark dataset.
Article
The recent evolution of generative artificial intelligence (GAI) leads to the emergence of groundbreaking applications such as ChatGPT, which not only enhances the efficiency of digital content production, such as text, audio, video, or even network traffic data, but also enriches its diversity. Beyond digital content creation, GAI’s capability in analyzing complex data distributions offers great potential for wireless communications, particularly amidst a rapid expansion of new physical layer communication technologies. For example, the diffusion model can learn input signal distributions and use them to improve the channel estimation accuracy, while the variational autoencoder can model channel distribution and infer latent variables for blind channel equalization. Therefore, this paper presents a comprehensive investigation of GAI’s applications for communications at the physical layer, ranging from traditional issues, including signal classification, channel estimation, and equalization, to emerging topics, such as intelligent reflecting surfaces and joint source channel coding. We also compare GAI-enabled physical layer communications with those supported by traditional AI, highlighting GAI’s inherent capabilities and unique contributions in these areas. Finally, the paper discusses open issues and proposes several future research directions, laying a foundation for further exploration and advancement of GAI in physical layer communications.
Article
Due to the openness of wireless channels, wireless communication is vulnerable to be eavesdropped, which results in confidential information leakage. Physical Layer security (PLS) technology provides a new way to solve this hidden danger of Internet of Things system. However, traditional PLS methods are often restricted by limited communication resources and unknown instantaneous channel state information of eavesdroppers, which makes it challenging to strike a balance between security and reliability in the communication system. Therefore, an adversarial defense embedded waveform design (ADEWD) method for physical layer reliable communication (PLRC) is proposed in this paper. Firstly, we use generative adversarial networks to generate amplitude controllable adversarial perturbation, and then superimpose it with original communication signal to form an adversarial signal. At the same time, we also design a demodulation network based on the modulation type of legitimate users to constrain the amplitude of the generated perturbations, to reduce the bit error rate (BER) loss after demodulation of the adversarial signal. With this waveform design, the adversarial signal not only enables reliable communication between legitimate users, but also utilizes embedded defense traps to prevent eavesdroppers from recognizing legitimate users. The experimental results demonstrate that our ADEWD method for PLRC has stronger defense capability and lower BER in both white-box and black-box scenarios, which reflects the defense robustness and communication reliability of the proposed waveform design method.
Article
Deep learning has been adopted for a wide range of wireless communication tasks, including modulation classification, because of its great classification capability. However, deep learning models have been shown to also introduce risks and vulnerabilities. For instance, adversarial attacks craft and introduce imperceptible perturbations that compromise the accuracy of deep learning-based modulation classifiers on wireless receivers. Therefore, in this article, we propose a novel wireless receiver architecture that enhances deep learning-based modulation classifiers to defend them against adversarial attacks. Our experimental results show that our defense technique significantly diminishes the accuracy reduction that is caused by adversarial attacks by protecting modulation classifiers at least 18% more than existing defense techniques.
Article
Internet of Things (IoT) based on cognitive radio (CR) exhibits strong dynamic sensing and intelligent decision-making capabilities by effectively utilizing spectrum resources. The federal learning (FL) framework based modulation recognition (MR) is an essential component, but its use of uninterpretable deep learning (DL) introduces security risks. This paper combines traditional signal interference methods and data poisoning in FL to propose a new adversarial attack approach. The poisoning attack in distributed frameworks manipulates the global model by controlling malicious users, which is not only covert but also highly impactful. The carefully designed pseudo-noise in MR is also extremely difficult to detect. The combination of these two techniques can generate a greater security threat. We have further advanced our proposal with the introduction of the new adversarial attack method called "Chaotic Poisoning Attack" to reduce the recognition accuracy of the FL-based MR system. We establish effective attack conditions, and simulation results demonstrate that our method can cause a decrease of approximately 80% in the accuracy of the local model under weak perturbations and a decrease of around 20% in the accuracy of the global model. Compared to white-box attack methods, our method exhibits superior performance and transferability.
Article
Adversarial attacks and defenses in machine learning and deep neural network (DNN) have been gaining significant attention due to the rapidly growing applications of deep learning in communication networks. This survey provides a comprehensive overview of the recent advancements in the field of adversarial attack and defense techniques, with a focus on DNN-based classification models for communication applications. Specifically, we conduct a comprehensive classification of recent adversarial attack methods and state-of-the-art adversarial defense techniques based on attack principles, and present them in visually appealing tables and tree diagrams. This is based on a rigorous evaluation of the existing works, including an analysis of their strengths and limitations. We also categorize the methods into counter-attack detection and robustness enhancement, with a specific focus on regularizationbased methods for enhancing robustness. New avenues of attack are also explored, including search-based, decision-based, dropbased, and physical-world attacks, and a hierarchical classification of the latest defense methods is provided, highlighting the challenges of balancing training costs with performance, maintaining clean accuracy, overcoming the effect of gradient masking, and ensuring method transferability. At last, the lessons learned and open challenges are summarized with future research opportunities recommended.
Article
Fifth-generation (5G) networks provide connectivity to a massive number of devices and boost a plethora of applications in several different domains. However, the large adoption of connected devices increases attack surfaces and introduces several security threats that can severely damage physical objects and risk people’s lives. Despite existing intrusion detection systems (IDSs), there are still several challenges to be addressed in the detection of cyber-attacks. For instance, while unsupervised IDSs are required to detect zero-day attacks, they usually present high false positive rates. Moreover, most existing IDSs rely on long short-term memory (LSTM) networks to consider time-dependencies among data. However, LSTM networks have recently been shown to present several drawbacks and limitations, which put into question their performance on sequence modeling tasks. Thus, in this paper, we investigate generative adversarial networks (GANs), a promising unsupervised approach to detecting attacks by implicitly modeling systems, and alternatives to LSTM networks to consider temporal dependencies among data. We propose a novel unsupervised GAN-based IDS that uses temporal convolutional networks (TCNs) and self-attention to detect cyber-attacks. The proposed IDS leverages edge computing and is proposed for edge servers, which bring computation resources closer to end nodes. Experiment results show that our proposed IDS can be configured to satisfy different detection rate and detection time requirements. Moreover, they show that our IDS is more accurate and at least 3.8 times faster than two state-of-the-art GAN-based IDSs that are used as baselines.
Article
In spite of unique advantages like higher recognition accuracy and better generalization capability, Automatic Modulation Classification (AMC) oriented Deep Neural Networks (ADNNs) are still vulnerable to adversarial examples (AEs). Recent results revealed that an attacker can easily fool ADNNs through adding a small and imperceptible perturbation to the original signal. Among different AE generation methods, Universal Adversarial Perturbation (UAP) has unique characteristics including input-agnostic and shift-invariance. However, applying UAP directly to RF signals faces three main challenges, i.e., perturbation neutralization, high perceptibility, and dependency of original signals. In this backdrop, a novel Universal Adversarial Perturbation under Frequency and Data constraints (UAP-FD) attack is put forward for solving these problems in this paper. First, an individual perturbation is filtered based on the representation visualization algorithm to counter the neutralization problem in perturbation integration. Second, the high-frequency components in the integrated UAP is eliminated through signal decomposition and reconstruction for promoting the imperceptibility. Third, a proxy signal generation method is proposed to help UAP-FD adapt to data-free black-box settings. A series of experiments is conducted to evaluate the aggressiveness and imperceptibility of UAP-FD attack in different settings on a public dataset. Results show that, compared with existing proposal, UAP-FD has a 40% higher fooling rate, and it can reduce the accuracy of the ADNN model from 83% to 9%, while maintaining a good imperceptibility and shift-invariance property. In addition, UAP-FD is applied to real world captured signals over the transmission channel; and it can reduce the model accuracy from 98.3% to 12.5%.
Article
Full-text available
With rapid progress and significant successes in a wide spectrum of applications, deep learning is being applied in many safety-critical environments. However, deep neural networks (DNNs) have been recently found vulnerable to well-designed input samples called adversarial examples. Adversarial perturbations are imperceptible to human but can easily fool DNNs in the testing/deploying stage. The vulnerability to adversarial examples becomes one of the major risks for applying DNNs in safety-critical environments. Therefore, attacks and defenses on adversarial examples draw great attention. In this paper, we review recent findings on adversarial examples for DNNs, summarize the methods for generating adversarial examples, and propose a taxonomy of these methods. Under the taxonomy, applications for adversarial examples are investigated. We further elaborate on countermeasures for adversarial examples. In addition, three major challenges in adversarial examples and the potential solutions are discussed.
Article
Deep learning-based automatic modulation classification (AMC) models are susceptible to adversarial attacks. Such attacks inject specifically crafted wireless interference into transmitted signals to induce erroneous classification predictions. Furthermore, adversarial interference is transferable in black box environments, allowing an adversary to attack multiple deep learning models with a single perturbation crafted for a particular classification model. In this work, we propose a novel wireless receiver architecture to mitigate the effects of adversarial interference in various black box attack environments. We begin by evaluating the architecture uncertainty environment, where we show that adversarial attacks crafted to fool specific AMC DL architectures are not directly transferable to different DL architectures. Next, we consider the domain uncertainty environment, where we show that adversarial attacks crafted on time domain and frequency domain features to not directly transfer to the altering domain. Using these insights, we develop our Assorted Deep Ensemble (ADE) defense, which is an ensemble of deep learning architectures trained on time and frequency domain representations of received signals. Through evaluation on two wireless signal datasets under different sources of uncertainty, we demonstrate that our ADE obtains substantial improvements in AMC classification performance compared with baseline defenses across different adversarial attacks and potencies.
Article
Deep learning (DL) models are vulnerable to adversarial attacks, by adding a subtle perturbation which is imperceptible to the human eye, a convolutional neural network (CNN) can lead to erroneous results, which greatly reduces the reliability and security of the DL tasks. Considering the wide application of modulation recognition in the communication field and the rapid development of DL, by adding a well-designed adversarial perturbation to the input signal, this article explores the performance of attack methods on modulation recognition, measures the effectiveness of adversarial attacks on signals, and provides the empirical evaluation of the reliabilities of CNNs. The results indicate that the accuracy of the target model reduce significantly by adversarial attacks, when the perturbation factor is 0.001, the accuracy of the model could drop by about 50% on average. Among them, iterative methods show greater attack performances than that of one-step method. In addition, the consistency of the waveform before and after the perturbation is examined, to consider whether the added adversarial examples are small enough (i.e., hard to distinguish by human eyes). This article also aims at inspiring researchers to further promote the CNNs reliabilities against adversarial attacks.
Article
Cyber-attacks on cyber-physical systems (CPSs) can lead to sensing and actuation misbehavior, severe damages to physical objects, and safety risks. Machine learning algorithms have been proposed for hindering cyber-attacks on CPSs, but the absence of labeled data from novel attacks makes their detection quite challenging. In this context, Generative Adversarial Networks (GANs) are a promising unsupervised approach to detect cyber-attacks by implicitly modeling the system. However, the detection of cyber-attacks on CPSs has strict latency requirements, since the attacks need to be stopped before the system is compromised. In this paper, we propose FID-GAN, a novel fog-based, unsupervised intrusion detection system (IDS) for CPSs using GANs. The IDS is proposed for a fog architecture, which brings computation resources closer to the end nodes and thus contributes to meeting low-latency requirements. In order to achieve higher detection rates, the proposed architecture computes a reconstruction loss based on the reconstruction of data samples mapped to the latent space. Other works that follow a similar approach struggle with the time required to compute the reconstruction loss, which renders them impractical for latency constrained applications. We address this problem by training an Encoder that accelerates the reconstruction loss computation. Experiments show that the proposed solution achieves higher detection rates and is at least 5.5 times faster than a baseline approach in the three studied datasets.
Article
Recent advancements in radio frequency machine learning (RFML) have demonstrated the use of raw in-phase and quadrature (IQ) samples for multiple spectrum sensing tasks. Yet, deep learning techniques have been shown, in other applications, to be vulnerable to adversarial machine learning (ML) techniques, which seek to craft small perturbations that are added to the input to cause a misclassification. The current work differentiates the threats that adversarial ML poses to RFML systems based on where the attack is executed from: direct access to classifier input, synchronously transmitted over the air (OTA), or asynchronously transmitted from a separate device. Additionally, the current work develops a methodology for evaluating adversarial success in the context of wireless communications, where the primary metric of interest is bit error rate and not human perception, as is the case in image recognition. The methodology is demonstrated using the well known Fast Gradient Sign Method to evaluate the vulnerabilities of raw IQ based Automatic Modulation Classification and concludes RFML is vulnerable to adversarial examples, even in OTA attacks. However, RFML domain specific receiver effects, which would be encountered in an OTA attack, can present significant impairments to adversarial evasion.
Conference Paper
The purpose of this study is to introduce new design-criteria for next-generation hyperparameter optimization software. The criteria we propose include (1) define-by-run API that allows users to construct the parameter search space dynamically, (2) efficient implementation of both searching and pruning strategies, and (3) easy-to-setup, versatile architecture that can be deployed for various purposes, ranging from scalable distributed computing to light-weight experiment conducted via interactive interface. In order to prove our point, we will introduce Optuna, an optimization software which is a culmination of our effort in the development of a next generation optimization software. As an optimization software designed with define-by-run principle, Optuna is particularly the first of its kind. We will present the design-techniques that became necessary in the development of the software that meets the above criteria, and demonstrate the power of our new design through experimental results and real world applications. Our software is available under the MIT license (https://github.com/pfnet/optuna/).