Optimizing Deep Learning Based Channel Estimation using Channel Response Arrangement

Abstract

The techniques used in deep learning for channel estimation are generally model-centric. These models have changed significantly over the years with each iteration yielding a better estimator than the last. Fundamentally, channel estimation works by exploiting correlations in an array of complex numbers, in particular the channel gains for a fading channel. In this paper, we study the effects of the spatial arrangement of channel response and input data, on channel estimation. With the right spatial arrangement, we improved the performance of our convolutional neural network that was used for estimation. Additionally, we optimized the training procedure simultaneously. We experimentally validate the importance of spatial arrangement of data in obtaining an accurate deep learning model for the channel.

Full text

Optimizing Deep Learning Based Channel
Estimation using Channel Response Arrangement
Satya Kumar Vankayala
Samsung R&D Institute
Bangalore, India
satyakumar.v@samsung.com
Swaraj Kumar
Samsung R&D Institute
Bangalore, India
swaraj.kumar@samsung.com
Issaac Kommineni
Samsung R&D Institute
Bangalore, India
issaac.k@samsung.com
Abstract—The techniques used in deep learning for channel es-
timation are generally model-centric. These models have changed
significantly over the years with each iteration yielding a better
estimator than the last. Fundamentally, channel estimation works
by exploiting correlations in an array of complex numbers, in par-
ticular the channel gains for a fading channel. In this paper, we
study the effects of the spatial arrangement of channel response
and input data, on channel estimation. With the right spatial
arrangement, we improved the performance of our convolutional
neural network that was used for estimation. Additionally, we
optimized the training procedure simultaneously. We experimen-
tally validate the importance of spatial arrangement of data in
obtaining an accurate deep learning model for the channel.
Index Terms—channel estimation, spatial arrangement, deep
learning, OFDM, CNN
I. INTRODUCTION
Accurate and efficient channel estimation in 5G communica-
tion poses a major challenge. Compared to earlier generations,
5G requires many more antennas and low SNR. Also, the
bandwidth for 5G systems are staggeringly higher than 4G sys-
tems. On the other hand, slot duration has come down in 5G by
a factor of 5. Delay requirement are also much more stringent
in 5G systems. There is a need to come up with novel channel
estimation method which is able to address these challenges.
In this paper we propose a deep learning solution for obtaining
efficient channel estimation. Applications of deep learning in
different domains have garnered the curiosity of researchers
worldwide. The neural network’s capability to learn complex
patterns in data has been exploited in a myriad of complex
tasks like computer vision, natural language processing and
many more. In particular, deep learning (DL) techniques
have found utility in the wireless communication domain, a
field dominated by algorithms and heuristics, as well [1]. In
the physical layer, DL has allowed for insightful solutions
for channel modelling, estimation, encoding, decoding and
equalisation, thus improving the quality and efficiency of the
communication [2]. As a result, significant advances have been
made in 5G wireless systems [3].
Deep neural networks (DNNs) are no longer restricted to
vanilla pattern recognition tasks. Coupled with a plethora of
new training methodologies and models, DNNs are being
used in other tasks such as finding missing data and sentence
completion. The authors in [4] demonstrated Convolutional
Neural Networks (CNN) ability to complete and extrapolate
images from incomplete input images. Similarly, Recurrent
Neural Networks (RNN) ability to complete sentences from
the missing input sentence has also been successfully demon-
strated in [5]. In this paper, we strive to do channel estimation
in Orthogonal frequency-division multiplexing (OFDM) [6]
system from the channel value at pilot locations, which is
input to our model. Owing to the knowledge of pilot locations,
we are not using the full channel response data as input. This
resembles the aforementioned usage of neural networks which
works with missing input data. A similar channel estimation
technique was devised by the authors in [7] where the authors
use the channel response as an image and apply a CNN model
to generate output. CNN has successfully placed itself as the
go-to neural network model for computer vision and image
processing tasks. Thus, we will treat subframes for channel
response and time-frequency channel response as images and
design a model based on CNN.
In this paper we present a novel approach based on exploit-
ing the intrinsic relationship between the channel response
values, which is made up of complex numbers, to enhance
the efficacy of our network. Increasing the layers is not the
solution for poor accuracy, as it slows down the training and
increases the number of parameters to be trained. Rather, we
extract the most out of our data by exploiting the intrinsic
relationship between the real and imaginary parts of the
channel response. Thus, they can’t be treated as two separate
entities and models cannot be trained efficiently if one doesn’t
consider their co-dependence. We introduce different ways of
spatially arranging the real and imaginary part of the channel
response. This yields a substantial improvement in perfor-
mance over simply treating them as discrete numbers with no
inter dependency. When compared with conventional method
for estimating time frequency channel response via Minimum
mean squared error (MMSE) [8] and approximated linear
version of the MMSE (ALMMSE) [9], our approach provides
a superior way for channel estimation. Incorporating the novel
spatial arrangement concept enables our model to outperform
the challenges offered by other DL based approaches like
ChannelNet [7].
II. RE LATE D WOR KS
Non-DL based OFDM channel estimation techniques such
as least squares (LS) and MMSE estimation have been

discussed in [10]. But these conventional methods tend to
have greater computational complexity. To overcome this,
DL based channel estimation techniques in OFDM systems
were developed. [11] and [12] provides end to end physical
layer architecture for OFDM systems. DL architectures in
[11] and [12] don’t provide channel estimation explicitly. In
[7], [13] and [14], they treat the channel matrix as an image
and use CNNs like super-resolution CNN (SRCNN) [15]
and Denoising CNN (DnCNN) [16] for channel estimation.
Apart from CNN, other alternative neural network models
have also been studied for channel estimation. 3 layered fully
connected neural net was used in [17] for MIMO-OFDM
channel estimation. RNN’s capability to learn time series data
has been exploited for channel estimation in [18] and via an
amalgamation of CNN and RNN in [19]. While numerous DL
approaches have been proposed, none of them study the impact
of the spatial arrangement of data in improving the efficacy
of their respective models.
III. PROP OS ED DESIGN
We employ a deep learning mechanism for estimating
the channel time-frequency response denoted by the matrix
H. Input data provided to the deep learning model is the
LS estimate value of the channel at the pilot location hLS
p.
Interpolation is applied to this channel response hLS
pmatrix.
This is then passed as an input to the CNN.
A. Dataset
We trained our models on the dataset used by [7], which
is obtained via a Single-input, Single-output (SISO) commu-
nication link. The dataset is generated using LTE simulator
developed by University of Vienna, Vienna LTE-A simulator
[20]. For wireless channel model, Vehicular-A (VehA) car-
rier frequency of 2.1GHz, bandwith of 1.6 MHz and user
equipment speed of 50 km/h are considered. Each subframe
of channel response hLS
pconsists of 14 time slots with 72
subcarriers. The resultant size of each frame of hLS
pand time-
frequency response of the channel His 72 X 14. We use a
corpus of 40000 such subframes of hLS
pand Has input and
output respectively. The values of hLS
pand Hare complex
numbers.
B. Data Pre-processing
Input to the model is estimated channel value hLS
pobtained
by LS estimation. We are going to use only those hvalues
which are present at the pilot locations. For our experiments,
we use a lattice type pilot arrangement which gives us the
hLS
p. Next, we apply radial basis function (RBF) interpolation
[21] on hLS
pto find the channel values at other points of
the subframe, other than the pilot locations. The resultant
subframe after the interpolation is the input to our model.
C. Model Architecture
Convolution Neural Networks [22] forms the basis of our
study. CNN consists of convolutional layers stacked together to
enable the learning of features in input data. Each convolution
layer employs a number of filters that enable the learning pro-
cess. The aim is to optimize the weights in these filters. This
is achieved via backpropagation [23]. Weights are initialized
using He normal [24]. Each convolution layer is followed by
Rectified Linear Units (ReLU) [25]. Incorporating ReLU, a
non-saturation activation function induces sparsity and sup-
ports faster training due to less computational requirements.
ReLU activation function is removed in the last layer because
ReLU only returns positive numbers. Hcan be negative also,
hence we remove the ReLU activation from the last layer. An
overview of the CNN model used is depicted in Fig. 1. It can
be seen that the input and output layer dimensions of CNN
are the same. This is required as each frame of H(output)
and hLS
p(input) have same dimensions. We use CNN in two
configurations, one with three convolutional blocks and the
other with six as shown in Fig. 1.
Fig. 1. Block diagram illustrating our model. 3 or 6 convolution layers are
being used. The loss function, Mean Squared Error (MSE) is used to update
the weights via back propagation as explained in section III-E
Studying the effects of different spatial arrangements of
real and imaginary numbers on training requires altering the
dimensions of input, output, and hidden layers. We change
layer dimensions while keeping the other hyper-parameters
namely number of filters, filter size, padding, stride, and
activation function unchanged. The hyper-parameter values for
3 layered CNN are mentioned in Table I. For a 6 layered
CNN, each layer in Table I is repeated twice. The weights
are initialized just as before. But as He normal initialization
depends on dimensions of the previous layer, the weights are

random but differ in range when varying the dimensions of the
input layer. Also, two hyper parameters, filter size in the first
layer and number of filters in the last layer are changed for one
of the spatial arrangements mentioned in section III-D2. For
evaluating performance on increasing the number of layers,
we increase the number of middle layers as shown in Fig. 1.
TABLE I
HYP ERPA RAM ET ERS VAL UES
Hyperparameters CNN layers
1 2 3
Number of filters 64 32 1
(2 for
section III-D2 )
Filter Size 9 X 9
(9X9X2for 1X1 5X5
section III-D2 )
Activation ReLU ReLU ReLU
Padding Same Same Same
Strides 1 1 1
D. Spatial Arrangements of Data
To exploit the correlation between real and imaginary part
of hLS
pand H, we analyze the effects of different spatial
arrangements of data. Same arrangement of complex numbers
is applied on input (hLS
p) and output (H). For generating
different arrangements, we first separate the real and imaginary
numbers of hLS
pand H. This results in two frames of size 72
X 14 where one is the real part and the other imaginary. With
each arrangement, we intend to try a different way to feed data
into our CNN. Four different arrangements are considered:
1) Separated: Model is trained in two segments, an upper
and a lower segment. It is trained on all the real frames in
upper segment, where it learns to estimate the corresponding
real frame of H. This is followed by training on imaginary
frames in the lower segment in a similar fashion. Here input
and output dimensions are 144 and 14 respectively.
2) Overlap in different channels: The real and imaginary
parts are passed in the model like an image with 2 channels.
The real frame forms the first channel and imaginary forms
the second. The input and output dimensions are 72 and
14 respectively for each of the two channels. As the input
to model has depth 2 (2 channels), the filters in the first
convolution layer have depth 2. The length and breadth of
the filters are kept the same as in the other three cases. So the
dimensions of filters in the first layer is 9 X 9 X 2. Rest all
other hidden layers have the same filter dimensions. 2 filters
are needed in the output layer so that the output also has two
channels.
3) Row-wise alternatively: A composite frame is obtained
by placing alternate rows of real and imaginary frames. The
rendered frame is twice the length of the real/imaginary frame.
Input/output dimensions= 144 X 14
4) Column-wise alternatively: The transpose of the Row-
wise alternatively gives the column-wise alternatively with
Input/output dimensions=72 X 28
For better visualization, low dimensional matrices (real and
imaginary) as input with the aforementioned spatial arrange-
ments have been illustrated in Fig. 2.
Fig. 2. Figure depicting the various spatial arrangements of input data. Blue
and red colour represents real and imaginary parts respectively. Arrangements
from top to bottom are - overlap in different channels, row-wise alternate,
column-wise alternate and separated
E. Loss Function
The model aims to learn to generate Hvia forward prop-
agation while updating weights of the CNN, using backward
propagation as depicted in Fig. 1. The objective of our model is
to minimize the following mean squared error (MSE) function:
L=1
kN k X
hp∈N
kf(Θsa;ˆ
hLS
p)−Hk2
2,(1)
where Θsa is the network parameter for a particular spatial
arrangement. f(Θsa;ˆ
hLS
p)represents the output that the model
generates in forward propagation having weights Θsa.Nis
the set of subframes in training batch.
F. Training
The model was trained for 300 epochs using Adam op-
timizer [26] with a learning rate of 0.001. 75 percent of the
data was used for training, 15 percent for the validation and 10
percent for testing. We trained our model on 4 Precision Tower
7820 workstation with Intel Xenon Gold 5120 processor (14
core and 28 threads each processor). The number of parameters

(weights) to be trained varies with the spatial arrangement
selected for the training.
Table II shows the exponential increase in the number of
trainable parameters. This in turn slows down the training
process. Also, it is important to note that the overlap chan-
nel arrangement of input has required a significantly greater
number of parameters to be trained vis a vis other three ar-
rangements. The two-dimensional spatial arrangement require
the same number of trainable parameters as the filters have
to convolve on the same area for these arrangements. With an
increased number of trainable parameters, training slows down
concomitantly. Training the models with additional hidden
layers increases the time of training as many more untrained
parameters are added with each convolution layer.
TABLE II
VARIATI ON I N THE N UM BER O F TR AIN ABL E PARA ME TER S FOR D IFF ERE NT
SPATIA L AR RAN GE MEN TS W ITH 3A ND 6L AYERE D CNN
Spatial Trainable Parameters
Arrangement 3 Conv 6 conv
Separated 8129 341025
Overlap in different channel 14114 347010
Row-wise alternatively 8129 341025
Column-wise alternatively 8129 341025
IV. RES ULT S
For testing the efficacy of our novel approach we train the
CNNs on two different SNRs, one at 12 dB and other at a
higher SNR of 22 dB. We use 48 pilot locations spread in a
lattice arrangement to carry out the interpolation. To get an in-
sight into the performance variation due to layers we train both
the 3 and 6 layered CNNs. Significant variations in MSE loss
is observed for different arrangements. The enhanced perfor-
mance of deeper networks is clearly exhibited in Fig. 3. While
adding convolutional blocks increases the accuracy, it comes
with the burden of a prolonged training schedule. This tradeoff
needs to be considered while deciding on the architecture of
CNNs. We observe that arrangement with alternate complex
and real parts outperforms the separated and overlap arrange-
ments. Overall, the column-wise alternative arrangement is the
best one for our task, though it is only having a minuscule
edge over the row-wise alternate arrangement. Further, we
carried out a comparative examination of our approach with
other deep learning-based channel estimation like [7], [13],
[14]. All these approaches use separated spatial arrangement.
We compare the performance of our approach with [7] and
other conventional techniques like MMSE, estimated MMSE
and ALMMSE. Fig. 4 shows that if our approach with 6
layered CNN is used in tandem with DnCNN then it is able
to outperform [7] by 46.6%which highlights the importance
of spatial arrangement of data in deep learning-based channel
estimation. We used DnCNN with 20 convolutional blocks.
Such a deep network impedes the training procedure. If
we remove DnCNN then our network with the best spatial
arrangement (column-wise alternate) is subpar to that of [7]
by 1.2%. But this minor reduction in performance comes
Fig. 3. Performance of 3 and 6 layered CNNs with different spatial
arrangements. Each CNN is trained for a specific SNR, either 12 dB or 22
dB.
with considerable gain as it cuts the training time by 80%.
Authors in [7] uses 23 convolutional layers in total. This
results in very high computational complexity. Our results
shows that we can achieve similar performance with just 6
layered CNN with column-wise spatial arrangement. Thus we
substantially reduce the computer complexity. Our approach
gives far better accuracy than conventional approaches such
as estimated MMSE and ALMMSE. Moreover our approach
does not require channel information. However, Ideal MMSE
is able to beat our results as it has complete information about
the channel.
Fig. 4. Performance of different channel estimation techniques at different
SNRs. Here CNN (worst) is 6 layered CNN with the separated spatial
arrangement and CNN (best) is 6 layered CNN with the column-wise alternate
spatial arrangement. CNN (worst) + DnNN is the network used in [7].
The results in Fig. 3 and Fig. 4 are obtained with 48 pilot
locations arranged in a lattice arrangement. But on decreasing
the number of pilot locations the accuracy decreases as de-
picted in Fig. 5. This can be explained by the weakening of
interpolation with reduction in the number of pilot locations.

Fig. 5. Variation in performance with the number of pilots for arrangements
mentioned in section III-D2 and III-D1 with SNR of 12 dB and 22 dB.
V. CONCLUSIONS AND FUTURE WORK
In this paper, we explored the effects that spatial arrange-
ments of data can produce in channel estimation models.
Our results provide compelling arguments that favour further
studies on data pre-processing and augmentation, especially
in the machine learning domain. In addition to this, denoising
the time frequency response output also increases the accuracy.
The denoising task can me made more prudent by using effi-
cient denoising models such as SkidNet [27]. Also, the intrin-
sic relationship between the complex numbers can be exploited
with the new variant of neural networks such as complex-
valued neural networks [28]. Accurate channel estimation is
one of the key module that can improve the throughput of the
system. Deep learning based channel estimation module can
be easily implemented on O-RAN/V-RAN and can become the
quintessential estimation module in the future communication
systems with its inimitable performance.
REFERENCES
[1] A. Zappone, M. Di Renzo and M. Debbah, ”Wireless Networks Design
in the Era of Deep Learning: Model-Based, AI-Based, or Both?” in
IEEE Transactions on Communications, vol. 67, no. 10, pp. 7331-7376,
Oct. 2019.
[2] T. Wang, C. Wen, H. Wang, F. Gao, T. Jiang and S. Jin, ”Deep learning
for wireless physical layer: Opportunities and challenges,” in China
Communications, vol. 14, no. 11, pp. 92-111, Nov. 2017.
[3] H. Huang et al., ”Deep Learning for Physical-Layer 5G Wireless
Techniques: Opportunities, Challenges and Solutions,” in IEEE Wireless
Communications, vol. 27, no. 1, pp. 214-222, February 2020.
[4] X. Wu et al., ”Deep Portrait Image Completion and Extrapolation,” in
IEEE Transactions on Image Processing, vol. 29, pp. 2344-2355, 2020.
[5] P. Mirowski, A. Vlachos, ”Dependency recurrent neural language models
for sentence completion,” in arXiv preprint arXiv:1507.01193. 2015 Jul
5.
[6] J. Armstrong, ”OFDM for Optical Communications,” in Journal of
Lightwave Technology, vol. 27, no. 3, pp. 189-204, Feb.1, 2009.
[7] M. Soltani, V. Pourahmadi, A. Mirzaei and H. Sheikhzadeh, ”Deep
Learning-Based Channel Estimation,” in IEEE Communications Letters,
vol. 23, no. 4, pp. 652-655, April 2019.
[8] Z. Luo and D. Huang, ”General MMSE Channel Estimation for MIMO-
OFDM Systems,” 2008 IEEE 68th Vehicular Technology Conference,
Calgary, BC, 2008, pp. 1-5.
[9] M. Simko, C. Mehlfhrer, M. Wrulich, and M. Rupp, “Doubly disper-
sivechannel estimation with scalable complexity,” in2010 International
ITGWorkshop on Smart Antennas (WSA), pp. 251–256, Feb 2010.
[10] M. K. Ozdemir and H. Arslan, “Channel estimation for wireless
OFDMsystems,”IEEE Commun. Surveys Tut., vol. 9, no. 2, pp. 18–48,
Second2007.
[11] V. Raj and S. Kalyani, ”Backpropagating Through the Air: Deep
Learning at Physical Layer Without Channel Models,” in IEEE Com-
munications Letters, vol. 22, no. 11, pp. 2278-2281, Nov. 2018.
[12] H. Ye, G. Y. Li and B. Juang, ”Power of Deep Learning for Channel
Estimation and Signal Detection in OFDM Systems,” in IEEE Wireless
Communications Letters, vol. 7, no. 1, pp. 114-117, Feb. 2018.
[13] X. Ru, L. Wei, Y. Xu, ”Model-Driven Channel Estimation for OFDM
Systems Based on Image Super-Resolution Network,” in arXiv preprint
arXiv:1911.13106. 2019 Nov 29.
[14] H. He, C. Wen, S. Jin, and G. Y. Li, “Deep learning-based channelesti-
mation for beamspace mmwave massive mimo systems,”IEEE Wireless
Communications Letters, vol. 7, pp. 852–855, Oct 2018.
[15] C. Dong, C. C. Loy, K. He, and X. Tang, “Image super-resolution
usingdeep convolutional networks,”IEEE Transactions on Pattern Anal-
ysisand Machine Intelligence, vol. 38, pp. 295–307, Feb 2016.
[16] K. Zhang, W. Zuo, Y. Chen, D. Meng, and L. Zhang, “Beyond a Gaus-
sian Denoiser: Residual Learning of Deep CNN for Image Denois-
ing,”IEEE Transactions on Image Processing, vol. 26, pp. 3142–3155,
July2017.
[17] K. Mei, J. Liu, X. Zhang, J. Wei, ”Machine Learning Based Channel
Estimation: A Computational Approach for Universal Channel Condi-
tions,” in arXiv preprint arXiv:1911.03886. 2019 Nov 10.
[18] Q. Bai, J. Wang, Y. Zhang and J. Song, ”Deep Learning-Based Channel
Estimation Algorithm Over Time Selective Fading Channels,” in IEEE
Transactions on Cognitive Communications and Networking, vol. 6, no.
1, pp. 125-134, March 2020.
[19] J. Yuan, H. Q. Ngo and M. Matthaiou, ”Machine Learning-Based
Channel Estimation in Massive MIMO with Channel Aging,” 2019 IEEE
20th International Workshop on Signal Processing Advances in Wireless
Communications (SPAWC), Cannes, France, 2019, pp. 1-5.
[20] C. Mehlfhrer, J. Colom Ikuno, M. Simko, S. Schwarz, M. Wrulich,and
M. Rupp, “The vienna lte simulators - enabling reproducibility inwire-
less communications research,”EURASIP Journal on Advances inSignal
Processing, vol. 2011, p. 29, Jul 2011.
[21] M. A. Abebe and J. Y. Hardeberg, ”Application of Radial Basis
Function Interpolation for Content Aware Image Retargeting,” 2018
14th International Conference on Signal-Image Technology and Internet-
Based Systems (SITIS), Las Palmas de Gran Canaria, Spain, 2018, pp.
174-183.
[22] S. Albawi, T. A. Mohammed and S. Al-Zawi, ”Understanding of a
convolutional neural network,” 2017 International Conference on En-
gineering and Technology (ICET), Antalya, 2017, pp. 1-6.
[23] Hecht-Nielsen, ”Theory of the backpropagation neural network,” Inter-
national 1989 Joint Conference on Neural Networks, Washington, DC,
USA, 1989, pp. 593-605 vol.1.
[24] K. He, X. Zhang, S. Ren and J. Sun, ”Delving Deep into Rectifiers: Sur-
passing Human-Level Performance on ImageNet Classification,” 2015
IEEE International Conference on Computer Vision (ICCV), Santiago,
2015, pp. 1026-1034.
[25] K. Hara, D. Saito and H. Shouno, ”Analysis of function of rectified
linear unit used in deep learning,” 2015 International Joint Conference
on Neural Networks (IJCNN), Killarney, 2015, pp. 1-8.
[26] S. Bock and M. Weiß, ”A Proof of Local Convergence for the Adam
Optimizer,” 2019 International Joint Conference on Neural Networks
(IJCNN), Budapest, Hungary, 2019, pp. 1-8.
[27] S. Dutta, S. Chaturvedi, S. Kumar and M. Bhatia, ”SkiDNet: Skip Image
Denoising Network for X-Rays,” 2019 International Joint Conference on
Neural Networks (IJCNN), Budapest, Hungary, 2019, pp. 1-8.
[28] J. Stankowicz, J. Robinson, J. M. Carmack and S. Kuzdeba, ”Complex
Neural Networks for Radio Frequency Fingerprinting,” 2019 IEEE West-
ern New York Image and Signal Processing Workshop (WNYISPW),
Rochester, NY, USA, 2019, pp. 1-5.