Content uploaded by Pan Shang
Author content
All content in this area was uploaded by Pan Shang on Feb 07, 2022
Content may be subject to copyright.
ARTICLE IN PRESS
Please cite this article as: P. Shang, X. Liu, C. Yu et al., A new ensemble deep graph reinforcement learning network for spatio-temporal traffic volume forecasting in a
freeway network, Digital Signal Processing, https://doi.org/10.1016/j.dsp.2022.103419
JID:YDSPR AID:103419 /FLA [m5G; v1.312] P.1 (1-13)
Digital Signal Processing ••• (••••)••••••
Contents lists available at ScienceDirect
Digital Signal Processing
www.elsevier.com/locate/dsp
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
A new ensemble deep graph reinforcement learning network for
spatio-temporal traffic volume forecasting in a freeway network
Pan Shang a, Xinwei Liu b, Chengqing Yu c, Guangxi Yan c, Qingqing Xiang d, Xiwei Mi a,∗
aSchool of Traffic and Transportation, Beijing Jiaotong University, Beijing, 100044, China
bSchool of Mechanical, Electronic and Control Engineering, Beijing Jiaotong University, Beijing, 100044, China
cSchool of Traffic and Transportation Engineering, Central South University, Changsha, 410075, China
dSchool of Traffic and Transportation, East China Jiaotong University, Nanchang, 330013, China
a r t i c l e i n f o a b s t r a c t
Article history:
Available online xxxx
Keywords:
Traffic volume forecasting
Graph convolutional network
Graph attention network
Deep reinforcement learning
Spatio-temporal traffic volume series
Spatio-temporal traffic volume forecasting technologies can effectively improve freeway traffic efficiency
and the travel comfort of humans. To construct a high-precision traffic volume forecasting model, this
study proposed a new ensemble deep graph reinforcement learning network. The modeling process of
the spatio-temporal prediction model mainly included three steps. In step I, raw spatiotemporal traffic
network datasets (traffic volumes, traffic speeds, weather, and holidays) were preprocessed and the
adjacency matrix was constructed. In step II, a graph attention network (GAT) and graph convolution
network (GCN) were used as the main predictors to build the spatio-temporal traffic volume forecasting
model and obtain the forecasting results, respectively. In step III, deep reinforcement learning was used
to effectively analyze the correlations between the forecasting results from these two neural networks
and the final results, so as to optimize the weight coefficient. The final result of the proposed model was
obtained by combining the forecasting results from the GAT and GCN with the weight coefficient. Based
on summarizing and analyzing the experimental results, it can be concluded that: (1) deep reinforcement
learning can effectively integrate the two different graph neural networks and achieve better results than
traditional ensemble methods; and (2) the presented ensemble model performs better than twenty-one
models proposed by other researchers for all studied cases.
2022 Elsevier Inc. All rights reserved.
1. Introduction
With the rapid development of modern cities, the rapid growth
of vehicles has caused many problems, such as traffic congestion
and environmental pollution. At present, traffic congestion has be-
come a main focus of concern [1]. As an indispensable part of
national economic development, freeways play an increasingly im-
portant role in traffic, and provide important links between cities
[2]. The establishment of an intelligent transportation planning and
dispatching system is an effective measure for alleviating traffic
congestion and improving travel quality [3]. Intelligent transporta-
tion systems provide travelers with information regarding city traf-
fic flows, congestion, and roads, thereby helping them to make
decisions [4]. Traffic volume forecasting technology is an impor-
tant and indispensable function in an intelligent traffic planning
system. A relatively accurate forecasting of the freeway traffic vol-
ume can provide effective suggestions for travelers, alleviating the
*Corresponding author.
E-mail address: mixiwei@bjtu.edu.cn (X. Mi).
short-term backlog, delay, and queuing problems caused by rela-
tively concentrated traffic flows (e.g., those in a toll station) [5].
Traffic volume forecasting has always been a popular topic in the
field of intelligent transportation. As the traffic volume is affected
by weather, holidays, time periods, road network distributions, ac-
cidents, occasional events, and other factors, it has complex trends,
periodicity, linkages, and randomness. The complexity of traffic
conditions and non-linearity of traffic volume data increase the dif-
ficulty of traffic volume prediction. Thus, traffic volume prediction
remains a challenging problem.
At present, the mainstream prediction methods proposed by
scholars can be divided into two categories: parametric models
and non-parametric models [6]. Parametric models mainly include
linear regression and auto-regressive integrated moving average
(ARIMA) approaches [7]. These models are constructed in advance
based on statistical theoretical assumptions, and then the model
parameters are calculated. Non-parametric models mainly include
the support vector machine (SVM) [8] and artificial neural net-
works [9], which are collectively referred to as artificial intelligence
(AI) models [10]. Owing to the nonlinearity and irregularity of traf-
fic volume data, AI models can better establish the correlations
https://doi.org/10.1016/j.dsp.2022.103419
1051-2004/2022 Elsevier Inc. All rights reserved.
ARTICLE IN PRESS
JID:YDSPR AID:103419 /FLA [m5G; v1.312] P.2 (1-13)
P. Shang, X. Liu, C. Yu et al. Digital Signal Processing ••• (••••)••••••
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
between the data [11]. Therefore, an increasing number of schol-
ars have applied AI models to forecast traffic volumes. As the most
popular AI algorithm, deep learning has excellent nonlinear mod-
eling and time series analysis capabilities [12]. Therefore, scholars
have increasingly used deep learning to build excellent traffic vol-
ume forecasting networks.
1.1. Related works
At present, scholars have proposed a variety of deep learning
algorithms and evaluated their performance in the field of traf-
fic flow prediction. Li et al. [13] proposed a new day-ahead traffic
flow forecasting model based on a deep belief network (DBN). The
results show that the DBN had a better predictive performance
than traditional statistical methods. Zhang et al. [14] utilized an
echo state network (ESN) to forecast short-term traffic volumes.
Experiments showed that the ESN could overcome shallow neural
networks and obtain the best forecasting results. Sun et al. [15]
presented a new gated recursive unit-based traffic volume fore-
casting model; they established an excellent urban traffic volume
forecasting model which performed better than other recursive
neural networks (RNNs). Lu et al. [16] proposed a novel traffic
flow forecasting model based on the long short-term memory net-
work (LSTM). Their results proved that the LSTM had excellent
modeling and analysis abilities for traffic time series. In general,
although the above-noted deep learning algorithms provided ex-
cellent traffic volume modeling abilities, they only considered the
traffic volume changes of a single node. At present, with the con-
tinuous expansion and increasing complexity of traffic networks, it
is very meaningful to establish a traffic network forecasting model
based on multi-nodes for analyzing traffic changes and improving
forecasting accuracy.
At present, aiming to establish the traffic network forecasting
models with high precision, scholars have proposed neural net-
work models based on pictures and graphs. Graph-based neural
networks can effectively aggregate the spatiotemporal character-
istic information of multiple nodes to improve analyses and pre-
dictions. Li et al. [17] built a new traffic flow forecasting model
based on a convolutional neural network long short-term mem-
ory network (CNN-LSTM). The experimental results showed that
CNN-LSTM could establish an excellent spatiotemporal traffic flow
prediction model. Chen et al. [18] used the ChebNet to established
a spatial environment prediction model and achieved better pre-
diction effect than traditional CNN. Peng et al. [19] proposed a new
dynamic spatio-temporal traffic volume forecasting model based
on a graph neural network (GNN). Their cases showed that the
GNN could achieve better forecasting results than statistics and
traditional neural networks. Yu et al. [20] utilized a graph convolu-
tional network (GCN) to establish accurate spatial-temporal traffic
speed forecasting. Their experiments proved that the GCN could ef-
fectively analyze the spatial-temporal correlations between nodes
and improve the forecasting accuracy. Guo et al. [21] combined
a graph attention network (GAT) and temporal convolutional net-
work (TCN) to propose a new spatial-temporal traffic forecasting
model. The experimental results showed that the proposed GAT-
TCN model had better capabilities than a traditional deep network,
and achieved satisfactory spatial-temporal forecasting results. Jin et
al. [22] proposed a new GAT-based traffic speed forecasting model.
Their comparative experiments showed that this model had better
forecasting and modeling performances than other GNN models.
Based on the literature investigation, it can be seen that graph
neural network has excellent capacity of spatio-temporal traffic
volume modeling. Besides, although the traditional GCN and GAT
algorithms have been proved to have excellent spatial modeling
ability, they are not able to extract time series features from orig-
inal data. To solve this problem, scholars usually use a variety of
RNN to optimize the performance of GNN. Zhang et al. [23] used
the GAT and Gated recursive unit (GRU) to get a wonderful spatio-
temporal modeling framework. The case proves that GRU optimizes
the performance of GCN. Liu et al. [24] applied the GRU and GCN
to obtain predictive models with excellent value. The above lit-
erature shows that the combination of gated neural network and
graph neural network can achieve excellent spatio-temporal mod-
eling results. Therefore, it is valuable to study the combination of
GRU with GAT and GCN.
Both GCN and GAT construct the feature representation of the
target node by aggregating the features of neighbor vertices to the
central node. GCN mainly uses the Labras matrix to achieve re-
lated functions. GAT mainly uses attention mechanism to realize
feature analysis [24]. According to the literature survey results, al-
though GCN and GAT have achieved satisfactory results in the field
of spatio-temporal traffic volume forecasting, they still have some
limitations. GCN algorithm makes full use of one-dimensional
edge features between nodes, that is, weight information between
edges. In addition, other important information is not extracted
by the GCN algorithm. GAT algorithm uses attention mechanism
to make the model fully integrates the topological structure and
node confidence of graph. However, GAT algorithm only uses the
connectivity between nodes and ignores important edge informa-
tion such as weights [25]. Therefore, GCN and GAT show different
adaptability in different spatio-temporal modeling domains. Egor
et al. [26] compared GAT and GCN algorithms and found that they
had different modeling effects on different data sets. Yu et al. [27]
combined GAT and GCN to optimize the performance of the pre-
diction model. The experimental results showed that the proposed
model can outperform the traditional GCN and GAT. Therefore, the
effective combination of GCN and GAT algorithm is an important
method to improve the overall modeling ability and generalization
performance of the model.
The ensemble learning method can integrate multiple neural
networks by using an optimization algorithm and weight coeffi-
cient; these improve the model’s ability to analyze and adapt to
different datasets [28]. Pulido et al. [29] utilized particle swarm
optimization (PSO) to build an new ensemble neural network fore-
casting model. The results showed that the PSO effectively inte-
grated several excellent neural networks, greatly improving the
accuracy of the model. Zhao et al. [30] used a fruit fly optimization
algorithm (FOA) to integrate a DBN and ARIMA. Contrastive exper-
iments showed that the proposed ensemble model based on FOA
outperformed all of the examined single models. Niu et al. [31]
proposed a novel ensemble time series prediction model based
on a gray wolf optimizer (GWO). Their comparative experiments
fully proved the excellent ensemble performance of the GWO. Li
et al. [32] presented a novel imperialist competitive algorithm
(ICA)-based ensemble model. The results showed that the ICA ef-
fectively integrated multiple neural networks and achieved better
results than these networks alone. Although heuristic algorithms
have achieved good results in the field of ensemble learning, it has
been difficult to make great breakthroughs. Therefore, there is an
important need to propose a new ensemble learning method to
improve the overall prediction abilities of these models [33].
In recent years, an agent-based reinforcement learning (RL)
algorithm has attracted wide attention in academic circles. RL
can improve the decision-making and analysis abilities of agents
through continuous training [34]. Therefore, this type of algorithm
has achieved excellent results in the fields of path planning and
game decision-making. Yin et al. [35] built an adaptive ensemble
model of predictors based on RL. Comparative experiments showed
that RL could effectively optimize the overall generalization per-
formance of the model. Wang et al. [36] proposed a new ensemble
traffic signal control model based on RL. Their experiments showed
that RL could adaptively optimize multiple traffic signals and ob-
2
ARTICLE IN PRESS
JID:YDSPR AID:103419 /FLA [m5G; v1.312] P.3 (1-13)
P. Shang, X. Liu, C. Yu et al. Digital Signal Processing ••• (••••)••••••
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
tain excellent decision results. Abdoos et al. [37] used RL and an
LSTM to propose an ensemble traffic forecasting model driven by
multiple traffic signals. The proposed method could build an ex-
cellent integration model and achieve better results than a single
LSTM. Based on the literature, however, it can be seen that al-
though RL algorithms can achieve excellent results in ensemble
learning, the decision of the agent is stored in the Q table, lim-
iting the decision storage range and universality of the agent [38].
To improve the storage spaces and intelligence of agents, schol-
ars have used neural networks to replace the traditional Q-tables,
i.e., deep RL [39]. Shen et al. [40] proposed a new CT image pa-
rameter optimization model based on a deep Q network (DQN).
The DQN model could effectively optimize the relevant parameters
to improve the overall recognition ability of the model according
to the characteristics of the input images. Ding et al. [41] used
a DQN as the main classifier to construct a bearing fault diag-
nosis model. Compared with traditional classification algorithms,
the DQN algorithm could extract parameter information from time
series data more effectively, and could optimize the classifica-
tion decision abilities of the agents. Deng et al. [42] proposed a
new dynamic financial system analysis model based on deep RL.
Their experiments showed that this model could analyze financial
data in depth, and obtain better decision results than a traditional
trading system. Pan et al. [43] proposed a new freeway vehicle
control and optimization model based on deep RL. Their experi-
ment proved that deep RL could effectively combine road network
states to optimize scheduling decisions and achieve better results
than the traditional methods. Based on the above literature survey
results, it can be found that deep RL provides excellent parame-
ter optimization and decision making performance. In addition, it
has been shown that GAT and GCN neural networks have pow-
erful capabilities for node aggregation and modeling analyses in
the field of spatial-temporal traffic volume forecasting. Therefore,
it would be very meaningful to study a combination of deep RL
and these two types of GNNs, so as to propose a new ensemble
spatio-temporal forecasting model.
1.2. The novelty of this study
According to above related works and literature survey results,
a new spatio-temporal traffic volume forecasting model based on
an ensemble deep graph RL network is presented herein. The con-
tributions of this paper can be mainly summarized as follows:
(1) In this study, two spatial GNNs (GCN and GAT) are used to
build spatial-temporal traffic volume forecasting models for
nine stations. In contrast to traditional single-point forecasting
models and deep neural networks, the GNNs can effectively
aggregate the characteristic information of each site, and effec-
tively integrate the spatio-temporal correlation characteristics
of the data. And holidays, traffic speed and weather will be
used as feature inputs to assist modeling. In addition, this pa-
per adopts GRU assisted GAT and GCN to further extract data
time features. To verify the performance of the GCN and GAT,
seven other neural networks are compared with them.
(2) Deep RL is used to effectively integrate the GAT and GCN to
further improve the overall space-time modeling capabilities
and forecasting performance of the model. A DQN combines
the characteristics of deep learning and RL, effectively improv-
ing the overall decision-making and adaptability of the agent
in the process of training. Therefore, the DQN can effectively
realize an ensemble of the GCN and GAT, and achieve better
results than a traditional heuristic algorithm or Q table-based
RL algorithm.
(3) The proposed ensemble deep graph RL network, comprising
the GCN, GAT and deep RL based ensemble method, is novel.
Therefore, it is significant to study the application value of the
model in the field of spatiotemporal traffic volume forecast-
ing. To verify that the model can establish a high-precision
framework for spatiotemporal data analysis and prediction, we
reproduce the work of twenty-one other researchers, and com-
pared their performances with those of the DQN-GCN-GAT.
2. Methodology
2.1. Entire framework of the presented traffic volume forecasting model
The proposed ensemble deep graph RL network mainly includes
three modules: a Data preprocessing module, a predictor module,
and an ensemble learning module based on deep RL. The overall
topology structure of the proposed model is shown in Fig. 1. The
details of these modules are as follows.
Module I: The raw traffic volume data of the freeway network
is preprocessed based on the sliding window method. In addition,
the traffic speed, weather, and holiday data are used as feature
information to help build forecasting models. In this paper, slid-
ing windows and data type conversion are used to preprocess all
feature data. In addition, in order to ensure the comprehensive
adaptability and generalization modeling ability of the model, this
paper adopts the ten fold cross validation.
Module II: The GAT and GCN networks are respectively used to
build the prediction models for highway networks, and the cor-
responding parameters are determined based on the training set
data.
Module III: The DQN algorithm is used to integrate the two
neural network models. Two weighting coefficients, w1and w2,
are introduced to integrate the trained GCN and GAT models. In
addition, the deep RL effectively optimizes the weight coefficient
to obtain higher model accuracy. Therefore, the output of the en-
semble model can be calculated using Equation (1).
ˆ
O(T)=w1ˆ
O1(T)+w2ˆ
O2(T)(1)
where, w1and w2are the weight coefficients of the GCN and
GAT, respectively. ˆ
O1(T)and ˆ
O2(T)are the outputs of the GCN
and GAT, respectively. The validation set is used to train the DQN
agent. The test set is used to evaluate the predictive power of the
proposed model.
2.2. Predictors
2.2.1. Gated recursive unit
The standard RNN has problems of gradient disappearance and
gradient explosion in the training process. Therefore, RNN is dif-
ficult to achieve long-term preservation of information [44]. LSTM
network preserves historical information by introducing memory
unit and gated unit. However, the internal structure of LSTM is
complex, and the time cost of model training is too high [45]. GRU
combines the input gate and forget gate in LSTM unit into update
gate, which is used to control the update of hidden state. The reset
gate is used to determine whether to retain the previously hidden
state [33]. In this study, the main function of GRU network is to
extract the depth time series characteristics of traffic volume time
series data, which can effectively improve the comprehensive mod-
eling effect of GAT and GCN.
2.2.2. Graph convolutional network
The GCN is derived from the traditional CNN. Graph convolution
methods are mainly divided into two categories: spectrum-based
methods and space-based methods [46]. From the point of view
of graph signal processing, the spectrum-based approach defines
3
ARTICLE IN PRESS
JID:YDSPR AID:103419 /FLA [m5G; v1.312] P.4 (1-13)
P. Shang, X. Liu, C. Yu et al. Digital Signal Processing ••• (••••)••••••
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
Fig. 1. Overall topology structure of DQN-GCN-GAT.
the graph convolution by introducing a filter. Therefore, spectrum-
based graph convolution can be understood as removing noise
from a graph signal [47]. The graph convolution method based on
space collects the information of neighbor nodes to construct the
graph convolution. When the graph convolution operates at the
node level, the graph pooling module and graph convolution can
be staggered and overlaid; thus, the graph can be coarsely pro-
cessed into a high-level subgraph [48]. At present, spectrum-based
graph convolution networks mainly rely on the Eigen decomposi-
tion of a Laplace matrix. Therefore, they have three defects [49], as
follows:
(1) Any perturbation to the graph will lead to a change in the
eigenvalue.
(2) The filters learned are domain-dependent, so they cannot be
extended to graphs with different structures.
(3) The time complexity of the feature decomposition is N3.
Therefore, the calculation is very time-consuming for graphs
with large amounts of data.
A GCN based on space mainly defines the graph convolution by
analyzing the spatial relations of nodes, and successfully mim-
ics the convolution operation in a traditional CNN [50]. A GCN
can effectively aggregate a node in the graph with its neigh-
bor nodes to construct a new characteristic representation of the
node. GCNs usually superimpose multiple graph convolution layers
together to effectively improve the depth and breadth informa-
tion of the node-receiving domain [51]. Based on the superpo-
sition mode for the convolution layer, the spatial graph convo-
lution can be divided into recursion-based spatial graph convo-
lution and synthesis-based spatial graph convolution. Recursion-
based graph convolution mainly uses the same graph convolution
layer to model and analyze the graph [52]. Combinatorial graph
convolution models and updates graphs using different convolu-
tion layers. The graph convolution operator is defined as follows
[53]:
sn+1
i=αX
j∈N(i)
1
cij
wnsn
i(2)
where, srepresents the characteristic information of the nodes; c
represents a normalization factor; wrepresents the weight infor-
mation of the nodes; and αstands for the activation function.
The graph convolution operation has three steps [54], as fol-
lows:
Step 1: Each node passes its characteristic information to its
neighbor nodes.
Step 2: Each node collects the characteristic information of its
neighbor nodes and itself to fuse the local structure.
Step 3: An activation function is (preferably) added in the graph
convolution to perform nonlinear transformations on the informa-
tion of nodes, so as to enhance the expression ability of the model.
Therefore, the key to the GCN is to learn a function that can
aggregate the characteristic information of the current node with
that of its neighbor nodes. The connections between nodes and
their length also affect correlation. This paper defines weighted
topology graph Gw=(V,E,W) according to the following for-
mula. The weight of the edge is shown in Eq. (3). Adjacency matrix
XWis expressed as Eq. (4).
Wij =median {len(vk)|vk∈V}
len(The shortest distance between vjand vi)(3)
4
ARTICLE IN PRESS
JID:YDSPR AID:103419 /FLA [m5G; v1.312] P.5 (1-13)
P. Shang, X. Liu, C. Yu et al. Digital Signal Processing ••• (••••)••••••
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
Fig. 2. Structure of proposed graph convolutional network.
Fig. 3. Structure of proposed graph attention network.
XW=
0W12 ... W1N
W21 0... W2N
... ... ... ...
WN1WN2... 0
(4)
where, len represents the calculated path length.
The main structure of the proposed GCN is shown in Fig. 2.
2.2.3. Graph attention network
The largest difference between the GAT and GCN is the intro-
duction of the attention mechanism. Therefore, the GAT can give
more weight to more important nodes [55]. In the end-to-end
framework, the attention weights and neural network parame-
ters are learned together. In the attention network, the weights
between nodes are parameterized, and can be continuously opti-
mized during the learning of the network [56]. The most signifi-
cant advantages of using an attention network are as follows [57]:
(1) Efficient: For adjacent nodes, the model can implement paral-
lel computing.
(2) Flexible: For nodes of different degrees, any weight corre-
sponding to them can be used.
(3) Portable: The models can be applied to graph-structured data
that has never been seen before, and which does not need to
be the same as the training set.
The GAT network is implemented by stacking multiple graph at-
tention layers [58]. For the first graph attention layer, the input of
the model is the feature set of the original node, and the output of
the model is a new feature set of that node [59]. In the GAT, the
most important thing is to build the weight coefficients between
the different nodes. To calculate the weight of each neighbor node,
the model builds a shared weight matrix Wto calculate the atten-
tion coefficient [60], as follows:
ei
j=aW hi,W h j(5)
where, hrepresents the input features of the nodes, Wrepresents
the weight matrix, and arepresents the attention mechanism.
In the modeling process, the attention mechanism is a single-
layer feedforward neural network. The model uses LeakyRelu as
the primary nonlinear activation function. The LeakyRelu function
can be described as follows [61]:
yi=(xi(xi≥0)
xi
ai(xi<0)(6)
where, airepresents a fixed parameter with a range of (1,+∞).
The main structure of the proposed GAT is shown in Fig. 3.
2.3. Ensemble method based on DQN
Aiming to propose a more intelligent and anthropomorphic al-
gorithm, scholars have combined deep learning with RL to propose
deep RL [62]. In contrast to traditional RL methods, deep RL uses a
neural network to store the Q values of discrete state action pairs
instead of a Q table, effectively solving the problem of traditional
methods, i.e., their difficulty in storing continuous state spaces and
action spaces [63]. Compared with Q-learning, the DQN provides
improvements in the following aspects [64]:
(1) the neural network is applied to approach the value function;
(2) the target Q network is used to update the target; and
(3) the experience playback is used.
Therefore, the deep RL algorithm has excellent perception and
decision-making abilities, enabling it to effectively realize the
learning and control of raw data [65]. In ensemble learning, re-
inforcement learning needs to be used to continuously process the
prediction results of GCN and GAT and optimize the weight coeffi-
cient. In addition, Q tables are difficult to store a large number of
state-action pairs. Therefore, it is valuable to adopt deep reinforce-
ment learning to realize model ensemble. The core parameters that
need to be defined for deep RL are the action, reward, and state
[66]; these are determined as follows:
State: In this study, the weight wiof the GCN and GAT is set
as the state, and is defined as follows:
S(t)=[w1,w2](7)
where, w1is the weight of the GCN; w2is the weight of the GAT.
5
ARTICLE IN PRESS
JID:YDSPR AID:103419 /FLA [m5G; v1.312] P.6 (1-13)
P. Shang, X. Liu, C. Yu et al. Digital Signal Processing ••• (••••)••••••
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
Action: The agent’s behavior aims to change the weight of the
prediction results of each predictor to minimize the forecasting er-
ror. The action is given as follows:
A(t)=[1w1,1w2](8)
where, 1wis the change of the weight.
Reward: The function of the reward is to allow the agent to
minimize the forecasting error. In particular, the mean squared er-
ror (MSE) is applied as the target function Error(t), as follows [67]:
⁀
O(t)=y1∗w1+y2∗w2(9)
Error (t)=N
X
t=1hO(t)−
⁀
O(t)i2/N(10)
where, O(t)is the actual traffic volume data;
⁀
O(t)is the fore-
casted traffic volume data; Nis the number of samples; y1is the
predicted result of the GCN; and y2is the predicted result of the
GAT.
The reward of agent is determined by the MSE obtained by the
model for taking the action. The rewards are determined as fol-
lows:
R(t)=(+1+ERR (t−1)−ERR (t)(ERR(t−1)≥ERR (t))
−1+ERR (t−1)−ERR (t)(ERR (t−1)<ERR(t))
(11)
The update criteria for the DQN are similar to those for Q-Learning.
They all predict each action according to the current environment,
and update it according to the actual situation of the next envi-
ronment [68]. In the updating process of the DQN, the network
parameters are updated based on the error between the predicted
value of each action of the current environment and the actual
situation of the next environment [69]. The basic parameters of
DQN-GCN-GAT algorithm are shown in Table 1. In our experiments,
these parameters can achieve the optimal effects.
3. Case study
3.1. Dataset
To verify the practicability and generalization performance of
the proposed freeway traffic volume prediction model, this study
collected the actual traffic volume data of nine stations of Chang-
sha freeway. Data from each site were sampled from June to Au-
gust 2020. The time interval of each data was 1 hour, and the
number of sampling points was 1500. The spatial correlations be-
tween the nine sites are shown in Fig. 4. Different colors represent
different areas divided by Changsha, and the data of the same color
comes from the same area. Considering the location of sites 1, 2
and 3 in the relative core of the freeway network, selecting them
as the target sites can effectively evaluate the modeling effect of
the graph neural network. Therefore, the datasets collected from
sites 1, 2, and 3 were used as the main forecasting objects, and
the datasets collected from the other sites were utilized as input
features to help build the GAT and GCN. In order to ensure the
scientific nature of the model results, we adopted the method of
ten-fold cross-validation. In addition, we also carried out 10 re-
peated tests, and took the average value of the error evaluation
index as the result of evaluating the model performance. All case
studies were mainly conducted on the Python 3.6.5 platform.
Table 1
The basic parameters of DQN-GCN-GAT algorithm.
Name of parameter Selected parameter
DQN
Learning rate 0.95
Optimizer Adam
Initial ε-greedy value 1.0
Final ε-greedy value 0.1
Batch size 32
Maximum episode 100
Discount factor 0.99
GRU
Learning rate 0.01
Optimizer Adam
Number of hidden layers 2
Training Epochs 200
Number of hidden layer units 128, 64
Number of output layer units 1
GAT
The number of parameters 96768
Learning rate 0.01
Optimizer Adam
Loss function Root mean square error
Training Epochs 200
History length 7
GCN
The number of parameters 80703
Learning rate 0.01
Optimizer Adam
Loss function Root mean square error
Training Epochs 200
History length 7
Fig. 4. Spatial correlations between all sites.
3.2. Data preprocessing
The datasets collected at each site included the following data:
traffic volume, traffic speed, weather, and holidays. To effectively
help the proposed integrated model to better analyze the nonlin-
earity between the data and improve the prediction accuracy, data
preprocessing was conducted based on the following perspectives.
(1) Traffic volume and traffic speed: In time series prediction
modeling, one-dimensional data is usually transformed into
multidimensional data to construct the format required for su-
pervised learning. Here, the sliding window method was used
to preprocess the raw traffic volume and traffic speed data.
This method could transform the original 1 ×Ndata into two
sets of data: M×(N−M)input feature data, and 1 ×(N−M)
label data. In this paper, Nis 1500 and Mis 7. In addition, the
raw data was standardized.
(2) Weather: The weather data used by each node is to measure
the weather conditions of the region where the site is located.
The weather data mainly contained the following types: clear,
6
ARTICLE IN PRESS
JID:YDSPR AID:103419 /FLA [m5G; v1.312] P.7 (1-13)
P. Shang, X. Liu, C. Yu et al. Digital Signal Processing ••• (••••)••••••
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
few clouds, broken clouds, clouds overcast clouds, scattered
clouds, haze, fog, mist, light intensity drizzle, light rain, mod-
erate rain, and proximity thunderstorm with rain. Since the
original form of these data is English words, the data needs to
be converted into a number format that can be recognized by
the neural network. Therefore, these data were preprocessed
by setting labels (clear: 0, few clouds: 11, broken clouds: 12,
clouds overcast clouds: 13, scattered clouds: 14, haze: 21, fog:
22, mist: 23, light intensity drizzle: 31, light rain: 32, moderate
rain: 33, proximity thunderstorm with rain: 34). The current
weather status data of different sites are also used as input to
the model. The weather conditions in the area where each site
is located was the weather features of that site.
(3) Holidays: This data mainly contained three main data types:
working days, rest days, and holidays. The original forms of
these three kinds of data are also English words. Therefore,
these three types of data were converted into corresponding
labels to simplify the data and assist the predictors in recog-
nition (working days: 0, rest days: 1, holidays: 2). The current
holidays data of different sites are also used as input to the
model. City holiday information will be as the feature of all
sites.
3.3. Performance evaluation indexes
A regression analysis index can reflect the ability of a predic-
tion algorithm to model and analyze data. In this study, three
traditional and classic regression analysis indexes, i.e., the mean
absolute error (MAE), mean absolute percentage error (MAPE), and
root mean square error (RMSE) were utilized to accurately show
the predictive power of all of the models. The promoting percent-
ages of the MAE (PMAE), RMSE (PRMSE ), and MAPE (PMAPE) were
used to analyze and compare the advantages and disadvantages of
the different models. These indexes are defined as follows [70]:
MAE =(
N
X
t=1O(t)−ˆ
O(t))/N(12)
RMSE =v
u
u
t(
N
X
t=1hO(t)−ˆ
O(t)i2
)/N(13)
MAPE =(
N
X
m=1(O(t)−ˆ
O(t))/O(t))/N(14)
PMAE =(MAE1−MAE2)/MAE1∗100% (15)
PRMSE =(RMSE1−RMSE2)/RMSE1∗100% (16)
PMAPE =(MAPE1−MAPE2)/MAPE1∗100% (17)
where, O(t)is the actual traffic volume data, ˆ
O(t)is the forecasted
traffic volume data, and Nis the number of samples.
3.4. Contrast experiment with alternative models
3.4.1. Contrast experiment of different neural networks
To fully analyze the GCN and GAT modeling and prediction ef-
fects, this study considered other seven types of neural networks
(Chebnet, CNN, LSTM, DBN, RNN, ESN and multi-layer perceptron
(MLP)), to provide a comparative analysis of these two types of
GNNs. From Table 2, the following conclusions can be drawn.
(1) The forecasting error of the MLP is higher than that of the
other deep neural networks and GNNs. The experiment proves
that a deep neural network has a better modeling ability for
Table 2
The error evaluation results.
Dataset Forecasting
models
MAE
(Volume/h)
MAPE
(%)
RMSE
(Volume/h)
Site 1 GCN 79.4564 7.2392 121.7641
GAT 78.7372 7.2247 121.3774
ChebNet 80.0301 7.2926 124.4544
CNN 83.7462 7.4156 123.7831
LSTM 88.2485 7.9688 134.3544
DBN 85.3547 7.7372 138.4732
RNN 89.3477 8.1854 131.4815
ESN 86.5347 8.2712 123.3547
MLP 103.4353 9.3544 150.3453
Site 2 GCN 85.7516 8.3133 112.6290
GAT 85.3150 8.3277 112.8088
ChebNet 85.7598 8.3834 112.9558
CNN 88.7832 8.4378 116.4377
LSTM 87.3783 8.4257 114.7737
DBN 89.3737 8.5271 115.4979
RNN 89.8343 8.5437 115.63437
ESN 106.9949 10.3738 138.2573
MLP 114.4373 11.2420 149.3543
Site 3 GCN 62.3434 8.4254 84.2435
GAT 62.2434 8.4043 84.4348
ChebNet 63.2754 8.6354 85.3541
CNN 64.4884 9.1583 85.9347
LSTM 67.3239 8.9344 90.1151
DBN 65.3454 8.7783 87.3437
RNN 70.4357 9.3473 95.3778
ESN 75.2787 10.2577 106.3782
MLP 81.0553 11.2587 108.8343
the spatiotemporal traffic volume than a shallow neural net-
work. One possible reason is that the deep learning network
can extract the depth features of the spatiotemporal traffic
data in more detail, which improves the overall analytical ca-
pability of the forecasting model.
(2) When comparing GNNs with other deep networks, GNNs can
obtain better forecasting results. A possible reason is that
these GNNs use a graph principle to analyze the spatio-
temporal correlations between the nine sites, and to establish
excellent spatio-temporal forecasting models. Therefore, GNNs
provide strong adaptability in the field of spatiotemporal traf-
fic volume forecasting.
(3) The forecasting accuracies of the GCN and GAT are higher than
those of alternative neural networks. This fully proves the ex-
cellent abilities of these two types of GNNs in the field of
spatio-temporal traffic volume forecasting. In particular, the
GCN, based on space, utilizes graph pooling and graph con-
volution operations to manage the spatio-temporal correla-
tions between nodes, effectively improving the accuracy of the
traffic volume prediction from the GCN model. In contrast,
by introducing the structure of an attention mechanism, the
GAT gives greater weight to the more important nodes, effec-
tively improving the quality of the network input features, and
further improving the prediction effect. Nevertheless, experi-
ments show that these two types of GNN have their own mod-
eling advantages for different data. Therefore, it is necessary to
use ensemble learning to improve the overall adaptability of
the models.
3.4.2. Contrast experiment of different ensemble methods
To prove that ensemble learning can further improve the ap-
plication value and generalization performance of the neural net-
works, the proposed DQN-GCN-GAT was compared with the two
types of single neural networks. In addition, to fully prove that
the DQN has excellent decision making and ensemble capabilities,
six other ensemble methods, based on Sarsa, Q-learning, ICA, PSO,
and genetic algorithm approaches, were compared with the DQN.
7
ARTICLE IN PRESS
JID:YDSPR AID:103419 /FLA [m5G; v1.312] P.8 (1-13)
P. Shang, X. Liu, C. Yu et al. Digital Signal Processing ••• (••••)••••••
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
Fig. 5. Scatter plots of traffic volume forecasting results (Reinforcement learning).
Fig. 6. Scatter plots of traffic volume forecasting results (Heuristic algorithm).
Fig. 5and Fig. 6respectively represent the Scatter plots of traffic
volume forecasting results of Reinforcement learning and Heuristic
algorithm. From Tables 3–5and Figs. 5-6, the following conclusions
can be drawn.
(1) The forecasting accuracies of ensemble learning models are
better than those of single GNNs. It is proven that ensemble
networks can comprehensively improve the overall robustness
and generalization performance of a model, and can effectively
reduce the forecasting error in a traffic volume forecasting
model.
(2) All of the RL methods can achieve better ensemble effects than
heuristic algorithms. This proves that the RL algorithm has ex-
cellent decision-making abilities based on weight coefficients
in ensemble learning. One possible reason for this is that RL
can continuously improve the ability of the agent to analyze
8
ARTICLE IN PRESS
JID:YDSPR AID:103419 /FLA [m5G; v1.312] P.9 (1-13)
P. Shang, X. Liu, C. Yu et al. Digital Signal Processing ••• (••••)••••••
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
Table 3
The error evaluation results.
Dataset Forecasting
models
MAE
(Volume/h)
MAPE
(%)
RMSE
(Volume/h)
Site 1 DQN-GCN-GAT 76.2013 6.9248 118.1276
Sarsa-GCN-GAT 77.3437 7.0246 118.5861
Q-GCN-GAT 77.2434 7.0140 118.2434
ICA-GCN-GAT 77.8330 7.0837 120.1244
PSO-GCN-GAT 77.6787 7.0777 120.4534
GA-GCN-GAT 77.8343 7.1345 119.2453
Site 2 DQN-GCN-GAT 80.9526 7.9644 103.5978
Sarsa-GCN-GAT 83.0345 8.1257 106.4538
Q-GCN-GAT 83.1835 8.1133 107.3458
ICA-GCN-GAT 83.4105 8.2377 108.2453
PSO-GCN-GAT 83.8518 8.2783 108.8767
GA-GCN-GAT 83.6012 8.2247 108.3437
Site 3 DQN-GCN-GAT 58.6279 7.6867 81.3107
Sarsa-GCN-GAT 58.8344 7.7753 81.3575
Q-GCN-GAT 59.2473 7.7945 82.1057
ICA-GCN-GAT 60.8344 7.8377 83.3453
PSO-GCN-GAT 59.6486 7.8284 83.1347
GA-GCN-GAT 61.2577 8.0453 83.2573
Table 4
The promoting percentages of the DQN by other ensemble methods.
Method Indexes Series #1 Series #2 Series #3
DQN-GCN-GAT
vs.
Sarsa-GCN-GAT
PMAE (%) 1.4770 2.5073 0.3510
PMAPE (%) 1.4207 1.9851 1.1395
PRMSE (%) 0.3866 2.6829 0.0575
DQN-GCN-GAT
vs.
Q-GCN-GAT
PMAE (%) 1.3491 2.6819 1.0454
PMAPE (%) 1.2717 1.8353 1.3830
PRMSE (%) 0.0979 3.4915 0.9683
DQN-GCN-GAT
vs.
ICA-GCN-GAT
PMAE (%) 2.0964 2.9468 3.6271
PMAPE (%) 2.2432 3.3177 1.9266
PRMSE (%) 1.6623 4.2935 2.4412
DQN-GCN-GAT
vs.
PSO-GCN-GAT
PMAE (%) 1.9019 3.4575 1.7112
PMAPE (%) 2.1603 3.7918 1.8101
PRMSE (%) 1.9309 4.8485 2.1940
DQN-GCN-GAT
vs.
GA-GCN-GAT
PMAE (%) 2.0980 3.1681 4.2930
PMAPE (%) 2.9392 3.1649 4.4573
PRMSE (%) 0.9373 4.3804 2.3381
Table 5
The promoting percentages of the DQN-GCN-GAT by single models.
Method Indexes Series #1 Series #2 Series #3
DQN-GCN-GAT
vs.
GAT
PMAE (%) 4.0967 5.5964 5.9597
PMAPE (%) 4.3430 4.1969 8.7675
PRMSE (%) 2.9865 8.0185 3.4813
DQN-GCN-GAT
vs.
GCN
PMAE (%) 3.2207 5.1133 5.8086
PMAPE (%) 4.1510 4.3625 8.5385
PRMSE (%) 2.6774 8.1651 3.7000
the correlations between weight coefficients and output re-
sults, so as select the global optimal result during the process
of training the agent.
(3) When comparing the DQN with traditional Q-table based RL, it
is shown that the DQN can better analyze the correlations be-
tween weight coefficients and predictors, and can achieve the
best ensemble results. This is because, as compared with the
traditional RL based on a Q-table, the neural network struc-
ture of deep RL can better store the state-action of the agent,
effectively improving the ability of the agent to optimize and
analyze the weight coefficient.
3.4.3. Contrast experiment of different components
In order to verify the excellent practical value of DQN-GCN-
GAT algorithm, this paper replaced different predictor compo-
Table 6
The error evaluation results.
Dataset Forecasting
models
MAE
(Volume/h)
MAPE
(%)
RMSE
(Volume/h)
Site 1 DQN-GCN-GAT 76.2013 6.9248 118.1276
DQN-GCN-GAT-CNN 77.3437 7.0345 119.2437
DQN-GCN-GAT-ChebNet 77.3344 7.0786 119.3453
DQN-GCN-GAT-ChebNet-CNN 77.5437 7.0243 119.4536
Site 2 DQN-GCN-GAT 80.9526 7.9644 103.5978
DQN-GCN-GAT-CNN 81.4561 8.0786 104.4534
DQN-GCN-GAT-ChebNet 81.1316 8.0456 104.3454
DQN-GCN-GAT-ChebNet-CNN 81.4105 8.0446 104.7341
Site 3 DQN-GCN-GAT 58.6279 7.6867 81.3107
DQN-GCN-GAT-CNN 59.6253 7.7754 81.8234
DQN-GCN-GAT-ChebNet 59.7837 7.7574 81.9677
DQN-GCN-GAT-ChebNet-CNN 59.3437 7.7254 81.8738
Fig. 7. MAEs of these state-of-the-art and classic models.
Fig. 8. MAPEs of these state-of-the-art and classic models.
Fig. 9. RMSEs of these state-of-the-art and classic models.
9
ARTICLE IN PRESS
JID:YDSPR AID:103419 /FLA [m5G; v1.312] P.10 (1-13)
P. Shang, X. Liu, C. Yu et al. Digital Signal Processing ••• (••••)••••••
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
Fig. 10. Final forecasting results and residual error of all involved models (site 1).
Fig. 11. Final forecasting results and residual error of all involved models (site 2).
nents to further evaluate the performance of the model. In this
section, DQN-GCN-GAT algorithm is compared with other three
ensemble frameworks, including DQN-GCN-GAT-CNN, DQN-GCN-
GAT-ChebNet and DQN-GCN-GAT-ChebNet-CNN. From Table 6, the
following conclusions can be drawn.
(1) Compared with other ensemble frameworks, DQN-GCN-GAT
proposed in this paper can achieve the best prediction results.
It is proved that the integrated structure proposed in this pa-
per can achieve excellent prediction results in spatio-temporal
traffic volume prediction.
(2) Increasing the number of ensemble model components did not
further improve the predictive performance of the model. The
possible reason is that the modeling effect of GAT and GCN
is better than that of ChebNet and CNN, which makes DQN
provide more weight information to GAT and GCN. Thus, even
with the addition of other components, the character energy
of the model does not improve.
3.5. Contrast experiment with existing models
Other scholars have proposed a variety of state-of-the-art and
traditional models, and have achieved excellent results in traffic
forecasting. To verify the advanced abilities and practicability of
the proposed model, two state-of-the-art models and two types
of classic models, i.e., contain Li’s model [17], Guo’s model [21],
the SVM, and the ARIMA, were reproduced and compared with the
DQN-GCN-GAT model. Figs. 7–9provide the MAE, RMSE, and MAPE
values of these state-of-the-art and classic models. Figs. 10–12
show the final forecasting results and residual errors of all involved
models. From Figs. 7–12, the following conclusions can be drawn.
(1) When comparing the state-of-the-art models with the classic
models, it is found that all of the state-of-the-art models can
obtain better forecasting results, showing that the state-of-the-
art models have better spatio-temporal traffic volume mod-
eling and analysis abilities than the classic algorithms. Com-
pared with the classic methods, these state-of-the-art models
improve the feature analysis and modeling capabilities by ef-
fectively aggregating the correlations between different sites,
effectively improving their forecasting accuracies.
(2) The ensemble DQN-GCN-GAT model presented in this study
achieves the best forecasting results in all of the case stud-
ies. The MAPEs of the DQN-GCN-GAT are all less than 8%.
The model uses the GCN and GAT models based on space
10
ARTICLE IN PRESS
JID:YDSPR AID:103419 /FLA [m5G; v1.312] P.11 (1-13)
P. Shang, X. Liu, C. Yu et al. Digital Signal Processing ••• (••••)••••••
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
Fig. 12. Final forecasting results and residual error of all involved models (site 3).
and temporal, which can effectively aggregate the spatiotem-
poral correlations of the traffic feature data between different
nodes. The ensemble learning method based on deep RL can
effectively analyze the intrinsic correlations between the fore-
casting results of different neural networks and real values
to intelligently optimize the weight coefficient. Therefore, the
model proposed in this study can achieve satisfactory predic-
tion results, and is superior to all existing alternative models.
Thus, this model has wide application value in the field of
spatio-temporal traffic volume forecasting.
4. Conclusion and future work
Spatio-temporal traffic volume prediction technologies for free-
ways provide technical references for alleviating traffic pressures
and ensuring travel quality. In this study, a new ensemble deep
graph RL network combining the GAT, GCN, and DQN based on
ensemble learning is proposed, aiming to build a spatio-temporal
traffic volume prediction model for a freeway network. This paper
can be summarized based on following three aspects.
Robustness: The proposed model can deeply analyze the deep
correlations of data among different nodes, making the model
more accurate when predicting the change trends of the traffic vol-
ume data at different nodes. The existing single models do not
have sufficient overall generalization performances, which limits
their modeling and prediction abilities. Therefore, the model pre-
sented in this paper has excellent robustness.
Accuracy: The DQN-GCN-GAT is compared with seventeen al-
ternative models and four existing models. All of the experimental
results show that the ensemble DQN-GCN-GAT model can achieve
the best forecasting accuracy in all cases. Its MAPE values are all
less than 8%. This fully proves that the model has excellent fore-
casting accuracy.
Practicability: The datasets are obtained from actual collected
traffic volume data of a freeway network. In addition, traffic in-
formation elements such as traffic speeds, weather, and holidays
are used as the input features to build the space-time prediction
model. All of the experimental results show that the DQN-GCN-
GAT model can obtain accurate and satisfactory prediction results.
Therefore, the model proposed in this paper has excellent practi-
cability.
In sum, the ensemble model proposed in this paper provides a
meaningful reference for the accurate forecasting of freeway net-
work traffic volumes. At present, intelligent big data platforms are
gradually becoming widely used in industry and academia [71].
In the future, the proposed model can be embedded into an in-
telligent big data platform to further improve the comprehensive
performance of the model, and to establish a complete intelligent
traffic management system.
CRediT authorship contribution statement
Shang Pan: Writing, Methodology.
Xinwei Liu: Data curation, Methodology.
Chengqing Yu: Software.
Guangxi Yan: Revision.
Qingqing Xiang: Revision.
Xiwei Mi: Conceptualization, Software.
Declaration of competing interest
The authors declare that they have no known competing finan-
cial interests or personal relationships that could have appeared to
influence the work reported in this paper.
Acknowledgments
This study is fully supported by National Natural Science Foun-
dation of China (52102471), China Postdoctoral Science Foundation
(2020M670127), National Key Research and Development Program
of China (2018YFB1201402), Fundamental Research Funds for the
Central Universities (2019RC057).
Appendix A. Model hyper parameters
The main hyper parameters of different algorithms are shown
in Table A.1.
References
[1] Y. Fan, B. Cai, W. Shangguan, J. Wang, D. Jiang, Multi-objective operation control
of rail vehicles, in: 2013 IEEE Intelligent Vehicles Symposium (IV), IEEE, 2013,
pp. 1265–1270.
[2] A. Karaer, M.B. Ulak, E.E. Ozguven, T. Sando, Reducing the non-recurrent free-
way congestion with detour operations: case study in Florida, Transp. Eng. 2
(2020) 100026.
[3] K. Niu, C. Cheng, J.L. Chang, H.Y. Zhang, T. Zhou, Real-time taxi-passenger pre-
diction with L-CNN, IEEE Trans. Veh. Technol. 68 (2019) 4122–4129.
[4] M. Lopez-Martin, B. Carro, A. Sanchez-Esguevillas, IoT type-of-traffic forecasting
method based on gradient boosting neural networks, Future Gener. Comput.
Syst. 105 (2020) 331–345.
11
ARTICLE IN PRESS
JID:YDSPR AID:103419 /FLA [m5G; v1.312] P.12 (1-13)
P. Shang, X. Liu, C. Yu et al. Digital Signal Processing ••• (••••)••••••
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
Table A.1
The main hyper parameters of different algorithms.
Name of parameter Selected parameter
Q-learning
Maximum iteration 50
Learning rate 0.95
Discount parameter 0.5
Sarsa
Maximum iteration 50
Learning rate 0.95
Discount parameter 0.5
ICA
Maximum iteration 50
Number of imperialist countries 10
Success rate of revolution 0.1
Coefficient of assimilation 1.5
Average cost factor 0.2
PSO
Maximum iteration 50
Inertia weight 0.8
Accelerated constant 2
(continued on next page)
[5] K. Anuar, M. Cetin, Estimating freeway traffic volume using shockwaves and
probe vehicle trajectory data, Transp. Res. Proc. 22 (2017) 183–192.
[6] H.-z. Wang, G.-q. Li, G.-b. Wang, J.-c. Peng, H. Jiang, Y.-t. Liu, Deep learning
based ensemble approach for probabilistic wind power forecasting, Appl. En-
ergy 188 (2017) 56–70.
[7] A. Picon, A. Alvarez-Gila, M. Seitz, A. Ortiz-Barredo, J. Echazarra, A. Johannes,
Deep convolutional neural networks for mobile capture device-based crop dis-
ease classification in the wild, Comput. Electron. Agric. 161 (2019) 280–290.
[8] G. Yan, C. Yu, Y. Bai, Wind turbine bearing temperature forecasting using a new
data-driven ensemble approach, Machines 9 (2021) 248.
[9] G. Yan, C. Yu, Y. Bai, A new hybrid ensemble deep learning model for train axle
temperature short term forecasting, Machines 9 (2021) 312.
[10] K. Gan, S. Sun, S. Wang, Y. Wei, A secondary-decomposition-ensemble learn-
ing paradigm for forecasting PM2.5 concentration, Atmos. Pollut. Res. 9 (2018)
989–999.
[11] A. Boukerche, Y. Tao, P. Sun, Artificial intelligence-based vehicular traffic flow
prediction methods for supporting intelligent transportation systems, Comput.
Netw. 182 (2020) 107484.
[12] X. Pu, G. Yan, C. Yu, X. Mi, C. Yu, Sentiment analysis of online course evaluation
based on a new ensemble deep learning mode: evidence from Chinese, Appl.
Sci. 11 (2021) 11313.
[13] L. Li, L. Qin, X. Qu, J. Zhang, Y. Wang, B. Ran, Day-ahead traffic flow forecast-
ing based on a deep belief network optimized by the multi-objective particle
swarm algorithm, Knowl.-Based Syst. 172 (2019) 1–14.
[14] Q. Zhang, H. Qian, Y. Chen, D. Lei, A short-term traffic forecasting model based
on echo state network optimized by improved fruit fly optimization algorithm,
Neurocomputing (2019).
[15] P. Sun, A. Boukerche, Y. Tao, SSGRU: a novel hybrid stacked GRU-based traffic
volume prediction approach in a road network, Comput. Commun. 160 (2020)
502–511.
[16] H. Lu, Z. Ge, Y. Song, D. Jiang, T. Zhou, J. Qin, A temporal-aware LSTM en-
hanced by loss-switch mechanism for traffic flow forecasting, Neurocomputing
427 (2021) 169–178.
[17] Y. Li, S. Chai, Z. Ma, G. Wang, A hybrid deep learning framework for long-term
traffic flow prediction, IEEE Access 9 (2021) 11264–11271.
[18] M.S. Chen, Z.B. Sun, J.M. Davis, C.S. Liu, W. Gao, Spatial interpolation of surface
ozone observations using deep learning, in: Conference on Remote Sensing and
Modeling of Ecosystems for Sustainability XV, San Diego, CA, 2018.
[19] H. Peng, H. Wang, B. Du, M.Z.A. Bhuiyan, H. Ma, J. Liu, L. Wang, Z. Yang, L. Du,
S. Wang, P.S. Yu, Spatial temporal incidence dynamic graph neural networks for
traffic flow forecasting, Inf. Sci. 521 (2020) 277–290.
[20] B. Yu, Y. Lee, K. Sohn, Forecasting road traffic speeds by considering area-wide
spatio-temporal dependencies based on a graph convolutional neural network
(GCN), Transp. Res., Part C, Emerg. Technol. 114 (2020) 189–204.
[21] G. Guo, W. Yuan, Short-term traffic speed forecasting based on graph attention
temporal convolutional networks, Neurocomputing 410 (2020) 387–393.
[22] C. Jin, T. Ruan, D. Wu, L. Xu, T. Dong, T. Chen, S. Wang, Y. Du, M. Wu, HetGAT:
a heterogeneous graph attention network for freeway traffic speed prediction,
J. Ambient Intell. Humaniz. Comput. (2021).
[23] X. Zhang, R.X. Cao, Z.Y. Zhang, Y. Xia, Crowd flow forecasting with multi-graph
neural networks, in: 2020 International Joint Conference on Neural Networks
(IJCNN), IEEE, 2020.
[24] X. Liu, M. Qin, Y. He, X. Mi, C. Yu, A new multi-data-driven spatiotemporal
PM2.5 forecasting model based on an ensemble graph reinforcement learning
convolutional network, Atmos. Pollut. Res. 12 (2021) 101197.
Table A.1 (continued)
Name of parameter Selected parameter
GA
Maximum iteration 50
Crossover rate 0.3
Mutation rate 0.1
LSTM
Learning rate 0.01
Optimizer Adam
History length 7
Number of hidden layers 2
Training Epochs 200
Number of hidden layer units 128, 64
Number of output layer units 1
ChebNet
Learning rate 0.01
Optimizer Adam
Loss function Root mean square error
Training Epochs 200
History length 7
RNN
Learning rate 0.01
Optimizer Adam
History length 7
Number of hidden layers 2
Training Epochs 200
Number of hidden layer units 128, 64
DBN
History length 7
Size of hidden units 32
Training Epochs 200
Size of output units 1
Momentum factor 0.0
Learning rate 0.01
ESN
History length 7
Size of hidden units 64
Training Epochs 200
Size of output units 1
MLP
History length 7
Training Epochs 200
Size of hidden units 32
Learning rate 0.01
Size of output units 1
[25] L.Y. Gong, Q. Cheng, I.C. Soc, Exploiting edge features for graph neural net-
works, in: IEEE/CVF Conference on Computer Vision and Pattern Recognition
(CVPR), Long Beach, CA, 2019, pp. 9203–9211.
[26] E. Shikov, M. Bardina, D. Vaganov, Predicting customer flow for the selected
location based on transactional data, Proc. Comput. Sci. 193 (2021) 361–370.
[27] J.J.Q. Yu, Citywide traffic speed prediction: a geometric deep learning approach,
Knowl.-Based Syst. 212 (2021).
[28] Y. Bai, Y. Li, B. Zeng, C. Li, J. Zhang, Hourly PM2.5 concentration forecast using
stacked autoencoder model with emphasis on seasonality, J. Clean. Prod. 224
(2019) 739–750.
[29] M. Pulido, P. Melin, O. Castillo, Particle swarm optimization of ensemble neu-
ral networks with fuzzy aggregation for time series prediction of the Mexican
stock exchange, Inf. Sci. 280 (2014) 188–204.
[30] J. Zhang, Y. Wei, Z. Tan, An adaptive hybrid model for short term wind speed
forecasting, Energy 190 (2020) 115615.
[31] M. Niu, Y. Wang, S. Sun, Y. Li, A novel hybrid decomposition-and-ensemble
model based on CEEMD and GWO for short-term PM2. 5 concentration fore-
casting, Atmos. Environ. 134 (2016) 168–180.
[32] C. Li, Z. Zhu, H. Yang, R. Li, An innovative hybrid system for wind speed fore-
casting based on fuzzy preprocessing scheme and multi-objective optimization,
Energy 174 (2019) 1219–1237.
[33] Y. Wang, Q. Hu, D. Srinivasan, Z. Wang, Wind power curve modeling and wind
power forecasting with inconsistent data, IEEE Trans. Sustain. Energy 10 (2018)
16–25.
[34] S. Dong, C. Yu, G. Yan, J. Zhu, H. Hu, A novel ensemble reinforcement learning
gated recursive network for traffic speed forecasting, in: 2021 Workshop on
Algorithm and Big Data, 2021, pp. 55–60.
[35] P.-Y. Yin, C.-H. Chao, Automatic selection of fittest energy demand predictors
based on cyber swarm optimization and reinforcement learning, Appl. Soft
Comput. 71 (2018) 152–164.
12
ARTICLE IN PRESS
JID:YDSPR AID:103419 /FLA [m5G; v1.312] P.13 (1-13)
P. Shang, X. Liu, C. Yu et al. Digital Signal Processing ••• (••••)••••••
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
[36] T. Wang, J. Cao, A. Hussain, Adaptive traffic signal control for large-scale
scenario with cooperative group-based multi-agent reinforcement learning,
Transp. Res., Part C, Emerg. Technol. 125 (2021) 103046.
[37] M. Abdoos, A .L.C. Bazzan, Hierarchical traffic signal optimization using rein-
forcement learning and traffic prediction with long-short term memory, Expert
Syst. Appl. 171 (2021) 114580.
[38] V. Mnih, K. Kavukcuoglu, D. Silver, A.A. Rusu, J. Veness, M.G. Bellemare, A.
Graves, M. Riedmiller, A.K. Fidjeland, G. Ostrovski, Human-level control through
deep reinforcement learning, Nature 518 (2015) 529.
[39] I. Momennejad, E.M. Russek, J.H. Cheong, M.M. Botvinick, N.D. Daw, S.J. Gersh-
man, The successor representation in human reinforcement learning, Nat. Hum.
Behav. 1 (2017) 680.
[40] C. Shen, Y. Gonzalez, L. Chen, S.B. Jiang, X. Jia, Intelligent parameter tuning in
optimization-based iterative CT reconstruction via deep reinforcement learning,
IEEE Trans. Med. Imaging 37 (2018) 1430–1439.
[41] Y. Ding, L. Ma, J. Ma, M. Suo, L. Tao, Y. Cheng, C. Lu, Intelligent fault diagnosis
for rotating machinery using deep Q-network based health state classification:
a deep reinforcement learning approach, Adv. Eng. Inform. 42 (2019) 100977.
[42] Y. Deng, F. Bao, Y. Kong, Z. Ren, Q. Dai, Deep direct reinforcement learning
for financial signal representation and trading, IEEE Trans. Neural Netw. Learn.
Syst. 28 (2016) 653–664.
[43] T. Pan, R. Guo, W.H.K. Lam, R. Zhong, W. Wang, B. He, Integrated optimal con-
trol strategies for freeway traffic mixed with connected automated vehicles:
a model-based reinforcement learning approach, Transp. Res., Part C, Emerg.
Technol. 123 (2021) 102987.
[44] Y. Yin, Y. Bai, F. Ge, H. Yu, Y. Liu, Long-term robust identification potential of
a wavelet packet decomposition based recursive drift correction of E-nose data
for Chinese spirits, Measurement 139 (2019) 284–292.
[45] S. Fukuyama, Dynamic game-based approach for optimizing merging vehicle
trajectories using time-expanded decision diagram, Transp. Res., Part C, Emerg.
Technol. 120 (2020) 102766.
[46] J. Tang, J. Liang, F. Liu, J. Hao, Y. Wang, Multi-community passenger demand
prediction at region level based on spatio-temporal graph convolutional net-
work, Transp. Res., Part C, Emerg. Technol. 124 (2021) 102951.
[47] W. Chen, M. Jiang, W.-G. Zhang, Z. Chen, A novel graph convolutional feature
based convolutional neural network for stock trend prediction, Inf. Sci. 556
(2021) 67–94.
[48] T. Ma, H. Wang, L. Zhang, Y. Tian, N. Al-Nabhan, Graph classification based on
structural features of significant nodes and spatial convolutional neural net-
works, Neurocomputing 423 (2021) 639–650.
[49] J.W. Lee, W.K. Lee, S.Y. Sohn, Graph convolutional network-based credit default
prediction utilizing three types of virtual distances among borrowers, Expert
Syst. Appl. 168 (2021) 114411.
[50] G. Zhao, P. Jia, A. Zhou, B. Zhang, InfGCN: identifying influential nodes in com-
plex networks with graph convolutional networks, Neurocomputing 414 (2020)
18–26.
[51] J. Lee, R. Asahi, Transfer learning for materials informatics using crystal graph
convolutional neural network, Comput. Mater. Sci. 190 (2021) 110314.
[52] Y.-D. Zhang, S.C. Satapathy, D.S. Guttery, J.M. Górriz, S.-H. Wang, Improved
breast cancer classification through combining graph convolutional network
and convolutional neural network, Inf. Process. Manag. 58 (2021) 102439.
[53] H. Taguchi, X. Liu, T. Murata, Graph convolutional networks for graphs contain-
ing missing features, Future Gener. Comput. Syst. 117 (2021) 155–168.
[54] Z. Yang, S. Dong, HAGERec: hierarchical attention graph convolutional network
incorporating knowledge graph for explainable recommendation, Knowl.-Based
Syst. 204 (2020) 106194.
[55] L. Yang, W. Li, Y. Guo, J. Gu, Graph-CAT: graph co-attention networks via local
and global attribute augmentations, Future Gener. Comput. Syst. 118 (2021)
170–179.
[56] P.N. Chowdhury, P. Shivakumara, S. Kanchan, R. Raghavendra, U. Pal, T. Lu, D.
Lopresti, Graph attention network for detecting license plates in crowded street
scenes, Pattern Recognit. Lett. 140 (2020) 18–25.
[57] C. Wen, X. Li, X. Yao, L. Peng, T. Chi, Airborne LiDAR point cloud classification
with global-local graph attention convolution neural network, ISPRS J. Pho-
togramm. Remote Sens. 173 (2021) 181–194.
[58] H. Xu, W. Jun, J. Wei, Recommending irregular regions using graph attentive
networks, Ad Hoc Netw. 113 (2021) 102383.
[59] F. Zhou, Q. Yang, T. Zhong, D. Chen, N. Zhang, Variational graph neural net-
works for road traffic prediction in intelligent transportation systems, IEEE
Trans. Ind. Inform. 17 (2021) 2802–2812.
[60] F. Wang, J.-F. Yang, M.-Y. Wang, C.-Y. Jia, X.-X. Shi, G.-F. Hao, G.-F. Yang, Graph
attention convolutional neural network model for chemical poisoning of honey
bees’ prediction, Sci. Bull. 65 (2020) 1184–1191.
[61] J. Zheng, Y. Wang, W. Xu, Z. Gan, P. Li, J. Lv, GSSA: pay attention to graph
feature importance for GCN via statistical self-attention, Neurocomputing 417
(2020) 458–470.
[62] S. Luo, Dynamic scheduling for flexible job shop with new job insertions by
deep reinforcement learning, Appl. Soft Comput. 91 (2020) 106208.
[63] Y. Du, F. Li, J. Munk, K. Kurte, O. Kotevska, K. Amasyali, H. Zandi, Multi-task
deep reinforcement learning for intelligent multi-zone residential HVAC con-
trol, Electr. Power Syst. Res. 192 (2021) 106959.
[64] Y. Du, H. Zandi, O. Kotevska, K. Kurte, J. Munk, K. Amasyali, E. McKee, F. Li,
Intelligent multi-zone residential HVAC control strategy based on deep rein-
forcement learning, Appl. Energy 281 (2021) 116117.
[65] R. Lian, J. Peng, Y. Wu, H. Tan, H. Zhang, Rule-interposing deep reinforcement
learning based energy management strategy for power-split hybrid electric ve-
hicle, Energy 197 (2020) 117297.
[66] X. Wu, H. Chen, C. Chen, M. Zhong, S. Xie, Y. Guo, H. Fujita, The autonomous
navigation and obstacle avoidance for USVs with ANOA deep reinforcement
learning method, Knowl.-Based Syst. 196 (2020) 105201.
[67] Z. Miljkovi ´
c, M. Miti´
c, M. Lazarevi´
c, B. Babi´
c, Neural network reinforcement
learning for visual control of robot manipulators, Expert Syst. Appl. 40 (2013)
1721–1736.
[68] Y. Li, B. Yang, L. Yan, W. Gao, Energy-aware resource management for uplink
non-orthogonal multiple access: multi-agent deep reinforcement learning, Fu-
ture Gener. Comput. Syst. 105 (2020) 684–694.
[69] G. Guo, Y. Wang, An integrated MPC and deep reinforcement learning approach
to trams-priority active signal control, Control Eng. Pract. 110 (2021) 104758.
[70] A. Rossi, G. Barlacchi, M. Bianchini, B. Lepri, Modelling taxi drivers’ behaviour
for the next destination prediction, IEEE Trans. Intell. Transp. Syst. 21 (2020)
2980–2989.
[71] Y. Zhao, H. Zhang, L. An, Q. Liu, Improving the approaches of traffic demand
forecasting in the big data era, Cities 82 (2018) 19–26.
PAN SHANG received the Ph.D. degree in civil en-
gineering from Tsinghua University, Beijing, China, in
2019. He is currently an Associate Professor in the
School of Traffic and Transportation, Beijing Jiaotong
University, Beijing, China. His main research inter-
ests include intelligent transportation system, train
timetabling, and service network design.
XINWEI LIU received the B.S. degree in Industry
Engineering from Beijing Jiaotong University, Beijing,
China, in 2019. He is currently pursuing the M.S.
degree in Mechanical, Electronic and Control Engi-
neering, Beijing Jiaotong University, Beijing, China. His
main research interests include deep learning, opera-
tions research, and optimization algorithm.
CHENGQING YU received the B.S. degree in Trans-
port Equipment and Control Engineering from Central
South University, Changsha, China, in 2019. He is cur-
rently pursuing the M.S. degree in Traffic and Trans-
portation Engineering with Central South University,
Changsha, China. His main research interests include
deep learning, reinforcement learning, and data min-
ing.
GUANGXI YAN received the B.S. degree from Wuh-
an University of technology, China, in 2010, and the
M.S. degree from Karlsruhe Institute of Technology,
Germany, in 2015, respectively. He is now PHD stu-
dent in School of Traffic and Transportation Engineer-
ing, Central South University, China. His research in-
terests include intelligent traffic system, fault analysis
and data mining.
QINGQING XIANG is currently pursuing a bache-
lor’s degree in the School of Traffic and Transportation
in East China Jiaotong University. Her may research
interests include optimization of the railway and in-
termodal services and intelligent transportation sys-
tem.
XIWEI MI received the Ph.D. degree in trans-
portation engineering from Central South University,
Changsha, China, in 2019. He is currently an Associate
Professor in the School of Traffic and Transportation,
Beijing Jiaotong University, Beijing, China. His main
research interests include intelligent transportation
system, automatic driving, artificial intelligence and
signal processing.
13
ARTICLE IN PRESS
JID:YDSPR AID:103419 /FLA [m5G; v1.312] P.14 (1-13)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
Sponsor names
Do not correct this page. Please mark corrections to sponsor names and grant numbers in the main text.
National Natural Science Foundation of China,country=China, grants=52102471
China Postdoctoral Science Foundation,country=China, grants=2020M670127
National Key Research and Development Program of China,country=China, grants=2018YFB1201402
Fundamental Research Funds for the Central Universities,country=China, grants=2019RC057
ARTICLE IN PRESS
JID:YDSPR AID:103419 /FLA [m5G; v1.312] P.15 (1-13)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
Highlights
•A spatiotemporal traffic volume forecasting model is proposed in freeway network.
•GCN and GAT is used as the main predictor.
•DQN is used to integrate GCN and GAT.
•The proposed model is compared with twenty-one mainstream forecasting models.
