Content uploaded by Haotian Shi
Author content
All content in this area was uploaded by Haotian Shi on Feb 24, 2024
Content may be subject to copyright.
1
SUBMITTED TO SUSTAINABLE CITIES AND SOCIETY
1
1Enhancing Urban Traffic Sustainability: Anomaly Detection in Time Series
2Incorporating Spatial Correlations and Temporal Evolutions
3
4Jiannan Mao
51 Department of Civil and Environmental Engineering
6 University of Wisconsin-Madison
7 1217 Engineering Hall 1415 Engineering Drive, Madison, WI, United States of America, 53705
82 School of Transportation and Logistics, Southwest Jiaotong University, 111 the Second Ring Road North,
9 Chengdu, Sichuan, China 610031
10 Email: jmao56@wisc.edu
11
12 Lan Liu
13 1School of Transportation and Logistics
14 2National and Local Joint Engineering Laboratory of Integrated Intelligent Transportation
15 Southwest Jiaotong University, 111 the Second Ring Road North, Chengdu, Sichuan, China 610031
16 Email: jiannan_l@home.swjtu.edu.cn
17
18 Bin Ran
19 Department of Civil and Environmental Engineering
20 University of Wisconsin-Madison
21 1217 Engineering Hall 1415 Engineering Drive, Madison, WI, United States of America, 53705
22 Email: bran@engr.wisc.edu
23
24 Hao Huang
25 School of Transportation and Logistics
26 Southwest Jiaotong University
27 111 the Second Ring Road North, Chengdu, Sichuan, China 610031
28 Email: haohuang_h@163.com
29
30 Tianli Tang
31 School of Transportation
32 Southeast University, Nanjing, Jiangsu, China
33 Email: T-Tang@seu.edu.cn
34
35 Weike Lu
36 1School of Rail Transportation, Soochow University, No. 8 Jixue Rd., Soochow, Jiangsu, China, 215131
37 2Alabama Transportation Institute, 248 Kirkbride Lane, Tuscaloosa, AL 35487
38 Email: wklu@suda.edu.cn
39
40 Haotian Shi (corresponding author)
41 Department of Civil and Environmental Engineering
42 University of Wisconsin-Madison
43 1217 Engineering Hall 1415 Engineering Drive, Madison, WI, United States of America, 53705
44 Email: hshi84@wisc.edu
45
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4737146
Preprint not peer reviewed
1
SUBMITTED TO SUSTAINABLE CITIES AND SOCIETY
1
Enhancing Urban Traffic Sustainability: Anomaly Detection in Time Series
Incorporating Spatial Correlations and Temporal Evolutions
Abstract
The efficiency of Intelligent Transportation Systems (ITS) in smart cities hinges on the accurate
identification of traffic anomalies, with traffic time series data serving as the primary data source.
However, the ever-increasing amount of data poses challenges to traffic practitioners in effectively
detecting or labeling anomalies, due to the prohibitive cost of manual calibration and the complex
spatial and temporal dynamics of urban traffic. Addressing this challenge, this paper investigates
the intrinsic spatial dependencies and temporal evolution within traffic time-series data, leading to
the development of two independent, data-driven anomaly detection algorithms for spatial and
temporal anomaly detection. The spatial anomaly detection algorithm employs the Maximal
Information Coefficient approach to identify abnormal roads with unusual traffic patterns. The
temporal anomaly detection algorithm leverages the mechanism of temporal evolution dynamics
in the Markov Transition Field, catering to analytical needs across different time granularities.
Validation is conducted on four real-world traffic speed datasets, and the results confirm the
effectiveness of the proposed algorithms. By simplifying data requirements and harnessing spatial-
temporal insights, the proposed anomaly detection algorithms can identify spatial and temporal
anomalies with just time-series data, representing a step towards more efficient and adaptable
traffic management systems for practical use.
Keywords: Anomaly Detection, Data-driven, Intelligent Transportation System (ITS), Time
Series
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4737146
Preprint not peer reviewed
2
SUBMITTED TO SUSTAINABLE CITIES AND SOCIETY
2
1. Introduction
Intelligent Transportation Systems (ITS) are crucial components of smart cities, aiding in
providing effective traffic management, enhancing mobility, and reducing energy costs and
environmental pollution (Rathore et al., 2021; Salman and Hasar, 2023). However, the
effectiveness of ITS is often hampered by abnormal traffic patterns, including traffic congestion,
accidents, special events, or atypical traffic on specific roads (Parkany and Xie, 2005; Zhang et al.,
2016). Traffic anomalies may result in stop-and-go traffic and heightened congestion within urban
networks (Qian et al., 2020), consequently impairing smart city operations and impeding the
development of a green and sustainable system. Detecting and understanding these abnormal
traffic patterns could help develop effective traffic management systems and control policies,
thereby reducing congestion duration, cutting emissions, and improving traffic safety (Zeroual et
al., 2019; Zhou et al., 2021).
During the current development stage of the ITS, traffic time-series data from sensors serves
as the fundamental dataset for further management (Ahmed et al., 2022). Within this framework,
time-series data analysis is instrumental in identifying both spatial and temporal anomalies.
Considering speed time-series data, generally, most road segments demonstrate some degree of
deterministic characteristics (Vlahogianni et al., 2008) owing to their periodic daily traffic patterns,
and traffic patterns on adjacent road segments often show notable similarities (Anbaroglu et al.,
2014; Zhang et al., 2016). However, certain road segments may exhibit more irregular or stochastic
patterns, as observed in their overall time-series data, leading to a diminished level of spatial
dependence with adjacent road segments. Such segments, characterized by their reduced
correlation or interaction with neighboring roads in terms of traffic conditions, can be identified as
spatial anomalies. Furthermore, analyzing the time-series data for each road segment at various
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4737146
Preprint not peer reviewed
3
SUBMITTED TO SUSTAINABLE CITIES AND SOCIETY
3
time granularities—ranging from minutes to days—reveals temporal anomalies. The temporal
anomalies are characterized by either rapid fluctuations or exceedingly low-speed values in the
specific time intervals of road segments, and can be attributed to factors such as traffic congestion,
accidents, special events, or holidays.
Fig. 1. Illustration of spatial and temporal anomalies in time-series traffic speed data
To visually demonstrate spatial and temporal anomalies in speed data, we present traffic speed
data collected from three vehicle-detector-stations (VDS) within the California Department of
Transportation (Caltrans) Performance Measurement System (PeMS)
1
. Fig.1 showcases one week
of traffic speed data, with a sampling granularity of one hour, obtained from three stations (VDS
ID 601590, 601591, and 601731, arranged from upstream to downstream). The data for each of
the three stations is depicted respectively in blue, green, and red. Both VDS-601591 and VDS-
601731 displayed relatively low values during the weekends (27
th
and 28
th
November 2021), which
1
http://pems.dot.ca.gov
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4737146
Preprint not peer reviewed
4
SUBMITTED TO SUSTAINABLE CITIES AND SOCIETY
4
could be classified as temporal anomalies. Additionally, speed drops observed at VDS-601731 on
23
rd
November could also be considered as a form of temporal anomaly. Regarding spatial
anomalies, one of the three stations, VDS-601590, displayed a different speed pattern from the
other two nearby stations. This discrepancy can be attributed to the poor data quality of VDS-
601590, where 0% of vehicles were observed, as reported in the data quality report from PeMS.
In comparison to its upstream and downstream counterparts, this abnormal station deviated from
the expected traffic pattern and exhibited less deterministic characteristics.
A basic assumption in anomaly detection is that the majority of traffic time-series data follow
similar patterns, while anomalies represent a minor proportion of the data (Ahmed et al., 2022; He
et al., 2023). Various efforts have been developed to identify abnormal traffic patterns, including
change point detection algorithms, traffic flow theory models, and prediction-based methods
(Alesiani et al., 2018). However, two challenges remain to be addressed as follows:
(ⅰ) How can anomalies be detected in limited data environments, particularly with single-
source traffic time-series data? The limited data environment refers to the current situation where
ITS in most cities can only provide limited traffic data. While accessing and analyzing high-
dimensional multi-source traffic data can offer system-wide efficiency and objective evaluations
(Yang et al., 2021), it is worth noting that the availability of such data in practical scenarios is often
limited or insufficient. In most cases, traffic data accessible through ITS are limited to basic meta-
information, such as time stamps, road or loop IDs, and some speed/flow/density combinations,
while only a few regions with well-developed ITS can provide more detailed traffic data. For
example, video surveillance systems that can greatly help in traffic anomaly detection are only
equipped on a few freeways (Alesiani et al., 2018).
Therefore, in practical scenarios, traffic data used for detecting anomalies may be limited,
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4737146
Preprint not peer reviewed
5
SUBMITTED TO SUSTAINABLE CITIES AND SOCIETY
5
where time-series data such as speed, volume, and occupancy are still the most commonly used
data sources. In such a scenario, developing data-driven detection methods that can be directly
applied to single-source traffic time-series data becomes particularly necessary. These methods
need to effectively handle limited data resources and accurately identify traffic anomalies without
requiring additional information support.
(ⅱ) How might one explore the spatial correlations and temporal evolutionary
characteristics in anomaly detection? Accurate spatial-temporal representation is important in
comprehensively depicting traffic dynamics and integrating ITS applications (Vlahogianni et al.,
2014). However, the exploration of spatial correlations and temporal evolutions within historical
traffic data remains insufficient. Despite extensive attempts to detect anomalies in data streams
based on immediate past traffic data, the wealth of historical traffic information continues to be
largely underutilized (Chakraborty et al., 2019). Simply extracting features at a specific time or
road segment is insufficient to fully capture the stochastic nature of traffic flow evolutions (Yang
et al., 2021) and spatial dependencies (Zhang et al., 2022). Moreover, while supervised learning
methods (Yuan et al., 2019; Zhang et al., 2018) can capture non-linear correlations to detect
anomalies, the efficacy of these learning-based approaches is constrained by the scarcity of
ground-truth anomaly labels in traffic time-series data. This limitation primarily stems from the
significant cost of manual calibration (He et al., 2023). Consequently, the necessity of developing
a data-driven approach becomes apparent, one that utilizes the data’s intrinsic patterns and
characteristics to identify anomalies within time-series data through unsupervised methods,
thereby obviating the need for exact anomaly labels.
Motivated by existing research gaps in traffic anomaly detection, this study introduces a
hybrid data-driven approach to address spatial and temporal anomaly detection tasks in traffic
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4737146
Preprint not peer reviewed
6
SUBMITTED TO SUSTAINABLE CITIES AND SOCIETY
6
time-series data. This approach leverages two distinct methods to explore the spatial correlations
and temporal evolutionary characteristics inherent in single-source time-series data. In the spatial
domain, the Maximal Information Coefficient (MIC) (Reshef et al., 2011; Reshef et al., 2016) is
employed to capture complex linear and non-linear spatial dependencies in time-series data across
roads. The MIC is capable of quantifying the mutual information (Cover, 1999) between time
series using an adaptive grid partition process (Reshef et al., 2011), thereby measuring a wide
range of associations stemming from real-world interactions. Regarding the temporal domain, the
Markov Transition Field (MTF) method (Wang and Oates, 2015) is used to depict the temporal
evolution characteristics within traffic time-series data, modeling the segment states of each time-
series sequence as transitioning probabilities over time. The MTF method effectively captures the
temporal dynamics and dependencies within the data, facilitating analysis and understanding how
the traffic patterns evolve.
By combining the spatial insights provided by the MIC with the temporal analysis capabilities
of the MTF method, this study achieves a comprehensive understanding of both spatial and
temporal dimensions in traffic time-series data. This hybrid anomaly detection approach not only
identifies spatial anomalies among roads but also captures temporal anomalies within traffic
evolution patterns. The main contributions of this work can be summarized as follows:
We establish a MIC-based spatial anomaly detection algorithm that provides a solution
for identifying abnormal roads. The algorithm can accurately capture the complex linear
and non-linear spatial dependencies among roads and identify abnormal roads with more
stochastic patterns, which has not been adequately addressed in clustering-based studies.
Combined with the MTF method, our proposed temporal anomaly detection algorithm
capitalizes on the inherent temporal evolution characteristics within traffic time-series
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4737146
Preprint not peer reviewed
7
SUBMITTED TO SUSTAINABLE CITIES AND SOCIETY
7
data. The algorithm can effectively detect temporal anomalies across various granularities
by quantifying the self-transition probabilities of temporal segment states. This method is
well-suited for limited data environments relying on single-source information (utilizing
only time-series data), independent of prior knowledge or external data sources.
The detection results of the proposed method enable the identification of roads with more
chaotic traffic patterns requiring additional monitoring for traffic management and control,
as well as the labeling of temporal anomaly points within a large amount of time-series
traffic data, all without necessitating manual calibration.
The rest is organized as follows: Section 2 reviews current studies on urban road traffic anomaly
detection; Section 3 explains the preliminaries and mathematical formulation of the spatial-
temporal anomalies detection method; Section 4 presents the detection results and pattern analyses
through real-world traffic speed data; Section 5 summarizes the conclusions and potential future
works.
2. Literature Review
2.1 Spatial Anomaly Detection
Spatial anomaly detection in existing studies can be broadly classified into two categories:
abnormal trajectory detection and abnormal traffic pattern detection. Abnormal trajectory detection
has been extensively studied in the context of airspace operations, where clustering techniques like
Principal Components Analysis (PCA) (Gariel et al., 2011), Gaussian Mixture Model (GMM) (Li
et al., 2016), and Density-Based methods (Deshmukh and Hwang, 2019; Wang et al., 2019) have
been applied to identify abnormal data. In the domain of road traffic, anomaly detection focuses
on GPS trajectories and aims to identify abnormal trajectories that deviate from the norm (Ahmed
et al., 2022). Furthermore, data fusion methods with other data sources, like loop data (Canepa and
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4737146
Preprint not peer reviewed
8
SUBMITTED TO SUSTAINABLE CITIES AND SOCIETY
8
Claudel, 2013), have been employed to improve anomaly detection accuracy.
In terms of abnormal traffic pattern detection, studies can be classified into image-based
anomaly detection and time-series anomaly detection (Javed et al., 2020). This paper primarily
focuses on detecting spatial anomalies from time series. The underlying assumption for detecting
spatial anomalies is that adjacent roads exhibit a certain degree of spatial correlation due to traffic
propagations. In general, detecting these types of spatial anomalies can be achieved by clustering
methods that employ various metrics such as density, flow and adjacent matrix, or secondary
developed measurements such as entropy and frequency of occurrences (Anbaroglu et al., 2014;
Zhang et al., 2016). However, investigating complex intrinsic correlations among roads with
limited data information remains a challenge when exploring spatial anomalies.
2.2 Temporal Anomaly Detection
As for the temporal domain, various methods have been proposed to detect temporal
anomalies, which are often associated with change point and event detection techniques (Gupta et
al., 2013). Cumulative sum algorithm (CUSUM) (Teng and Qi, 2003), together with California
algorithms and their variants (Petty et al., 2002; Stephane De S and Chassiakos, 1993), were used
to identify different traffic patterns by pair-wise comparison between upstream and downstream
sensors. Besides, some studies tended to set a threshold based on historical data to distinguish
normal and abnormal traffic conditions. The most representative methods for this purpose were
standard normal deviation (SND) and its derivations (Chakraborty et al., 2019; Snelder et al., 2013;
Tang and Gao, 2005). In order to prevent interferences from outliers, the SND family of methods
required the exclusion of accurate start and end times of traffic incidents when calculating the
threshold. In addition, hyperparameter tuning was necessary for these methods, adding complexity
to the process. Consequently, the SND methods required stringent data requirements and
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4737146
Preprint not peer reviewed
9
SUBMITTED TO SUSTAINABLE CITIES AND SOCIETY
9
adjustments.
Although these studies achieved satisfactory performances, some claimed there was still room
for improvement, e.g., considering traffic impacts by combining traffic flow theories. For instance,
Asakura et al. (Asakura et al., 2017) employed shock wave theory in the GPS trajectory of probe
vehicles to detect traffic incidents. Other studies utilized the Lighthill–Whitham–Richards (LWR)
model (Canepa and Claudel, 2013; Zeroual et al., 2019; Zeroual et al., 2017) and fundamental
diagrams (Kalair and Connaughton, 2021) as detection aids. However, these methods that
incorporated traffic flow theories often required a robust database with multiple types of traffic
data (such as flow and density) to construct accurate fundamental diagrams. Moreover,
implementing shock wave analysis at a network scale can be costly and challenging.
In recent years, data mining methods such as machine learning and deep learning have gained
popularity due to the availability of large amounts of data. Chen et al. (Chen et al., 2016) proposed
a systematic approach by secondary incident identification and K-Nearest Neighbor (KNN) pattern
matching to analyze non-recurrent congestions. Mercader and Haddad (Mercader and Haddad,
2020) applied the isolation forest to detect isolated anomalies in traffic data from Bluetooth sensors.
Support Vector Machine (SVM) has also been used for incident detection, as demonstrated in
(Wang et al., 2013). Additionally, several studies took advantage of deep learning methods and
proposed incident detection methods based on Deep Belief Network (DBN) (Zhang et al., 2018),
Long Short-Term Memory (LSTM) (Yuan et al., 2019), Convolutional Neural Network (CNN)
(Cai et al., 2020), and Autoencoder (Islam et al., 2021).
Despite the progress in traffic speed data collection and computation (Mercader and Haddad,
2020), the effectiveness of learning-based models in anomaly detection relies heavily on the
quality of input data, particularly accurate anomaly labels. However, obtaining precise anomaly
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4737146
Preprint not peer reviewed
10
SUBMITTED TO SUSTAINABLE CITIES AND SOCIETY
10
labels is challenging, and their availability is often limited, even when the dataset is extensive.
Manual efforts are typically required to label abnormal instances accurately, which is time-
consuming and expensive. Moreover, it is crucial to exclude abnormal data points from the training
data to ensure the development of a reliable model that represents normal traffic patterns accurately
(Chakraborty et al., 2019). Manually improving the quality of collected data is impractical,
especially for large-scale urban networks. Therefore, there is a pressing need for data-driven
algorithms that leverage the time-series characteristics of traffic data to label temporal anomalies
automatically. Such algorithms can significantly reduce the dependence on external labels and
enable the development of more effective and scalable anomaly detection models.
3. Method
3.1 Preliminaries
This subsection introduces the foundational concepts and overall structure of the proposed
method. The time-series data used in this study consists of multivariate spatial-temporal
information, including traffic speed data for each road segment across a specified timeframe.
Mathematically, considering a discrete-time period of
T
(
T N
), the traffic speed data of
S
(
S N
) road sections can be regarded as a matrix
T S
V
.
1
1 1
1
=
S
T S
S
T T T S
V V
V V
V
(1)
Formally, the spatial and temporal anomalies in this paper are defined as follows:
(ⅰ) Spatial anomalies refer to road sections with traffic patterns that diverge from the majority
within the urban network. While traffic propagation typically leads to similar patterns among
adjacent roads, spatial anomalies often feature time-series patterns with much more stochastic
characteristics than typical roads. (Anbaroglu et al., 2014; Zhang et al., 2016). In matrix
T S
V
, the
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4737146
Preprint not peer reviewed
11
SUBMITTED TO SUSTAINABLE CITIES AND SOCIETY
11
spatial anomalies are identified as abnormal columns.
(ii) Temporal anomalies in this paper refer to rapid changes or extremely low-speed values in
time-series data at different levels of temporal granularity, such as every few minutes and every
day. The occurrence of low-speed values may indicate traffic congestion or accidents, which could
result in a decrease in traffic service level. In matrix
T S
V
, the temporal anomalies are regarded
as abnormal points in each column. An anomaly score will be assigned to each temporal stamp,
and a larger score will indicate anomalies (Kandanaarachchi, 2022).
Moving from the foundational definitions, we now introduce the overall framework structure
of our hybrid data-driven spatial and temporal anomaly detection method, depicted in Fig. 2, laying
the groundwork for the subsequent mathematical details.
Fig. 2. Hybrid data-driven spatial and temporal anomaly detection method
The framework outlines the hybrid data-driven approach for detecting anomalies in traffic
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4737146
Preprint not peer reviewed
12
SUBMITTED TO SUSTAINABLE CITIES AND SOCIETY
12
speed time-series data, encompassing both spatial and temporal domains. Starting with the speed
matrix as input, spatial dependencies between roads are computed through the MIC-based method
and represented in a matrix form. Kernel Density Estimation (KDE) (Chen, 2017) is then utilized
to establish a frequency distribution, with a subsequent threshold set to identify weak spatial
dependencies. For each road segment, the cumulative instances of spatial dependence falling
below the threshold with other segments are calculated. A histogram of these cumulative values is
constructed using the Freedman-Diaconis rule (Freedman and Diaconis, 1981). Road segments
exhibiting anomalous behavior are then identified through the premise that spatial dependencies
follow a long-tail distribution. In parallel, the speed matrix is transformed into the MTF matrix,
focusing on self-transition probabilities along the main diagonal to reflect the temporal stability of
traffic states. Anomaly scores for both daily and minute granularity are calculated, with daily
anomalies categorized by K-means clustering and minute-level anomalies identified using a
topside quantile bin threshold. This hybrid approach is adept at identifying spatial anomalies,
which are presented in histograms, as well as temporal anomalies, depicted through point plot
graphs for both daily and minute intervals. This facilitates a thorough analysis of traffic anomalies
from both qualitative and quantitative perspectives. Details of the method are discussed in
subsequent sections.
3.2 MIC-based Spatial Anomaly Detection Algorithm
The objective of detecting spatial anomalies in the multivariate speed matrix
T S
V
is to identify
road sections that exhibit weak inner spatial dependences with others, as such roads violate the traffic
propagation law or display much more stochastic than ordinary roads. However, it is challenging to
capture certain non-linear spatial dependencies in multivariate traffic data through some commonly
used linear-based correlation measurements such as Spearman and Person correlation coefficient (Dai
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4737146
Preprint not peer reviewed
13
SUBMITTED TO SUSTAINABLE CITIES AND SOCIETY
13
et al., 2019; Habtemichael and Cetin, 2016). To overcome this challenge, we employed the MIC
(Reshef et al., 2011; Reshef et al., 2016) to mine both linear and non-linear relationships in large-scale
time-series traffic data, as follows:
*
( ) 2
(( ; ), , )
( ; ) max min( , )
ab B n
I X Y a b
MIC X Y log a b
(2)
where,
( ; )MIC X Y
is the MIC value between variable sets
X
and
Y
;
2
min( , )log a b
is used
for normalization.
n
is the number of road sections and
0.6
( )= B n n
, according to (Reshef et al.,
2016).
*
(( ; ), , )I X Y a b
is the maximum value of mutual information (Cover, 1999) obtained by
the grid partition with
a
rows and
b
columns on variable sets
X
and
Y
. Note that the
calculation formula of mutual information is omitted here because it can be induced by grid
partition, where the value is proportional to the number of data points falling inside the box (Reshef
et al., 2011).
Basically, the MIC can encapsulate the linear or non-linear relationships between two variable sets
by partitioning a grid of the scatterplot of the two sets (Reshef et al., 2011). To speed up the calculation,
we employ a consistent estimator called MIC
e
(Reshef et al., 2016) by simplifying the gird partition
process. The MIC
e
is defined as follows:
*
( ) 2
*
( ) 2
(( ; ), ,[ ])
max if
min( , )
( ; ) (( ; ),[ ], )
max otherwise,
min( , )
e
ab B n
e
e
ab B n
I X Y a b b a
log a b
MIC X Y I X Y a b
log a b
(3)
where
*
( ,[ ])
(( ; ), ,[ ]) max (( ; ) )
e G
G G a b
I X Y a b I X Y
;
( ,[ ])G a b
represents the grid partition with
a
rows and
b
columns on variable sets
X
and
Y
; Note that
[ ]b
denotes the partition condition
that for
b a
, the a-by-b grids are equipartition of size
b
in column and analogous to the
a b
condition (Reshef et al., 2016). The calculation results of MIC
e
are normalized within the range of
[0,1], where a value of 1 indicates a strong correlation between the two variables, while a value of 0
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4737146
Preprint not peer reviewed
14
SUBMITTED TO SUSTAINABLE CITIES AND SOCIETY
14
indicates independence between them.
Given the multivariate speed matrix
T S
V
, Eq. (3) calculates the spatial dependence between
any pair of column sets (road sections) as
( ; )
e T S T S
MIC
V V
. This approach captures both inner
linear or non-linear spatial dependencies among road sections. The resulting spatial dependency
matrix
S S
SD
can be obtained as shown in Eq.(4).
1,1 ,1
,
1, ,
S
e e
i j
S S e
S S S
e e S S
mic mic
mic
mic mic
SD
(4)
where the element
,i j
e
mic
(
,
[0,1]
i j
e
mic
) denotes the spatial dependence value between road
i
and road
j
. A higher value of
,i j
e
mic
indicates a stronger relationship between road
i
and road
j
is. Notably, the spatial dependence value in the leading diagonal, which represents the auto-
correlation of the road, is set to the maximum value of 1. It can be easily inferred that the spatial
dependency matrix
S S
SD
is symmetric and
, ,i j j i
e e
mic mic
.
Based on the spatial dependency matrix, the spatial anomaly detection algorithm can be
defined as follows (Algorithm 1).
Algorithm 1 Spatial anomaly detection algorithm
Input: Spatial dependency matrix
S S
SD
.
Output: Cumulative number of MIC
e
values
1
,...,
S
cn cncn
; anomaly road sections set
1
= ,...,
c
ar arAR
where
c
is the total number of anomaly road sections.
1: for each column j in
S S
SD
do
2: Compute frequency distribution of MICe values in each column (kernel density estimation is
used);
3: end for
4: Determine a low threshold
( [0,1])p p
through the frequency distribution result and set
0Flag
;
5: for each column j in
S S
SD
do
6: for each row
i
in column j do
7: if
i
e j
mic p
then
8:
1Flag Flag
;
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4737146
Preprint not peer reviewed
15
SUBMITTED TO SUSTAINABLE CITIES AND SOCIETY
15
9: end if
10: end for
11: Let j
cn Flag
,
0Flag
;
12: end for
13: According to FD rules, compute the frequency distribution of elements in
1
,...,
S
cn cncn
;
14: Set anomalies divided by FD rules as
1
,...,
c
ar arAR
;
Firstly, the frequency distribution of MIC
e
values of each column (road) is computed
according to kernel density estimation to determine a threshold
p
of MIC
e
values (as shown in
lines 1~4 in Algorithm 1). The value of
p
can be determined according to the distribution of
MIC
e
values. If the MIC
e
value of a pair of roads is smaller than
p
, these two roads are considered
as less correlated. The cumulative number of MIC
e
values below the threshold
p
is counted for
each road, and the values are stored in a set
1
= ,..., ,...,
x S
cn cn cncn
, where
x
cn
denotes the
cumulative number of MIC
e
below the threshold of road section
( )x x S
. This procedure refers
to lines 5~12.
Now each road section has its cumulative number, which can be used to determine anomalies.
A large cumulative number indicates that few roads have strong correlations with the target road,
and this target road may be an abnormal one. Subsequently, the frequency distribution of elements
in the cumulative number set
cn
is computed using the Freedman-Diaconis rule (FD) (Freedman
and Diaconis, 1981). The FD is designed to select the width of the bins in a histogram, and based
on the FD result, the algorithm identifies the outlier part of the distribution that can be treated as
an anomaly set of roads (i.e., lines 13-14 in Algorithm 1). Note that the FD estimator is robust
(resilient to outliers) for long-tail distributions, where anomalies are typically located. By
implementing Algorithm 1, the anomalous road sections
1
= ,...,
c
ar arAR
can be deciphered,
where
c
is the total number of anomalous road sections.
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4737146
Preprint not peer reviewed
16
SUBMITTED TO SUSTAINABLE CITIES AND SOCIETY
16
3.3 MTF-based Temporal anomaly detection algorithm
In this study, temporal anomalies are defined as rapid changes or extremely low values of
traffic speed, and may include accidents, road constructions, public events, and holidays. To
capture the inherent temporal dependencies present within each column (representing individual
roads) of the traffic speed matrix , this study introduces a temporal anomaly detection method
utilizing the Markov Transition Field (MTF) approach (Wang and Oates, 2015). This method
models traffic time-series features as a first-order Markov chain, preserving sequential
characteristics within the temporal domain. It allows for measuring transition probabilities within
time-series traffic data, thereby enabling the identification of temporal anomalies. The specific
details of the temporal anomaly detection method utilizing the MTF approach are outlined as
follows:
Given a time-series (i.e., speed data of road
x
)
1
( ,..., )
x x
T
V V
, the domain of values is divided
into
Q
quantile bins, where each bin has the same number of points. Thus a
Q Q
Markov
matrix can be built by computing the transitions among bins in the manner of a first-order Markov
chain (Wang and Oates, 2015). The quantile bin
( [1, ])
i
q q Q
represents the bin that contains
data of bin
q
at time
i
. Then the MTF is derived by extending the Markov matrix along with
the temporal positions as Eq. (5).
1 1 1
1
| , | ,
11 1
1| , | ,
x x x x
i j i T j
x x x x
T i j T i T j
ij v q v q ij v q v q
T
ii
T TT ij v q v q ij v q v q
w w
M M
M
M M w w
M
(5)
where
ij
M
is the transition probability of bin
i
q
transferring to bin
j
q
(that is,
i j
q q
), and
ij
w
is the frequency of a point in
j
q
followed by the other point in
i
q
; Through this matrix, we
can obtain transition probabilities from one timestamp to another and identify patterns of temporal
T S
V
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4737146
Preprint not peer reviewed
17
SUBMITTED TO SUSTAINABLE CITIES AND SOCIETY
17
anomalies.
Note that the main diagonal of MTF, ii
M
, corresponds to the self-transition probability of
each bin, indicating the probability of maintaining the current traffic state in the next sample time
period. A high self-transition probability implies that the traffic state is more likely to remain the
same rather than transfer to other states. Generally, in situations where the traffic speed is
extremely low, it is less likely for the traffic state to change, and hence, the timestamps with high
self-transition probabilities can be identified as temporal anomalies.
In contrast to the original purpose of MTF for visualizing time series as a graph, our proposed
algorithm leverages the transition dynamics embedded in MTF to detect temporal anomalies at
multi-granularity. Specifically, in this study, we evaluate two temporal granularities, namely, 10
minutes (the minimum sample period in our dataset) and one day (144 sample periods). The multi-
granularity temporal anomaly detection algorithm is presented in Algorithm 2.
Algorithm 2 Temporal anomaly detection algorithm
Input: Multivariate traffic speed matrix
T S
V
; The quantile bins number q
n
;
Output: Anomaly score sets
1
= ,...,
T T
S
MA ma ma
,
/144 /144
1
= ,...,
T T
S
DA da da
and anomaly points
{ ,..., }
1 S
min min min
anp anp anp
,
1
{ ,..., }
S
day day day
anp anp anp
for every 10 minutes and every
day, respectively;
1: Set the number of quantile bin sets for every 10-minute and daily time intervals as
( )
min q
n nQ
and
( )
day q
n nQ
;
// self-transition probability calculation
2: for each column
i
in
T S
V
do
3: Compute the MTF matrix
min
i T T
M
and
/144 /144
day
i T T
M
of road
i
using Eq. (4);
4: Assign the main diagonal of min
i T T
M
and /144 /144
day
i T T
M
to the anomaly score vector
T
i
ma
and /144T
i
da
;
5: Store the division result in
T
i
ma
and
/144T
i
da
;
6: end for
7: Set
1
,...,
T T
S
MA ma ma
,
/144 /144
1
,...,
T T
S
DA da da
,
1
,...,
S
min min min
Q Q Q
, and
1
,...,
S
day day day
Q Q Q
;
8: for each column j in
S
do
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4737146
Preprint not peer reviewed
18
SUBMITTED TO SUSTAINABLE CITIES AND SOCIETY
18
// anomaly detection for the granularity of 10-minute
9: Choose the topside values of j
min
Q
for every 10 minutes as the threshold of road j, denoted
as
1
,
q
jn
min
Q
;
10: if
1
,
( )
q
jn
t t T
j min j j
ma Q ma ma
then
11: Designate the anomaly point
,min
j
t
anp
(time
t
, road
j
);
12: end if
13: Set the anomaly points (with the total number of m) of road j as
min 1,min ,min
,...,
j j j
m
anp anpanp
;
// anomaly detection for the granularity of one-day
14: Apply K-means method on the anomaly score set
DA
;
15: Designate high anomaly score values as anomaly points cluster
1, ,
,...,
j j j
day day d day
anp anpanp
;
16: end for
17: Set the anomaly point sets
{ ,..., }
1 S
min min min
anp anp anp
and
1
{ ,..., }
S
day day day
anp anp anp
.
As in Algorithm 2, the algorithm starts by dividing the time-series data values into quantile bins.
Considering the trade-off between computational efficiency and division accuracy, the number of quantile
bins
q
n
is set within 10~20 for both 10-minute and daily granularity scenarios (line 1 in Algorithm 2). The
main diagonals of the MTF matrix,
1
= ,...,
T T
S
MA ma ma
and
/144 /144
1
= ,...,
T T
S
DA da da
, are
computed for each road under two temporal granularities. At the same time, the results of Q bins’ division
in
MA
and
DA
are also recorded as
1
= ,...,
S
min min min
Q Q Q
and
1
= ,...,
S
day day day
Q Q Q
. These two
steps, which can be regarded as the self-transition probability calculation, are shown in lines 2~7 in
Algorithm 2.
Subsequently, temporal anomaly detection is conducted at different granularities. For temporal
anomaly detection with the granularity of every 10 minutes (lines 8~13), the algorithm sets a threshold
value for each road based on the topside value
j
min
Q
of road
j
. If the self-transition probability exceeds
the threshold value, the timestamp is designated as an anomaly point. The algorithm then stores the anomaly
points every 10 minutes in
{ ,..., }
1 S
min min min
anp anp anp
for all roads.
For temporal anomaly detection with the granularity of every day (lines 14~16), the algorithm applies
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4737146
Preprint not peer reviewed
19
SUBMITTED TO SUSTAINABLE CITIES AND SOCIETY
19
the K-means clustering algorithm on the anomaly score set
/144 /144
1
= ,...,
T T
S
DA da da
. The cluster number
is chosen using the elbow method. The algorithm then designates the day in the cluster of high anomaly
score values as anomaly points of road
j
, denoted as
1, ,
,...,
j j j
day day d day
anp anpanp
.
Finally, the algorithm outputs the anomaly sets,
{ ,..., }
1 S
min min min
anp anp anp
and
1
{ ,..., }
S
day day day
anp anp anp
, for every road, under two temporal granularities (line 17). These sets can
be used to identify and analyze temporal anomalies in traffic data.
4. Result and Analysis
Based on the above methodology, this section details the experimental design to validate the proposed
hybrid data-driven method for spatial and temporal anomaly detection.
First, we illustrate the spatial
anomaly detection results, identifying the anomalies to showcase their patterns, alongside a quantitative
validation to test the spatial anomaly detection method’s effectiveness.
Subsequently, the effectiveness of
the temporal anomaly detection method is evaluated using two distinct datasets.
Further analysis of the
temporal anomaly detection results is conducted across two distinct temporal granularities.
This section
confirms the method’s ability to capture complex dependencies and dynamic temporal evolutions in urban
traffic.
4.1 Data Description
4.1.1 Traffic speed data from three cities
Real-world masking traffic speed data of three cities (Chengdu, Xian, and Shenzhen) from 1
st
March
to 30
th
June 2018 are used to test the proposed spatial-temporal anomaly detection method. The data contain
road ID and corresponding speed records every 10 minutes. A data pre-processing step is performed to
remove roads that (i) experienced more than ten percent speed data loss and (ii) are located outside the
central area of the city. These criteria exclude roads with poor data quality while retaining the main urban
area. The rest of the missing values are filled by the linear interpolation method. After the data pre-
processing procedure, the traffic speed data of the three cities are chosen as test bases, which contain every
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4737146
Preprint not peer reviewed
20
SUBMITTED TO SUSTAINABLE CITIES AND SOCIETY
20
10-minute traffic speed data (km/h) of 628, 659, and 707 road sections for Chengdu, Xian, and Shenzhen
City, respectively. For simplicity, the names of these three cities will be shortened to CD, XA, and SZ.
4.1.2 Traffic speed data from PeMS
As anomaly labels are not available in the time series data of the three cities, we faced similar
challenges to those described in the former study (Wang and Sun, 2021). To address this, we conducted
additional comparison experiments using a different data source provided by PeMS to evaluate the
performance of our method in detecting temporal anomalies. The dataset comprises time-series speed data
collected every five minutes from a single sensor of I-880 between 8:00 p.m. and 12:00 p.m. from 1
st
May
to 31
st
July 2018. The dataset was carefully calibrated, containing 188 anomalies in 17,664 time stamps.
Speed data can be collected from loops, sensors, GPS equipment installed in taxis, and even
smartphones of ride-hailing services. Also, speed data is the major data source in most ITS deployed in
cities and plays a critical role in identifying traffic conditions on urban roads. By integrating these datasets,
this study aims to comprehensively assess the effectiveness and applicability of the proposed spatial and
temporal anomaly detection method.
4.2 Spatial Anomaly Detection Considering Spatial Correlations
Three spatial dependency matrices of CD, XA, and SZ (
628 628
SD
,
659 659
SD
, and
707 707
SD
) are
computed through the speed matrix of each city. Then, these spatial dependency matrices are adopted in the
spatial anomaly detection algorithm as described in Algorithm 1. As per Algorithm 1, the initial step
involves selecting a threshold value, denoted as
p
. Intuitively, the threshold can be set according to the
distribution of spatial dependence values. Therefore, kernel density estimate (KDE) plots of different cities
are illustrated to visualize the frequency distribution (Fd) of spatial dependence values of road sections, as
shown in Fig.3.
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4737146
Preprint not peer reviewed
21
SUBMITTED TO SUSTAINABLE CITIES AND SOCIETY
21
(a) KDE plot of CD (d) Fd of roads in CD
(b) KDE plot of XA (e) Fd of roads in XA
(c) KDE plot of SZ (f) Fd of roads in SZ
Fig. 3. KDE plot of spatial dependence values and frequency distribution of roads.
The top of Fig. 3 displays the KDE plots of the spatial dependency matrices for the three cities.
Each KDE line corresponds to a specific road section, distinguished by different colors. The x-axis
represents the MIC
e
values, and the y-axis is the estimated density of MIC
e
values. Notably, certain
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4737146
Preprint not peer reviewed
22
SUBMITTED TO SUSTAINABLE CITIES AND SOCIETY
22
road sections in each city exhibit spatial dependencies that cluster around lower levels (indicated
by green dot-dash lines), namely below 0.2, 0.15, and 0.1 for CD, XA, and SZ, respectively. It
suggests relatively weak correlations between these roads and others. Accordingly, the thresholds
for CD, XA, and SZ are set at 0.2, 0.15, and 0.1, respectively. At the bottom of Fig.2, the
cumulative numbers of MIC
e
values that fall below these thresholds are computed for each road in
the different cities, and the resulting frequency distribution is depicted. In Fig.3 (d), (e), and (f),
the x-axis denotes the cumulative number of MICe values below the threshold for each road, and
the y-axis is the number of roads located in a specific range determined by Freedman-Diaconis
rules. Each histogram is divided by a purple dot-dash line, the critical boundary.
It is important to note that spatial anomaly detection operates assuming that the frequency
distribution of road segments concerning the MIC
e
value accumulation follows a long-tail
distribution under normal conditions. The presence of a long-tail distribution to the left of the
purple dot-dash line represents a widespread phenomenon in complex networks, signifying that a
small number of road segments have a large cumulative count below the predetermined threshold.
This observation is consistent with spatial correlation patterns, where most roads in the network
exert mutual influence upon one another. Roads located on the right side of the purple line deviate
from this distribution pattern and are thus identified as anomalies; these roads exhibit a significant
quantity of MIC
e
values falling below the threshold. For instance, in Fig.3 (d), the rightmost
histogram indicates 26 roads with over 600 MIC
e
values below 0.2. Note that anomaly detection
relies on the assumption that, under normal conditions, the frequency distribution of roads
regarding the cumulative number of MICe values follows a long-tail distribution. By employing
Algorithm 1, the anomaly road sets can be detected, with the numbers 48, 54, and 64 for CD, XA,
and SZ, respectively.
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4737146
Preprint not peer reviewed
23
SUBMITTED TO SUSTAINABLE CITIES AND SOCIETY
23
(a) Abnormal roads in CD (d) Road patterns in CD
(b) Abnormal roads in XA (e) Road patterns in XA
(c) Abnormal roads in SZ (f) Road patterns in SZ
Fig. 4. Results of the spatial anomaly detection algorithm
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4737146
Preprint not peer reviewed
24
SUBMITTED TO SUSTAINABLE CITIES AND SOCIETY
24
Following the stream of Algorithm 1, the results of the spatial anomaly detection algorithm
are presented in Fig.4. In Fig.4(a), (b), and (c), the identified spatial anomalies are marked in brown.
These anomalies are predominantly found near the railway station, scenic spots, central business
district (CBD), beltway, and urban roads along the river. Although some of these anomalies may
be attributed to loop failures or measurement errors, it is intriguing to note that these abnormal
roads, detected through the analysis of spatial dependencies in time-series data, appear to be
potential bottlenecks that warrant additional attention from traffic management authorities.
In Fig.4(d), (e), and (f), three types of typical roads are selected to present the differences
between normal and abnormal roads. Specifically, the green line plots depict the most normal roads,
characterized by the minimal cumulative number of MICe values that are below the thresholds (as
observed in the leftmost histograms of Fig. 3 (d), (e), (f)). These normal roads exhibit typical daily
traffic patterns, with distinct morning and evening peak hours clearly evident. The yellow line
plots represent the most abnormal roads, showcasing the maximum cumulative number of MICe
values below the thresholds (as observed in the rightmost histograms of Fig. 3 (d), (e), (f)). These
anomalous roads lack distinct periodic peak hours, displaying chaotic and random characteristics.
This could be attributed to their unique geographical location, traffic planning issues, or other
factors. Lastly, the red line plots illustrate the critical point roads, which serve as the boundary
between normal and abnormal roads (indicated by the purple dot-dash lines in Fig. 3 (d), (e), (f)).
These roads demonstrate some degree of daily periodic patterns, yet compared to the most normal
roads, their periodic characteristics are less pronounced, indicating lower levels of deterministic
behaviors (Vlahogianni et al., 2008).
To further validate our proposed spatial anomaly detection method quantitatively, we employ
the determinism (DET) metric from Recurrence Quantification Analysis (RQA) (Huang et al.,
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4737146
Preprint not peer reviewed
25
SUBMITTED TO SUSTAINABLE CITIES AND SOCIETY
25
2023; Vlahogianni and Karlaftis, 2012). The DET metric measures the determinism of time series
data, with values ranging from 0 (100% stochastic, i.e., fully chaos) to 1 (100% deterministic). As
the spatial anomalies defined before, the DET metric can then be used to evaluate whether time-
series data from a road section is abnormal or not. A larger DET value means that the time-series
data of one road is more deterministic with regular periodic patterns, that is, less abnormal in the
spatial domain. A validation result of three kinds of typical roads from CD, XA, and SZ, is shown
in Fig. 5.
Fig. 5. DET values of three types of roads in different cities
From Fig. 5, it is observed that DET values demonstrate a decreasing trend transitioning from
normal roads to abnormal roads. This indicates a gradual intensification of chaotic characteristics
in the time-series data as one transitions from normal to anomalous roads. This changing pattern
validates the effectiveness of the spatial anomaly detection method in identifying roads with high
levels of chaos and anomalous behavior. Additionally, it confirms the accuracy of selecting critical
roads based on the long-tail distribution assumption. Notably, in SZ, the critical road shows a
smaller DET difference relative to normal roads than in the other two cities. This may be due to a
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4737146
Preprint not peer reviewed
26
SUBMITTED TO SUSTAINABLE CITIES AND SOCIETY
26
larger number of roads in the ‘tail’ of the long-tail distribution, as demonstrated in Fig. 3(f).
In summary, these qualitative and quantitative results together affirm the effectiveness of the
proposed spatial anomaly detection algorithm, providing a powerful tool for analyzing and
detecting anomalies in traffic patterns within urban traffic systems.
4.3 Temporal Anomaly Detection Considering Traffic Evolutions
This subsection focuses on validating the effectiveness of the temporal anomaly detection
method and analyzing patterns of the anomaly detection results. First, performance comparisons
are conducted between the proposed MTF-based method and other popular anomaly detection
methods using well-labeled traffic time-series data. Second, the temporal anomaly detection
algorithm is applied to data from three cities. Given that the datasets for the three cities lack actual
anomaly labels, this study indirectly evaluates the performance of the method by comparing its
results with those of various clustering methods. Finally, pattern analysis for the three cities is
conducted at two levels of granularity: minute and daily, utilizing the temporal anomaly detection
results from the perspectives of individual roads and the entire network.
4.3.1 Effectiveness validation in well-labeled data
Comparison experiments utilize well-labeled PeMS data, as described in section 4.1.2. This
dataset was carefully calibrated, comprising 188 anomalies across 17,664 timestamps. We
compared our method with several well-established time-series anomaly detection methods,
including KNN (Ramaswamy et al., 2000), LinearPCA (Veeramachaneni et al., 2016), KernelPCA
(Hoffmann, 2007), iNNE (isolation using Nearest Neighbor Ensemble) (Bandaragoda et al., 2018),
IF (Isolation Forest) (Liu et al., 2012), BD (Boundary and distance) (Jiang et al., 2011),
WFRDA(Weighted Fuzzy-Rough Density_based Anomaly) (Yuan et al., 2023). We evaluated the
performance of each method using Precision rate, Recall Rate, and AUC metrics, which are
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4737146
Preprint not peer reviewed
27
SUBMITTED TO SUSTAINABLE CITIES AND SOCIETY
27
commonly used in highly imbalanced traffic time-series anomaly detection tasks. A higher value
for these metrics indicates a better detection performance. The comparison results are presented in
Table 1, and the best performance is bold-marked.
Table 1 Comparison results using well-labeled data
Methods Precision Recall AUC
KNN 75.30% 77.70% 0.887
LinearPCA 38.00% 40.43% 0.699
KernelPCA 79.80% 81.90% 0.908
iNNE 58.46% 60.64% 0.801
IF 71.60% 75.00% 0.873
BD 59.90% 58.00% 0.788
WFRDA 86.80% 87.20% 0.935
MTF(this paper) 85.60% 88.30% 0.941
It can be seen from Table 1 that our method performs the best with the highest recall rate and
AUC value, although it has a slightly lower precision rate of 85.6% compared to the WFRDA
method, which is based on rough set theory. It is worth noting that for this test dataset with nearly
20,000 timestamps, our MTF-based detection method can be executed in seconds through parallel
computing techniques, whilst WFRDA requires almost 1,500 seconds. In summary, our method
demonstrates superior performance in temporal traffic anomaly detection tasks.
4.3.2 Effectiveness validation using data from the three cities
As there is a lack of accurate anomaly labels in the time-series data from the cities CD, XA,
and SZ, in this subsection, the temporal anomaly detection task is transformed into an unsupervised
clustering problem, and the effectiveness of the proposed method is assessed through the
evaluation of clustering results. Based on the speed data at ten-minute granularity in the three cities,
several clustering methods are employed for comparative analysis, including K-means (MacQueen,
1967), Gaussian Mixture Model (GMM) (Reynolds, 2009; Zhang et al., 2021), Balanced Iterative
Reducing and Clustering using Hierarchies (BIRCH) (Sathiaraj et al., 2019; Zhang et al., 1996),
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4737146
Preprint not peer reviewed
28
SUBMITTED TO SUSTAINABLE CITIES AND SOCIETY
28
and Agglomerative Clustering (Agglomerative) (Murtagh and Contreras, 2012; Pasupathi et al.,
2021).
In evaluating clustering results, compactness and separation serve as criteria to group similar
objects within the same cluster and ensure they remain distinct across different clusters. Previous
studies (Ezugwu et al., 2022; Liu et al., 2010) have shown that the S_Dbw index excels across five
key aspects: monotonicity, density, subclusters, skewed distributions, and noise. These aspects are
crucial for anomaly detection, notably skewed distributions, and noise, which indicate data with
outliers. We selected S_Dbw as an evaluation metric for assessing the results. S_Dbw is calculated
as follows:
1
1
0, ( , ) ( )
( , )
1.
NC
i
i
if d x u C
f x u NC
otherwise
(6)
1
( )
1
( ) ( )
NC
i
i
C
Scat NC NC D
(7)
1 1,
( , )
1
_ ( ) ( 1) max ( , ), ( , )
i j
i j i j
NC NC ij
x C C
i j j i i j
x C C x C C
f x u
Dens bw NC NC NC f x c f x c
(8)
_ ( ) ( ) _ ( )S Dbw NC Scat NC Dens bw NC
(9)
where,
D
is data set and
NC
is the number of clusters;
i
C
is the i-th cluster and
( )
i
C
is
the variance vector of
i
C
;
i
c
and
j
c
are centers of cluster
i
C
and
j
C
, respectively;
ij
u
is the
middle point of the segment defined by cluster centers;
( , )d x u
represent the distance between
x
and
u
;
denotes the l2-norm calculation;
( )Scat NC
and
_ ( )Dens bw NC
indicates
compactness and separation, respectively;
_ ( )S Dbw NC
is S_Dbw value of the cluster’s result
and a small value of
_ ( )S Dbw NC
indicates well-separated clusters.
Our method, designated as MTF, was tested alongside the four previously mentioned methods
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4737146
Preprint not peer reviewed
29
SUBMITTED TO SUSTAINABLE CITIES AND SOCIETY
29
on each road across three cities. The number of clusters for all methods was aligned with that of
the MTF method to ensure a fair comparison. Table 2 displays the mean values and standard
deviations of the S_Dbw index of the whole network. The best results, characterized by the lowest
mean values and standard deviations for each city, are highlighted in bold.
Table 2 S_Dbw result of different methods
Method Mean value Standard deviation
CD XA SZ CD XA SZ
K-means 0.407 0.420 0.442 0.047 0.064 0.058
GMM 0.409 0.422 0.446 0.052 0.066 0.068
BIRCH 0.403 0.420 0.436 0.056 0.075 0.065
Agglomerative 0.408 0.421 0.441 0.047 0.063 0.056
MTF(this paper) 0.408 0.417 0.431 0.043 0.051 0.044
The MTF method clearly exhibits superior performance, achieving almost the lowest mean
values (except in CD city) and the lowest standard deviation values for the S_Dbw index across
three cities. The results showcase not only the efficiency and precision of our method during the
clustering process, characterized by notable compactness and separation of clusters but also its
reliability across varied road conditions within a road network. This underscores the proposed
method’s high adaptability and robustness in managing complex urban traffic systems and offers
reliable evidence supporting its application in various cities in the future.
4.3.3 Pattern analysis of MTF matrix
In Algorithm 2, the MTF matrix serves a fundamental role by providing traffic transition
probabilities in time-series data, which can further be utilized to quantify traffic evolution on roads.
This subsection will present illustrations and examine the distinct patterns of the MTF matrix for
different types of roads across three cities. Fig. 6 displays the MTF matrices used in the temporal
anomaly detection algorithm, showcasing heatmaps of the MTF matrices for the most normal and
abnormal roads in each of the three cities. A sample time of 24 hours (one day) is selected to ensure
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4737146
Preprint not peer reviewed
30
SUBMITTED TO SUSTAINABLE CITIES AND SOCIETY
30
clarity in the presentation.
(a) MTF matrix heatmap for normal and abnormal roads in CD
(b) MTF matrix heatmap for normal and abnormal roads in XA
(c) MTF matrix heatmap for normal and abnormal roads in SZ
Fig. 6. Illustration of MTF matrices of the normal roads and abnormal roads in CD, XA, and SZ
The color bars on the right side of Fig.6(a), (b), and (c) denote the transition probabilities of
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4737146
Preprint not peer reviewed
31
SUBMITTED TO SUSTAINABLE CITIES AND SOCIETY
31
elements in the MTF matrix. On the left side of Fig.6(a), (b), and (c), a clear periodic characteristic
is observed in the MTF matrix of the normal road in each city. This periodic pattern is manifested
both in the main diagonal and across the entire matrix, reflecting the inherent property of daily
traffic under normal conditions. Notably, the comparison between the normal and abnormal roads
in each city reveals significant differences. In the MTF matrix of each city, the latticed blocks
corresponding to the normal road are much more prominent and evenly distributed. This finding
serves as additional supporting evidence for the effectiveness of our spatial anomaly detection
algorithm, as it highlights the lack of apparent periodicity in the daily traffic of abnormal roads.
4.3.4 Temporal anomaly detection: a minute granularity level
As described in Algorithm 2, the main diagonal (self-transition probability) of the MTF matrix
is selected for detecting temporal anomalies. After the temporal anomaly detection algorithm, we
present
MA
, the anomaly score every 10 minutes under different scenes (the most normal single
road and network scale) in Fig. 7.
(a) Network scale self-transition and median speed
values of CD
(d) Self-transition and median speed values of a
single road in CD
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4737146
Preprint not peer reviewed
32
SUBMITTED TO SUSTAINABLE CITIES AND SOCIETY
32
(b) Network scale self-transition and median speed
values of XA
(e) Self-transition and median speed values of a
single road in XA
(c) Network scale self-transition and median speed
values of SZ
(f) Self-transition and median speed values of a
single road in SZ
Fig. 7. Self-transition probabilities and median speed values every 10 minutes
In all sub-plots of the three cities, the x-axis is the time of day, and the y-axis on the left side
is the date. Further, a line plot of the median traffic speed every 10 minutes (marked in red) on
different dates is added on the right-side y-axis to explore the relationship between daily traffic
speeds and anomaly scores. The color bar on the bottom of each subplot is decided by its own
anomaly score set
MA
under different scenarios.
Fig. 7 depicts the distribution of anomaly scores in both the network scale (Fig. 7(a), (b), and
(c)) and individual roads (Fig.7(d), (e), and (f)). Despite some variations resulting from averaging
operations across roads, a similar pattern of anomaly score distribution can be observed. High
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4737146
Preprint not peer reviewed
33
SUBMITTED TO SUSTAINABLE CITIES AND SOCIETY
33
anomaly scores (indicated by bright yellow areas) are concentrated during specific time periods,
including midnight (from 3 a.m. to 7 a.m.), morning rush hours (from 8 a.m. to 9 a.m.), and evening
rush hours (from 5 p.m. to 7 p.m.). It is not surprising that the high anomaly scores gather in these
periods because traffic speed data in the urban network remain at a pretty high (midnight) or low
(rush hours) level, which increases the probability of maintaining their current states. During rush
hours, the congestion slows down traffic propagation along the road, leading to low inter-transition
probabilities to others and higher self-transition probabilities. Similarly, during midnights, the
urban traffic is in free flow conditions with relatively high speed, allowing roads to maintain their
current states.
Furthermore, a detailed illustration of the temporal anomaly detection results for every 10
minutes (i.e., the temporal anomaly points
min
anp
on a single road) is shown in Fig.8. The
snapshot in Fig.8 illustrates the detection result for the most normal road in CD during the period
from 8
th
May to 11
th
May. The reason for choosing this period is that two typical types of temporal
anomalies are included in this period through manual verifications. It is a suitable example to
demonstrate the effectiveness of the proposed algorithm.
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4737146
Preprint not peer reviewed
34
SUBMITTED TO SUSTAINABLE CITIES AND SOCIETY
34
Fig. 8. Fragments of temporal anomaly detection results for a single road in CD.
In Fig.8, the x-axis represents the time of day, and the y-axis is the traffic speed. The red points
correspond to detected anomalies, which can be categorized into point anomalies (marked by red
dashes) and period anomalies (marked by yellow dashes). It is clear that anomalies occur during
periods of rapid change and valleys in the time series. In the case of period anomalies, the start
and endpoints are denoted by yellow dash lines and are filled in yellow. Typically, period anomalies
arise during rush hours. For example, most period anomalies in the chosen road are concentrated
between 6 p.m. and 8 p.m., corresponding to the evening rush hour. Also, from the line plots of
traffic speed Fig. 7d), we can observe that the duration of the morning rush hour on the chosen
road is shorter compared to that of the evening rush hour. This explains why only one morning
rush hour (around 8 a.m. on 8
th
May) is detected in Fig.8. Regarding point anomalies, three
anomalies are detected and marked by red dashed lines. One anomaly is located near the rush hours
on 9
th
May, and we speculate that it can be considered part of an extended period of high traffic
congestion. The other two anomalies occur around 12 p.m., during which sudden drops in traffic
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4737146
Preprint not peer reviewed
35
SUBMITTED TO SUSTAINABLE CITIES AND SOCIETY
35
speed occur, likely caused by accidents, monitoring faults, or other unforeseen factors.
4.3.5 Temporal anomaly detection: a daily granularity level
To analyze the performance of the temporal anomaly detection algorithm under daily
granularity, we aggregate the anomaly score set of different roads in each city to the network scale.
This is done by calculating the average MTF values for each road. The results are presented in
Fig.9, where the daily temporal anomalies
day
anp
are identified according to Algorithm 2, and
the vertical gray dot-dash lines label weekends. The scatter plot displays different colors to
represent the cluster results obtained from the k-means algorithm. The number of clusters is
determined as 4 using the elbow method.
(a) CD, March-June
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4737146
Preprint not peer reviewed
36
SUBMITTED TO SUSTAINABLE CITIES AND SOCIETY
36
(b) XA, March-June
(c) SZ, March-June
Fig. 9. Network temporal anomaly scores from 1
st
March to 30
th
June.
In Fig.9(a), (b), and (c), the MTF values of the network are grouped into four clusters, where
two clusters with the highest values (colored in red and coral) exhibit distinct patterns compared
to the other clusters. These two groups are identified as temporal anomaly sets for daily granularity.
Upon examining the dates associated with these groups, it becomes evident that they correspond
to holidays observed in China. Additionally, other instances related to public activities specific to
each city are not listed here. This observation indicates that the temporal anomaly detection
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4737146
Preprint not peer reviewed
37
SUBMITTED TO SUSTAINABLE CITIES AND SOCIETY
37
algorithm can identify holidays and public activities that significantly impact urban traffic patterns.
Another interesting finding is the presence of a weekly periodicity in the anomaly scores (MTF
values). In Fig.9, the weekends are indicated by vertical gray dot-dash lines. Apart from the
temporal anomaly sets and the days directly affected by them, the anomaly scores exhibit a
consistent pattern. From Monday to Thursday, the scores remain relatively low, but they experience
a significant increase from Friday to Sunday. This pattern aligns with the daily traffic patterns
observed in urban cities, where traffic flows on weekends and Fridays differ from those on
weekdays, and thus can be considered as anomalies.
In summary, the temporal anomaly detection algorithm can detect anomalies from different
temporal granularities. In this paper, we conducted temporal anomaly detections at two specific
temporal granularities: every ten minutes and daily intervals. It is worth noting that the proposed
method is effectiveness for any other temporal granularities and can be readily adjusted to
accommodate new data with different sample time values, making it adaptable and flexible for
various time resolutions.
4.4. Discussion
There are still some aspects that require further discussion. First, our objective is to develop
an anomaly detection method that can fully take advantage of the profits of data explosion in the
new information era. However, transitioning to a mature smart city system is time-consuming, and
current systems frequently grapple with limited data environments. Currently, traffic data sources
are still limited, and the growing volume of traffic time-series data, being the primary accessible
data source, presents a challenge in effectively labeling anomalies for traffic management. As it is,
the advantage of this approach lies in its capacity to capture dynamic changes in road speed and
abnormal traffic patterns, utilizing only time-series data from individual road segments. This
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4737146
Preprint not peer reviewed
38
SUBMITTED TO SUSTAINABLE CITIES AND SOCIETY
38
renders it an efficient data-driven method with minimal operational requirements, accounting for
spatial correlations and temporal evolutions in traffic based on speed data.
It is important to acknowledge that integrating temporal and spatial factors can enhance the
accuracy of anomaly detection methods. However, some studies in short-term prediction (Wang et
al., 2020; Zhang et al., 2019) have shown that the most significant contributions to prediction
results come from the data of the road segment (or detector) itself. The influence of upstream and
downstream road segments on the target segment diminishes sharply with increasing distance.
1
Additionally, the minimum sample interval for data in our study is set as 10 minutes. Given the
relatively large time interval, the impacts of traffic propagation on nearby road segments may be
underestimated, potentially diminishing the advantages of integrating upstream and downstream
data in anomaly detection. Considering these factors, we conducted temporal anomaly detection
based solely on the speed data of the road segment itself (and, in fact, the results also demonstrate
the effectiveness of our method).
Second, the dynamic homogeneity of traffic networks (Geroliminis and Daganzo, 2008) may
affect the spatial anomaly detection algorithm. Homogeneous clusters of roads may exhibit higher
values of spatial dependence, causing the KDE of spatial dependence values to aggregate at a low
value, and impeding the spatial anomaly detection results. This assumption may weaken the
applicability of the spatial anomaly detection algorithm, particularly in cities with many clusters,
such as those located in areas with several rivers that naturally group a number of roads into
clusters. Therefore, future research should investigate the relationship between dynamic
homogeneity and spatial dependence within traffic networks. Adjusting and optimizing the spatial
anomaly detection algorithm to improve its applicability in these scenarios requires further
1
this discussion does not consider the traffic interactions of road segments due to their inherent characteristics (such as Points of
Interest and land use attributes)
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4737146
Preprint not peer reviewed
39
SUBMITTED TO SUSTAINABLE CITIES AND SOCIETY
39
investigation despite the rarity of such city types compared to typical urban layouts.
Third, although the anomaly detection method has been proven to be effective, some
differences exist when applying the method to different cities. Initially, the density of road
networks can impact the result of our temporal anomaly detection method. As seen in Fig. 7(b),
XA exhibits higher anomaly scores (represented by brighter yellow colors) during morning and
evening peaks compared to the other two cities. The road network density in XA is 5.49 km/km
2
,
which is lower than CD (8.02 km/km
2
) and SZ (9.50 km/km
2
). A small road network density
restricts the capability of urban networks, resulting in higher anomaly scores. Similar findings are
observed in Fig. 9(b), where network anomaly scores of XA are higher than the other two cities.
Fortunately, the impacts of such differences are blunted when dealing with the same city.
5. Conclusion
Spatial-temporal anomalies in urban traffic can significantly affect the efficiency of ITS in
smart cities, resulting in extra fuel consumption and increased vehicle emissions. This study
addresses the challenge of accurately and effectively detecting anomalies in raw traffic data under
limited data environments without relying on complex parameter tuning. The proposed data-driven
spatial-temporal anomaly detection method offers an efficient solution for handling the dynamic
and complex nature of traffic data. Experimental analyses using real-world data validate the
effectiveness of the proposed method in identifying abnormal roads with distinct traffic patterns
and labeling anomalous periods.
The primary objective of this study is to identify roads that require additional management
and control measures while providing a strong foundation for the implementation of potent
learning-based anomaly detection methods. The practical implications of this study extend to urban
traffic and city management departments. The spatial anomaly detection algorithm enables the
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4737146
Preprint not peer reviewed
40
SUBMITTED TO SUSTAINABLE CITIES AND SOCIETY
40
identification of roads with more stochastic traffic patterns that require specialized traffic
management measures during daily operations. Furthermore, the temporal anomaly detection
algorithm allows for exploring traffic patterns at different time granularities, such as minute-by-
minute, hourly, and daily intervals. It can identify sudden speed drops, congestion during rush
hours, as well as special events and holidays reflected in speed data. City transportation managers
can leverage this information to improve traffic flow and optimize traffic management strategies.
By adopting this data-driven approach, the reliance on manual operations can be reduced, and
well-labeled data can be generated using basic traffic data, facilitating further development goals.
Future works will focus on developing real-time anomaly detection methods based on this
work and deep learning models. Developing methods based on Generative Adversarial Networks
(Tang et al., 2023) or tensor decomposition (Xing et al., 2023) to utilize the limited data source
will be a promising direction. Additionally, the performance of spatial detection methods on
different road types will also be the next research topic.
Acknowledgments
This work is supported by the China Scholarship Council (No. 202207000072), National Natural
Science Foundation of China (No. 62103292), and Chengdu Technology Innovation R&D Project
of Key R&D Support Program (No. 2022-YF05-00302-SN).
References
Ahmed, U., Srivastava, G., Djenouri, Y., Lin, J.C.-W., 2022. Knowledge graph based trajectory outlier
detection in sustainable smart cities. Sustainable Cities and Society 78, 103580.
Alesiani, F., Moreira-Matias, L., Faizrahnemoon, M., 2018. On learning from inaccurate and incomplete
traffic flow data. IEEE Transactions on Intelligent Transportation Systems 19(11), 3698-3708.
Anbaroglu, B., Heydecker, B., Cheng, T., 2014. Spatio-temporal clustering for non-recurrent traffic
congestion detection on urban road networks. Transportation Research Part C: Emerging
Technologies 48, 47-65.
Asakura, Y., Kusakabe, T., Nguyen, L.X., Ushiki, T., 2017. Incident detection methods using probe vehicles
with on-board GPS equipment. Transportation research part C: emerging technologies 81, 330-341.
Bandaragoda, T.R., Ting, K.M., Albrecht, D., Liu, F.T., Zhu, Y., Wells, J.R., 2018. Isolation‐based anomaly
detection using nearest‐neighbor ensembles. Computational Intelligence 34(4), 968-998.
Cai, Q., Abdel-Aty, M., Yuan, J., Lee, J., Wu, Y., 2020. Real-time crash prediction on expressways using
deep generative models. Transportation research part C: emerging technologies 117, 102697.
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4737146
Preprint not peer reviewed
41
SUBMITTED TO SUSTAINABLE CITIES AND SOCIETY
41
Canepa, E.S., Claudel, C.G., 2013. A framework for privacy and security analysis of probe-based traffic
information systems, Proceedings of the 2nd ACM international conference on High confidence
networked systems, pp. 25-32.
Chakraborty, P., Hegde, C., Sharma, A., 2019. Data-driven parallelizable traffic incident detection using
spatio-temporally denoised robust thresholds. Transportation research part C: emerging technologies
105, 81-99.
Chen, Y.-C., 2017. A tutorial on kernel density estimation and recent advances. Biostatistics &
Epidemiology 1(1), 161-187.
Chen, Z., Liu, X.C., Zhang, G., 2016. Non-recurrent congestion analysis using data-driven spatiotemporal
approach for information construction. Transportation Research Part C: Emerging Technologies 71,
19-31.
Cover, T.M., 1999. Elements of information theory. John Wiley & Sons.
Dai, X., Fu, R., Zhao, E., Zhang, Z., Lin, Y., Wang, F.-Y., Li, L., 2019. DeepTrend 2.0: A light-weighted
multi-scale traffic prediction model using detrending. Transportation Research Part C: Emerging
Technologies 103, 142-157.
Deshmukh, R., Hwang, I., 2019. Incremental-learning-based unsupervised anomaly detection algorithm for
terminal airspace operations. Journal of Aerospace Information Systems 16(9), 362-384.
Ezugwu, A.E., Ikotun, A.M., Oyelade, O.O., Abualigah, L., Agushaka, J.O., Eke, C.I., Akinyelu, A.A., 2022.
A comprehensive survey of clustering algorithms: State-of-the-art machine learning applications,
taxonomy, challenges, and future research prospects. Engineering Applications of Artificial
Intelligence 110, 104743.
Freedman, D., Diaconis, P., 1981. On the histogram as a density estimator: L 2 theory. Zeitschrift für
Wahrscheinlichkeitstheorie und verwandte Gebiete 57(4), 453-476.
Gariel, M., Srivastava, A.N., Feron, E., 2011. Trajectory clustering and an application to airspace
monitoring. IEEE Transactions on Intelligent Transportation Systems 12(4), 1511-1524.
Geroliminis, N., Daganzo, C.F., 2008. Existence of urban-scale macroscopic fundamental diagrams: Some
experimental findings. Transportation Research Part B Methodological 42(9), 759-770.
Gupta, M., Gao, J., Aggarwal, C.C., Han, J., 2013. Outlier detection for temporal data: A survey. IEEE
Transactions on Knowledge and data Engineering 26(9), 2250-2267.
Habtemichael, F.G., Cetin, M., 2016. Short-term traffic flow rate forecasting based on identifying similar
traffic patterns. Transportation research Part C: emerging technologies 66, 61-78.
He, D., Kim, J., Shi, H., Ruan, B., 2023. Autonomous anomaly detection on traffic flow time series with
reinforcement learning. Transportation Research Part C: Emerging Technologies 150, 104089.
Hoffmann, H., 2007. Kernel PCA for novelty detection. Pattern recognition 40(3), 863-874.
Huang, H., Mao, J., Lu, W., Hu, G., Liu, L., 2023. DEASeq2Seq: An attention based sequence to sequence
model for short-term metro passenger flow prediction within decomposition-ensemble strategy.
Transportation Research Part C: Emerging Technologies 146, 103965.
Islam, Z., Abdel-Aty, M., Cai, Q., Yuan, J., 2021. Crash data augmentation using variational autoencoder.
Accident Analysis & Prevention 151, 105950.
Javed, A.R., Usman, M., Rehman, S.U., Khan, M.U., Haghighi, M.S., 2020. Anomaly detection in
automated vehicles using multistage attention-based convolutional neural network. IEEE Transactions
on Intelligent Transportation Systems 22(7), 4291-4300.
Jiang, F., Sui, Y., Cao, C., 2011. A hybrid approach to outlier detection based on boundary region. Pattern
recognition letters 32(14), 1860-1870.
Kalair, K., Connaughton, C., 2021. Anomaly detection and classification in traffic flow data from
fluctuations in the flow–density relationship. Transportation Research Part C: Emerging Technologies
127, 103178.
Kandanaarachchi, S., 2022. Unsupervised anomaly detection ensembles using item response theory.
Information Sciences 587, 142-163.
Li, L., Hansman, R.J., Palacios, R., Welsch, R., 2016. Anomaly detection via a Gaussian Mixture Model
for flight operation and safety monitoring. Transportation Research Part C: Emerging Technologies
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4737146
Preprint not peer reviewed
42
SUBMITTED TO SUSTAINABLE CITIES AND SOCIETY
42
64, 45-57.
Liu, F.T., Ting, K.M., Zhou, Z.-H., 2012. Isolation-based anomaly detection. ACM Transactions on
Knowledge Discovery from Data (TKDD) 6(1), 1-39.
Liu, Y., Li, Z., Xiong, H., Gao, X., Wu, J., 2010. Understanding of internal clustering validation measures,
2010 IEEE international conference on data mining. IEEE, pp. 911-916.
MacQueen, J., 1967. Some methods for classification and analysis of multivariate observations,
Proceedings of the fifth Berkeley symposium on mathematical statistics and probability. Oakland, CA,
USA, pp. 281-297.
Mercader, P., Haddad, J., 2020. Automatic incident detection on freeways based on Bluetooth traffic
monitoring. Accident Analysis & Prevention 146, 105703.
Murtagh, F., Contreras, P., 2012. Algorithms for hierarchical clustering: an overview. Wiley
Interdisciplinary Reviews: Data Mining and Knowledge Discovery 2(1), 86-97.
Parkany, E., Xie, C., 2005. A Complete Review of Incident Detection Algorithms & Their Deployment:
What Works and What Doesn't. Technology Report for the New England Transportation Consortium
Pasupathi, S., Shanmuganathan, V., Madasamy, K., Yesudhas, H.R., Kim, M., 2021. Trend analysis using
agglomerative hierarchical clustering approach for time series big data. The Journal of
Supercomputing 77, 6505-6524.
Petty, K.F., Ostland, M., Kwon, J., Rice, J., Bickel, P.J., 2002. A new methodology for evaluating incident
detection algorithms. Transportation Research Part C: Emerging Technologies 10(3), 189-204.
Qian, X., Lei, T., Xue, J., Lei, Z., Ukkusuri, S.V., 2020. Impact of transportation network companies on
urban congestion: Evidence from large-scale trajectory data. Sustainable Cities and Society 55, 102053.
Ramaswamy, S., Rastogi, R., Shim, K., 2000. Efficient algorithms for mining outliers from large data sets,
Proceedings of the 2000 ACM SIGMOD international conference on Management of data, pp. 427-
438.
Rathore, M.M., Paul, A., Rho, S., Khan, M., Vimal, S., Shah, S.A., 2021. Smart traffic control: Identifying
driving-violations using fog devices with vehicular cameras in smart cities. Sustainable Cities and
Society 71, 102986.
Reshef, D.N., Reshef, Y.A., Finucane, H.K., Grossman, S.R., McVean, G., Turnbaugh, P.J., Lander, E.S.,
Mitzenmacher, M., Sabeti, P.C., 2011. Detecting novel associations in large data sets. science
334(6062), 1518-1524.
Reshef, Y.A., Reshef, D.N., Finucane, H.K., Sabeti, P.C., Mitzenmacher, M., 2016. Measuring dependence
powerfully and equitably. The Journal of Machine Learning Research 17(1), 7406-7468.
Reynolds, D.A., 2009. Gaussian mixture models. Encyclopedia of biometrics 741(659-663).
Salman, M.Y., Hasar, H., 2023. Review on Environmental Aspects in Smart City Concept: Water, Waste,
Air Pollution and Transportation Smart Applications using IoT Techniques. Sustainable Cities and
Society, 104567.
Sathiaraj, D., Huang, X., Chen, J., 2019. Predicting climate types for the Continental United States using
unsupervised clustering techniques. Environmetrics 30(4), e2524.
Snelder, M., Bakri, T., Van Arem, B., 2013. Delays caused by incidents: Data-driven approach.
Transportation Research Record 2333(1), 1-8.
Stephane De S, Y.J., Chassiakos, A.P., 1993. Freeway incident detection through filtering. Transportation
Research Part C Emerging Technologies 1(3), 219-233.
Tang, S., Gao, H., 2005. Traffic-incident detection-algorithm based on nonparametric regression. IEEE
Transactions on Intelligent Transportation Systems 6(1), 38-42.
Tang, T., Liu, R., Choudhury, C., Fonzone, A., Wang, Y., 2023. Predicting hourly boarding demand of bus
passengers using imbalanced records from smart-cards: A deep learning approach. IEEE Transactions
on Intelligent Transportation Systems.
Teng, H., Qi, Y., 2003. Detection-delay-based freeway incident detection algorithms. Transportation
Research Part C: Emerging Technologies 11(3-4), 265-287.
Veeramachaneni, K., Arnaldo, I., Korrapati, V., Bassias, C., Li, K., 2016. AI^ 2: training a big data machine
to defend, 2016 IEEE 2nd international conference on big data security on cloud (BigDataSecurity),
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4737146
Preprint not peer reviewed
43
SUBMITTED TO SUSTAINABLE CITIES AND SOCIETY
43
IEEE international conference on high performance and smart computing (HPSC), and IEEE
international conference on intelligent data and security (IDS). IEEE, pp. 49-54.
Vlahogianni, E.I., Karlaftis, M.G., 2012. Comparing traffic flow time-series under fine and adverse weather
conditions using recurrence-based complexity measures. Nonlinear Dynamics 69, 1949-1963.
Vlahogianni, E.I., Karlaftis, M.G., Golias, J.C., 2008. Temporal evolution of short‐term urban traffic flow:
a nonlinear dynamics approach. Computer‐Aided Civil and Infrastructure Engineering 23(7), 536-548.
Vlahogianni, E.I., Karlaftis, M.G., Golias, J.C., 2014. Short-term traffic forecasting: Where we are and
where we’re going. Transportation Research Part C: Emerging Technologies 43, 3-19.
Wang, Li, Liao, S.S, Hua, 2013. A Hybrid Approach for Automatic Incident Detection. IEEE Transactions
on Intelligent Transportation Systems 14(3), 1176-1185.
Wang, X., Guan, X., Cao, J., Zhang, N., Wu, H., 2020. Forecast network-wide traffic states for multiple
steps ahead: A deep learning approach considering dynamic non-local spatial correlation and non-
stationary temporal dependency. Transportation Research Part C: Emerging Technologies 119,
102763.
Wang, X., Sun, L., 2021. Diagnosing spatiotemporal traffic anomalies with low-rank tensor autoregression.
IEEE Transactions on Intelligent Transportation Systems 22(12), 7904-7913.
Wang, Y.F., Jiong, Y., Su, G.P., Qian, Y.R., 2019. A new outlier detection method based on OPTICS.
Sustainable cities and society 45, 197-212.
Wang, Z., Oates, T., 2015. Encoding time series as images for visual inspection and classification using
tiled convolutional neural networks, Workshops at the twenty-ninth AAAI conference on artificial
intelligence.
Xing, J., Liu, R., Anish, K., Liu, Z., 2023. A Customized Data Fusion Tensor Approach for Interval-Wise
Missing Network Volume Imputation. IEEE Transactions on Intelligent Transportation Systems.
Yang, H., Du, L., Mohammadi, J., 2021. A shock wave diagram based deep learning model for early alerting
an upcoming public event. Transportation research part C: emerging technologies 122, 102862.
Yuan, J., Abdel-Aty, M., Gong, Y., Cai, Q., 2019. Real-time crash risk prediction using long short-term
memory recurrent neural network. Transportation research record 2673(4), 314-326.
Yuan, Z., Chen, B., Liu, J., Chen, H., Peng, D., Li, P., 2023. Anomaly detection based on weighted fuzzy-
rough density. Applied Soft Computing, 109995.
Zeroual, A., Harrou, F., Sun, Y., 2019. Road traffic density estimation and congestion detection with a
hybrid observer-based strategy. Sustainable cities and society 46, 101411.
Zeroual, A., Harrou, F., Sun, Y., Messai, N., 2017. Monitoring road traffic congestion using a macroscopic
traffic model and a statistical monitoring scheme. Sustainable cities and society 35, 494-510.
Zhang, T., Ramakrishnan, R., Livny, M., 1996. BIRCH: an efficient data clustering method for very large
databases. ACM sigmod record 25(2), 103-114.
Zhang, W., Yan, S., Li, J., 2022. TCP-BAST: A novel approach to traffic congestion prediction with bilateral
alternation on spatiality and temporality. Information Sciences 608, 718-733.
Zhang, Y., Li, M., Wang, S., Dai, S., Luo, L., Zhu, E., Xu, H., Zhu, X., Yao, C., Zhou, H., 2021. Gaussian
mixture model clustering with incomplete data. ACM Transactions on Multimedia Computing,
Communications, and Applications (TOMM) 17(1s), 1-14.
Zhang, Z., He, Q., Gao, J., Ni, M., 2018. A deep learning approach for detecting traffic accidents from social
media data. Transportation research part C: emerging technologies 86, 580-596.
Zhang, Z., He, Q., Tong, H., Gou, J., Li, X., 2016. Spatial-temporal traffic flow pattern identification and
anomaly detection with dictionary-based compression theory in a large-scale urban network.
Transportation Research Part C: Emerging Technologies 71, 284-302.
Zhang, Z., Li, M., Lin, X., Wang, Y., He, F., 2019. Multistep speed prediction on traffic networks: A deep
learning approach considering spatio-temporal dependencies. Transportation research part C:
emerging technologies 105, 297-322.
Zhou, S., Ng, S.T., Yang, Y., Xu, J.F., 2021. Integrating computer vision and traffic modeling for near-real-
time signal timing optimization of multiple intersections. Sustainable Cities and Society 68, 102775.
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4737146
Preprint not peer reviewed
