ArticlePDF Available

An integrated architecture for autonomous vehicles simulation

Authors:

Abstract and Figures

Modeling and simulation tools are being increasingly acclaimed in the research field of autonomous vehicles systems, as they provide suitable test beds for the development and evaluation of such complex systems. However, these tools still do not account for some integration capabilities amongst several state-of-the-art Intelligent Transportation Systems, e.g. to study autonomous driving behaviors in human-steered urban traffic scenarios, which are crucial to the Future Urban Transport paradigm. In this paper we describe the modeling and implementation of an integration architecture of two types of simulators, namely a robotics and a traffic simulator. This integration should enable autonomous vehicles to be deployed in a rather realistic traffic flow as an agent entity (on the traffic simulator), at the same time it simulates all its sensors and actuators (on the robotics counterpart). Also, the statistical tools available in the traffic simulator will allow practitioners to infer what kind of advantages such a novel technology will bring to our everyday's lives. Furthermore, an architecture for the integration of the aforementioned simulators is proposed and implemented in the light of the most desired features of such software environments. To assess the usefulness of the platform architecture towards the expected realistic simulation facility, a comprehensive system evaluation is performed and critically reviewed, leveraging the feasibility of the integration. Further developments and future perspectives are also suggested.
Content may be subject to copyright.
An Integrated Architecture for Autonomous Vehicles
Simulation
José L. F. Pereira, Rosaldo J. F. Rossetti
Artificial Intelligence and Computer Science Laboratory
Department of Informatics Engineering
Faculty of Engineering, University of Porto
Rua Dr. Roberto Frias, S/N 4200-465 Porto Portugal
T: (+351) 225081566 F: (+351) 225574103
{jlpereira, rossetti}@fe.up.pt
ABSTRACT
Modeling and simulation tools are being increasingly ac-
claimed in the research field of autonomous vehicles sys-
tems, as they provide suitable test beds for the development
and evaluation of such complex systems. However, these
tools still do not account for some integration capabilities
amongst several state-of-the-art Intelligent Transportation
Systems, e.g. to study autonomous driving behaviors in
human-steered urban traffic scenarios, which are crucial to
the Future Urban Transport paradigm.
In this paper we describe the modeling and implementa-
tion of an integration architecture of two types of simula-
tors, namely a robotics and a traffic simulator. This inte-
gration should enable autonomous vehicles to be deployed in
a rather realistic traffic flow as an agent entity (on the traffic
simulator), at the same time it simulates all its sensors and
actuators (on the robotics counterpart). Also, the statistical
tools available in the traffic simulator will allow practition-
ers to infer what kind of advantages such a novel technology
will bring to our everyday’s lives. Furthermore, an architec-
ture for the integration of the aforementioned simulators is
proposed and implemented in the light of the most desired
features of such software environments.
To assess the usefulness of the platform architecture to-
wards the expected realistic simulation facility, a compre-
hensive system evaluation is performed and critically re-
viewed, leveraging the feasibility of the integration. Further
developments and future perspectives are also suggested.
Keywords
Autonomous Driving, Autonomous Vehicles Simulation,
Robotics Simulation, Microscopic Traffic Simulation, Agent-
based Simulation
Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that copies
bear this notice and the full citation on the first page. To copy otherwise, to
republish, to post on servers or to redistribute to lists, requires prior specific
permission and/or a fee.
SAC’12 March 25-29, 2012, Riva del Garda, Italy.
Copyright 2011 ACM 978-1-4503-0857-1/12/03 ...$10.00.
1. INTRODUCTION
The autonomous vehicle paradigm has been around in tin-
kerers minds for some years now, as it envisions to radically
change the concept of mobility. With the technology boom,
researchers are now more focused on the software challenges
that such a complex system requires, as hardware itself is be-
coming more affordable. Also, some authors agree that the
autonomous vehicle paradigm is rather a software problem
([4], [11] or [15]). To accomplish such a goal, current traffic
modeling and simulation tools need to be adapted having in
mind its requirements, as they already account for a multi-
tude of features related to traffic research and engineering,
which can be great helpers when applied to the development
of driverless vehicles.
A careful analysis throughout the recent Defense Advanced
Research Projects Agency (DARPA) challenge competition
demonstrates the embrionary state of such tools. Often the
teams used robotics simulators with no cooperation capabil-
ities, which implemented simplified models for vehicle and
object collision, not offering any kind of realistic behaviour
or traffic analysis tools [6, 7, 8, 16].
In this paper we intend to contribute to the autonomous
vehicles simulation tools introducing an integrated architec-
ture, seamlessly coupling both a traffic and a robotics sim-
ulator. Such a tool would allow practitioners to have in
consideration the robotic nature of an autonomous vehicle
which uses a set of sensors/actuators to perceive the real
world, making intelligent actions on navigation and planning
– at the same time it simulates it within a virtual urban traf-
fic network.
The remainder of the paper is organized as follows. We
first discuss on the details of the proposed architecture, em-
phasizing on integration aspects. The following section is
dedicated to presenting the developed prototype, where some
technological decisions are made. We illustrate the approach
with the implementation of a reactive agent and conclude
with suggesting further developments.
2. THE PROPOSED ARCHITECTURE
Depicted in Figure 1, a software architecture for the au-
tonomous vehicle simulation in a traffic environment is pro-
posed. This architecture has a distributed nature, as each
simulator must use the maximum possible resources. It con-
sists of four major modules, briefly described below:
Microscopic Traffic Simulator Simulates almost all ve-
286
microscopic
traffic simulator
robotics simulator
game engine
traffic network
data
networked
data
robotics framework
autonomous
driver agent
robotics framework
autonomous
driver agent
Figure 1: The proposed architecture for autonomous vehicle
simulation in a traffic environment
hicles with a resemblance to the macroscopic simula-
tion of real traffic streams. It also maintains all in-
frastructure systems, such as induction loops or traffic
light plans. Given the extensibility of the simulator,
a higher level statistical framework may be coupled in
order to study traffic behavior patterns of individual
and cooperative strategies for autonomous vehicles;
Robotics Simulator Performs the simulation of all au-
tonomous vehicles in the environment, along with all
its sensors and actuators, and mirrors every surround-
ing object. It also features a game engine for immer-
sive 3D animation through both powerful physics and
visualization modules;
Coherent Network Data Represents the traffic network
topology model, as well as its realistic 3D environment
data;
Autonomous vehicle interface and control Manages the
brain of the vehicle. Typically an external software
driver can be deployed to perform the autonomous ve-
hicle high-level tasks using an agent-based methodol-
ogy. A Hardware Abstraction Layer (HAL) is under-
neath for seamless real/virtual world development, and
sensor/actuator permutation.
We can acknowledge the bidirectional communication be-
tween each simulator, whereas the autonomous vehicle pro-
vides its kinematic variables to the traffic simulator, and the
traffic simulator calculates the world state and return back
its surrounding data. All of these transactions should occur
in the same time step.
3. PROTOTYPE DEVELOPMENT
3.1 Simulator Selection
As pointed out earlier, using two types of simulators may
be a feasible approach when simulating autonomous vehi-
cles on an urban environment. Such an idea was already
introduced in [5], although the selected software architec-
ture and frameworks did not provide satisfactory results. In
the former paper, the chosen traffic and robotics simulation
software did not have the sufficient maturity level, which
led to various implementation difficulties and, consequently,
to an inefficient platform. However, its findings have pro-
vided some lights on the usefulness of the integration and
difficulties to be overcome.
In [9], the authors present a comprehensive review to the
simulation of autonomous vehicles, having in mind both
robotics and traffic simulators. This review was crucial to
the decision of using both SUMO and USARSim traffic and
robotics simulator to this project. Although their maturity
level is inferior to their commercial counterparts, they have a
strong support from the open-source community and would
allow for full core access which should be favorable for low
level optimizations.
SUMO [14] is a highly portable, microscopic road traffic
simulation package designed to handle large road networks
and has a strong commitment with the academia and re-
search community. It is mainly developed at Institute of
Transportation Systems at the German Aerospace Center
and it is written is C++.
USARSim [3] is an open-source high-fidelity robotics sim-
ulator based on the Unreal Tournament game engine. Due
to Unreal license restrictions, USARSim is written in Unreal
Script, the official scripting language which offers interface
with the engine core. This particular issue can be critical
as there is no direct access to all the features of the engine.
However, USARSim’s current development state, high qual-
ity sensor simulation and physics rendering makes it the best
choice to the project. Another major drawback could be its
dependency on Windows operating system, a constraint by
Unreal Engine 3. However, as there are no other dependen-
cies to the remaining software, all the produced software is
multi-platform.
To describe the proposed framework a prototype was de-
veloped using SUMO and USARSim. Its methodological
approach is many-fold:
Modify SUMO simulator SUMO is a vehicle- and edge-
based microscopic simulator. Therefore, it is not pre-
pared to allow lane-independent vehicles to circulate.
Also, a spawning mechanism must be implemented to
carefully place and identify an autonomous vehicle in
the network. Finally, a communication class must be
implemented for it to efficiently communicate with US-
ARSim;
Modify USARSim simulator USARSim is also not fully
prepared to accommodate a urban autonomous vehi-
cle. The latest version (based on Unreal Engine 3)
does not provide a LIDAR range scanner, and there
are no 3D models of a four wheeled conventional vehi-
cle and realistic terrain. Similarly to the previous step,
a communication class needs to be implemented;
Integrate simulators Apply the required modifications to
integrate both simulators;
Implement a control agent To successfully validate the
proposed architecture and prototype in its essence,
a simple agent was devised with a simple dashboard
showing real-time sensor data from the autonomous ve-
hicle. The control algorithm moves the vehicle through
a white line, obeying the car-following rule, and possi-
bly deviating from random objects.
287
(a) Aliados
in Open-
StreetMaps
(b) Aliados in
netEditor
(c) Aliados in a 3D
model
Figure 2: Aliados in several representations.
3.2 Reference Network
The reference network for which the autonomous vehi-
cles were simulated on is part of the Aliados area in Porto,
Portugal. Its model has to be represented in two forms,
as each one will fit the requirements of the two simulators.
Such models are denominated a topological model and a 3D
model for SUMO and USARSim respectively.
In the SUMO case, the netEditor [10], a pluginable traffic
network editor, was used to import the required network
from OpenStreetMaps (OSM), implement an O-D Matrix
for starting nodes, and select all allowed directions on each
road. Figure 2a represents the OSM Aliados topology, and
Figure 2b the imported network from OSM.
Regarding the USARSim simulator, the 3D model from
Aliados was gathered using a software designed and devel-
oped in [12]. This software uses Procedural Modeling and
geographically referenced information to render a realistic
scenario, tipically used on computer games. Giving that this
work intended to make the simulation framework as realis-
tic as possible, a rapid prototyping tool for 3D scenarios was
imperative. Figure 2c depicts Aliados 3D model rendered in
the Autodesk 3ds Max modeling tool [1].
3.3 Surrounding Vehicles Integration
We are ready to manage all surrounding vehicle infor-
mation from SUMO to USARSim. SUMO was accordingly
modified to calculate the surrounding vehicles around an au-
tonomous vehicle and therefore, an algorithm to handle such
vehicles was devised. It uses a hash-table comprehending ve-
hicles already created on the scene, and calculates their next
movements. Its pseudo-code is represented by Algorithm 1.
AbsMove(veh), Create(veh) and Kill(veh) corresponds to the
WorldController USARSim command definitions [2].
3.4 Network Coherence
Having successfully imported the Aliados road network
into SUMO using the netEditor and to USARSim using the
procedural modeling application, we implemented a network
calibration method between each simulator scenario model.
As both networks for the two simulators rely on different
sources, albeit they both resemble the same physical zone
of Aliados, in Porto city, their coordinate references are not
the same. To overcome this, an Euclidean transformation
method was used to map a point from one coordinate sys-
tem to the other. Also, to calculate the transformation pa-
rameters (A matrix and b
1
, b
2
), the practitioner would only
need to know the location of two points in both simulators
Algorithm 1 Surrounding vehicles control algorithm
surrV ehicles = myV ehicle.getSurrV ehicles()
for curV eh = surrV ehicles.f irst()
surrV ehicles.last() do
if vehHashMap.exists(curV eh) then
AbsMove(curV eh)
else
Create(curV eh)
AbsMove(curV eh)
vehHashM ap.insert(curV eh)
end if
curV eh curV eh + 1
end for
for curV eh = lastSurroundingV ehicles.first()
lastSurroundingV ehicles.last() do
if !vehHashMap.exists(curV eh) then
Kill(curV eh)
end if
curV eh curV eh + 1
end for
lastSurroundingV ehicles surrV ehicles
coordinate system.
We must note that despite the 3D nature of the USARSim
simulator, as the 3D model contains plain roads, the z value
will always remain constant, therefore this transformation is
T : R
2
R
2
.
This coordinate transformations were implemented on SUMO
simulator given its C++ native implementation and a stan-
dard library containing the trigonometric functions.
Having the coordinate mapping between the two simula-
tors ready, the vehicles from SUMO were correctly trans-
lated to USARSim and the autonomous vehicle was also
shown on SUMO. However, this improvements have shown
a network disparity between the two models, as the road
lengths and positions from each other were not exactly the
same thus mirroring the vehicles in a slight different position
from where they should have been drawn. To address this
issue, the Aliados SUMO network description was edited in
netEditor to approximate the two network positions, how-
ever the lack of a proper tool that could render both net-
works at the same time prevented us to carry out this task
on the most approximate way.
3.5 Multi-agent communication
The USARSim platform implements a Wireless Commu-
nications Server to act as a middle man between robots, sim-
ulating message and connection dropping when its distance
is not realistically feasible which can be used for Multi-Agent
Systems based coordination methodologies. In the following
section an example implementation of a reactive agent to
control an autonomous vehicle in USARSim is detailed, as
a mean to introduce a practitioner to the development of
agent-based vehicle control in the proposed platform. How-
ever, for the sake of time this agent does not feature any
type of communication abilities with other vehicles. Refer
to the USARSim documentation for further elucidation on
the Wireless Communications Server protocol [2].
288
Reactive Agent
Rule: Condition ->
-> Action
Internal State
of the World
Filter of
Actions
Sensors
Action
Environment
Figure 3: Simple reactive agent diagram (adapted from [13]).
Figure 4: Optical Camera and LIDAR sensor visualization
interface on a preliminary simulation in USARSim.
4. SIMPLE REACTIVE AGENT
To validate the proposed framework and its implementa-
tion as already discussed earlier, a simple autonomous agent
was developed to spawn a vehicle in the simulation, receive
its sensors information and calculate a trajectory based on
it. It follows a reactive agent methodology, with the model
as depicted in Figure 3.
This application was coded in C++ with the Qt4 frame-
work from Nokia, which provides cross-platform Graphical
User Interface (GUI) and networking capabilities.
4.1 Agent Architecture
The simple reactive agent software architecture is depicted
in Figure 5. It consists of a central class controller which
sets up the communication QTcpSocket with USARSim and
instantiates an usarsim sensor for each sensor in the vehicle.
It uses an hash-table that maps each sensor id to its corre-
spondent object, while redirecting the received sensor mes-
sage for parsing. Each sensor also inherits a QWidget class,
containing sensor information in a graphical format depen-
dent of the sensor type. For example, a LIDAR (rangeS-
canner3D) sensor provides a 3D point cloud visualization in
OpenGL as illustrated in Figure 4.
The reactive agent rules set is represented by the rule
set
class, to which the controller feeds with received sensor data,
and retrieves the proper driving speed an steer angle to ap-
ply onto the robot vehicle. In this reactive agent example,
the camera is used to process the lane position and tries to
maintain its aim at it, whereas the LIDAR sensor prevents
the vehicle to collide with close objects. Figure 6 depicts
usar_sensor
1
*
rangeScanner3D
gps
imu
odometer
camera
QWidget
GUI
rule_set
controller
1
1
QTcpSocket
1
1
Figure 5: Simple reactive agent architecture.
a sequence diagram with the most important transactions
between the simulators and the reactive agent.
4.2 Navigation
This section provides more detailed description on the
Drive and Steer commands sent to USARSim, using LIDAR
and camera sensors respectively, as an example of a simple
autonomous control for an urban vehicle. The reader must
note that these methods are not validated to be used on ro-
bust autonomous vehicle control, but only to model a very
simple navigation algorithm for the sake of demonstration.
To drive the vehicle, only the front side point data from
the LIDAR sensor is considered to make it aware of its front
vehicles. Furthermore, we select a square window with hor-
izontal and vertical angles (typically 5 to 10 degrees), and
apply the mean to it. Afterwards, the Drive speed is calcu-
lated using the following formula (Equation 1):
drive speed = max speed · (1 e
m/k
1
) · (1 e
α/k
2
) (1)
The expressions in parenthesis attenuate the speed de-
pending on the measured mean m to make the vehicle slow
down in other vehicles’ presence, and on the steering angle
α, to prevent it to steer in high speeds.
The vision-based method to support vehicle steer deci-
sions is processed in the following steps:
Image binarization with a predefined threshold to filter
lanes only (in white);
Calculate white pixel density of both left (ρ
left
) and right
(ρ
right
) side of the image;
Implement a proportional control with the density dif-
ference, given by Equation 2.
α = K · (ρ
right
ρ
left
) (2)
Therefore if the left pixel density is higher, the steering
angle α will be negative and the vehicle will move towards
289
create robot
SUMO
USARSim
Reactive
Agent
start()
spawn vehicle
done
done
[for each timestep]
get sensor data
get surroundings
done
step()
step()
sensor data
process sensors ()
set drive and steer
parameters
done
loop
Figure 6: Sequence diagram stating the interaction between
simulators and the reactive agent.
Figure 7: Framework synchronization aspect. On the first
row, SUMO view is presented (autonomous vehicle yellow
painted) followed by USARSim’s. The camera and LIDAR
sensors from the reflective agent are depicted in last.
Figure 8: The binarization process of the vehicle front cam-
era image.
PC1
USARSim
PC2
SUMO
reflective
agent
1
Network flow
1
1 *
Figure 9: Computer-application arrangement to perform
evaluation tests.
left. If the right pixel density is higher, α will be positive
and the vehicle will move towards right.
From Figure 7, we can state the successfulness of the test.
On the first row we can visualize the autonomous vehicle
as the yellow car, which is surrounded by three other cars;
one at front, one at the back and another on the left side.
From the USARSim view we can see the same environment
in a 3D model which confirms the synchronization of the two
simulators. A careful look at the network topology yields a
slight offset amongst the vehicles position in each simula-
tor, as a consequence of the calibration issues already dis-
cussed in Section 3.4. The last row on the figure depicts
the camera image, which is seeing the front vehicle on the
top, whereas a LIDAR point cloud data clearly detecting
the road is depicted below. Given the limited resolution of
the range sensor it is not evident the presence of a car just
using its data.
5. RESULTS AND DISCUSSION
To properly validate the integration architecture and im-
plementation devised for the project, several metrics were
analyzed to measure its effectiveness and therefore give a
more consistent critic about each part of the system. This
way, we can assess the selected choices of the traffic and
the robotics simulators, SUMO and USARSim respectively,
which seemed the most appropriate to be used in this project.
Table 1: Two computer set-ups used to evaluate the inte-
grated platform.
PC PC1 PC2
CPU Intel C2D E8500 Intel C2D E6550
Clock speed 3.16GHz 2.33GHz
RAM size 3.71GB 3.00GB
Graphics Card Intel Q45 Xpress Nvidia GF 8600GT
Operating System Windows 7 Ubuntu 11.04
This evaluation tests were performed on two relatively
recent desktop computers available in our department de-
nominated PC1 and PC2. Table 1 illustrates the computers
specifications, which should be considered in the critical as-
sessment of the result values. Also, Figure 9 shows which
application was executed on which computer.
To evaluate the platform performance to be eventually
scrutinized against other frameworks, the CPU usage of both
simulators were assessed as well as the network bandwidth
in several test runs.
290
Figure 10: Performance Test Run 1 - SUMO simulator per-
formance with several connected clients.
These test runs use the bundled operating system profiling
tools to measure the aforementioned metrics. Thus, on Win-
dows 7 (the PC1 machine) the metrics are accessed using the
Performance” tool on Administrative Tools > Performance.
In Ubuntu Linux, the top and iftop commands provided the
CPU and network usage respectively.
The first approach was to simulate a SUMO network con-
necting several autonomous vehicle entities and evaluating
the CPU usage and network bandwidth used by them. As
one of the main goals of the framework is to retain a real-
time refresh-rate, i.e. a minimum 30Hz, when the network
latency is superior to 33ms the test-run is stopped. The
steps to be taken into account in this SUMO performance
test are:
1. Start SUMO loaded with Aliados network in PC2;
2. Start the number of instances of the client sample as
required in PC2;
3. Wait until the simulation reaches 10s simulation time;
4. Record both CPU usage and network bandwidth on
PC2.
Figure 10 depicts the achieved results from the test run.
It really looks promising as we can simulate over 1000 au-
tonomous vehicles without significant loss. However, we
have to note that when the simulation grows with more ve-
hicles, the consequent number of vehicles surrounding the
autonomous vehicles will increase, and so will the network
flow and CPU usage.
The next test run depicts the CPU usage and network
bandwidth of USARSim connected from one to several au-
tonomous vehicle agents, with all the sensors from the im-
plemented autonomous vehicle. Therefore, the steps to re-
produce the test run are:
1. Start USARSim loaded with Aliados network in PC1;
2. Start the number of instances of the reflective agent as
required in PC2;
3. Wait until the simulation reaches 10s simulation time;
4. Record both CPU usage and network bandwidth on
PC1.
Figure 11 plots the results acquired during this second
test run. Despite being run on PC1, the fastest machine,
USARSim consumes a lot of CPU power given its realistical
simulation complexity. Furthermore, the lack of a dedicated
Physics Processing Unit (PPU) leaves all physics calcula-
tions to the CPU. The required network bandwidth is also
significantly higher than in its SUMO counterpart, as sen-
sor data is quite large (5 sensors) and the network protocol
is not serialized thus wasting higher bandwidth (typically it
Figure 11: Performance Test Run 2 - USARSim simulator
performance with several reactive agents.
needs 50 to 100 bytes per each message whereas serialized
buffer only require 5 to 20 or even less).
From the practical results we can infer that the required
network bandwidth for the framework application can rep-
resent a bottleneck if certain conditions are met. Therefore,
researchers who intend to adapt this architecture are encour-
aged to comprehensively study the message passing efficacy
when selecting the simulation tools.
6. CONCLUSIONS
This paper has presented the methodological and techni-
cal challenges of a practical solution for the coupling of two
simulators, namely a traffic simulator and a robotics sim-
ulator to increase the reliability of simulation with several
self-driving vehicles, seamlessly integrated within a common
traffic environment. An integration architecture was devised
pinpointing the main aspects towards an efficient communi-
cation among simulators, which have been addressed to some
extend. This flexible approach was not bound to any spe-
cific simulation software architecture thus opening up new
opportunities to the development of other platforms. Also,
a prototype to demonstrate the integration was also devised
and effectively implemented using both SUMO and USAR-
Sim, which were accordingly patched to meet the integration
requirements. To couple these two simulators, a method
for the management of seamless vehicle mirroring was im-
plemented using efficient data types and networking. The
robotics simulator provided its position to the traffic one,
whereas the latter calculates all microsimulation vehicles,
feeding back their position to the robotics simulator in the
same time step.
The effectiveness of the framework has culminated in the
need for modeling and developing the reactive agent, which
comprehends a simple navigation and control algorithm as a
means to exemplify and measure the framework’s promising
potentiality and efficiency.
Giving the complexity of the simulation frameworks used
in this project, the next step towards its solidification would
be to fix some occasional bugs that might still persist, partic-
ularly on SUMO as it was subjected to several modifications
and the simulation of real complex traffic networks.
Ultimately, some coordinate mapping techniques could be
assessed to study the possibility of using three dimensional
topologies of traffic roads in the robotic simulator side, while
using the same plain roads in the traffic simulator. The
network model calibration method should be evaluated as
the result of importing from two different data sources.
Another long-term goal would be to foster the integration
of this platform with even one more simulator, namely a
pedestrian simulator, whereby it can push further the real
291
implementation of an even more realistic Artificial Trans-
portation System.
As we are all aware, the advent of autonomous vehicles
will boost the need for modern R&D tools for such complex
systems to be evaluated. Furthermore, there is a recognized
potential on this proposed framework, as it should be able
to approximate the modern robotics research methodologies
to Future Urban Transport Systems.
7. REFERENCES
[1] Autodesk autocad 2011. http://www.autodesk.com/,
2011.
[2] Usarsim reference documentation.
http://iweb.dl.sourceforge.net/sourceforge/
usarsim/usarsim-manual_3.1.3.pdf, 2011.
[3] S. Carpin, M. Lewis, J. Wang, S. Balakirsky, and
C. Scrapper. USARSim: a robot simulator for research
and education. Proceedings 2007 IEEE International
Conference on Robotics and Automation, pages
1400–1405, Apr. 2007.
[4] E. Dickmanns, R. Behringer, D. Dickmanns,
T. Hildebrandt, M. Maurer, F. Thomanek, and
J. Schiehlen. The seeing passenger car ’vamors-p’. In
Intelligent Vehicles ’94 Symposium, Proceedings of the,
pages 68 73, oct. 1994.
[5] M. C. Figueiredo. An Approach to Simulation of
Autonomous Vehicles in Intense Traffic Scenarios.
MSc. Thesis, (June), 2009.
[6] G. Franken. DARPA Urban Challenge Princeton
University Technical Paper. Access, 2005.
[7] T. Leader, C. Reinholtz, R. Hall, M. Code, V. Tech,
T. Alberi, J. Farmer, S. Frash, C. Terwelp, and
A. Wicks. DARPA Urban Challenge Technical Paper.
Defense, 2007.
[8] U. Ozguner, K. Redmill, S. Biddlestone, A. Yazici,
and C. Toth. Simulation and testing environments for
the DARPA Urban Challenge. 2008 IEEE
International Conference on Vehicular Electronics and
Safety, pages 222–226, Sept. 2008.
[9] J. L. F. Pereira and R. J. F. Rossetti. Autonomous
vehicles simulation: A comprehensive review. In
Proceedings of the 25th European Simulation and
Modelling Conference, ESM’2011, October 2011.
[10] J. L. F. Pereira, R. J. F. Rossetti, and E. Oliveira.
Towards a cooperative traffic network editor. In
Y. Luo, editor, Cooperative Design, Visualization, and
Engineering, volume 5738 of Lecture Notes in
Computer Science, pages 236–239. Springer Berlin /
Heidelberg, 2009.
[11] D. Pomerleau and T. Jochem. Rapidly adapting
machine vision for automated vehicle steering. IEEE
Expert, 11(2):19 –27, apr 1996.
[12] R. Rodrigues, A. Coelho, and L. Reis. Data model for
procedural modelling from textual descriptions. In
Evolutionary Computation (CEC), 2010 IEEE
Congress on, pages 1 –8, july 2010.
[13] S. Russell and P. Norvig. Artificial Intelligence: A
Modern Approach. Prentice-Hall, Englewood Cliffs,
NJ, 2nd edition edition, 2003.
[14] C. Sommer, Z. Yao, R. German, and F. Dressler.
Simulating the influence of ivc on road traffic using
bidirectionally coupled simulators. In INFOCOM
Workshops 2008, IEEE, pages 1 –6, april 2008.
[15] B. Ulmer. VITA-an autonomous road vehicle (ARV)
for collision avoidance in traffic. In Intelligent
Vehicles’ 92 Symposium., Proceedings of the, pages
36–41. IEEE, 1992.
[16] M. Xie, H. Chen, X. F. Zhang, X. Guo, and Z. P. Yu.
Development of Navigation System for Autonomous
Vehicle to Meet the DARPA Urban Grand Challenge.
2007 IEEE Intelligent Transportation Systems
Conference, pages 767–772, Sept. 2007.
292
... and CARLA [6] encompass mainly the physics aspects of the simulations and do not address concepts like sensors communication, security, Cloud communication, and external data sources. In this sense, co-simulation is a popular approach explored in the literature in the last decade [4,16,19]. For instance, in [16], the authors integrate Gazebo with an external Wireless-Sensors Network (WSN) simulator to detect misbehavior in vehicular networks. ...
... Through the SmartData abstraction, replacing a data source or introducing a new one becomes transparent to the application. The majority of the solutions available in the literature lack of transparent integration of components [4,16,19], being one of the main advantages of our approach. Moreover, the proposed solution promotes time synchronization and the addition of security whenever needed, as both concepts are intrinsic to SmartData. ...
... In [19], the authors propose an architecture for integrating robotics and traffic simulators for AVs, specifically USARSim and SUMO [18]. They modify SUMO and USARSim applications to marshal information between simulators, enabling vehicle creation, movement, or deletion synchronously. ...
Article
Full-text available
Simulations are key steps in the design, implementation, and verification of autonomous vehicles (AV). Parallel to this, typical simulation tools fail to integrate the entirety of the aspects related to the complexity of AV applications, such as data communication delay, security, and the integration of software/hardware-in-the-loop and other simulation tools. This work proposes a SmartData-based middleware to integrate AV simulators and external tools. The interface models the data used on a simulator and creates an intermediary layer between the simulator and the external tools by defining the inputs and outputs as SmartData. A message bus is used for communication between SmartData following their Interest relations. Messages are exchanged following a specific protocol. Nevertheless, the architecture presented is agnostic of protocol. Moreover, we present a data-centric AV design integrated into the middleware. The design considers the standardization of the data interfaces between AV components, including sensing, perception, planning, decision, and actuation. Therefore, the presented design promotes a transparent integration of the AV simulation with other simulators (e.g., network simulators), cloud services, fault injection mechanisms, digital twins, and hardware-in-the-loop scenarios. Moreover, the design allows for transparent, runtime component replacement and time synchronization, the modularization of the vehicle components, and the addition of security aspects in the simulation. We present a case-study application with an AV simulation using CARLA, and we measure the end-to-end delay and overhead incurred in the simulation by our middleware. An increase in the end-to-end delay was measured once data communication was not acknowledged in the original scenario, and data was assumed to be ready for processing with no communication delay between sensors, decision-making, and actuation units.
... The additional networks assisting the driving such as GPS and IMUs map the data to accurately determine the vehicle's position within its environment for safety. Based on the localized data, a path can be planned that would be followed by the AV, and the acceleration, braking and driving decisions would be processed in real time using sensor fusion and high-performance computing [50]. The control system present in the dense compute engine makes sure that the vehicle follows the traffic rules and safety requirements. ...
Article
Real-time applications always have security issues present in their ecosystem which needs to be improved. Blockchain technology can be considered as a solution that can ensure network security. The integration of blockchain technology into autonomous systems has now become an important candidate to resolve network security issues in devices that work on p2p communications. As humans become more reliant on autonomous vehicles, the importance of finding a solution to the problems linked with data security needs to be addressed to comply with international security laws. The paper aims to find the importance of blockchain technology and how it can be integrated into different classes of autonomous vehicles. In the research, an analysis of existing papers will be included along with a proposed design that can be deployed to solve the problems associated with data security using blockchain technology. Besides the improvements offered by blockchain technology, all the additional challenges associated with designing a new system and deploying this technology will be reviewed. The limitations of the works along with the future research opportunities that might open up will be presented in this paper.
... Early studies aimed to build a framework for modeling sensors, digitizing the real world, and gathering datasets using driving fleets with attached automotive sensors. Gschwandtner et al. [20] introduced a 3D scan simulator for commercial LiDAR sensors, while Pereira et al. [21] presented an integrated framework for a traffic simulator based on automotive sensor models (LiDAR and camera). ...
Article
Full-text available
The Light Detection and Ranging (LiDAR) sensor has become essential to achieving a high level of autonomous driving functions, as well as a standard Advanced Driver Assistance System (ADAS). LiDAR capabilities and signal repeatabilities under extreme weather conditions are of utmost concern in terms of the redundancy design of automotive sensor systems. In this paper, we demonstrate a performance test method for automotive LiDAR sensors that can be utilized in dynamic test scenarios. In order to measure the performance of a LiDAR sensor in a dynamic test scenario, we propose a spatio-temporal point segmentation algorithm that can separate a LiDAR signal of moving reference targets (car, square target, etc.), using an unsupervised clustering method. An automotive-graded LiDAR sensor is evaluated in four harsh environmental simulations, based on time-series environmental data of real road fleets in the USA, and four vehicle-level tests with dynamic test cases are conducted. Our test results showed that the performance of LiDAR sensors may be degraded, due to several environmental factors, such as sunlight, reflectivity of an object, cover contamination, and so on.
Chapter
Testing and evaluation for vehicle safety (TEVS) is to measure, analyze and judge vehicle safety performance through a series of techniques and methods with reference to certain standards, and then make qualitative or quantitative evaluation conclusions.
Article
Automated and connected vehicles are emerging in the market. Currently, solutions are being proposed to use these technologies for cooperative driving, which can significantly improve road safety. Vehicular safety applications must be tested before deployment. It is challenging to verify and validate them in the real world. Therefore, simulation is used for this purpose. Modeling this technology necessitates coupled use of traffic flow, vehicle dynamics, and communication network simulators. State-of-the-art tools exist in these domains; however, they are difficult to integrate or lack full domain coverage. This article analyzes the requirements for an integrated connected and automated vehicle simulation environment for simulating vehicular cooperative driving automation with consideration of surrounding objects’ influence. For this purpose, we have assessed the existing challenges and practices. Vehicular simulation tools, signal propagation, and cooperative perception models are reviewed and analyzed. In our review, we focus mainly on autonomous driving simulators with 3D graphical environments as they have not yet been assessed for cooperative driving task fitness. Further, the current state of connected and automated vehicle simulation studies using these tools is surveyed, including single-tool and co-simulation approaches. We discuss the shortcomings of existing methods and propose an architecture for an integrated simulation environment (ISE) with full domain coverage using open source tools. The obtained conclusions can be further used in the development of connected and automated vehicle ISEs.
Article
Due to recent advancements in connected and autonomous vehicles (CAVs), freeway traffic control (FTC) has become a trendy area of research. Combining CAVs and control measures has opened more efficient traffic flow possibilities. However, while there is extensive research on how traffic control and CAVs can enhance traffic flow performance, their safety benefits should be evaluated more comprehensively. It is challenging to distinguish their contributions to traffic flow safety. Studies on the safety of CAVs in traffic flow indicate that traffic flow safety improves with higher market penetration rates (MPR) of CAVs. However, this finding only sometimes aligns with the traffic flow performance effects of CAVs on freeways. This paper introduces a framework for assessing the safety effects of FTC strategies in mixed traffic scenarios involving human-driven vehicles and CAVs. The proposed framework is tested through a microsimulation-based case study in Istanbul, Turkey. The results show that, despite varying MPRs, the safety effects of CAVs and FTC methods do not consistently reduce the number of conflicts in the traffic flow context.
Article
Autonomous transportation systems (ATSs) focus on traffic services and have become more capable of handling stochastic traffic scenarios. The preliminary work for designing and constructing ATS management and control systems is to develop an instructive and systematic architecture. However, the construction process of a conventional traffic system generally follows a function- and demand-oriented way. Thus, few studies have addressed architecture integrity evaluation and verification. In this work, we propose a collaboration mechanism-based approach to ATS architecture integrity evaluation. First, to provide a clear description of ATSs, ATS theory was systematically introduced through the generation, definition, and property of its basic elements, and then the relations between each element were analyzed based on the concept of microservices. Second, a collaboration mechanism on physical architecture was proposed to realize the adaptive construction of the architecture. Additionally, to fulfill the evaluation of architecture integrity, a contrast scheme combined with system function and information flow (IF) was determined, and accordingly, two independent ways were determined to obtain both the reference and calculated results. Finally, the scenario of automatic vehicles passing through an intersection was illustrated to demonstrate the feasibility of this static simulation evaluation method for the scenario architecture, and achieved a 100% function integrity rate and 93.3% IF integrity rate. A successful application in the case study shows that the proposed method could be extended to more general traffic scenarios.
Conference Paper
Full-text available
This paper presents USARSim, an open source high fidelity robot simulator that can be used both for research and education. USARSim offers many characteristics that differentiate it from most existing simulators. Most notably, it constitutes the simulation engine used to run the virtual robots competition within the Robocup initiative. We describe its general architecture, describe examples of utilization, and provide a comprehensive overview for those interested in robot simulations for education, research and competitions.
Conference Paper
Full-text available
The most common cause of traffic accidents is arguably the driver error due to lack of attention. And it is very unlikely this is going to change soon thanks to increasingly cell-phone usage, in-car entertainment systems, and naturally the more frequent traffic jams in highly populated areas. Autonomous vehicles, such as driverless cars, are a promising approach to decrease traffic accidents, as well as congestions. To test this approach, simulations are a safer, more efficient, and cheaper way than live testing. This paper presents an approach to implement a simulator to test such vehicles. It includes a study of the state of the art in driverless car simulation and discusses on the specific objectives that this particular simulator aims to achieve in order to aid testing the interactions of multiple driverless cars in urban networks.
Conference Paper
Full-text available
In this paper we explore the potential benefits of concepts such as visual interactive modelling and simulation to devise and implement a cooperative network editor embedded in a collaborative environment for transport analysis. Traditional approaches lack adequate means to foster integrated analyses of transport systems either because they are strict in terms of purpose or because they do not allow multiple users to dynamically interact on the same description of a model. The use of a common geographical data model of the application domain promotes the means for different experts to interact seamlessly in a collaborative environment.
Article
Research on autonomous vehicles has come a long way since first findings, and its software tools are increasingly acclaimed by the research community. Particularly with robotics simulators, autonomous vehicles have been provided with a suitable test-bed for experimentation of new methodologies such as long-term navigation algorithms, map building and intelligent reasoning. How-ever, when it concerns the deployment and validation of such vehicles in a larger urban traffic scenario, robotics simulators do not seem to provide the required functionality for road traffic analysis, or inter-vehicular communication infrastructure as they seem present in today's traffic simulators. The improvement of such features is the key for the successful practical deployment of such a critical system. In this paper, we present a study on the currently available robotics and traffic simulators focused in its applicability on autonomous vehicles research. The simulator integration approach was introduced by Figueiredo et al. (2009), although the selected software architecture and frameworks did not provide satisfactory results. In the former paper, the traffic and robotics simulation softwares chosen did not have the sufficient maturity level, which led to various implementation difficulties and, consequently, to an inefficient platform. However, its findings have provided some lights on the usefulness of newer possible integrations and difficulties to be overcome. In this paper we present the current features and issues on some of the most known robotics and traffic simulators, and propose a taxonomy for selecting these simulators, in order to foster further developments on the topic.
Conference Paper
In this paper we provide an overview of the simulation and testing environments developed for the Ohio State Universitypsilas DARPA Urban Challenge vehicle OSU-ACT (Autonomous City Transport). These environments were used in the research and development process of the actual challenge vehicle, and also extended for classroom teaching purposes and for general autonomous vehicle research.
Chapter
The long-anticipated revision of this best-selling book offers the most comprehensive, up-to-date introduction to the theory and practice of artificial intelligence. Intelligent Agents. Solving Problems by Searching. Informed Search Methods. Game Playing. Agents that Reason Logically. First-order Logic. Building a Knowledge Base. Inference in First-Order Logic. Logical Reasoning Systems. Practical Planning. Planning and Acting. Uncertainty. Probabilistic Reasoning Systems. Making Simple Decisions. Making Complex Decisions. Learning from Observations. Learning with Neural Networks. Reinforcement Learning. Knowledge in Learning. Agents that Communicate. Practical Communication in English. Perception. Robotics. For those interested in artificial intelligence.
Conference Paper
The generation of three-dimensional models of urban environments using procedural modelling is presented as being a solution that allows financial and temporal gains, maintaining an acceptable visual fidelity level. Nevertheless, the modelling of anchor buildings (or monumental), identifying certain urban areas, needs a more careful modelling, due to the elevated level of detail necessary, using, generally, manual modelling. This paper proposes a data model to be used on an architecture for high level modelling of monumental buildings, through the introduction of additional knowledge from textual information. The firsts results shows that the data model is flexible enough to build distinct models of churches and other buildings. This data model also provides a high level three-dimensional model that can be used by other procedural modelling solutions to create detailed monumental buildings. The results also demonstrate that it is possible to use simple text to create a rudimentary 3D model, showing the way to further investigation, allowing non-specialized users to increase effectiveness using a procedural modelling system.
Article
Humankind has given itself the scientific name homo sapiens--man the wise--because our mental capacities are so important to our everyday lives and our sense of self. The field of artificial intelligence, or AI, attempts to understand intelligent entities. Thus, one reason to study it is to learn more about ourselves. But unlike philosophy and psychology, which are also concerned with AI strives to build intelligent entities as well as understand them. Another reason to study AI is that these constructed intelligent entities are interesting and useful in their own right. AI has produced many significant and impressive products even at this early stage in its development. Although no one can predict the future in detail, it is clear that computers with human-level intelligence (or better) would have a huge impact on our everyday lives and on the future course of civilization. AI addresses one of the ultimate puzzles. How is it possible for a slow, tiny brain{brain}, whether biological or electronic, to perceive, understand, predict, and manipulate a world far larger and more complicated than itself? How do we go about making something with those properties? These are hard questions, but unlike the search for faster-than-light travel or an antigravity device, the researcher in AI has solid evidence that the quest is possible. All the researcher has to do is look in the mirror to see an example of an intelligent system. AI is one of the newest disciplines. It was formally initiated in 1956, when the name was coined, although at that point work had been under way for about five years. Along with modern genetics, it is regularly cited as the ``field I would most like to be in'' by scientists in other disciplines. A student in physics might reasonably feel that all the good ideas have already been taken by Galileo, Newton, Einstein, and the rest, and that it takes many years of study before one can contribute new ideas. AI, on the other hand, still has openings for a full-time Einstein. The study of intelligence is also one of the oldest disciplines. For over 2000 years, philosophers have tried to understand how seeing, learning, remembering, and reasoning could, or should, be done. The advent of usable computers in the early 1950s turned the learned but armchair speculation concerning these mental faculties into a real experimental and theoretical discipline. Many felt that the new ``Electronic Super-Brains'' had unlimited potential for intelligence. ``Faster Than Einstein'' was a typical headline. But as well as providing a vehicle for creating artificially intelligent entities, the computer provides a tool for testing theories of intelligence, and many theories failed to withstand the test--a case of ``out of the armchair, into the fire.'' AI has turned out to be more difficult than many at first imagined, and modern ideas are much richer, more subtle, and more interesting as a result. AI currently encompasses a huge variety of subfields, from general-purpose areas such as perception and logical reasoning, to specific tasks such as playing chess, proving mathematical theorems, writing poetry{poetry}, and diagnosing diseases. Often, scientists in other fields move gradually into artificial intelligence, where they find the tools and vocabulary to systematize and automate the intellectual tasks on which they have been working all their lives. Similarly, workers in AI can choose to apply their methods to any area of human intellectual endeavor. In this sense, it is truly a universal field.