Development of a Co-Simulation Framework for Systematic Generation of Scenarios for Testing and Validation of Automated Driving Systems*

Abstract

Due to the sheer infinite number of test scenarios
for test and validation of automated driving, stand-alone onroad testing of these systems is not reasonable, calling for the
development of X-in-the-loop test methods. Recent advances in
simulation methods are often based on simulation techniques
where test scenarios are built considering stochastic traffic
or deterministic predefined manoeuvres. To ensure realism,
numerical robustness and usability of the test scenarios for both
approaches, increasing effort must be invested in modelling the
driving environment as well as vehicle and traffic dynamics.
Especially traffic models are rarely realistically modelled in
most current scenario generation and testing techniques. The
goal of the present paper is to introduce a co-simulation
framework for automated scenario generation with calibrated
traffic flow models using measured data from an official test
road in Austria and modelled in PTV Vissim. Combined
with the Multi-Body-System vehicle development software IPG
CarMaker, the presented co-simulation framework provides an
approach for generation of realistic scenarios. This approach
is demonstrated for a Highway Chauffeur function and allows
future systematic testing.

Full text

Development of a Co-Simulation Framework for Systematic Generation
of Scenarios for Testing and Validation of Automated Driving Systems*
Demin Nalic1, Arno Eichberger2,Georg Hanzl3, Martin Fellendorf4, Branko Rogic5
Abstract— Due to the sheer infinite number of test scenarios
for test and validation of automated driving, stand-alone on-
road testing of these systems is not reasonable, calling for the
development of X-in-the-loop test methods. Recent advances in
simulation methods are often based on simulation techniques
where test scenarios are built considering stochastic traffic
or deterministic predefined manoeuvres. To ensure realism,
numerical robustness and usability of the test scenarios for both
approaches, increasing effort must be invested in modelling the
driving environment as well as vehicle and traffic dynamics.
Especially traffic models are rarely realistically modelled in
most current scenario generation and testing techniques. The
goal of the present paper is to introduce a co-simulation
framework for automated scenario generation with calibrated
traffic flow models using measured data from an official test
road in Austria and modelled in PTV Vissim. Combined
with the Multi-Body-System vehicle development software IPG
CarMaker, the presented co-simulation framework provides an
approach for generation of realistic scenarios. This approach
is demonstrated for a Highway Chauffeur function and allows
future systematic testing.
I. INTRODUCTION
Automated Driving systems (AD) are playing a very
important role in making driving more safe, efficient and
comfortable. A significant part of road accidents are caused
by human misconduct, main causes are listed in [1]. The
reduction of human error by using different ADAS/AD in
vehicles such as the Anti-Lock Braking System (ABS), Elec-
trical Stability Program ESP, Adaptive Cruise Control (ACC)
and others are expected to yield to a continuous decreasing
number of accidents. In [2] the potential of these different
systems to avoid accidents or at least decrease injury risk and
severity was investigated, showing the increasing potential of
ADAS/AD with increasing degree of automation. Together
with the increasing complexity of these technologies, in
terms of the used hardware and software, the test and vali-
dation process becomes time and resource consuming. In [3]
the importance and complexity of testing automated driving
systems is motivated and presented. The not yet answered
*This work is founded by the Austrian Federal Ministry of Transport,
Innovation and Technology as part of the FFG Program ”EFREtop”
1D.Nalic,2A.Eichberger are with the Automo-
tive Engineering Department, Technical Univer-
sity Graz, Austria. demin.nalic@tugraz.at,
arno.eichberger@tugraz.at
4M. Fellendorf and 3G. Hanzl are with the Highway
Engineering and Transport Planning Department, Technical
University Graz, Austria. martin.fellendorf@tugraz.at,
georg.hanzl@tugraz.at
5B. Rogic is Development Manager Electric/Electronics
and Driver Assistance at MAGNA Steyr Fahrzeugtechnik.
branko.rogic@magna.com
question which comes with respect to test and validation
is which test scenarios and how many kilometres should
be tested to ensure the safety and reliability of automated
driving systems. Especially for vehicles with high automation
levels defined in the SAE Standard [4], the test effort grows
exponentially. A statistical approach for this answer was
shown in the work [5] where one of the key findings was
that automated driving vehicles should be driven hundreds
of millions and in some cases billions of kilometres to prove
reliability in terms of fatality and injuries. Several countries
are concerned with this issue and are already developing
rules and laws for test and validation of automated driving
systems which should ensure the reliability and safety of
AD systems. In [6] one can find a short summary of the
standards and requirements for testing of automated driving
vehicles on real roads in countries which are intensively
developing AD systems. For test and validation methods
of AD systems there are find different approaches and
testing platforms available which are developed and used
by variety of AD systems developers [7]. Hardware-in-the-
Loop techniques are presented in [8-10], Vehicle-in-the-Loop
in [11-12] and frameworks or testing platforms are presented
in [13-17]. In all these techniques the traffic model is either
not considered or not modelled well enough to ensure a
realistic traffic environment, that limits the needed effort
for testing. In order to perform numerous virtual tests in
realistic traffic conditions a co-simulation which includes the
pairing between vehicle dynamics (IPG CarMaker) and the
microscopic traffic flow simulation software PTV VISSIM
was used [18]. This two software tools allow the testing
and generating of scenarios with consideration of detailed
single vehicle dynamics for the ego vehicle and realistic
traffic representations. Using the co-simulation framework,
a huge number of data can be generated where ADAS/AD
algorithms can be systematically investigated.
The first part of the work describes the concept and structure
of the framework. After this part a short description of the
tested Highway Chauffeur (HWC) is described. This HWC
serves as an example of an automated driving system which
is implemented and tested with the developed framework.
The section traffic describes a detailed traffic model built
and configured with measured data from a public motorway.
The last part shows a simulation result for one chosen test
run.
II. CO-SIMULAT ION F RAMEWORK
The co-simulation framework consists of three software
tools which are used for different tasks during and after the
2019 IEEE Intelligent Transportation Systems Conference (ITSC)
Auckland, NZ, October 27-30, 2019
978-1-5386-7024-8/19/$31.00 ©2019 IEEE 1895

1. Vehicle dynamic
2. Enviroment
3. Network
4. Controllers
1. Input volumes on road entrances
2. Speed distributions
3. Vehicle compositions
4. Driver models
Matlab Apllication
1. CarMaker parameter
configuration
2. Traffic parameter
configuration
1. Signal Processing
2. Scenario selection
3. Report Generation
Corner cases
Reports
Evaluation
Postprocessing
Simulation Controller
Co-simulation CarMaker - Vissim
Multi Body Simualtion
Traffic Simulation
Fig. 1. The main concept and process flow of the co-simulation framework.
simulation. The main concept of the framework is shown in
figure 1. As in figure 1 the presented framework consists of
following three parts:
•Co-Simulation Controller
•Co-Simulation between CarMaker and VISSIM and
•Post-processing
These three parts build a closed cycle which is essential
for the scenario generation. After every simulation cycles
the data is processed and forwarded to the main simulation
controller for new scenario configurations. All parts are
described in following subsections separately.
A. Simulation Controller
The simulation controller from figure 1 is the main con-
trolling part of the framework. This part is implemented as an
MATLAB application with a graphical user interface (GUI).
For using this GUI, no knowledge in CarMaker and VISSIM
is required. Via the GUI, CarMaker and VISSIM parameters
are editable. To adjust these parameters, two interface com-
munication protocols are used. The communication between
the GUI and CarMaker is established via the Dynamic Data
Exchange (DDE) interface which is available in IPG Car-
Maker. The DDE is used in several other testing procedures
presented in [19] and [20]. The developed framework uses
the DDE interface to edit and configure desired vehicle and
test run parameters/settings in CarMaker. Figure 2 shows the
CarMaker settings which can be edited in Matlab GUI using
DDE the interface.
Via the COM-application programming interface (API) in
VISSIM [21], traffic configuration, driving behaviour and
driver models can be edited using the MATLAB GUI. A
description of how this can be programmed, is presented
in [22]. Figure 3 shows adjustable data for the traffic flow
simulation in VISSIM.
In order to calibrate the traffic flow simulation, extensive
measurements of the Austrian motorway with instrumenta-
tion for AD testing were performed [23]. The road geometry
was surveyed by laser scanners and imported into CarMaker
1. Set a desired test run
2. Set a desired test road
3. Set initial conditions of the vehicle under test
4. Define simulation stop conditions for example:
Co-simualtion Control
Dynamic Data Exchange
IPG CarMaker
Interface
Matlab Application
Parameter Settings
a. Stop simulation after a collision appears
b. Stop simulation after a desired time
c. Stop simulation after a desired distance
5. Start simulation
6. Stop simulation
7. Configure simulation data which should be saved
Fig. 2. Process flow between the Matlab Application and IPG CarMaker.
Co-simualtion Control
Vissim COM
PTV Vissim
Interface
Matlab Application
Parameter Settings
1. Chouse the desired traffic scenario
2. Configure input volumes for all road entrances
3. Configure vehicle composition for all
road entrances
4. Configure the speed distribution of traffic
vehicles
5. Configure driver modells
Fig. 3. Process flow between the Matlab Application and PTV Vissim.
1896

Road Measurement
OpenDRIVE Format
Road width and length
Road marking position
Road bank
Road elevation profile
Surface profile and condition
Road File .rd5
conv.
conv.
OpenDRIVE Format
IPG CarMaker
Network File .anm
PTV Vissim
Co-Simulation
Fig. 4. Network data formats and conversions of the co-simulation
framework.
.rd5 and into PTV VISSIM road network format. Manual
adaption within the .rd5 were required to eliminate laser
scanner errors. The adapted network was transferred to
VISSIM native .anm file format via a converter provided
by IPG. While the lane width varies over distance in reality
and in CarMaker, VISSIM requires fixed lane width for each
link. Lane width may change at each link, which required
simplifications including the elimination of cross fall. This
converter is part of the IPG CarMaker co-simulation soft-
ware, the conversion process which is required for the co-
simulation framework is shown in figure 4. Further details
of the network development are presented in section IV.
B. Co-Simulation between CarMaker and VISSIM
The co-simlation part of the framework cycle is the part
where CarMaker and VISSIM are running synchronously
and generating the data. The visualisation of this process is
shown in figure 5. In this figure, VISSIM traffic visualisation
(top) and a high quality 3D CarMaker Video (down) are
demonstrated.
C. Data processing
The data which is processed during and after the co-
simulation, consists of VISSIM traffic and CarMaker vehicle
data. The data which is recorded during the simulation and
which is relevant for the scenario generation is listed and
described in tables I and II. Based on this online data,
different critical conditions are defined to stop the simulation
and prepare the data for the post processing phase (see figure
1). The current stop conditions, which are defined in the
presented co-simulation framework are listed in table III.
After the simulation, CarMaker Vehicle data is saved as .erg
output file and immediately converted to Matlab native .mat
files for further analysis and processing. The traffic data
which is generated by VISSIM is saved as .txt files which
contains traffic data from the table II. With a C++ data
converter the data is converted to .mat files for the post
processing step of the framework. If during a test run a
critical condition appears, the data from this test run will be
considered as potential data with relevant testing scenarios.
Nevertheless, all test runs are saved and will be used for any
developed scenario generation methods. The generated data
Fig. 5. The main concept and process flow of the co-simulation framework.
TABLE I
REC ORDE D VEHI CLE DATA VIA CA RMAKE R
Parameter Name Description
Distance Vehicle travelling distance
Position Translation of the vehicle at mounted posi-
tion in x,y,z direction
Vehicle Velocity Velocity vector of the vehicle in x,y,z direc-
tions
Vehicle Acceleration Acceleration vector of the vehicle in x,y,z
directions
Control Signals All signals which are related to the devel-
oped HWC
can be used for the scenario generation methods presented
in [24-27] and works similar.
III. HIGHWAY CHAUFFEUR
For the testing and validation process, an SAE level 3
Higway Chauffeur function, which is defined in [28], is
developed applying an IPG-Simulink interface. Requirements
and functionality of the HWC are summarized bellow:
•Filed of application
The HWC is only available on highways under good
weather and visibility conditions. Automatic entering
and exiting the highway as well as driving in construc-
tion sites on the highway can not be handled by this
function
•Functionality
The HWC is intended to take control of the lateral an
longitudinal driving task. Information from monitoring
the environment provided by virtual sensor models
to make decisions when a lane change manoeuvre is
possible or which target is relevant for the ACC.
1897

TABLE II
REC ORDE D TRAFFI C DATA VIA VISSIM
Parameter Name Description
Vehicle ID Number of each traffic participant
Position Rear Position of the vehicle rear frame in x,y,z
directions in a defined coordination system
Position Front Position of the vehicle front frame in x,y,z
directions in a defined coordination system
Velocity Vehicle velocity in x,y,z directions in a
defined coordination system
Acceleration Vehicle acceleration in x,y,z directions in a
defined coordination system
TABLE III
STOP C ONDI TIO NS FOR T HE CO-S IMUL ATI ON PROC ESS
Condition Description
Vehicle Collision Detection Collision detection of the EGO vehicle
with other traffic participants
Traffic Collision Traffic Collision detection of the traffic vehicle
among each other in the vicinity of the
EGO vehicle
Minimum distance A defined minimum distance of the
ACC controller is exceeded for 20 %
of the desired minimum distance
High Vehicle Deceleration The EGO vehicle exceeded the decel-
eration value of -1.5 m/s2
High Traffic Deceleration Minimum two traffic vehicles deceler-
ated with more then -1.5 m/s2in the
vicinity of the EGO vehicle
The HWC consists of an Adaptive Cruise Controller ACC,
a Lane Keeping Assist (LKA), and a Lane Change Assist
(LCA) based on approaches presented in [28-30]. A short
description of the control structure in IPG CarMaker is
shown in figure 6. To meet the requirements of the SAE
level 3 HWC [28], an additional Automatic Emergency Brake
(AEB) system should be considered which not implemented
in the presented study.
The general structure of IPG Simulink interface is shown in
figure 6. The vehicle control port in Simulink is adjusted with
the HWC. Via Simulink CarMaker interface where CarMaker
input signals are taken as inputs for the developed ACC,
LKA and LCA algorithms. The calculated outputs using the
HWC implementation are forwarded as new inputs for the
vehicle in CarMaker, see figure 7. Other signal which are
Driver Vehicle
Control IPG Vehicle Output data
Input Data
Virtual Enviroment
Vehicle
Driver
Road
IPG CarMaker
Matlab/Simulink
Fig. 6. General structure of CarMaker Simulink.
Vehicle
Control
LKA
LCA
ACC
Highway Chauffeur
... ...
...
Gas
Steering Angle
Steering Angle Vel.
Steering Angle Acc.
Steering Torque
...
Gas
Steering Angle
Steering Angle Vel.
Steering Angle Acc.
Steering Torque
IPG CarMaker I/O data which is not
relevant for the HWC
Driver Data Vehicle Data
Driver Data
Simulink Enviroment
Vehicle Data
Fig. 7. HWC Implementation in CarMaker Simulink.
not relevant for the HWC functionalities are not manipulated.
This implementation part from figure 7 can be exchanged by
any other ADAS/AD algorithms and tested in the presented
co-simulation.
TABLE IV
REC ORDE D DATA FOR E VERY CRO SS SE CT IO N OF T HE T ES T ROA D
Parameter Description Metric
T imeStamp Recorded time stamp YYYY:MM:DD
hh:mm:ss
NodRef Cross section ID none
QKF Z Traffic Volume - cars (Veh) Veh/min
QLKW Traffic Volume - trucks (Trk) Trk/min
VP LW Average speed of all cars km/h
VLKW Average speed of all trucks km/h
T headway s
B Occupancy of the presence detector %
SKF Z Standard deviation of average speed km/h
VKF Z Average speed of all vehicles km/h
TABLE V
CLU ST ER D EFI NI TI ON O F DA ILY TR AFFI C CO ND IT IO N S
Cluster 1 Working days between 24.12.2017 - 06.01.2018
Cluster 2 Monday to Thursday between 01.10.2017 - 24.10.2017
Cluster 3 Public holidays 01.10.2017-31.03.2018
Cluster 4 Saturdays between 01.10.2017-31.03.2018
Cluster 5 Sundays and holidays between 01.10.2017-31.03.2018
Cluster 6 Monday to Thursday between 25.10.2017-23.12.2017
Cluster 7 Monday to Thursday between 07.01.2018-31.03.2018
IV. TR AFF IC MO DEL
The modelling and calibration of the traffic was carried
out with the traffic simulation software PTV VISUM and
VISSIM [32]. For calibration, the cross sectional measure-
ment data of the ALP.lab motorway section was taken from
points shown in figure 8.
1898

Intersections
Legend
Cross sections
Highway
0 500 1000 1500 2000 2500
MP. at CS 1
MP. at CS 2
MP. at CS 3
MP. at CS 12
MP. at CS 11
MP. at CS 4
MP. at CS 10
MP. at CS 9
MP. at CS 5
MP. at CS 6
MP. at CS 7
MP. at CS 8
MP. at CS 15
MP. at CS 14
MP. at CS 13
MP. at CS 16
MP. at CS 17
MP. at CS 19
MP. at CS 18
Fig. 8. Measurement Points (MP) at the cross sections (CS) of the
considered ALP.lab test road.
0
1,000
2,000
3,000
4,000
Number of vehicles
Cluster 1 Cluster 2
0
1,000
2,000
3,000
4,000
Number of vehicles
Cluster 3 Cluster 4
0
1,000
2,000
3,000
4,000
Number of vehicles
Cluster 5
0 5 10 15 20 25
time in hours
Cluster 6
0 5 10 15 20 25
0
1,000
2,000
3,000
4,000
time in hours
Number of vehicles
Cluster 7
MP at CS 1 MP at CS 2
MP at CS 3 MP at CS 4
MP at CS 5 MP at CS 6
MP at CS 7 MP at CS 8
MP at CS 9 MP at CS 10
MP at CS 11 MP at CS 12
MP at CS 13 MP at CS 14
MP at CS 15 MP at CS 16
MP at CS 17 MP at CS 18
MP at CS 19 MP at CS 20
MP at CS 21
Fig. 9. Vehicle volumes of the defined classes for all measurements at the
cross sections.
The measured data per cross section is show in table IV.
With the measured data a traffic scenario catalogue based on
various traffic situations (traffic composition, traffic volume)
and environmental conditions (Time of day) was built.
70 90 110 130 150 180 200
0
50
100
150
Frequency distribution of all vehicles
70 90 110 130 150 180 200
0
50
100
150
Frequency distribution of cars
70 90 110 130 150 180 200
0
50
100
Distrubution of all vehicles in %
70 90 110 130 150 180 200
0
50
100
Distrubution of cars in %
70 90 110 130 150 180 200
0
50
100
speeds in km/h
Distrubution of hgv in %
70 90 110 130 150 180 200
0
50
100
150
speeds in km/h
Frequency distribution of hgv
Fig. 10. Speed and frequency distribution for cross section 3 for all
vehicles, cars and hgv (heavy goods vehicles) for the time between 9:00-
10:00 AM
The calibrated network and the generated scenarios are
then used for co-simulation process. The modelled scenarios
include morning and evening peak at average working days
and leisure traffic at weekends. Average traffic conditions for
each scenario were generated by determining seven clusters
(table V) of daily traffic conditions into typical volume
profiles by the k-means clustering method, see figure 9.
0 5 10 15 20 25 30 35 40 45
0
100
200
300
400
500
600
500
700
Time in s
Distribution of cars
0 5 10 15 20 25 30 35 40 45
0
100
Frequency distribution of cars %
Fig. 11. Headway distribution for cross section 3 for the time period
between 9:00 - 10:00
With the vehicle data from table VI the speed distributions
were calculated. The calculation of the speed distributions
was done for every coss-section from figure 8 and an average
speed distribution was implement to VISSIM for the co-
simulation. The result of the speed distribution for cross
section 3 is presented in figure 10. The headway is the
time interval between two consecutive moving vehicles. It
1899

TABLE VI
SIN GL E VE HI CL E DATA MEA SU RE D FO R EV E RY TR AFFI C PARTI CI PAN T
Parameter Metric
Time s
Lane 1,2,3
Vehicle Type Car or Truck
Velocity km/h
Headway s
Fig. 12. Critical cut in manoeuvre of the ego vehicle causing a near-
collision .
is measured from the moment when the leading vehicle
leaves the measure point until the following vehicle reaches
this point. If this time span is smaller than 5 seconds, the
following vehicle is influenced by the vehicle ahead. In
figure 11 the result of the headway calculation for a specific
scenario is shown. The results of the calibrated data were
used for different scenarios in the co-simulation.
V. S IMU LATION RESULTS
For demonstration of the approach, the described HWC
AD function was tested with the co-simulation framework.
The ego vehicle was sent repeatedly through the virtual
motorway section resulting in a total number of 45425
kilometres. Within this driven distance, the evaluation met-
rics found a total number of 26 collisions and 378 near-
collisions. Figure [12] shows a scenario where the HWC
function showed a malfunction by using the LCA algorithm.
The decision making part of the LCA algorithm initiated the
cut-in manoeuvre. In this situation the cut-in was very risky
because of the close the distance among the vehicles, see
figure 12. After a close analysis of the same situation from
figure 12 the rear traffic vehicle almost caused a collision
after the lane change of the ego vehicle. In figure 13 its
shown that the rear traffic vehicle succeed to break and
overtake the ego vehicle. In this situation the ego vehicle
caused a critical scenario for the surrounding traffic. Since
not only critical objective measurement such as collisions,
Fig. 13. Critical cut in manoeuvre of the ego vehicle influencing the
surrounding traffic vehicles.
small headway but other metrics such as driver comfort,
influence on the surrounding traffic, energy efficiency and
vehicle throughput are essential, additional metrics will be
implemented in future.
VI. CO NCL USI ON
The presented work offers an approach and implemen-
tation of a virtual framework for testing and validation
of AD systems. The developed framework consists of a
comprehensive traffic flow model which is calibrated with
real traffic measurements. This ensures a realistic traffic
environment where any AD system can be tested with a high
number of considerable traffic scenarios using a Matlab GUI
in combination with the co-simulation between CarMaker
and VISSIM. Based on these real traffic scenarios a sheer
infinite amount of realistic traffic scenarios can be generated
and used for testing AD vehicles and scenario generation
methods within the presented environment. For the validation
of this results other techniques and real car test should
be considered and realised. To increase and improve the
presented co-simulation more complex sensor models, traffic
objects and environmental conditions will be implemented
additionally for future work. Also focus will be given to the
extension of evaluation metrics that can be implemented to
automatically detect malfunctions for AD developers.
REF ER E NC E S
[1] Bundesministerium fur Inneres. Straßenverkehrstote in
¨
Osterreich. Available at https://www.bmi.gv.at/202/
Verkehrsangelegenheiten/files/Jahresvergleich_
Verkehrstote_2017.pdf. Accessed on 3 April 2019.
[2] Eichberger, Arno. Contributions to primary, secondary and integrated
traffic safety. Verlag Holzhausen GmbH, 2011.
[3] A. Broggi, M. Buzzoni, S. Debattisti, P. Grisleri, M. C. Laghi, P.
Medici and P. Versari, ”Extensive Tests of Autonomous Driving Tech-
nologies”, IEEE Transactions on Intelligent Transportation Systems,
vol. 14, pp. 1403-1415, 2013.
[4] Taxonomy and Definitions for Terms Related to Driving Automa-
tion Systems for On-Road Motor Vehicles, SAE International Std.
J3016, 06 2018. [Online]. Available: https://www.sae.org/
standards
[5] Kalra, Nidhi and Susan M. Paddock, Driving to Safety: How
Many Miles of Driving Would It Take to Demonstrate Au-
tonomous Vehicle Reliability?. Santa Monica, CA: RAND Cor-
poration, 2016. https://www.rand.org/pubs/research_
reports/RR1478.html. Accessed 4 April 2019.
[6] Bundesministerium Verhehr, Inovation und Technologie
”Automatisiertes Fahren auf Straßen mit ¨
offentlichen Verkehr -
Rechntliche Rahmenbedinugnen im Verlgeich”, Available https://
www.bmvit.gv.at/verkehr/automatisiertesFahren/
downloads/laendervergleich_ua.pdf, Accessed 11 April
2019.
1900

[7] J. E. Stellet, M. R. Zofka, J. Schumacher, T. Schamm, F. Niewels
and J. M. Z¨
ollner, ”Testing of Advanced Driver Assistance Towards
Automated Driving: A Survey and Taxonomy on Existing Approaches
and Open Questions”, in 2015 IEEE 18th International Conference on
Intelligent Transportation Systems, 2015, pp. 1455-1462.
[8] Y. Chen, S. Chen, T. Zhang, S. Zhang and N. Zheng, ”Autonomous
Vehicle Testing and Validation Platform: Integrated Simulation System
with Hardware in the Loop*”, in 2018 IEEE Intelligent Vehicles
Symposium (IV), 2018, pp. 949-956.
[9] A. Soltani and F. Assadian. (2016, 2016). A Hardware-in-the-Loop
Facility for Integrated Vehicle Dynamics Control System Design and
Validation. Ifac-Papersonline 49(21), pp. 32-38.
[10] O. Gietelink, J. Ploeg, B. De Schutter and M. Verhaegen, ”Develop-
ment of advanced driver assistance systems with vehicle hardware-in-
the-loop simulations”, Veh. Syst. Dyn., vol. 44, pp. 569-590, 2006.
[11] T. Bokc, M. Maurer and G. Farber, ”Validation of the Vehicle in
the Loop (VIL); A milestone for the simulation of driver assistance
systems”, in 2007 IEEE Intelligent Vehicles Symposium, 2007, pp.
612-617.
[12] T. Tettamanti, M. Szalai, S. Vass and V. Tihanyi, ”Vehicle-In-the-Loop
Test Environment for Autonomous Driving with Microscopic Traffic
Simulation”, in 2018 IEEE International Conference on Vehicular
Electronics and Safety (ICVES), 2018, pp. 1-6.
[13] B. Kim, Y. Kashiba, S. Dai and S. Shiraishi, ”Testing Autonomous
Vehicle Software in the Virtual Prototyping Environment”, IEEE
Embedded Systems Letters, vol. 9, pp. 5-8, 2017.
[14] M. R. Zofka, S. Klemm, F. Kuhnt, T. Schamm and J. M. Z¨
ollner,
”Testing and validating high level components for automated driving:
simulation framework for traffic scenarios”, in 2016 IEEE Intelligent
Vehicles Symposium (IV), 2016, pp. 144-150.
[15] Wang, J., Li, S., Hang, X. and Li, K. Driving simulation platform ap-
plied to develop driving assistance systems. IET Intelligent Transport
Systems. 2009, 4(2), 121-127.
[16] M. Rodrigues, A. McGordon, G. Gest and J. Marco, ”Developing
and testing of control software framework for autonomous ground
vehicle”, in 2017 IEEE International Conference on Autonomous
Robot Systems and Competitions (ICARSC), 2017, pp. 4-10.
[17] Y. Chen, S. Chen, T. Zhang, S. Zhang and N. Zheng, ”Autonomous
Vehicle Testing and Validation Platform: Integrated Simulation System
with Hardware in the Loop*”, in 2018 IEEE Intelligent Vehicles
Symposium (IV), 2018, pp. 949-956.
[18] PTV Group and IPG Automotive, ’PTV Group and IPG Automotive
Offer New Interface for Virtual Test Driving’, 2017 [online].
Available: http://newsroom.ptvgroup.com/uploads/tx_
ptvpress/PR_EN_2017-03- 21_PTV_IPG_Partnership.
pdf. Accessed:03.04.2019
[19] T. Wu, Y. Wan, Y. Xi and C. Chen, ”Study on the Automatic
Test Framework Based on Three-tier Data Driven Mechanism”, in
2009 Eighth IEEE/ACIS International Conference on Computer and
Information Science, 2009, pp. 996-1001.
[20] Zhu Jian-Jun and Sha Hong, ”Simulation and researching of control
system based on Matlab and WinCC”, in 2010 International Confer-
ence on Computer and Communication Technologies in Agriculture
Engineering, 2010, pp. 138-140.
[21] PTV, ”PTV Vissim 10 User Manual” PTV Planung Transport Verkehr
AG, Karlsruhe: PTV Planung Transport Verkehr AG., 2019.
[22] Tettamanti, T. and Horv ´
ath, M.T., “A practical manual for Vissim
COM programming in Matlab 2nd edition for Vissim versions 6, 7,
and 8,” Budapest University of Technology and Economics, Dept. for
Control of Transportation and Vehicle Systems, 1-5, 2015.
[23] Payerl, C., Schmidt, D., Hrassnig, P., Bernsteiner, S.. Absicherung
automatisierter Fahrfunktionen mithilfe von Fahrzeug-und Infrastruk-
turdaten. ATZ-Automobiltechnische Zeitschrift, 121(4), 16-21, 2019
[24] Beglerovic, H., Schloemicher, T., Metzner, S., Horn, ”M. Deep Learn-
ing Applied to Scenario Classification for Lane-Keep-Assist Systems”.
Appl. Sci. 2018, 8, 2590.
[25] C. E. Tuncali, G. Fainekos, H. Ito and J. Kapinski, ”Simulation-based
Adversarial Test Generation for Autonomous Vehicles with Machine
Learning Components”, in 2018 IEEE Intelligent Vehicles Symposium
(IV), 2018, pp. 1555-1562.
[26] M. Althoff and S. Lutz, ”Automatic Generation of Safety-Critical Test
Scenarios for Collision Avoidance of Road Vehicles”, in 2018 IEEE
Intelligent Vehicles Symposium (IV), 2018, pp. 1326-1333.
[27] G. E. Mullins, P. G. Stankiewicz and S. K. Gupta, ”Automated
generation of diverse and challenging scenarios for test and evaluation
of autonomous vehicles”, in 2017 IEEE International Conference on
Robotics and Automation (ICRA), 2017, pp. 1443-1450.
[28] PEGASUS. The Highway-Chauffeur. Available at https://www.
pegasusprojekt.de/files/tmpl/PDF-Symposium/03_
The-Highway- Chauffeur.pdf. Accessed on 7 April 2019
[29] S. Samiee, S. Azadi, R. Kazemi, A. Eichberger, Towards a decision-
making algorithm for automatic lane change manoeuvre considering
traffic dynamics. Promet - Traffic - Traffico. 28, 91–103 (2016).
[30] S. Bernsteiner. ”Integration of Advanced Driver Assistance Systems
on Full-Vehicle Level - Parametrization of an Adaptive Cruise Control
System Based on Test Drives” . Technical University Graz, 2016
[31] A. Benine-Neto u. a. ”Dynamic controller for lane keeping and
obstacle avoidance assistance system”. In: 2010 13th International
IEEE Conference on Intelligent Transportation Systems (ITSC). Sep.
2010, S. 1363–1368. doi: 10.1109/ITSC.2010.5625021.
[32] Fellendorf, M & Vortisch, P 2010, Microscopic Traffc Flow Simulator
VISSIM. in Fundamentals of Traffic Simulation. 1 edn, vol. 145,
International Series in Operations Research & Management Science,
Springer Science+Business Media, New York, pp. 63-94.
1901