Content uploaded by Gunter Nitzsche
Author content
All content in this area was uploaded by Gunter Nitzsche on Mar 27, 2020
Content may be subject to copyright.
Proceedings of 8th Transport Research Arena TRA 2020, April 27-30, 2020, Helsinki, Finland
AutoTruck and helyOS: Enabling highly efficient yard operation
by automation
Gunter Nitzschea
*
, Sebastian Wagnera, Nikolay Belova, Carlos Eduardo Viol Barbosaa
aFraunhofer-Institute for Transportation and Infrastructure Systems IVI, Dresden, Germany
Abstract
The paper shows a new approach for highly efficient yard operation by automation with a new control centre
software approach for gated areas, called helyOS. This software is currently developed in different research
projects. This paper describes the concept of helyOS along with its capabilities, features and possible future
extensions. Its first-time application is the fully automated, electrically driven AutoTruck – a distribution truck,
which can drive autonomously within a distribution centre. By this example, the authors show how helyOS can
aid an efficient handling of automated vehicles.
Keywords: Autonomous Driving, Yard Automation, Web Service, Path Planning, Control Centre
* Corresponding author. Tel.: +49-351-4640-621;
E-mail address: gunter.nitzsche@ivi.fraunhofer.de
Nitzsche et. al. / TRA2020, Helsinki, Finland, April 27-30, 2020
1
Nomenclature
DGPS Differential Global Positioning System
ECU Electronic Control Unit
helyOS Highly Efficient Online Yard Operating System
ROS Robotic Operating System
1. Introduction
The Highly Efficient Online Yard Operating System (helyOS) is developed by the Fraunhofer Institute for
Transportation and Infrastructure Systems IVI in the project “AutoTruck”, which is funded by the German Federal
Ministry for Economic Affairs and Energy (BMWi). The AutoTruck-project consortium, which consists of
industrial and research partners and an end user, develops a delivery truck for automated driving in distribution
centres. However, the vehicle shall also be homologated for driving with a driver on public roads. Fig. 1 shows
the targeted application scenario of such a vehicle. The driver parks the truck at a designated point or the parking
area, when entering the distribution centre. After the driver has left the vehicle, the truck will drive autonomously
on the distribution centre area e.g. to loading docks or charging stations. When the vehicle has finished its tasks,
it drives back to the parking area again, where the driver can enter it and leave the distribution centre. The project
website (Fraunhofer IVI, n.d.) provides a short video of the concept. This concept can have the following
advantages for distribution centre applications or similar scenarios:
The driver can leave the vehicle earlier and either:
o can take his rest period rather than manoeuvring the truck on the distribution centre, or
o can already take care of the paperwork, while his truck drives to the loading dock itself
The number of accidents can be reduced, because manoeuvring in tight spaces is a challenging and error
prone task for human drivers
Fig. 1 Schematic functional overview of the Automation Zone controlled by helyOS
Nitzsche et. al. / TRA2020, Helsinki, Finland, April 27-30, 2020
2
The authors envision automated driving of trucks on restricted areas like distribution centres as a good migration
path towards autonomous driving. Such systems are already known as intelligent transportation systems
(short: ITS). However, these automated vehicles drive only in the restricted areas. The new concept of AutoTruck
is the usage of the vehicle with both conventional driving on public roads and automated driving in restricted areas.
In addition, the autonomous driving mode preferably shall only use these sensors, which are already installed or
are anticipated to be installed in future trucks for advanced driver assistance systems, as shown e.g. in (Banerjee,
Kniess, & Tenbrock, 2019).
Automation in gated and fenced areas also has the advantage of significantly lower challenges regarding safety
and homologation. The main reason is the well-known area, which can be adapted to autonomous driving vehicles
e.g. by marking driving zones. Furthermore, the variety of traffic scenarios is much lower than e.g. in inner-city
traffic, which reduces the overall complexity. This can even be more favourable by instructing the people –
typically personnel – within the area in order to prevent risky behaviour and troublemakers. This all allows an
easier first step into automation before moving towards automated driving on public roads. It also gives the
opportunity to gain experience with the necessary technology in real world scenarios.
This paper describes the two major tasks of the AutoTruck-project. First, the development of the actual testing
vehicle, referred to as AutoTruck. Second, the development of a control centre based on modern web technologies.
These technologies make it available on various platforms and accessible either on premise via a local network or
globally via the internet. This Highly Efficient Online Yard Operating System (short: helyOS) enables the operator
of a yard to assign missions to autonomous vehicles. The project will demonstrate the automated driving to a
loading/unloading dock. In future, helyOS will support more complex tasks like moving swap bodies or semi-
trailers.
2. The Vehicle
Within the AutoTruck project, the partners Orten, Wabco, Götting and Fraunhofer IVI in collaboration with
additional suppliers built up the testing vehicle shown in Fig. 2. The vehicle was fitted with a new electric drivetrain
and sensor, actuator and control technologies for automated driving. Nevertheless, the vehicle shall still be
homologated for normal operation on public roads. The project partners installed technologies, which are expected
to be included in future serial trucks. This shall show that future serial trucks can easily be changed to be used with
the new yard automation concept.
One crucial point during the development of the vehicle was that the vehicle should be able to perform driving
missions autonomously and safely. HelyOS sends the mission to the truck. From this point on, the vehicle itself
shall be responsible for safety. Thus, the safety requirements for the data transmission between vehicle and helyOS
can be lowered, but are increased on vehicle level.
Fig. 3 shows an overview of the functional components in the Autotruck. Orten installed an electric powertrain
into the truck. This powertrain supports drive-by-wire and allows to control the vehicle easily and smoothly in
longitudinal direction. An electric steering system provides steer-by-wire and thus allows for lateral control. Both
systems are controlled by the central vehicle control, which so can manoeuvre the vehicle.
Nitzsche et. al. / TRA2020, Helsinki, Finland, April 27-30, 2020
3
Fig. 2 The AutoTruck testing vehicle
Fig. 3 Structure of the functional components within the AutoTruck
The electric powertrain also enables electric braking. In addition, Wabco provides access to the conventional
braking system. This can be used as an additional safety layer in critical situations. Furthermore, Wabco provides
a communication solution between the vehicle and the stationary helyOS system. The communication solution is
based on IEEE 802.11p.
Götting KG provides a localization solution for the testing vehicle. The system is based on three technologies,
which are combined via sensor fusion. This shall ensure a high accuracy and availability of the measured vehicle
position und varying conditions. Differential GPS (DGPS) provides a precise global reference. Additionally, scan
matching is used, which is based on laser scanner data in combination with a digital map. Six laser scanners can
monitor the surroundings of the vehicle with 360° for this purpose. As third system, the localization includes
odometry data from the powertrain and steering system. All three systems have their advantages and disadvantages
in certain scenarios. The combination can provide a robust and accurate position, which is used for lateral and
longitudinal control of the vehicle.
Nitzsche et. al. / TRA2020, Helsinki, Finland, April 27-30, 2020
4
As described above, the vehicle itself is responsible for safety while performing the driving mission. Therefore,
two additional safety laser sensors monitor the driving path at the front and rear of the vehicle (for forward and
rearward driving) for obstacles like people or other vehicles. The laser sensors provide a safety integrity level,
which suits the requirements for autonomous vehicles in non-public areas.
The Fraunhofer IVI is responsible for transmitting the data between helyOS and the truck via the communication
channel provided by Wabco. The following section will describe this in more detail. In addition, Fraunhofer IVI
was responsible for the integration of the subsystems and the underlying vehicle control. The functional blocks
described above are connected by a variety of interfaces like digital signals, CAN and Ethernet communication.
One important technology is the Robotic Operating System (ROS), which is described e.g. in (O'Kane, 2014).
With ROS, functionalities can be divided in nodes, which enables a flexible and fast development. The nodes are
then interconnected either via standard interfaces defined by ROS or via proprietary interfaces.
For the control of the vehicle, different approaches are considered. The first one, described in (Kolb, Nitzsche, &
Wagner, 2019), is a simple approach with very easy calculations and thus suitable for vehicle control units. It will
not be suitable for high speed driving, but should be sufficient for low speed manoeuvring in yard automation
scenarios. The second approach provides an enhanced, but also quite simple control method, (Kanayama, Kimura,
Miyazaki, & Noguchi, 1990). Also this approach should be implementable on a vehicle control unit. Tests of both
control algorithms in the real vehicle will show how they perform under real world conditions and whether the
assumptions above are correct.
3. helyOS Architecture
HelyOS is the actual control centre software, which allows one operator managing multiple autonomous vehicles,
e.g. in distribution centres. The software is designed as a web application. This allows the software to run on
different devices, like desktop computers or tablets, without any changes. It consists of a backend and a frontend,
which enables many degrees of freedom regarding the setup and usage in various application field of helyOS. The
following paragraphs describe the components shown in Fig. 4.
Fig. 4 Structure of the functional components in the helyOS infrastructure
The frontend is the actual user interface for the operator of the yard. It presents a task-oriented view of the current
situation to the operator, including a digital map with live positions of the automated vehicles. The interactive user
interface allows easy access to real-time vehicle status information and allows controlling the automated vehicles
by scheduling missions. This frontend can easily be adapted to different applications of the helyOS software, e.g.
for yard automation and for agricultural automation. As described above, it is developed with widely adopted web
technologies, which allows it to run on virtually every mobile or desktop device. This gives a great flexibility for
stationary use and mobile operation. The requirements for the frontend hardware are quite small, because the
expensive calculations are handled by the backend, described in the following paragraphs.
The backend provides the necessary computation power, a database as data storage, the business logic for the
different use-cases and a path planning algorithm. The backend can be either on-site or accessed via the internet.
Nitzsche et. al. / TRA2020, Helsinki, Finland, April 27-30, 2020
5
Together with the flexible frontend approach, this allows system adaptations to the needs of the end user. The end
user can decide to have its own dedicated backend, e.g. because of security reasons. However, the end user can
also rent a backend, if he does not want to take care of the backend infrastructure.
The central database saves all information about missions, vehicle statuses and operator interactions. It is used to
exchange data between the system components (frontend, business logic) and the automated vehicles. Furthermore,
it enables a detailed tracking and statistical evaluation of the performed vehicle missions.
The operator can enter complex driving tasks via the frontend. The business logic is responsible to split complex
tasks into elementary actions and to schedule these. For example, the abstract task of unloading and loading of a
truck at a certain gate is broken down into the following steps: 1. Drive to the gate, 2. Dock at the gate, 3. Wait
until the unloading/loading process is finished, 4. Drive to the parking area, 5. Wait at the assigned parking spot.
While the automated vehicle(s) process the elementary actions, the business logic supervises the status and
provides information to the operator on the frontend via the central database. When creating the elementary actions,
the business logic also calls other modules, e.g. the path planning module to get feasible routes to reach a certain
destination. A scheduling algorithm for the driving actions can already avoid collisions with other automated
vehicles already in the planning stage. Since the current positions and driving missions of all automated vehicles
on the yard are known, the scheduler can consider this information during the planning of new driving missions.
It is therefore important, that the business logic ensures consistency and integrity of the data to provide a good
scheduling. Nevertheless, the vehicles will avoid collisions themselves, e.g. with emergency stops. However, the
overall system efficiency will be higher, if such unnecessary stops can be avoided in the first place. Finally, the
business logic combines the scheduled elementary actions into a mission description, and stores it in the central
database, from where it is sent to the vehicle. It needs to be highlighted, that helyOS does not control the vehicles
directly via steering, propulsion or braking commands. This would unnecessarily increase the safety requirements
for the wireless communication and would also be limited by latencies. The vehicle control is part of the vehicle
itself, which follows the driving mission, which it receives from helyOS.
One key task, which the backend performs, is the path planning, if the operator task requires a driving action.
Fraunhofer IVI developed a path planning algorithm called TruckTrix, which considers obstacles and the non-
holonomic vehicle kinematics, see (Beyersdorfer & Wagner, 2013). This path planning algorithm generates
feasible paths for the vehicles from start to destination if they exist. For this purpose TruckTrix uses a digital map,
which includes driving areas and obstacles. Even temporary obstacles like swap bodies and containers that are not
part of the fixed infrastructure can be considered, because of the short calculation times. The path planning
algorithm currently cannot take account of moving obstacles like other automated vehicles. Hence, the scheduling
algorithm is necessary.
The vehicle communication module interacts wireless with the vehicle via a ROS-network. As already described
for the internals of the vehicle, this allows for a flexible development approach. However, during the development
it became evident that current versions of ROS have significant drawbacks regarding data safety, security and the
protection of intellectual property. Thus, for future developments this communication shall be replaced with
different methods, which can overcome these drawbacks. Independent of the actual transmission protocol, the
vehicle communication module selects the next scheduled vehicle action from the central database and sends it to
the vehicle. In the opposite direction, this module receives live data from the vehicle that it stores in the central
database for visualization in the frontend. Such data can include for example the vehicle position, its velocity, the
state of charge of the traction battery and other status information. Another important set of tasks of the vehicle
communication module are the check-in and check-out procedures, when a new vehicle enters or a vehicle leaves
the yard. During check-in the vehicle provides relevant information to the backend, like its own kinematic structure
and measurements as well as capabilities. The path planning and scheduling algorithms require this information
for the calculation of feasible and collision free trajectories. In return the vehicle receives a digital map of the yard
for its own localization methods, e.g. scan-matching. The check-out procedure is as simple as making the vehicle
unavailable for further task assignments and setting the vehicle back in non-autonomous mode for conventional
driving on public roads.
As an optional extension, helyOS also supports an interface to external sensors, which can be mounted in the yard.
These could include e.g. cameras, stationary laser sensors or distance sensors at loading docks. Such sensors can
especially help to observe blind spot areas. This would help especially when operating autonomous vehicles in
mixed traffic with conventional vehicles and pedestrians, by providing such information to the autonomous
Nitzsche et. al. / TRA2020, Helsinki, Finland, April 27-30, 2020
6
vehicles via the vehicle communication module. Furthermore, these external sensors can report new objects and
obstacles, like swap-bodies, to the business logic, which then uses this information for further mission planning in
its digital map.
A first version of helyOS is developed within the AutoTruck project for a distribution centre application. This first
version proofs the concept and shows the capabilities of such an approach. However, the vision for helyOS is to
develop a flexible operating system for yards, see Fig. 5. This operating system shall interact with the hardware
(vehicles and stationary sensors), perform tasks like scheduling, path planning, managing resources and creating
a live map. Use-case specific applications can access these functionalities via an application interface, enabling a
significantly faster development. helyOS encapsulates basic tasks and provides general services like path planning,
scheduling of actions based on available resources and a current status view of the yard.
Every vehicle, which complies with the hardware communication interface, shall be able to connect to helyOS.
Thus, vehicle manufacturers can develop own vehicles, with proprietary localization and control technology,
which does not have to be made available. Only high level information, like current position and status is sent
from the vehicles to helyOS. In return a mission description is received by the vehicles, which they shall perform
autonomously.
This flexible approach will be used in follow-up projects for different application scenarios like agriculture.
Fig. 5 helyOS as operating system for different applications of automated vehicle fleets
4. helyOS user interface
This section will look a little bit more into the frontend – the user interface for the operator. As already described,
this is a web-application, which runs in widely available internet browsers like Firefox, Safari and Google Chrome,
enabling a great flexibility in the underlying hardware. This allows the operator to interact with the autonomous
vehicles, via almost any desktop or laptop computer as well as tablets or even his mobile phone. However, bigger
screens are recommended for a better user experience. Depending on the backend location (either on site or off
site), the frontend hardware requires either an internet connection or a connection to a local network.
Fig. 6 shows a screenshot of the frontend. In the top right, the map of the yard is displayed. This map can provide
information like obstacles or drivable areas. In this particular screenshot, it shows the current path of the selected
vehicle with green dots. The map also shows the vehicles for the operator to have a live-view of the current
situation.
Nitzsche et. al. / TRA2020, Helsinki, Finland, April 27-30, 2020
7
Fig. 6 Screenshot of the helyOS user interface for the AutoTruck project
The map is a digital three-dimensional representation of the yard, which optionally can be combined with Google
Earth images or construction drawings of the yard. The third dimension can be used, if heights of special vehicles
need to be considered during the planning of driving missions. Interactive boxes mark key positions in the yard.
These include loading/unloading docks, parking spots or in the case of electric vehicles also charging stations.
When the operator selects a vehicle on the map, the left pane shows a small mini map, which includes the vehicle,
and status information of the mission and other information provided by the vehicle. Also new missions can be
assigned to this vehicle and already planned missions can be aborted. Assigning a new mission to an autonomous
vehicle just takes three clicks, including confirmation: 1. Select the vehicle, 2. Select the new target position, 3.
Confirm. The backend handles the remaining steps as described in the previous section. Once the mission is
actually scheduled, helyOS sends it to the vehicle.
The left pane also provides a list of the vehicles currently registered at the yard from which the operator can directly
select a vehicle.
At the bottom of the screen, the user interface provides a Gantt-like view, which shows the missions of the selected
vehicle. The operator gets an overview over the performed, the current and the scheduled driving missions for this
vehicle. It also allows to manually schedule tasks, if a driving mission shall not start immediately, but just at a
given point in time. With a click on a task bar, further information for this mission is shown, including starting
times and estimated or recorded end times.
Due to the flexible frontend-backend-architecture, the look and feel of the user interface can be changed easily,
which allows for an easy adaptation for different application scenarios. Additional views can be added, which can
provide specific information for specific applications. The underlying technologies especially in the backend
remain unchanged.
Nitzsche et. al. / TRA2020, Helsinki, Finland, April 27-30, 2020
8
5. Summary
The helyOS control centre software utilizes the capabilities of automated vehicles to create a benefit for customers.
A single operator is able to control and monitor many automated vehicles at once. Due to the used web
technologies, this can even be done remotely via the internet.
The application fields have a wide range from distribution centres, yards of production sites, airfields and harbours
to agriculture. The basic idea is to generalize common tasks like communication, mission management, etc. in the
context of automation of mobile machines. An application interface can be used to develop apps tailored to
different application fields. In order to support the application development, helyOS provides several services like
path planning (also for complex vehicles) and a scheduler, which coordinates the movements of many automated
vehicles.
Currently the system is still in development. A first demonstration of the AutoTruck project is planned at a
distribution centre of the end user and project partner Emons. These tests will be used to evaluate the practicality
of the proposed control centre approach. Vehicle measurements will be used to evaluate the localization and control
algorithms under real world conditions.
In future, helyOS will be further developed step by step with respect to its architecture, functionality, availability
and safety within research projects and/or in cooperation with industry partners.
Acknowledgement
The project AutoTruck was funded by the German Federal Ministry for Economic Affairs and Energy (BMWi).
References
Banerjee, A., Kniess, P., & Tenbrock, F. (2019). ZF swap body assist - autonomous loading and unloading of swap
bodies in depots. VDI Commercial Vehicles 2019. Baden Baden: VDI-Verlag.
Beyersdorfer, S., & Wagner, S. (2013). Novel model based path planning for multi-steered heavy load vehicles.
16th International IEEE Conference on Intelligent Transportation Systems (ITSC). Den Haag: IEEE.
Fraunhofer IVI. (n.d.). Project Website. Retrieved from https://aeroflex-project.eu/
Kanayama, Y., Kimura, Y., Miyazaki, F., & Noguchi, T. (1990). A stable tracking control method for an
autonomous mobile robot. Proceedings of IEEE International Conference on Robotics and Automation
(pp. 384 - 389). Cincinnati, OH, USA: IEEE.
Kolb, J. K., Nitzsche, G., & Wagner, S. (2019). A simple yet efficient Path Tracking Controller for Autonomous
Trucks. Proceedings of the 10th IFAC Symposium on Intelligent Autonomous Vehicles, (pp. 307 - 312).
Danzig.
O'Kane, J. M. (2014). A Gentle Introduction to ROS. Columbia.
