ChapterPDF Available

Co-simulation of Semi-autonomous Systems: The Line Follower Robot Case Study

Authors:

Abstract and Figures

Semi-autonomous systems are capable of sensing their environment and perform their tasks autonomously, but they may also be supervised by humans. The shared manual/automatic control makes the dynamics of such systems more complex, and undesirable and hardly predictable behaviours can arise from human-machine interaction. When these systems are used in critical applications, such as autonomous driving or robotic surgery, the identification of conditions that may lead the system to violate safety requirements is of main concern, since people actually entrust their life on them. In this paper, we extend an FMI-based co-simulation framework for cyber-physical systems with the possibility of modelling semi-autonomous robots. Co-simulation can be used to gain more insights on the system under analysis at early stages of system development, and to highlight the impact of human interaction on safety. This approach is applied to the Line Follower Robot case study, available in the INTO-CPS project.
Content may be subject to copyright.
Co-simulation of semi-autonomous systems: the
Line Follower Robot case study?
Maurizio Palmieri12, Cinzia Bernardeschi2, and Paolo Masci3
1Dipartimento di Ingegneria dell’Informazione, University of Pisa
2Dipartimento di Ingegneria dell’Informazione, University of Florence
3HASLab/INESC TEC and Universidade do Minho, Braga
Abstract. Semi-autonomous systems are capable of sensing their envi-
ronment and perform their tasks autonomously, but they may also be
supervised by humans. The shared manual/automatic control makes the
dynamics of such systems more complex, and undesirable and hardly
predictable behaviors can arise from human-machine interaction. When
these systems are used in critical applications, such as autonomous driv-
ing or robotic surgery, the identification of conditions that may lead the
system to violate safety requirements is of main concern, since people ac-
tually entrust their life on them. In this paper, we extend an FMI-based
co-simulation framework for cyber-physical systems with the possibility
of modeling semi-autonomous robots. Co-simulation can be used to gain
more insights on the system under analysis at early stages of system de-
velopment, and to highlight the impact of human interaction on safety.
This approach is applied to the Line Follower Robot case study, available
in the INTO-CPS project.
1 Introduction
Cyber-Physical Systems (CPS) are complex physical systems operated by digital
controllers. The physical part (the plant) may be an entirely engineered system
(e.g., a chemical plant) as in traditional control system, but can also be a natural
system (e.g., a patient) as in medical applications. From the computational point
of view, the existence of digital and physical components requires the use of
different kinds of mathematical formalisms, e.g., discrete logic-based models for
controllers, and continuous models based on differential equations for plants.
In addition, the physical parts of a same CPS may need to be modeled with
different languages and tools. Because of this, an efficient way of simulating
CPS is by using co-simulation frameworks, which enable integrated simulation
of heterogeneous models using multiple tools.
?Published in: A.Cerone, M. Roveri (Eds.): SEFM 2017 Workshops.
Lecture Notes in Computer Science, vol 10729. Springer, 2018.
The final authenticated version is available online at
https://doi.org/10.1007/978-3-319-74781-1 29.
Semi-autonomous systems are a particular kind of CPS. In these systems,
the user interface of the system has an important role, as it allows an opera-
tor to interact with the system, e.g., to override its autonomous behavior when
desired or necessary. A common example is a car’s cruise control that automat-
ically adjusts the speed of the car. The driver can take over control at any time
just by pressing either the brake or the accelerator pedal. Another example is
robotic-assisted surgery, where a surgeon console registers the hand’s gestures of
a surgeon and translates them into micro-movement of robotic arms.
Simulation and prototyping are important technologies for early detection
of design problems in user interfaces of CPS, as they facilitate the discussion
of scenarios and design aspects in a multi-disciplinary team of developers, hu-
man factors specialists, and domain experts. In critical application domain such
as automotive and healthcare, however, simulation and prototyping alone may
not be sufficient to ensure that the system meets the safety levels required by
regulatory frameworks they can be used to explore only a finite set of sce-
narios and user input sequences. Formal methods technologies can be used to
extend simulation results and reach the necessary safety level. They are based
on mathematical proofs, and allow developers to gain additional confidence that
a system meets given safety requirements. It is therefore desirable to integrate as
much as possible formal methods with simulation and prototyping frameworks,
to make the overall development process more efficient.
In our previous work [2], we developed a CPS co-simulation framework that
integrates the Prototype Verification System (PVS) [23] and Simulink [25]. PVS
is used for modeling, simulation, and verification of critical CPS software compo-
nents. Simulink [25] is used for modeling and simulation of continuous aspects of
the CPS. Ad-hoc APIs were used to execute two models in lockstep and for time
synchronization. In the present work, we enhance this PVS-based co-simulation
framework with a Functional Mockup Interface (FMI), a tool-independent co-
simulation interface that is becoming a de-facto industry standard.
Contribution. This paper reports on our work on developing an FMI-compliant
interface for the Prototype Verification System (PVS) [23]. An example based on
a semi-autonomous vehicle is used to demonstrate the utility and capabilities of
the developed interface. A controller is modeled and verified in PVS; the PVSio-
web [18] toolkit is then used to connect the verified PVS model with a realistic
interactive dashboard prototype; a co-simulation is then carried out by using
the developed FMI interface to integrate the PVS model and dashboard with
the rest of the system components (vehicle engine, sensors, and environment)
simulated with other tools.
Structure. Section 2 presents related work on simulation and verification of
CPS. Section 3 illustrates background concepts and tools used in this work. Sec-
tions 4 and 5 present the main contributions of this work, i.e., the development
of an FMI-compliant interface for PVS, and an example application based on a
semi-autonomous system. Section 6 concludes the paper.
2 Related work
In [7], an approach is presented, based on expressing the discrete-event model
in the Vienna Development Method (VDM) [8] and the continuous-time model
in the Bond-Graph notation [14]. The simulation environment Overture [15]]
for VDM and the simulation environment 20-sim [5] for Bond-Graphs are in-
tegrated into the Crescendo tool [16]. The information needed to co-ordinate
the two models, including shared parameters, variables, and events are identi-
fied in a contract listing. Synchronization and data exchange is managed by a
co-simulation engine.
In [22], the ForSyDe modeling framework [24] is extended to support het-
erogeneous co-simulation. A ForSyDe model is a hierarchical set of processes,
where each process may belong to a Model of Computation (MoC) [17]. A MoC
represents the underlying time, synchronization, and communication model as-
sumed by a process. The framework enables processes with different MoCs to
co-execute. This framework has been extended with wrapper processes interact-
ing with external simulators or hardware components.
The INTO-CPS project [12] created an integrated tool chain for compre-
hensive Model-Based Design of CPS based on the Functional Mockup Interface
(FMI) standard [3, 4]. The core of INTO-CPS is an FMI-compliant Co-simulation
Orchestration Engine that enables a coordinated simulation of heterogeneous
models in a distributed environment. Even if many challenges are still open,
such as establishing the correctness of the co-simulation of mix continuous and
discrete behavior [6], there are efforts to apply this standard in industry.
Other works address the problem of simulating CPS by using only one spec-
ification formalism for both continuous and discrete systems (like for example,
HybridSim [27]) or extend original languages to new features, for example, in [26,
13] the integration of MATLab/Simulink with UML is proposed. For a recent
survey of co-simulation technologies for cyber-physical systems, readers may re-
fer to [10].
A complementary approach to the analysis of CPS is formal verification. In
this respect, KeYmaera [9] is a theorem prover for differential dynamic logic.
It has been applied successfully for proving correctness of automotive, avionics
and medical CPS. Our work differs from KeYmaera in that we aim to integrate
formal verification by theorem proving in PVS (see for example [1]) with the
possibility of performing co-simulation of the system. Our aim is ultimately
to facilitate the introduction of formal verification technologies in development
processes that routinely use simulation and prototyping.
3 Background
3.1 The Functional Mockup Interface
The Functional Mockup Interface (FMI) [3,4] is a tool-independent standard
to support both model exchange and co-simulation of dynamic models. Co-
simulation is performed by a number of Functional Mockup Units (FMUs), each
Fig. 1. FMI communication schema.
responsible for simulating a single sub-model. An FMU contains a sub-model
represented in the native formalism of the tool that created it, and the infor-
mation or tools needed for its execution. That is, an FMU may carry a whole
simulation environment, or just information needed by an FMI-compliant host
environment to simulate the model contained in the FMU. An FMI-compliant
host environment provides a master program that communicates with the FMUs
acting as slaves. The FMI defines a standard API for the FMUs and standard
methods of deployment for them.
The FMU’s APIs include functions called at the initialization phase, functions
to trigger one simulation step (fmi2DoStep()), and functions to exchange data.
The latter have a standard signature fmi2Get<TYPE>and fmi2Set<TYPE>,
where <TYPE>is a concrete type name, e.g., Integer or Real. These func-
tions are used to transmit data from and to the FMUs, respectively. Other two
functions, fmi2Termminate and fmi2FreeInstance, can be used to terminate
simulation and release resources. Figure 1 shows the communication pattern for
these functions.
3.2 INTO-CPS
INTO-CPS [12] is a EU-funded project that is finalizing the development of an
integrated tool-chain for model-based design of CPS based on FMI-compliant
co-simulation. The tool-chain currently supports simulation of models produced
by various tools, including OpenModelica, 20-sim, and Overture. SysML is used
to specify the overall architecture of the system to be simulated, by representing
the interconnections among the submodels.
In this work, we embrace the INTO-CPS tool-chain and a case study devel-
oped in the INTO-CPS project (the Line Follower Robot4), and use them to
demonstrate the FMI extensions we have developed for PVS (additional details
on the case study are in Section 5).
3.3 The Prototype Verification System (PVS)
The Prototype Verification System (PVS) [23] is an interactive theorem prov-
ing environment for higher-order logic. The PVS specification language provides
basic types, such as Booleans, naturals, integers, reals, and others, and type
constructors to define more complex data-types (e.g., records) and subtypes. The
mathematical properties of each type are defined axiomatically in a set of fun-
damental theories, called the prelude. New types are declared using the TYPE
keyword. A record type is a tuple whose elements are referred to by their respec-
tive field name. For example, given the declarations:
Wheels: TYPE = [ # l e ft : real , r ig ht : real #]
axl e : Whe e l s = (# l e f t := 1. 0 , r i g ht := 0 . 5 #)
the expressions left(axle) and right(axle) denote the speeds of the left
and right wheels of axle, respectively. Equivalent notations are axle‘left and
axle‘right. The override expression WITH [ .. ] can be used for in-line re-
definition of record field values. For example, for the declarations above, the
expression axle WITH [ left := -1.0 ] denotes the record value (# left :=
-1.0, right := 0.5 #). An example PVS subtype is the following:
S pe ed : TYPE = { x : real | x >= - 1 AND x <= 1 }
which defines type Speed as a subtype of real numbers in the interval [1,1].
Subtypes can be used in function definitions to define safety constraints and
limits that shall be checked. We will use these feature in Section 5, to verify
that, e.g., the velocity commanded by the controller does never exceed the robot’s
engine specifications.
3.4 PVSio and PVSio-web
PVSio [21] is a ground evaluator that computes the value of ground (variable-
free) expressions. The PVSio evaluator acts as an interactive interpreter for
the logic language of PVS, and can be used by developers to simulate a PVS
model. At the PVSio prompt, the user types a ground PVS expression (which
is equivalent to a function call of imperative languages) and PVSio returns the
result of the evaluation. For example, if a PVS theory contains the following
function definition
compute_velocity(v:real , a : real , t : real) : real = v + a * t
4https://github.com/into-cps/case-study line follower robot
then its value for a particular triple of arguments can be computed with the
following function application: compute velocity(3.5, 1.8, 3.0);
PVSio-web [18] is an open source toolkit that extends PVSio with functions
for creating and executing realistic user interface prototypes based on PVS mod-
els. Using PVSio-web, developers can define the visual appearance of the proto-
type, as well as capture user actions over input widgets, and render feedback on
displays and other output widgets on the user interface. These functionalities are
used in Section 5 to create an interactive dashboard for driving a Line Follower
robot.
4 Development of an FMI-compliant interface for PVS
We developed an FMI-compliant interface for PVS by creating a C wrapper
module that implements the FMI interface and spawns a PVSio process. A the
beginning of the co-simulation, the wrapper starts a PVSio instance, loads a
given PVS model in PVSio, and waits for input. In more detail, the wrapper
performs the following two actions at each simulation step:
Translates calls to FMI functions into appropriate commands for PVSio;
Receives replies from PVSio and translates them into FMI calls.
For co-simulation of interactive systems, as in the present work, the wrapper
creates a server module to exchange commands and data with interactive
Graphical User Interfaces (GUIs) realized as PVSio-web prototypes.
Communication between the wrapper and PVSio relies on standard Unix pipes,
and communication between the wrapper and GUI uses the WebSocket5pro-
tocol. This latter choice allows us to have a loose coupling between the FMU
and the GUI, which promotes separation of concerns between the visual appear-
ance of the user interface from its functional behavior (Model-Controller-View
architectural pattern). The overall architecture is summarized in Figure 2.
4.1 Implementation of the FMU
The FMU module implements four core functions: fmi2Instantiate, which ini-
tializes the FMU; fmi2DoStep, which executes a simulation step; and a battery
of fmi2Get/fmi2Set functions for data exchange. To use a PVS model in the
FMU, the model needs to provide at least two functions: init, that initializes
the PVS model to the initial state (where the state is represented using a PVS
record type); and step, that performs a simulation step.
The initialization function (fmi2Instantiate) starts the PVSio interpreter,
redirects the PVSio standard I/O towards a Unix pipe, invokes the init function
of the PVS model, and stores the result of the evaluation in a variable of the
FMU. Finally, a WebSocket server necessary for communication with a PVSio-
web prototype is created.
The fmi2DoStep function sends PVSio a string encoding a call to the PVS
function step. A function argument is included, representing the current state
of the system. The result of the evaluation is stored in a variable in the FMU.
5https://www.websocket.org
Fig. 2. Architecture of the FMU module for PVS.
4.2 Implementation of the PVSio-web prototype
The PVSio-web prototype is a JavaScript module executed in a web browser.
The prototype builds on the PVSio-web communication architecture to open
a Websocket connection to Websocket server executed in the PVS FMU. A
picture is used as a basis to create the visual appearance of the prototype (e.g.,
to create the remote dashboard controller prototype illustrated in Section 5, we
used a photo of a joypad and a smartphone see left side of Figure 4). Control
and display elements in the prototype are created using a library of input and
output widgets provided by PVSio-web. Button widgets capture user actions
on certain areas of the prototype (e.g., pressing a button) and translate these
actions into commands for the FMU. Display widgets are used for rendering
visible state attributes of the system, using an appropriate visualization style
(e.g., the velocity of a vehicle can be rendered with speedometer gauges). The
FMU, in turn, translates these commands into calls to functions defined in the
PVS model. State updates are periodically sent by the FMU to the PVSio-web
prototype, to keep the user interface updated with the actual system state.
5 Case study
Our case study is based on the Line Follower Robot example provided by the
INTO-CPS European project. In the original example, a autonomous unicycle-
like robot (see Figure 3) has the goal of following a line painted on the ground.
The controller of the robot receives the readings from two light sensors placed
on the front of the robot, and sends commands to the left and right motors of
Fig. 3. The INTO-CPS Line Follower Robot (from [12]).
the wheels to keep the robot on a path (the dark line in Figure 3). The INTO-
CPS project provides the FMU of the robot mechanics (created with the 20-sim
tool), the FMU of the sensors (created with 20-sim and OpenModelica), and
the FMU of the controller (created with the Overture tool). It also provides a
SysML model necessary to link these components.
In the present work, we replaced the original controller of the robot with a
more advanced controller developed in PVS. The new controller allows a driver
to override the automatic cruise control of the robot, and operate the robot
manually, using a dashboard. The sensors and the mechanics of the robot are
unaltered with respect to the original INTO-CPS example.
The prototype of the dashboard (see left side of Figure 4) provides a navi-
gation display with the trajectory of the robot, speedometer gauges to monitor
speed and engine status, and navigation buttons to allow a driver to acceler-
ate or brake, change direction of the robot (turn right,turn left), and change
gear (drive,reverse,neutral ). There is also a command (home ) to switch cruise
control from manual back to automatic mode.
In the following sub-section, a description of the PVS controller and the
PVSio-web prototype of the dashboard are presented. The full example, includ-
ing the PVS theory and the PVSio-web prototype, can be downloaded from our
repository on github6.
5.1 PVS theory of the new controller
Theory advanced controller defines the characteristics and functionalities of
the new controller. The initial part of the theory defines the structure of the
controller state (lines 9-13), and the data-types of the state attributes:
1advanced_controller: T H E ORY BE G I N
2%-- typ e d e f i n i t i o n s
6https://github.com/PaoloMasci/pvsioweb-fmi-extensions
3LS R : TYPE = { x : n o n n e g _ real | x <= 255 }
4LightSensors: TYPE = [# l ef t : L SR , r ig h t : L SR # ]
5S pe ed : TYPE = { x : real | x >= - 1 AND x <= 1 }
6M ot o rS pe e d : TYPE = [# l e ft : Spe e d , r ight : S peed # ]
7C ru i se Co n tr o l : TYPE = { A UTO , M AN UA L }
8%-- contro l l e r s t a te
9S ta te : TYPE = [#
10 l ig h tS en s or s : L i gh tS e ns or s ,
11 m ot o rS pe e d : M ot or S pe ed ,
12 g ea r : Ge a r ,
13 cc : C r u is e C o nt r o l # ]
14 %-- .. . mor e d e f i n i t i o n s o m itted f o r b r evity
15 EN D advanced_controller
Field lightSensors in the state of the controller (line 10 in the snippet above)
holds the input values received from the light sensors, ranging from 0 to 255,
according to the robot’s sensors specifications; motorSpeed holds the rotation
speed of the robot wheels, ranging from -1 to 1, according to the robot’s engine
specifications; gear is an extension used to represent possible gears of a car-like
system. It can be DRIVE,REVERSE or NEUTRAL;cc stores the cruise control mode,
which can be either AUTO or MANUAL.
The step function called at every simulation step updates the rotation speed
of the left and right engine when cruise control is in mode AUTO.
1ste p ( st: Stat e ) : Stat e =
2IF c c ( s t ) = A UT O
3THEN st WITH [
4motor S p e e d : = ( #
5l ef t : = u p da t e _ le f t _ m ot o r _ sp e e d ( s t ) ,
6right := update_right_motor_speed (st)
7#) ] ELSE st END I F
Function update left motor speed (update right motor speed) in the snip-
pet above updates the left (right) engine speed using a simple control algorithm
based on a threshold and the current light sensors reading. The same algorithm
was also used in the Overture model of original controller of the robot.
1update_left_motor_speed ( s t : S t a te ) : S p eed =
2LE T ls = lightSensors(st)
3IN C OND ls
`
rig h t < 15 0 AND ls
`
l ef t < 1 50 - > 0 .4 ,
4ls
`
rig h t > 15 0 AND ls
`
l ef t < 1 50 - > 0 .5 ,
5ls
`
rig h t < 15 0 AND ls
`
l ef t > 1 50 - > 0 .1 ,
6ELSE - > m ot o r S pe e d ( s t )
`
left ENDCOND
In the snipped above, the LET-IN construct introduces local definition that can
be used in the expressions following IN. The COND-ENDCOND expression is a many-
way switch composed of clauses of the form condition expression, where all
conditions must be mutually exclusive and must cover all possible combinations
of their truth values (an ELSE clause provides a catch-all). The PVS type checker
verifies that these constraint are satisfied.
For each control provided on the dashboard, the PVS theory provides a
matching function. For example, the accelerate button is associated with the
PVS function accelerate, which is defined as follows:
1accel e r a t e ( st : State ) : S tate = s t WITH [
2cc := M ANUA L ,
3motor S p e e d : = ( #
4lef t : = COND
5g ea r ( s t ) = D R IV E
6- > in c _C W _s p ee d ( m ot o rS p ee d ( st )
`
le f t , A CC _ S TE P ) ,
7g ea r ( s t ) = R E VE R S E
8- > i nc _C C W_ s pe e d ( mo t or S pe e d (s t )
`
le f t , A CC _ S TE P ) ,
9g ea r ( s t ) = N E UT R A L
10 - > m ot o r S pe e d ( s t )
`
left ENDCOND ,
11 rig h t : = COND
12 g ea r ( s t ) = D R IV E
13 - > i nc _C C W_ s pe e d ( mo t or S pe e d (s t )
`
ri g ht , A C C_ S T EP ) ,
14 g ea r ( s t ) = R E VE R S E
15 - > in c _C W _s p ee d ( m ot o rS p ee d ( st )
`
ri g ht , A C C_ S T EP ) ,
16 g ea r ( s t ) = N E UT R A L
17 - > m ot o r S pe e d ( s t )
`
rig h t ENDCOND # ) ]
When function accelerate is executed, cruise control is automatically changed
to MANUAL (line 2 in the snippet above). The speed of the robot is increased
by updating the rotation speed of the left and right engines of the robot by
an acceleration step ACC STEP. The specific direction of rotation of the engines
(clockwise, or counter-clockwise) depends on the gear selected by the driver,
and on which wheel the engine controls (e.g., to move the robot forward, the left
engine needs to rotate clockwise, but the right engine needs to rotate counter-
clockwise). When the gear is DRIVE (lines 5-6 and 12-13) the rotation speed of
the left and right engines is set to move the robot forward. When the gear is
REVERSE (lines 7-8 and 14-15) the rotation speed of the left and right engine
is set to move the robot backwards. When the gear is NEUTRAL (lines 9-10 and
16-17), the engine speed is kept unchanged.
For the developed theory, the PVS type-checker automatically generated 22
proof obligations to ensure correct use of types, coverage of conditions, and
disjointness of conditions. All proof obligations were automatically verified by
the PVS theorem prover in just 0.29 seconds on a standard laptop (Intel Core
i7-5500U, 8GB RAM). This ensures that the developed models does not have
bugs such as division by zero, or logic errors such as erroneous control definitions
that overshoot the robot’s engines specifications.
5.2 PVSio-web prototype of dashboard
We developed a dashboard prototype in PVSio-web (see Figure 4) to allow a
driver interact with the PVS controller and, by doing so, operate the robot and
monitor its speed and position. The prototype is based on a joypad: directional
arrows are used to steer the robot, and a number of buttons can be used to effect
Fig. 4. Screenshot taken during a co-simulation run. The dashboard prototype is on
the left. The output of the INTO-CPS co-simulation window is on the right.
actions of the robot. For example, if the robot had a mechanical arm, some of
the buttons could be used to move the arm, or if the robot represents a car-like
vehicle, as in our case, they can simulate a gear shift control.
The developed dashboard prototype uses directional arrows to control the
direction and accelerate/brake. The home button at the center for the joypad
can be used to activate automatic cruise control. These interactive controls were
created by overlaying the picture of the joypad with transparent interactive areas
that can detect button presses. Gears can be shifted using the joypad buttons
Y(drive), B(neutral), and A(brake). This is the standard approach used in
PVSio-web to create interactive controls.
A smartphone mounted at the top of the joypad is used to render speedometer
with the current speed of the robot and the current gear, and a navigation display
with the current position and direction of the robot on a map. A frameless display
at the top of the smartphone shows the cruise control mode (auto/manual). Two
additional gauge displays are placed at the bottom of the joypad, to monitor the
current rotation speed of the wheels. All these display elements were created
by overlaying the picture of the joypad with digital displays available in the
PVSio-web distribution. An external JavaScript library (d3-gauge-plus7) is used
to render gauges. The navigator display is implemented using HTML5 Scalable
Vector Graphics (SVG). The gauge and navigator displays are part of a new
domain-specific library for the PVSio-web, which will be released with the next
version of the toolkit.
5.3 Results
Several co-simulation runs were performed by connecting the PVS FMU to the
INTO-CPS Co-Simulation Engine. All experiments were configured with a fixed
7https://github.com/gimbo/d3-gauge-plus
Fig. 5. Cruise control validation. Fig. 6. Automatic vs. Manual drive.
simulation step of 0.01 seconds, and a duration of 35 seconds of simulated time.
Figure 4 shows a screen-shot from an ongoing simulation. On the left side, the
image shows the joypad prototype displaying the trajectory and speed of the
robot. On the right side, the image shows the INTO-CPS application window
hosting the co-simulation.
As a first experiment, we checked that the behavior of the new PVS controller
in automatic cruise control mode was the same of that produced by the original
Overture controller developed in the INTO-CPS project. As shown in Figure 5,
this check was successful: the trajectory of the robot is identical in the two cases
(modulo small differences due to mathematical approximation).
Other experiments were then performed to check that the robot was following
the commands given by a driver with the dashboard prototype. In one experi-
ment, we tried to use the manual drive to follow the same path as the automatic
controller, obtaining the result in Fig. 6 (manual driving is shown with a dashed
line). The plot shows that it is possible to approximately follow the same path.
The low accuracy in some sections of the track are mainly due to the relatively
high speed used by the driver to move the robot, which did not allow an accurate
control of the robot’s direction.
Finally, experiments were also performed to check the robot behavior when
switching cruise control from automatic to manual, and vice-versa. Switching
from manual to automatic mode highlighted some interesting scenarios where
the robot had an unexpected behavior. For example, the robot was sometimes
overshooting the path so much that a U-turn was then necessary to get back
on track (see Figure 7). This happened, e.g., when the control of the robot was
initially set to manual, the driver accelerated the robot to its maximum speed,
and then switched to automatic cruise control. When switching from manual to
automatic cruise control, the robot keeps the same speed set by the driver until
a direction adjustment is needed. Because of this, when the robot encounters
the line painted on the track, the speed is too high and the directional change
issues by the controller is not enough to perform the necessary sharp turn.
Fig. 7. U-turn due to high speed. Fig. 8. Missed turn.
Another example abnormal situation is shown in Figure 8, where the robot
under automatic cruise control does not perform the directional change necessary
to bring the robot on track. This seems to be a boundary case of the cruise control
algorithm: when the robot reaches the path perpendicularly, both sensors return
the same value, and cruise control decides not to turn.
6 Conclusions
We have presented the implementation of the FMI-compliant interface for the
PVS system. This allows to use PVS in FMI-based frameworks, such as the
INTO-CPS tool-chain. An example based on a semi-autonomous vehicle was
developed. The example builds on the Line Follower Robot case study of the
INTO-CPS project. We extended the example by developing a new controller
that allows an operator to drive manually the robot using a joypad-like controller.
The logic of operation of he new controller is entirely specified in PVS, which
allows the use of the PVS theorem prover to verify use-related safety properties
of the controller, such as consistency of response to user actions and visibility of
device modes (see also [11, 19, 20]).
Acknowledgments. We would like to thank the INTO-CPS team for their support
with the INTO-CPS tool-chain and the Line Follower Robot example. Paolo Masci’s
work is supported by the North Portugal Regional Operational Programme (NORTE
2020) under the PORTUGAL 2020 Partnership Agreement, and by the European Re-
gional Development Fund (ERDF) within Project “NORTE-01-0145-FEDER-000016”.
References
1. Cinzia Bernardeschi and Andrea Domenici. Verifying safety properties of a nonlin-
ear control by interactive theorem proving with the Prototype Verification System.
Information Processing Letters, 116(6):409–415, 2016.
2. Cinzia Bernardeschi, Andrea Domenici, and Paolo Masci. A pvs-simulink inte-
grated environment for model-based analysis of cyber-physical systems. IEEE
Transactions on Software Engineering, PP(99):1–1, 2017.
3. T. Blochwitz, M. Otter, M. Arnold, C. Bausch, C. Clauß, H. Elmqvist, A. Jung-
hanns, J. Mauß, M. Monteiro, T. Neidhold, D. Neumerkel, H. Olsson, J.-V. Peetz,
and S. Wolf. The Functional Mockup Interface for Tool independent Exchange of
Simulation Models. In Proc. of the 8th International Modelica Conference, pages
105–114. Link¨oping University Electronic Press, 2011.
4. Torsten Blochwitz, Martin Otter, Johan ˚
Akesson, Martin Arnold, Christoph Clauß,
Hilding Elmqvist, Markus Friedrich, Andreas Junghanns, Jakob Mauß, Dietmar
Neumerkel, Hans Olsson, and Antoine Viel. Functional Mockup Interface 2.0: The
Standard for Tool independent Exchange of Simulation Models. In Proceedings of
the 9th International Modelica Conference, pages 173–184. The Modelica Associa-
tion, 2012.
5. J. F. Broenink. Modelling, simulation and analysis with 20-sim. Journal A, 38(3):22
–25, September 1997.
6. Fabio Cremona, Marten Lohstroh, David Broman, Stavros Tripakis, Edward A.
Lee, and Michael Masin. Hybrid co-simulation: It’s about time. Technical Report
UCB/EECS-2017-6, EECS Department, University of California, Berkeley, Apr
2017.
7. John Fitzgerald, Peter Gorm Larsen, Ken Pierce, Marcel Verhoef, and Sune
Wolff. Integrated Formal Methods: 8th International Conference, IFM 2010, Nancy,
France, October 11-14, 2010. Proceedings, chapter Collaborative Modelling and Co-
simulation in the Development of Dependable Embedded Systems, pages 12–26.
Springer Berlin Heidelberg, Berlin, Heidelberg, 2010.
8. John S. Fitzgerald, Peter Gorm Larsen, and Marcel Verhoef. Vienna Development
Method. John Wiley & Sons, Inc., 2007.
9. F. Franchetti, T. M. Low, S. Mitsch, J. P. Mendoza, L. Gui, A. Phaosawasdi,
D. Padua, S. Kar, J. M. F. Moura, M. Franusich, J. Johnson, A. Platzer, and
M. M. Veloso. High-assurance spiral: End-to-end guarantees for robot and car
control. IEEE Control Systems, 37(2):82–103, April 2017.
10. Cl´audio Gomes, Casper Thule, David Broman, Peter Gorm Larsen, and Hans
Vangheluwe. Co-simulation: State of the art. arXiv preprint arXiv:1702.00686,
2017.
11. Michael D Harrison, P. Masci, Jos´e C Campos, and Paul Curzon. Verification of
user interface software: the example of use-related safety requirements and pro-
grammable medical devices. IEEE Transactions on Human-Machine Systems, to
appear., 2017.
12. Integrated Tool Chain for Model-based Design of Cyber-Physical Systems
®
project, Horizon 2020, grant number 644047.
13. N. Mulyar J. Hooman and L. Posta. Coupling simulink and uml models. In
In Proc. Symposium FORMS/FORMATS, Formal Methods for Automation and
Safety in Railway and Automotive Systems, B. Schnieder and G. Tarnai (eds.),
pages 304–311. 2004.
14. Dean Karnopp and Ronald Rosenberg. Analysis and simulation of multiport sys-
tems; the bond graph approach to physical system dynamics. M.I.T. Press, Cam-
bridge, MA, USA, 1968.
15. Peter Gorm Larsen, Nick Battle, Miguel Ferreira, John Fitzgerald, Kenneth Laus-
dahl, and Marcel Verhoef. The Overture Initiative Integrating Tools for VDM.
SIGSOFT Softw. Eng. Notes, 35(1):1–6, January 2010.
16. Peter Gorm Larsen, Carl Gamble, Kenneth Pierce, Augusto Ribeiro, and Kenneth
Lausdahl. Support for Co-modelling and Co-simulation: The Crescendo Tool, pages
97–114. Springer, 2014.
17. E. A. Lee and A. Sangiovanni-Vincentelli. A framework for comparing models of
computation. IEEE Transactions on Computer-Aided Design of Integrated Circuits
and Systems, 17(12):1217–1229, Dec 1998.
18. Paolo Masci, Patrick Oladimeji, Yi Zhang, Paul Jones, Paul Curzon, and Harold
Thimbleby. PVSio-web 2.0: Joining PVS to HCI, pages 470–478. Springer Inter-
national Publishing, 2015.
19. Paolo Masci, Rimvydas Rukˇsenas, Patrick Oladimeji, Abigail Cauchi, Andy Gim-
blett, Yunqiu Li, Paul Curzon, and Harold Thimbleby. The benefits of formalising
design guidelines: A case study on the predictability of drug infusion pumps. In-
novations in Systems and Software Engineering, 11(2):73–93, 2015.
20. Paolo Masci, Yi Zhang, Paul Jones, Paul Curzon, and Harold Thimbleby. Formal
verification of medical device user interfaces using pvs. In ETAPS/FASE2014, 17th
International Conference on Fundamental Approaches to Software Engineering.
Springer Berlin Heidelberg, 2014.
21. C. Mu˜noz. Rapid prototyping in PVS. Technical Report NIA 2003-03, NASA/CR-
2003-212418, National Institute of Aerospace, Hampton, VA, USA, 2003.
22. S. H. Attarzadeh Niaki and I. Sander. Co-simulation of embedded systems in a
heterogeneous MoC-based modeling framework. In 2011 6th IEEE International
Symposium on Industrial and Embedded Systems, pages 238–247, June 2011.
23. S. Owre, J.M. Rushby, and N. Shankar. PVS: A prototype verification system. In
Deepak Kapur, editor, Automated Deduction CADE-11, volume 607 of Lecture
Notes in Computer Science, pages 748–752. Springer Berlin Heidelberg, 1992.
24. I. Sander and A. Jantsch. System modeling and transformational design refinement
in ForSyDe. IEEE Transactions on Computer-Aided Design of Integrated Circuits
and Systems, 23(1):17–32, Jan 2004.
25. Simulink
®
web site.
26. Carl-Johan Sj¨ostedt, Martin orngren, Jianlin Shi, De-Jiu Chen, and Viktor
Ahlsten. Mapping simulink to uml in the design of embedded systems:investigating
scenarios and transformations. In OMER4 Post-proceedings, 2008, pages 137–160,
2008. QC 20100810.
27. B. Wang and J. S. Baras. Hybridsim: A modeling and co-simulation toolchain
for cyber-physical systems. In Distributed Simulation and Real Time Applications
(DS-RT), 2013 IEEE/ACM 17th International Symposium on, pages 33–40, Oct
2013.
... In [33,34], the authors extended the INTO-CPS cosimulation framework with FMUs based on the PVSio tool. Such FMUs can also implement a user interface using PVSioweb [30], allowing user interaction in the co-simulation. ...
... The framework proposed in [33] to generate an FMU starting from a PVS theory enables the usage of the co-simulation for validation of the system under attack. Modeling attacks in PVS enables the usage of the theorem prover for verification. ...
Article
Full-text available
This paper presents a methodology for the formal modeling of security attacks on cyber-physical systems, and the analysis of their effects on the system using logic theories. We consider attacks only on sensors and actuators. A simulated attack can be triggered internally by the simulation algorithm or interactively by the user, and the effect of the attack is a set of assignments to the variables defined in the Controller. The global effects of the attacks are studied by injecting attacks in the system model and co-simulating the overall system, including the system dynamics and the control part. Interesting properties of the behavior of the system under attack can also be formally proved by theorem proving. The INTO-CPS framework has been used for co-simulation, and the methodology is applied to the Line follower robot case study of the INTO-CPS project. The theorem prover of PVS has been used for deriving formal proofs of invariants of the system under attack.
... However, FAAS also controls aircraft flight, making flight paths dynamic [13]. Semi-autonomous systems can sense their environment and perform their tasks autonomously, but humans may also supervise them [14]. ...
Article
Full-text available
Bird damage to fruit crops causes significant monetary losses to farmers annually. The application of traditional bird repelling methods such as bird cannons and tree netting become inefficient in the long run, requiring high maintenance and reducing mobility. Due to their versatility, Unmanned Aerial Vehicles (UAVs) can be beneficial to solve this problem. However, due to their low battery capacity that equals low flight duration, it is necessary to evolve path planning optimization. A novel path planning optimization algorithm of UAVs based on Particle Swarm Optimization (PSO) is presented in this paper. This path planning optimization algorithm aims to manage the drone’s distance and flight time, applying optimization and randomness techniques to overcome the disadvantages of the traditional systems. The proposed algorithm’s performance was tested in three study cases: two of them in simulation to test the variation of each parameter and one in the field to test the influence on battery management and height influence. All cases were tested in the three possible situations: same incidence rate, different rates, and different rates with no bird damage to fruit crops. The field tests were also essential to understand the algorithm’s behavior of the path planning algorithm in the UAV, showing that there is less efficiency with fewer points of interest, but this does not correlate with the flight time. In addition, there is no association between the maximum horizontal speed and the flight time, which means that the function to calculate the total distance for path planning needs to be adjusted. Thus, the proposed algorithm presents promising results with an outstanding reduced average error in the total distance for the path planning obtained and low execution time, being suited for this and other applications.
Chapter
While many modeling and simulation environments provide tools for the generation of FMI-compliant FMUs, developers often have to design an FMU from scratch in order to co-simulate their own code or code from a third-party framework. This paper reports on the authors’ experience in FMU development and presents some simple guidelines based on that experience. In particular, FMU generation is discussed in the context of a model predictive control framework using a robot arm as running example.KeywordsCo-simulationModel predictive controlFMI standardDesign space explorationCyber-physical system
Book
This volume constitutes revised selected papers from the four workshops collocated with the 19th International Conference on Software Engineering and Formal Methods, SEFM 2021, held virtually during December 6–10, 2021. The 21 contributed papers presented in this volume were carefully reviewed and selected from a total of 29 submissions. The book also contains 3 invited talks. SEFM 2021 presents the following four workshops: CIFMA 2021 - 3rd International Workshop on Cognition: Interdisciplinary Foundations, Models and Applications; CoSim-CPS 2021 - 5th Workshop on Formal Co-Simulation of Cyber-Physical Systems; OpenCERT 2021 - 10th International Workshop on Open Community approaches to Education, Research and Technology; ASYDE 2021 - 3rd International Workshop on Automated and verifiable Software sYstem Development. Due to the Corona pandemic this event was held virtually.
Chapter
Designing a Model Predictive Control system requires an accurate analysis of the interplay among three main components: the plant, the control algorithm, and the processor where the algorithm is executed. A main objective of this analysis is determining if the controller running on the chosen hardware meets the time requirements and response time of the plant. The constraints, in turn, should be met with a satisfactory tradeoff between algorithm complexity and processor performance. To carry out these analyses for an autonomous vehicle control, this paper proposes to leverage parallel co-simulation between the plant, the model predictive controller and the processor.KeywordsModel predictive controlCo-simulationAutonomous vehicles
Article
Full-text available
Unmanned aerial vehicle (UAV) co-operative systems are complex cyber-physical systems that integrate a high-level control algorithm with pre-existing closed implementations of lower-level vehicle kinematics. In model-driven development, simulation is one of the techniques that are usually applied, together with testing, in the analysis of system behaviours. This work proposes a method and tools to validate the design of UAV co-operative systems based on co-simulation and formal verification. The method uses the Prototype Verification System, an interactive theorem prover based on a higher-order logic language, and the Functional Mock-up Interface, a widely accepted standard for co-simulation. In this paper, results on the co-simulation and proofs of safety requirements of a representative co-ordination algorithm are shown and discussed in a scenario where quadcopters are deployed and perform space-coverage operations.
Chapter
This paper reports on the approach used to augment a transition system tool with automatic Functional Mock-up Units (FMU) generation. To verify the FMU properties, the same transition system can be translated into a formal language. Among intrinsic system properties, transition systems are associated with the following two: the disjointedness and the coverage, which assert that the controller is deterministic and defined for every possible input. This paper shows how both properties are enforced when proving the type checking conditions derived by the PVS theorem prover.
Article
The growing complexity of embedded and cyber-physical systems makes the design of all system components from scratch increasingly impractical. Consequently, already from early stages of a design flow, designers rely on prior experience, which comes in the form of legacy code or third-party intellectual property (IP) blocks. Current approaches partly address the co-simulation problem for specific scenarios in an ad hoc style. This work suggests a general method for co-simulation of heterogeneous IPs with a system modeling and simulation framework. The external IPs can be integrated as high-level models running in an external simulator or as software- and hardware-in-the-loop simulation with minimal effort. Examples of co-simulation scenarios for wrapping models with different semantics are presented together with their practical usage in two case studies. The presented method is also used to formulate a refinement-by-replacement workflow for IP-based system design.
Article
Full-text available
Model-based design methodologies are commonly used in industry for the development of complex cyber-physical systems (CPSs). There are many different languages, tools, and formalisms for model-based design, each with its strengths and weaknesses. Instead of accepting some weaknesses of a particular tool, an alternative is to embrace heterogeneity, and to develop tool integration platforms and protocols to leverage the strengths from different environments. A fairly recent attempt in this direction is the functional mock-up interface (FMI) standard that includes support for co-simulation. Although this standard has reached acceptance in industry, it provides only limited support for simulating systems that mix continuous and discrete behavior, which are typical of CPS. This paper identifies the representation of time as a key problem, because the FMI representation does not support well the discrete events that typically occur at the cyber-physical boundary. We analyze alternatives for representing time in hybrid co-simulation and conclude that a superdense model of time using integers only solves many of these problems. We show how an execution engine can pick an adequate time resolution, and how disparities between time representations internal to co-simulated components and the resulting effects of time quantization can be managed. We propose a concrete extension to the FMI standard for supporting hybrid co-simulation that includes integer time, automatic choice of time resolution, and the use of absent signals. We explain how these extensions can be implemented modularly within the frameworks of existing simulation environments.
Article
Full-text available
One part of demonstrating that a device is acceptably safe, often required by regulatory standards, is to show that it satisfies a set of requirements known to mitigate hazards. This paper is concerned with how to demonstrate that a user interface software design is compliant with use-related safety requirements. A methodology is presented based on the use of formal methods technologies to provide guidance to developers about addressing three key verification challenges: 1) how to validate a model, and show that it is a faithful representation of the device; 2) how to formalize requirements given in natural language, and demonstrate the benefits of the formalization process; and 3) how to prove requirements of a model using readily available formal verification tools. A model of a commercial device is used throughout the paper to demonstrate the methodology. A representative set of requirements are considered. They are based on US Food and Drug Administration (FDA) draft documentation for programmable medical devices, and on best practice in user interface design illustrated in relevant international standards. The methodology aims to demonstrate how to achieve the FDA’s agenda of using formal methods to support the approval process for medical devices.
Article
Full-text available
It is essential to find new ways of enabling experts in different disciplines to collaborate more efficient in the development of ever more complex systems, under increasing market pressures. One possible solution for this challenge is to use a heterogeneous model-based approach where different teams can produce their conventional models and carry out their usual mono-disciplinary analysis, but in addition, the different models can be coupled for simulation (co-simulation), allowing the study of the global behavior of the system. Due to its potential, co-simulation is being studied in many different disciplines but with limited sharing of findings. Our aim with this work is to summarize, bridge, and enhance future research in this multidisciplinary area. We provide an overview of co-simulation approaches, research challenges, and research opportunities, together with a detailed taxonomy with different aspects of the state of the art of co-simulation and classification for the past five years. The main research needs identified are: finding generic approaches for modular, stable and accurate coupling of simulation units; and expressing the adaptations required to ensure that the coupling is correct.
Article
This paper presents a methodology, with supporting tool, for formal modeling and analysis of software components in cyber-physical systems. Using our approach, developers can integrate a simulation of logic-based specifications of software components and Simulink models of continuous processes. The integrated simulation is useful to validate the characteristics of discrete system components early in the development process. The same logic-based specifications can also be formally verified using the Prototype Verification System (PVS), to gain additional confidence that the software design complies with specific safety requirements. Modeling patterns are defined for generating the logic-based specifications from the more familiar automata-based formalism. The ultimate aim of this work is to facilitate the introduction of formal verification technologies in the software development process of cyber-physical systems, which typically requires the integrated use of different formalisms and tools. A case study from the medical domain is used to illustrate the approach. A PVS model of a pacemaker is interfaced with a Simulink model of the human heart. The overall cyber-physical system is co-simulated to validate design requirements through exploration of relevant test scenarios. Formal verification with the PVS theorem prover is demonstrated for the pacemaker model for specific safety aspects of the pacemaker design.
Article
Cyberphysical systems (CPSs), ranging from critical infrastructures such as power plants, to modern (semi) autonomous vehicles, are systems that use software to control physical processes. CPSs are made up of many different computational components. Each component runs its own piece of software that implements its control algorithms, based on its model of the environment. Every component then interacts with other components through the signals and values it sends out. Collectively, these components, and the code they run, drive the complex behaviors modern society has come to expect and rely on. Due to these intricate interactions between components, managing the hundreds to millions of lines of software to ensure that the system, as a whole, performs as desired can often be unwieldy.
Conference Paper
We describe INTO-CPS, a project that aims to realise the goal of integrated tool chains for the collaborative and multidisciplinary engineering of dependable Cyber-Physical Systems (CPSs). Challenges facing model-based CPS engineering are described, focussing on the semantic diversity of models, management of the large space of models and artefacts produced in CPS engineering, and the need to evaluate effectiveness in industrial settings. We outline the approach taken to each of these issues, particularly on the use of semantically integrated multi-models, links to architectural modelling, code generation and testing, and evaluation via industry-led studies. We describe progress on the development of a prototype tool chain from baseline tools, and discuss ongoing challenges and open research questions in this area.
Chapter
We describe tool support for multidisciplinary modelling of embedded systems using the Crescendo tool which allows discrete-event models given in the VDM notation using the Overture tool to co-simulate with continuous-time models that are developed using the 20-sim tool. The linking of discrete and continuous models via contracts, and co-simulation under the control of predefined scenarios injecting disturbances are presented.