Fig 2 - uploaded by Yliès Falcone
Content may be subject to copyright.
Source publication
Runtime enforcement (RE) is a technique to ensure that the (untrustworthy) output of a black-box system satisfies some desired properties. In RE, the output of the running system, modeled as a stream of events, is fed into an enforcement monitor. The monitor ensures that the stream complies with a certain property, by delaying or modifying events i...
Contexts in source publication
Context 1
... automata syntax and semantics. Before going into the formal definitions, we intro- duce timed automata on an example. The timed automaton in Fig. 2 defines the requirement "In every 10 time units, there cannot be more than 1 alloc action". The set of locations is L = {l 0 , l 1 , l 2 }, and l 0 is the initial location 4 . The set of actions is Σ = {alloc, rel }. There are transitions between locations upon actions. A finite set of real-valued clocks is used to model real-time ...
Similar publications
This paper investigates whether the female self-employed are more affected by the COVID- 19 crisis than the male self-employed using longitudinal data four months following the first ‘lockdown’ in the UK. We specifically test the role of family/social, economic and psychological factors on gendered differential impact. We find that self-employment...
Runtime enforcement (RE) is a technique to ensure that the (untrustworthy) output of a black-box system satisfies some desired properties. In RE, the output of the running system, modeled as a sequence of events, is fed into an enforcer. The enforcer ensures that the sequence complies with a certain property, by delaying or modifying events if nece...
Citations
... In another extension [48], a generic notion of enforcement monitors based on a memory device, finite sets of control states, and enforcement operations are introduced. Also, a systematic technique to produce a monitor from the automaton recognizing a given safety, guarantee, obligation, or response property is proposed [49]. In all these works, the monitor is usually modeled as an automaton (finite state machine), which dictates its behavior according to the input action and current state. ...
The Simplex Architecture is a runtime assurance framework where control authority may switch from an unverified and potentially unsafe advanced controller to a backup baseline controller in order to maintain the safety of an autonomous cyber-physical system. In this work, we show that runtime checks can replace the requirement to statically verify safety of the baseline controller. This is important as there are many powerful control techniques, such as model-predictive control and neural network controllers, that work well in practice but are difficult to statically verify. Since the method does not use internal information about the advanced or baseline controller, we call the approach the Black-Box Simplex Architecture. We prove the architecture is safe and present two case studies where (i) model-predictive control provides safe multi-robot coordination, and (ii) neural networks provably prevent collisions in groups of F-16 aircraft, despite the controllers occasionally outputting unsafe commands. We further show how to safely blend commands from the advanced and baseline controllers in multi-agent systems, reducing the performance impact when switching is necessary to preserve safety.
... Runtime enforcement (RE) is a technique [8,[17][18][19][20]27] to monitor the execution of a system at runtime and ensure its compliance against a set of formal requirements. An enforcement monitor (EM) is generally synthesized from a property expressed in a high-level formalism [8,[17][18][19][20]27]. ...
... Runtime enforcement (RE) is a technique [8,[17][18][19][20]27] to monitor the execution of a system at runtime and ensure its compliance against a set of formal requirements. An enforcement monitor (EM) is generally synthesized from a property expressed in a high-level formalism [8,[17][18][19][20]27]. Based on how an enforcement monitor is permitted to change the input sequence, a variety of RE models have been proposed such as [4,8,9,17,18,[20][21][22]27]. ...
... Similar to enforcement mechanisms in [19,20], we focus on online enforcement of regular properties defined as automata, where an EM (operating at runtime) is placed between an event emitter and an event receiver that execute asynchronously. An EM takes as input a sequence of events (modeling an untrustworthy execution) and transforms it into a sequence of events o that complies with a given property to be enforced. ...
Runtime enforcement is a methodology used to enforce that the output of a running system satisfies a desired property. Given a property, an enforcement monitor modifies an (untrusted) sequence of events into a sequence that complies to that property. In practice, we may have not one, but many properties to enforce. Moreover, new properties may arise as new capabilities are added to the system. It is thus important to construct not a single, i.e., monolithic monitor, but rather several monitors, one for each property. The question is to what extent such monitors can be composed, and how. In this paper, we study two enforcement monitor composition schemes, serial and parallel composition. We show that, runtime enforcement is compositional for general regular properties with respect to one of the parallel composition schemes defined. We also show that runtime enforcement is not compositional with respect to serial composition for general regular properties, but it is for certain subclasses of regular properties. The proposed compositional runtime enforcement framework is formalized and implemented. Our experimental results demonstrate the pros and cons of using the compositional approach versus the monolithic with respect to performance.
... Predictive enforcement mechanisms. Inspired by the predictive semantics of runtime verification monitors [94], predictive enforcement mechanisms were proposed in [78,79]. Predictive enforcement mechanisms leverage some apriori knowledge of the system to output some events faster, instead of delaying them until more events are observed (or permanently). ...
This chapter describes how to use in-the-field runtime techniques to improve the dependability of software systems. In particular, we first present an overall vision of the problem of ensuring highly-dependable behaviours at runtime based on the concept of autonomic monitor, and then we present the two families of relevant approaches for this purpose. First, we present techniques related to runtime enforcement that can prevent the system producing bad behaviours. Second, we describe healing techniques that can detect if the system has produced a bad behaviour and react to the situation accordingly (e.g., moving the system back to a correct state).
... Predictive enforcement mechanisms. Inspired by the predictive semantics of runtime verification monitors [94], predictive enforcement mechanisms were proposed in [78,79]. Predictive enforcement mechanisms leverage some apriori knowledge of the system to output some events faster, instead of delaying them until more events are observed (or permanently). ...
... In [15,16], the authors introduced a predictive runtime enforcement framework that allows to build an enforcement monitor with or without a-priori knowledge of the system. The enforcement monitor ensures that the system complies with a certain property, by delaying or modifying events. ...
We introduce a novel framework for runtime enforcement of safe executions in component-based systems with multi-party interactions modeled using BIP. Our technique frames runtime enforcement as a sequential decision making problem and presents two alternatives for learning optimal strategies that ensure fairness between correct traces. We target both finite and infinite state-spaces. In the finite case, we guarantee that the system avoids bad-states by casting the learning process as a one of determining a fixed point solution that converges to the optimal strategy. Though successful, this technique fails to generalize to the infinite case due to need for building a dictionary, which quantifies the performance of each state-interaction pair. As such, we further contribute by generalizing our framework to support the infinite setting. Here, we adapt ideas from function approximators and machine learning to encode each state-interaction pairs’ performance. In essence, we autonomously learn to abstract similar performing states in a relevant continuous space through the usage of deep learning. We assess our method empirically by presenting a fully implemented tool, so called RERL. Particularly, we use RERL to: (1) enforce deadlock freedom on a dining philosophers benchmark, and (2) allow for pair-wise synchronized robots to autonomously achieve consensus within a cooperative multi-agent setting.
... This paper extends the results of [20], and provides the following additional contributions: ...
Runtime enforcement (RE) is a technique to ensure that the (untrustworthy) output of a black-box system satisfies some desired properties. In RE, the output of the running system, modeled as a sequence of events, is fed into an enforcer. The enforcer ensures that the sequence complies with a certain property, by delaying or modifying events if necessary. This paper deals with predictive runtime enforcement, where the system is not entirely black-box, but we know something about its behavior. This a priori knowledge about the system allows to output some events immediately, instead of delaying them until more events are observed, or even blocking them permanently. This in turn results in better enforcement policies. We also show that if we have no knowledge about the system, then the proposed enforcement mechanism reduces to standard (non-predictive) runtime enforcement. All our results related to predictive RE of untimed properties are also formalized and proved in the Isabelle theorem prover. We also discuss how our predictive runtime enforcement framework can be extended to enforce timed properties.
... Our recent work on predictive runtime enforcement [18,19] (see related work in Section 7) motivated us to work on the predictive runtime verification problem for timed properties that we present in this paper. ...
... (e.g., a safety requirement). Recently in [18], we presented predictive runtime enforcement framework for untimed properties, where a- priori knowledge of the system ? allows to output some events immediately (whenever possible) instead of delaying them until more events are observed. ...
... allows to output some events immediately (whenever possible) instead of delaying them until more events are observed. The predictive RE approach in [18] uses operations on automata such as product and negation, and emptiness-check to decide whether an input event can be released immediately. How- ever, there are several aspects that need to be explored to be able to extend the predictive RE approach in [18] for timed properties, since delaying a timed event has the affect of changing the timed event at the output of the enforcement mechanism. ...
Runtime verification (RV) techniques are used to continuously check whether the (un-trustworthy) output of a black-box system satisfies or violates a desired property. When we consider runtime verification of timed properties, physical time elapsing between actions influences the satisfiability of the property. This paper introduces predictive runtime verification of timed properties where the system is not entirely a black-box but something about its behaviour is known a priori. A priori knowledge about the behaviour of the system allows the verification monitor to foresee the satisfaction (or violation) of the monitored property. In addition to providing a conclusive verdict earlier, the verification monitor also provides additional information such as the minimum (maximum) time when the property can be violated (satisfied) in the future. The feasibility of the proposed approach is demonstrated by a prototype implementation, which is able to synthesize predictive runtime verification monitors from timed automata.
Twenty years have passed since the first workshop on runtime verification—the area has grown and evolved with hundreds of papers published and a sizeable number of mature tools developed. In a special occasion like this it is good to look back, but it is also good to look forward to the future. In this paper, we outline a very brief history of runtime verification, and propose a way of passing the knowledge down to future generations of academics and industry practitioners in the form of a roadmap for teaching runtime verification. The proposal, based on our experience, not only equips students with the fundamental theory underpinning runtime verification, but also ensures they have the required skills to engineer it into industrial systems. Our hope is that this would increase uptake and eventually give rise to the establishment of industry-grade tools.
We develop a behavioural theory for monitors, computational entities that passively analyse the runtime behaviour of systems so as to infer properties about them. First, we present a monitor language and an instrumentation relation used for piCalculus process monitoring. We then identify contextual behavioural preorders that allow us to relate monitors according to criteria defined over monitored executions of piCalculus processes. Subsequently, we develop alternative monitor preorders that are compositional, since they allow us to relate monitors without resorting to their composite behaviour when they instrumented with systems. Importantly, we show that the latter alternative preorders are sound and complete with respect to the contextual preorders. Finally, we demonstrate how these preorders can assist the development of correct monitor synthesis tools.
Thesis Submitted in partial fulfilment of the requirements for the degree of Bachelor of Technology in Computer Science and Engineering by Ankit Pradhan (16CS01014), School of Electrical Sciences, IIT Bhubaneswar.
