Toward a hybrid causal framework for autonomous vehicle safety analysis

Abstract

Autonomous Vehicles (AVs), also known as self-driving cars, are a potentially transformative technology, but developing and demonstrating AV safety remains an open question. AVs offer some unique challenges that stretch the limits of traditional safety engineering practices. Most current safety standards and methodologies in the AV industry were not originally intended for application to autonomous vehicles, and they have significant limitations and shortcomings. In this article, we analyze the literature to first build an argument that a new safety framework is needed for AVs. We then use the identified limitations of current methodologies as a basis to formulate a set of fundamental requirements that must be met by any proposed AV safety framework. We propose a new AV safety framework based on the Hybrid Causal Logic (HCL) methodology, which combines Event Sequence Diagrams (ESDs), Fault Tree Analysis (FTA), and Bayesian Networks (BNs). The HCL framework is developed at a conceptual level and then evaluated versus the identified fundamental requirements. To further illustrate how the framework may meet the requirements, a simple example of an AV perception system scenario is developed using the HCL framework and evaluated. The results demonstrate that the HCL framework provides an integrated approach that has the potential to satisfy more completely the fundamental requirements than the current methodologies.

Full text

1 Accepted Manuscript – Journal of Risk and Reliability – Aug 2021
Toward a Hybrid Causal Framework for Autonomous Vehicle Safety Analysis
Stephen Thomas and Katrina M. Groth
Center for Risk and Reliability, Mechanical Engineering Department
University of Maryland
Abstract
Autonomous Vehicles (AVs), also known as self-driving cars, are a potentially transformative
technology, but developing and demonstrating AV safety remains an open question. AVs offer
some unique challenges that stretch the limits of traditional safety engineering practices. Most
current safety standards and methodologies in the AV industry were not originally intended for
application to autonomous vehicles, and they have significant limitations and shortcomings. In
this article, we analyze the literature to first build an argument that a new safety framework is
needed for AVs. We then use the identified limitations of current methodologies as a basis to
formulate a set of fundamental requirements that must be met by any proposed AV safety
framework. We propose a new AV safety framework based on the Hybrid Causal Logic (HCL)
methodology, which combines Event Sequence Diagrams (ESDs), Fault Tree Analysis (FTA),
and Bayesian Networks (BNs). The HCL framework is developed at a conceptual level and then
evaluated versus the identified fundamental requirements. To further illustrate how the
framework may meet the requirements, a simple example of an AV perception system scenario is
developed using the HCL framework and evaluated. The results demonstrate that the HCL
framework provides an integrated approach that has the potential to satisfy more completely the
fundamental requirements than the current methodologies.
Keywords
Autonomous vehicles, risk assessment, functional safety, hybrid causal logic, Bayesian network
1 Introduction and Motivation
Autonomous vehicles (AVs) promise to be one of the most transformative technologies of the
21st century. Self-driving cars have the potential to transform cities, help the environment, bring
affordable mobility to the disabled, and even save lives.
AV technology is poised to solve a major public safety issue. Globally, traffic accidents currently
account for over 1.3 million deaths each year and are the leading cause of death for children and
young adults aged 5-29 years.1 The U.S. National Highway Transportation Safety Administration
(NHTSA) estimates that 94% of all traffic accidents can be attributed to human error. AVs are
well-positioned to eliminate human error and significantly improve automotive safety. Even
conservative estimates indicate that the widespread deployment of safe self-driving cars could
eliminate the majority of traffic accidents, i.e., saving over half a million lives per year
worldwide.
However, developing demonstrably safe AVs has taken longer and been more difficult than
anticipated. Safety incidents with AVs and partially automated vehicles have shaken the public’s
confidence in a self-driving future. Developing credible, understandable, and trustworthy safety

2 Accepted Manuscript – Journal of Risk and Reliability – Aug 2021
engineering has become a leading focus of the AV industry. AVs offer some unique challenges
that stretch the limits of traditional safety engineering practices. Several researchers 2–4 have
highlighted novel challenges that AVs present for safety engineering, identifying limitations
including:
§ Infeasibility of complete vehicle level testing due to the enormous number of
operational scenarios and the ambiguity in requirements in an “open world”
environment
§ Increased system complexity and integrity requirements due to unavailability of a
human driver for exception handling and fallback control
§ Sensor data complexity, volume, and uncertainty
§ Complex and uncertain failure modes of autonomous systems
§ Lack of explicit requirements and functional understanding of machine learning,
stochastic algorithms, and other forms of inductive machine reasoning, making it
difficult to apply traditional system safety engineering methods
§ Interaction and tight coupling of hardware, software, environment, and human
behaviors
§ Lack of established quantitative metrics and risk assessment methods for autonomous
systems to support decision-making in the presence of uncertainty
§ Rapidly evolving technology as new sensors, computing, and algorithms continually
change AV designs and furthermore provide system adaptivity unseen in conventional
automated systems.
In addition to the novel challenges introduced by the technology, the AV industry faces unique
challenges in demonstrating system safety through standards and current methodologies. None of
the safety standards and methodologies available to the self-driving car industry were developed
with autonomous systems in mind. Current practices are something of a patchwork of standards
and methodologies, each with its own scope, assumptions, and limitations.
Modern autonomous systems pose a challenge to the paradigms used in traditional functional
safety standards. These standards have focused on system simplification, rigorous analysis, and
proven work processes to support software safety. However, autonomous systems often use high-
performance multi-core processors running complex, multi-threaded, non-deterministic
algorithms in a constantly changing operating environment. These algorithms must transform
gigabytes of diverse raw sensor data every second into a symbolic representation of the
environment. Functional safety practices that were originally intended for embedded
microcontrollers and conventional feedback control algorithms may not be adequate for the new
autonomous domain.
There is clearly an opportunity, and indeed an urgent need, for an integrated methodology that
addresses the fundamental needs of autonomous system safety, but what are those requirements?
In this article, we intend to help unify and improve AV risk assessment and safety analysis by
proposing a set of requirements for defining a safety framework, and then defining a new
analysis framework based on the Hybrid Causal Logic (HCL) methodology. This work is a
conceptual paper and is not intended to develop all details of the framework. Rather, we will
demonstrate the need for a new framework by identifying the limitations with current approaches
and identifying a set of fundamental requirements for AV safety analysis. Second, through a

3 Accepted Manuscript – Journal of Risk and Reliability – Aug 2021
combination of analysis and example, we intend to demonstrate how the HCL framework fulfils
those requirements.
The article is structured as follows. Through a literature review we demonstrate that significant
limitations and shortcomings have been found when applying existing methods to AV analysis.
We identify common themes in the literature, and, based on those themes, we then define a list of
fundamental requirements for any potential new AV safety analysis framework. We then propose
a new conceptual framework based on HCL to meet these fundamental requirements. After
defining the conceptual HCL framework and process, we evaluate the framework against each of
the fundamental requirements. To demonstrate application of the framework, we present a simple
example applied to a single operational scenario for a simplified AV system. Finally, we analyze
the example to further demonstrate how the proposed framework meets the fundamental
requirements while addressing AV-specific problems.
2 Challenges with Current Approaches
Currently, safety engineering of AVs is conducted using a combination of several approaches.
The ISO 26262 functional safety standard is the best established and most widely used approach.
However, the standard has significant shortcomings for AV analysis. Due to its fundamental
importance, ISO 26262 will receive the most focus in our analysis. Other approaches such as
ISO 21448, Systems Theoretic Accident Model and Process (STAMP), Safety Cases, and the
new UL 4600 standard have been proposed to supplement ISO 26262. Each of these is also
briefly analyzed for strengths and weaknesses.
The purpose of this review is twofold. First, we seek to demonstrate the need for a new AV
safety framework by identifying significant weaknesses in all of the current approaches to AV
safety. Second, we intend to identify common themes among these weaknesses so that we can
clearly describe a set of fundamental requirements for any proposed new framework.
We acknowledge that the methodologies discussed in this section are not a comprehensive list of
all AV safety methodologies currently in use or in development. There are numerous standards,
specifications, and whitepapers in progress across multiple countries and organizations. AV
technology is a fast-moving field. The scope of this review is to capture an accurate picture of
the current state of the art as reflected in published and well-known standards and practices. To
avoid the appearance of bias, the challenges identified in the proceeding sections are based as
much as possible on results in published literature rather than our own analysis.
2.1 ISO 26262 and ISO 21448 Standards
The ISO 26262 functional safety standard 5 is the automotive industry standard derived from the
IEC 61508 umbrella standard for functional safety. Similar to IEC 61508, ISO 26262 uses
simplified order-of-magnitude risk assessment to determine target levels of safety performance,
called Automotive Safety Integrity Levels (ASIL). Hardware safety is probabilistically
quantified with traditional reliability tools such as Failure Modes & Effects Analysis (FMEA)
and Fault Tree Analysis (FTA), while qualitative methods such as rigorous design processes and
coding standards are used for software safety.
The ISO 21448 standard6 extends ISO 26262 with a scenario-based approach to analyze
functional insufficiencies and triggering events for the system, known as Safety of the Intended

4 Accepted Manuscript – Journal of Risk and Reliability – Aug 2021
Functionality (SOTIF). However, the narrative driven description of scenarios does not provide a
clear framework for unambiguously specifying or quantifying scenarios.
ISO 26262 was originally developed in 2011 for conventional automobile electronics and did not
consider autonomous vehicles, which were still a futuristic vision at the time the standard was
initially developed. As such, there are inherent difficulties in trying to extend the standard to a
new paradigm. Likewise, the ISO 21448 standard was originally developed for advanced driver
assistance systems (ADAS) and partially automated vehicles. These standards are closely related,
so it makes sense to analyze them together.
Challenges and limitations of both ISO 26262 and 21448 for Autonomous Vehicles have been
discussed by a number of authors. Key shortcomings are described below. The shortcomings are
grouped by common themes (in italics) to aid in our later summary of fundamental requirements.
Theme: Integrated & Comprehensive
• Excludes failures caused by human error, component interactions, environmental error,
and other system failures.7
Theme: Adaptable to Novel Technology
• No methods for the unique requirements of Machine Learning safety assurance.8,9
• Novel technologies and the absence of humans in the loop complicates safety 10,11
Theme: Scenario-based Analysis
• Models analyzing one function failure at a time may not be appropriate when the
complex interactions blur the clear definition of functions.12
• Function analysis may miss specification errors and emergent properties. Difficulty in
eliciting a complete list of operational situations, and no sufficient process currently
exists.9
• ISO 21448 does not provide an adequate framework for identifying, recording, and
quantifying scenarios.13
Themes: Maintainable Model and High Fidelity
• ISO 26262 does not adequately address environmental impacts over the vehicle lifecycle,
e.g., periodic testing, degradation, etc. 14
Themes: Quantitative Assessment and Justifiable Model
• Difficulty of obtaining statistically valid failure probabilities for ISO 26262 exposure and
controllability assessment 14, which may lead to bias in selection of these parameters.9
• Quantitative methods and metrics need to be developed for autonomy.9
Not much discussed in the literature are the assumptions underlying the probabilistic hardware
failure models in the ISO standards. A constant failure rate exponential model is assumed with
only very limited consideration for uncertainty.
2.2 UL 4600 Standard
The UL 4600 Standard for the Evaluation of Autonomous Products 15 is the newest of the
relevant standards, and it is the first to explicitly target autonomous systems. It takes a safety
case-based approach, and it includes guidance for machine learning applications and validation
of autonomous systems. There is limited literature discussing the challenges and limitations, so
we make just a few observations:

5 Accepted Manuscript – Journal of Risk and Reliability – Aug 2021
Theme: Integrated & Comprehensive
• UL 4600 is a goal based standard and does not specify a certain methodology for risk
assessment or safety analysis. For example, one approach recommended by UL 4600 for
risk assessment is Bayesian estimation, but no further guidance is provided.
• A large number of acceptable methodologies are listed in the standard, but there is
limited guidance on integrating them.
Theme: Quantitative Assessment
• UL 4600 requires a large number of quantified Safety Performance Indicators not
covered by other standards, but does not propose a methodology to calculate them
Theme: Compatibility with Standards
• UL 4600 is intended to supplement and not replace ISO 26262 and ISO 21448 and
provides cross-references to demonstrate compatibility. For acceptance of supplementary
methodologies, it is important demonstrate compatibility with consensus international
standards.
Note that since UL 4600 is built around a safety case model, it also shares some limitations of
that framework, discussed below.
To a greater or lesser extent, all of the standards discussed above are largely goal-oriented and
non-prescriptive. Therefore, to get a clearer picture of current issues, we will also examine
potential weakness of some of the underlying methodologies that are referenced in the standards.
2.3 Safety Case Frameworks
A safety case is a structured argument, often graphical, supported by evidence intended to
demonstrate that a system is sufficiently safe. Common approaches to safety cases include Goal
Structuring Notation (GSN) 16,17 and Claims Argument Evidence (CAE).18,19
A safety case framework is required by both ISO 26262 and UL 4600. While the UL 4600
standard is entirely built around a safety case approach, ISO 26262 gives only minimal guidance
for the safety case. Neither standard fully specifies the details of the safety case construction.
Safety case literature highlights the following general issues:
Theme: Quantitative Assessment and Justifiable Model
• Safety cases are generally qualitative and do not provide a way to estimate failure
probabilities or confidence in the assessment without additional analysis tools.20–23
Theme: Justifiable Model and Risk-based Assessment and Integrated & Comprehensive
• Safety case arguments may be limited to likely or expected outcomes rather than worst
cases. Arguments can lead to confirmation bias, especially when important factors are
omitted.24
Theme: Maintainable Model and Justifiable Model
• Automotive safety cases may lack argumentation and simply be bulky documentation of
a compliance process. Lack of published evidence and consensus in the industry of the
effectiveness and value of a safety case.25
In theory a safety case framework could be used to make any type of argument, however in
practice there is not yet any structured way to capture a scenario-based assessment. For

6 Accepted Manuscript – Journal of Risk and Reliability – Aug 2021
example, the UL 4600 version of a safety case proposes a fault model without reference to
operational scenarios.
2.4 STAMP/STPA Framework
Systems-Theoretic Accident Model and Process (STAMP) is an accident model based on
systems theory 26, and Systems-Theoretic Process Analysis (STPA) is a hazard analysis process
derived from STAMP.27 STAMP and STPA are referenced, but not required, in both ISO 21448
and UL 4600.
STPA is based on the systems theory concept of a hierarchical control model with feedback
control loops between each layer of the model. The model may include any type of feedback
loop, including software, human, organizational, and societal interactions. The analysis uses a
keyword-based approach to identify unsafe control actions (UCA) and their systemic causes.
STPA has seen growing interest in autonomous systems industries due to its potential strengths
in dealing with complexity. However, it should be noted that STPA is primarily a hazard analysis
technique and not a risk assessment technique per se. The literature highlights several key issues:
Theme: Risk-based Assessment and Quantitative and Standards Compatibility
• STPA does not use a risk-based approach (i.e., it explicitly excludes likelihood), which
limits its utility in decision-making and makes it not fully compatible with risk-based
standards.7,28,29
Theme: Incorporate Uncertainty
• STPA is not quantitative and does not consider epistemic uncertainty in the causal
model.30,31
Theme: Scenario-based Analysis
• STPA does not define any framework for an operational scenario-based analysis,
although it could be adapted to be used in such an analysis
2.5 Traditional PRA Framework
The HCL framework we are proposing is an extension of traditional Probabilistic Risk Analysis
(PRA). For completeness, we will also briefly analyze the strengths and weaknesses of
traditional PRA so that we can evaluate how the other frameworks potentially mitigate these
weaknesses.
None of the current AV standards recommend PRA in the structured form that is commonly
practiced in the nuclear and aerospace sectors. However, ISO 26262 and UL 4600 do reference
many tools that would be familiar to PRA practitioners, including FMEA, FTA, and Event Tree
Analysis (ETA). The challenges listed below are referring to traditional PRA, but they should
equally apply to the non-integrated or overly simplified use of the tools.
Theme: High Fidelity
• Challenges of identifying complex risk scenarios, limitations of binary logic and
deterministic cause-effect relationships.32
• Inadequate assumptions of direct linear causality and independence of events. Failure to
account for changes over time.33

7 Accepted Manuscript – Journal of Risk and Reliability – Aug 2021
Theme: Integrated & Comprehensive and Adaptable to Technology
• Inadequate consideration of sociotechnical factors (e.g., organization factors, safety
culture), component interactions, design errors, human behaviors, and complex
dependencies.33,34
Themes: Justifiable Model, High-fidelity, and Incorporate Uncertainty
• Analysis relying on undocumented assumptions, design models inaccurately predicting
system behavior, inadequate treatment of uncertainty, insufficient scientific basis for the
analysis.35,36
Regarding the other fundamental requirements PRA is inherently scenario-based, risk-based, and
quantitative. Bayesian updating can enable maintainability, but it lacks some of the flexibility of
the HCL model to adapt the model over time.
In the following chapters, we will discuss in more detail how HCL addresses the limitations
identified above. In short, with the inclusion of the Bayesian Network (BN) layer, the HCL
model is no longer a “linear causal” model, and the traditional PRA limitations no longer apply.
2.6 Fundamental Requirements for a New Framework
As can be seen from the discussion above, none of the existing standards really proposes a
cohesive, integrated, and comprehensive framework for risk assessment and safety analysis. The
new UL 4600 standard perhaps comes the closest, but its goal-oriented approach is something of
a laundry list of alternatives rather than a cohesive approach. A structured approach is needed to
integrate logically the tools of system safety.
We have taken the observed shortcomings from the literature analysis and have used these as a
basis to create a concise list of high-level requirements for an ideal AV risk assessment and
safety analysis framework. Each of the requirements in
Table 1 is directly derived from one of the themes identified above. The description attempts to
concisely summarize the issues raised in the literature in the form of positive requirements.
Table 1: Fundamental Requirements for a New Framework
Requirement
Description
Robustness
Integrated &
Comprehensive
Must be capable of considering all sources of hazards & causal
factors, e.g., hardware, software, human, component interactions,
design errors, systematic faults, environment factors, organization
factors, security, in single cohesive model
High-fidelity
Model structure must adequately represent complex interactions and
behaviors, including potential non-binary states and non-
deterministic causes. Must account for common cause / dependencies
between events, including indirect dependencies and latent factors

8 Accepted Manuscript – Journal of Risk and Reliability – Aug 2021
Adaptable to
Novel
Technology
Must be inherently flexible and capable of adequately model novel
technologies such as Machine Learning (ML) and Artificial
Intelligence (AI) algorithms, statistical algorithms, or future
technologies
Scenario-based
Analysis
Must be able to model and communicate the progression of scenarios
and demonstrate consideration of a complete set of potentially
hazardous operating scenarios
Decisions
Risk-based
Assessment
Must consider both the likelihood and consequence of hazards for
risk prioritization and risk-informed decision-making
Quantitative
Assessment
Must provide estimated numerical accident frequencies and other
quantitative safety performance indicators to enable transparent and
rational decision-making
Incorporate
Uncertainty
Must explicitly consider both aleatory and epistemic uncertainty in
both the data and the risk model
Acceptance
Justifiable Model
Must provide justification for assumptions and probability values
employed, especially including use of expert opinion and historical
data. Must provide clear communication of the argumentation
structure and analysis results
Maintainable
Model
Must support the incorporation of changes to the model and/or
additional evidence over the entire life cycle without invalidating
prior model or evidence
Standards
Compatible
Must be compatible with current industry safety standards such as
ISO 26262, ISO 21448, and UL 4600
To illustrate the inter-relationship among some of the requirements, we have also organized them
into clusters. The requirements in the Robustness cluster are related to the power of the model,
i.e., its completeness, accuracy, and flexibility. The requirements in the Decisions cluster impact
the usefulness of the model for risk-informed decision-making. The Acceptance cluster relates to
the usefulness of the model over the lifecycle, i.e., will it be accepted by stakeholders and
maintained over the long term.
3 Proposed Methodology: Hybrid Causal Logic
Hybrid Causal Logic (HCL) 37 is an extension of traditional PRA that uses an interconnected
three-layer model composed of Event Sequence Diagrams (ESD), Fault Tree Analysis (FTA),
and Bayesian Networks (BN).
The top ESD layer represents a hazardous scenario to be analyzed. The scenario is evaluated at a
high-level (either functional or behavior) and is broken down into temporal sequence of discrete
pivotal events that determine the path to the scenario end state, which may be either a safe or

9 Accepted Manuscript – Journal of Risk and Reliability – Aug 2021
unsafe outcome. ESDs are similar to traditional Event Trees, but they provide some additional
modelling flexibility. While ESDs have traditionally been used to evaluate failure-initiated
scenarios, we intend to adapt the approach to incorporate ESDs initiated by specific operating
scenarios.
The middle FTA layer typically represents the failure logic for the hardware, software, and
human actors in the system. Each pivotal event in an ESD is typically connected to either the top
event in a fault tree or a node in the BN that further expands the relevant causal logic.
The bottom BN layer of the model was originally intended to capture non-deterministic causal
factors such as human, environment, software, and organizational factors. We intend to
additionally demonstrate how the BN layer will allow more accurate modelling of complex
autonomous driving algorithms and behaviors. Figure 1 below provides an overview of the
structure of an HCL model.
Figure 1: Generic Structure of an HCL Model
3.1 Application of HCL in Diverse Industries
The HCL methodology adds new capabilities and flexibility to the traditional PRA framework,
addressing the known shortcomings of traditional FTA and ETA methods. HCL is a relatively
new methodology, but it has already been successfully applied in a number of complex, high-risk
industries, as discussed below.
The proposed HCL framework for AVs was developed in cooperation with a leading developer
of commercial AV technology. Early application of the framework has provided valuable
feedback leading to methodological improvements and increased confidence in its effectiveness
and practicality. However, since the full application of the framework is still ongoing, we can
look to applications across other industries to evaluate the ultimate potential of the HCL
methodology.
One of the largest applications of HCL to date has been the development of a comprehensive
HCL-based causal model for air traffic safety in the Netherlands.38 This model, involving over

10 Accepted Manuscript – Journal of Risk and Reliability – Aug 2021
1400 events and 5000 probabilistic relationships, was validated versus historical accident data
from the U.S. and the Netherlands. The Netherlands model extended fundamental research
funded by the FAA SASO program to develop a hybrid causal model as part of a holistic systems
approach to safety for commercial aviation in the U.S. 39
NASA has completed the first phase of a project to develop an HCL-based methodology for
assuring resilience for autonomous space missions. In their report, 40 NASA highlights the
potential applicability of the HCL-based methodology to self-driving cars and unmanned aerial
vehicles. Before the advent of HCL, the ESD layer was already a standard NASA approach for
analyzing complex accident scenarios.41 For example, the International Space Station risk
assessment used ESDs to analyze over 400 different accident sequences.42
In the transportation industries, the HCL methodology has been applied in a case study to
develop a model of ship foundering risk based on 28 accident scenarios in China. 43 Another
study analyzed 50 ship collision accident reports to develop an HCL model of ship collision
risk.44 This model was empirically validated versus the annual frequency of collisions in the
Singapore Strait. In a railway application, HCL was used to analysis automatic train protection
systems on high speed railway, including uncertain sociotechnical causal relationships. 45
In the nuclear industry, Diaconeasa 46 has developed an HCL-based simulation framework to
analyze highly dynamic and complex accident scenarios in the nuclear power plant domain.
Another study 47 used an HCL-based approach to integrate human reliability analysis into
traditional nuclear power plant PRA.
In the petroleum industry, HCL was evaluated as an extension to conventional barrier-based risk
analysis, and they found it offered additional modeling flexibility when needed. 48 Another study
applied HCL to include uncertain sociotechnical factors in the assessment of offshore fire risk. 49
In the refining sector, an HCL model was developed to assess safety culture in process plants and
incorporate into risk assessments.50 In a recent study, an HCL-based methodology was developed
to incorporate human reliability analysis into oil refinery risk assessment based on a variety of
inputs from real-world refinery operations.51
Beyond strictly HCL models, there is also a large body of literature on the application of pure
BN approaches to risk assessment.52,53 The successful applications of pure BN approaches also
support the proposed HCL-based approach since they demonstrate the capabilities of a
standalone BN layer. One of the advantages of HCL is that it allows flexibility in how much of
the BN layer to incorporate. The power and flexibility of the BN comes at the cost of increased
complexity and resource commitment. The HCL methodology allows the analyst to choose the
correct balance for their application.
The HCL methodology has been proven in diverse applications, but no one has yet attempted to
apply HCL to an autonomous road vehicle. In the next section we will discuss how HCL
modelling may address the fundamental requirements identified above for a new safety analysis
framework for autonomous vehicles.
3.2 HCL Addresses the Fundamental Requirements
The purpose of this study is to evaluate the use of HCL for AVs. We can start by evaluating how
well it fulfills the fundamental requirements for AV safety analysis listed in

11 Accepted Manuscript – Journal of Risk and Reliability – Aug 2021
Table 1 above. This initial analysis relies on the available literature, and it will be reinforced by
an AV-specific example in the next section.
Integrated & Comprehensive
HCL is demonstrably integrated & comprehensive since it is capable of considering all of the
factors of interest for AV safety analysis. HCL and its constituent BNs have been used
previously for many of the areas of interest for AV analysis, including software safety54,55,
organizational factors50,56,57, human performance51,58,59, complex interactions60, and security61.
HCL provides a logic context for integrating ESDs, FTAs, and BNs, as well as indirectly
incorporating other analysis tools such as FMEA or Markov Chains. Research has also been
undertaken to integrate HCL into a modern Model-Based Systems Engineering framework. 62
High-fidelity
The bottom BN later of HCL provides great modelling power and flexibility. High fidelity is
achieved since not only can all factors be covered, but very complex and uncertain relationships
between the factors can be modelled.
In his seminal paper mapping fault trees into Bayesian networks, Bobbio 63 explains that BNs are
inherently more powerful than fault trees because they are a more general logical formalism. Key
additions are the ability to include multi-state variables and capability for modeling direct and
indirect dependencies among variables. BNs are inherently more suitable than FTA to represent
complex dependencies among components and to include causal factors and uncertainty in
modeling.
Adaptable to Novel Technology
The modelling power discussed above also supports adaptability to novel technology. The ESD
and FTA layers provide a powerful and proven framework for scenario-based analysis. On top
of that, the novel three-layer HCL architecture helps the model adapt to changing technology
(and modelling needs) by allowing the powerful but more complex BN layer to be used only as
needed 64.
The flexibility and adaptability of HCL have recently led to its application in other emerging
autonomous sectors. Research is currently underway to develop HCL models for autonomous
ships65 and autonomous spacecraft 40. Our example later will demonstrate HCL adaptability by
demonstrating how the event sequence analysis can be easily adapted to accommodate the
multitude of AV operational scenarios.
Scenario-based Analysis
One of the novel challenges of AV safety analysis is the need to identify hazards across a wide
range of “open world” operational scenarios. Traditional PRA and HCL are scenario-based, but
the emphasis is on a large number of accident scenarios derived from a relatively small number
of operational scenarios or modes. In the proposed AV framework, we will modify the
traditional PRA approach to better handle the large number of AV operational scenarios.
In typical PRA and HCL applications, the initiating events (IE) in the ESD layer are system
malfunctions. For the AV version of HCL, we will adapt the ESD such that the IE for each ESD
is the occurrence of a specific operational scenario. System malfunctions will be captured as
pivotal events in the ESD in the context of a specific operational scenario. This operational
scenario-based approach will help ensure the completeness of the Hazard Analysis and Risk

12 Accepted Manuscript – Journal of Risk and Reliability – Aug 2021
Assessment (HARA) and the adequacy of the Functional Safety Concept (FSC). This concept
will be demonstrated in the AV-specific example in the next section.
Risk-based, Quantitative, and Incorporating Uncertainty
The requirements to be risk-based, to be quantitative, and to handle uncertainty are key features
inherent to HCL and are largely inherited from traditional PRA. HCL enhances the handling of
uncertainty by incorporating model uncertainty (e.g., noisy gates) in addition to traditional data
uncertainty.
Current quantitative, risk-based approaches in automotive such as ISO 26262 are limited in both
scope and capability. ISO 26262 goes so far as to say that the main probabilistic metrics
calculated for hardware “do not have an absolute significance”5, a surprising admission from a
risk-based standard. On the other hand, systems-oriented approaches for complex systems such
as STAMP are neither risk-based nor quantitative. The proposed new HCL methodology intends
to bridge this gap with meaningful probabilities and explicitly acknowledged uncertainty.
Justifiable Model
Justifiable and explainable risk arguments are some of the strengths of the safety case approach
when using typical graphical tools such as the Goal Structuring Notation (GSN). The tree and
graph structures of HCL should be able to offer similar benefits 64, allowing application domain
experts to help build and understand the model.
Assumptions and justifications may be made explicit in the HCL model similar to how they are
shown in a GSN diagram. In fact, it is worth noting that several researchers, including the
inventor of GSN, have shown how GSN safety cases may be converted to BNs 20,21 to provide
quantitatively supported model justification.
Maintainable Model
One of the criticisms of safety cases and PRAs, and indeed artifacts from any poor safety culture,
is that they may become paper exercises that ultimately gather dust on a shelf. The new AV
safety methodology needs to be maintainable and predictive over the full lifecycle of the product.
The Bayesian approach to probability inherently has the ability to update the analysis based on
diverse types of new evidence. For HCL, this updating may be applied either at the BN level or
for individual components in the FTA or ESD. Bayesian updating approaches have shown
promise in recent AV safety research.66,67 We believe this updating capability will provide a key
incentive for maintaining the model for decision-making throughout the product lifecycle.
Although diagnostics and prognostics are not a focus of the current research, HCL and BNs have
demonstrated great utility in these areas as well.60 The ability to perform real-time risk
assessment and generate actionable inferences can provide a strong incentive to maintain the
HCL model over the lifecycle.
Standards Compatible
Lastly, the HCL methodology is only a part of a larger safety lifecycle process. It is important
that HCL fit into established frameworks such as the ISO 26262 standard. Failure to conform to
industry standards could result in lack of acceptance by the public, assessors, and regulators. As
we develop the methodology in the proceeding sections, we will show how HCL ties into the
functional safety work processes and deliverables.

13 Accepted Manuscript – Journal of Risk and Reliability – Aug 2021
3.3 Assessment of Frameworks Versus Requirements
Based on the analysis above, Table 2 below shows that the HCL methodology meets all of the
fundamental requirements for an AV safety framework, while each of the other methodologies
has several important gaps.
Table 2: Assessment of Frameworks Versus Requirements
Legend: Y = Yes, N = No, P = Partial
Requirement
ISO
Standards
UL / Safety
Case
STAMP
/STPA
PRA
HCL
Robustness
Integrated & Comprehensive
N
P
Y
P
Y
High-fidelity
N
Y
Y
P
Y
Adaptable to Technology
N
Y
Y
P
Y
Scenario-based
P
N
N
Y
Y
Decisions
Risk-based
Y
Y
N
Y
Y
Quantitative
Y
P
N
Y
Y
Incorporate Uncertainty
N
P
N
P
Y
Acceptanc
e
Justifiable Model
P
P
N/A
Y
Y
Maintainable Model
Y
P
P
P
Y
Standards Compatible
Y
P1a
P1b
P1a
P1a
Note 1: Regarding standards compatibility, none of the alternative frameworks ensure conformance with the consensus
standards, so we divide the partial conformance into two levels: a) Generally compatible with the requirements of the
standards, and b) Some aspects of framework are not compatible with the requirements of the standards.
The gap assessment is derived from the challenges discussed above. The ISO standards have
limitations in the robustness area, with a notable lack of comprehensiveness. UL 4600 and the
safety case approach allow for a more robust model, but they do not provide a clear quantitative
methodology for decision-making. The STAMP/STPA methodology is also robust, but its
explicit rejection of likelihood makes decision-making a challenge. Traditional PRA provides at
least partial coverage of all of the requirements, but the addition of the BN layer in the HCL
framework is required for full coverage of the requirements.
4 Development of HCL Model and Methodology for AVs
Now that we have established the requirements and potential advantages of the HCL framework,
we turn our attention to developing the methodology. The intent of this section is to develop the
methodology at a conceptual level for the complete HCL AV framework. We will identify the
individual steps in the methodology and provide a description of each step.
To further demonstrate standards compatibility, the steps in the methodology are placed in the
context of the ISO 26262 lifecycle, which uses a traditional V-model of system development.

14 Accepted Manuscript – Journal of Risk and Reliability – Aug 2021
The initial phases of the V-model development model are (i) the concept phase, (ii) system-level
development, and (iii) product development (including hardware and software). Our focus will
be on these first three phases to establish the conceptual HCL framework. The context is briefly
described for each step, but it is not necessary to know the details of the ISO 26262 phases to
understand the methodology. Other models of system development, such as Agile, Spiral, and
DevOps are generally not covered in current AV safety standards and are out of scope for this
study. However, as safety practices for these models become better established, integration of the
HCL framework may be an area of future research.
4.1 Concept Phase with ESD Analysis
Figure 2 below shows the initial steps of the HCL methodology in the context of the ISO 26262
concept phase. The white rounded boxes show HCL steps, while the grey boxes show
relationships to conventional ISO 26262 deliverables. The process uses familiar ISO 26262
inputs: the Item Definition which describes functional behaviors, constraints, and item
boundaries and the Operational Design Domain (ODD) definition which describes operational
and environmental conditions. The HCL process fulfils the HARA and FSC development steps
in ISO 26262 while producing compatible HARA and FSC outputs in addition to the HCL
model. We will discuss each of these steps in turn to demonstrate the application and necessary
adaptations of HCL for AVs.
Figure 2: Methodology for HCL model Part 1: ISO 26262 Concept Phase
4.1.1 Scenario Elicitation
Elicitation of a complete set of driving scenarios is a largely inductive process, and it is difficult
to demonstrate completeness. To help achieve completeness, scenario elicitation may draw from
a variety of sources, including but not limited to:
• NHTSA pre-crash scenarios68
• Regulatory required test cases
• Company developed test and simulation scenarios (and results)
• AV driving logs and disengagement reports
• Expert analysis of the ODD
• Scenario and maneuver taxonomy literature
• AV accident reports
• Structured brainstorming13

15 Accepted Manuscript – Journal of Risk and Reliability – Aug 2021
Simply adopting the HCL methodology does not automatically solve the scenario completeness
problem, but the methodology is expected to make the development and management of
scenarios easier through (i) reduction in scenario variants, (ii) visual representation of accident
sequences (iii) reuse of behaviors and functions in ESDs, and (iv) an iterative development
approach.
Scenario variants that change the probability of events (e.g., weather, speed, lighting), but not the
structure of the sequence, are captured in the conditional probabilities of the BN layer instead of
the ESD. This approach reduces the total number of required scenario permutations by orders of
magnitude.
The ESD format provides a clear, complete visual representation of accident sequences that can
be easily understood, and even built, by non-specialists. Pivotal events in the ESD can represent
any type of uncertain event, including system failures, system decisions, the presence of scenario
actors, or the actions of scenario actors. Unlike conventional ISO 26262 HARA, the ESD can
also visually communicate multiple possible accident sequences and end states in one diagram.
The ESD formalism provides additional capabilities beyond traditional Event Tree Analysis
(ETA) that provide confidence that it will be capable of adequately modeling the diverse range
of scenarios that an AV may encounter. A simple ESD may superficially resemble a traditional
event tree, but the ESD formalism extends ETA with several key features69,70, including:
• Random and deterministic time delays
• Time conditions and competing events
• Physical variable conditions
• Concurrent independent sequences
• Synchronization of sequences
• Converging sequences and cyclic processes
There are examples in the literature to illustrate the ESD capability for modeling complex,
dynamic sequences in the aerospace71, chemicals72, and autonomous maritime73 domains. We
believe these provide strong evidence that the ESD formalism will be adequate for modeling any
AV scenario, but further work is needed to fully develop the ESD methodology for AVs.
The concise, unambiguous format of the ESD also makes it possible to identify “equivalence
classes” between scenarios and sub-scenarios, reducing duplication in the model. For example,
failure to detect a vehicle in front of the AV makes little difference whether the vehicle is
stopping, slowing down, or travelling slowly. The structure of the sequence is largely the same
and can be reused across multiple scenarios.
Flexibility in the level of detail employed in the ESD allows valuable results to be obtained using
a high-level model with fewer, less detailed sequences, then later refined the ESDs with more
detailed sequences. We have demonstrated this flexibility in the proposed methodology by
splitting the ESD development into a behavioral ESD (for HARA) and a functional ESD (for
FSC), but the proposed approach could be modified to meet the needs of the development
process.
4.1.2 Behavioral ESD Development
The behavioral ESD is a high-level ESD that captures the top-level behaviors (or vehicle-level
functions) that the AV must exhibit to navigate a scenario successfully. The primary purpose of

16 Accepted Manuscript – Journal of Risk and Reliability – Aug 2021
this ESD is to allow a scenario-based HARA to be performed in order to identify safety goals
and associated ASIL targets.
Using pivotal events, the behavioral ESD also captures the high-level misbehaviors that the AV
could exhibit through the scenario. This analysis could be aided by a behavioral guide word list.
It is important to note that this level of analysis is implementation independent and is different
from traditional functional analysis such as Functional Hazard Analysis (FHA). The ESD allows
conditional relationships between functions to be mapped out explicitly, avoiding the limitations
of a function-by-function analysis. For example, misbehavior by one function may create a new
sub-sequence which triggers the need for a different behavior from other functions.
4.1.3 ESD Hazard Analysis
This step can be very similar to the typically ISO 26262 HARA process. Each “path” in the ESD
from the initiating event (i.e., the scenario) to a hazardous end state is a line item to be
considered in the HARA. Based on the properties of the initiating event, pivotal events, and end
states, the Exposure, Controllability, and Severity parameters of each path can be estimated.
Then safety goals may be associated which each line item (i.e., path) as appropriate. Similar to
traditional HARA, safety goals may be shared between scenarios if they are deemed equivalent.
4.1.4 Functional ESD Development
In this step, the behavioral ESD is revised and extended by decomposing behaviors into
functions and sub-functions. Each path with an associated safety goal is individually analyzed to
identify potential safety barriers and other pivotal events.
While the initial ESD analysis is intended to be purely behavioral, the Functional ESD is
intended to capture lower level functional behaviors and misbehaviors. The behavioral ESD
could equally describe human or machine behaviors (e.g., detect pedestrian), but the functional
ESD begins to posit specific machine functions (e.g., primary perception and backup perception).
The ESD structure may be informed by a preliminary system architecture.
The functional ESD may also include other scenario factors such as time delays, competing
events, and other random events that may affect the scenario structure, i.e., not just hardware and
software failures. Pivotal events may capture any event with an uncertain outcome. Specifically,
the ESD pivotal events may also capture SOTIF triggering events and weaknesses, allowing an
integrated model of functional safety and SOTIF.
4.1.5 ESD Architecture Analysis
In the earlier hazard analysis step, each path to a hazardous end state in each ESD was analyzed
to yield safety goals and ASIL targets. Then in the functional ESD development step, each path
was individually analyzed to identify potential safety barriers and other pivotal events.
In this final step of the concept phase, the detailed paths are evaluated to determine which pivotal
events will be credited to reduce the risk, and which functions will be designated safety-critical.
Safety-critical functions will be assigned ASIL targets consistent with top-level safety goals, and
functional safety requirements will be derived. Each functional safety requirement corresponds
to a pivotal event in the functional ESD. In ISO 26262 terminology, this step corresponds to the
functional safety concept and ASIL decomposition.

17 Accepted Manuscript – Journal of Risk and Reliability – Aug 2021
4.2 System and Product Phases with FTA/BN Analysis
The ESD analysis completes the conceptual phase of the lifecycle. In the system-level and
product development phases, the FTA and BN are the primary tools.
Figure 3 below continues showing the steps of the HCL methodology, this time in the context of
the ISO 26262 system and product development phases. The formatting is similar to Figure 2.
We will discuss each of the HCL steps in turn to develop the conceptual application of the FTA
and BN layers in an AV context.
Figure 3: Methodology for HCL model Part 2: System and Product Phases
4.2.1 System-Level FTA Development
System-level FTA development proceeds much as it would in a conventional ISO 26262 process.
The Safety Goals (SGs) and Functional Safety Concept (FSC) from the previous steps provide
input to propose a Technical Safety Concept (TSC) is implement the higher-level requirements.
Together, the SGs, FSC, and TSC form a hierarchical tree of requirements that form the
structural basis for the initial system-level FTA. This mapping of the requirements to the FTA
kicks off an iterative process of refining both the TSC and the FTA. The FTA results provide
feedback to the design team on issues such as single point faults, common cause, and latent
failures that may require the TSC to be revised. The initial system-level FTA is typical
qualitative and focuses on modelling the structure of the system and ensuring a complete,
consistent, and feasible set of technical safety requirements to fulfil the safety goals.
Developing the FTA in an HCL context has a few additional features which are briefly outlined
next. First, each FTA top event is derived from a specific ESD pivotal event in a specific
operational scenario. As a result, the FTA can be built knowing the exact operational context,
enabling a more detailed and accurate analysis. For example, rather than analyzing “failure to
perceive vehicle”, the ESD richer context might give us “failure to perceive a vehicle
approaching from the left at an occluded intersection.” The more specific ESD context both
simplifies the FTA analysis and makes the analysis more accurate by including only the relevant
variables (e.g., sensors on the left of the vehicle).
As the FTA is built, we can also look for situations where the logical relationships between
events is more complex than can be accurately captured by the strict AND/OR logic of FTA.
Similarly, we can begin to identify “soft” causal factors that would typically be left out of FTA.

18 Accepted Manuscript – Journal of Risk and Reliability – Aug 2021
For example, the speed of the approaching vehicle effects both the potential severity and the
probability of a collision. These relationships and factors provide input to the BN layer in the
next step.
One side-effect of integrating the ESD and FTA layers is that the HCL model typically includes
a relatively large number of fault trees, but each fault tree is smaller (compared to a fault tree
only approach). This feature is helpful for modularity, but it can lead to some redundancies in the
model. When evaluating a large number of scenarios, there is the potential for repetition between
ESD pivotal events and FTA events with only small differences. Fortunately, available HCL
software is able to handle this complexity via transfer gates and other features.37
4.2.2 Conceptual BN Development
This step takes the soft relationships and causal factors identified in the previous step and begins
developing the structure for a qualitative BN. Depending on the system characteristics and
modelling needs, these causal factors could include:
• Organizational factors (safety culture, competence, time pressures, etc.)
• Complex system behaviors (ML/AI, Kalman filters, complex software, etc.)
• Human actions (errors, reactions, etc.)
• ODD conditions (weather, speed, lighting, features, etc.)
During this step, basic events in the FTA are connecting to the influencing factors in the BN. In
some cases, entire subtrees of the FTA may be replaced if the BN provides a significantly more
accurate representation.
Note that due to the larger number of available parameters, the BN layer always has the
capability to produce a more accurate model than the FTA at the expense of increased
complexity. It is up to the judgement of the modeler to determine where that trade-off is
warranted.
4.2.3 Detailed FTA Development and Detailed BN Development
The final steps in the proposed methodology are the development of the detailed FTA and BN
layers. The current paper is focused on the development of the conceptual framework, so the
details of these steps are reserved for future work.
5 Example of an HCL Model for an AV Scenario
To illustrate the application of the conceptual framework described above, we develop a typical
operational scenario for a self-driving car. One of the most common two-car accident scenarios
is another vehicle stopped in the lane ahead of the AV. In this section we develop notional HCL
models for the initiating event “vehicle stopped in lane” ahead of the AV.
5.1 Conceptual Phase
At a conceptual level, there is a well-defined sequence of events that the AV must complete after
the initiating event “vehicle stopped in lane” to avoid an accident. The AV must know what lane
it is in, detect the stopped vehicle in its lane, decide how to respond, and execute the response
(i.e., maneuvering) safely and correctly. This sequence is captured in a simple behavioral ESD in
Figure 4 below. Note that we use the term ego in the diagram to identify the AV and differentiate
it from other AV and non-AV actors in the scenario.

19 Accepted Manuscript – Journal of Risk and Reliability – Aug 2021
Figure 4: Example Behavioral ESD for Vehicle Stopped in Lane
When combined with some basic ODD information (e.g., speed limit, traffic density), this simple
ESD is sufficient to identify several HARA scenarios and estimate exposure, controllability, and
severity. In ISO 26262 terminology, each unique path leading to a hazardous end state would be
a hazard scenario to be recorded in the HARA. Top-level safety goal and ASIL targets may be
identified for each hazard scenario (and likely reused in future scenarios).
The next step in the process is to develop each path into a functional ESD, shown in Figure 5
below. For this example, we have chosen to only develop the perception path starting with
pivotal event 2 highlighted in Figure 4 above. The connection between PE-2 in the behavioral
ESD and the functional ESD can be thought of as a transfer gate where additional detail for PE-2
is developed in the functional ESD.
Why split the behavioral ESD and the functional ESD at all? The answer lies in the product
development phases. Building the behavioral ESD first allows the HARA to be performed as
early as possible based on behavioral requirements only and without even a conceptual design
developed. The functional ESD is where conceptual architectural options may be explored.
Similarly, we use the functional ESD rather than the FTA to propose conceptual architectures
because it allows architectural proposals to made sooner using the simpler ESD format.
In this example, we propose a conceptual design in the functional ESD. The perception function
has been broken into primary (2-A) and backup (2-B) functions. This is clearly a design proposal
that is not driven by the behavioral requirements. These pivotal events will be developed further
in the FTA layer.
For clarity of the example, we have limited the Figure 5 ESD to only two pivotal events, but that
could be extended with additional events as necessary for a more nuanced model. For example,
we could include a pivotal event for “partially successful” functioning if that is a possible system
behavior. This flexibility allows the ESD to model complex scenarios with multiple possible end
states. Note that there is no assumption of independence between pivotal events in the ESD.

20 Accepted Manuscript – Journal of Risk and Reliability – Aug 2021
Figure 5: Example Functional ESD for Perception Path Only
5.2 System and Product Development Phase
In the previous Concept Phase step, a perception system design did not yet exist. The designers
conceptually proposed a redundant perception architecture in pivotal events 2-A and 2-B. To
continue this example into the System phase, let us assume that the perception system designers
have further developed this proposed design as described below.
5.2.1 Example System Description
The proposed design places the primary (2-A) and backup (2-B) perception functions on
redundant computing hardware using diverse ML algorithms. The primary system uses LIDAR
and a camera, while the backup system uses RADAR and a different camera. The redundant
object detections are sent to the rest of the AV system for further processing (not shown). A
block diagram of this design is shown in Figure 6 below.

21 Accepted Manuscript – Journal of Risk and Reliability – Aug 2021
Figure 6: Example Perception Architecture Block Diagram
Further, we assume that estimates have been made for the probability of failure (in a dangerous
failure mode) for each component in the architecture, shown in Table 3 below. Note that these
probability values are for illustration only.
Table 3: Assumed Component Failure Probabilities
Component
Failure Probability
Compute
0.005
ML Software
0.04
Lidar
0.02
Radar
0.05
Camera
0.01
For purposes of this example, we will be focusing on the common cause factors among the
sensors. For simplicity, we are neglecting common cause effects among the compute and ML
software components as they are ancillary to the example.
5.2.2 Development of FTA and BN Layers
A simple system-level FTA is developed for pivotal events 2-A and 2-B and shown in Figure 7
below. For this simple example we have skipped the qualitative FTA step and gone directly to a
quantitative FTA. As noted above, this FTA neglects factors such as other failure modes, mission
times, and common cause (except as noted below).
As can be seen in Figure 7, the FTA formalism is inadequate to model the AV with high fidelity.
The notes in Figure 7 highlight places where the FTA fails to adequately model the system. In
particular, for the sensors, the failure of an individual sensor does not automatically cause
perception to fail, so it is not properly an OR gate. However, the failure of one sensor
significantly increases the probability that perception will fail, so it is not properly an AND gate
either. This type of complex interaction is more accurately handled by a “noisy” gate in the BN
layer.
Further, the behavior of individual sensors may not be adequately described by “failed” or “not
failed” FTA states. While these states may be adequate for the scope of ISO 26262, they do not
cover the wide range of SOTIF factors. For example, the performance of the individual sensors
could also be non-deterministically affected by driving conditions, resulting in an uncertain

22 Accepted Manuscript – Journal of Risk and Reliability – Aug 2021
range of in-between states such as “mildly degraded” or “very degraded”. Soft causal factors in
the SOTIF domain go beyond sensor performance and may include weather, lighting, driver
behaviors, speeds, etc.
Soft causal influences on the ML algorithms are also best handled by conditional probabilities in
the BN layer. No simple FTA common cause model (e.g., a Beta model) will adequately model
the complex interactions among the sensors and environmental conditions. The ML algorithms
are diverse, but they are likely indirectly influenced by common environmental causal factors of
the sensors. For this simple example, we concentrated on sensor causal factors, but another factor
could be that the distance (or time-to-collision) required for a “successful” detection varies based
on the minimum braking distance, which may be influenced by ODD conditions such as rain.
Figure 7: Example of Flawed System-level Fault Trees for Perception Events
The final step of this example corresponds to the developed of the detailed FTA and BN layers to
address the issues highlighted above. In this step, the flawed FTA in Figure 7 is modified, and
the BN layer is added to address the limitations identified in Figure 7. The combined FTA and
BN layers are shown in Figure 8 below. The conditional probability tables (CPTs) for all of the
BN nodes are shown in Figure 9.
Figure 8 illustrates how the BN allows seamless integration of functional safety and SOTIF
concerns. In the BN layer, the concept of sensor “failure” has been extended with the more
general concept of sensor “performance”. In other words, electronic faults and failures of the
sensor (i.e., functional safety) are just one potential cause of poor sensor performance. Other
causes include SOTIF triggers such as rain or poor lighting. We have not included degraded
sensor performance states in this example, but they could be easily added.
As with the FTA, note that all quantitative values in this example are for illustrative purposes
only. The methodology for rigorous quantification of the FTA and BN layers for an AV system
is not developed in this example and is reserved for future work. The example probabilities in the

23 Accepted Manuscript – Journal of Risk and Reliability – Aug 2021
CPTs are intended to illustrate the format and flexibility of the model. The conditional
probabilities are logically consistent (e.g., heavy rain is worse than light rain), but they are not
derived from real-world data.
Figure 8: Example of Integration between FTA and BN Layers

24 Accepted Manuscript – Journal of Risk and Reliability – Aug 2021
Figure 9: Conditional Probability Tables for BN Layer
6 Discussion
Once all three layers of the model structure have been developed and the nodes have been
quantified, then the overall HCL model is quantified using algorithms that have already been
described in the HCL literature.37,64 Since the system Safety Goals were defined based on the
ESD layer, the quantification of the model can also provide quantification of both the ASIL
hardware metrics and SOTIF performance metrics consistent with ISO 26262 and ISO 21448.
6.1 Discussion of the Example Results
The BN model can more accurately capture the relationships between multiple types of sensors
involved in location perception. These dependencies are captured both in the structure of the BN
model and in the conditional probability tables (CPTs). For the sensor model, the AND gate has
been removed from the FTA and replaced with a BN event Sensor Performance. By making this
change, we have extended the strict AND-gate logic of the FTA to include additional uncertainty
and flexibility in the BN. In the updated model, if both sensors fault, the sensor set certainly
faults. However, if only one sensor faults, we still allow for a possible fault through common
cause. In the case where neither sensor is faulted, we allow for some “leakage” to account for
unknowns. This table illustrates the flexibility offered by the BN noisy gates. The CPT for the
Sensor Performance are shown in Figure 9(a) above.
In addition, we have demonstrated the incorporation of soft causal factors by identifying Rain
and Lighting as causal factors for sensor performance. Their respective probabilistic impacts on
Lidar and camera performance have been quantified in the BN. Note that the Rain node has three
possible states, illustrating the BN capability for non-binary variables. The camera performance
is affected by both rain and lighting, so it has the most complex CPT, shown in Figure 9(b). The
CPT covers the conditional probabilities for all combinations of rain and lighting, capture six
different scenario variants in one compact table.
Another benefit of the BN layer that may not be intuitively obvious is the inherent BN ability to
apply observed evidence (or hypothetical evidence) to each BN node and update the conditional
probabilities across the network. For example, we may ask the question: “What is the probability
that perception will fail if we are driving at night in a light rain, and the radar sensor has already
faulted?” A summary of the relevant probabilities for this question is shown in Table 4 below.

25 Accepted Manuscript – Journal of Risk and Reliability – Aug 2021
Table 4: Example of Applying Evidence in the BN
Variable
Prior
Probability
Posterior
Probability
Comment
Pr(Rain=Light Rain)
30%
100%
Observed value
Pr(Lighting=Night)
25%
100%
Observed value
Pr(Radar Perf=Fault)
5%
100%
Observed value
Pr(SensorPerf1=Fault)
5.4%
6.4%
Propagated in BN
Pr(SensorPerf2=Fault)
5.4%
22.6%
Propagated in BN
Pr(PercDetection=Fail)
0.30%
1.4%
Top level failure probability increases
The propagation of probability updates in the BN is also bidirectional. This ability is rooted in
the use of Bayes’ Theorem:
!
"
#
$
%
&
' ( 𝑷(𝑩|𝑨)𝑷(𝑨)
𝑷(𝑩)
(1)
Bayes’ Theorem allows a network to compute the conditional probability of B|A from the
conditional probability of A|B and vice versa. The implication is that analysts can conduct
reasoning forward from A to B (causal reasoning), but also backward from B to A (evidential
reasoning).
So, in addition to updating top-level system probabilities with causal reasoning as shown above,
the BN may also be used in a prognostic mode with evidential reasoning. In this mode, evidence
is placed on higher level nodes and the conditional probabilities are automatically propagated to
lower level nodes. In this way, the BN can be used to identify the most likely causal factors of an
observed high-level failure. These methods may also be used in combination (intercausal
reasoning) to reason in the presence of both observed causes and observed effects in complex
systems. See these references74–76 for further discussion of reasoning in BNs.
Figure 10 below illustrates the use of bidirectional (i.e., intercausal) inference in the BN layer. In
this example, we have obtained evidence that the Sensor 2 set has failed, so the probability of the
Fault state is set to 100% on the Sensor Performance 2 node. This evidence is automatically
propagated throughout the BN, as shown via the dashed arrows. Initially the information
propagates backward (i.e., evidential reasoning), but once it reaches the Rain and Lighting nodes
it begins to propagate forward (i.e., causal reasoning). Perhaps counterintuitively, the observation
of Sensor 2 failure actually increases the probability that Sensor 1 will fail because they share
Rain and Lighting as common cause factors.

26 Accepted Manuscript – Journal of Risk and Reliability – Aug 2021
Figure 10: Intercausal Inference in the BN Layer
6.2 Discussion of Fundamental Requirements in the Example
The above example gives a simple illustration of applying the proposed HCL framework to a
single scenario for the perception portion of an AV. The example demonstrates many of the
fundamental principles identified in
Table 1. The potential for the HCL methodology to fulfill fundamental requirements was detailed
earlier in section 3.2. Here we will summarize the requirements that were specifically illustrated
in our example.
Integrated & Comprehensive
The case study gives a simple example of how functional safety and SOTIF issues may be
integrated into a single comprehensive probabilistic model. The scenario-based model is
sufficiently flexible and comprehensive to account for various complexities and “soft” factors in
AV safety analysis that would normally be excluded from an ISO 26262 analysis.
The HCL framework represents a potential approach for quantitatively analyzing the functional
dependencies and triggering events covered by ISO 21448. This standard is still evolving and
does not yet propose a specific methodology. Although our example was limited to a generic
perception system, in the future we envision the framework being used to characterize specific
functions and behaviors (e.g., detect an occluded pedestrian at night) consistent with the
standard.
High-fidelity
The integrated FTA and BN layers in Figure 8 illustrate how the BN extends the FTA to capture
complex interdependencies and uncertainties typical of an AV application. There are no
unrealistic assumptions about independence, and the BN layer allows sophisticated
characterization of dependent failures. As illustrated in Table 4, the model can answer complex

27 Accepted Manuscript – Journal of Risk and Reliability – Aug 2021
queries such as “What is the probability of system failure given a radar failure at night in the
rain?” to allow risk-informed decision making.
Adaptable to Novel Technology
One of the key advantages of the HCL model is the capability to explicitly acknowledge and
characterize the uncertainty around the performance of new technologies. The example
illustrated using the HCL model to characterize a system that included complex perception
sensors and machine learned algorithms in an uncertain operating environment. Where
performance is well understood, the CPTs can be crisply defined. In cases where the analyst is
less confident, the “noisy” BN gates allow for incorporating imperfect knowledge into the
analysis. As novel technology becomes better understood, the Bayesian updating process
provides a natural route for updating the knowledge in the HCL model.
Scenario-based Analysis
The ESDs in Figure 4 and Figure 5 demonstrate how operational scenario sequences may be
unambiguously specified, as well as how the development process may be phased to meet the
development process needs. The behavioral ESDs may in fact be developed before the
architecture of the product is even defined.
As discussed in section 4.1.1, the ESD layer provides the capability to probabilistically model
much more complex scenarios than our example. Our conceptual example was necessarily a
simplified scenario, although it does represent one of the leading automotive fatal accident
scenarios. Further development of the ESD methodology for complex AV scenarios is reserved
for future work.
It is possible, although we think unlikely, that some AV scenarios are so complex that it is not
feasible to accurately represent them using the ESD and FTA formalisms, even with the
additional BN layer. In these rare cases, the HCL framework may act as a “stepping-stone” to a
fully BN-based model for those scenarios. A pure BN model offers the maximum flexibility and
fidelity at the cost of increased complexity. This approach has been demonstrated in the literature
by Roelen et al 38 where an HCL model was ultimately converted to a pure BN model. The
authors emphasized that the HCL step was essential in eliciting scenarios and building
understanding of the system. Note that a pure BN model for a scenario could still be part of an
overall HCL model since the scope of the three HCL layers is at the discretion of the analyst.
Risk-based, Quantitative, and Incorporating Uncertainty
This paper was intended to develop and justify the HCL methodology for AVs at a conceptual
level, so neither the framework definition nor the example developed a rigorous approach for
model quantification. However, quantification and the incorporation of uncertainty are important
aspects of the complete methodology, so we respond to some potential concerns below.
On the feasibility of quantification: This common objection often comes from those not familiar
with the Bayesian view of probability. Simply put, any qualitative judgement has an implicit
quantitative measure associated with it. The quantitative measure and its uncertainties can be
rigorously elicited through a variety of techniques. The Bayesian approach will allow us to
rigorously combine hard evidence, soft evidence, and expert opinion to produce quantitative
values and uncertainties as needed. There is a large volume of PRA and HCL literature that
support the feasibility and utility of the Bayesian approach.32

28 Accepted Manuscript – Journal of Risk and Reliability – Aug 2021
On the importance of quantification: Besides the inherent engineering usefulness of having
quantitative values, they also provide another important benefit in the form of Bayesian
updating. As new data is received from the field, having prior quantitative values allows the new
evidence to be rigorously incorporated into an updated model. This updating process allows the
model to be continually updated during the development process and maintained over the
lifecycle of the system.
Justifiable Model
A key aspect of a justifiable model is an understandable model. The explainability of the
HCL model is well illustrated in Figure 8 where the interconnected FTA and BN graph
visually convey the relevant causal factors in the system. Both layers are explicitly
connected to events in an operational scenario described in the ESD in Figures 4 and 5. The
graphical structure of the model helps make clear to the non-specialist all of the factors
incorporated into the model.
The justification of the model data was beyond the scope of this conceptual paper. However,
using illustrative data, we demonstrated how the flawed and incomplete logic in the Figure 7
FTA could be rectified by incorporating the more flexible BN layer. The “noisy gates” and
soft causal factors in the BN were fully quantified in the CPTs to provide a more accurate
probability for design decisions and/or risk acceptance decisions. Ultimately, we believe
that the requirement to fully quantify the CPTs inherently drives rigor in the selection and
justification of the underlying data.
Maintainable Model
Since we were not demonstrating quantification technique in the example, we did not
explicitly demonstrate the maintainability aspects of the model in the example. However, we
can qualitatively observe that our simple BN layer in the example could be easily extended
in the future based on either ODD expansions or field experience. For example, existing
models could have additional causal factors added (e.g., snow or fog), or entirely new causal
networks could be added (e.g., organizational factors). Probabilities assigned in the network
may also be updated based on operational data using well-known Bayesian updating
methods.
Standards Compatible
The example illustrated the practical application of the methodology through steps that were
shown to be compatible with ISO 26262 in Figure 2 and Figure 3. The treatment of both
functional safety and SOTIF in the HCL methodology provides a unifying methodology
between ISO 26262 and ISO 21448. Obviously many of the details of ISO 26262
conformance were not explicitly covered in our example (e.g., ASIL targets, safety
requirements, etc.), but those pieces are tangential to the safety analysis process covered in
the article.
6.3 Future Work
This initial research was focused on developing a holistic framework for AV safety at a
conceptual level. As such there are several areas of remaining work to develop the fully detailed
framework.
The concept phase of the framework needs further development, starting with the methodology
for developing a comprehensive set of scenario-based ESDs. Additional definition of the process

29 Accepted Manuscript – Journal of Risk and Reliability – Aug 2021
for building ESDs is needed, including methods for handling the inevitable repetition from
developing a very large set of scenarios. For the FTA and BN layers, we need to establish a
process to identify where FTA models are inadequate, and where the BN layer is most valuable
for AVs. A rigorous methodology for quantifying the BN layer with uncertainty is needed.
Finally, development of more detailed practical examples is necessary to demonstrate and
validate the HCL methodology. At the ESD layer, more complicated and detailed scenarios can
be demonstrated. At the lower layers, the simple perception example in this paper could be
expanded with realistic detail. Other functions of the AV, including localization, prediction, and
motion planning are also excellent candidates for HCL modelling. A comprehensive model of
the ODD causal factors and their interaction with the entire AV stack would be the ultimate goal.
7 Conclusions
In this study, we developed the fundamental requirements for a new autonomous vehicle risk
assessment and safety analysis framework, and we proposed a new framework based on the
established Hybrid Causal Logic (HCL) methodology. The new framework allows the
integration of functional safety and SOTIF analysis, including soft causal factors such as ODD
conditions and organizational factors. This work is the first to propose applying the HCL
methodology in the self-driving car domain.
To ensure the proposed framework meets the needs of the AV domain, we developed a set of
fundamental requirements for any new AV safety framework. To develop and justify the
requirements, we systematically reviewed the literature to identify current challenges with AV
safety as well as identified shortcomings or limitations of current methodologies. Current
methodologies and the new methodology were evaluated against these requirements. These
requirements could also be used to evaluate any proposed methodology for AV safety.
The proposed HCL methodology was systematically reviewed against the identified fundamental
requirements, indicating that HCL meets all of the fundamental requirements. A step-by-step
HCL methodology was developed for AV applications consistent with the ISO 26262 product
development lifecycle. The methodology was then applied to a simple AV operational scenario
to demonstrate its utility. The results of this example were then evaluated against the
fundamental requirements and were shown to fulfil these requirements as predicted by the
general HCL requirements analysis.
The HCL methodology in this article was adapted and applied in novels ways for the
autonomous vehicle domain. Rather than a failure-driven approach for the ESD layer, we
developed an operational scenario-driven approach to make feasible the analysis of a wide range
of AV scenarios. The application of the BN layer for characterizing the SOTIF performance of
the AV in diverse ODD conditions is a novel approach in the AV domain. For practical
purposes, it was clearly shown how the HCL framework could seamlessly extend the existing
ISO 26262 methodology to provide a unifying framework addressing both functional safety and
SOTIF issues for an AV.
There is a clear need for an integrated methodology that addresses the fundamental needs of
autonomous vehicle system safety. Based on an analysis of the fundamental requirements for
such a methodology, the Hybrid Causal Logic methodology appears to be an excellent candidate
and should be further developed for application to autonomous vehicles.

30 Accepted Manuscript – Journal of Risk and Reliability – Aug 2021
8 References
1. Webb CN. Motor Vehicle Traffic Crashes as a Leading Cause of Death in the United
States, 2015. United States. National Highway Traffic Safety Administration, 2018.
2. Koopman P, Wagner M. Autonomous Vehicle Safety: An Interdisciplinary Challenge.
IEEE Intelligent Transportation Systems Magazine 2017; 9: 90–96.
3. Kalra N, Paddock SM. Driving to safety: How many miles of driving would it take to
demonstrate autonomous vehicle reliability? Transportation Research Part A: Policy and
Practice 2016; 94: 182–193.
4. Alexander RD, Ashmore R, Banks A. The State of Solutions for Autonomous Systems
Safety. Report for the Safety Critical Systems Club, York, UK,
http://eprints.whiterose.ac.uk/127573/ (2018, accessed September 12, 2020).
5. ISO 26262:2018. Road vehicles - Functional safety.
6. ISO/PAS 21448:2019. Road vehicles - Safety of the intended functionality.
7. Abdulkhaleq A, Wagner S, Lammering D, et al. Using STPA in Compliance with ISO
26262 for Developing a Safe Architecture for Fully Automated Vehicles. In: Automotive-
Safety & Security 2017-Sicherheit und Zuverlässigkeit für automobile
Informationstechnik. 2017, pp. 149–162.
8. Salay R, Queiroz R, Czarnecki K. An Analysis of ISO 26262: Machine Learning and
Safety in Automotive Software. SAE Technical Paper (No. 2018-01-1075).
9. Adedjouma M, Pedroza G, Bannour B. Representative safety assessment of autonomous
vehicle for public transportation. In: Proceedings - 2018 IEEE 21st International
Symposium on Real-Time Computing, ISORC 2018. Institute of Electrical and Electronics
Engineers Inc., 2018, pp. 124–129.
10. Koopman P, Ferrell U, Fratrik F, et al. A Safety Standard Approach for Fully
Autonomous Vehicles. In: International Conference on Computer Safety, Reliability, and
Security. Springer, Cham, 2019, pp. 326–332.
11. Koopman P, Wagner M. Challenges in Autonomous Vehicle Testing and Validation.
SAE International Journal of Transportation Safety 2016; 4: 15–24.
12. Martin H, Tschabuschnig K, Bridal O, et al. Functional safety of automated driving
systems: Does ISO 26262 meet the challenges? In: Automated Driving: Safer and More
Efficient Future Driving. Springer International Publishing, pp. 387–416.
13. Kramer B, Neurohr C, Büker M, et al. Identification & Quantification of Hazardous
Scenarios for Automated Driving. In: International Symposium on Model-Based Safety
and Assessment. Springer, Cham, 2020, pp. 163–178.
14. Hommes QVE. Assessment of safety standards for automotive electronic control
systems. United States. National Highway Traffic Safety Administration, 2016.
15. ANSI/UL 4600:2020. Standard for Evaluation of Autonomous Products.

31 Accepted Manuscript – Journal of Risk and Reliability – Aug 2021
16. Kelly T, Weaver R. The Goal Structuring Notation-A Safety Argument Notation. In:
Proceedings of the dependable systems and networks 2004 workshop on assurance cases.
2004, p. undefined.
17. The Assurance Case Working Group. Goal Structuring Notation Community Standard
Version 2, http://www.goalstructuringnotation.info/ (2018).
18. Bishop P, Bloomfield R. A Methodology for Safety Case Development. In: Industrial
Perspectives of Safety-critical Systems. Springer London, 1998, pp. 194–203.
19. Bloomfield R, Bishop P, Jones C, et al. The Adelard Safety Case Development Manual.
Adelard, 1998.
20. Wu W, Kelly T. Combining Bayesian belief networks and the goal structuring notation
to support architectural reasoning about safety. In: Lecture Notes in Computer Science
(including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in
Bioinformatics). Springer Verlag, pp. 172–186.
21. Denney E, Pai G, Habli I. Towards measurement of confidence in safety cases. In:
International Symposium on Empirical Software Engineering and Measurement. IEEE
Computer Society, 2011, pp. 380–383.
22. Lin C, Shen W, Drager S. Measure confidence of assurance cases in safety-critical
domains. In: 2018 IEEE/ACM 40th International Conference on Software Engineering:
New Ideas and Emerging Technologies Results (ICSE-NIER) . IEEE, 2018, pp. 13–16.
23. Wang R, Guiochet J, Motet G. Confidence assessment framework for safety arguments.
In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial
Intelligence and Lecture Notes in Bioinformatics). Springer Verlag, 2017, pp. 55–68.
24. Leveson N. The Use of Safety Cases in Certification and Regulation. Massachusetts
Institute of Technology. Engineering Systems Division,
https://dspace.mit.edu/handle/1721.1/102833 (2011).
25. Sujan MA, Habli I, Kelly TP, et al. Should healthcare providers do safety cases? Lessons
from a cross-industry review of safety case practices. Safety Science 2016; 84: 181–189.
26. Leveson N. A new accident model for engineering safer systems. Safety Science 2004;
42: 237–270.
27. Leveson N, Thomas J. STPA Handbook. Cambridge, MA, 2018.
28. Tirado AM, Brown R, Banda OV. Risk and safety management of autonomous systems:
a literature review and initial proposals for the maritime industry,
https://aaltodoc.aalto.fi/handle/123456789/37509 (2019).
29. Zhang J, Kim H, Liu Y, et al. Combining system-theoretic process analysis and
availability assessment: A subsea case study. Proceedings of the Institution of Mechanical
Engineers, Part O: Journal of Risk and Reliability 2019; 233: 520–536.
30. Bjerga T, Aven T, Zio E. Uncertainty treatment in risk analysis of complex systems: The
cases of STAMP and FRAM. Reliability Engineering & System Safety 2016; 203–209.

32 Accepted Manuscript – Journal of Risk and Reliability – Aug 2021
31. Leong C, Kelly T, Alexander R. Incorporating epistemic uncertainty into the safety
assurance of socio-technical systems. In: Workshop on Causal Reasoning for Embedded
and Safety-critical Systems Technologies. York, 2017.
32. Mosleh A. PRA: A Perspective on strengths, current Limitations, and possible
improvements. Nuclear Engineering and Technology 2014; 46: 1–10.
33. Leveson NG. Shortcomings of the Bow Tie and Other Safety Tools Based on Linear
Causality, http://sunnyday.mit.edu/Bow-tie-final.pdf.
34. Leveson N. Engineering a Safer World: Systems Thinking Applied to Safety. The MIT
Press, 2011.
35. John A, Rae A, Alexander R, et al. Fixing the cracks in the crystal ball : A maturity
model for quantitative risk assessment. Reliability Engineering & System Safety 2014; 67–
81.
36. Mcdermid J, Alexander, Jia Y, et al. Towards a Framework for Safety Assurance of
Autonomous Systems. Artificial Intelligence Safety 2019; 1–7.
37. Groth KM, Wang C, Mosleh A. Hybrid causal methodology and software platform for
probabilistic risk assessment and safety monitoring of socio-technical systems. Reliability
Engineering and System Safety 2010; 95: 1276–1285.
38. Roelen A, Wever R, Mosleh A. Development and validation of a comprehensive hybrid
causal model for safety assessment and management of aviation systems. In: Ninth
International Probabilistic Safety Assessment and Management Conference, PSAM. 2008.
39. Mosleh A, Dias A, Eghbali G, et al. An Integrated Framework for Identification,
Classification, and Assessment of Aviation Systems Hazards. In: Probabilistic Safety
Assessment and Management. London: Springer, 2004, pp. 2384–2390.
40. NASA Open Data Portal. Assured Resilience for Autonomous Systems, Phase I,
https://data.nasa.gov/dataset/Assured-Resilience-for-Autonomous-Systems-Phase-I/ukck-
ewfz (accessed September 11, 2020).
41. Stamatelatos M, Dezfuli H, Apostolakis G, et al. Probabilistic risk assessment
procedures guide for NASA managers and practitioners,
https://ntrs.nasa.gov/search.jsp?R=20120001369 (2011, accessed October 31, 2020).
42. Conference CS-JE-NS-FS, 2002 undefined. Probabilistic Risk Assessment for the
International Space Station. adsabs.harvard.edu,
http://adsabs.harvard.edu/full/2002ESASP.486..319S (accessed October 31, 2020).
43. Zhang K, Zhang D, Fan C, et al. Risk analysis of ship foundering using the hybrid causal
logic methodology. In: PSAM 2018 - Probabilistic Safety Assessment and Management.
International Association for Probabilistic Safety Assessment and Management
(IAPSAM), 2018.
44. Wang T, Wu Q, Diaconeasa MA, et al. On the use of the hybrid causal logic
methodology in ship collision risk assessment. Journal of Marine Science and
Engineering 2020; 8: 485.

33 Accepted Manuscript – Journal of Risk and Reliability – Aug 2021
45. Ren D, Zheng W, Wu D. Hybrid causal methodology in quantitative risk assessment for
the on-board ATP of high speed railway. In: 2014 17th IEEE International Conference on
Intelligent Transportation Systems, ITSC 2014. 2014.
46. Diaconeasa M. Integration of Qualitative and Quantitative Hybrid Causal Logic into a
Simulation-based Platform for Probabilistic Risk Assessment of Nuclear Power Plants.
PhD Thesis, UCLA, Los Angeles, CA, USA, 2017.
47. Groth K, Oxstrand J, Mosleh A, et al. A Model-Based Approach to HRA: Example
Application and Quantitative Analysis. In: Proceedings of the International Conference
on Probabilistic Safety Assessment and Management (PSAM 2012). Helsinki, 2012.
48. Røed W, Mosleh A, Vinnem JE, et al. On the use of the hybrid causal logic method in
offshore risk analysis. Reliability Engineering and System Safety 2009; 94: 445–455.
49. Wang YF, Xie M, Roohi SF. Quantitative risk assessment using hybrid causal logic
model. In: International Topical Meeting on Probabilistic Safety Assessment and Analysis
2011, PSA 2011. 2011.
50. Son C. A Study of Safety Culture Assessment Framework for Process Industries and its
Application to a Bayesian Belief Network Analysis. MS Thesis, Texas A&M University,
College Station, TX, USA, 2016.
51. Ramos MA, López Droguett E, Mosleh A, et al. A human reliability analysis
methodology for oil refineries and petrochemical plants operation: Phoenix-PRO
qualitative framework. Reliability Engineering and System Safety 2020; 193: 106672-.
52. Weber P, Medina-Oliva G, Simon C, et al. Overview on Bayesian networks applications
for dependability, risk analysis and maintenance areas. Engineering Applications of
Artificial Intelligence 2012; 25: 671–682.
53. Kabir S, Papadopoulos Y. Applications of Bayesian networks and Petri nets in safety,
reliability, and risk assessments: A review. Safety Science 2019; 115: 154–175.
54. Fenton N, Bieman J. Software metrics: a rigorous and practical approach. 3rd ed. Boca
Raton, FL: CRC Press, 2014.
55. Cai Y, Wu Y, Zhou J, et al. Quantitative software reliability assessment methodology
based on Bayesian belief networks and statistical testing for safety-critical software.
Annals of Nuclear Energy 2020; 145: 107593.
56. Mohaghegh Z, Kazemi R, Mosleh A. Incorporating organizational factors into
Probabilistic Risk Assessment (PRA) of complex socio-technical systems: A hybrid
technique formalization. Reliability Engineering and System Safety 2009; 94: 1000–1018.
57. Pence J, Sakurahara T, Zhu X, et al. Data-theoretic methodology and computational
platform to quantify organizational factors in socio-technical risk analysis. Reliability
Engineering and System Safety 2019; 185: 240–260.
58. Groth KM, Smith R, Moradi R. A hybrid algorithm for developing third generation HRA
methods using simulator data, causal models, and cognitive science. Reliability
Engineering and System Safety 2019; 191: 106507.

34 Accepted Manuscript – Journal of Risk and Reliability – Aug 2021
59. Mkrtchyan L, Podofillini L, Dang VN. Bayesian belief networks for human reliability
analysis: A review of applications and gaps. Reliability Engineering and System Safety
2015; 139: 1–16.
60. Groth KM, Denman MR, Darling MC, et al. Building and using dynamic risk-informed
diagnosis procedures for complex system accidents. Proceedings of the Institution of
Mechanical Engineers, Part O: Journal of Risk and Reliability 2020; 234: 193–207.
61. Chockalingam S, Pieters W, Teixeira A, et al. Bayesian network models in cyber
security: A systematic review. In: Lecture Notes in Computer Science (including subseries
Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Springer
Verlag, pp. 105–122.
62. Diaconeasa MA, Mosleh A, Morozov A, et al. Model-based resilience assessment
framework for autonomous systems. In: ASME International Mechanical Engineering
Congress and Exposition, Proceedings (IMECE). American Society of Mechanical
Engineers (ASME), 2019.
63. Bobbio A, Portinale L, Minichino M, et al. Improving the analysis of dependable
systems by mapping Fault Trees into Bayesian Networks. Reliability Engineering and
System Safety 2001; 71: 249–260.
64. Wang C. Hybrid Causal Logic Methodology for Risk Assessment. PhD Thesis,
University of Maryland, College Park, MD, USA, 2007.
65. Ramos MA, Thieme CA, Utne IB, et al. Human-system concurrent task analysis for
maritime autonomous surface ship operation and safety. Reliability Engineering and
System Safety; 195.
66. Kovaceva J, Bálint A, Schindler R, et al. Safety benefit assessment of autonomous
emergency braking and steering systems for the protection of cyclists and pedestrians
based on a combination of computer simulation and real-world test results. Accident
Analysis and Prevention 2020; 136: 105352.
67. Zhao X, Salako K, Strigini L, et al. Assessing Safety-Critical Systems from Operational
Testing: A Study on Autonomous Vehicles. Information and Software Technology 2020;
128: 106393.
68. Najm W, Smith J, Yanagisawa M. Pre-crash scenario typology for crash avoidance
research. United States. National Highway Traffic Safety Administration, 2007.
69. Swaminathan S, Smidts C. The event sequence diagram framework for dynamic
probabilistic risk assessment. Reliability Engineering and System Safety 1999; 63: 73–90.
70. Swaminathan S, Smidts C. Mathematical formulation for the event sequence diagram
framework. Reliability Engineering and System Safety 1999; 65: 103–118.
71. Luo P, Hu Y. System risk evolution analysis and risk critical event identification based
on event sequence diagram. Reliability Engineering and System Safety 2013; 114: 36–44.
72. Zhou J, Reniers G, Khakzad N. Application of event sequence diagram to evaluate
emergency response actions during fire-induced domino effects. Reliability Engineering
and System Safety 2016; 150: 202–209.

35 Accepted Manuscript – Journal of Risk and Reliability – Aug 2021
73. Ramos MA, Thieme CA, Utne IB, et al. Human-system concurrent task analysis for
maritime autonomous surface ship operation and safety. Reliability Engineering and
System Safety 2020; 195: 106697.
74. Pearl J. Fusion, Propagation, and Structuring in Belief Networks. Artificial Intelligence
1986; 29: 241–288.
75. Pearl J. Causality: Models, Reasoning, and Inference. Cambridge: Cambridge University
Press, 2000.
76. Koller D, Friedman N. Probabilistic Graphical Models: Principles and Techniques. MIT
Press, 2009.