Conference PaperPDF Available

Requirements engineering for autonomous vehicles: a systematic literature review

Authors:

Figures

Content may be subject to copyright.
Requirements Engineering for Autonomous Vehicles: A
Systematic Literature Review
Quelita A. D. S. Ribeiro
Universidade Federal de Pernambuco
Recife, Pernambuco
qadsr@cin.ufpe.br
Moniky Ribeiro
Universidade Federal de Pernambuco
Recife, Pernambuco
smsr@cin.ufpe.br
Jaelson Castro
Universidade Federal de Pernambuco
Recife, Pernambuco
jbc@cin.ufpe.br
ABSTRACT
Context:
Autonomous Vehicles (AVs) will transform the way we
live and work. Several benets are envisaged, including: reduction
in trac deaths, drop in harmful emissions, improvement in fuel
economy, reduction in travel time, and consumer savings. Indeed,
the trend in the automobile industry is the development of AVs
in preparation for the introduction and mass implementation of
driverless vehicles. However, given the complexity and increasing
connectivity of the AV, the challenges for eective and ecient
development are immense. Many problems are related to miscon-
ceptions in the requirements engineering phase for AVs. Hence, a
Requirements Engineering (RE) process is crucial in the develop-
ment of AVs.
Objective:
The purpose of this work is to identify
and analyse the current RE approaches used for AVs development.
The analysis is based on answers related to the type of RE problems
addressed by the study, the RE phases covered by the approach,
requirements modelling styles used, the type of requirements de-
scribed in the study, the specic AVs considered in the study, and
the open problems reported.
Method:
We conducted a Systematic
Literature Review (SLR) as the basis for our work.
Results:
Our
SLR draws on 31 studies in which we identied the RE problems
addressed by the studies and the phases of the RE processes con-
sidered. We also uncovered the languages and description styles
used to describe the requirementes of the AVs. Special attention
was paid to the non-functional requirements of interest. Dierent
types of AVs were identied. Last but not least, several challenges
were revealed.
Conclusions:
This paper reports the current state
of the art of RE for AVs and identies some open issues that deserve
further investigation.
CCS CONCEPTS
Software and its engineering Requirements analysis
;
Applied computing Transportation;
KEYWORDS
Autonomous Vehicles, Requirements Engineering, Systematic Lit-
erature Review.
Permission to make digital or hard copies of all or part of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
for prot or commercial advantage and that copies bear this notice and the full citation
on the rst page. Copyrights for components of this work owned by others than ACM
must be honored. Abstracting with credit is permitted. To copy otherwise, or republish,
to post on servers or to redistribute to lists, requires prior specic permission and/or a
fee. Request permissions from permissions@acm.org.
SAC ’22, April 25–29, 2022, Virtual Event,
©2022 Association for Computing Machinery.
ACM ISBN 978-1-4503-8713-2/22/04. . . $15.00
https://doi.org/https://doi.org/10.1145/3477314.3507004
ACM Reference Format:
Quelita A. D. S. Ribeiro, Moniky Ribeiro, and Jaelson Castro. 2022. Re-
quirements Engineering for Autonomous Vehicles: A Systematic Literature
Review. In The 37th ACM/SIGAPP Symposium on Applied Computing (SAC
’22), April 25–29, 2022, Virtual Event, . ACM, New York, NY, USA, Article 4,
10 pages. https://doi.org/https://doi.org/10.1145/3477314.3507004
1 INTRODUCTION
Autonomous Vehicle (AV) is a car that makes driving decisions
without the intervention of a human. As such, autonomy exists
in many aspects of a car today: cruise control and anti-lock brake
systems are excellent examples of systems that exhibit autonomous
behavior. Additional systems already exist on some models, includ-
ing advanced cruise control, lane maintenance support, lane change
warning and obstacle prevention systems, all of which expand the
range of autonomous behavior [27].
The AV consists of orchestration of hardware components with
complex software implementations and multiple internal networks
connecting intelligent sensor nodes with electronic control units
and actuators. Therefore, many messages are exchanged within mil-
liseconds [
31
]. Such connectivity and cooperation enable vehicles
to connect with other road users and the infrastructure systems
[
32
]. Furthermore, critical data dependencies exist, e.g., compo-
nents that are mandatory for the core functionality of the vehicle
communicate with noncritical components that provide comfort
features for the passengers [31].
According to [
39
] automated driving will change the future’s
private transport and is currently the most discussed and disruptive
technology in the automotive domain. Indeed, AVs are attracting
more interest with each passing day in the industry, as well as the
general public.
Given the long development time for an AV, it may take a con-
siderable time before a prototype becomes available to experiment
with. Hence, it is important to get the requirements right from the
beginning, because it contributes to reduce costs, resources, time,
and eort in the other phases of software development, mainly in
the software development and testing. However, very few studies
have focused on expressing the requirements of AV at a high level.
Moreover, system building is subject to requirements change ac-
cording to the practice tests on the prototype. Therefore, a critical
issue is how to consider those changes and integrate them in the
current specication [36].
It is well known that inappropriate RE practices may result in
incomplete requirements, incorrect elicitation and specication,
high complexity, and economic or human loss. Hence, it is necessary
to investigate how RE is being adapted to deal Autonomous Vehicles
to avoid inadequate or misunderstood requirements. RE for AV is
SAC ’22, April 25–29, 2022, Virtual Event, elita A. D. S. Ribeiro, Moniky Ribeiro, and Jaelson Castro
Table 1: Research questions and motivations
Research Question Description and Motivation
RQ1. What are the problems of the AVs
that were addressed by the studies that
are related to requirements engineer-
ing?
The purpose of this question is to identify problems/situations that are being addressed in the
RE process in AVs by requirements engineers and researchers. Hence, someone developing an
autonomous vehicle and facing a similar problem may benet from the reported results.
RQ2. What phases of the requirements
engineering process have been sup-
ported?
This question helps to identify the phases of the RE Process that are attracting more attention
from the community and the ones that have been neglected. Hence, it indicates the coverage of
the proposed approach as well which phases deserve more attention. It also may justify the
need to develop a more comprehensive RE Process or to extend a given approach to cover some
missing RE phase. The phases considered are in accordance with the Unied Requirements
Engineering Process Maturity Model (Uni-REPM 1).
RQ3. What requirements description
style are supported by the approaches?
Given that there are many ways to describe the requirements (text, goal oriented, scenario-
based, formal language, UML, logical description, etc.) it is important to nd out the styles that
are attracting the most attention, the strengths, and weaknesses of each style in the eld of
autonomous vehicles. Hence, it may help to identify popular requirements styles that are used
by the AV community. Descriptions in dierent styles could be mapped to the most popular
ones, helping to improve the communication among dierent professional teams.
RQ4. What are the non-functional
requirements supported by the ap-
proaches?
Autonomous vehicles must meet several Non-Functional Requirements (NFRs) such as safety,
security, privacy, usability, etc. The purpose of this question is to identify the dierent types of
NFRs that are addressed by the studies. The reader will become aware of the several approaches
proposed to deal with the NFRs.
RQ5. What are the types of autonomous
vehiches that were used in the RE
study/work?
Given that there are many types of autonomous vehicles (such platoon, Cycab, AUTOelfe, Ego,
etc.) with dierent levels of autonomy, it may be worth knowing if the approach proposed by
the study is a generic one or specially tailored for a specic type of autonomous vehicle. It may
be the case that a certain type of AV is more challenging than others.
RQ6. What are the RE challenges related
to autonomous vehicles?
The purpose of this question is to identify the open problems reported by the studies. This
information is useful for identifying gaps in current research and suggesting areas for future
research.
challenging since it has unique properties that make it complex,
expensive and error-prone as compared with other categories, such
as information systems [20, 30].
The rising complexity of automated driving functions makes
it hard to dene all concerning requirements in detail, formulate
the development goal in more than an abstract way, as well as
to estimate the development eort [
39
]. Therefore, requirements
engineering problems in the domain of AVs continue to occur de-
spite the eorts and advances in their understanding. Due to their
properties, dierent approaches, methods, and tools are required
to improve their quality. Some studies provide insights into the
practice of RE for AVs [
10
,
13
,
35
,
39
]. However, to the best of our
knowledge there is no systematic attempt to perform an extensive
identication at all stages of requirements engineering, mapping be-
tween functional and non-functional requirements, and constraints
of requirements approach for AVs.
Autonomous vehicles dier from other systems in that they have
several main concerns and their operation is dynamic, for example
they need to comply with federal and state laws in their region.
In case the car leaves your region, any necessary changes must
be determined and integrated into the system as safety require-
ments, including relevant speed limits, trac signs and other signs.
Therefore, it is necessary to improve security analysis techniques,
ethics, certication, transparency in the process due to a collision
or other situation, traceability for specifying requirements and fast
retrieval of information about the requirements. Furthermore, the
autonomous vehicle is a novelty with several challenges and doubts.
For that, we chose to conduct a systematic review in order to
identify and report research that eectively supports the results of
treatments in a given population, as described in [
24
]. Moreover,
the studies had their quality assessed and were evaluated in details
and the results synthesized.
Hence, this work aims to conduct a Systematic Literature Review
(SLR), following the guidelines of [
22
], to identify and analyze the
RE approaches for AVs. In doing so, we investigate the RE research
problems addressed by them. We want to nd out which RE phases
are attracting more attention to the AVs community. It is also impor-
tant to investigate the requirements modeling styles that are used in
this domain. It is worth discovering whether the RE approaches for
AVs have been deployed to manage functional and non-functional
requirements and which non-functional requirements deserve spe-
cial care.
In this paper, we report the results of the SLR performed to
evaluate and synthesize the evidence available in the literature
to answer research questions (see Table 1) related to the use of
approaches, methods, techniques, and processes to support the RE
in the AVs domain.
The results presented in this systematic review can be bene-
cial to both researchers and practitioners (automotive industry
Requirements Engineering for Autonomous Vehicles: A Systematic Literature Review SAC ’22, April 25–29, 2022, Virtual Event,
team, requirements engineer, safety requirements engineer, secu-
rity specialists, designers, and developers) since it gathers evidence
from the primary studies included in the review, forming a body of
knowledge regarding requirements engineering for AVs to improve
the requirements process and reduce the risk of undetected errors
and deciencies.
The remainder of this paper is organized as follows: Section 2
presents related work. The research method adopted to conduct
the SLR is outlined in Section 3. The results and the analysis of our
research questions are described in Section 4. Finally, conclusions
and future works are shown in Section 5.
2 RELATED WORKS
We did not nd any systematic review related to requirements
engineering targeting autonomous vehicles.
The only systematic review we uncovered was aimed at investi-
gating test coverage criteria for Verication and Validation (V&V)
and ensuring the safety of AVs in software engineering [
40
]. The
authors also portray that one of the biggest obstacles for AVs to
reach public roads is the lack of public condence in the safety
aspect.
We can note that despite the evolution, some features are still
poorly understood in the automotive industry of conventional vehi-
cles. For example, the authors of [
28
] portray the diculty in terms
of usability and the type of information automotive head-up dis-
plays (HUDs) should display in dierent situations to better serve
the driver. For this reason, a systematic review was conducted to
identify the functional requirements of automotive HUDs from the
perspectives of the developer, researcher and user [28].
In contrast to other papers, our goal is to identify and analyse
the current RE approaches used for AVs development. Furthermore,
we identify implications for practice and dene a research agenda
for the RE community with a view to improve AV development.
3 RESEARCH METHOD
In order to perform this SLR, we used guidelines and the proto-
col template proposed by Kitchenham, and Charters [
24
], whose
process involves several activities grouped into three main phases:
planning, conducting, and reporting of the review. It consists of
the following steps: (1) identication of the need for a systematic
review, (2) development of a review protocol, (3) a comprehensive,
exhaustive search for primary studies, (4) quality assessment of in-
cluded studies, (5) data extraction and monitoring, (6) data analysis
and synthesis, and (7) report-writing.
This SLR aims to identify and analyse the current RE approaches
for AVs. The analysis is based on RQs answers related to the type
of RE problems addressed by the study, the RE phases covered
by the approach, requirements modelling styles used, the type of
requirements described in the study, the specic AVs considered in
the RE study, and the open problems reported.
An automatic search was conducted in the following electronic
databases: ACM Digital Library, IEEE Xplore, Science Direct, Scopus,
and Springer.
The search period starts in 1970 and nished in December 21st,
2020. The search was executed in title, keywords, and abstract based
on terms presented in Table 2. We used the descriptive qualitative
research for the qualitative analysis of the data.
In this paper, we want to collect information about requirements
engineering for AVs. Thus, we had focused on terms of the RE area,
autonomous vehicles, and kind of contribution.
Our procedure for selecting studies consisted of six main steps,
as shown in Fig. 1.
3.1 Inclusion and exclusion criteria
We are interested only in primary studies that contribute to require-
ments engineering for AVs in the English language, which satises
a minimum quality threshold.
Secondary studies, short papers (
5 pages), studies that are not
related to research questions, non-peer-reviewed studies, duplicated
studies (only one copy of each study was included), non-English
written papers, studies that do not discuss requirements engineering
in AVs development, grey literature, redundant paper of the same
authorship, and ongoing work were considered exclusion criteria.
Table 2: Terms of the search.
# Related words
(T1) ("requirements engineering" OR "requirements elicitation" OR "require-
ments analysis" OR "requirements specication" OR "requirements man-
agement" OR "requirements validation" OR "requirements verication"
OR "requirements modeling" OR "requirements modelling")
(T2) ("autonomous vehicle" OR "autonomous driving" OR "autonomous fam-
ily vehicle" OR "self-driving car" OR "automated driving" OR "automated
vehicle" OR "intelligent system" OR "cyber-physical vehicle" )
3.2 Quality assessment
In order to avoid bias and maximize internal and external validity,
we performed a quality assessment (QA) of the selected studies. All
studies were assessed against a set of 8 quality criteria (See Table
3). The source of the criteria is also identied, the quality scores for
each criterion of the selected studies are available in Table 4.
We calculate the quality score of the studies using an arithmetic
mean:
𝑆𝑢𝑚𝑂 𝑓 𝑇 ℎ𝑒𝐴𝑛𝑠𝑤 𝑒𝑟𝑠
8
100, where the numerator is the sum of
all answers, and the denominator is the total of questions taken
into account in this quality assessment.
We considered 60% as the minimum score for a study to accepted.
The overall quality of the selected studies is acceptable since the
mean of quality is 72,98%. It is worth noting that all studies were
accepted because their quality score were above 60%.
3.3 Threats to validity
We used the four categories of threats presented by [
44
], which in-
cludes construct, internal, external, and conclusion validity threats.
Construct validity: According to [
44
], bias in the selection of
studies and inaccuracy in data extraction are the main threats to
the validity of systematic reviews. With the aim to avoid these
threats, we followed the guidelines provided by [
22
] to develop a
reliable and auditable research protocol. The protocol was validated
employing inspection and comparison between already published
SLR protocols as well as control papers. The search string of an SLR
SAC ’22, April 25–29, 2022, Virtual Event, elita A. D. S. Ribeiro, Moniky Ribeiro, and Jaelson Castro
Figure 1: Systematic literature review steps. Adapted from [23].
Table 3: Quality assessment criteria.
# Questions Possible answer
1 Is there a rationale provided for why the study was undertaken? [15] Y=1, N=0, P=0.5
2 Is the paper based on research (or is it merely a lessons learned report based
on expert opinion)? [12]
Y=1, N=0
3 Is there a clear statement of the goals of the research? [12] Y=1, N=0, P=0.5
4 Is the proposed approach clearly described? [3] Y=1, N=0, P=0.5
5 The research context was described at an adequate level (industry, laboratory
setting, products used and so on)? [3]
Y=1, N=0, P=0.5
6 Is the study supported by a tool? [11] Y=1, N=0
7 Is there a clear description of the open issues related to the study that was
carried out?
Y=1, N=0, P=0.5
8 Does the research also add value to the industrial community? [3] [12] Y=1, P=0.5
N = Not, P = Partial, Y = Yes
was evaluated several times to avoid the risk of omitting relevant
studies and we used various synonyms of the search string terms to
be able to cover the required papers. We also consider a period that
encompasses all research in AV development, being that from 1970
to the present day. Another important threat was not performing
the snowballing search due to time constraints, which could com-
promise the completeness of the results. Finally, another threat is
that due to resource limitations, not all aspects could be extracted
from the data. For instance, variations of the "requirement" term in
search string could have been used to better serve non-traditional
development processes such as agile development.
Internal validity: We tried to mitigate threats due to personal bias
on study understanding. The three authors are experts in require-
ments engineering. The rst two are PhD students and the third
one is an experienced reseacher, which increases the reliability of
the results obtained. However, during data extraction, subjective
decisions may have occurred since some papers did not provide a
clear description of their objectives and results.
External validity: We are concerned with the generalizability
of the SLR results, which is related to the degree to which the
primary studies are representative of the review topic. To mitigate
external threats, the search was dened after several trial searches
and validated with the consensus of all authors. By choice of our
exclusion criteria, we excluded gray literature papers. The analysis
made by the three authors also improves the external validity by
incrementally improving the quality of the dataset used to draw
general conclusions.
Conclusion validity: The selection process and the inclusion and
exclusion criteria were carefully designed and discussed by authors
to minimize the risk of exclusion of relevant studies.
Requirements Engineering for Autonomous Vehicles: A Systematic Literature Review SAC ’22, April 25–29, 2022, Virtual Event,
Table 4: List of studies and quality scores.
Study Q1. Q2. Q3. Q4. Q5. Q6. Q7. Q8. Total Score Qual %
[2] 1 1 1 1 0.5 1 1 1 0,88 87,50
[10] 1 1 1 0.5 0 0 1 1 0,63 62,50
[13] 1 1 0.5 0.5 1 1 0 1 0,63 62,50
[37] 1 1 1 1 1 1 0 1 0,88 87,50
[39] 1 1 0.5 0.5 0 1 1 1 0,63 62,50
[42] 1 0 1 1 1 0 0 1 0,63 62,50
[21] 1 0 1 1 1 1 0 1 0,75 75,00
[20] 1 1 1 1 1 1 1 1 1,00 100,00
[31] 1 1 1 1 0 1 1 0.5 0,75 75,00
[17] 1 1 1 1 0.5 1 0.5 0.5 0,63 62,50
[26] 1 1 1 1 0 1 0.5 0.5 0,63 62,50
[5] 1 1 1 1 0 1 1 1 0,88 87,50
[30] 1 1 1 1 0 0 0 1 0,63 62,50
[18] 1 1 0.5 1 1 1 1 1 0,88 87,50
[32] 1 1 1 1 0 0 0 1 0,63 62,50
[19] 1 1 1 1 0.5 1 0 1 0,75 75,00
[34] 1 1 1 0.5 0 1 0 1 0,63 62,50
[4] 1 1 1 0 0 0 1 1 0,63 62,50
[1] 1 1 1 1 1 1 1 1 1,00 100,00
[43] 1 1 0 0 0 1 1 1 0,63 62,50
[33] 1 0 1 1 1 0 0 1 0,63 62,50
[35] 1 0 1 1 1 0 0.5 1 0,63 62,50
[14] 1 1 0.5 1 1 1 1 1 0,88 87,50
[25] 1 1 1 1 1 1 0 1 0,88 87,50
[9] 1 1 0 1 1 1 1 1 0,88 87,50
[16] 1 1 0 1 0.5 0 1 1 0,63 62,50
[7] 1 1 0.5 1 0.5 0 1 1 0,63 62,50
[6] 1 1 1 0 0 1 0 1 0,63 62,50
[36] 1 1 1 1 1 0 0.5 1 0,75 75,00
[29] 1 1 1 1 0 1 0.5 1 0,75 75,00
[38] 1 0.5 0.5 1 1 1 1 1 0,75 75,00
4 RESULTS AND ANALYSIS
In this section, we present the results and analysis of the selected
studies. We rst provide general information about the studies such
as the year of publication, the context in which it was developed
and the evaluation method used. Then we answer the research
questions.
4.1 Publication year, context, and research
method.
According to Figure 2 the studies were published between 2009 and
December 2020.
The academic and industrial contexts were considered in this
classication. We classied twenty studies(64.5%) as a result of joint
work between academia and industry. Ten studies (32.3%) involved
only the academia and one study (3.2%) was developed exclusively
at the industry. Hence, the industry is usually cooperating with the
academia in the investigation of RE for AVs.
We classied the research method of the publications according
to [
11
]. The majority of the papers are illustrative scenarios (17
papers, 54.8%), followed by case study (10 papers, 32.3%), not ap-
plicable (3 papers, 9.7%), and controlled experiment (1 paper, 3.2%).
We can observe that research in RE for AVs is focused on small
examples and case studies to evaluate contributions (See Table 5).
Figure 2: Temporal view of the studies.
4.2 RQ1. What are the requirements engineering
problems addressed by the studies?
We reviewed the RE problems considered by the studies. The results
of this question are depicted in Table 6. During this literature review,
it was clear that the studies address RE using specic viewpoints,
SAC ’22, April 25–29, 2022, Virtual Event, elita A. D. S. Ribeiro, Moniky Ribeiro, and Jaelson Castro
Table 5: Evaluation method.
Evaluation Method Freq. %
Illustrative scenario 17 54.8%
Case study 10 32.3%
Not applicable 3 9.7%
Controlled experiment 1 3.2%
and some papers are classied in more than one category. The
authors discuss specic problems, and therefore, our classication
included 7 RE problems.
The most important problem addressed by the RE community is
the complexity of AVs. Indeed, 12 studies (38.70%) mention the need
for a new process, methodology, method, process, or framework
(or improvement of an existing one) that can deal with the multi-
faceted nature of AVs.
We were able to nd few proposals aimed at specic aspects
of RE for AVs. For example, a hybrid-agile process with shorter,
predictable life cycles [
13
]; a proposal of a method to derive safety
goals for automated driving systems [
42
]; an approach for scenario-
based systems engineering for automated driving functions [
39
]; a
rigorous requirements engineering approach that integrates adapta-
tion and tailoring [
36
]; a proposal that provides a process to specify
in a exible way a Cycab requirement model [35].
Nine studies (29.03%) indicate the need to support variability,
maintenance, reuse, traceability, and evolution of requirements.
For example, [
43
] helps the developer to dene system objectives,
requirements and constraints at an early development phase. They
relied on scenario-based and executable behavior models that can
be analyzed automatically; [
2
] supports the analysis and verication
of functional and non-functional requirements; while [
34
] is aimed
at scenario-based testing; [
37
] introduces variability at the early
phase of requirements engineering; and [
1
] proposes an automated
approach to detect undesired feature interactions in self-driving
systems at an early stage.
Another important issue, mentioned in seven studies (22.58%), is
the need to align safety and security. We were able to uncover some
works addressing this issue. For example, [
29
] illustrates how GORE
(Goal-Oriented Requirements Engineering) can cope with the co-
engineering of security and safety properties; while [
7
] describes
a Unied Safety and Security analysis method (US2) which uses a
simple quantication scheme to assess safety hazards and security
threats simultaneously; and [
32
] proposes a holistic approach to
identify and analyze safety and security risk.
There are six (19.35%) studies only concerned with safety re-
quirement. For example, in [
6
] a methodology is described where
formal models play a crucial role both for supporting the percep-
tion of operational situations and for dynamically assessing the
safety risks and planning the behaviors of AVs; and [
26
] proposes a
semi-automatic safety analysis and optimization (SASAO) process.
Several issues are mentioned only by one study. For example,
[
16
] focuses on the elicitation and analysis of Machine Learning
safety requirements and how such requirements should drive the
assurance activities within the data management and model learn-
ing phases. Risk assessment to derive security requirements for
automotive systems is addressed in [
19
]. While [
17
] focused on
the discovery of the user requirements and the specication of
the system requirements that will eliminate the problem of driver
overload.
Table 6: Problems in the RE process in AVs.
#
Problems in the RE process in AVs Studies
1
Avs Complexity
[1] [10] [13]
[14] [18] [30]
[33] [35] [36]
[38] [39] [42]
2
Variability, maintenance, reuse,
traceability or evolution
of requirements
[
1
] [
2
] [
21
]
[
30
] [
34
] [
35
]
[
36
] [
37
] [
43
]
3
Safety and security alignment
[
4
] [
7
] [
20
]
[
25
] [
29
] [
31
]
[32]
4
Safety concerns
[
1
] [
5
] [
6
] [
9
]
[26] [42]
5
Security concerns [19]
6
Machine Learning in the RE [16]
7
Human requirements concerns [17]
4.3 RQ2. What phases of the requirements
engineering process were supported by the
papers?
The phases taken into account to answer this question are the
following: Organizational Support, Requirements Process Manage-
ment, Requirements Elicitation, Requirements Analysis, Release,
Planning, Documentation and Requirements Specication, and Re-
quirements Validation. These phases are the ones proposed in Uni-
REPM
2
(Universal Requirements Engineering Process Maturity
Model). Hence, the process described in each study was carefully
checked to identity its coverage. The results of this question are de-
picted in Table 7. All requirements engineering phases are covered.
The predominant phase that we identied was Documentation
and Requirements Specication (77.4 %), followed by Requirements
Analysis (61.3%), Requirements Elicitation (51.6%), Requirements Val-
idation (41.9%), Requirements Process Management (12.9%),Organi-
zational Support (6.5%), and Release Planning (6.5%). It is important
to note that a study could have addressed more than one phase of
the RE process.
More than 77% of the studies address the Documentation and
Requirements Specication phase. This result is expected because
AVs need to be well documented. In fact, according to RQ3, they
are described in many dierent styes/languages (feature models,
textual requirements, goal-oriented, scenario-based, etc).
2http://www.unirepm.com/publications.php
Requirements Engineering for Autonomous Vehicles: A Systematic Literature Review SAC ’22, April 25–29, 2022, Virtual Event,
Table 7: Phases presents of RE.
Phases/Stages Presents of RE Freq. %
Documentation and Requirements Specication 24 77.4%
Requirements Analysis 19 61.3%
Requirements Elicitation 16 51.6%
Requirements Validation 13 41.9%
Requirements Process Management 4 12.9%
Organizacional Support 2 6.5%
Release Planning 2 6.5%
The Requirements Analysis phase is also representative (61.3%)
because many papers are concerned with the analysis phase. They
examine how the high-level requirements are decomposed in detail,
identifying risks, determining unclear, incomplete, ambiguous, or
contradictory requirements. Mainly analyzing safety requirements.
The percentage of papers addressed in the Elicitation phase is
51.6%. Several studies investigate the issues and challenges of tra-
ditional requirements elicitation techniques when applied to the
context AVs.
The Validation phase is also addressed by a considerable num-
ber of studies (more than 40%). The safety critical nature of AVs
demands special care with the validation of the requirements. There-
fore, the studies are interested in improving the RE process in order
to support tests that are based on model and scenario-based descrip-
tions. Besides that, the review, audit, and inspection of procedures
are needed to avoid AVs accidents.
However, we can identify that little attention is given to the
requirements process management, organizational support, and
release planning phases.
4.4 RQ3. What requirements description style
was supported by the approaches?
In order to guide our classication, we used seven requirements
styles presented in the work of [
11
], except for Description logic
since it was discovered during the classication. The results pre-
sented in Table 8 were dened according to the distribution of the
studies.
Table 8: Style of RE modeling.
Style of RE Modeling Freq. %
Feature models 10 32.3%
Textual requirements 7 22.6%
Goal-oriented 7 22.6%
Scenario-based 6 19.4%
Description logic 2 6.5%
Non-specic 2 6.5%
UML 1 3.2%
It is interesting to note in Table 8 that the number of studies on
a specic style exceeds the total number of studies because one
study can use several requirements description styles.
The predominant style identied was feature models followed
by textual requirements,goal-oriented, and scenario-based. The au-
tomotive sector has a well-planned software development process.
Many models are developed to address specic safety and security
concerns. Mainly because of auditing and certication. Because
of the need to express variability in the requirements, the use of
feature models is quite common.
Many authors considered the use of textual language to describe
the requirements, as it facilitates the communication between stake-
holders. This may be due to multi-disciplinar nature of development
team required to address several concerns of the automotive in-
dustry, such as security, safety, testers, auditors, and certication
experts.
Goal-oriented and scenario-based descriptions are also used by
many studies to dene the requirements of AVs.
The development of AVs requires stakeholders from dierent
expertise. Thus, a tool should be able to organize the specication in
a structured manner to help to cope with the dierent requirements
expressed in dierent requirements languages.
4.5 RQ4. What are the non-functional
requirements supported by the approaches?
We were able to identify the non-functional requirements addressed
by the studies (See Table 9).
Table 9: Type of non-functional requirements.
Type of Requirements (non-functional) Freq. %
Safety 21 67.7%
Security 11 35.5%
None 7 22.6%
Usability 3 9.7%
Privacy 2 6.5%
Law 1 3.2%
Performance 1 3.2%
Reliability 1 3.2%
Testability 1 3.2%
Safety and security requirements were the most cited among the
studies.
Some authors report the use in the AVs development of well-
known safety analysis techniques such as FMEA, FTA, STPA, and
HARA [
6
,
7
,
9
,
29
]. Other researchers proposed new approaches
based on these frameworks and also ISO 26262 [
26
,
32
]. It is inter-
esting to note that some works are integrating security and safety
issues relying on ISO 26262 and SAE J3061 [5, 19, 25].
AVs are also subject to cyber attacks which can expose personal
information on other connected devices. Hence it deserves to be
explored more rigorously. In fact, vehicle security requirements is
already attracting much attention (35.5% of studies).
SAC ’22, April 25–29, 2022, Virtual Event, elita A. D. S. Ribeiro, Moniky Ribeiro, and Jaelson Castro
Some papers did not mention non-functional requirements (22.6%)
because they only directed their research to functional require-
ments.
Three papers cited usability because the authors focused on
the requirements for users concerning automated functions, au-
tonomous taxi, and family vehicle (for children, the elderly, or
people with disabilities).
Other three non-functional requirements (performance, relia-
bility, and testability) were mentioned only by a single study. In
section 4.7 we discuss some challenges related to non-functional
requirements.
4.6 RQ5. What are the types of autonomous
vehicles?
The answer to this question may help to recognize which type of AV
attracts more attention concerning requirements engineering. The
classication was based on the analysis of the vehicles investigated.
Some AV types found in the analyzed papers are dened below:
A platoon consists of some vehicles, it maintains a constant
speed for all vehicles in the platoon, and can safely reduce the
distance between vehicles in the platoon. The leader vehicle
is driven manually by a human driver to ensure conformance
with laws.
Cycab is a public vehicle with automated driving capabilities.
Automated functions are advanced driver assistance systems.
AUTOelfe is an autonomous taxi, a family car that can be
used independently even by those who cannot drive a con-
ventional car.
Level-4 AD (dened by SAE J3016
3
) indicates that the driver
is not required under certain conditions.
Ego is an autonomous vehicle used for tests, experiments or
scenarios.
According to Figure 3, we can see that 11 dierent types were
identied. The most frequent type is Generic (11 papers, 35.5%)
followed by Ego vehicle (5 papers, 16.1%), Platoon (5 papers, 16.1%),
Cycab vehicle (3 papers, 9.7%), and Automated functions, car of sim-
ulation, Wheel loaders, Trucks, AUTOelfe, Intelligent Transportation,
and Level-4: SAE J3016 (1 paper each, 3.2%). We can note that the
majority of the studies are general purpose. However, if a reader
is interested in a specic type of AVs (eg. Platoon) some specially
tailored approaches are available.
4.7 RQ6. What are the RE challenges related to
autonomous vehicles?
Despite the signicant progress reported in the studies, there are
several weaknesses that still need to be addressed.
Evaluation in real settings:
AV is known for its high complex-
ity, the increasing number of functions and the escalating number
of interactions between dierent functions. However, most research
in RE for AVs was evaluated using simple scenarios. Hence, there
is a need to evaluate them in real settings with real users during
complex AVs development. In particular, it is important to uncover
errors or inaccuracies in the requirements as these ndings can
3https://www.sae.org/blog/sae-j3016-update
Figure 3: Autonomous vehicles.
be used to discuss the existing problems with the stakeholders of
autonomous vehicle [43].
Integrate communication for a multidisciplinary team:
The
AV industry must address several concerns, such as security, safety,
privacy, usability, legal requirements. Thus, it often relies on a mul-
tidisciplinary team, consisting of engineers, designers, developers,
testers, auditors, and certication experts. Hence, it may face many
communication obstacles. As shown in Table 8, dierent require-
ments modelling styles are used. The challenge is to integrate these
dierent perspectives of the system. For example, it could be worth-
while to consider the usefulness of scenario descriptions together
with goal models to describe the requirements of AVs [
38
], as well
as the denition a coherent set of extensions to foster goal model-
ing of AVs [
10
] in connection with textual requirements. A future
concern is related to automated generation of scenarios that could
be used in the validation and verication phases [
39
]. Likewise, in
order to facilitate the communication among stakeholder the combi-
nation of natural language and model based requirements could be
considered [
14
]. However, the mapping from the requirements into
the software design could be facilitated if a formal requirements
language was used [36].
Formulation of safety & security requirements:
In relation
to safety concerns, further investigation is required to incorporate
fault injection analysis to add security aspects. It may include formal
safety contracts (assumptions & guarantees) to describe dynamic
behavior to support the analysis of functional insuciencies of
AVs [
26
]. Another challenge is to consider both the formulation of
technical security requirements at the system level, as well as in
hardware and software.
Evaluation of AV safety & security requirements:
A better
understanding and evaluation of AV safety & security requirements
is necessary [
19
]. Current approaches only partially address the
security threats and safety hazards present in the AVs. The reuse
of security & safety solution knowledge may help to reduce the
engineering eort [
20
]. This may be achieved by the creation of
catalogues of threat and failure modes [
32
]. Such information will
improve the understanding of the threats and vulnerabilities of
Requirements Engineering for Autonomous Vehicles: A Systematic Literature Review SAC ’22, April 25–29, 2022, Virtual Event,
existing components and systems and their cyber-physical impli-
cations. It is worth noting that security is also an important con-
sideration for safety of the intended functionality. Therefore the
co-engineering of safety and cybersecurity needs to consider the
potential adversarial impact on the environment of the AV [
5
]. It is
necessary to analyze the consequences when a new vulnerability
or attack vector is identied in the AV domain [
32
]. It also needs to
consider compliance and certication obligations [
29
]. It is likely
that other initiative for RE in safety critical domains, such as [
41
],
could be applied in the domain of AVs.
Use of Machine Learning (ML) for safety requirements:
Another important line of research is the use of Machine Learning
(ML) for safety requirements in the data management and model
learning phases [
16
]. However, it is not clear how this approach
can be extended to consider machine learning verication and
deployment, which are two crucial aspects for a compelling safety
case.
Traceability between product and process requirements:
Given that AVs are safe critical system, if an accident happens and
software is to blame, it is of paramount importance for the manu-
facturer or suppliers to quickly identify the problem and implement
a x that does not create additional problems. But this requires an
eective traceability support system [
21
]. However, with a higher
volume of updates on increasingly complex AVs, traceability is go-
ing to be more dicult to manage and more important to maintain.
So, the challenge is to support the traceability between product and
process requirements from standards to the implemented develop-
ment process [37].
AVs evolution:
Last but not least, we must address the issue
of AVs evolution. AVs are developed as product lines and features
and subsystems reuse common core sets of functionalities. Hence,
it is important to consider the problem of evolution management
from a more holistic perspective where consistency among dierent
heterogeneous models must be veried [30].
5 CONCLUSIONS
In this work we presented the results of a Systematic Literature
Review conducted to investigage how Requirements Engineering
is being applied to Autonomous Vehicles.
Our SLR draws on 31 studies, selected out of 1068 over 50 years,
through a multi-stage process. Thus, it gives us more profound in-
sights into the state-of-the-art content of requirements engineering
for AVs.
We identied the problems addressed by the studies as well as the
phases of the RE processes that are attracting more attention. We
also revealed the languages and description styles used to specify
the requirementes of the AVs. The most frequent non-functional
requirements were also pinpointed, dierrent types of AVs were
recognized. Finally, several challenges were disclosed.
The most relevant ndings from this review and their implica-
tions for further research are as follows:
Requirements engineering phases.
The majority of studies
only partially address RE process phases. Only two studies [
5
]
[
39
] considered all seven phases, indicating that few approaches
consider the whole RE process. However, they are quite limited,
because [
5
] is only focused on Safety & Security co-engineering,
while [39] aims at driving functions.
Lack of a specic requirements engineering process for
AVs.
We did not nd a well-dened, standardized, and known re-
quirements engineering process to guide companies. Hence, there
is a need to investigate and develop a specic requirements engi-
neering process that consider all aspects of AVs requirements.
To improve the specication and analysis of the NFRs.
De-
spite the current contributions to ameliorate the non-functional
requirements problem, several open issues are to be solved. Mainly
because few studies mention transparency, usability, privacy, law,
performance, reliability, testability. Besides, for users to trust and
be willing to buy and use AV, the system must be transparent and
allow users to ask for information, such as why the AV decided
to break instead of turning, informations about accidents, injuries,
and death where the police will want to evaluate the guilt [8].
To consider RE standards.
There are dierent requirements
engineering standards such as IEEE Std 29148:2011, ISO/IEC 15408,
ISO 26262, ISO PAS 21448, ISO/IEC/IEEE 15288, SAE J3061, and
IEEE Std 830:1998. Nevertheless, most RE approaches for AVs are
not compatible with them.
Need for industry evaluation. Despite the emerging interest
of the community to investigate requirements engineering for AVs,
there is still a need to implement the new proposals described in
the studies in real industry projects with real users to improve and
assess the relevance of the research.
Motivated by the results of this SLR, we propose a research
agenda for the RE community that is aimed at the AVs domain:
(1)
How can we develop an RE process tailored for AVs that
address organizational support, process management, elici-
tation, analysis, release planning, documentation and speci-
cation, and validation phases?
(2)
What is the core set of information that requirements engi-
neers should consider in AVs development?
(3)
What are the main requirements engineering patterns and
how they can be used in an RE process for AVs?
(4)
What are the requirements and constraints needed to develop
a tool capable of dealing with integration with other tools
that support all RE phases for AVs?
(5)
How to measure the feasibility of using requirements engi-
neering approaches for AVs?
(6)
What are the primary non-functional requirements, and how
are they related to AVs functions?
(7)
How to improve the maturity level of requirements engi-
neering processes for AVs?
In future work, we intend to create a methodology that encom-
passes the activities included in the RE phases for AV, considering
the safety analysis based on STPA. Requirements description style
is still being discussed, but we want to use a language that is easy for
stakeholders to understand. Finally, we will evaluate the methodol-
ogy in the industry in order to improve the research and receive
feedback from engineers.
6 ACKNOWLEDGMENTS
This work was partially supported by CNPq, CAPES, and FACEPE.
SAC ’22, April 25–29, 2022, Virtual Event, elita A. D. S. Ribeiro, Moniky Ribeiro, and Jaelson Castro
REFERENCES
[1]
Raja Ben Abdessalem, Annibale Panichella, Shiva Nejati, Lionel C Briand, and
Thomas Stifter. 2018. Testing autonomous cars for feature interaction failures
using many-objective search. In 2018 33rd IEEE/ACM International Conference on
Automated Software Engineering (ASE). IEEE, 143–154.
[2]
Dhaminda B Abeywickrama, Nicola Bicocchi, Marco Mamei, and Franco Zam-
bonelli. 2020. The SOTA approach to engineering collective adaptive systems.
International Journal on Software Tools for Technology Transfer 22, 4 (2020), 399–
415.
[3]
Philip Achimugu, Ali Selamat, Roliana Ibrahim, and Mohd Naz’ri Mahrin. 2014.
A systematic literature review of software requirements prioritization research.
Information and Software Technology 56, 6 (2014), 568–585.
[4]
Adina Aniculaesei, Jörg Grieser, Andreas Rausch, Karina Rehfeldt, and Tim
Warnecke. 2018. Toward a holistic software systems engineering approach for
dependable autonomous systems. In 2018 IEEE/ACM 1st International Workshop
on Software Engineering for AI in Autonomous Systems (SEFAIAS). IEEE, 23–30.
[5]
Robert Bramberger, Helmut Martin, Barbara Gallina, and Christoph Schmittner.
2020. Co-engineering of safety and security life cycles for engineering of auto-
motive systems. ACM SIGAda Ada Letters 39, 2 (2020), 41–48.
[6]
DeJiu Chen, Fredrik Asplund, Kenneth Östberg, Eugene Brezhniev, and Vy-
acheslav Kharchenko. 2015. Towards an ontology-based approach to safety
management in cooperative intelligent transportation systems. In International
Conference on Dependability and Complex Systems. Springer, 107–115.
[7]
Jin Cui and Giedre Sabaliauskaite. 2018. US2: An Unied Safety and Security
Analysis Method for Autonomous Vehicles. In Future of Information and Commu-
nication Conference. Springer, 600–611.
[8]
Luiz Marcio Cysneiros, Majid Ra, and Julio Cesar Sampaio do Prado Leite.
2018. Software Transparency as a Key Requirement for Self-driving Cars. In 26th
International Requirements Engineering Conference (RE). IEEE, 382–387.
[9]
Youssef Damak, Marija Jankovic, Yann Leroy, and Bernard Yannou. 2018. Analysis
of safety requirements evolution in the transition of land transportation systems
toward autonomy. In 15th International Design Conference-DESIGN 2018, Vol. 6.
2845–2854.
[10]
Marian Daun, Viktoria Stenkova, Lisa Krajinski, Jennifer Brings, Torsten
Bandyszak, and Thorsten Weyer. 2019. Goal modeling for collaborative groups
of cyber-physical systems with GRL: reections on applicability and limita-
tions based on two studies conducted in industry. In Proceedings of the 34th
ACM/SIGAPP Symposium on Applied Computing. 1600–1609.
[11]
Diego Dermeval, Jéssyka Vilela, Ig Ibert Bittencourt, Jaelson Castro, Seiji Isotani,
Patrick Brito, and Alan Silva. 2015. Applications of ontologies in requirements
engineering: a systematic review of the literature. Requirements Engineering
(2015), 1–33.
[12]
Tore Dybå and Torgeir Dingsøyr. 2008. Empirical studies of agile software
development: A systematic review. Information and software technology 50, 9
(2008), 833–859.
[13]
Otto Emmerich, Huaji Wang, Guillermo Garcia, Ilaria Pezzulla, Alexander Dar-
lington, and Bo Gao. 2020. A Systems Engineering Framework and Application to
an Open Automated Driving Platform. In IECON 2020 The 46th Annual Conference
of the IEEE Industrial Electronics Society. IEEE, 2393–2398.
[14]
Markus Fockel. 2016. ASIL Tailoring on Functional Safety Requirements. In
International Conference on Computer Safety, Reliability, and Security. Springer,
298–310.
[15]
Matthias Galster, Danny Weyns, Dan Tofan, Bjoern Michalik, and Paris Avgeriou.
2014. Variability in software systems—a systematic literature review. Software
Engineering, IEEE Transactions on 40, 3 (2014), 282–306.
[16]
Lydia Gauerhof, Richard Hawkins, Chiara Picardi, Colin Paterson, Yuki Hagiwara,
and Ibrahim Habli. 2020. Assuring the safety of machine learning for pedestrian
detection at crossings. In International Conference on Computer Safety, Reliability,
and Security. Springer, 197–212.
[17]
Andreas Gregoriades, Jack Hadjicosti, Christos Florides, Maria Pampaka, and
Harris Michail. 2015. A driving simulator for discovering requirements in complex
systems. In Proceedings of the Conference on Summer Computer Simulation. 1–10.
[18]
Rong Gu, Eduard Enoiu, and Cristina Seceleanu. 2020. TAMAA: UPPAAL-based
mission planning for autonomous agents. In Proceedings of the 35th Annual ACM
Symposium on Applied Computing. 1624–1633.
[19]
Majul Md Islam, Aljoscha Lautenbach, Christian Sandberg, and Tomas Olovs-
son. 2016. A risk assessment framework for automotive embedded systems. In
Proceedings of the 2nd ACM International Workshop on Cyber-Physical System
Security. 3–14.
[20]
Sergej Japs. 2020. Security & Safety by Model-based Requirements Engineering.
In 2020 IEEE 28th International Requirements Engineering Conference (RE). IEEE,
422–427.
[21]
Henning Jost, Silke Köhler, and Frank Köster. 2011. Towards a safer development
of driver assistance systems by applying requirements-based methods. In 2011
14th International IEEE Conference on Intelligent Transportation Systems (ITSC).
IEEE, 1144–1149.
[22]
Stas Keele. 2007. Guidelines for performing systematic literature reviews in
software engineering. In Technical report, Ver. 2.3 EBSE Technical Report. EBSE.
[23]
Barbara Kitchenham, O Pearl Brereton, David Budgen, Mark Turner, John Bai-
ley, and Stephen Linkman. 2009. Systematic literature reviews in software
engineering–a systematic literature review. Information and software technology
51, 1 (2009), 7–15.
[24]
B. Kitchenham and S Charters. 2007. Guidelines for performing Systematic
Literature Reviews in Software Engineering.
[25]
Georg Macher, Alexander Much, Andreas Riel, Richard Messnarz, and Chris-
tian Kreiner. 2017. Automotive SPICE, safety and cybersecurity integration. In
International Conference on Computer Safety, Reliability, and Security. Springer,
273–285.
[26]
Peter Munk, Andreas Abele, Eike Thaden, Arne Nordmann, Rakshith Amarnath,
Markus Schweizer, and Simon Burton. 2018. Semi-automatic safety analysis and
optimization. In Proceedings of the 55th Annual Design Automation Conference.
1–6.
[27]
Umit Ozguner, Christoph Stiller, and Keith Redmill. 2007. Systems for safety and
autonomous behavior in cars: The DARPA Grand Challenge experience. Proc.
IEEE 95, 2 (2007), 397–412.
[28]
Juhee Park and Woojin Park. 2019. Functional requirements of automotive head-
up displays: A systematic review of literature from 1994 to present. Applied
ergonomics 76 (2019), 130–146.
[29]
Christophe Ponsard, Gautier Dallons, and Philippe Massonet. 2016. Goal-oriented
co-engineering of security and safety requirements in cyber-physical systems. In
International Conference on Computer Safety, Reliability, and Security. Springer,
334–345.
[30]
Sudha Ramesh, Birgit Vogel-Heuser, Wanli Chang, Debayan Roy, Licong Zhang,
and Samarjit Chakraborty. 2017. Specication, verication and design of evolv-
ing automotive software. In Proceedings of the 54th Annual Design Automation
Conference 2017. 1–6.
[31]
Rhea C Rinaldo and Timo F Horeis. 2020. A Hybrid Model for Safety and Security
Assessment of Autonomous Vehicles. In Computer Science in Cars Symposium.
1–10.
[32]
Christoph Schmittner, Zhendong Ma, Erwin Schoitsch, and Thomas Gruber. 2015.
A case study of FMVEA and CHASSIS as safety and security co-analysis method
for automotive cyber-physical systems. In Proceedings of the 1st ACM Workshop
on Cyber-Physical System Security. 69–80.
[33]
Tobias Schräder, Torben Stolte, Inga Jatzkowski, Robert Graubohm, and Markus
Maurer. 2019. An approach for a requirement analysis for an autonomous family
vehicle. In 2019 IEEE Intelligent Vehicles Symposium (IV). IEEE, 1597–1603.
[34]
Barbara Schütt, Thilo Braun, Stefan Otten, and Eric Sax. 2020. SceML: a graph-
ical modeling framework for scenario-based testing of autonomous vehicles.
In Proceedings of the 23rd ACM/IEEE International Conference on Model Driven
Engineering Languages and Systems. 114–120.
[35]
Farida Semmak, Christophe Gnaho, Joêl Brunet, and Régine Laleau. 2009. How
to Adapt the KAOS Method to the Requirements Engineering of Cycab Vehicle..
In ENASE. 87–94.
[36]
Farida Semmak, Christophe Gnaho, and Régine Laleau. 2009. Extended kaos
method to model variability in requirements. In Evaluation of Novel Approaches
to Software Engineering. Springer, 193–205.
[37]
Farida Semmak, Régine Laleau, and Christophe Gnaho. 2009. Supporting variabil-
ity in goal-based requirements. In 2009 Third International Conference on Research
Challenges in Information Science. IEEE, 237–246.
[38]
Irish Singh and Seok-Won Lee. 2017. Self-adaptive requirements for intelligent
transportation system: A case study. In 2017 International Conference on Informa-
tion and Communication Technology Convergence (ICTC). IEEE, 520–526.
[39]
Christoph Sippl, Florian Bock, Christoph Lauer, Aaron Heinz, Thomas Neumayer,
and Reinhard German. 2019. Scenario-based systems engineering: an approach
towards automated driving function development. In 2019 IEEE International
Systems Conference (SysCon). IEEE, 1–8.
[40]
Zaid Tahir and Rob Alexander. 2020. Coverage based testing for V&V and Safety
Assurance of Self-driving Autonomous Vehicles: A Systematic Literature Review.
In 2020 IEEE International Conference On Articial Intelligence Testing (AITest).
IEEE, 23–30.
[41]
Jéssyka Vilela, Jaelson Castro, Luiz Eduardo G Martins, and Tony Gorschek. 2018.
Safety Practices in Requirements Engineering: The UNI-REPM Safety Module.
IEEE Transactions on Software Engineering 46, 3 (2018), 222–250.
[42]
Fredrik Warg, Martin Skoglund, Anders Thorsén, Rolf Johansson, Mattias
Brännström, Magnus Gyllenhammar, and Martin Sanfridson. 2020. The quan-
titative risk norm-a proposed tailoring of HARA for ads. In 2020 50th Annual
IEEE/IFIP International Conference on Dependable Systems and Networks Workshops
(DSN-W). IEEE, 86–93.
[43]
Carsten Wiecher. 2020. A feature-oriented approach: from usage scenarios to
automated system of systems validation in the automotive domain. In Proceed-
ings of the 23rd ACM/IEEE International Conference on Model Driven Engineering
Languages and Systems: Companion Proceedings. 1–6.
[44]
C Wohlin, P Runeson, M Host, MC Ohlsson, B Regnell, and A Wesslen. 2000.
Experimentation in software engineering: an introduction. 2000.
... In the development of Autonomous Vehicles (AV), vehicles can execute a broad range of tasks without human intervention or partial intervention, such as controlling the car's speed and switching lanes [50]. There has yet to be a consensus about the complete autonomy of these vehicles, as some researchers have proposed strategies for controlling their level of automation according to their location (e.g., highway, commercial street, residential street), application concerns (e.g., safety, security, improvement in fuel economy); or even to the driving style (e.g., aggressive, normal, calm). ...
... There has yet to be a consensus about the complete autonomy of these vehicles, as some researchers have proposed strategies for controlling their level of automation according to their location (e.g., highway, commercial street, residential street), application concerns (e.g., safety, security, improvement in fuel economy); or even to the driving style (e.g., aggressive, normal, calm). Ribeiro et al. [50] present a literature review about the requirements involved in the development of AVs, identifying different types of autonomous vehicles with varying levels of autonomy. Based on this literature review, we identified and classified the factors that can make AVs assume different degrees of autonomy. ...
Article
Full-text available
The need to support complex human and machine collaboration has increased because of recent advances in the use of software and artificial intelligence approaches across various application domains. Building applications with more autonomy has grown dramatically as modern system development capability has significantly improved. However, understanding how to assign duties between humans and machines still needs improvement, and there is a need for better approaches to apportion these tasks. Current methods do not make adaptive automation easy, as task assignments during system operation need to take knowledge about the optimal level of automation (LOA) into account during the collaboration. There is currently a lack of explicit knowledge regarding the factors that influence the variability of human-system interaction and the correct LOA. Additionally, models have not been provided to represent the adaptive LOA variation based on these parameters and their interactions and interdependencies. The study, presented in this paper, based on an extensive literature review, identifies and classifies the factors that affect the degree of automation in autonomous systems. It also proposes a model based on feature diagrams representing the factors and their relationships with LOAs. With the support of two illustrative examples, we demonstrate how to apply these factors and how they relate to one another. This work advances research in the design of autonomous systems by offering an adaptive automation approach that can suggest levels of automation to facilitate human-computer interactions.
... While the format for requirements descriptions are still being explored, it is important to choose a language that both developers and stakeholders can readily comprehend, as stated in [20]. The authors also state that there are many ways to describe requirements: NL, goal-oriented, scenario-based, formal language, UML, logical description, and more. ...
Article
Full-text available
This study presents three structures for clearly expressing functional requirements (FRs) and quantitative non-functional requirements (qt-NFRs). Expressing requirements with these structures will allow the understanding of requirements by stakeholders and software developers. The first structure is the SURE format, which is composed of three main sections: a title, a short definition, and a detailed description. The second proposed structure is a template to facilitate the definition of the title and description of unambiguous FRs. It is based on the application of CRUD operations on a certain entity, calling it the “CRUDE” structure. Finally, the third structure serves as a template to make it easier to clearly define the description and title of qt-NFRs. It is based on the application of system properties to computer events or actions, calling it the “PROSE” structure. In this, it is very important to specify those metric values that are desired or expected by the stakeholder. To know how much the definition of FRs and qt-NFRs improved when the proposed structures were used, 46 requirement specification documents elaborated as homework by students of the “Requirement Engineering” course offered at the University of Guayaquil between 2020 and 2022 were evaluated by five experts with more than 10 years of experience in software development for Ecuadorian companies. The findings showed that students reduced the percentage of unambiguous FRs and qt-NFRs from over 80% to about 10%. In conclusion, the findings demonstrate how crucial the three structures proposed in this paper are to helping students develop the ability to clearly express requirements.
... In addition to the above three categories of requirements, safety assurance is imperative for AD [119], [120]. We should consider safety at a fundamental level of importance which underlies and complements the three high-level requirements of trustworthy AI for AD While requirements for trustworthy AI often include safety, safety assurance often places hard constraints on the behaviour of AI systems as opposed to the more high-level criteria of other aspects of trustworthy AI, and so safety should be viewed as a distinct set of requirements. ...
Preprint
Full-text available
Artificial Intelligence (AI) shows promising applications for the perception and planning tasks in autonomous driving (AD) due to its superior performance compared to conventional methods. However, inscrutable AI systems exacerbate the existing challenge of safety assurance of AD. One way to mitigate this challenge is to utilize explainable AI (XAI) techniques. To this end, we present the first comprehensive systematic literature review of explainable methods for safe and trustworthy AD. We begin by analyzing the requirements for AI in the context of AD, focusing on three key aspects: data, model, and agency. We find that XAI is fundamental to meeting these requirements. Based on this, we explain the sources of explanations in AI and describe a taxonomy of XAI. We then identify five key contributions of XAI for safe and trustworthy AI in AD, which are interpretable design, interpretable surrogate models, interpretable monitoring, auxiliary explanations, and interpretable validation. Finally, we propose a modular framework called SafeX to integrate these contributions, enabling explanation delivery to users while simultaneously ensuring the safety of AI models.
... Research has also looked specifically at RE for AD, e.g., providing an overview of AD RE techniques [33]. Riberio et al. identified AD RE challenges addressed by the literature and identified the languages and description styles used to describe AD requirements, with special attention given to NFRs [34]. Heyn et al. investigated challenges with context and ODD definition in ML-enabled perception systems [35], including a lack of standardization for context definitions, ambiguities in deriving ODDs, missing documentation, and lack of involvement of function developers while defining the context. ...
Article
Full-text available
Driving automation systems, including autonomous driving and advanced driver assistance, are an important safety-critical domain. Such systems often incorporate perception systems that use machine learning to analyze the vehicle environment. We explore new or differing topics and challenges experienced by practitioners in this domain, which relate to requirements engineering (RE), quality, and systems and software engineering. We have conducted a semi-structured interview study with 19 participants across five companies and performed thematic analysis of the transcriptions. Practitioners have difficulty specifying upfront requirements and often rely on scenarios and operational design domains (ODDs) as RE artifacts. RE challenges relate to ODD detection and ODD exit detection, realistic scenarios, edge case specification, breaking down requirements, traceability, creating specifications for data and annotations, and quantifying quality requirements. Practitioners consider performance, reliability, robustness, user comfort, and—most importantly—safety as important quality attributes. Quality is assessed using statistical analysis of key metrics, and quality assurance is complicated by the addition of ML, simulation realism, and evolving standards. Systems are developed using a mix of methods, but these methods may not be sufficient for the needs of ML. Data quality methods must be a part of development methods. ML also requires a data-intensive verification and validation process, introducing data, analysis, and simulation challenges. Our findings contribute to understanding RE, safety engineering, and development methodologies for perception systems. This understanding and the collected challenges can drive future research for driving automation and other ML systems.
... Among various AV safety requirements used in the literature [35,44,45], considering the capability of CARLA and the major functionality of our target MLC (i.e., the object detection module), we focus on the following safety requirement: "AV should have a distance no less than from the vehicle in front." To detect safety violations, if any, during the simulation of a complete solution (i.e., the combination of a scenario and an MLC behavior), we measure the distance between the ego vehicle and the vehicle in front for each simulation time step. ...
Preprint
In Machine Learning (ML)-enabled autonomous systems (MLASs), it is essential to identify the hazard boundary of ML Components (MLCs) in the MLAS under analysis. Given that such boundary captures the conditions in terms of MLC behavior and system context that can lead to hazards, it can then be used to, for example, build a safety monitor that can take any predefined fallback mechanisms at runtime when reaching the hazard boundary. However, determining such hazard boundary for an ML component is challenging. This is due to the space combining system contexts (i.e., scenarios) and MLC behaviors (i.e., inputs and outputs) being far too large for exhaustive exploration and even to handle using conventional metaheuristics, such as genetic algorithms. Additionally, the high computational cost of simulations required to determine any MLAS safety violations makes the problem even more challenging. Furthermore, it is unrealistic to consider a region in the problem space deterministically safe or unsafe due to the uncontrollable parameters in simulations and the non-linear behaviors of ML models (e.g., deep neural networks) in the MLAS under analysis. To address the challenges, we propose MLCSHE (ML Component Safety Hazard Envelope), a novel method based on a Cooperative Co-Evolutionary Algorithm (CCEA), which aims to tackle a high-dimensional problem by decomposing it into two lower-dimensional search subproblems. Moreover, we take a probabilistic view of safe and unsafe regions and define a novel fitness function to measure the distance from the probabilistic hazard boundary and thus drive the search effectively. We evaluate the effectiveness and efficiency of MLCSHE on a complex Autonomous Vehicle (AV) case study. Our evaluation results show that MLCSHE is significantly more effective and efficient compared to a standard genetic algorithm and random search.
Article
In Machine Learning (ML)-enabled autonomous systems (MLASs), it is essential to identify the hazard boundary of ML Components (MLCs) in the MLAS under analysis. Given that such boundary captures the conditions in terms of MLC behavior and system context that can lead to hazards, it can then be used to, for example, build a safety monitor that can take any predefined fallback mechanisms at runtime when reaching the hazard boundary. However, determining such hazard boundary for an ML component is challenging. This is due to the problem space combining system contexts (i.e., scenarios) and MLC behaviors (i.e., inputs and outputs) being far too large for exhaustive exploration and even to handle using conventional metaheuristics, such as genetic algorithms. Additionally, the high computational cost of simulations required to determine any MLAS safety violations makes the problem even more challenging. Furthermore, it is unrealistic to consider a region in the problem space deterministically safe or unsafe due to the uncontrollable parameters in simulations and the non-linear behaviors of ML models (e.g., deep neural networks) in the MLAS under analysis. To address the challenges, we propose MLCSHE (ML Component Safety Hazard Envelope), a novel method based on a Cooperative Co-Evolutionary Algorithm (CCEA), which aims to tackle a high-dimensional problem by decomposing it into two lower-dimensional search subproblems. Moreover, we take a probabilistic view of safe and unsafe regions and define a novel fitness function to measure the distance from the probabilistic hazard boundary and thus drive the search effectively. We evaluate the effectiveness and efficiency of MLCSHE on a complex Autonomous Vehicle (AV) case study. Our evaluation results show that MLCSHE is significantly more effective and efficient compared to a standard genetic algorithm and random search.
Chapter
Full-text available
Background: Driving automation systems (DAS), including autonomous driving and advanced driver assistance, are an important safety-critical domain. DAS often incorporate perceptions systems that use machine learning (ML) to analyze the vehicle environment. Aims: We explore new or differing requirements engineering (RE) topics and challenges that practitioners experience in this domain. Method: We have conducted an interview study with 19 participants across five companies and performed thematic analysis. Results: Practitioners have difficulty specifying upfront requirements, and often rely on scenarios and operational design domains (ODDs) as RE artifacts. Challenges relate to ODD detection and ODD exit detection, realistic scenarios, edge case specification, breaking down requirements, traceability, creating specifications for data and annotations, and quantifying quality requirements. Conclusions: Our findings contribute to understanding how RE is practiced for DAS perception systems and the collected challenges can drive future research for DAS and other ML-enabled systems.KeywordsMachine learningRequirements engineeringPerception systemsDriving automation systemsAutonomous driving
Preprint
Full-text available
Background: Driving automation systems (DAS), including autonomous driving and advanced driver assistance, are an important safety-critical domain. DAS often incorporate perceptions systems that use machine learning (ML) to analyze the vehicle environment. Aims: We explore new or differing requirements engineering (RE) topics and challenges that practitioners experience in this domain. Method: We have conducted an interview study with 19 participants across five companies and performed thematic analysis. Results: Practitioners have difficulty specifying upfront requirements, and often rely on scenarios and operational design domains (ODDs) as RE artifacts. Challenges relate to ODD detection and ODD exit detection, realistic scenarios, edge case specification, breaking down requirements, traceability, creating specifications for data and annotations, and quantifying quality requirements. Conclusions: Our findings contribute to understanding how RE is practiced for DAS perception systems and the collected challenges can drive future research for DAS and other ML-enabled systems.
Conference Paper
Cyber-physical systems (CPS), like autonomous vehicles, are intelligent and networked. The development of such systems requires interdisciplinary cooperation between different stakeholders. A lack of system understanding between stakeholders can lead to unidentified security threats & safety hazards in requirements engineering, resulting in high costs in product development. In particular, a lack of an integrative consideration of security threats & safety hazards can compromise safety compliance for CPS. Model-based requirements engineering (MBRE) improves the understanding of systems between stakeholders by additionally creating supporting models to system requirements. However, MBRE approaches only partially address security threats & safety hazards. In particular, their integrative consideration is not taken into account. Established security & safety approaches are either only applicable to specific disciplines or only partially consider security threats & safety hazards. Overall, existing approaches do not fully cover the MBRE process. In the context of this paper, the results of three scientific papers are consolidated with the aim to create a basis for a holistic MBRE approach, which considers security threats & safety hazards integratively. In each of the papers, sub-criteria of the holistic MBRE approach are presented. Furthermore, elaborated and planned tools for the individual process steps are presented.
Conference Paper
New mobility solutions can be characterized as a System of Systems (SoS). SoS characteristics such as emergent system behavior and the operational and managerial independence of the constituent systems pose particular challenges for requirements analysis and system validation. In this paper, we formulate research goals for an automated and model-based analysis of behavior requirements in an automotive SoS context. Based on a problem statement, we propose an integrative requirements analysis and system validation approach. This approach realizes a closed chain from stakeholder requirements to the validation of system behavior, whereby new information is continuously provided via automated checks and short feedback loops. This allows an early and iterative refinement of system objectives and requirements, and reduces uncertainty during SoS development. The current research status is summarized and future work is outlined.
Chapter
Machine Learnt Models (MLMs) are now commonly used in self-driving cars, particularly for tasks such as object detection and classification within the perception pipeline. The failure of such models to perform as intended could lead to hazardous events such as failing to stop for a pedestrian at a crossing. It is therefore crucial that the safety of the MLM can be proactively assured and should be driven by explicit and concrete safety requirements. In our previous work, we defined a process that integrates the development and assurance activities for MLMs within safety-related systems. This is used to incrementally generate the safety argument and evidence. In this paper, we apply the approach to pedestrian detection at crossings and provide an evaluation using the publicly available JAAD data set. In particular, we focus on the elicitation and analysis of ML safety requirements and how such requirements should drive the assurance activities within the data management and model learning phases. We explain the benefits of the approach and identify outstanding challenges in the context of self-driving cars.
Conference Paper
One of the major challenges of automated driving systems (ADS) is showing that they drive safely. Key to ensuring safety is eliciting a complete set of top-level safety requirements (safety goals). This is typically done with an activity called hazard analysis and risk assessment (HARA). In this paper we argue that the HARA of ISO 26262:2018 is not directly suitable for an ADS, both because the number of relevant operational situations may be vast, and because the ability of the ADS to make decisions in order to reduce risks will affect the analysis of exposure and hazards. Instead we propose a tailoring using a quantitative risk norm (QRN) with consequence classes, where each class has a limit for the frequency within which the consequences may occur. Incident types are then defined and assigned to the consequence classes; the requirements prescribing the limits of these incident types are used as safety goals to fulfil in the implementation. The main benefits of the QRN approach are the ability to show completeness of safety goals, and make sure that the safety strategy is not limited by safety goals which are not formulated in a way suitable for an ADS.
Conference Paper
Machine Learnt Models (MLMs) are now commonly used in self-driving cars, particularly for tasks such as object detection and classification within the perception pipeline. The failure of such models to perform as intended could lead to hazardous events such as failing to stop for a pedestrian at a crossing. It is therefore crucial that the safety of the MLM can be proactively assured and should be driven by explicit and concrete safety requirements. In our previous work, we defined a process that integrates the development and assurance activities for MLMs within safety-related systems. This is used to incrementally generate the safety argument and evidence. In this paper, we apply the approach to pedestrian detection at crossings and provide an evaluation using the publicly available JAAD data set. In particular, we focus on the elicitation and analysis of ML safety requirements and how such requirements should drive the assurance activities within the data management and model learning phases. We explain the benefits of the approach and identify outstanding challenges in the context of self-driving cars.
Article
Nowadays systems are becoming more and more connected. Consequently, the co-engineering of (cyber)security and safety life cycles becomes paramount. Currently, no standard provides a structured co-engineering process to facilitate the communication between safety and security engineers. In this paper, we propose a process for co-engineering safety and security by the explicit systematization and management of commonalities and variabilities, implicitly stated in the requirements of the different standards. Our process treats the safety and security life cycles as members of a security-informed safety-oriented process line and so it forces safety and security engineers to come together and brainstorm on what might be considered a commonality and what might be considered a variability. We illustrate the usage of our process by systematizing commonalities and variabilities at risk analysis phase in the context of ISO 26262 and SAE J3061. We then draw lessons learnt. Finally, we sketch some directions for future work.