Content uploaded by Chouki Sentouh
Author content
All content in this area was uploaded by Chouki Sentouh on Jul 18, 2018
Content may be subject to copyright.
Content uploaded by Chouki Sentouh
Author content
All content in this area was uploaded by Chouki Sentouh on Jul 18, 2018
Content may be subject to copyright.
Content uploaded by Boussaad Soualmi
Author content
All content in this area was uploaded by Boussaad Soualmi on Jan 25, 2018
Content may be subject to copyright.
C. Guo et al. / Transportation Research Part F 00 (0000) 000–000 1
Cooperation between Driver and Automated Driving System:
Implementation and Evaluation
Chunshi Guoa,b,c,* , Chouki Sentouhb, Jean-Christophe Popieulb, Jean-Baptiste Hauéa,
Sabine Langloisa,c, Jean-Jacques Loeilletc, Boussaad Soualmic, Thomas Nguyend
a Technocentre Renault, Cognitive Ergonomics & HMI, 1 avenue du Golf, 78280 Guyancourt, France
b University of Valenciennes, LAMIH - CNRS UMR 8201, Mont Houy- F-59313 Valenciennes, France
c Technological Research Institute SystemX, LRA, 8 Avenue de la Vauve, 91120, Palaiseau, France, France
d OKTAL, 19 Boulevard des Nations Unies, 92190, Meudon, France
Abstract
Recent years have witnessed rapid advancement of automated driving technologies. In this context, driver-vehicle cooperation as
a new interaction paradigm offers an opportunity to improve the driving performance through the exploitation of human-
automation synergy. This paper presents the implementation and evaluation of a new cooperation principle between the driver and
the automated driving system. Within a use case concerning highway merging management, we describe the scenario modelling,
system functional and HMI design. We evaluate the cooperation principle and the designed HMI through a user study on driving
simulator. Based on test results, we discuss user’s perception of the cooperation principle and the potential of this principle to
handle highway merging situations. Through this paper, we also aim to demonstrate the strengths of driving simulation in terms of
interaction design for automated driving systems.
Keywords: Automated driving; driver-vehicle cooperation; use case method; HMI; scenario modelling
1. Introduction
1.1. Context and motivation
Considered as a promising solution to improve the safety, efficiency and comfort of our road transport system,
automated driving (AD) technologies receive increasingly research efforts and make continuous advancement.
According to SAE’s taxonomy of levels of vehicle automation (SAE, 2014), AD systems refer to those systems that
are capable of performing all driving tasks without human’s monitoring, covering the highest three levels of
automation (from Level 3 to Level 5). Demonstrations such as autonomous vehicles in the DAPRA Urban Challenge
(Urmson et al., 2009) and the Google car (Markoff, 2010) suggest that it may be technically feasible to develop such
an AD system. In this context, how a future AD system should interact with a human driver constitutes a key question
of research for AD system designers.
For AD systems of SAE Level 3, most human factors studies focus on authority transitions between manual driving
(MD) and AD (Marinik et al., 2014). Particularly, the transition from AD to MD initiated by the automation in so-
called “takeover scenarios” receives a great degree of attention (Blanco et al., 2015; Gold, Damböck, Lorenz, &
Bengler, 2013; Lorenz, Kerschbaum, & Schumann, 2014; Walch, Lange, Baumann, & Weber, 2015), since this
* Corresponding author. Tel.: +33-169080662 .
E-mail address: chunshi.guo@renault.com .
2 C. Guo et al. / Transportation Research Part F 00 (0000) 000–000
transition could cause safety issues if the driver fails to take over vehicle control. However, few works address the
case in which an operating AD system shares the authority with the human driver.
This paper is a result of our studies on driver-vehicle cooperation (referred to as “cooperation” hereafter), a new
interaction paradigm to address the shared authority issue between the AD system and the driver. Through this
paradigm, the driver can cooperate with the operating AD system on driving tasks. Our motivations to propose such
a paradigm are derived from the following two reasons.
Firstly, a human driver may interfere with the AD system in driving tasks. As users of AD systems, human drivers
may have expectations on system’s driving behaviors. Current complex safety concepts (Hörwick & Siedersberger,
2010) and risk-based decision strategies (Vanholme, Gruyer, Lusetti, Glaser, & Mammar, 2013) could make an AD
vehicle behave over conservatively compared with human drivers. A driver that monitors the automation may have
needs to intervene in AD mode, in case that he is not satisfied with the driving behaviors performed by the latter. An
interaction paradigm that enables the AD system to share the authority with the driver is expected to meet such
potential user needs. It is also expected to improve the usability of the system, because the driver need not disengage
the system every time he would like to intervene.
Secondly, we intend to explore synergy between humans and machines. Human driving activities exhibit strong
social patterns, e.g. cues like eye contacts and hand gestures, which are difficult for a machine to interpret (Färber,
2016). Compared with a machine, a human driver can detect these patterns easily and make an adequate decision
following social rules. Therefore, by allowing the human driver to intervene, the AD system can benefit from the help
of the human driver to better handle interactions with other vehicles manually driven by humans.
1.2. Theoretical background
Over the past decades, human-machine cooperation has been gaining interest in the academic community as a
framework to address the shared authority issue within a human-machine system. According to the definition of Hoc
(2001), “cooperation” in the context of human-automation interaction implies that the human and the automation
“interfere with each other on goals, resources, procedures, etc.” However, “each one tries to manage the interference
to facilitate the individual activities and/or the common task”. Among other fields, the concept of human-machine
cooperation has been applied in telerobotics (Sheridan, 1995), air traffic control (Pacaux-Lemoine & Debernard,
2002) and robotic manipulation (Mörtl et al., 2012). In the automotive domain, human-machine cooperation has been
implemented within shared control framework for vehicle control (Abbink & Mulder, 2010; Anderson, Karumanchi,
Iagnemma, & Walker, 2013; Soualmi, Sentouh, Popieul, & Debernard, 2014). Its application for automotive systems
has also been approached from a cognitive perspective (Hoc, Young, & Blosseville, 2009; Flemisch, Bengler, Bubb,
Winner, & Bruder, 2014). Recent European projects like HAVEit (Hoeger et al., 2008) and ABV (Sentouh, Popieul,
Debernard, & Boverie, 2014) demonstrated the interest of applying human-machine cooperation for AD systems.
In our previous work on the design of cooperation for AD systems (Guo, Sentouh, Soualmi, Haue, & Popieul,
2015), we adopted a top-down approach. We firstly identified a common hierarchy between a classical functional
architecture for AD systems (Urmson et al., 2009) and the Michon’s model for human driving behavior (Michon,
1985). Then at each level of the hierarchy, we proposed a cooperation principle as a generic interaction method to
manage the interference at this level. A cooperation principle is implemented by commands (from the user to the
system) and information feedback (from the system to the user). Moreover, a cooperation principle takes a user’s
perspective with the objective to make the cooperative system easy to understand and use by a user.
The cooperation principle concerned in this paper is situated at the maneuver level, namely deliberative maneuver
cooperation. This principle originates from the human supervisory control framework (Sheridan, 1992). In this
principle, the cooperation is initiated by the system. The system shows its intended maneuver and plausible
alternatives to the driver (information feedback), while the latter can select an alternative if he does not agree with
the intention of the system (commands). “Deliberative”, a term borrowed from the domain of multi-agent system
(“Deliberative agent,” 2014), refers to the “thinking ahead” ability exhibited by the human and the automation in this
principle. Finally, this principle is consistent with SAE Level 3, i.e., the AD system executes its originally intended
maneuver plan in the absence of driver’s feedback and it holds the authority in the AD mode.
C. Guo et al. / Transportation Research Part F 00 (0000) 000–000 3
1.3. Objective
The first objective of this paper is to present a use case method for cooperation principle implementation. This use
case served as a common basis for driving scenario modeling, system functional and human-machine interface (HMI)
design. The use of driving simulators to support the design of AD systems has already aroused the attention from AD
developers (Boer, Della Penna, Utz, Pedersen, & Sierhuis, 2015). Within this method, we demonstrate how we
implemented the design of system functions and HMI in the driving simulation environment.
As the second objective, we present the evaluation of this cooperation principle through a user study. This study
was oriented to test user experiences, since it was the first time to implement such an interaction principle. Hence, a
main objective of the user study was to investigate how future users will perceive the cooperation principle and HMI.
What’s more, this test allowed us to examine whether cooperation facilitates the interaction of the AD vehicle with a
traffic merging vehicle in this use case.
2. Use case method
Use cases are widely employed in both object-oriented software engineering and HMI design. In this paper, use
cases correspond to driving situations. A main advantage of relating use cases to driving situations is that possible
behaviors of an actor (the driver or the automation) can be predicted from the knowledge of a few characteristics of
the outer environment (the driving situation). Accordingly, the potential user needs and interferences between the
driver and the automation can be identified.
2.1. Use case definition
Strong interaction exists among road vehicles at highway entry sections. Constrained by the end of acceleration
lane, on-ramp vehicles have to merge into the mainline. However, they should give way to those vehicles already on
the mainline according to traffic regulations. This special configuration at highway entry sections leads to different
interaction patterns between merging and mainline vehicles. A merging vehicle can filter in by forcing a mainline lag
vehicle to decelerate. In an inverse case, a mainline vehicle may voluntarily decelerate to facilitate the merge. In
congestions, it can be observed in some countries that vehicles alternate between passing and yielding near the lane
closure area in a “zipper” fashion (Cassidy & Ahn, 2005).
These complicated interactions among vehicles motivate us to select highway merging management as one of use
cases for cooperation design. In this use case, the AD ego vehicle encounters a merging vehicle at a highway entry
section, as shown in Fig. 1. Multiple possible interaction patterns with the merging vehicle could result in the
interference of the driver and the automation on maneuver plan (e.g., pass, yield or lane change), thus creating
potential user needs to intervene.
Fig. 1. A typical driving scene at a highway entry section.
Once the use case had been selected, a catalog of merging situations was elaborated for scenario modelling. In this
catalog, possible merging situations were classified into nine groups according to the following two criteria:
Macroscopic criterion: mainline traffic density
AD
•Ego vehicle •Lead vehicle
•Merging vehicle
4 C. Guo et al. / Transportation Research Part F 00 (0000) 000–000
Microscopic criterion: merging behaviors
1
For the traffic density criterion, three qualitative levels are set: fluid, dense and congested levels. In addition, three
types of merging behaviors are defined: nominal, hesitant and forced types. Details of behavior description and
modelling will be presented in the next section.
2.2. Scenario modelling tool
Modelling such a highly interactive merging scenario imposes a new challenge: the scenario must be interactive
and reproducible at the same time. “Interactive” implies that the surrounding vehicles should naturally adapt their
behaviors to the ego AD vehicle. But the scenario should be reproducible in the sense that each participant of the user
test can encounter the same kind of situation. For the chosen use case, a target merging vehicle needs to be generated
such that it always meets the ego vehicle with a configurable relative gap across different test runs. In the meanwhile,
its microscopic merging behavior shall be controllable.
To meet this challenge, we developed a generic scenario modelling tool prototype in this study. This tool is based
on the traffic and the scenario modules of the SCANeRTM studio software (That & Casas, 2011) and meets the
following two requirements. First, this tool shall be re-usable for other scenarios or projects. Thus the dependency on
the road network and on the scene has to be minimal. Second, it shall be fully configurable, i.e., every parameter
controlling the behavior of the actors has to be accessible and modifiable from high-level scripts in the scenario
module.
We identified three major functions for the scenario modelling tool: the “meeting control”, the “gap control” and
the “lane change control”. The role of the “meeting control” function is to ensure that the target merging vehicle
always meets the ego AD vehicle under configurable conditions including the meeting location, the gap between the
vehicles and the speed of the target merging vehicle at the meeting location. The “gap control” function” is responsible
for controlling the speed of the target merging vehicle so that it keeps resting in proximity in front of the ego AD
vehicle. In this way, the predefined hesitant behavior can be realized. The last function “lane change control” aims at
controlling the decision of the target merging vehicle to merge behind or in front of the ego AD vehicle. Conditions
for merging decision vary according to the merging behavior. For the nominal merging, only the gap between the
vehicles is considered by the target merging vehicle as a decisional factor. For the hesitant merging, the deceleration
of the ego AD vehicle, showing a cooperative attitude, triggers the decision of merging. At the proximity of the end
of the merging lane, a forced merging decision can be taken even with smaller gaps than the threshold used by nominal
merging decision.
2.3. System functional design
An AD system of SAE L3 was developed in this use case. The system is partitioned into three layers: perception,
maneuver planning and control layers. We took the SCANeRTM studio as the driving simulation platform. We
employed radar models in SCANeRTM studio to detect traffic vehicles and to obtain their positions and speeds. A
longitudinal controller (Guo, Sentouh, Popieul, Soualmi, & Haué, 2015) and a lane-keeping controller (Soualmi et
al., 2014) developed in Simulink were easily integrated in the platform thanks to the APIs of SCANeRTM studio. After
the functions of perception and control had been realized, maneuver planning was the focus of our design.
Maneuver planning is responsible for generating maneuver plans for vehicle controllers on the one hand and for
implementing the cooperation principle on the other hand. To simplify the problem, it was assumed that the system
plans maneuvers only in the longitudinal dimension. This simplification can be justified by the fact that interactions
between vehicles at a merging section are mainly influenced by their longitudinal positions and dynamics.
We modelled the maneuver planning function using a hierarchical finite state machine (HFSM, Kurt & Özgüner,
2013). The HFSM has two first-level states: “intention phase” and “decision phase”, as shown in Fig. 2. The “intention
phase” corresponds to the phase in which the system adapts AD vehicle’s relative position to the merging vehicle
according to its intention. In “intended pass” state, the system attempts to reduce the gap with the merging vehicle by
1
For the reason of simplicity, it is assumed that there is only a merging vehicle on ramp at the current stage.
C. Guo et al. / Transportation Research Part F 00 (0000) 000–000 5
keeping the current speed or accelerating if possible. In “intended yield” state, conversely, the system increases the
gap by decelerating in order to facilitate the merge of the merging vehicle. Between these two states, the system
always enables the one with a lower risk. A “no intention” state is set purposely for congested traffics in which the
intention to pass or yield is contextual and influenced mainly by social rules. Therefore, the system in “no intention”
state keeps its initial car-following task without interacting with the merging vehicle. The “decision phase”
corresponds to the phase in which the system engages an action to terminate the interaction with the merging vehicle.
If the gap with the merging vehicle is sufficiently small that the merge is deemed infeasible, the system engages a
pass maneuver to surpass the merging vehicle. If the merging vehicle is detected to initiate the merge, the system state
transitions to the “engaged yield”. The longitudinal controller then treats the merging vehicle as the new lead vehicle
and establishes a new safe gap with it.
Fig. 2. HFSM-based maneuver planning for highway merging context
The cooperation occurs in the “intention phase”. The principle was implemented by third-level states named
“alternative available” (represented by green circles in Fig. 2) and transitions from them (green arrows). When the
risk of an alternative maneuver is below a threshold, the corresponding “alternative available” sub-state becomes
active. In this case, the driver can trigger the transition to the destination state. As a result, the original alternative
becomes the system’s intended maneuver. For instance, the “intended pass” is active and “yield” is an alternative
maneuver. If the risk of “yield” (deceleration) is smaller than the threshold, the “yield available” sub-state becomes
active and the driver’s input can hence trigger the transition from “intended pass” to “intended yield”. In this principle,
the system assumes the final authority. The fact that the driver asks an available alternative does not mean this
maneuver will be engaged by the system definitively. This kind of situations may occur in highly dynamic situations
where an available maneuver may no more be feasible for the control layer one instant later.
2.4. HMI design
In addition to the interaction logic at the system functional level, HMI is another key factor enabling efficient
cooperation between the driver and the system. The design decisions are formulated as the following three HMI
principles:
1. Showing the driving context. This principle is consistent with the concept of common frame of reference
proposed by Hoc (2001). The driving context serves as a common reference on which the driver and the AD
system share their intentions. This principle was implemented by a representation of merging scene in a
Intended pass No intention
Highway merging context
Intended yield
Intention phase
Engaged yield
Engaged pass
Decision phase
Transition by defaut (risk based)
Transition by cooperation
6 C. Guo et al. / Transportation Research Part F 00 (0000) 000–000
windshield Head-Up Display (HUD, Fig. 3-left) and a yellow semi-transparent rectangle tracking the target
merging vehicle (Fig. 3-upper right). This yellow rectangle is the simulation of augmented reality (AR). The
appearance and disappearance of these HMI elements are consistent with the entry and the exit of the HFSM
“highway merging context”.
2. Showing the intention and available alternative. First, we used triangle to symbolize maneuver, with
triangle(s) forwards for “pass” and backwards for “yield”. Then we designed color codes to distinguish
maneuver states. Three blue triangles represent an “intended/engaged” maneuver, whereas a single green one
represents an “available” alternative. Intention and alternative symbols are shown within the representation of
merging scene in the HUD (Fig. 3-left).
3. Providing a way for the user to choose an alternative. This principle was implemented by two capacitive
backlit buttons (Fig. 3-lower right). The up button means “pass” and the down button “yield”. As long as a
maneuver is available, the corresponding button twinkles in green and remains active. The press on an active
button will trigger a transition to the alternative in the HFSM (Fig. 2). As for an acknowledgement, the pressed
button will become blue (lasting 2s). If the user presses when the green light is off, this button will temporally
become red (2s) to signify a refusal.
Fig. 3 shows a synthesis of the HMI prototypes installed on the driving simulator.
Fig. 3. Designed HMI for deliberative maneuver cooperation principle
3. User study
3.1. Objectives
Through a user study based on the driving simulator, we intended to evaluate the following three aspects of the
proposed cooperation principle:
O1: to evaluate the intuitiveness of the proposed cooperation principle and its HMI;
O2: to assess the user’s performance to cooperate with the system through the HMI;
O3: to assess the effects of cooperation on the interaction between the AD vehicle and the merging vehicle.
3.2. Participants
Twenty-two participants, average age of 41.3 years (ranging from 24 to 61) took part in the experiment. They were
employees of Renault Technocentre and IRT SystemX. They have a driving license of 22.45 years on average and
drove on average 5.8 days/week.
HUD HMI: merging context
The HMI indicates:
•Merging scene representation
•AD intended maneuver:
yield (blue triangles)
•Available alternative: pass
(green triangle)
Command
The green-twinkling button
indicates the availability of
the alternative (green
triangle in the HUD)
Simulated augmented reality
The merging vehicle is highlighted
C. Guo et al. / Transportation Research Part F 00 (0000) 000–000 7
3.3. Test setup
3.3.1. Driving simulator
The driving simulator “Dr SiHMI” of the IRT SystemX (“Dr SiHMI platform,” n.d.) was used in the experiment.
The simulator uses SCANeRTM studio as a simulation platform in which are integrated different modules such as the
AD system, the scenario modelling tool and HMI controllers. The visual system of the simulator is composed of three
projectors and a curved screen that can cover a field of view in horizontal 170° and vertical 40°. The simulator cockpit
is modular and thus facilitates the prototyping and integration of HMI solutions.
3.3.2. Test scenarios
Based on the catalogue of merging situations, we have modelled two baseline test scenarios: merging management
in a fluid mainline (referred to as Fluid) and in a congested mainline (referred to as Congestion). According to the
modal logics in the HFSM, the AD system has its intended maneuver (“intended pass” or “intended yield”) in Fluid,
while in Congestion the AD system is in the “no intention” state. This implies that the subject has an alternative to
choose in Fluid but two alternatives in Congestion. In both scenarios, the AD mode is active by default so that each
subject can totally be disengaged from vehicle control. Fig. 4 illustrates two scans of driving scene in Fluid and Congestion.
Fig. 4. Baseline scenarios: Fluid (left) and Congestion (right)
For this type of scenario in which the ego vehicle needs to interact with other traffic vehicles, it is of interest to
vary behaviors of traffic vehicles. Otherwise, a same merging behavior across all the test runs could not only decrease
the immersion of subjects but also lead to strong memory effects that may bias the assessment of cooperation
performance. As a solution, we used a random variable generator to generate random parameters for the scenario
modelling tool. In this way, based on a baseline scenario (used as a template), different test scenarios can be generated
automatically.
3.4. Procedure and instructions
Prior to the test drive, the subjects were familiar with the AD system and the driving simulator in a training drive.
The test drive was divided into two phases. The objective of the first phase (PH1) was to evaluate the intuitiveness of
the HMI and the cooperation principle. In order to incite subjects to cooperate with the system, we modelled a hesitant
merging vehicle that keeps oscillating ahead. In Fluid, facing this problematic vehicle, the system adopted a
conservative maneuver plan—“yield” while leaving the driver a possibility to change to “pass”. In this phase, each
subject participated in three test runs. Each run consisted of a Fluid and a Congestion scenario. The first run served
as a reference in which none of HMI was activated. Before this run, we explained briefly the functionality of the
system (full automatic control) and asked the subject to observe the driving scene as a new user. In the second run,
we added HMI displays (HUD and the yellow rectangle). We informed the subject of the new added HMI displays
but did not explain their meaning. In the third run, we activated the button command interface in addition to HMI
displays. We simply indicated the driver that he had a new interface allowing him to change the intention of the
system, however, without any instruction on how to use the buttons. As illustrated by the schema in Fig. 5, the second
run was set to assess how the subjects understood the HMI displays whilst the third run was dedicated to evaluate
their comprehension of the cooperation principle. At the end of PH1, the subjects were asked to fill the prepared
questionnaires.
8 C. Guo et al. / Transportation Research Part F 00 (0000) 000–000
The second phase (PH2) aimed to test subject’s performance on cooperation. Therefore, HMI displays and the
buttons were both active. Each subject participated in four sequences of Fluid and four sequences of Congestion in a
random order. While test scenarios were varied across test sequences within subjects, they remained the same as a
whole across subjects
2
. Before the start of PH2, we explained the HMI displays and the correct way to use buttons.
Furthermore, each subject was instructed to use these buttons according to their needs, i.e., they were not obliged to
use them.
Fig. 5. Schema of the procedure in PH1
3.5. Data collection and metrics
Both quantitative and qualitative data were collected. The quantitative data were collected through data log from
the simulator and a questionnaire. To obtain the qualitative data, we employed “thinking aloud” technique and
conducted interviews. Results were principally derived from the quantitative analysis, while qualitative data were
used to give complementary information.
To evaluate how subjects perceived the cooperation principle (O1) and their performance on cooperation (O2), we
proposed two metrics concerning button use. According to the button state when it was pressed, we defined three
types of button pressing:
Pressing Available Alternative maneuver (PAA): a subject pressed the button corresponding to an available
alternative. It characterizes a good use.
Pressing Intended maneuver (PI): a subject pressed the button corresponding to system’s intended maneuver. It
can be interpreted that the subject shared the same intention with the system.
Pressing Pre-Intention (PPI): a subject pressed a button before the system showed its intended maneuver (in Fluid)
or available alternatives (in Congestion).
The first metric of button use was the distribution of button press types which indicates the successful rate (represented
by the ratio of PAA).
The second metric was related to the time on button use. Within each record of a test run, by setting the time when
an alternative became available as the origin (t=0), we computed at which time a subject pressed a button for the first
time. The statistical information on the time of the first button presses was exploited in different ways in PH1 and
PH2. In PH1, this information allowed us to assess whether a subject was aware of the time window of an alternative’s
availability. In PH2, given that subjects had known the cooperation principle through our explanations, the average
time they needed to press the button is a metric of their efficiency on cooperation.
Concerning O3, we queried the data within a time span in which the AD vehicle was interacting with the merging
vehicle. The time span starts from the moment when the merging vehicle entered in a zone proximate to the AD
vehicle until it quitted this zone either by being surpassed by the AD vehicle or by merging into the mainline. This
zone is defined by [-l, 2v] on the acceleration lane, where l is the length of AD car body and the time headway is 2s.
The selected metrics within this time span are shown in Table 1. The interaction duration is characterized by tinter. Due
to the limited range available for the merging vehicle on the acceleration lane, the longer tinter is, the more urgent the
merge of the merging vehicle is and hence the more critical the situation is for the AD vehicle. The speed ratio reflects
2
In such a way, a subject had four different Fluid scenarios and four different Congestion scenarios. To ensure the consistency of the test, the
total eight scenarios remained the same for each subject. This was achieved by using the same random seed for each subject’s test (The MathWorks,
Inc., n.d.).
Run 1
Reference Run 2
+ HMI display Run 3
++ Cooperation
Comprehension of the
HMI displays Comprehension of the
cooperation principle
C. Guo et al. / Transportation Research Part F 00 (0000) 000–000 9
the ability of the AD system to maintain its initial speed during the interaction. At last, the acceleration variation is a
metric for comfort.
Table 1. Metrics used to evaluate the driving performance
Metrics
Description
tinter
Length of the time span in which the AD vehicle was interacting with the merging vehicle
vfin/v0
Ratio of the end speed of the AD vehicle to the initial speed
a
Difference between the maximum and the minimum accelerations of the AD vehicle
4. Results
4.1. Intuitiveness of the HMI and cooperation principle
Subjective evaluation of the HMI intuitiveness was made based on the results of the questionnaire (see Fig. 6).
Among all the HMI elements, the simulated AR, the yellow rectangle tracking the merging vehicle, has received the
highest rating (M = 3.68, SD = 0.57) in terms of the ease of understanding. The representation of merging context in
HUD-HMI was rated as “rather easy” (M = 2.95, SD = 0.90). The rather high SD indicates that the answers were
varied. According to the verbal protocols, several subjects were confused with the meaning of arrows. They
interpreted the blue arrows as actual accelerations of the ego vehicle. Due to the small size of the pictogram limited
by the HUD projection area, 12 subjects out of 22 reported unaware of the green arrow. As for the button command,
while the button function were better understood by subjects (M = 3.36, SD = 0.66), the color code of the button was
worse rated (M = 2.73, SD = 1.03). Owning to the position of the button interface, several subjects did not associate
the twinkling green state of a button with the appearance of the green arrow in the HUD-HMI. Consequently, those
subjects were confused with the meaning of the green color.
Fig. 6. Subjective evaluation of the intuitiveness of the HMI
To assess how users perceived the cooperation principle, we first examined the distribution of the types of button
pressing in PH1. As shown in Fig. 7, PAA accounted for only 21% of the presses in Fluid. It suggests that subjects
did not well understand the logic to change the intention of the system. Rather, the high ratio of PI (50%) reveals that
subjects expected that pressing system’s intention would have effect. Compared to Fluid, the logic in Congestion was
better perceived by the subjects given the 53% for PAA.
Very difficult (1) Rather difficult (2) Rather easy (3) Very easy (4)
Merging
context
Yellow
rectangle
Button
function
Button
color
Was it easy or difficult for you to understand …?
10 C. Guo et al. / Transportation Research Part F 00 (0000) 000–000
Fig. 7. Results on button press distribution in PH1
The distribution of the time of the first button presses in PH1 is represented in form of the histogram in Fig. 8. In
each histogram, the presses falling within the range of [-SD, SD] were counted and shown. The time window of the
availability of the alternative was underlined in green and its initial time served as the origin of time axis. The average
time of the first button press as well as the SD were indicated too. It can be clearly seen that the button presses in
Fluid were more dispersed, whilst the presses in Congestion were more concentrated to the average value. In view of
the one third presses falling out of the time window of the “available pass” in Fluid, it can be inferred that subjects
did not well perceive this time windows in Fluid.
Fig. 8. Histograms of the time of the first button press in PH1with the interval of 1s (left: Fluid; right: Congestion).
4.2. Cooperation performance
The overall performance of the subjects on button use was improved in PH2, given the 62% and the 86% for PAA
in Fluid and Congestion respectively in Fig. 9. However, PI in Fluid still occupied a non-negligible part (27%)
compared with in Congestion. It indicates that several subjects had tendency to confirm the system’s intention, even
though they were informed of the correct button use—to press an available alternative. With a more important part of
PAA, the performance in Congestion was better than in Fluid.
Fig. 9. Results on button press distribution in PH1
21%
10%
50%
19%
Distribution of button press types in PH1
Fluid Congestion
53%
3%
13%
31%
PAA PPI OthersPI
0
1
2
3
4
5
6
7
8
9
10
0 9.35
Average = -0.45
-SD SD=9.06
Available pass
Fluid
Frequency
1
2
3
1
2
3
0
1
2
3
4
5
6
7
8
9
10
-3.2 -2.2 -1.2 -0.2 0.8 1.8 2.8 3.8 4.8 5.8 6.8 7.8 8.8 9.8 10.8 11.8
026.35
Time (s)
Average = 3.86
-SD SD=8.11
Available pass
Congestion
Frequency
4
5
Distribution of button press types in PH2
Fluid Congestion
PAA PPI OthersPI
62%
12%
27%
0%
86%
0% 9%
5%
C. Guo et al. / Transportation Research Part F 00 (0000) 000–000 11
Fig. 10 shows that the distributions of the time of the first button press in both scenarios of PH2 were more
concentrated to the average than in PH1. What’s more, most button presses fell into the time windows of the
alternative’s availability. It means that the subjects perceived better this time window and hence understood better the
cooperation principle. Coincidently, the averages of both two scenarios were equal to 3.87s.
Fig. 10. Histograms of the time of the first button press in PH2 with the interval of 1s (left: Fluid; right: Congestion).
4.3. The effects of cooperation on the interaction with the merging vehicle
Table 2 shows the descriptive statistics of the performance metrics of the AD system in interaction with the merging
vehicle in PH2. The sample of system’s performance without cooperation in either Fluid or Congestion consists of
nine randomly generated scenarios; the sample of system’s performance with cooperation (41 in Fluid and 59 in
Congestion) is made of the test runs with effective cooperation, i.e., with a PAA, performed in these nine scenarios
of Fluid or Congestion. An independent-samples t-test was conducted to compare system’s performance with and
without cooperation (see Table 3). In Fluid, all the performance metrics yielded better results with cooperation than
without cooperation, manifested by shorter interaction time (tinter, p < 0.03), higher speed-keeping ability (vfin/v0, p <
1.19E-11) and smaller acceleration variation (
a, p < 0.04). In Congestion, tinter with cooperation was significantly
reduced (p < 5.56E-04). The ratio of the final speed to the initial speed (vfin/v0) with cooperation was higher, however,
without significant difference (p < 0.06). The average acceleration variation was stronger with cooperation in
Congestion. It can be explained by the fact that the AD vehicle manifested its intention to the merging vehicle by
small amounts of acceleration as a result of the driver’s intervention.
Table 2. Means and standard deviations for the metrics on the performance of the AD system in PH2
Source
Number
Metrics
tinter (s)
vfin/v0 (%)
a (m/s2)
n
M
SD
M
SD
M
SD
Fluid – without cooperation
9
20.87
5.52
69.26
5.01
2.05
0.77
Fluid – cooperation
41
16.04
3.00
98.18
9.04
1.40
0.64
Congestion – without cooperation
9
19.63
7.00
78.70
15.46
0.88
0.29
Congestion – cooperation
59
7.17
4.04
90.47
20.45
1.28
0.29
5. Summary and discussion
We summarize the main results from the user study and give a discussion after each result.
Concerning the HMI, the simulated AR in the driving scene—the yellow rectangle that tracks the merging vehicle
was considered as the one the easiest to understand among all the HMI elements. Besides other factors that could
influence the intuitiveness of an HMI like graphical grammar, we want to highlight the role of the perceptibility,
considering that the HMI is dispatched into three areas in the field of view of the driver in the current configuration.
0
2
4
6
8
10
12
14
16
18
20
0.56 1.56 2.56 3.56 4.56 5.56 6.56 7.56 8.56
010.2
Average = 3.87
-SD SD=4.3
Available pass
Frequency
3
6
9
11
0
2
4
6
8
10
12
14
16
18
20
1.93 2.93 3.93 4.93 5.93 6.93
0 14.54
Time (s)
Average = 3.87
-SD SD=2.93
Available pass
4
8
12
16
20
Frequency
Fluid Congestion
12 C. Guo et al. / Transportation Research Part F 00 (0000) 000–000
According to our observations, most of the subjects monitored the merging vehicle located on the right part of the
scene. The verbal protocol suggested that they tended to infer the system’s intention from the change of the distance
to the merging vehicle. Consequently, the HUD-HMI (in the center of the scene) and the back-lit buttons (near the
gear shift lever) were situated in the peripheral vision of those subjects. The low perceptibility of the HUD-HMI and
the button interface may influence subject’s average level of understanding on the meanings of their contents.
Furthermore, subjects needed to switch attention between the merging vehicle and the button command (to check
whether a button became green). It could increase their cognitive demand. Thus, it may be beneficial for the usability
of HMI to display the essential information in the driving scene, especially in the zone the driver attends to. In our
case, it is of interest to show the system’s intention and the button’s state near the merging vehicle. To achieve this
goal, the AR technology is necessary.
Table 3. T-test for equality of means (independent samples)
Scenario
Metrics
t
df
p*
Mean differences
Fluid
tinter (s)
-2.54
9
0.03
-4.83
vfin/v0 (%)
13.23
21
1.19E-11
28.92
a (m/s2)
-2.38
11
0.04
-0.65
Congestion
tinter (s)
-5.21
9
5.56E-04
-12.46
vfin/v0 (%)
2.03
13
0.06
11.77
a (m/s2)
3.85
11
2.68E-3
0.40
* Two tails with a 5% alpha level
With regard to the cooperation principle, the experiment results indicate that the logic in Congestion (being
possible to choose any of the alternatives) was easier to understand than that in Fluid (being possible to choose an
alternative other than the system’s intention). The average performance of the subjects on button use in PH2 confirms
this conclusion too. Therefore, it seems better to provide a way for the driver just to indicate his intention, regardless
of the intention of the system. In case that the driver and the system share the same intention, the system can take the
input of the driver as a confirmation. This modification of the cooperation principle will be tested in the future works.
As to the temporal aspect of the cooperation performance, the average time needed for the first button press after this
button became available was about 3.9s. This implies that the system needs to offer a time window long enough (at
least longer than 3.9s under the current HMI configuration) so that the driver has enough time to reason and to give
his choice. Of course, time window length depends on the behavior of the merging vehicle, but it can be extended if
the AD system is able to predict the situation’s evolution with a longer look ahead. Therefore, this imposes high
requirements on the situation assessment function of the AD system. On the other hand, how to design an intuitive
HMI to reduce the driver’s reaction time, especially when he is engaging in non-driving tasks, remains another
research question of interest.
The performance metrics of the AD vehicle in interaction with the merging vehicle suggest that the intervention
from the driver could be beneficial to the AD system in terms of managing merging situations. Nevertheless, it is
difficult to generalize the same conclusion to the real world for lack of the knowledge of the validity of these simulated
scenarios. As argued by Boer, Della Penna, Utz, Pedersen and Sierhuis (2015), to evaluate the interaction of the AD
vehicle with other road users constitutes one important usage of driving simulators in the AD vehicle design. To
achieve this goal, it needs to ensure the realism of other road user’s behaviors. One plausible solution is to integrate
state-of-the-art microscopic traffic simulation models into a scenario modelling tool as the one developed in this
study. In this way, one could make traffic vehicles interactable vis-à-vis the ego AD vehicle while guaranteeing the
realism of their behaviors, because those models are already validated by naturalistic data.
C. Guo et al. / Transportation Research Part F 00 (0000) 000–000 13
6. Conclusion
This paper presented the implementation and evaluation of a driver-vehicle cooperation principle. This principle
was implemented within a use case: highway merging management. Based on this use case, we designed a scenario
modelling tool, a maneuver planning function and a set of HMIs. In a user study on driving simulator, we evaluated
the intuitiveness of the cooperation principle, user’s performance on cooperation and the effects of the driver-vehicle
cooperation on AD vehicle’s interaction with the merging vehicle. Test results show the interest of using AR to
enhance the perceptibility of HMI. In addition, we discussed user’s perception of the proposed cooperation principle
and the positive effect of driver-vehicle cooperation on AD vehicle’s performance. We also pointed out some future
directions to improve the cooperation principle and HMI design.
At last, we would like to highlight the strengths of driving simulation in interaction design for AD systems. Without
using driving simulation, it would be quite complicated to prepare the target scenarios in a real environment. A driving
simulation environment offers the flexibility to prototype new types of HMI such as the simulation of AR. To design
new interaction principles for the future system, driving simulation also renders driver-in-loop tests possible at an early
design stage.
Acknowledgements
This work is part of the project “Localization & Augmented Reality”, which is supported by the French
government as part of PIA (French acronym for Program of Future Investments) within the Technological Research
Institute SystemX.
References
Abbink, D. A., & Mulder, M. (2010). Neuromuscular analysis as a guideline in designing shared control. In M. Hosseini (Ed.), Advances in
Haptics. InTech.
Anderson, S. J., Karumanchi, S. B., Iagnemma, K., & Walker, J. M. (2013). The intelligent copilot: A constraint-based approach to shared-
adaptive control of ground vehicles. IEEE Intelligent Transportation Systems Magazine, 5(2), 45–54.
Blanco, M., Atwood, J., Vasquez, H., Trimble, T., Fitchett, V., Radlbeck, J., … Sheldon, R. (2015). Human factors evaluation of Level 2 and
Level 3 automated driving concepts (No. DOT HS 812 182) (p. 320).
Boer, E. R., Della Penna, M., Utz, H., Pedersen, L., & Sierhuis, M. (2015). The role of driving simulators in developing and evaluating
autonomous vehicles. In Proceedings of DSC 2015 Europe Driving Simulation Conference & Exhibition (pp. 3–10). Tübingen, Germany.
Cassidy, M., & Ahn, S. (2005). Driver turn-taking behavior in congested freeway merges. Transportation Research Record: Journal of the
Transportation Research Board, 1934, 140–147.
Deliberative agent. (2014). In Wikipedia. https://en.wikipedia.org/w/index.php?title=Deliberative_agent&oldid=638039836 [02.11.2016]
Dr SiHMI platform. (n.d.). http://www.irt-systemx.fr/en/valorisation/plateformes/plateforme-dr-sihmi/ [18.12.2016]
Färber, B. (2016). Communication and communication problems between autonomous vehicles and human drivers. In M. Maurer, J. C. Gerdes,
B. Lenz, & H. Winner (Eds.), Autonomous Driving (pp. 125–144). Springer Berlin Heidelberg.
Flemisch, F. O., Bengler, K., Bubb, H., Winner, H., & Bruder, R. (2014). Towards cooperative guidance and control of highly automated
vehicles: H-Mode and Conduct-by-Wire. Ergonomics, 57(3), 343–360.
Gold, C., Damböck, D., Lorenz, L., & Bengler, K. (2013). “Take over!” How long does it take to get the driver back into the loop? Proceedings
of the Human Factors and Ergonomics Society Annual Meeting, 57(1), 1938–1942.
Guo, C., Sentouh, C., Popieul, J.-C., Soualmi, B., & Haué, J.-B. (2015). Shared control framework applied for vehicle longitudinal control in
highway merging scenarios. In 2015 IEEE International Conference on Systems, Man, and Cybernetics (SMC) (pp. 3098–3103).
Guo, C., Sentouh, C., Soualmi, B., Haue, J.-B., & Popieul, J.-C. (2015). A hierarchical cooperative control architecture for automated driving
systems. Presented at the 22nd ITS World Congress, Bordeaux, France.
Hoc, J.-M. (2001). Towards a cognitive approach to human–machine cooperation in dynamic situations. International Journal of Human-
Computer Studies, 54(4), 509–540.
Hoc, J.-M., Young, M. S., & Blosseville, J.-M. (2009). Cooperation between drivers and automation: implications for safety. Theoretical Issues
in Ergonomics Science, 10(2), 135–160.
Hoeger, R., Amditis, A., Kunert, M., Hoess, A., Flemish, F., Krueger, H.-P., … Pagle, K. (2008). Highly automated vehicles for intelligent
transport: Have-It approach. Presented at the 15th World Congress on Intelligent Transport Systems and ITS America’s 2008 Annual
Meeting.
Hörwick, M., & Siedersberger, K. H. (2010). Strategy and architecture of a safety concept for fully automatic and autonomous driving assistance
systems. In 2010 IEEE Intelligent Vehicles Symposium (pp. 955–960).
14 C. Guo et al. / Transportation Research Part F 00 (0000) 000–000
Kurt, A., & Özgüner, Ü. (2013). Hierarchical finite state machines for autonomous mobile systems. Control Engineering Practice, 21(2), 184–
194.
Lorenz, L., Kerschbaum, P., & Schumann, J. (2014). Designing take over scenarios for automated driving: How does augmented reality support
the driver to get back into the loop? Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 58(1), 1681–1685.
Marinik, A., Bishop, R., Fitchett, V. L., Morgan, J. F., Trimble, T. E., & Blanco, M. (2014). Human factors evaluation of Level 2 and Level 3
automated driving concepts: Concepts of Operation (Task Report No. DOT HS 812 044). Blacksburg, VA: Virginia Tech Transportation
Institute.
Markoff, J. (2010, October 9). Google cars drive themselves, in traffic. The New York Times.
http://www.nytimes.com/2010/10/10/science/10google.html [24.12.2016]
Michon, J. A. (1985). A critical view of driver behavior models: What do we know, what should we do? In L. Evans & R. C. Schwing (Eds.),
Human Behavior and Traffic Safety (pp. 485–524). Springer US.
Mörtl, A., Lawitzky, M., Kucukyilmaz, A., Sezgin, M., Basdogan, C., & Hirche, S. (2012). The role of roles: Physical cooperation between
humans and robots. The International Journal of Robotics Research, 31(13), 1656–1674.
Pacaux-Lemoine, M. P., & Debernard, S. (2002). Common work space for human–machine cooperation in air traffic control. Control
Engineering Practice, 10(5), 571–576.
SAE. (2014). Taxonomy and definitions for terms related to on-road motor vehicle automated driving systems (Standard No. J3016_201401) (p.
12). SAE International.
Sentouh, C., Popieul, J.-C., Debernard, S., & Boverie, S. (2014). Human-machine interaction in automated vehicle: The ABV Project. IFAC
Proceedings Volumes, 47(3), 6344–6349.
Sheridan, T. B. (1992). Telerobotics, Automation, and Human Supervisory Control. Cambridge, MA, USA: MIT Press.
Sheridan, T. B. (1995). Teleoperation, telerobotics and telepresence: A progress report. Control Engineering Practice, 3(2), 205–214.
Soualmi, B., Sentouh, C., Popieul, J. C., & Debernard, S. (2014). Automation-driver cooperative driving in presence of undetected obstacles.
Control Engineering Practice, 24, 106–119.
That, T. N., & Casas, J. (2011). An integrated framework combining a traffic simulator and a driving simulator. Procedia - Social and
Behavioral Sciences, 20, 648–655.
The MathWorks, Inc. (n.d.). Generate random numbers that are repeatable - MATLAB & Simulink.
https://www.mathworks.com/help/matlab/math/generate-random-numbers-that-are-repeatable.html [24.12.2016]
Urmson, C., Anhalt, J., Bagnell, D., Baker, C., Bittner, R., Clark, M. N., … Ferguson, D. (2009). Autonomous driving in urban environments:
Boss and the Urban Challenge. In M. Buehler, K. Iagnemma, & S. Singh (Eds.), The DARPA Urban Challenge (pp. 1–59). Springer Berlin
Heidelberg.
Vanholme, B., Gruyer, D., Lusetti, B., Glaser, S., & Mammar, S. (2013). Highly automated driving on highways based on legal safety. IEEE
Transactions on Intelligent Transportation Systems, 14(1), pp 333-347.
Walch, M., Lange, K., Baumann, M., & Weber, M. (2015). Autonomous Driving: Investigating the feasibility of car-driver handover assistance.
In Proceedings of the 7th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (pp. 11–18). New
York, NY, USA: ACM.
