Human-Robot Mutual Trust in (Semi)autonomous Underwater Robots

Abstract

It is envisioned that a human operator is able to monitor and control one or more (semi)autonomous underwater robots simultaneously in future marine operations. To enable such operations, a human operator must trust the capability of a robot to perform tasks autonomously, and the robot must establish its trust to the human operator based on human performance and follow guidance accordingly. Therefore, we seek to i model the mutual trust between humans and robots (especially (semi)autonomous underwater robots in this chapter), and ii) develop a set of trust-based algorithms to control the human-robot team so that the mutual trust level can be maintained at a desired level. We propose a time series based mutual trust model that takes into account robot performance, human performance and overall human-robot system fault rates. The robot performance model captures the performance evolution of a robot under autonomous mode and teleoperated mode, respectively. Furthermore, we specialize the robot performance model of a YSI EcoMapper autonomous underwater robot based on its distance to a desired waypoint. The human performance model is inspired by the Yerkes-Dodson law in psychology, which describes the relationship between human arousal and performance. Based on the mutual trust model, we first study a simple case of one human operator controlling a single robot and propose a trust-triggered control strategy depending on the limit conditions of the desired trust region. The method is then enhanced for the case of one human operator controlling a swarm of robots. In this framework, a periodic trust-based control strategy with a highest-trust-first scheduling algorithm is proposed. Matlab simulation results are provided to validate the proposed model and control strategies that guarantee effective real-time scheduling of teleoperated and autonomous controls in both one human one underwater robot case and one human multiple underwater robots case.

Full text

Human-Robot Mutual Trust in (Semi)autonomous
Underwater Robots
Yue Wa ng
1
, Zhenwu Shi
2
, Chuanfeng Wang
3
, and Fumin Zhang
2
1
Department of Mechanical Engineering, Clemson University,
Fluor Daniel Building, Clemson, SC 29634
yue6@clemson.edu
2
School of Electrical and Computer Engineering,
Georgia Institute of Technology, Atlanta, GA 30332
{zwshi,fumin}@gatech.edu
3
George W. Woodruff School of Mechanical Engineering,
Georgia Institute of Technology, Atlanta, GA 30332
cwang329@gatech.edu
Abstract. It is envisioned that a human operator is able to monitor and con-
trol one or more (semi)autonomous underwater robots simultaneously in future
marine operations. To enable such operations, a human operator must trust the
capability of a robot to perform tasks autonomously, and the robot must establish
its trust to the human operator based on human performance and follow guidance
accordingly. Therefore, we seek to i) model the mutual trust between humans and
robots (especially (semi)autonomous underwater robots in this chapter), and ii)
develop a set of trust-based algorithms to control the human-robot team so that
the mutual trust level can be maintained at a desired level. We propose a time se-
ries based mutual trust model that takes into account robot performance, human
performance and overall human-robot system fault rates. The robot performance
model captures the performance evolution of a robot under autonomous mode
and teleoperated mode, respectively. Furthermore, we specialize the robot per-
formance model of a YSI EcoMapper autonomous underwater robot based on its
distance to a desired waypoint. The human performance model is inspired by the
Yerkes-Dodson law in psychology, which describes the relationship between hu-
man arousal and performance. Based on the mutual trust model, we first study a
simple case of one human operator controlling a single robot and propose a trust-
triggered control strategy depending on the limit conditions of the desired trust
region. The method is then enhanced for the case of one human operator control-
ling a swarm of robots. In this framework, a periodic trust-based control strategy
with a highest-trust-first scheduling algorithm is proposed. Matlab simulation re-
sults are provided to validate the proposed model and control strategies that guar-
antee effective real-time scheduling of teleoperated and autonomous controls in
both one human one underwater robot case and one human multiple underwater
robots case.
Keywords: Human-Robot Interaction, Mutual Trust, Real-Time Scheduling, (Semi)
autonomous Underwater Robots.
A. Koubˆaa and A. Khelil (eds.), Cooperative Robots and Sensor Networks 2014, 115
Studies in Computational Intelligence 554,
DOI: 10.1007/978-3-642-55029-4_6,
c
 Springer-Verlag Berlin Heidelberg 2014

116 Y. Wang et al.
1 Introduction
(Semi)autonomous underwater robots have been widely used for military and civil-
ian applications such as seafloor mapping, ocean exploration, fisheries and plankton-
acoustics research, underwater mine detection and etc. However, current approaches
are limited to multiple human operators controlling a single (semi)autonomous under-
water robot. The high level of manpower required to operate such a (semi)autonomous
underwater vehicle inevitably leads to high labor costs as well as human errors.
As the autonomy technology advances, the number of human operators per under-
water robot will be significantly reduced [7] within the very near future. To enable a
single human operator controlling one or multiple robots, it is of particular importance
to build mutual trust between a human operator and underwater robots. On one hand,
the human operator must trust the capability of robots and allow them to operate au-
tonomously under normal conditions. On the other hand, a robot must also build its
trust on a human operator based on the operator’s performance so that the robot can
follow human guidance accordingly. This is especially crucial for the monitoring and
control of underwater robots since underwater communication is more challenging than
that of ground and ariel robots.
In this paper, we propose a novel model for the mutual trust between the hu-
man operator and (semi)autonomous underwater robots. This model is inspired by
the time series trust model proposed in [19, 29]. Different from the unilateral human-
to-robot trust in [19, 29], since we consider the reciprocal trust between human and
robot partners, our model includes both robot performance, human performance, and
overall fault rates of the human-robot systems. The robot performance will be modeled
based on impact of human’s neglect (respectively, intervention) on the robot and evalu-
ated in autonomous (reps. teleoperated) mode. In particular, we obtain the performance
model of a YSI EcoMapper autonomous underwater robot based on its distance to a de-
sired waypoint, which will be used in the simulation section 5. The human performance
model establishes a mathematical description of the Yerkes-Dodson law in psychology,
which is a function of both human workload and task difficulty.
Based on the mutual trust model, trust-based algorithms are developed to guaran-
tee effective real-time scheduling of teleoperated and autonomous control of the un-
derwater robots. To achieve this objective, we first investigate the case of one human
operator controlling a single robot. A trust-triggered control strategy depending on the
limit conditions of the desired trust region is obtained. Further, we consider the case
of one human operator controlling a swarm of robots. To overcome the limitations
of the trust-triggered control strategy for a single robot, the real-time scheduling al-
gorithms [35,39] are adapted for the allocation between teleoperated and autonomous
control mode for multiple robots. A periodic trust-based control strategy with a highest-
trust-first scheduling algorithm is proposed such that the mutual trust can be maintained
at a desired level for each human-robot pair.
The main contributions of the chapter are two fold. First of all, we systematically
model the human robot mutual trust based on a time series model. While the existing
works in the literature have been mainly focused on human-to-robot trust and are mostly
qualitative [17, 24], our model captures the mutual trust between human and robot
partners, which is especially important for underwater robots applications. Second, we

Human-Robot Mutual Trust in (Semi)autonomous Underwater Robots 117
develop trust-based control and real-time scheduling algorithms which guarantee de-
sired trust level for the human-robot team. The mutual trust study is especially relevant
for distributed multi-robot systems or mobile sensing networks where one human oper-
ator is asked to interact with networked robots.
The outline of the chapter is as follows. In Section 2, we introduce the related works
in human robot interaction (HRI), trust in HRI, and related research in supervisory con-
trol theory. In Section 3, we propose a time series mutual trust model as a function of
robot performance, human performance, and fault rate. The robot and human perfor-
mance models are then discussed in detail. Section 4 discusses the control strategies
to drive the human-robot mutual trust to the desired trust region. We first develop a
trust-triggered control strategy based on the mutual trust model for the human operator
to control one single (semi)autonomous underwater vehicles in Section 4.1. We then
extend the case to multi-robot team in Section 4.2. A periodic control strategy with
highest-trust-first scheduling algorithm is developed. Section 5 provides a set of Matlab
simulation results. We conclude the paper in Section 6.
2 Related Works
Research in human-robot interaction (HRI) has grown rapidly in the last two decades.
The seminal book [32] gives a good introduction of much of the early work in human
and computer systems interactions. The book [16] provides a survey on the foundations
and trends of HRI. It gives a unified treatment of HRI-related problems, reviews the
history and formation of the HRI field, identifies key themes and solution approaches,
discusses challenge problems that are likely to shape the field in the near future, and
related fields such as teleoperation, automation science, human-computer interaction,
artificial intelligence and so on. The book [20] gives an overview of HRI in advanced
manufacturing systems with highlights on three broad areas including human factors,
safety, design and implementation. Multi-disciplinary problems in HRI such as opera-
tor cognitive, robot software, industrial robot capabilities, organizational and manage-
ment structures, and safety are discussed. The book [33] deals with human-automation
systems from design to control and performance of both humans and machines. The
special issue on HRI in the IEEE Transactions on Robotics [23] presents a wide range
of HRI related research in robotics including multimodal ways by which humanoids
interact with human beings, the design of a robot head expressing emotions, navigation
algorithms for a mobile humanoid robot, control solutions for humanoid and assistive
robots, human robot interfaces, multiple human-robot team interactions, human subject
testings, and the evaluation for HRI methods. The special issue on HRI in the Inter-
national Journal of Social Robotics [18] presents recent advances in understanding the
expectations, intentions, and actions of both users and robots, and designing appropriate
robot behaviors based on its understanding of the world. Several international confer-
ence proceedings are dedicated to HRI research such as [22] for human-robot personal
relationships, [15] for social robotics, [1] for robot and human interactive communi-
cation, and [21] for general HRI research. HRI research has immersed in almost all
aspects of robotics and automation and is of significant importance to real life applica-
tions. On one hand, robots can extend human capabilities and compensate for human

118 Y. Wang et al.
limitations, especially in some hazardous environments. On the other hand, human has
superior abilities to handle unanticipated and poorly addressed scenarios, and hence
provides good guidance to semiautonomous and autonomous robots. Therefore, the in-
tegration of human and robot capabilities will greatly improve system performance in
demanding tasks.
In particular, the mutual trust between humans and robots is fundamental for effec-
tive HRI in order to fulfill tasks cooperatively and successfully [3, 17]. Humans and
robots act like partners in this shared relationship and their mutual trust affects how
each of the members behaves. The human robot mutual trust is a reciprocal relation-
ship and includes both human-to-robot trust and robot-to-human trust. Human-to-robot
trust involves understanding capabilities of a robot and allowing a robot to perform
certain tasks autonomously. It affects the willingness of the human operator to accept
robot-produced information and thus benefit from the advantages inherent in robotic
systems [17]. The analysis on human-to-robot trust is especially useful for operating
multiple robots simultaneously [3]. The trust level affects the difference between ne-
glect time and activity time of a robot, which has an important impact on multiplexing
human attention among multiple robots. Robot-to-human trust is based on understand-
ing the proficiency of human operator, establishing trust according to this understand-
ing, and following guidance while maintaining a certain level of autonomy. Humans
and robots will either gain or lose trust based on the progress of the task [24]. Over-
trust often leads to misuse of the robot and human resources and hence causes decreas-
ing effectiveness during the execution of a desired task. Contrarily, under-trust will put
limitation on the autonomy of robots and underestimates human’s capability. Hence,
ensuring an appropriate level of human robot mutual trust can be a challenge.
Existing studies on the trust in HRI have mostly been qualitative and descriptive.
More generally, trust in robot is very much related to trust in automation [24,33]. Qual-
itative analysis has been performed in [17] to evaluate the effects of human, robot and
environment factors on perceived trust in HRI. A collaborative performance model is
developed in [14] to capture the critical performance attributes of human-robot trust.
Furthermore, a decision-analytical based measure of trust is developed, which is given
by the ratio of the expected loss to the number of operator overrides.
Despite of the modeling effort, most of the literature has been focused on unilat-
eral human’s trust to robots. However, in a collaborative operation where humans and
robots work together with each other as partners to complete a task efficiently, the trust
between humans and robots should be bilateral and an appropriate level of mutual trust
will eventually yield the best collaborative performance. To model human-robot mutual
trust, the analogy on human-animal trust has been drawn to model human-robot trust
based on experimental data. In [6], two separate notions are employed to determine
human-robot trust by analogous to human-dog trust, i.e., (i) knowing how a partner will
respond and (ii) trusting oneself to interpret a partner’s behavior. The paper [10] argues
that different contexts and perception can be applied to study human-robot relations by
comparisons with human-animal relations. Related research includes mixed initiative
interaction where the key is not only the ability of the human to understand and pre-
dict robot performance, but also the robot’s ability to identify human needs and select
intervention points to assume different levels of initiative [8].

Human-Robot Mutual Trust in (Semi)autonomous Underwater Robots 119
Apart from the above qualitative works, a time series trust model has been pro-
posed to characterize the quantitative dynamic relationship between human-to-robot
trust and corresponding factors. A time-series model of human-to-robot trust and the
human operator’s self-confidence is proposed in [29]. In [19], the relationship between
the changes in human’s control strategy and trust to a semi-automatic pasteurization
plant is investigated using subjective rating scales.
Related works in supervisory control refer to intermittent human interaction with a
remote, automated system in order to manage a controlled process or task [34]. Several
works discuss about the supervisory control for teleoperation of multi-agent coordina-
tion, which involve the control of a lead agentorthecenterofmassoftheformationby
an external human operator [4,13]. In [9], a human supervised robot forage problem is
proposed where the two alternative, forced choice human decision-making task is con-
sidered. The use of human supervisory control for swarming networks was discussed
in [12]. Multi-operator supervisory control has been studied in [27,37].
However, there still lacks systematic modeling of the dynamic evolution of mutual
trust between human and robot partners, especially for the case when a human operator
is required to monitor and control multiple robots at the same time. This then moti-
vates the current chapter. The multi-robot case further raises the problem of real-time
scheduling and allocation of human interaction with each robot, which we will address
in more details in Section 4.
3 Dynamic Model of Mutual Trust
In this section, we first model the mutual trust between the human operator and one
(semi)autonomous robot. Inspired by the time series human-to-robot trust model [19,
25,29], we propose the following dynamic model for human-robot mutual trust
T (t)=AT (t − 1) + B
1
P
R
(t)+B
2
P
R
(t − 1) + C
1
P
H
(t)+C
2
P
H
(t − 1) +
D
1
F (t)+D
2
F (t − 1), (1)
where A, B
i
,C
i
,D
i
,i=1, 2 are real constant coefficients and their values depend on
the tasks to be performed as well as the specific robot and human operator. The cur-
rent mutual trust level T (t) is determined by the previous trust level T (t − 1), current
and previous performance of the robot P
R
(t),P
R
(t − 1), current and previous perfor-
mance of the human operator P
H
(t),P
H
(t − 1), and current and previous fault rate
F (t),F(t − 1) under autonomous or teleoperated mode, respectively. The faults may
include machine malfunction or human error. Let T
d
be the trust level adopted by the
expert human operator, which is gained from experiments. T
u
and T
l
are the upper and
lower limit of the desired trust region such that T
l
<T
d
<T
u
.
3.1 Robot Performance Model
The robot performance model adopted in this paper is inspired by the qualitative analy-
sis in [11]. The model is based on the following assumptions derived from experimental

120 Y. Wang et al.
data: i) the likely performance of a robot will gradually degrade due to accumulated er-
ror when a human operator neglects the robot (as shown in Figure 12(a)), and ii) the
likely performance of a robot will increase as a human operator interacts with the robot
(as shown in Figure 12(b)). Hence, we can assume that a robot working in two modes:
the autonomous mode, and the teleoperated mode, with different performance models
given by the following two difference equations.
P
R
(t)=

(1 − k
R
)P
R
(t − 1) + k
R
P
R,min
, autonomous mode
(1 − k
H
)P
R
(t − 1) + k
H
P
R,max
, teleoperated mode
, (2)
where P
R,max
,P
R,min
∈ [0, 1] stand for the maximum and minimum robot performance,
and k
R
,k
H
∈ (0, 1) is the performance coefficients for autonomous mode and tele-
operated mode, respectively. The robot performance model (2) guarantees that P
R
is
bounded between [P
R,min
,P
R,max
]. We acknowledge that this model may not apply to
all robots and all applications. More sophisticated models can be derived which depend
on specific robots and applications. Nevertheless this model provides a starting point to
study mutual trust between human and robots. Next, we will show that this model can
be used to model the waypoint-based navigation performance of the YSI EcoMapper.
Time-off-task
Robot Performance
(Semi)autonomous
Robot
(a)
Robot Performance
(Semi)autonomous
Robot
Time-on-task
(b)
Fig.1. (a) Impact of human neglect (time-off-task) on the robot performance, and (b) Impact of
human intervention (time-on-task) on the robot performance. This figure is inspired by the case
of semiautonomous robots in [11]. However, in [11], the performance for autonomous robots is
independent of human control.
The EcoMapper, as shown in Figure 2, is a versatile autonomous underwater vehi-
cle (AUV) equipped with water quality sensors for aquatic environmental monitoring
applications. It can operate in autonomous mode or teleoperated mode [30]. A dynam-
ical model of the EcoMapper has recently been obtained though computational fluid
dynamics analysis and experimental efforts [36]. Motion of AUVs like the EcoMapper
are subjected to disturbances from ocean current.
When an AUV navigates autonomously to a waypoint, the error between the desired
and actual position will be driven to zero so that the AUV can arrive at the destination
point. Although the autonomous navigation is more accurate, the AUV usually takes

Human-Robot Mutual Trust in (Semi)autonomous Underwater Robots 121
Fig.2. YSI EcoMapper AUV
more time to adjust its orientation and thus spends more time to reach the destination.
Therefore, human interventions are often necessary.
The dynamics of the EcoMapper can be expressed by the following equations
M
˙
V+C(V)V + D(V)V = τ
˙η = J (η)V (3)
where η =[x, y, z, φ, θ, ψ]
T
represents the AUV position and Euler angles, V =
[u, v, w, p, q, r]
T
is the vector of linear and angular velocities, τ is the vector of ex-
ternal forces and moments. M is the inertia matrix, C(V) is the matrix of Coriolis and
centripetal terms, D(V) is the damping matrix, and J(η) is a transformation matrix
from the body-fixed frame to the earth-fixed frame [36].
For the sake of simplicity, we consider only the planar motion of the EcoMapper.
The first order Euler approximation can be used to obtain the discrete-time dynamics.
Assume that the EcoMapper is initially at some point A and its destination is point B.
Define the distance between the current position at time t and the destination as D(t).
We can use either the autonomous or teleoperated mode to drive the EcoMapper to its
destination point B.
In the teleoperated mode, the human operator can quickly adjust the heading of the
EcoMapper as well as stop the EcoMapper when it is close to the destination point B.
Therefore, the human operator is able to control the EcoMapper so that it goes towards
the destination at full speed u
max
. However, because human’s observation of speed and
distances are not very accurate and that there is no automatic compensation to the exter-
nal disturbances (e.g., wind, water current) applied to the EcoMapper, the actual speed
executed by the EcoMapper may not be constant. Hence, we assume that the actual
speed of the EcoMapper is given by u
max
− δ
1
,whereδ
1
represents a random noise.
In other words, the actual speed of the EcoMapper fluctuates around u
max
,andδ
1
rep-
resents the perturbation applied to the EcoMapper speed. Hence, in this case, we have
D(t)=D(t−1)−(u
max
−δ
1
). When the distance D(t) < 2m, the human operator will
deem that the EcoMapper has reached its destination and does not apply control on it
any more. Consequently, the position of the EcoMapper will only be affected by some
disturbances. Therefore, perturbation is added to the velocity components of the AUV
dynamics to simulate the disturbances and we use δ
2
to denote this perturbation. In
summary, the distance function D(t) under the teleoperated mode is given as follows:
D(t)=

D(t − 1) − (u
max
− δ
1
), if D>2
D(t − 1) + δ
2
, if D ≤ 2

122 Y. Wang et al.
In the autonomous mode as shown in Figure 3, the EcoMapper is controlled by a
self-docking controller which controls the vertical fin angle α(t) and the propeller ro-
tation speed n(t). Assume at some time t, the position of the EcoMapper is [x(t); y(t)]
and the heading angle is Φ(t). Let the desired heading angle at time t be Φ
d
(t)=
arctan
y(t)−y
d
(t)
x(t)−x
d
(t)
. That is, the heading of the EcoMapper needs to be turned by Φ
d
(t) −
Φ(t) . Therefore, the vertical fin angle should be α(t)=k
1
(Φ
d
(t) − Φ(t)),where
k
1
> 0 is a control gain. Because we want to make sure that the EcoMapper stops when
it reaches the destination point B, the propeller rotation speed is set as n(t)=k
2
D(t),
where k
2
> 0 is another control gain.
X
Y
Φ
d
(t)
x(t) x
d
(t)
y
d
(t)
Φ(t)
y(t)
Fig.3. Illustration of the autonomous mode
The path D(t) traveled by an EcoMapper can be computed from dynamics (3). Let
the initial position be [0, 0] and destination be [x
d
; y
d
] = [50m; 0]. For the teleoperated
mode, we set u
max
=2m/s, δ
1
∈ (−0.1, 0.1),andδ
2
∈ (−0.05, 0.05). For the au-
tonomous mode, we set k
1
=0.5 and k
2
=60. Because there are some constraints on
the vertical fin angle and propeller rotation speed, we assume that the limits of the verti-
cal fin angle are 35
◦
and -35
◦
and the limit for the propeller rotation speed is 2000rpm.
Based on the data D(t), we can determine the parameters in the robot performance
model which result in a best match. Under the teleoperated mode, define P
R
(t)=
D
max
−D(t)
D
max
,whereD
max
is the maximum distance traveled by the EcoMapper. We can
determine the robot performance model under the teleoperated mode as represented by
Equation (2) by setting P
R,max
=0.96 and k
H
=0.09. Under the autonomous mode,
define the normalized distance to the destination point B as the robot performance,
i.e., P
R
(t)=
D(t)
D
max
. The robot performance model (2) under the autonomous mode
can be determined by setting k
R
=0.05 and P
R,min
=0.15. Figures 4(a) and 4(b)
plot the actual robot performance and its corresponding model approximation under the
teleoperated and autonomous mode, respectively.
3.2 Human Performance Model
The Yerkes-Dodson law [38] describes human performance as an empirical model with
respect to human arousal, which is always assumed to be proportional to the workload

Human-Robot Mutual Trust in (Semi)autonomous Underwater Robots 123
0 50 100 150
0
0.2
0.4
0.6
0.8
1
t
D
max
−D/D
max
Teleoperated Mode
Actual Robot Performance
Model Approximation
(a)
0 50 100 150
0
0.2
0.4
0.6
0.8
1
t
D/D
max
Autonomous Mode
Actual Robot Performance
Model Approximation
(b)
Fig.4. The YSI EcoMapper performance under (a) the teleoperated mode, and (b) the au-
tonomous mode
[5]. Here human performance means the capability and efficiency of the human operator
to perform a given task. In this paper, we represent the human workload as an utilization
function that is formally defined as follows.
Definition 1. At any time t, the utilization r(t) is defined as the ratio of the total time
when a human operator is performing some task (versus free) within the time interval
[0,t] to the entire time length t.
Based on the above definition, we can obtain the following mathematical model for
human performance
P
H
(t)=

r(t)
β

β

1 − r(t)
1 − β

1−β
, (4)
where β ∈ (0, 1) represents the difficulty of the task for human and a smaller value
of β represents a more difficult task. Figure 5 illustrates the effect of utilization r and
difficulty of the task β on human performance P
H
. Note that the human performance
model (4) guarantees that P
H
is bounded between [0, 1]. We can observe from the in-
verted U-shape curve that the performance of human increases with utilization at the
beginning. However, when the level of utilization r exceeds β, i.e., a moderate level
of workload and stress, the human performance will drop. Furthermore, comparing the
human performance under different task difficulty β, we can observe that the level of
utilization for optimal performance decreases when the difficulty of task increases [28].
For example, for a relatively difficult task with a small value of β, the peak human per-
formance occurs with a low workload and arousal level. However, as the time increases,
the human operator easily gets more stressed with more workload and the correspond-
ing human performance decreases.
According to Definition 1, the total “busy” time, i.e., the workload of a human oper-
ator within [0,t− 1] is (t − 1)r(t − 1). If the robot is in the autonomous mode within

124 Y. Wang et al.
0 0.2 0.4 0.6 0.8 1
0
0.2
0.4
0.6
0.8
1
β
1
=0.2
β
2
=0.5
β
3
=0.8
Task Difficulty
Human Performance
r
P
H
Fig.5. Human Performance
[t − 1,t], the workload of a human operator within [0,t] will remain (t − 1)r(t − 1).
Thus, we have the utilization r(t) in the autonomous mode as
r(t)=
(t − 1)r(t − 1)
t
=(1−
1
t
)r(t − 1). (5)
On the other hand, if the robot is in the teleoperated mode within [t−1,t],theworkload
of a human operator within [0,t] will be (t−1)r(t−1)+1. Thus, we have the utilization
r(t) in the teleoperated mode as
r(t)=
(t − 1)r(t − 1) + 1
t
=(1−
1
t
)r(t − 1) +
1
t
. (6)
According to Equations (5) and (6), the utilization r(t) in both modes is as follows
r(t)=r(t − 1) +
u(t) − r(t − 1)
t
, (7)
u(t)=

1 teleoperated mode
0 autonomous mode
,
where u(t) denotes the control mode of the robot. Note that the utilization r(t) in-
creases in teleoperated mode and decreases in autonomous mode. However, it is always
bounded between the minimum utilization ratio 0 and the maximum utilization ratio 1.
This model is especially useful in the case when one human operator supervises
multiple robots and the workload of the human is given by the summation of all the
control efforts allocated to different robots.
4 Trust-Triggered Control and Real-Time Task Allocation
Strategies
4.1 Single Robot Control Strategy
In this section, we first develop a simple trust-triggered control strategy based on limit
conditions to drive the mutual trust between the human operator and one single robot

Human-Robot Mutual Trust in (Semi)autonomous Underwater Robots 125
to a desired trust region determined by the expert trust level. As shown in Figure 6,
based on the EcoMapper dynamics (3) and the current mode, the robot performance
(2) can be determined. The human performance (4) is based on the current utilization r
and task difficulty β. The mutual trust (1) can be computed from the robot and human
performance in the presence of faults. We design trust-triggered control strategy to dy-
namically switch between the teleoperated mode and autonomous mode based on the
information from the mutual trust.
Teleoperated
Mode
Autonomous
Mode
EcoMapper
Dynamics
Robot
Perfomance
Trust-Triggered Control
Switching
Mutual
Trust
Utilization
Task Difficulty
Human
Perfomance
F
Fig.6. Trust-Triggered Control Strategy
Define a desired trust region determined by the expert trust level T
d
as [T
l
,T
u
].Let
the triggering conditions correspond to the upper limit T
u
and lower limit T
l
of the
trust region [T
l
,T
u
]. When the trust level exceeds the upper limit, i.e., T (t) ≥ T
u
,the
system reaches the ‘over-trust’ situation and teleoperation u(t)=1will be adopted.
On the other hand, when the trust level goes below the lower limit, i.e., T (t) ≤ T
l
,
the system reaches the ‘under-trust’ situation and automatic control u(t)=0will be
adopted. Otherwise, the control scheme at the previous time step will be used. Hence,
the above trust-triggered control strategy can be represented as
u(t)=
⎧
⎨
⎩
1,T(t) ≥ T
u
0,T(t) ≤ T
l
u(t − 1), Otherwise
. (8)
The trust-triggered control strategy in Equation (8) is designed to drive the human-
robot mutual trust to the desired trust region in the presence of faults. Figure 7 shows an
illustrative example of the above trust-triggered control strategy when the desired trust
level is within range of T (t) ∈ [1.5, 2] with a constant desired trust level T
d
=1.75 and
fault rate modeled as a Gaussian white noise with standard deviation 0.004.

126 Y. Wang et al.
0 50 100 150 200 250 300
0
0.5
1
1.5
2
2.5
trust
control
expert
Fig.7. Illustration of trust-triggered control strategy based on limit conditions
Table 1 summarizes the trust-triggered control strategy.
Table 1. Summary of trust-triggered control strategy
Autonomous Mode u(t)=0 Teleoperated Mode u(t)=1
Robot PRFM. P
R
(t) (Eqn. (2)) (1 − k
R
)P
R
(t − 1) + k
R
P
R,min
(1 − k
H
)P
R
(t − 1) + k
H
P
R,max
Human PRFM. P
H
(t) (Eqns (4), (7))
P
H
(t)=

r(t)
β

β

1−r(t)
1−β

1−β
(1 −
1
t
)r(t − 1) (1 −
1
t
)r(t − 1) +
1
t
Mutual Trust T (t) Eqn. (1)
Triggering Condition T (t) ≤ T
l
T (t) ≥ T
u
4.2 Multi-robot Allocation and Real-Time Scheduling
We now extend the above results to the case when a human operator monitors and
controls a team of robots {R
1
, ..., R
N
}.LetR
n
denote an individual robot in the team.
According to the mutual trust dynamic model in Equation (1), the mutual trust level
between the human operator and each robot R
n
can be easily defined as follows
T
n
(t)=A
n
T
n
(t − 1) + B
n,1
P
n,R
(t)+B
n,2
P
n,R
(t − 1) + C
n,1
P
H
(t)+
C
n,2
P
H
(t − 1) + D
n,1
F
n
(t)+D
n,2
F
n
(t − 1) (9)
where the subscript n denotes the index of robots, T
n
(t) the mutual trust between the
human operator and R
n
, P
n,R
(t) the performance of R
n
, P
H
the human performance,
F
n
(t) the fault rate of R
n
,andA
n
, B
n,1
, B
n,2
, C
n,1
, C
n,2
, D
n,1
and D
n,2
are constant
coefficients depending on characteristics of each robot R
n
. Moreover, for each robot
R
n
,weuseT
n,d
to denote the trust level adopted by the expert human operator, T
n,u
denote the upper limit of the desired trust region, and T
n,l
denote the lower limit of
the desired trust region. As disccused in Equation (4), the human perfomrnace P
H
(t)

Human-Robot Mutual Trust in (Semi)autonomous Underwater Robots 127
is determined by the utilization ratio r(t). In the case of multiple robots, the utilization
ratio r(t) depends on the interactions with all robots. Therefore, we extend the definition
of r(t) as follows
r(t)=r(t − 1) +

N
n=1
u
n
(t) − r(t − 1)
t
, (10)
u
n
(t)=

1 teleoperated mode
0 autonomous mode
,
where u
n
(t) denotes the control mode of robot R
n
. As shown in Equation (10), the
utilization ratio r(t) is determined by control modes of all robots. It is assumed that the
human operator can only interact with one robot at a time, therefore, the utilization ratio
r(t) is still bounded between 0 and 1.
The goal of controlling a team of robots is similar to the goal of controlling a single
robot, as introduced in Section 4.1. The human operator must use appropriate control
strategy to achieve the desired mutual trust with each robot in the team. Since the human
operator can only interact with one robot at a time, the trust-triggered control strategy
developed for a single robot in Section 4.1 cannot be directly applied here for the fol-
lowing two reasons. First, in the trust-triggered control strategy, the human operator
will start to interact with the robot once the mutual trust level reaches the upper limit of
the desired trust region. However, for the case of multiple robots, this control strategy
will fail when more than one robot reach their respective upper limits at the same time.
Second, in the trust-triggered control strategy, once the human operator starts to inter-
act with a robot, this interaction will not stop until the mutual trust level falls below the
lower limit of the desired trust region. However, for the case of multiple robots, such
long time of interaction with one robot could waste the capability of human operator,
which should have been spent on other robots that are in more urgent need of inter-
action. To overcome the limitations of the trust-triggered control strategy for a single
robot, the human operator needs a new trust-based control strategy for controlling mul-
tiple robots. This new strategy must satisfy two requirements. First, it should guarantee
that each robot can receive a fair share of interaction with the human operator. Second,
no more than one robot will reach the upper limit of desired trust region at the same
time.
Here we propose a periodic trust-based control strategy. In this new control strategy,
the human operator must interact with each robot R
n
for I
n
amount of time within
every period of L
n
. For each robot R
n
, the choice of L
n
is a fixed value that depends
on the desired trust region [T
n,l
,T
n,u
]. A wider trust region allows for a larger choice
of L
n
. The choice of I
n
is a little bit involved and will dynamically change according
to the mutual trust level within the previous period. Algorithm 1 discusses the detailed
implementation for updating I
n
at the beginning of each period. First, we calculate the
maximum and minimum mutual trust level within the previous period, as shown in Line
1 and Line 2. When the mutual trust goes too high, we increase the amount of interac-
tion time I
n
, as shown in Line 3 and Line 4. Note that the value of I
n
cannot go beyong
L
n
. On the other hand, when the mutual trust goes too low, we decrease the amount
of interaction time I
n
, as shown in Line 5 and Line 6. Note that the value of I
n
can-
not be smaller than zero. Note that 
1
and 
2
are arbitrarily small values guaranteeing

128 Y. Wang et al.
that I
n
is adjusted before the mutual trust level goes beyond upper and lower limit.
Figure 8 shows an example of periodic trust-based control strategy between the human
operator and one robot. As we can see, the mutual trust can always stay within the de-
sired region [1.5, 2]. Moreover, comparing Figure 8 with Figure 7, we can see that the
mutual trust under the periodic trust-based control strategy is closer to the expert level
than trust-triggered control strategy. This is because the human operator under the peri-
odic trust-based control strategy may switch between the autonomous and teleoperated
mode frequently even before the mutual trust reaches the limits of the desired region.
Note that the value of I
n
depends on the range of desired trust region. The higher the
trust region is, the smaller the value of I
n
is.
Algorithm 1. Update Task
Data: T
n
(t), {I
n
,L
n
}
N
n=1
Result: {I
n
,L
n
}
N
n=1
1
for each τ
n
∈ Γ do
2 T
n,max
=max
t−L
n
≤t≤t
T
n
(t);
3 T
n,min
=min
t−L
n
≤t≤t
T
n
(t);
4 if T
n,max
>T
n,u
− 
1
or T
n,min
>T
n,d
then
5 I
n
=min{I
n
+1,L
n
} ;
6 else if T
n,min
<T
n,l
+ 
2
or T
n,max
<T
n,d
then
7 I
n
=max{I
n
− 1, 0} ;
8 return {I
n
,L
n
}
N
n=1
;
0
50
100
150
200
250
300
0
0.5
1
1.5
2
2.5
trust
control
expert
Fig.8. An example of periodic trust-based control strategy between the human operator and one
robot

Human-Robot Mutual Trust in (Semi)autonomous Underwater Robots 129
Based on the above analysis, we can conclude that controlling a team of robots shares
similarities to scheduling a set of periodic tasks on a single processor, which is a classic
research topic in real-time scheduling [2,26,31]. More specifically, the control of each
robot R
n
can be understood as the execution of an individual task τ
n
, and the human
operator can be understood as the single processor as he/she can only control one robot
at a time. Therefore, controlling a set of robots {R
1
, ··· ,R
n
} corresponds to executing
asetoftasksΓ = {τ
1
,...,τ
N
}. However, the new challenge here is that the time I
n
required to interact with a robot R
n
within each period L
n
will dynamically change ac-
cording to the robot and human performance. This case has not been studied in real-time
scheduling to the best of our knowledge. Because the human operator can only interact
within one robot at a time, we now propose a highest-trust-first scheduling algorithm to
schedule the interaction with different robots. In the highest-trust-first scheduling algo-
rithm, the human operator always chooses to interact with the robot that has the highest
mutual trust level and non-zero remaining interaction time. According to Equation (9),
we know that the calcuation of mutual trust level depends on the human performance
P
H
(t), which in turn depends on the utilization ratio r(t) as show in Equation (4). The
value of r(t) reflects the accumulative operation of the human operator from time 0 to
current time t. Therefore, the value of r(t) needs to be continuously updated as time
propagates. Moreover, the remaining interaction time of each robot also needs to be
dynamically updated as time propagates. To achieve these goals, we adopt the dynamic
timing model for real-time scheduling proposed in [35, 39]. The dynamic timing model
contains a set of state variables that represent the dynamic updates of system status
according to the evolution rules decided by the scheduling policy. The remaining inter-
action time of each robot can be defined as one of the state variables in the model and
gets updated dynamically. After updating these state variables in the dynamic timing
model, we obtain the human utilization ratio r accordingly. We now define three state
variables that completely represent the current status of robots at any time t.
Definition 2. Define the dynamic arrival as Q(t)=[q
1
(t), ..., q
N
(t)], where any
q
n
(t) ∈ Q(t) denotes how long after t the next instance of τ
n
will arrive. For any
task τ
n
, we have the evolution of q
n
(t) as follows
q
n
(t +1)=

q
n
(t) − 1,q
n
(t) > 0
L
n
,q
n
(t)=0
. (11)
Definition 3. Define the residue as S(t)=[s
1
(t), ..., s
N
(t)], where any s
n
(t) ∈ S(t)
represents the remaining interaction time required after time t by the current instance
of τ
n
. For any task τ
n
, we have the evolution of s
n
(t) as follows
s
n
(t +1)=
⎧
⎨
⎩
s
n
(t) − 1,u
n
(t)=1,q
n
(t) =0
s
n
(t),u
n
(t)=0,q
n
(t) =0
I
n
,q
n
(t)=0
. (12)
Definition 4. Define the gap as E(t)=[e
1
(t), ..., e
N
(t)], where any e
n
(t) ∈ E(t)
represents the difference between the current trust level T
n
(t) of the robot R
n
and its
desired trust upper limit T
n,u
, i.e., e
n
(t)=T
n,u
− T
n
(t).

130 Y. Wang et al.
Note that the higher the trust is, the smaller the value of e
n
(t) is. The choice of I
n
is designed in Algorithm 1 such that T
n
(t) will not exceed T
n,u
and hence e
n
(t) is
always positive. When the mutual trust level associated with the robot is higher, the
human operator has the trend to over trust the robot. Therefore, the highest-trust-first
scheduling algorithm states that the smaller e
n
(t) is, the more urgent the human needs
to manually control the robot R
n
. Based the state variables Q(t),S(t),E(t), we can
schedule the human operator’s interaction with a team of robots according to Algo-
rithm 2. At each time step t, the inputs to Algorithm 2 are the current system sta-
tus including the human and robot performance P
H
(t), {P
n,R
(t)}
N
n=1
, human utiliza-
tion ratio r(t), mutual trust {T
n
(t)}
N
n=1
, state variables of the dynamic timing model
Q(t),S(t),E(t), and the period and interaction time {L
n
,I
n
}
N
n=1
obtained from Al-
gorithm 1. We define a set G which is a set of tasks with the non-zero remaining in-
teraction time s
n
.IfG is not an empty set, the robot with minimum difference e
n
will
be chosen for teleoperation. The outputs of Algorithm 2 are the updated system status
P
H
(t +1), {P
n,R(t+1)
}
N
n=1
,r(t +1), {T
n
(t +1)}
N
n=1
,Q(t +1),S(t +1),E(t +1)
and the scheduling decision {u
n
(t +1)}
N
n=1
. If the human operator is scheduled to in-
teract with the robot R
n
at time t, u
n
(t +1) = 1 and if the human operator is scheduled
NOT to interact with the robot R
n
at time t, u
n
(t +1)=0.
We now summarize the periodic trust-based control strategy composed of Algorithm
1 and 2. At every time step t, we first use Algorithm 1 to evalute the choice of I
n
for each task τ
n
by checking whether the mutual trust T
n
(t) falls within the desired
trust region [T
n,l
,T
n,u
] during the time interval t ∈ [t − L
n
,t]. If not, the value of I
n
will be adjusted accordingly. To be more specific, Algorithm 1 can guarantee the proper
choice of I
n
so that T
n
(t) will not exceed T
n,u
or go below T
n,l
. Hence, e
n
(t) is always
positive. Given the output {I
n
,L
n
}
N
n=1
of Algorithm 1, we use Algorithm 2 to compute
the mutual trust T
n
(t +1)for each robot and then decide which is the next robot the
human operator will interact with. The above process repeats as time propagates.
5 Simulation Results
In this section, we will show that the periodic trust-based control strategy can effectively
maintain the mutual trust between the human operator and robots within a desired re-
gion.
5.1 Simulation Setup
We consider the case when a human operator controls three EcoMapper AUV robots
{R
1
,R
2
,R
3
}. All three robots are asked to perform station keeping using switched
autonomous and teleoperation strategy. Their performances are measured by the dis-
tance (in meters) to the goal points as described by Equation (2). Since the AUVs
may have different sensor packages installed, we assume the three robots have dif-
ferent parameters as listed in Table 2. Each robot has its initial performance as
[P
1,R
(0),P
2,R
(0),P
3,R
(0) = [0.18, 0.25, 0.21]. The human operator has his/her per-
formance as described by Equation (4). We assume that the task difficulty for the human
operator is β =0.8. The initial human performance is P
H
(0) = 0.25 and the initial
utilization ratio is r(0) = 0.1.

Human-Robot Mutual Trust in (Semi)autonomous Underwater Robots 131
Algorithm 2. Highest-Trust-First Scheduling Algorithm
Data: r(t), P
H
(t), {P
n,R
(t),T
n
(t),q
n
(t),s
n
(t)}
N
n=1
, {L
n
,I
n
}
N
n=1
Result: {u
n
(t+1)}
N
n=1
, r(t+1), P
H
(t+1),
{P
n,R
(t+1),T
n
(t+1),q
n
(t+1),s
n
(t+1)}
N
n=1
/* Update P
n,R
, P
n,H
and T
n
*/
1 for each τ
n
∈ Γ do
2 P
n,R
(t +1)
Eq.(2)
←−−−−{P
n,R
(t),u
n
(t)};
3 r(t +1)
Eq.(10)
←−−−−−{r(t), {u
n
(t)}
N
n=1
};
4 P
H
(t +1)
Eq.(4)
←−−−− r(t +1);
5 T
n
(t +1)
Eq.(9)
←−−−−{T
n
(t),P
n,R
(t +1),P
n,R
(t),P
H
(t +1),P
H
(t)};
/* Update state variables */
6 for each τ
n
∈ Γ do
7 q
n
(t +1)
Eq.(11)
←−−−−−{q
n
(t),L
n
};
8 s
n
(t +1)
Eq.(12)
←−−−−−{s
n
(t),u
n
(t),q
n
(t),I
n
};
9 e
n
(t +1)=T
n,u
− T
n
(t);
/* G is a set of tasks with the non-zero remaining interaction time*/
10 G =[];
11 for each τ
n
∈ Γ do
12 if s
n
(t +1)> 0 then
13 G =[G, τ
n
];
/* Select the next task for execution*/
14 for each τ
n
∈ Γ do
15 u
n
(t +1)=0;
16 if G is not empty then
17 i =min
τ
n
∈G
e
n
(t +1);
18 u
i
(t +1)=1;
19 return {u
n
(t+1)}
N
n=1
, r(t+1), P
H
(t+1),
{P
n,R
(t+1),T
n
(t+1),q
n
(t+1),s
n
(t+1)}
N
n=1

132 Y. Wang et al.
Table 2. Coefficients in Robot Performance Model
k
R
k
H
P
Rmin
P
Rmax
R
1
0.15 0.09 0.05 0.96
R
2
0.12 0.09 0.02 0.85
R
3
0.15 0.12 0.05 0.95
The mutual trust between the human operator and each robot R
n
follows the dynamic
model discussed in Equation (9). The constant coefficients in Equation (9) are chosen
as A
n
=1,B
n,1
= −1,B
n,2
=1,C
n,1
= −1,C
n,2
=1,D
n,1
=0.002,D
n,2
=
0.001 and the fault rates follow the normal distribution. The initial mutual trust be-
tween the human operator and three robots are assumed to be [T
1
(0),T
2
(0),T
3
(0)] =
[1.83, 1.8, 1.88]. The goal of the human operator is to make sure that the mutual trust
T
n
(t) with each robot R
n
stays within a desired trust region as time propagates. In this
simulation, we choose a desired trust region with the lower bound T
n,l
=1.5, the upper
bound T
n,u
=2, and the ideal expert level T
n,d
=1.75 for each robot.
To achieve the above goal, the human operator applies our proposed periodic strategy
to switch control among three robots. As discussed in Section 4.2, we choose the initial
parameters in the periodic strategy as
[I
1
,L
1
]=[3, 16]s [I
2
,L
2
]=[4, 18]s [I
3
,L
3
]=[5, 12]s (13)
where each pair [I
n
,L
n
] for n =1, 2, 3 denotes that the human operator must interact
with the robot R
n
for I
n
seconds within every L
n
seconds. Note that the value of I
n
will dynamically change according to Algorithm 1 and Algorithm 2. If the mutual trust
T
n
(t) is too close to the lower bound, the human operator will interact less with this
robot and thus the value of I
n
will decrease. On the other hand, if the mutual trust
T
n
(t) is too close to the upper bound, the value of I
n
will increase.
5.2 Results and Discussions
Figure 9 shows the evolution of the mutual trust T
n
(t) (n =1, 2, 3) for three robots
within the time interval t ∈ [0, 300] seconds, under the periodic control strategy. The
green lines represent the upper bound and lower bound of the desired trust region, the
black dashed lines represent the ideal expert level, the blue lines represent the mutual
trust between the human operator and robots, and the red lines represent the control of
the human operator. For the red line, “1” means that the human operator is interacting
with the robot; and “0” means that the human operator is NOT interacting with the robot.
It can be observed that the mutual trust T
n
(t) of each human-robot pair is consistently
bounded between [1.5, 2]. Moreover, by comparing the mutual trust in Figure 9 with
that in Figure 7, we can see that the mutual trust under the periodic control strategy in
Figure 9 stays closer to the ideal expert level (black dashed line) than that under the
trust-triggered strategy in Figure 7.

Human-Robot Mutual Trust in (Semi)autonomous Underwater Robots 133
50 100 150 200 250 300
0
0.5
1
1.5
2
2.5
Time
Mutual Trust
Control Mode
Expert Level
(a) T
1
(t) between human and R
1
50 100 150 200 250 300
0
0.5
1
1.5
2
2.5
Time
Mutual Trust
Control Mode
Expert Level
(b) T
2
(t) between human and R
2
50 100 150 200 250 300
0
0.5
1
1.5
2
2.5
Time
Mutual Trust
Control Mode
Expert Level
(c) T
3
(t) between human and R
3
Fig.9. Mutual Trust within [0, 300] seconds, under the periodic control strategy
50 60 70 80 90 100
0
0.5
1
1.5
2
2.5
Time
Mutual Trust
Control Mode
Expert Level
(a) The First Robot
50 60 70 80 90 100
0
0.5
1
1.5
2
2.5
Time
Mutual Trust
Control Mode
Expert Level
(b) The Second Robot
50 60 70 80 90 100
0
0.5
1
1.5
2
2.5
Time
Mutual Trust
Control Mode
Expert Level
(c) The Third Robot
Fig. 10. Mutual Trust within [50, 100]
0 50 100 150 200 250 300
0
0.5
0
0.5
0
0.5
P
1,R
P
2,R
P
3,R
Time
Fig. 11. Robot Performance

134 Y. Wang et al.
0 50 100 150 200 250 300
0.25
0.3
0.35
0.4
0.45
0.5
P
H
Time
(a)
0 50 100 150 200 250 300
0.1
0.12
0.14
0.16
0.18
0.2
0.22
0.24
Time
r(t)
(b)
Fig. 12. (a) Human Performance, and (b) Utilization
For the sake of illustration, we further zoom in the evolution of the mutual trust
within a smaller time interval [50, 100] in Figure 10. We can clearly see that the human
operator can only interact with one robot at a time. Moreover, for each robot, the amount
of interaction time will dynamically change according to the mutual trust.
Figure 11 shows the evolution of robot performance within [0, 300]. As seen from
Figure 11, due to the different robot performance at each time step, the teleoperated
and autonomous control allocation for each robot is dynamically changing. Figure 12
shows the evolution of human performance and utilization within [0, 300], respectively.
Because the utilization is always less than the optimal value β =0.8, the evolution
of human performance has the same trend as the utilization. All the performance is
bounded between [0, 1].
6Conclusion
In this paper, we propose a mutual trust model to capture the interactions between hu-
man and underwater robots. A set of trust based control strategies are developed to
allocate the teleoperated and autonomous mode in real-time. Especially, we present a
real-time scheduling algorithm for the multi-robot case. The mutual trust level depends
on both robot and human performance. More specifically, we investigate the robot per-
formance of the YSI EcoMapper AUV and establish the human performance model
based on Yerkes-Dodson law. A trust-triggered control strategy is first developed for
single robot and human pair. We further extend the results to multi-robot case and pro-
pose a periodic trust-based control strategy with highest-trust-first scheduling algorithm
for real-time task allocation. Simulation results are presented to show the effectiveness
of the proposed strategies.
Although we present a novel and attractive tool to study HRI, in particular, human-
robot mutual trust, this preliminary work is a first step towards systematic study of the
model and control of the dynamical evolution of HRI in collaborative human and AUV
teams. Further works will focus on the extensions on both performance models and
control strategies for human-robot mutual trust in underwater applications. Human per-
formance model specific to AUV operators will be investigated. We will also consider

Human-Robot Mutual Trust in (Semi)autonomous Underwater Robots 135
non-periodic trust-based scheduling for multiple human multiple robot case and the sta-
bility proof for the switched control strategies. Since the execution time of each task is
dynamically changing according to the performance of the robots, we will develop al-
gorithms to predict the robot performance within finite time horizon and also integrate
the highest-trust-first scheduling algorithm into the schedulability test.
References
1. IEEE RO-MAN: The 21st IEEE International Symposium on Robot and Human Interactive
Communication, Paris, France (September 2012)
2. Arzen, K.E., Cervin, A., Eker, J., Sha, L.: An introduction to control and scheduling co-
design. In: Proceedings of IEEE Conference on Decision and Control, pp. 4865–4870 (2000)
3. Bainbridge, W.A., Hart, J., Kim, E.S., Scassellati, B.: Fan-out: Measuring human control of
multiple robots. In: The 17th IEEE International Symposium on Robot and Human Interac-
tive Communication, RO-MAN, pp. 701–706 (August 2008)
4. Balch, T., Arkin, R.: Behavior-based formation control for multirobot teams. IEEE Transac-
tions on Robotics and Automation 14(6), 926–939 (1998)
5. Bertrand, J.W.M., van Ooijen, H.P.G.: Workload based order release and productivity: a miss-
ing link. Production Planning and Control 13(7), 665–678 (2002)
6. Billings, D.R., Schaefer, K.E., Chen, J.Y., Kocsis, V., Barrera, M., Cook, J., Ferrer, M., Han-
cock, P.A.: Human-animal trust as an analog for human-robot trust: A review of current
evidence. Technical report, DTIC Document (2012)
7. Naval Studies Board. Autonomous vehicles in support of naval operations. National Research
Council, Washington, DC (2005)
8. Bruemmer, D.J., Marble, J.L., Dudenhoeffer, D.D.: Mutual initiative in human-machine
teams. In: Proceedings of the 2002 IEEE 7th Conference on Human Factors and Power
Plants, pp. 7–22. IEEE (2002)
9. Cao, M., Stewart, A., Leonard, N.E.: Integrating human and robot decision-making dynamics
with feedback: Models and convergence analysis. In: 47th IEEE Conference on Decision and
Control, pp. 1127–1132 (December 2008)
10. Coeckelbergh, M.: Humans, animals, and robots: A phenomenological approach to human-
robot relations. International Journal of Social Robotics 3(2), 197–204 (2011)
11. Crandall, J.W., Goodrich, M.A., Olsen Jr., D.R., Nielsen, C.W.: Validating human-robot in-
teraction schemes in multitasking environments. IEEE Transactions on Systems, Man and
Cybernetics, Part A: Systems and Humans 35(4), 438–449 (2005)
12. Cummings, M.: Human supervisory control of swarming networks. In: 2nd Annual Swarm-
ing: Autonomous Intelligent Networked Systems Conference (2004)
13. Ding, X., Powers, M., Egerstedt, M., Young, S., Balch, T.: Executive decision support:
Single-agent control of multiple uavs. IEEE Robotics & Automation Magazine 16(2), 73–81
(2009)
14. Freedy, A., DeVisser, E., Weltman, G., Coeyman, N.: Measurement of trust in human-robot
collaboration. In: International Symposium on Collaborative Technologies and Systems, pp.
106–114 (2007)
15. Ge, S.S., Khatib, O., Cabibihan, J.-J., Simmons, R., Williams, M.-A. (eds.): ICSR 2012.
LNCS (LNAI), vol. 7621. Springer, Heidelberg (2012)

136 Y. Wang et al.
16. Goodrich, M.A., Schultz, A.C. (eds.): Human-Robot Interaction: A Survey (Foundations and
Trends(r) in Human-Computer Interaction). Now Publishers Inc. (2008)
17. Hancock, P.A., Billings, D.R., Schaefer, K.E., Chen, J.Y.C., de Visser, E.J., Parasuraman, R.:
A meta-analysis of factors affecting trust in human-robot interaction. Human Factors: The
Journal of the Human Factors and Ergonomics Society 53, 517–527 (2011)
18. Hanheide, M., Lohse, M., Zender, H.: Special issue on expectations, intentions & actions in
human-robot interaction. International Journal of Social Robotics 4(2) (April 2012)
19. John, L., Moray, N.: Trust, control strategies and allocation of function in human-machine
systems. Ergonomics 35(10), 1243–1270 (1992)
20. Karwowski, W., Rahimi, M. (eds.): Human-Robot Interaction. Taylor & Francis (1992)
21. Kuzuoka, H., Evers, V., Imai, M., Forlizzi, J.: In: Proceedings of the 8th ACM/IEEE Interna-
tional Conference on Human-Robot Interaction, Tokyo, Japan (March 2013)
22. Lamers, M.H., Verbeek, F.J.: HRPR 2010. Revised Selected Papers Series: LNICST, vol. 59
(June 2010)
23. Laschi, C., Breazeal, C., Nakauchi, Y.: Guest editorial special issue on human-robot interac-
tion. IEEE Transactions on Robotics 23(5) (2007)
24. Lee, J.D., See, K.A.: Trust in automation: Designing for appropriate reliance. Human Fac-
tors 46, 50–80 (2004)
25. Lewandowsky, S., Mundy, M., Tan, G.: The dynamics of trust: Comparing humans to au-
tomation. Journal of Experimental Psychology: Applied 6(2), 104 (2000)
26. Liu, C.L., Layland, J.W.: Scheduling algorithms for multiprogramming in a hard- real-time
environment scheduling algorithms for multiprogramming. Journal of the Association for
Computing Machinery 20, 46–61 (1973)
27. Majji, M., Rai, R.: Autonomous task assignment of multiple operators for human robot in-
teraction. In: American Control Conference, pp. 6454–6459 (June 2013)
28. Mendl, M.: Performing under pressure: stress and cognitive function. Applied Animal Be-
haviour Science 65(3), 221–244 (1999)
29. Moray, N., Inagaki, T., Itoh, M.: Adaptive automation, trust, and self-confidence in fault
management of time-critical tasks. Journal of Experimental Psychology: Applied 6(1), 44
(2000)
30. Murphy, R.R., Steimle, E., Hall, M., Lindemuth, M., Trejo, D., Hurlebaus, S., Medina-
Cetina, Z., Slocum, D.: Multi-robot port-controlled hamiltonian systems under quantization.
In: IEEE International Workshop on Safety, Security & Rescue Robotics (SSRR) (2009)
31. Sha, L., Arzen, K.E., Abdelzaher, T., Cervin, A., Baker, T., Burns, A., Buttazzo, G., Cac-
camo, M., Lehoczky, J., Mokd, A.K.: Real time scheduling theory: A historical perspective.
Real-Time Systems 28, 101–155 (2004)
32. Sheridan, T.B.: Telerobotics, Automation, and Human Supervisory Control. The MIT Press
(1992)
33. Sheridan, T.B. (ed.): Humans and automation: System design and research issues. Wiley
(2002)
34. Sheridan, T.B.: Supervisory control of remote manipulators, vehicles and dynamic processes:
Experiments in command and display aiding. Technical report, DTIC Document (1983)
35. Shi, Z., Zhang, F.: Predicting time-delays under real-time scheduling for linear model predic-
tive control. In: International Conference on Computing, Networking and Communications.
Workshops Cyber Physical System, pp. 205–209 (2013)
36. Wang, C., Zhang, F., Schaefer, D.: Dynamic modeling of an autonomous underwater vehicle:
the ecomapper. In: Journal of Marine Science and Technology (2013) (under review)

Human-Robot Mutual Trust in (Semi)autonomous Underwater Robots 137
37. Whetten, J., Goodrich, M., Guo, Y.: Beyond robot fan-out: Towards multioperator supervi-
sory control. In: IEEE International Conference on Systems Man and Cybernetics (SMC),
pp. 2008–2015 (2010)
38. Yerkes, R.M., Dodson, J.D.: The relation of strength of stimulus to rapidity of habit-
formation. Journal of Comparative Neurology and Psychology 18(5), 459–482 (2004)
39. Zhang, F., Shi, Z., Mukhopadhyay, S.: Robustness analysis for battery supported cyber-
physical systems. ACM Transactions in Embedded Computing Systems 12(3), Article 6 (27
pages) (2013)