Output-Feedback Adaptive Neural Control for Stochastic Nonlinear Time-Varying Delay Systems With Unknown Control Directions

Abstract

This paper presents an adaptive output-feedback neural network (NN) control scheme for a class of stochastic nonlinear time-varying delay systems with unknown control directions. To make the controller design feasible, the unknown control coefficients are grouped together and the original system is transformed into a new system using a linear state transformation technique. Then, the Nussbaum function technique is incorporated into the backstepping recursive design technique to solve the problem of unknown control directions. Furthermore, under the assumption that the time-varying delays exist in the system output, only one NN is employed to compensate for all unknown nonlinear terms depending on the delayed output. Moreover, by estimating the maximum of NN parameters instead of the parameters themselves, the NN parameters to be estimated are greatly decreased and the online learning time is also dramatically decreased. It is shown that all the signals of the closed-loop system are bounded in probability. The effectiveness of the proposed scheme is demonstrated by the simulation results.

Full text

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 1
Output-Feedback Adaptive Neural Control for
Stochastic Nonlinear Time-Varying Delay
Systems With Unknown Control Directions
Tieshan Li, Senior Member, IEEE, Zifu Li, Dan Wang, and C. L. Philip Chen, Fellow, IEEE
Abstract— This paper presents an adaptive output-feedback
neural network (NN) control scheme for a class of stochastic
nonlinear time-varying delay systems with unknown control
directions. To make the controller design feasible, the unknown
control coefficients are grouped together and the original system
is transformed into a new system using a linear state trans-
formation technique. Then, the Nussbaum function technique is
incorporated into the backstepping recursive design technique to
solve the problem of unknown control directions. Furthermore,
under the assumption that the time-varying delays exist in
the system output, only one NN is employed to compensate
for all unknown nonlinear terms depending on the delayed
output. Moreover, by estimating the maximum of NN parameters
instead of the parameters themselves, the NN parameters to be
estimated are greatly decreased and the online learning time is
also dramatically decreased. It is shown that all the signals of the
closed-loop system are bounded in probability. The effectiveness
of the proposed scheme is demonstrated by the simulation results.
Index Terms—Adaptive output feedback control, neural
network (NN), stochastic nonlinear systems, time-varying delays,
unknown control directions.
I. INTRODUCTION
RECENTLY, stochastic nonlinear control has received
more and more attention due to the existence of sto-
chastic disturbances in many practical systems. With the
proposition of many important concepts of stochastic stability
theory [1], quite a number of research results on deterministic
nonlinear systems have been extended to stochastic nonlinear
Manuscript received July 2, 2013; revised June 13, 2014; accepted June 23,
2014. This work was supported in part by the National Natural Science
Foundation of China under Grant 51179019, Grant 61374114, and Grant
61273137, in part by the Program for Liaoning Excellent Talents in University
under Grant LR2012016, in part by the Applied Basic Research Program,
Ministry of Transport of China, under Grant 2011-329-225-390 and Grant
2013-329-225-270, in part by the Fundamental Research Funds for the Central
Universities under Grant 313-2014-321, in part by the National Fundamental
Research 973 Program of China under Grant 2011CB302801, in part by
the Macau Science and Technology Development Foundation, Macau, China,
under Grant 008/2010/A1, and in part by the Multiyear Research Grants.
(Corresponding author: Zifu Li.)
T. Li is with Navigation College, Dalian Maritime University, Dalian
116026, China (e-mail: tieshanli@126.com).
Z. Li is with Navigation College, Dalian Maritime University, Dalian
116026, China, and also with Navigation College, Jimei University, Xiamen
361021, China (e-mail: lzfxmjmu1019@163.com).
D. Wang is with the Marine Engineering College, Dalian Maritime Univer-
sity, Dalian 116026, China (e-mail: dwangdl@gmail.com).
C. L. P. Chen is with the Faculty of Science and Technology, University of
Macau, Macau 99999, China (e-mail: philip.chen@ieee.org).
Color versions of one or more of the figures in this paper are available
online at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/TNNLS.2014.2334638
systems, for example, Sontags stabilization formula, nonlin-
ear optimality, and backstepping technique [2], [3], where
the backstepping approach was developed to the point of a
step-by-step design procedure in [4]. The main technical
obstacle in the Lyapunov design for stochastic systems is that
the Itˆostochastic differentiation involves not only the gradient
but also the higher order Hessian term. Pan and Basar [5]
were the first to solve the stochastic stabilization problem for
a class of strict-feedback systems based on a risk-sensitive
cost criterion. By employing the quadratic Lyapunov functions,
Deng et al. [2], [3] extended the backstepping design method
to output-feedback stabilization and state-feedback stabiliza-
tion of stochastic nonlinear systems, respectively. However,
the combined problem of the control of stochastic nonlinear
systems with nonlinear uncertainties simultaneously is still a
cumbersome issue.
As well known, both neural network (NN) and fuzzy logic
system (FLS) have been proved to be a useful tool for solv-
ing the control problem of uncertain systems with unknown
nonlinear functions in practical applications [6]. By combining
NNs or FLSs with the backstepping design technique, several
adaptive NN or adaptive fuzzy backstepping control schemes
have been developed for several classes of stochastic nonlinear
systems with mismatched conditions [7]–[11]. For example,
a novel adaptive fuzzy backstepping control scheme was
proposed based on the observer design in [8] for a class of
stochastic nonlinear strict-feedback systems. In [9] and [10],
two constructive output-feedback control approaches were
developed based on adaptive NNs for two classes of stochastic
nonlinear systems, respectively. In addition, by combining
backstepping technique with stochastic small-gain approach,
a novel robust adaptive fuzzy output feedback controller was
presented for a class of stochastic nonlinear systems in [11].
Recently, [12] and [13] extended the above results to a class
of stochastic nonlinear large-scale systems, respectively. More
recently, a novel adaptive control scheme was proposed for a
class of uncertain nonlinear stochastic systems based on fuzzy
NN approximation in [14].
However, when approximating the unknown smooth func-
tions using either NNs or FLSs, the number of parameters to
be tuned online, i.e., the neural weights of hidden units in a
NN, or the fuzzy weights in a FLS, will grow rapidly with the
dimension of the argument vector of the function to be approx-
imated [15], which causes the explosion of learning para-
meters. Consequently, it makes the complex NNs or FLSs
2162-237X © 2014 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
2IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
unsuitable for the real-time systems that are sensitive to time
delays, and the time-consuming process is unavoidable during
the implementation of these control algorithms. This problem
was pointed out by Fischle and Schroder [16] and first solved
by Yang and Ren [17] in their pioneering work, where the
so-called minimal learning parameter (MLP) algorithm con-
taining much less online adaptive parameters were constructed
by fusion of traditional backstepping technique and radial-
basis-function (RBF) NNs. Recently, by combining dynamic
surface control and MLP techniques, Li et al. [18] first
proposed an algorithm that can simultaneously solve both
problems of the explosion of learning parameters and the
explosion of computation complexity. Then, a novel adaptive
neural output feedback controller based on reduced-order
observer was explored for a class of uncertain nonlinear
Single Input Single Output systems in [19]. More recently,
by estimating the maximum of NN parameters, Yu et al. [20]
designed an adaptive neural controller containing only one
adaptive parameter. Nevertheless, little work has been done
to investigate the adaptive NN or fuzzy output-feedback con-
trol problem for stochastic nonlinear systems with unknown
control directions.
It is well known that the control directions, defined as
signs of the control gains, are normally required to be known
aprioriin adaptive control literature. However, the unknown
control directions often exist in many practical nonlinear
systems. It is also a major source of resulting in instability
of the control systems. When the signs of control gains are
unknown, the adaptive control problem becomes much more
difficult, since we cannot decide the direction along which the
control operates. This problem has been remained open till the
Nussbaum-type gain was first introduced in [21] for adaptive
control of a class of first-order linear systems. Later on, the
Nussbaum gain was extended to adaptive control of nonlinear
systems with unknown control directions incorporated with
different techniques. For example, Liu [22] investigated the
output-feedback adaptive stabilization for a class of nonlinear
systems with unknown control directions using the linear
state transformation technique. Wen and Ren [23] proposed
a state observer-based adaptive neural control scheme for the
systems with unknown control directions and unmeasurable
states. To handle the unknown control directions in stochastic
nonlinear systems, Wang et al. [24] and Yu et al. [25] pro-
posed adaptive fuzzy or neural backstepping control methods
for stochastic nonlinear systems using the Nussbaum func-
tion technique under state feedback framework, respectively.
Recently, Wang et al. [26] developed an adaptive fuzzy output-
feedback control scheme for a class of stochastic nonlinear
systems with unknown control direction, in which only one
control coefficient was assumed unknown. However, little
work was dedicated to the unknown control direction problem
for stochastic nonlinear systems along with time delays.
In many practical control systems, such as biological sys-
tem, microwave oscillators, nuclear reactors, network system
and so on, there often exists time delay, which is frequently
a source of instability, and the control performance of these
systems is often degraded. Over the past years, the problem
of adaptive control design along with the stability analysis
for the time-delay systems has been a popular topic and
significant progress has been achieved [27]–[40].For example,
Ge et al. [33] and Hong et al. [34] successfully constructed
state feedback controllers for nonlinear time-delay systems,
respectively. Hua et al. [35] and Tong et al. [36] investigated
the output-feedback control problems for nonlinear time-delay
systems, respectively. Then, Chen et al. [38] developed an
adaptive consensus control scheme for a class of nonlinear
multiagent time-delay systems with the help of NNs approxi-
mation. Recently, Chen and Jiao [39] addressed the problem of
adaptive NN output-feedback control for a class of uncertain
stochastic nonlinear strict-feedback systems with time-varying
delays, where the problem of nonlinear observer design was
solved by introducing the circle criterion. More recently,
without controllable linearization and by first employing the
adding-a-power-integer technique to solve the stabilization
problem of stochastic time-delay systems, Chen et al. [41]
investigated the state-feedback stabilization problem for a class
of lower-triangular stochastic time-delay nonlinear systems.
However, to the best of the authors knowledge, there is no
result reported on the output-feedback control problem of
stochastic nonlinear systems with both time-varying delays and
unknown control directions.
Motivated by the above observations, incorporating the
linear state transformation with MLP techniques, an adap-
tive backstepping output-feedback neural control scheme is
proposed for a class of stochastic nonlinear systems with
both time-varying delays and unknown control directions.
The unknown control coefficients are grouped together and
the original system is transformed into a new system by
employing the linear state transformation technique. Then, the
Nussbaum-type gain function is used to deal with the unknown
parameters caused by the unknown control directions in the
new system. In addition, an RBF NN is used to approximate
all unknown nonlinear terms depending on the delayed output.
Still, the MLP technique is used to alleviate the computa-
tional burden by estimating the maximum of NN parameters.
Compared with the existing results, the main contributions in
this paper lie in the following: 1) for the first time, the linear
state transformation technique is introduced to the stochastic
nonlinear time-varying delay systems with unknown control
directions. Moreover, unlike [23] and [39], the assumption on
the value of the unknown control coefficients is not needed,
both control singularity and unknown control direction can be
tackled in this paper; 2) all the unknown output-dependent
functions are grouped into a suitable unknown function that
is compensated only by one NN. This simplifies the design
procedure and reduces the computation loads dramatically; and
3) by estimating the maximum of NN parameters instead of
the parameters themselves, the parameters to be estimated are
greatly decreased. Hence, the explosion of learning parameters
is solved efficiently.
The rest of this paper is organized as follows. Section II
provides some notations and preliminary knowledge. The
problem formulation is presented in Section III. In Section IV,
the adaptive output-feedback control design and stability
analysis are presented for the stochastic nonlineartime-varying
delay systems with unknown directions. Section V gives

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
LI et al.: OUTPUT-FEEDBACK ADAPTIVE NEURAL CONTROL FOR STOCHASTIC NONLINEAR TIME-VARYING DELAY SYSTEMS 3
some simulation examples to illustrate the effectiveness of the
method proposed in this paper, followed by the conclusion
presented in Section VI.
II. NOTATIONS AND PRELIMINARY KNOWLEDGE
The following notations will be used throughout this
paper. R+denotes the set of all nonnegative real numbers.
Rndenotes the real n-dimensional space. Rn×rdenotes the
real n×rmatrix space. For a given vector or matrix X,
XTdenotes its transpose. Tr(X)denotes its trace when X
is square. |X|denotes the Euclidean norm of a vector X,
and the corresponding induced norm for matrix Xis denoted
by X.XFdenotes the Frobenius norm of Xdefined
by XF=Tr(XTX)with properties X≤XFand
XYF≤XFYF.λmin(X)and λmax (X)denote the
minimal eigenvalue and maximal eigenvalue of symmetric real
matrix X, respectively. Cidenotes the set of all functions with
continuous ith partial derivatives. C2,1(Rn×[−d,∞);R+)
denotes the family of all nonnegative functions V(x,t)on
Rn×[−d,∞);R+with C2in xand C1in t.C2,1denotes
the family of all functions with C2in the first argument and
C1in t.
Consider an n-dimensional stochastic time-delay system
dx =f(t,x(t), x(t−d(t)))dt
+g(t,x(t), x(t−d(t)))dw(∀t≥0)(1)
with initial condition {x(s):−d≤s≤0}=ξ∈
Cb
F0×[(−d,0);Rn],whered(t):R+→[0,d]is a
Borel measurable function. f:R+×Rn×Rn→Rnand
g:R+×Rn×Rn→Rn×rare locally Lipschitz, and
wis r-dimensional standard Brownian motion defined on the
complete probability space ( , F,{Ft}t≥0,P)with being a
sample space, with Fbeing a σfield, {Ft}t≥0being a filtration,
and Pbeing the probability measure.
Define a differential operator Lknown as infinitesimal
generator for twice continuously differentiable function
V(x,t)∈C2,1as follows:
LV =∂V
∂t+∂V
∂xf(x(t), x(t−d(t)), t)+1
2Tr gT∂2V
∂x2.
(2)
Definition 2.1: N (·)is an even smooth Nussbaum-type
function, if it satisfies
limt→∞ sup 1
ss
0N(v)dv
limt→∞ inf 1
ss
0N(v)dv.(3)
From the definition, we know that Nussbaum functions
should have infinite gains and switching frequencies. There
are many continuous function satisfying these conditions,
such as ev2cos(v),ln(v +1)cos √ln(v +1),andv2cos(v).
In this paper, the even Nussbaum function v2cos(v)
is used.
Lemma 2.1 [24]: Consider the stochastic nonlinear
system (1), and assume that there exists a smooth function ξ:
R+→R, a function V(x,t)∈C2,1(Rn×[−d,∞);R+),and
a Nussbaum type even function N(·).Letχbe a nonnegative
random variable, M(t)be a real valued continuous local
martingale with M(0)=0 such that
V(x,t)≤χ+e−ct t
0(δN(ξ ) ˙
ξ+˙
ξ)ecτdτ+M(t). (4)
Then, the functions V(x,t), ξ(t)and t
0(δN(ξ ) ˙
ξ+˙
ξ)dτmust
be bounded in probability.
In this paper, the RBF NN will be used to approx-
imate any unknown continuous function F(Z), namely
Fnn(Z)=WTS(Z),whereZ∈Z⊂Rqis the input
vector with qbeing the input dimension of NNs, W=
[w1,w
2,...,w
l]T∈Rlis the weight vector with l>1
being the node number of a NN, and S(Z)means the basis
function vector with si(Z)being chosen as a Gaussian function
si(Z)=exp[−(Z−μi)T(Z−μi)/ς2],i=1,2,...,l,where
μi=[μi;,μ
i2,...,μ
iq]Tis the center of the receptive
field, and ς>0 is the width of the basis function si(Z).
It has been proven that an RBF NN can approximate any
continuous function over a compact set Z⊂Rqto an
arbitrary accuracy as
F(Z)=W∗TS(Z)+δ(Z). (5)
The ideal weight vector W∗is an artificial quantity required
for analytical purposes. It is defined as
W∗:= arg min
ˆ
W∈Rqsup
Z∈Zhnn(Z)−ˆ
WTS(Z).
Assumption 2.1 [25]: For ∀Z∈Z, there exists an ideal
constant weight vector W∗such that W∗∞≤wmax and
|δ(Z)|≤δmax with bounds wmax and δmax >0. It is obvious
that
W∗TS(Z)+δ(Z)≤W∗TS(Z)+|δ(Z)|
≤
l

i=1|si(Z)|wmax +δmax
≤θβ (Z)(6)
where
β(Z)=


(1+l)l

i=1s2
i(Z)+1,
θ=max{δmax,w
max}.
Assumption 2.2: For 1 ≤i≤n, according to the mean value
theorem, the following equalities hold:
fi(y,y(t−d(t))) =y(t−d(t)) ¯
fi(y,y(t−d(t))) (7)
hi(y,y(t−d(t))) =y(t−d(t)) ¯
hi(y,y(t−d(t))) (8)
where fi(y,0)=0, hi(y,0)=0, for 0 <ϑ
fi,ϑ
hi<1, the
unknown functions satisfy
¯
fi(y,y(t−d(t))) =∂fi(y,s)
∂ss=ϑfiy(t−d(t))
¯
hi(y,y(t−d(t))) =∂hi(y,s)
∂ss=ϑhiy(t−d(t)).
Remark 2.1: These unknown nonlinear functions depending
on the delayed output will be grouped into a suitable unknown
function that will be compensated by only one NN.

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
4IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
III. PROBLEM FORMULATION
Consider the following stochastic nonlinear time-varying
delay system described by:
⎧
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎨
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎩
dxi=gixi+1+φi(y)+fi‘(y,y(t−d(t)))dt
+hi(y,y(t−d(t)))dw
.
.
.
dxn=[gnu+φn(y)+fn(y,y(t−d(t)))]dt
+hn(y,y(t−d(t)))dw
y=x1
(9)
where xi∈R(i=1,...n),u∈R,andy∈Rare the unmea-
sured system states, the control input, and the system output,
respectively. gi= 0, (i=1,2,...,n)called control coeffi-
cients, are unknown constants with unknown directions. φi(y)
are known smooth nonlinear functions with φi(0)=0. fi(·):
R2→Rand hi(·):R2→Rrare unknown locally Lipschitz
smooth functions with fi(y,0)=0andhi(y,0)=0.
The uncertain time-varying delay d(t):R+→[0,d]sat-
isfies ˙
d(t)≤ς<1, with ςbeing an unknown con-
stant. The initial condition {x(s):−d≤s≤0}=
ξ∈Cb
F0×[(−d,0);Rn]is unknown. whas been defined
in (1). Only the system output ycan be available for
measurement.
Remark 3.1: It is known that nonlinear systems with
unknown directions exist in a wide range of physical and
engineering practices, such as in ship control problem and
air space control problem. When all the control directions
of the nonlinear systems are unknown, the control design
for such systems will become more challenging. It has
been shown that the Nussbaum function technique is an
effective tool to deal with the unknown control coeffi-
cients, and various results have been obtained, but most
of them are for state feedback, or output feedback with
only one unknown control coefficient. Therefore, (9) is more
general.
Remark 3.2: In this paper, we only consider that the smooth
functions φi(·)are the function of the output y, but not the
function of the system state vector ¯xi.φi(0)=0 implies
that the origin is the equilibrium point of (9). giare positive
or negative but not zero, which satisfies the controllability
condition of (9).
Remark 3.3: The time-varying delays of (9) only exist in
the system output. Similar to [10], [39], and [41], the time-
varying delay problem will be solved by constructing an
appropriate Lyapunov–Krasovskii function. However, they did
not consider the cases with all unknown control directions
along with time-varying delays in the stochastic nonlinear
systems.
Since the values and signs of the unknown control coef-
ficients are unknown, the output-feedback control design
becomes very hard. To make the output-feedback control
design feasible, (9) is transformed into a new system using the
linear state transformation technique. Therefore, the original
unknown control coefficients are grouped together in the new
states.
Let ζi=xi/n
j=igj,fori=1,...,n, then, the new state
variables ζ=[ζ1,...,ζ
n]are given by
⎧
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎨
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎩
dζi=ζi+1+1
n
j=igj(φi(y)+fi(y,y(t−d(t))))dt
+1
n
j=igjhi(y,y(t−d(t)))dw
.
.
.
dζn=u+1
gn(φn(y)+fn(y,y(t−d(t))))dt
+1
gnhn(y,y(t−d(t)))dw.
(10)
It is obvious that all the control coefficients are known in (10).
If all the states of (10) are available, it is easy to design a
controller. But, because of the existence of unknown control
coefficients in the linear state transformation, all the states
of (10) are unavailable. Therefore, a full-order state observer
must be established first to estimate the unmeasured states.
Then, a novel NN adaptive output-feedback control scheme
will be explored based on the designed observers.
Remark 3.4: After the linear state transformation, (9)
becomes a strict-feedback uncertain nonlinear system with
both unknown parameters 1/n
j=igjand unknown time-
varying delays. Therefore, it is much feasible to design a better
controller.
Consider the observers for (10) as follows:
⎧
⎪
⎪
⎨
⎪
⎪
⎩
˙
ˆ
ζi=ˆ
ζi+1−kiˆ
ζ1
.
.
.
˙
ˆ
ζn=u−knˆ
ζ1
(11)
where k1,...,knare optional positive constants such that the
following matrix:
A=⎡
⎢
⎣
−k1In−1
.
.
.
−kn0··· 0⎤
⎥
⎦
is asymptotically stable. Denote ˆ
ζ=[
ˆ
ζ1,...,ˆ
ζn]and let
˜
ζ=ζ−ˆ
ζbe the observe error. For convenience, define
ˆ
ζn+1=uand ˜
ζn+1=0. By combining (10) and (11), the
observer error dynamics of ˜
ζcan be obtained as follows:
d˜
ζ=[A˜
ζ+(y)+F(y,y(t−d(t))) +Kζ1]dt
+H(y,y(t−d(t)))dw(12)
where
(·)=#φ1
n
j=1gj,...,φn
gn$T
K=[k1,...,kn]T
F(·)=#f1
n
j=1gi,..., fn
gn$T
H(·)=#h1
n
j=1gj,..., hn
gnT
.
Hence, there exists a positive definite matrix P=PTsuch
that ATP+PA =−I.

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
LI et al.: OUTPUT-FEEDBACK ADAPTIVE NEURAL CONTROL FOR STOCHASTIC NONLINEAR TIME-VARYING DELAY SYSTEMS 5
Then, the complete system can be expressed as
⎧
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎨
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎩
d˜
ζ=A˜
ζ+(y)+F(y,y(t−d(t))) +Kζ1dt
+H(y,y(t−d(t)))dw
dy =[¯gζ2+φ1(y)+fi(y,y(t−d(t))]dt
+h1(y,y(t−d(t)))dw
dˆ
ζi=ˆ
ζi+1−kiζidt
.
.
.
dˆ
ζn=[u−knζn]dt
(13)
where ¯g=n
j=1gj.
IV. ADAPTIVE OUTPUT-FEEDBACK CONTROL DESIGN
AND STABILITY ANALYSIS
In this section, the adaptive backstepping technique will
be used to design an adaptive NN output-feedback controller.
To simplify the design procedure, some derivations are omit-
ted, and only the main design procedures are given.
Step 1: Following the adaptive backstepping design idea,
we define the error variables as follows:
z1=y,z2=ˆ
ζ2−α1(y,ξ, ˆ
ζ1,ˆ
θ, ˆ
l)
where α1(·)is a smooth function to be determined later, and
ξis the variable of Nussbaum-type function N(ξ ).ˆ
θdenotes
the estimate of θand the error ˜
θ=θ−ˆ
θ.ˆ
ldenotes the estimate
of land the error ˜
l=l−ˆ
l,lis an unknown constant defined as
l=max 3¯g4/3
4δi1,3¯g4/3
4δi2,3¯g4/3
4δj3
where i=1,...,n,j=2,...,n,δi1,andδi2are known
parameters to be designed later. For simplicity, here and
hereafter, fi=fi(y,y(t−d(t))),hi=hi(y,y(t−d(t))),
F=F(y,y(t−d(t))),andH=H(y,y(t−d(t))).
Then, the differential of z1is
dz1=¯g(˜
ζ2+ˆ
ζ2)+φ1(y)+f1dt +h1dw. (14)
Remark 4.1: By defining the unknown constant l,rather
than limiting the value of unknown parameters ¯giin (10),
the difficulty caused by the unknown control directions can
be overcome using integrator backstepping approach together
with tuning function technique and a Nussbaum-type function.
Consider the following Lyapunov function candidate:
V1=b
2%˜
ζTP˜
ζ&2+1
4z4
1+1
2˜
lT−1˜
l.(15)
It follows from (2), (13), and (14) that:
LV1=−b˜
ζTP˜
ζ|˜
ζ|2+2b˜
ζTP˜
ζ(
T(y)P˜
ζ)
+bTr'(2P˜
ζ˜
ζTP+˜
ζTP˜
ζP)HHT(+3
2z2
1h1hT
1
+2b˜
ζTP˜
ζ((Kζ1)TP˜
ζ)+2b˜
ζTP˜
ζ(FTP˜
ζ)
+z3
1¯g(˜
ζ2+ˆ
ζ2)+φ1(y)+f1−˜
lT−1˜
l.(16)
Then, using (16) and (A.1)–(A.8) in Appendix A, one has
LV1≤z3
1¯gα1+φ1(y)+z1)2ˆ
l+3
4λ11 +3
4λ12 *
+1
4δ12z4
2−b˜
ζTP˜
ζ|˜
ζ|2+1−˜
lT−1(˙
ˆ
l−τ1)(17)
where τ1=2z4
1. According to the well-known mean value
theorem, we can get φi(y)=y¯
φi(y).
Step 2: Define z3=ˆ
ζ3−α2(y,ξ, ˆ
ζ1,ˆ
ζ2,ˆ
θ, ˆ
l)with
α2(·)being a smooth function to be determined later. Then
from (13), the differential of z2is
dz2=ˆ
ζ3−k2ˆ
ζ1−∂α1
∂y(¯g(˜
ζ2+ˆ
ζ2)+φ1(y)+f1)
−∂α1
∂ˆ
ζ1
(ˆ
ζ2−k1ˆ
ζ1)−∂α1
∂ˆ
θ˙
ˆ
θ−∂α1
∂ˆ
l˙
ˆ
l
−1
2
∂2α1
∂y2h1hT
1−∂α1
∂ξ ˙
ξdt −∂α1
∂yh1dw. (18)
Consider the following Lyapunov function candidate:
V2=V1+1
4z4
2.(19)
The derivative of V2satisfies
LV2
≤LV1+z3
2ˆ
ζ3−k2ˆ
ζ1−∂α1
∂y%¯g%˜
ζ2+ˆ
ζ&+φ1(y)+f1&
−∂α1
∂ˆ
ζ1
(ˆ
ζ2−k1ˆ
ζ1)−∂α1
∂ˆ
θ˙
ˆ
θ−∂α1
∂ˆ
l˙
ˆ
l−1
2
∂2α1
∂y2h1hT
1
−∂α1
∂ξ ˙
ξ+3
2z3
2)∂α1
∂y*2
h1hT
1.(20)
Then, following (17), (20), and (A.9)–(A.14), where i=2,
and (A.15) in Appendix A, one has:
LV2≤z3
1¯gα1+φ1(y)+z1)2ˆ
l+3
4λ11 +3
4λ12 *
−b˜
ζTP˜
ζ|˜
ζ|2+2+1
4δ23y4+z3
2(α2−2)
+1
4λ24z4
3−˜
lT−1[˙
ˆ
l−τ2](21)
where τ2=τ1+z4
2(∂α1/∂y)4/3(2+¯α4/3
1).
Step i: (3≤i≤n−1)Define the error variable zi=
ˆ
ζi−αi−1,zi+1=ˆ
ζi+1−αi,whereαi(·)is a smooth function
to be determined later. Then, from (13), the differential of ziis
dzi=ˆ
ζi+1−kiˆ
ζ1−∂αi−1
∂y(¯g(˜
ζ2+ˆ
ζ2)+φ1(y)+f1)
−
i−1

j=1
∂αi−1
∂ˆ
ζj
(ˆ
ζj+1−kjˆ
ζ1)−∂αi−1
∂ˆ
θ˙
ˆ
θ−∂αi−1
∂ˆ
l˙
ˆ
l
−1
2
∂2αi−1
∂y2h1hT
1−∂αi−1
∂ξ ˙
ξdt −∂αi−1
∂yh1dw.
(22)
Consider the following Lyapunov function candidate:
Vi=Vi−1+1
4z4
i.(23)

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
6IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
Similar to the Step 2, and noting (A.9)–(A.14), (A.16), and
(A.18) in Appendix A, the differential of Visatisfies
LVi≤z3
1¯gα1+φ1(y)+z1)2ˆ
l+3
4λ11 +3
4λ12 *
−b˜
ζTP˜
ζ|˜
ζ|2+z3
2[α2−2]+
i

j=3
z3
j[αj−i]
+i+1
4
i

j=3
λj4z4
j+1−˜
lT−1[˙
ˆ
l−τi](24)
where τi=τi−1+z4
i(∂αi−1/∂y)4/3(2+¯α4/3
i−1).
Step n: Define the error variable zn=ˆ
ζn−αn−1. Then,
from (13), the differential of znis
dzn=u−knˆ
ζ1−∂αn−1
∂y(¯g(˜
ζ2+ˆ
ζ2)+φ1(y)+f1)
−
n−1

i=1
∂αn−1
∂ˆ
ζi
(ˆ
ζi+1−kiˆ
ζ1)−∂αn−1
∂ˆ
θ˙
ˆ
θ−∂αn−1
∂ˆ
l˙
ˆ
l
−1
2
∂2αn−1
∂y2h1hT
1−∂αn−1
∂ξ ˙
ξdt −∂αn−1
∂yh1dw.
(25)
Consider the following Lyapunov function candidate:
Vn=Vn−1+1
4z4
n.(26)
Repeating the similar operation in the former steps, and with
the help of Itˆo formula and (A.17) in Appendix A, the
differential of Vnsatisfies
LVn≤z3
1¯gα1+φ1(y)+z1)2ˆ
l+3
4λ11 +3
4λ12 *
−b˜
ζTP˜
ζ|˜
ζ|2+#1
4
n

i=1
δj1+3
2bP2ε
4
3
1
+ε4(2bP+bPλmax(p)) +3
2bP2ε
4
3
2
+3
2bP2ε
4
3
3|˜
ζ|4+z3
2[α2−2]
+
n−1

i=3
z3
i[αi−i]+ [u−n]+y4θβ(Z)
+y4(t−d(t))e−r¯
dψ(y,y(t−d(t)))
−1
1−ςy4ψ(y,y(t)) −˜
lT−1[˙
ˆ
l−τn](27)
where ris a known positive scalar, ¯
d=max{d(t)},ψ(y,y(t−
d(t))) =ψ1(y,y(t−d(t)))+ψ2(y,y(t−d(t))), which satisfies
ψ1(y,y(t−d(t))) =1
2ε4
2
er¯
dbP2n

i=11
n
j=igj¯
fi4
+1
ε4er¯
d(2bP+bPλmax(p))
×
n

i=11
n
j=igj¯
hi4
(28)
ψ2(y,y(t−d(t))) =1
4er¯
d
n

i=1
λi1¯
f4
1+3
4er¯
d
n

i=1
λj2¯
h4
1
+1
4er¯
d
n

i=2
λj3¯
h4
1.(29)
Now, define a new smooth function as follows:
f=1
4
n

i=2
δi3+1
2ε4
1
bP2n

i=11
n
j=igj¯
φi(y)4
+1
2ε4
3
bP2n

i=1ki1
n
j=1gj4
+1
1−ςψ(y,y(t)).
Since giare unknown control coefficients, fcannot be directly
implemented to construct the virtual controller α1. Thus, the
RBF NN approximation property will be employed here so
that fcan be approximated by an RBF NN in the form of (6)
on the compact set zas follows:
f=θβ(Z). (30)
Now, consider the following positive definite function for the
whole closed-loop system:
V=Vn+1
2λ−1˜
θ2+e−r¯
d
1−ςt
t−d(t)
erυψ(y,y(υ))dυ(31)
where λis a design parameter and ψ(·)is a positive continuous
function to be determined. Notice that ˙
d(t)≤ς<1, the
differential of Vsatisfies
LV ≤z3
1¯gα1+φ1(y)+z1)2ˆ
l+3
4λ11 +3
4λ12 *
+ˆ
θβ(Z)tanh z4
1β(Z)
σ$
−bλmin(p)−3
2bP2ε
4
3
1−1
4
n

i=1
δj1−3
2bP2ε
4
3
2
−3
2bP2ε
4
3
3−ε4(2bP+bPλmax(p))|ζ|4
+z3
2[α2−2]+
n−1

i=3
z3
i[αi−i]+[u−n]
−λ−1˜
θ#˙
ˆ
θ−λλˆ
θ+λz3
1)β(Z)tanh )z4
1β(Z)
σ**$
−re−r¯
d
1−ςt
t−d(t)
erυψ(y,y(υ))dυ−1
2ω˜
l2
−1
2λ˜
θ2+D−˜
lT−1˙
ˆ
l−(−ωˆ
l+τn)(32)
where D=1/2ωl2+1/2λθ2+0.2785σθ.From(32),wecan
design the virtual control law, the actual control law and the
adaptive laws as follows:
α1=N(ξ)η (33)

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
LI et al.: OUTPUT-FEEDBACK ADAPTIVE NEURAL CONTROL FOR STOCHASTIC NONLINEAR TIME-VARYING DELAY SYSTEMS 7
α2=−c2z2+k2ˆ
ζ1+∂α1
∂yφ1(y)+∂α1
∂ˆ
ζ1
(ˆ
ζ2−k1ˆ
ζ1)
+∂α1
∂ˆ
θ˙
ˆ
θ+∂α1
∂ˆ
l˙
ˆ
l+∂α1
∂ξ ˙
ξ−z2
×⎡
⎣
1
4δ12 +1
4δ22 +3
4λ24 +3
4λ21 )∂α1
∂y*4
3
+1
4λ23 )∂2α1
∂y2*2
z2
2+3
4λ22 )∂α1
∂y*4
+ˆ
l)∂α1
∂y*4
3%2+¯α
4
3
1&⎤
⎦(34)
αi=−cizi+kiˆ
ζ1+∂αi−1
∂yφ1(y)+
i−1

j=1
∂αi−1
∂ˆ
ζj
(ˆ
ζj+1−kjˆ
ζ1)
+∂αi−1
∂ˆ
θ˙
ˆ
θ+∂αi−1
∂ˆ
l˙
ˆ
l+∂αi−1
∂ξ ˙
ξ−zi1
4λi−1,4
+3
4λi4+1
4δi2+3
4λi4+3
4λi1)∂αi−1
∂y*4
3
+1
4λi3)∂2αi−1
∂y2*2
z2
i+3
4λi2)∂αi−1
∂y*4
+ˆ
l)∂αi−1
∂y*4
3%2+¯α
4
3
i&⎤
⎦(35)
u=−cnzn+knˆ
ζ1+∂αn−1
∂yφ1(y)+∂αn−1
∂ˆ
θ˙
ˆ
θ+∂αn−1
∂ˆ
l˙
ˆ
l
+
n−1

j=1
∂αn−1
∂ˆ
ζj
(ˆ
ζj+1−kjˆ
ζ1)+∂αn−1
∂ξ ˙
ξ
−zn⎡
⎣
1
4λn−1,4+1
4δn2+3
4λn1)∂αn−1
∂y*4
3
+1
4λn3)∂2αn−1
∂y2*2
z2
n+3
4λn2)∂αn−1
∂y*4
+ˆ
l)∂αn−1
∂y*4
3%2+¯α
4
3
n&⎤
⎦(36)
˙
ˆ
l=−ωˆ
l+τ1+
n

j=2
z4
j)∂αi−1
∂y*4
3)2+¯α
4
3
1*(37)
˙
ˆ
θ=−λλˆ
θ+λz4
1β(Z)tanh )z4
1β(Z)
σ*(38)
where ˙
ξ=z3
1η,τ1=2z4
1,η=z1[c1+2ˆ
l+3/4λ11+3/4λ12+
ˆ
θβ(Z)tanh(z4
1β(Z)/σ )]+φ1(y). Substituting (33)–(38) into
(32) yields
LV ≤−cV +[¯gN(ξ ) +1]˙
ξ+D(39)
where c=min{2c0,4ci,λ
λ, ω},i=1,2,...,n,and
c0=bλmin(P)−1
4
n

i=1
δj1−3
2bP2ε
4
3
1−3
2bP2ε
4
3
2
−3
2bP2ε
4
3
3−ε4(2bP+bPλmax(p)). (40)
Theorem 4.1: Consider the closed-loop system consist-
ing of (9), the observer (11), the control law (36), and
the adaptive laws (37) and (38). Under the Assump-
tions 2.1 and 2.2, the following properties can be obtained:
1) all the involved signals are bounded in probability and
2) the error signal can converge to the compact set defined
as z:= {zi∈R|E(+n
i=1z4
i)≤4}.
Proof: See Appendix B.
V. SIMULATION EXAMPLE
In this section, two simulation examples will be given
to illustrate the effectiveness of the proposed control
method.
Example 5.1: Consider the following second-order stochas-
tic nonlinear system:
⎧
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎨
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎩
dx1(t)=[g1x2(t)+y2(t)+y(t)y(t−d(t))]dt
+sin(y(t−d(t)))dw
dx2(t)=[g2u+y(t)+sin(y(t)y(t−d(t))]dt
+y(t−d(t))e−y2(t−d(t))dw
y(t)=x1(t)
(41)
where g1and g2represent the unknown control direction
coefficients, d(t)=2+0.5cos(t).
Manipulating the linear state transformations with
ζ1=x1/(g1g2)and ζ2=x2/g2, the following system can be
obtained:
⎧
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎨
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎩
dζ1(t)=ζ2(t)+1
g1g2y2(t)+1
g1g2y(t)y(t−d(t))dt
+1
g1g2sin(y(t−d(t)))dw
dζ2(t)=u+1
g2y(t)+1
g2sin(y(t)y(t−d(t))dt
+1
g2y(t−d(t))e−y2(t−d(t))dw
dy(t)=g1g2ζ2(t)+y2(t)+y(t)y(t−d(t))dt
+sin(y(t−d(t)))dw.
(42)
Design the observer for (42) as follows:
⎧
⎨
⎩
˙
ˆ
ζ1=ˆ
ζ2−ˆ
ζ1
˙
ˆ
ζ2=u−ˆ
ζ1.
(43)
Define the error variables z1=yand z2=ˆ
ζ2−α1(y,ξ,
ˆ
ζ1,ˆ
θ, ˆ
l). By virtual of the design procedures and results given
in Section IV, the signal ξ, the adaptive output feedback virtual

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
8IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
Fig. 1. Response of x1,ˆx1, and the estimate error ˜x1.
control law α1, the actual control law u, and the adaptive laws
are given by ˙
ξ=z3
1η,α1=N(ξ)η,¯α1=α1/y
η=z1c1+2ˆ
l+3
4λ11 +3
4λ12 +ˆ
θβ(Z)tanh )z4
1β(Z)
σ*
+y2(t)
u=−c2z2+ˆ
ζ1+∂α1
∂yφ1(y)+∂α1
∂ˆ
ζ1
(ˆ
ζ2−ˆ
ζ1)+∂α1
∂ˆ
θ˙
ˆ
θ
+∂α1
∂ˆ
l˙
ˆ
l+∂α1
∂ξ ˙
ξ
−z2⎡
⎣
1
4δ22 +1
4λ24 +3
4λ21 )∂α1
∂y*4
3
+1
4λ23 )∂2α1
∂y2*2
z2
2
+3
4λ22 )∂α1
∂y*4
+ˆ
l)∂α1
∂y*4
3)2+¯α
4
3
1*⎤
⎦
˙
ˆ
l=−ωˆ
l+2z4
1+z4
2)∂α1
∂y*4
3)2+¯α
4
3
1*
˙
ˆ
θ=−λλˆ
θ+λz4
1β(Z)tanh )z4
1β(Z)
σ*.
In this simulation, one RBF NN is used to approximate the
unknown function, which contains 100 nodes and the width of
the basis function is chosen as five. The Nussbaum function
is chosen as N(ξ) =ξ2cos(ξ ). The following initial condi-
tions and suitable parameters can be chosen as x1(0)=0.5,
ζ2(0)=0.5, ˆ
ζ1(0)=0.7, ˆ
ζ2(0)=1, ξ(0)=0.5, ˆ
θ(0)=0.8,
ˆ
l(0)=0.5, c1=c2=1, λ11 =λ12 =5, σ=0.05, δ22 =0.05,
λ21 =λ22 =λ23 =200, λ24 =1, ω=400, =0.01,
λ=0.02, and λ=200. If the unknown control coefficients
are chosen as g1=−0.8andg2=−0.5, the simulation results
are shown in Figs. 1–6. Then, changing the unknown control
coefficients as g1=+0.8andg2=+0.5, the simulation
results, with the same initial conditions, the parameters, the
controller and the adaptive laws, are shown in Figs. 7–12.
From the simulation results, it can be seen that the proposed
controller in this paper is effective.
Example 5.2: To further illustrate the effectiveness of our
results, we consider the following practical example of a
pendulum system with stochastic disturbances [42]:
ml ¨q=−mgsin q−klq +1
lu(44)
Fig. 2. Response of x2,ˆx2, and the estimate error ˜x2.
Fig. 3. Nussbaum function signal ξ.
Fig. 4. Control input u.
Fig. 5. Adaptive parameter θ.
where uis the torque applied to the pendulum, qis the
anticlockwise angle between the vertical axis through the
pivot point and the rod, gis the gravity acceleration, and
the constants k,l,andmdenote a coefficient of friction,

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
LI et al.: OUTPUT-FEEDBACK ADAPTIVE NEURAL CONTROL FOR STOCHASTIC NONLINEAR TIME-VARYING DELAY SYSTEMS 9
Fig. 6. Adaptive parameter l.
Fig. 7. Response of x1,ˆx1, and the estimate error ˜x1.
Fig. 8. Response of x2,ˆx2, and the estimate error ˜x2.
Fig. 9. Nussbaum function signal ξ.
the length of the rod, and the mass of the bob, respectively.
It is assumed that the constant lis unknown. Since there are
not stochastic disturbances and time delays in the pendulum
systems, we introduce the stochastic disturbances and the
Fig. 10. Control input u.
Fig. 11. Adaptive parameter θ.
Fig. 12. Adaptive parameter l.
time-delay coefficient s, which satisfy s∈[0,1].Let
x1=q/k,x2=ml ˙q+klq. The nonlinear stochastic system
with time delays can be expressed as follows:
⎧
⎪
⎪
⎪
⎪
⎪
⎪
⎨
⎪
⎪
⎪
⎪
⎪
⎪
⎩
dx1=(g1x2−sk
mx1−(1−s)x1(t−d(t)))dt
+0.5(x1(t−d(t)))2dw
dx2=(g2u−smgsin kx1+(1−s)sin(x1(t−d(t))))dt
+0.5(x1(t−d(t)))2dw
y=x1
(45)
where the time-delay coefficient is chosen as s=0.5,
g1=1/mkl and g2=1/lrepresent the unknown con-
trol direction coefficients. d(t)=0.5(1+sin t(t)) is the
time delay. After the same linear state transformation as the
Example 5.1, (45) can be rewritten in the form of (10).

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
10 IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
Then, the following system can be obtained:
⎧
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎨
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎩
dζ1(t)=ζ2(t)−1
g1g2sk
mx1−1
g1g2(1−s)x1(t−d(t))dt
+1
g1g20.5(x1(t−d(t)))2dw
dζ2(t)=u−1
g2smg sin kx+1
g2(1−s)sin(x1(t−d(t)))
dt
+1
g20.5(x1(t−d(t)))2dw
dy(t)=g1g2ζ2(t)−sk
mx1−(1−s)x1(t−d(t))dt
+0.5(x1(t−d(t)))2dw.
(46)
Based on the proposed control scheme, the foregoing control
problem is easily solved. First, the observer is designed as
follows:
˙
ˆ
ζ1=ˆ
ζ2−ˆ
ζ1
˙
ˆ
ζ2=u−ˆ
ζ1.(47)
Then, define the error variables z1=y,z2=ˆ
ζ2−α1,and
follow the same manipulations in Section IV, the signal ξ,
the virtual control law α1, the actual control law u,andthe
adaptive laws can be given as follows ˙
ξ=z3
1η,α1=N(ξ)η,
¯α1=α1/y:
η=z1c1+2ˆ
l+3
4λ11 +3
4λ12 +ˆ
θβ(Z)tanh )z4
1β(Z)
σ*
−sk
mx1
u=−c2z2+ˆ
ζ1+∂α1
∂yφ1(y)+∂α1
∂ˆ
ζ1
(ˆ
ζ2−ˆ
ζ1)+∂α1
∂ˆ
θ˙
ˆ
θ
+∂α1
∂ˆ
l˙
ˆ
l+∂α1
∂ξ ˙
ξ
−z21
4δ22 +1
4λ24 +3
4λ21 )∂α1
∂y*4
3
+1
4λ23 )∂2α1
∂y2*2
z2
2+3
4λ22 )∂α1
∂y*4
+ˆ
l)∂α1
∂y*4
3)2+¯α
4
3
1*
˙
ˆ
l=−ωˆ
l+2z4
1+z4
2)∂α1
∂y*4
3)2+¯α
4
3
1*
˙
ˆ
θ=−λλˆ
θ+λz4
1β(Z)tanh )z4
1β(Z)
σ*.
Similar to the Example 5.1, in this simulation, one RBF NN
is taken for the unknown function, which contains 100 nodes
and the width of the basis function is chosen as five. The
Nussbaum function is chosen as N(ξ ) =ξ2cos(ξ).Thefol-
lowing initial conditions and suitable parameters can be cho-
sen as m=2,k=0.98, l=1, g=9.8, x1(0)=1,
ζ2(0)=0,ˆ
ζ1(0)=0.5, ˆ
ζ2(0)=0, ζ(0)=0, ˆ
θ(0)=0,
ˆ
l(0)=0, c1=c2=2, λ11 =λ12 =10, σ=0.1, δ22 =0.01,
λ21 =λ22 =λ23 =12, λ24 =26, ω=0.4, =0.005,
λ=0.1, and λ=10. The simulation results are shown in
Fig. 13. Response of x1,ˆx1, and the estimate error ˜x1.
Fig. 14. Response of x2,ˆx2, and the estimate error ˜x2.
Fig. 15. Control input u.
Figs. 13–15, from which we can see that the control perfor-
mance is still very well. To demonstrate the superiority of the
algorithm proposed in this paper, a comparative simulation
on (45) will be also given using the algorithm proposed
in [42]. The initial conditions and suitable design parameters
are chosen as x1(0)=0.1, ˆx1(0)=0.1, x2(0)=ˆx2(0)=0,
m=2, k=0.98, l=1, g=9.8, k1=0.001, k2=0.002,
r1=r2=1, ¯r1=5, ¯r2=6, σ1=0.1, σ2=0.12, ¯σ1=0.01,
¯σ1=0.01, c1=0.5, c2=0.01, v1=0.01, v2=0.05,
k=0.005, η=0.1, b=1, and n=2. All the meaning
of the design parameters can be founding [42], which will
not be explained in this paper. The simulation results are
shown in Figs. 16–18. Comparing the results in Figs. 16–18
with the results in Figs. 13–15, it is obvious that it takes
more time to keep (45) stable using the algorithm in [42]
than the algorithm proposed in this paper. This indicates that
the algorithm proposed in this paper has a better transient
performance than the algorithm proposed in [42].

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
LI et al.: OUTPUT-FEEDBACK ADAPTIVE NEURAL CONTROL FOR STOCHASTIC NONLINEAR TIME-VARYING DELAY SYSTEMS 11
Fig. 16. Response of x1and ˆx1in [42].
Fig. 17. Response of x2and ˆx2in [42].
Fig. 18. Control input uin [42].
VI. CONCLUSION
In this paper, the output-feedback adaptive neural control
has been investigated for a class of stochastic nonlinear sys-
tems with time-varying delays and unknown control directions.
Through a linear state transformation, the unknown control
coefficients are grouped together and the original system is
transformed into a new system that makes the control design
become feasible. The proposed adaptive neural controller
contains only one adaptive parameter that is relevant to the
given NNs such that the online learning time is dramatically
decreased. Notably, only one NN is employed to compensate
for all the unknown functions. The stability analysis guarantees
that all the signals in the closed-loop system are bounded in
probability. The simulation examples demonstrate the perfor-
mance of the proposed approach. Future works will focus on
the virtual control gain function and the function φ( ¯xi)rather
than φ(y), which will make the design more challenging.
APPENDIX A
In this appendix, we use Young’s inequality [43]
xy ≤εp
p|x|p+1
qεp|y|q
where ε>0, the constants p>1andq>1 satisfy
(p−1)(q−1)=1, and (x,y)∈R2. Applying these
inequalities leads to
2b˜
ζTP˜
ζT(y)P˜
ζ≤1
2ε4
1
bP2n

i=11
n
j=igjφi(y)4
+3
2bP2ε
4
3
1|˜
ζ|4(A.1)
2b˜
ζTP˜
ζFTP˜
ζ≤1
2ε4
2
bP2
n

i=11
n
j=igjfi4
+3
2bP2ε
4
3
2|˜
ζ|4(A.2)
2b˜
ζTP˜
ζ(Kζ1)TP˜
ζ≤3
2bP2ε
4
3
3|˜
ζ|4
+3
2ε4
3
bP2n

i=1
(kiζ1)4(A.3)
bTr'(2P˜
ζ˜
ζTP+˜
ζTP˜
ζP)HHT(
≤1
ε4[2bP+bPλmax(p)]
n

i=11
n
j=igjhi4
+ε4[2bP+bPλmax(p)]|˜
ζ|4(A.4)
¯gz3
1˜
ζ1≤1
4δ11|˜
ζ2|4+3
4δ11 ¯g4
3z4
1≤δ11|˜
ζ|4
+3
4δ11 ¯g4
3z4
1(A.5)
¯gz3
1z2≤1
4δ12z4
2+3
4δ12 ¯g4
3z4
1(A.6)
z3
1f1≤3
4λ11 z4
1+1
4λ11 y4(t−d(t))
ׯ
f4
1(y,y(t−d(t))) (A.7)
3
2z2
1h1hT
1≤3
4λ12 y4(t−d(t)) ¯
h4
1(y,y(t−d(t))) +3
4λ12 z4
1
(A.8)
−∂αi−1
∂y¯gz3
i˜
ζi≤1
4δi1|˜
ζi|4+3
4δi1¯g4
3)∂αi−1
∂y*4
3z4
i
≤1
4δi1|˜
ζ|4+3
4δi1¯g4
3)∂αi−1
∂y*4
3z4
i(A.9)
−∂αi−1
∂y¯gz3
i˜
ζi≤1
4δi1|˜
ζi|4+3
4δi1¯g4
3)∂αi−1
∂y*4
3z4
i
≤1
4δi1|˜
ζ|4+3
4δi1¯g4
3)∂αi−1
∂y*4
3z4
i(A.10)
−∂αi−1
∂y¯gz3
iˆ
ζi=−∂αi−1
∂y¯gz3
i(zi+y¯αi−1)
≤1
4δi2z4
i+3
4δi2¯g4
3)∂αi−1
∂y*4
3z4
i
+3
4δi3¯g4
3)∂αi−1
∂y*4
3
¯α
4
3
i−1z4
i+1
4δi3y4
(A.11)

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
12 IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
−∂αi−1
∂yz3
ifi−1≤3
4λi1)∂αi−1
∂y*4
3z4
i
+1
4λi1y4(t−d(t)) ¯
f4
i−1(A.12)
−1
2
∂2αi−1
∂y2z3
ihi−1hT
i−1≤1
4λi3)∂2αi−1
∂y2*2
z6
i
+1
4λi3y4(t−d(t)) ¯
h4
i−1(A.13)
3
2z2
i)∂αi−1
∂y*2
hi−1hT
i−1≤3
4λi2)∂αi−1
∂y*4
z4
i
+3
4λi2y4(t−d(t)) ¯
h4
i−1.(A.14)
Then, we can define
2=k2ˆ
ζ1+∂α1
∂yφ1(y)+∂α1
∂ˆ
ζ1
(ˆ
ζ2−k1ˆ
ζ1)+∂α1
∂ˆ
θ˙
ˆ
θ
+∂α1
∂ˆ
l˙
ˆ
l−z21
4δ12 +1
4δ22 +3
4λ21 )∂α1
∂y*4
3
+1
4λ22 )∂2α1
∂y2*2
z2
2+3
4λ23 )∂α1
∂y*4
+3
4λ24 +ˆ
l)∂α1
∂y*4
3)2+¯α
4
3
1* (A.15)
i=kiˆ
ζ1+∂αi−1
∂yφ1(y)+
i−1

j=1
∂αi−1
∂ˆ
ζj
(ˆ
ζj+1−kjˆ
ζ1)
+∂αi−1
∂ˆ
θ˙
ˆ
θ+∂αi−1
∂ˆ
l˙
ˆ
l+∂αi−1
∂ξ ˙
ξ
−zi1
4λi−1,4+3
4λi4+1
4δi2+3
4λi4+3
4λi1)∂αi−1
∂y*4
3
+1
4λi3)∂2αi−1
∂y2*2
z2
i+3
4λi2)∂αi−1
∂y*4
+ˆ
l)∂αi−1
∂y*4
3)2+¯α
4
3
i* (A.16)
n=knˆ
ζ1+∂αn−1
∂yφ1(y)+
n−1

j=1
∂αn−1
∂ˆ
ζj
(ˆ
ζj+1−kjˆ
ζ1)
+∂αn−1
∂ˆ
θ˙
ˆ
θ+∂αn−1
∂ˆ
l˙
ˆ
l+∂αn−1
∂ξ ˙
ξ
−zn1
4λn−1,4+1
4δn2+3
4λn1)∂αn−1
∂y*4
3
+1
4λn3)∂2αn−1
∂y2*2
z2
n
+3
4λn2)∂αn−1
∂y*4
+ˆ
l)∂αn−1
∂y*4
3)2+¯α
4
3
n*
(A.17)
i=y4bP2⎡
⎣
1
2ε4
1
n

m=11
n
j=mgj¯
φm(y)4
+1
2ε4
3
n

m=1km1
n
j=1gj4⎤
⎦
+#1
4
i

i=1
δj1+3
2bP2ε
4
3
1+ε4(2bP
+bPλmax(p)) +3
2bP2ε
4
3
2
+3
2bP2ε
4
3
3$|˜
ζ|4+y4(t−d(t))
×⎡
⎣
1
2ε4
2
bP2n

m=11
n
j=mgj¯
fm4
+1
4
i

j=1
λj1¯
f4
1
+3
4
i

j=1
λj2¯
h4
1+1
4
i

j=2
λj3¯
h4
1⎤
⎦.(A.18)
APPENDIX B
PROOF OF THEOREM 4.1
1) Define W(t,x)=V(t,x)ect . Applying [It ˆos formula to
the function W(t,x)yields
d(V(t,x)ect)=ect (V(t,x)+LV)dt
+ect ∂V
∂z1h1−
n

i=2
∂V
∂zi
∂αi−1
∂yh1+∂V
∂˜
ζHdw
≤ect ([¯gN(ξ ) +1]˙
ξ+D)dt
+ect ∂V
∂z1h1−
n

i=2
∂V
∂zi
∂αi−1
∂yh1+∂V
∂˜
ζHdw.
(B.1)
Integrating (B.1) over (0,t), we obtain
V(t,x)≤e−ct V(0,x(0)) +D
c
+t
0
e−c(t−s)([¯gN(ξ) +1]˙
ξ)ds
+t
0
e−c(t−s)∂V
∂z1h1−
n

i=2
∂V
∂zi
∂αi−1
∂yh1
+∂V
∂˜
ζH*dw(s). (B.2)
The stochastic integral (second integral) in (B.2) is local
martingale. Then, according to Lemma 1, the conclu-
sion can be obtained that the functions V(t,x),ξ(t),and
t
0e−c(t−s)([¯gN(ξ) +1]˙
ξ)ds must be bounded in probability.
2) Taking expectations on (B.2) and applying the Founds
lemma, we have
EV(t,x)≤EV(0,x(0)) +D
c+¯σ(B.3)

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
LI et al.: OUTPUT-FEEDBACK ADAPTIVE NEURAL CONTROL FOR STOCHASTIC NONLINEAR TIME-VARYING DELAY SYSTEMS 13
where ¯σ=sup t
0([¯gN(ξ ) +1]˙
ξ)ds. Then
1
4E
4

i=1
z4
i≤EV(t,x)≤EV(0,x(0)) +D
c+¯σ.
Denote =EV(0,x(0)) +D
c+¯σ,thenE4
+
i=1
z4
i≤4.
Therefore, there exists a compact set z, such that zi∈z.
Thus, the proof is completed.
REFERENCES
[1] P. Florchinger, “Lyapunov-like techniques for stochastic stability,” SIAM
J. Control Optim., vol. 33, no. 4, pp. 1151–1169, 1995.
[2] H. Deng and M. Krsti´c, “Output-feedback stabilization of stochastic
nonlinear systems driven by noise of unknown covariance,” Syst. Control
Lett., vol. 39, no. 3, pp. 173–182, 2000.
[3] H. Deng, M. Krsti´c, and R. J. Williams, “Stabilization of stochastic
nonlinear systems driven by noise of unknown covariance,”
IEEE Trans. Autom. Control, vol. 46, no. 8, pp. 1237–1253,
Aug. 2001.
[4] M. Krstic, I. Kanellakopoulos, and P. V. Kokotovic, Nonlinear and
Adaptive Control Design. New York, NY, USA: Wiley, 1995.
[5] Z. Pan and T. Basar, “Backstepping controller design for nonlinear
stochastic systems under a risk-sensitive cost criterion,” SIAM J. Control
Optim., vol. 37, no. 3, pp. 957–995, 1999.
[6] S. S. Ge and C. Wang, “Direct adaptive NN control of a class
of nonlinear systems,” IEEE Trans. Neural Netw., vol. 13, no. 1,
pp. 214–221, Jan. 2002.
[7] W. S. Chen, L. C. Jiao, and Z. B. Du, “Brief paper: Output-feedback
adaptive dynamic surface control of stochastic non-linear systems
using neural network,” IET Control Theory Appl., vol. 4, no. 12,
pp. 3012–3021, Dec. 2010.
[8] S. Tong, Y. Li, Y. Li, and Y. Liu, “Observer-based adaptive fuzzy
backstepping control for a class of stochastic nonlinear strict-feedback
systems,” IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 41, no. 6,
pp. 1693–1704, Dec. 2011.
[9] J. Li, W. Chen, J.-M. Li, and Y. Fang, “Adaptive NN output-feedback
stabilization for a class of stochastic nonlinear strict-feedback systems,”
ISA Trans., vol. 48, no. 4, pp. 468–475, 2009.
[10] W. Chen, L. C. Jiao, and J. Wu, “Decentralized backstepping
output-feedback control for stochastic interconnected systems with
time-varying delays using neural networks,” Neural Comput. Appl.,
vol. 21, no. 6, pp. 1375–1390, Sep. 2011.
[11] S. Tong, T. Wang, Y. Li, and B. Chen, “A combined backstepping and
stochastic small-gain approach to robust adaptive fuzzy output feedback
control,” IEEE Trans. Fuzzy Syst., vol. 21, no. 2, pp. 314–327, Apr. 2013.
[12] J. Li, W. Chen, and J.-M. Li, “Adaptive NN output-feedback
decentralized stabilization for a class of large-scale stochastic nonlinear
strict-feedback systems,” Int. J. Robust Nonlinear Control, vol. 21, no. 3,
pp. 452–472, Mar. 2011.
[13] Y. Li, Y. Li, and S.-C. Tong, “Adaptive fuzzy decentralized
output feedback control for stochastic nonlinear large-scale systems,”
Neurocomputing, vol. 83, pp. 38–46, Apr. 2012.
[14] C. L. P. Chen, Y.-J. Liu, and G.-X. Wen, “Fuzzy neural network-based
adaptive control for a class of uncertain nonlinear stochastic systems,”
IEEE Trans. Cybern., vol. 44, no. 5, pp. 583–593, May 2014.
[15] A. Papadimitropoulos, G. A. Rovithakis, and T. Parisini, “Fault detection
in mechanical systems with friction phenomena: An online neural
approximation approach,” IEEE Trans. Neural Netw., vol. 18, no. 4,
pp. 1067–1082, Jul. 2007.
[16] K. Fischle and D. Schroder, “An improved stable adaptive fuzzy control
method,” IEEE Trans. Fuzzy Syst., vol. 7, no. 1, pp. 27–40, Feb. 1999.
[17] Y. Yang and J. Ren, “Adaptive fuzzy robust tracking controller design
via small gain approach and its application,” IEEE Trans. Fuzzy Syst.,
vol. 11, no. 6, pp. 783–795, Dec. 2003.
[18] T.-S. Li, D. Wang, G. Feng, and S.-C. Tong, “A DSC approach to robust
adaptive NN tracking control for strict-feedback nonlinear systems,”
IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 40, no. 3, pp. 915–927,
Jun. 2010.
[19] Y.-J. Liu, S.-C. Tong, D. Wang, T.-S. Li, and C. L. P. Chen, “Adaptive
neural output feedback controller design with reduced-order observer
for a class of uncertain nonlinear SISO systems,” IEEE Trans. Neural
Netw., vol. 22, no. 8, pp. 1328–1334, Aug. 2011.
[20] Z. Yu, S. Li, and H. Du, “Razumikhin–Nussbaum-lemma-based adaptive
neural control for uncertain stochastic pure-feedback nonlinear systems
with time-varying delays,” Int. J. Robust Nonlinear Control, vol. 23,
no. 11, pp. 1214–1239, 2012.
[21] R. D. Nussbaum, “Some remarks on a conjecture in parameter adaptive
control,” Syst. Control Lett., vol. 3, no. 5, pp. 243–246,
Nov. 1983.
[22] Y.-G. Liu, “Output-feedback adaptive control for a class of nonlinear
systems with unknown control directions,” Acta Autom. Sinica, vol. 33,
no. 12, pp. 1306–1312, 2007.
[23] Y. Wen and X. Ren, “Neural networks-based adaptive control for
nonlinear time-varying delays systems with unknown control direction,”
IEEE Trans. Neural Netw., vol. 22, no. 10, pp. 1599–1612, Oct. 2011.
[24] W. Ying-Chun, Z. Hua-Guang, and W. Yi-Zhong, “Fuzzy adaptive
control of stochastic nonlinear systems with unknown virtual control
gain function,” Acta Autom. Sinica, vol. 32, no. 2, pp. 170–178,
Mar. 2006.
[25] Z. Yu, Z. Jin, and H. Du, “Adaptive neural control
for a class of non-affine stochastic non-linear systems with
time-varying delay: A Razumikhin–Nussbaum method,” IET Control
theory Appl., vol. 6, no. 1, pp. 14–23, Jan. 2012.
[26] T. Wang, S. Tong, and Y. Li, “Robust adaptive fuzzy output
feedback control for stochastic nonlinear systems with unknown control
direction,” Neurocomputing, vol. 106, pp. 31–41, Apr. 2013.
[27] Y. Fu, Z. Tian, and S. Shi, “Output feedback stabilization for a class of
stochastic time-delay nonlinear systems,” IEEE Trans. Autom. Control,
vol. 50, no. 6, pp. 847–851, Jun. 2005.
[28] H. Zhang, Y. Wang, and D. Liu, “Delay-dependent guaranteed cost
control for uncertain stochastic fuzzy systems with multiple time delays,”
IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 38, no. 1, pp. 126–140,
Feb. 2008.
[29] J. Fu, H. Zhang, T. Ma, and Q. Zhang, “On passivity
analysis for stochastic neural networks with interval time-varying
delay,” Neurocomputing, vol. 73, nos. 4–6, pp. 795–801, 2010.
[30] Z. Li, X. Cao, and N. Ding, “Adaptive fuzzy control for synchronization
of nonlinear teleoperators with stochastic time-varying communication
delays,” IEEE Trans. Fuzzy Syst., vol. 19, no. 4, pp. 745–757, Apr. 2011.
[31] L. Wu and W. X. Zheng, “L2–L∞control of nonlinear fuzzy ItÔ
stochastic delay systems via dynamic output feedback,” IEEE Trans.
Syst., Man, Cybern. B, Cybern., vol. 39, no. 5, pp. 1308–1315, Oct. 2009.
[32] Y. Xia, M. Fu, P. Shi, Z. Wu, and J. Zhang, “Adaptive backstepping
controller design for stochastic jump systems,” IEEE Trans. Autom.
Control, vol. 54, no. 12, pp. 2853–2859, Dec. 2009.
[33] S. S. Ge, F. Hong, and T. H. Lee, “Adaptive neural network control
of nonlinear systems with unknown time delays,” IEEE Trans. Autom.
Control, vol. 48, no. 11, pp. 2004–2010, Nov. 2003.
[34] F. Hong, S. S. Ge, and T. H. Lee, “Practical adaptive neural control of
nonlinear systems with unknown time delays,” IEEE Trans. Syst., Man
Cybern. B, Cybern., vol. 35, no. 4, pp. 849–854, Aug. 2005.
[35] C. Hua, X. Guan, and P. Shi, “Robust backstepping control for a class
of time delayed systems,” IEEE Trans. Autom. Control, vol. 50, no. 6,
pp. 894–899, Jun. 2005.
[36] S.-C. Tong, Y. M. Li, and H.-G. Zhang, “Adaptive neural
network decentralized backstepping output-feedback control for
nonlinear large-scale systems with time delays,” IEEE Trans. Neural
Netw., vol. 22, no. 7, pp. 1073–1086, Jul. 2011.
[37] Z. Liu, G. Lai, Y. Zhang, X. Chen, and C. L. P. Chen, “Adaptive
neural control for a class of nonlinear time-varying delay systems with
unknown hysteresis,” IEEE Trans. Neural Netw. Learn. Syst., 2014, doi:
10.1109/TNNLS.2014.2305717.
[38] C. L. P. Chen, G. Wen, Y. Liu, and F. Wang, “Adaptive consensus
control for a class of nonlinear multiagent time-delay systems using
neural networks,” IEEE Trans. Neural Netw. Learn. Syst., vol. 25, no. 6,
pp. 1217–1226, Jun. 2014.
[39] W. Chen, L. Jiao, J. Li, and R. Li, “Adaptive NN backstepping
output-feedback control for stochastic nonlinear strict-feedback systems
with time-varying delays,” IEEE Trans. Syst. Man Cybern. B, Cybern.,
vol. 40, no. 3, pp. 939–950, Jun. 2010.
[40] S.-J.Liu,S.S.Ge,andJ.-F.Zhang, “Adaptive output-feedback control
for a class of uncertain stochastic non-linear systems with time delays,”
Int. J. Control, vol. 81, no. 8, pp. 1210–1220, 2008.
[41] W. Chen, J. Wu, and L. C. Jiao, “State-feedback stabilization for a class
of stochastic time-delay nonlinear systems,” Int. J. Robust Nonlinear
Control, vol. 22, no. 17, pp. 1921–1937, Nov. 2012.

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
14 IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
[42] Y. Li, S. Tong, and Y. Li, “Observer-based adaptive fuzzy
backstepping control for strict-feedback stochastic nonlinear systems
with time delays,” Int. J. Innovative Comput., Inform., Control,vol.8,
no. 12, pp. 8103–8114, Dec. 2012.
[43] G. Hardy, J. E. Littlewood, and G. Polya, Inequalities, 2nd ed.
Cambridge, U.K.: Cambridge Univ. Press, 1989.
Tieshan Li (M’08–SM’12) received the B.S. degree
from the Ocean University of China, Qingdao,
China, in 1992, and the Ph.D. degree in trans-
portation from Dalian Maritime University (DMU),
Dalian, China, in 2005.
He was a Post-Doctoral Scholar with Shanghai
Jiao Tong University, Shanghai, China, from 2007
to 2010. He also visited the City University of Hong
Kong, Hong Kong, from 2008 to 2009, as a Senior
Research Associate, and the University of Macau,
Macau, China, in 2013, as a Visiting Scholar. He
is currently a Professor at DMU. His current research interests include
decentralized adaptive control, fuzzy control and neural-network control for
nonlinear systems, and their applications to marine control.
Zifu Li received the B.S. degree in marine technol-
ogy and the M.S. degree in transportation informa-
tion engineering and control from Dalian Maritime
University (DMU), Dalian, China, in 1998 and 2004,
respectively, where he is currently pursuing the
Ph.D. degree in transportation information engineer-
ing and control.
His current research interests include adaptive con-
trol, robust control, and neural-network control for
stochastic nonlinear systems.
Dan Wang was born in Dalian, China. He received
the B.Eng. degree in industrial automation engi-
neering from the Dalian University of Technol-
ogy, Dalian, in 1982, the M.Eng. degree in marine
automation engineering from Dalian Maritime Uni-
versity, Dalian, in 1987, and the Ph.D. degree in
mechanical and automation engineering from the
Chinese University of Hong Kong, Hong Kong, in
2001.
He was with Dalian Maritime University as a
Lecturer from 1987 to 1998, and an Associate
Professor in 1992. From 2001 to 2005, he was a Research Scientist with
Temasek Laboratories, National University of Singapore, Singapore. Since
2006, he has been with Dalian Maritime University, where he is a Professor
with the Department of Marine Electrical Engineering, Marine Engineering
College. His current research interests include nonlinear control theory and
applications, neural networks, adaptive control, robust control, fault detection
and isolation, and system identification.
C. L. Philip Chen (S’88–M’88–SM’94–F’07)
received the M.S. degree in electrical engineering
from the University of Michigan, Ann Arbor, MI,
USA, in 1985, and the Ph.D. degree in electrical
engineering from Purdue University, West Lafayette,
IN, USA, in 1988.
He is currently the Chair Professor with the
Department of Computer and Information Science
and the Dean of the Faculty of Science and Tech-
nology at the University of Macau, Macau, China.
Dr. Chen is a fellow of the American Association
for the Advancement of Science. He is currently the President of the IEEE
Systems, Man, and Cybernetics Society. In addition, he is an Accreditation
Board of Engineering and Technology Education Program Evaluator for
Computer Engineering, Electrical Engineering, and Software Engineering
programs.