Content uploaded by Jinghua Guo
Author content
All content in this area was uploaded by Jinghua Guo on May 15, 2018
Content may be subject to copyright.
3230 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, VOL. 17, NO. 11, NOVEMBER 2016
Nonlinear Coordinated Steering and Braking Control
of Vision-Based Autonomous Vehicles
in Emergency Obstacle Avoidance
Jinghua Guo, Ping Hu, and Rongben Wang
Abstract—This paper discusses dynamic control design for
automated driving of vision-based autonomous vehicles, with a
special focus on the coordinated steering and braking control in
emergency obstacle avoidance. An autonomous vehicle is a
complex multi-input and multi-output (MIMO) system, which
possesses the features of parameter uncertainties and strong non-
linearities, and the coupled phenomena of longitudinal and lateral
dynamics are evident in a combined cornering and braking ma-
neuver. In this work, an effective coordinated control system for
automated driving is proposed to deal with these coupled and
nonlinear features and reject the disturbances. First, a vision algo-
rithm is constructed to detect the reference path and provide the
local location information between vehicles and reference path in
real time. Then, a novel coordinated steering and braking control
strategy is proposed based on the nonlinear backstepping control
theory and the adaptive fuzzy sliding-mode control technique, and
the asymptotic convergence of the proposed coordinated control
system is proven by the Lyapunov theory. Finally, experimental
tests manifest that the proposed control strategy possesses favor-
able tracking performance and enhances the riding comfort and
stability of autonomous vehicles.
Index Terms—Autonomous vehicles, nonlinear coordinated con-
trol, vision algorithm, fuzzy sliding mode control, steering and
braking control.
I. INTRODUCTION
IN THE past two decades, such social concerns related to
traffic accidents and energy consumption have been in-
creased rapidly [1]. Autonomous vehicles apply information,
sense and control techniques to enhance driving safety and
efficiency, which are regarded as one of the effective ways
to improve traffic safety and reduce fuel consumption. Due
to these potential benefits, recently, researches on autonomous
vehicles have attracted more and more attentions.
Manuscript received September 12, 2015; revised January 13, 2016 and
March 11, 2016; accepted March 16, 2016. Date of publication April 21,
2016; date of current version October 28, 2016. This work was supported
in part by the National Natural Science Foundation of China under Grant
61304193, by the “973” National Basic Research Project of China under Grant
2011CB711204, and by the China Postdoctoral Science Foundation under
Grant 2013M530607. The Associate Editor for this paper was J. E. Naranjo.
J. Guo is with the Department of Mechanical and Electrical Engineering,
Xiamen University, Xiamen 361005, China (e-mail: guojing_0701@live.cn).
P. Hu is with the School of Autonomous Engineering, Dalian University of
Technology, Dalian 116024, China.
R. Wang is with the College of Traffic, Jilin University, Changchun 130012,
China.
Color versions of one or more of the figures in this paper are available online
at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/TITS.2016.2544791
The function of autonomous vehicles can be classified into
two main aspects, known as assistant driving and automatic
driving. Assistant driving, which is devoted to improve safety
and riding comfort, is materialized by the emergence of new
Advanced Driver Assistance Systems (ADAS), and the devices
of ADAS, such as Adaptive Cruise Control (ACC), Forward
Collision Avoidance (FCA) and Lane Departure Warning Sys-
tem (LDWS), are more and more available on the market.
Automatic driving is the highest level of autonomous vehicles,
and it is considered to be one of the toughest challenges in
the exploitation of autonomous vehicles within the field of
intelligent transportation system (ITS).
Automatic driving control system is a crucial component of
autonomous vehicles in ITS, which mainly includes lateral and
longitudinal motion control. The fundamental mission of lateral
and longitudinal control is to automatically and accurately track
the desired trajectory at the set speed while ensuring the safety,
stability and riding comfort of autonomous vehicles [2].
A great deal of practice and research on the lateral motion
control has been done in recent years. A nested PID steer-
ing control architecture with two independent control loops
in vision-based autonomous vehicles is proposed and it can
reject the disturbances on the curvature which increase linearly
with respect to time [3]. In order to simulate human decision
making and analogical reasoning, an intelligent fuzzy steering
control strategy is given in [4] and [5]. Furthermore, an optimal
fuzzy control system is constructed, in which the parameters
of membership functions and rule base are determined by
genetic algorithms [6]. In [7], a nonholonomic single-track
vehicle model in local coordinates is given and a linear steering
control law which can real-time reduce the tracking errors and
avoid unpredictable overshoots is designed. In [8], a real world
application of the lane-guidance technologies is discussed, and
a new low-speed vehicle model that explains the source of the
oscillation is proposed, the corresponding low-order steering
controller is validated and refined through the LMI optimization
synthesis. The input-output feedback linearization method is
applied to the design of automatic steering control system in
[9] and [10], however, the accurate knowledge of the plant
dynamics needs to be known in advance. An active front
steering system is designed by model predictive control (MPC)
theory, and by introducing a constraint on the tire slip angle
which stabilizes the vehicle at high speed, the performances
of the proposed system is enhanced [11]. In [12], an adaptive
fuzzy sliding mode lateral controller is proposed to deal with
1524-9050 © 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
GUO et al.: NONLINEAR COORDINATED STEERING AND BRAKING CONTROL OF VISION AUTONOMOUS VEHICLES 3231
parametric uncertainties and strong nonlinearities, and the as-
ymptotic stability of the closed-loop lateral control system is
proven. Moreover, Mammar et al. [13] design the assistant
steering control system using hybrid automata theory and syn-
thetically composite Lyapunov theory, and the practical imple-
mentation confirms the effectiveness of proposed approach.
The task of longitudinal control for autonomous vehicles
is to track the desired velocity or the desired safe distance
in real-time while maintaining stability and riding comfort. In
[14], a nonlinear cascade longitudinal control system with inner
and outer loops is proposed to ensure safety and comfort of
autonomous vehicles. In the practical implementation of lon-
gitudinal velocity control, sliding mode control technique is a
popular method[15]–[17], but, it is liable to cause the chattering
phenomenon. An intelligent longitudinal vehicle following con-
trol system is developed in [18], and in this control system, the
adaptive output recurrent cerebellar model articulation control
(ORCMAC) is the main tracking controller to mimic an ideal
backstepping control, and the robust controller is utilized to
attenuate the effects caused by lumped uncertainty term. A
vehicle spacing control system using robust fuzzy control with
pole placement in an LMI region for TS model is described
in [19], and the results indicate that the designed control law
is robust enough to reject parametric uncertainties and the
variations of operating conditions (e.g., wind, road surface). A
longitudinal assistance control system including adaptive cruise
control and forward collision warning/avoidance is developed
in [20], which is adaptive to driver behavior, and the parameters
of this presented control system is identified from the data in
the manual operation phase. A novel time-varying parameter
adaptive speed control algorithm is presented to improve the
tracking capability under different working conditions, and the
performance of the proposed control algorithm is validated by
experimental tests [21].
Under the conditions of emergency obstacle avoidance, ve-
hicle lateral and longitudinal dynamics has the strong coupled
and nonlinear characteristics, and the coupled effects mainly
embody in tire forces coupling, load-transfer coupling and
kinematic coupling. In addition, the coupled effects become
increasingly significant as maneuvers involving higher accel-
erations, larger tire forces, or reduced road friction. The perfor-
mance of lateral and longitudinal controllers would be degraded
if the features of coupling and nonlinearities of vehicles are
neglected. Consequently, how to effectively and reasonably
deal with the coupled behaviors between vehicle lateral and
longitudinal dynamics is the emphasis and difficulty of motion
control system design for autonomous vehicles [2].
In this paper, to deal with the coupled and nonlinear features
of autonomous vehicles under the conditions of emergency
obstacle avoidance, an coordinatedsteering and braking control
system for automated driving is proposed. Firstly, a vision
algorithm is constructed to detect the reference path and provide
the local location information in real-time. Then, an adaptive
nonlinear coordinated control strategy is proposed to overcome
the strong nonlinearities and parametric uncertainties, and the
asymptotic convergence of the proposed coordinated control
system is proven by the Lyapunov theory. Finally, experimental
tests manifest that the proposed control system possesses favor-
Fig. 1. Architecture of automatic driving control system.
able tracking performance and enhances the riding comfort and
stability of autonomous vehicles.
The rest of this paper is organized as follows. Section II
gives a particular description of nonlinear vehicle model and a
vision algorithm to provide scene information of autonomous
vehicles. Section III constructs a novel coordinated steering
and braking control strategy. Experimental results of the pro-
posed control architecture under adverse operating conditions
are shown in Section IV. Finally, conclusions are drawn in
Section V.
II. SYSTEM DESCRIPTION
A. System Architecture
Automatic driving control system devotes to achieve better
performance on riding comfort, traffic safety, fuel efficiency
and environmental protection. As shown in Fig. 1, the proposed
automated driving control architecture consists of perception
layer, decision and control layer, and execution layer.
The perception layer includes vision system, radar sensor,
vehicle-2-vehicle (V2V) and vehicle-2-infrastructure (V2I) de-
vices for data collection, feature extraction and information
fusion, and the full scale identification on traffic environment
and vehicle states is achieved in the perception layer [22].
Vision system consists of a stingray CCD camera and a PC-
based central processing system. The camera module provides
a 656 ×492 pixel image at 84 frames per second (FPS), and
the CCD output signal is gathered by the IEEE 1394b card,
besides, the processing time of the presented vision system is
less than 20 ms per frame. V2V and V2I devices at 5.9 GHZ
offers the potential to effectively support wireless data commu-
nications between vehicles and infrastructure based on IEEE
802.11p protocols. The decision and control layer receives the
fused information from the perception layer in real-time and
is responsible for detecting the desired reference geometric
path and regulating the lateral and longitudinal motions of
3232 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, VOL. 17, NO. 11, NOVEMBER 2016
Fig. 2. Vehicle dynamics model.
autonomous vehicles. The execution layer consists of executing
devices and fault-tolerant system which has the actuator fault-
diagnosis and fault-tolerant capabilities. As a consequence, the
proposed automated driving control system has the features as
follows [23].
1) Structure sharing for sophisticated and redundant system
is adopted to improve source effectiveness and lower
total cost.
2) Data information about traffic environment and vehicle
status collecting from multiple sensors is fused.
3) Multi-objective coordinated control system for the lat-
eral and longitudinal motions of autonomous vehicles is
achieved to improve the system performance.
B. Vehicle Dynamics Model
Autonomous vehicle is a nonlinear multivariate system in the
presence of strong coupled and uncertain properties. Since this
paper focuses on studying the coordinated steering and braking
control strategy, the driving input is not considered. The model
is derived under the following assumptions: i) ignore vertical,
roll, and pitch motion; ii) approximate the braking and steering
dynamics as linear first-order systems; iii) discount the effect of
suspension on the tire axels [2]. A simplified nonlinear vehicle
dynamics model with three degrees of freedom (see Fig. 2)
which can be effectively described in terms of longitudinal
velocity, lateral velocity and yaw rate is
˙vx=−fRg−cxv2
x
m+vy˙
ψ+2Cf
vy+lf˙
ψ
mvx
δf+1
m
KbPb
rw
+gsin θ+τ(Δx)
˙vy=−2(Cf+Cr)
mvx
vy−vx+2(Cflf−Crlr)
mvx˙
ψ
+2Cf
mδf−cyv2
y
m+τ(Δy)
¨
ψ=−
2Cfl2
f+Crl2
r
Izvx
˙
ψ−2(Cflf−Crlr)
Izvx
vy
+2Cflf
Iz
δf+τ(Δ ˙
ψ)(1)
where vx,vy,andψrepresent the longitudinal velocity, the lat-
eral velocity and the yaw angle, respectively. mis the total mass
of vehicle. Izis the yaw inertia. lfand lrare the distances of the
front and rear axles from the CG, respectively. cxand cyare the
longitudinal and lateral air resistance coefficients, respectively.
fRis the rolling resistance coefficient. Cfand Crare the
cornering stiffness of the front and rear tires, respectively. rw
is the vehicle radius, θis the road grade, δfis the front wheel
steering angle. Pbdenotes the braking pressure, Kbdenotes
the braking pressure coefficient. τ(Δx),τ(Δy),andτ(Δ ˙
ψ)
denote the external disturbances and uncertainties caused by the
time varying parameters and unmodeled dynamics. Fx,Fy,and
Mzdenote the total forces and moment acting on vehicle. αf
and αrrepresent the tire slip angles. Fxi(i=f,r)and Fyj(j=f,r)
represent the longitudinal and lateral tire forces, respectively.
The simplified vehicle dynamics model (1) can be rewritten
in canonical form
˙vx=f0+g0δf+g1Pb+τ(Δx)
˙vy=f1+g2δf+τ(Δy)
¨
ψ=f2+g3δf+τ(Δ ˙
ψ)(2)
with
f0=−fRg−cxv2
x
m+vy˙
ψ
f1=−2(Cf+Cr)vy
mvx−vx+2(Cflf−Crlr)
mvx˙
ψ−cyv2
y
m
f2=−2Cfl2
f+Crl2
r˙
ψ
Izvx−2(Cflf−Crlr)vy
Izvx
g0=2Cf(vy+lf˙
ψ)
mvx
;g1=Kb
mrw
g2=2Cf
m;g3=2Cflf
Iz
.(3)
Assumption 1: The uncertainties and external disturbances in
the vehicle dynamics model (2) are limited in a certain range,
and there exists known continuous functions ¯τi(i=1, 2, 3)
which satisfy the following inequality conditions
τ(Δx)≤¯τ1(vx,v
y,˙
ψ)
τ(Δy)≤¯τ2(vx,v
y,˙
ψ)
τ(Δ ˙
ψ)≤¯τ3(vx,v
y,˙
ψ).(4)
Steering and braking actuators are modeled as linear first
order systems using the recursive least-square identification
method, here, the transfer function models of steering and
braking actuators are established as
G1(s)= δf
δfd
=M1
M2s+M3
(5)
G2(s)= Pb
Pbd
=N1
N2s+N3
(6)
where δfd is the desired front wheel steering angle, Pbd is the
desired braking pressure. M1,M2,M3and N1,N2,N3are the
system parameters.
Due to the multiple driving requirements and dynamic co-
operation of various components of autonomous vehicles, the
technology of coordinated steering and braking control under
the condition of emergency obstacle avoidance needs to be
GUO et al.: NONLINEAR COORDINATED STEERING AND BRAKING CONTROL OF VISION AUTONOMOUS VEHICLES 3233
Fig. 3. Geometric relationships of vehicle and trajectory.
studies. The task of longitudinal braking control is to guarantee
the vehicles automatically and smoothly achieve the desired
speed/acceleration by adjusting the braking pressure according
to the specified control strategy. Given a desired velocity vpand
an actual velocity vx, the time derivative of velocity tracking
error veis defined as
˙ve=˙vx−˙vp(7)
The basic principle of steering control is to ensure the
autonomous vehicles accurately track the planned reference
trajectory, as shown in Fig. 3. The vision system can capture
the real-time road scene and then determines the angular and
lateral errors. In this paper, angular error ϕeis shaped by the
vehicle centerline and the tangent of reference trajectory, lateral
error yeis the horizontal distance between the vehicle position
and the reference trajectory at a look-ahead distance DL.The
evolution of the measurements can be described as [6], [24]
˙ϕe=vxKL−˙
ψ
˙ye=vxϕe−vy−˙
ψDL(8)
where KLis the road curvature.
C. Vision Algorithm
Accurate and intact traffic environment information plays
an important role that ensures the automatic driving control
system of vehicles achieve the desired dynamic performance.
With visual data that was grabbed from a single camera that
is mounted on the roofline, the real-time vision system is
mainly capable of estimating the vehicle location relative to the
desired trajectory. Here, the proposed vision algorithm consists
of five stages, the first few stages are responsible for trajectory
detection, whereas the last stage realizes the curve fitting of the
desired trajectory. The process of proposed vision algorithm is
designed as follows.
Image Filter and Enhancement: Owing to the impact of
surrounding background, images contain amount of noise dur-
ing the process of generation and transmission. Firstly, the
Gaussian filter method is adopted to reduce the influence of
these noise disturbances. Then, a local contrast enhancement
algorithm is adopted to effectively improve the whole or partial
characteristics of image. For the point (x, y)in the image, the
implementation scheme of image enhancement is given as
1) Calculating the histogram equalization in a rectangular
region with the center of point (x, y)as
pW(rk)= nk
W2(9)
2) Establishing the cumulative distribution function PW(·)as
PW(rk)=
k
i=0
pW(i)(10)
3) Achieving the following gray transformation as
T(f(x, y)) = 255PW(f(x, y)) (11)
where nkis the number of pixels that have gray-scale
θk,T(·)is the gray transform function. The size of
rectangular region Wis the only one control parameter
in the local area histogram equalization.
Edge Detection: In order to obtain the edge information of
shooting environment, Canny’s edge detection is proposed and
the corresponding algorithm process is given as follows.
1) Using a Gaussian filter to eliminate the noise.
2) Employing the 3 ×3 Sobel operator to calculate the
gradient values (Gx,G
y)of the input image as shown
in equations (12) and (13), the gradient magnitudes is
calculated as equation (14), and the direction of the edges
is determined as equation (15).
Gx(x, y)= {f(x+1,y−1)+2f(x+1,y)+f(x+1,y+1)}
−{f(x−1,y −1)+2f(x−1,y)+f(x−1,y+1)}
(12)
Gy(x, y)= {f(x−1,y+1)+2f(x, y +1)+f(x+1,y+1)}
−{f(x−1,y −1)+2f(x,y−1)+f(x+1,y −1)}
(13)
G=G2
x+G2
y(14)
θ=Arc tan Gy
Gx(15)
3) Applying the non-maximum suppression to suppress any
pixel value that is not considered to be an edge.
4) Applying the double thresholding method to determine
potential edges.
Contour Extraction: Edge images contain not only the infor-
mation of the reference path, but also a number of little non-
path. Usually, the reference path has the features of continuous
and tenuous contour, however, most of the interference sources
do not possess these features, as a consequence, the edge points
of non-path can be removed based on the different profile
features of path and non-path, here, a 8-neighbour contour
extraction method is proposed to obtain the profile of each edge
3234 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, VOL. 17, NO. 11, NOVEMBER 2016
chain [25]. After all the edges are found, the contour described
by each edge chain is distinguished by the following principle.
1) Counting the number of edge points at each contour, the
contour in which the number of edge points is below a set
threshold is reviewed as non-path.
2) Calculating the envelop rectangle of the remaining con-
tour, the contour that has the long and narrow rectangle is
reviewed as non-path, and vice versa.
Morphology Processing: Due to the external disturbances,
the edge information of reference path obtained by the above
contour extraction is often intermittent, hence, mathematical
morphology is applied to fill the gap among the interrupted edge
points, here, the dilation and erosion operations are carried out
for the same structuring element of edge images and given as
(f⊕b)(s, t)=max{f(s−x, t −y)+b(x, y)|s−x,
t−y∈Df,x+y∈Db}
(fΘb)(s, t)=min{f(s+x, t +y)−b(x, y)|s+x,
t+y∈Df,x+y∈Db}.(16)
The entity of the path can be highlighted by the dilation and
erosion operations, and the location of path in images is not
changed. The inner edge points of the left or right reference
path can be detected from the center line to the both sides of
processed images.
Model Fitting: In order to obtain the geometric model which
can accurately describes the features of reference path, the
model fitting is achieved by the least square method, assuming
asimpledataset(xi,y
i)consists of mpoints in the image, then
the geometry model of reference path is usually in the form of
a polynomial such as
y(x)=
n
j=0
cjϕj(x).(17)
The goal of the problem is to seek for the values of c0,c
1,...,
cnsuch that the sum of square errors is minimized, it can be
written as
s=
m
i=0
(y(xi)−yi)2.(18)
Fig. 4 indicates the extracted results of the proposed vision
algorithm under different working conditions. It is interesting
to note that the fitted curve model is in good coincidence
with the actual reference trajectory under different illumination
conditions. Meanwhile, the results manifest that the proposed
vision algorithm provides a powerful guarantee to supply real-
time location information between the vehicle and the reference
trajectory for the follow-up coordinated steering and braking
control system.
III. NONLINEAR COORDINATED STEERING
AND BRAKING CONTROLLER
Under the conditions of emergency obstacle avoidance, the
steering and braking dynamics of autonomous vehicles have
the strong coupled, nonlinear and parametric uncertain features,
Fig. 4. Extracted results in different working conditions.
Fig. 5. Block diagram of coordinated steering and braking control strategy.
and the performance of steering and braking motion controllers
would be degraded if these characteristics are neglected. How to
effectively and reasonably deal with the nonlinear and coupled
behaviors between vehicular steering and braking dynamics
is the emphasis and difficulty of automatic driving control
system design for vehicles. In this section, As shown in Fig. 5,
a coordinated steering and braking control system based on
the nonlinear backstepping control theory and the adaptive
fuzzy sliding mode control (FSMC) technique is constructed to
guarantee uniformly ultimately bounded and global asymptotic
stability of close loop system, and a major advantage of the
proposed control strategy is that it has the greater flexibility to
pursue the multi-objective control performances and effectively
overcome the parametric uncertainties and nonlinearities [12].
A. Nonlinear Backstepping Equivalent Control Strategy
Nonlinear steering and braking coupled dynamics model of
autonomous vehicles can be yielded by combining equations
(2) and (7), (8), this vehicle dynamics model consists of six
state variables and two input variables such as Pband δf,
which has semi-strict feedback form in the presence of external
disturbances and parametric uncertainties. To deal with these
features, a coordinated steering and braking equivalent control
strategy based on nonlinear backstepping control technique is
designed as follows.
GUO et al.: NONLINEAR COORDINATED STEERING AND BRAKING CONTROL OF VISION AUTONOMOUS VEHICLES 3235
Step 1: The first error vector s1is defined from the lateral
error as
s1=ye.(19)
Choosing the Lyapunov function as Vlat0 =1/2s2
1,andthe
time derivative of Vlat0 is obtained as
˙
Vlat0 =s1˙s1=s1˙ye=s1(vxϕe−vy−˙
ψDL).(20)
In equation (20), viewing the term vxϕe−˙
ψDLas the virtual
control input, and the condition for which yetends towards zero
is that ˙
Vlat0 must be negative definite such that
˙
Vlat0 =−k1s2
1≤0 (21)
where k1is a positive constant. Thus, the desired virtual control
input α1can be obtained as
α1=−k1s1+vy.(22)
Defining the difference between the virtual control input
vxϕe−˙
ψDLand its desired value α1to be the second error
variable s2, and it is given by
s2=vxϕe−˙
ψDL−α1.(23)
Substituting equation (23) into equation (20), yields
˙
Vlat0 =s1s2−k1s2
1.(24)
Obviously, when s2=0, ˙
Vlat0 =−k1s2
1≤0 is satisfied. The
target of next step is to search the control input variables Pb
and δfwhich can ensure the error variable s2converge to zero
or a small value. As a consequence, the error variable s1is
guaranteed to asymptotically converge to zero or be uniformly
ultimately bounded.
Step 2: Choosing the Lyapunov function as
Vlat1 =Vlat0 +1
2s2
2.(25)
The time derivative of equation (25) can be obtained as
˙
Vlat1 =˙
Vlat0 +s2˙s2=−k2s2
1+s1s2+s2˙s2.(26)
Let
−k2s2=s1+(f0+g0δf+g1Pb)ϕe+˙ϕevx
−(f2+g3δf)DL+k1˙s1−(f1+g2δf)+η1.(27)
Substituting equation (27) into equation (26), thus
˙
Vlat1 =−k1s2
1−k2s2
2≤0 (28)
where k2is a positive constant, η1is an uncertain term that
caused by the time derivative of the error variable s2.
Based on the assumption 1, there exists known continuous
positive function β1(vx,v
y,˙
ψ)which satisfies
η1≤β1(vx,v
y,˙
ψ).(29)
Step 3: Considering the longitudinal braking process of
autonomous vehicles, the first error vector is defined as
p1=ve.(30)
Choosing the Lyapunov function as
Vlog it0=1
2p2
1.(31)
The time derivative of equation (31) is obtained as
˙
Vlog it0=p1˙p1=p1˙ve=p1(˙vx−˙vp).(32)
The condition for which p1tends toward zero is that ˙
Vlog it0
must be negative definite such that
˙
Vlog it0=−l1p2
1≤0.(33)
Let
f0+g0δf+g1Pb+ξ1−˙vp=−l1p1(34)
where l1is a positive constant, ξ1is an uncertain term which
caused by the time derivative of p1.
Based on the assumption 1, there exists known continuous
positive function γ(vx,v
y,˙
ψ), which satisfies
ξ1≤γ(vx,v
y,˙
ψ)(35)
Combining equation (27) and equation (34), the equivalent
control input can be obtained as
ueq =Pbeq
δfeq=g1g0
g1ϕe(g0ϕe−g2−g3DL)−1σ1
σ2
(36)
with
σ1=−f0+˙vp−l1p1−p1γ2
2ς1
σ2=−s1−f0ϕe−˙ϕevx+f2DL+f1−k1˙s1−k2s2−s1β2
2ε1
(37)
where −(p1γ2/2ς1)and −(s2β2/2ε1)are nonlinear damp-
ing terms to compensate for the disturbances caused by the
parametric uncertainties, and they exhibit lower gains at small
tracking errors to enhance the riding comfort of vehicles and
higher gains at large tracking errors to improve the safety of
vehicles, both ς1and ε1are the positive constants.
Autonomous vehicles have the characteristics of parametric
uncertainties and external disturbances. In order to restrain the
influence of these uncertainties and disturbances, the variable
structure reaching law is designed as follows:
us=Pbs
δfs=λ1sign(p1)
λ2sign(s2)(38)
where λ1and λ2are the positive constants, respectively, and
sign(·)is the sign function.
Remark 1: The reaching law (38) is discontinuous across the
sliding mode hyperplane, thus it will cause the high frequency
chattering phenomena near the sliding hyperplane.
B. Adaptive Fuzzy Sliding Mode Reaching Law
In order to deal with the chattering problem caused by the
sliding mode control law (38), an adaptive fuzzy sliding mode
control scheme is proposed, and the two-input single-output
3236 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, VOL. 17, NO. 11, NOVEMBER 2016
TAB L E I
RULE BASE
fuzzy logic control systems used for tuning reaching phase
instead of sign function are constructed in the variable structure
reaching control part. The main advantage of this method is that
the robust behavior of the system is guaranteed. The second
advantage of the proposed scheme is that the performance of
the system in the sense of removing chattering is improving
in comparison with the same sliding mode control technique
without using fuzzy logic control [26]. The fuzzy variable
structure reaching law is rewritten as follows:
us=Pbs
δfs=λ1uFSMC(p1,˙p1)
λ2uFMSC(s2,˙s2)(39)
In the fuzzy variable structure reaching control law (39),
both of the fuzzy logic control systems taking the place of
sign function are modeled with two input variables and one
output variable. For the reaching control law of steering part,
the two input variables are the sliding signal s2(t)and the rate
of change of sliding signal ˙s2(t), respectively, two trapezoidal
and five triangular membership functions are defined to depict
each input variable, and seven single membership functions are
defined to describe output variable. All membership functions
are decomposed into seven fuzzy partitions expressed as posi-
tive small (PS), positive medium (PM) and positive big (PB),
zero (ZE), negative big (NB), negative medium (NM), negative
small (NS).
As seen in Table I, the rule base of fuzzy control system
consists of 49 rules and represents as the mapping of the input
and output linguistic variables, which can be defined heuristi-
cally in the following format:
R(i):if s2(t)is Ei
1and ˙s2(t)is Ei
2then uFSMC(s2,˙s2)is Fi
where Ei
1,Ei
2,andFiare the corresponding linguistic terms of
the input and output fuzzy sets. i=1, 2,...,49 is the number
of the fuzzy if-then rule. For instance, a sample fuzz rule is
given as
if s2(t)is negative big (NB) and ˙s2(t)is negative big (NB)
then uFSMC(s2,˙s2)is positive big (PB)
It could be comprehended as the system states are below the
sliding hyperplane and are moving away from it, therefore, in
order to make the system states return to the sliding hyperplane
quickly, the control action uFSMC (s2,˙s2)should be PB.
The fuzzy inference is carried out by the Mamdani operator,
and the implement of defuzzifier is utilized by the center of
gravity method. For the regulation of braking dynamics part,
the design flow of fuzzy logic system to take the place of sign
function in reaching law (39) is same as above. Consequently,
the total coordinated steering and braking control law can be
expressed as
u=Pbd
δfd=ueq +us.(40)
Theorem 1: Consider the closed-loop system consisting of
vehicle dynamics (2) and (7), (8) with the coordinated steering
and braking controller (40), all signals in the closed-loop system
are bounded, and the tracking errors asymptotically converge to
zero.
Proof: With regard to the regulation of lateral dynamics,
the following nonlinear steering control law can be obtained
from the coordinated control law (40) as
u=Pbd
δfd=Pbeq +λ1uFSMC(p1,˙p1)
δfeq +λ2uFSMC(s2,˙s2).(41)
As seen in equation (36), it is interesting to note that the
equivalent control terms of front steering angle δfeq and brak-
ing pressure Pbeq satisfy the following equality as:
(f0+g0δfeq +g1Pbeq )ϕe+˙ϕevx−(f2+g3δfeq)DL
+k1˙s1−(f1+g2δfeq)=−s1−k2s2−s2β2
2ε1
.(42)
Defining Lyapunov function as
Vlat =Vlat1 +1
2˜e2
y(43)
where ˜eyis the inevitable measurement error due to the lack of
light and signal blockage of vision system, and it is assumed to
be bounded as
|˜ey||˙
˜ey|≤ν|s2|(44)
where ˙
˜eyis the time derivative of inevitable measurement error,
and νis the positive constants.
The time derivative of equation (23) is substituted into
the equation (26), and then the following equality can be
obtained as:
˙
Vlat =−k1s2
1+s1s2+s2˙s2+˜ey˙
˜ey
=−k1s2
1+s1s2+s2(˙vxϕe+˙ϕevx−¨
ψDL−˙a1)+ ˜ey˙
˜ey
=−k1s2
1+s1s2+s2
×((f0+g0δf+g1Pb)ϕe+˙ϕevx−(f2+g3δf)DL
+k1˙s1−(f1+g2δf)+η1)+ ˜ey˙
˜ey.(45)
Let δf=δfd and Pb=Pbd, then, substituting the equivalent
control law (36) and the fuzzy reaching law (39) into the above
GUO et al.: NONLINEAR COORDINATED STEERING AND BRAKING CONTROL OF VISION AUTONOMOUS VEHICLES 3237
equation (45), therefore, the equation (45) can be rewritten as
˙
Vlat =−k1s2
1+s1s2+s2˙s2+˜ey˙
˜ey
=−k1s2
1+s1s2+s2−s1−k2s2−s2β2
2ε1
+η1+˜ey˙
˜ey
+s2(g1ϕePbs +(g0˙ϕe−g2−g3DL)δfs)
=−k1s2
1−k2s2
2−s2
2β2
2ε1
+η1s2+s2λ1g1ϕeuFSMC(p1,˙p1)
+s2λ2(g0˙ϕe−g2−g3DL)uFSMC(s2,˙s2)+˜ey˙
˜ey.
(46)
Since η1≤β, the polynomial term −(s2
2β2/2ε1)+η1s2can
be rewritten as
−s2
2β2
2ε1
+η1s2≤−s2β
√2ε1−√ε1
√22
+ε1
2≤ε1
2(47)
Assuming k2>k
1,let
κ=λ2(g0ϕe−g2−g3DL)+λ1g1ϕe.(48)
The output of fuzzy logic system are normalized in the inter-
val (−1, 1), then |uFSMC (p1,˙p1)|≤1and|uFSMC(s2,˙s2)|≤1,
and the equation (44) can be rewritten as
˙
Vlat ≤−k1s2
1−k1s2
2−(k2−k1)s2
2+ε1
2+κ|s2|+ν|s2|
≤−2k1Vlat −(k2−k1)s2
2+ε1
2+(κ+ν)|s2|
≤−2k1Vlat −k2−k1|s2|− (κ+ν)
2√k2−k12
+ε1
2−(κ+ν)2
4(k2−k1)
≤−2k1Vlat +ε1
2.(49)
Consequently, limt→∞ Vlat ≤ε1/4k1, the control error vec-
tor s1is uniformly ultimately bounded. Similarly, the error
vector p1is uniformly ultimately bounded, proof is the same
as above.
Remark 2: In order to effectively eliminate the chattering
phenomenon and overcome the parametric uncertainties and
external disturbances, a fuzzy logic system uFSMC(p1,˙p1)can
be used to completely replace the sign function sign(p1).
IV. FIELD EXPERIMENTS AND DISCUSSIONS
To confirm the performance of the coordinated steering and
braking control system, both simulation and experimental tests
which show the behaviors of the proposed control system are
implemented, and the correspondingprototype vehicle is called
Tiggo automated vehicle.
Firstly, the robustness of the coordinated control strategy
against model uncertainties and disturbance is verified by
simulation. The external disturbance is assumed as a random
process, and the uncertain parameters of tire stiffness are
changed from Cf=Cr=50 KN to Cf=Cr=20 KN in test.
Fig. 6. Simulation results.
Fig. 7. Reference path in experimental test I.
The reference trajectory is straight, besides, the initial lateral
and angular errors are set to 0.1 m and 0.04 rad, respectively.
Fig. 6 shows the simulation results of proposed control system
in the different working condition. It can be seen that the pro-
posed system has strong robustness and high control accuracy
with system uncertainties and disturbances.
Furthermore, experimental tests are carried out. To study
the contribution of proposed controller, the dynamic behaviors
of the proposed control system are analyzed and compared
with the uncoordinated control system, which consists of a
linear time varying steering controller [9] and a sliding mode
longitudinal braking controller [15].
As shown in Fig. 7, the reference trajectory used in the
experimental test I is consisted of several curve segments with
different curvature radius, besides, Fig. 7 shows the desired
velocity of autonomous vehicle in braking case, at first, the
vehicle runs at a constant velocity of 90 km/h, then, it begins
to decelerate since 40 m, in the final stage, it returns to run at
a uniform velocity. Fig. 7 manifests that the longitudinal and
lateral coupled and nonlinear dynamic features are occurred
in experimental test I. The initial lateral, angular and velocity
errors are set to 0.3 m, 0.05 rad, and −11 km/h, respectively.
A series of dynamic behaviors of the proposed coordinated
control system and the LTV+SMC uncoordinated system are
depicted in Fig. 8. Fig. 8(a) describes the response results of
3238 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, VOL. 17, NO. 11, NOVEMBER 2016
Fig. 8. Response results of experimental test I.
lateral error, it can be seen that the maximum steady-state
lateral error of the proposed control system and the LTV+SMC
control system are bounded to ±0.1 m and ±0.2 m, respec-
tively, which occurs in the tough road with largest curvature
Fig. 9. Reference path in experimental test II.
of 0.015 m−1. Besides, the overshoot of lateral error controlled
by the proposed control system is lower than the LTV+SMC
control system. Fig. 8(b) shows the response results of angu-
lar error, it is clear that both the angular errors of proposed
control system and LTV+SMC control system are limited and
their maximum steady-state values are with in ±0.05 rad and
±0.1 rad, respectively. Fig. 8(a) and (b) manifest that the
proposed control algorithm has less overshoot and smaller
oscillation than the LTV+SMC control system.
Fig. 8(c) shows the response results of longitudinal velocity,
it is interesting to note that the response curve of velocity
for the coordinated control system basically coincided with
the desired values. But, the response of longitudinal velocity
for the LTV+SMC control system has a certain deviation,
and the deviation is increased with the variations of path
curvature.
Fig. 8(d) shows the comparison results of yaw rate, it can be
observed that both the control strategies can ensure the yaw rate
limit in a preconcert range, but the oscillation frequency of the
LTV+SMC control system is enhanced obviously, which will
make passengers uncomfortable. Consequently, compared with
the LTV+SMC control strategy, the proposed control strategy
can effectively decrease the oscillations and improvethe control
accuracy.
The contrasting results of corresponding front wheel steering
angle and brake pressure are shown in Fig. 8(e) and (f), respec-
tively. It is worth noting that the control inputs of the proposed
coordinated control strategy are smoother than the LTV+SMC
control strategy.
The reference trajectory and velocity of autonomous vehicles
used in the experimental test II are shown in Fig. 9. The initial
lateral, angular and velocity errors are set to 0.4 m, −0.05 rad,
and 4 km/h, respectively.
Fig. 10(a) and (b) describe the response results of lateral and
angular errors, it can be seen that the tracking accuracy of the
proposed control system is better than the LTV+SMC system.
The maximum steady-state lateral error of the proposed control
system is bounded to ±0.15 m, and the maximum steady-
state angular error of the proposed control system is limited in
±0.05 rad.
Fig. 10(c) shows the response results of longitudinal velocity,
it is worth noting that the proposed control system not only
ensure the steady-state velocity error converge to zero, but also
reject the adverse effects of the variations of path curvature.
Nevertheless, the robustness of the LTV+SMC control strategy
is relatively weaker.
GUO et al.: NONLINEAR COORDINATED STEERING AND BRAKING CONTROL OF VISION AUTONOMOUS VEHICLES 3239
Fig. 10. Response results of experimental test II.
Fig. 10(d) indicates that the proposed control strategy could
effectively deal with the nonlinear features and take advantage
of the interactions between the steering and braking dynamics
to improve the riding comfort and stability of autonomous
vehicles. The contrasting results of front wheel angle and brake
pressure are shown in Fig. 10(e) and (f), respectively.
The comparative experiment results exhibited in this section
manifest that the proposed coordinated control strategy not only
significantly improves the control accuracy and yields transient
performances, but also can enhance the riding comfort, stability
and safety of autonomous vehicles.
V. C ONCLUSION
This paper has presented a novel automated driving control
system for the coordinated management of steering and braking
dynamics of vision-based autonomous vehicles, which is aimed
to effectively improve the safety and riding comfort properties.
The vision algorithm consisting of five stages is designed
to real-time detect the desired path and provide the relative
location information between the autonomous vehicle to the
reference path.
Additionally, aiming at the coupled and nonlinear features of
autonomous vehicles in the conditions of emergency obstacle
avoidance, a nonlinear coordinated steering and braking control
system consisting of a backstepping equivalent control law and
a fuzzy sliding mode reaching control law is constructed, and
the two-input single-output fuzzy logic control systems used
for tuning reaching phase take the place of sign function in the
reaching control law.
Furthermore, the overall proposed control system has been
implemented on a prototype autonomous vehicle, and the re-
sults from the simulation and experimental tests demonstrate
that the proposed control strategy possesses better tracking
performances and enhances the riding comfort and stability of
autonomous vehicles, even under adverse driving conditions.
REFERENCES
[1] K. G. Akram, N. Farnoosh, and M. Viliam, “Highway accident modeling
and forecasting in winter,” Transp. Res. A, Policy Pract., vol. 59, no. 1,
pp. 384–396, Jan. 2014.
[2] J. Guo, K. Li, and Y. Luo, “Coordinated control of autonomous four drive
electric wheels for platooning and trajectory tracking using a hierarchi-
cal architecture,” ASME J. Dyn. Syst. Meas. Control, vol. 137, no. 10,
pp. 1–18, Oct. 2015.
[3] R. Marino, S. Scalzi, and M. Netto, “Nest PID steering control for lane
keeping in autonomous vehicles,” Control. Eng. Pract., vol. 19, no. 12,
pp. 1459–1467, Sep. 2011.
[4] K. Erdal, K. Erkan, R. Herman, and K. Okyay, “Towards agrobots: Tra-
jectory control of an autonomous tractor using type-2 fuzzy logic con-
trollers,” IEEE/ASME Trans. Mechatronics, vol. 20, no. 1, pp. 2912–2924,
Feb. 2015.
[5] J. Perez, V. Milanes, and E. Onieva, “Cascade architecture for lat-
eral control in autonomous vehicles,” IEEE Trans. Intell. Transp. Syst.,
vol. 12, no. 1, pp. 73–82, Mar. 2011.
[6] J. Guo, P. Hu, L. Li, and R. Wang, “Design of automatic steering con-
troller for trajectory tracking of unmanned vehicles using genetic al-
gorithms,” IEEE. Trans. Veh. Technol., vol. 61, no. 7, pp. 2913–2924,
Sep. 2012.
[7] S. Kammel, J. Ziegier, and B. Pitzer, “Team Annie WAY’s autonomous
system for the 2007 DARPA Urban Challenge,” J. Field Robot., vol. 25,
no. 9, pp. 615–639, Aug. 2007.
[8] H. S. Tan, F. Bu, and B. Bougler, “A real-world application of
lane-guidance technologies automated snowblower,” IEEE Trans. Intell.
Transp. Syst., vol. 8, no. 3, pp. 538–548, Sep. 2007.
[9] J. Huang and M. Tomizuka, “LTV controller design for vehicle lateral
control under fault in rear sensors,” IEEE/ASME Trans. Mechatronics,
vol. 10, no. 1, pp. 1–7, Feb. 2005.
3240 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, VOL. 17, NO. 11, NOVEMBER 2016
[10] R. Rajamani, C. Zhu, and L. Alexander, “Lateral of a backward driven
front-steering vehicle,” Control. Eng. Pract., vol. 11, no. 5, pp. 531–540,
May 2003.
[11] P. Falcone, F. Borrelli, J. Asgari, and H. Tseng, “Predictive active steer-
ing control for autonomous vehicle systems,” IEEE Trans. Control. Syst.
Technol., vol. 15, no. 3, pp. 566–580, May 2007.
[12] J. Guo, L. Li, and K. Li, “An adaptive fuzzy sliding lateral control strat-
egy of automated vehicles based on vision navigation,” Veh. Syst. Dyn.,
vol. 51, no. 10, pp. 1502–1517, Jun. 2013.
[13] N. Enache, S. Mammar, M. Netto, and B. Lusetti, “Driver steering assis-
tance for lane-departure avoidance based on hybrid automata and com-
posite Lyapunov function,” IEEE Trans. Intell. Transp. Syst., vol. 11,
no. 1, pp. 28–39, Mar. 2010.
[14] A. Rachid, O. Rodolfo, and B. Michel, “Nonlinear cascade strategy
for longitudinal control in autonomous vehicle guidance,” Control. Eng.
Pract., vol. 29, no. 8, pp. 225–234, Mar. 2014.
[15] L. Nouveliere and S. Mammar, “Experimental vehicle longitudinal con-
trol using a second order sliding mode technique,” Control. Eng. Pract.,
vol. 15, no. 8, pp. 934–954, Aug. 2007.
[16] A. Ferrara and C. Vecchio, “Second order sliding mode control of vehicles
with distributed collision avoidance capabilities,” Mechatronics, vol. 19,
no. 4, pp. 471–477, Jun. 2009.
[17] P. Hu, J. Guo, L. Li, and R. Wang, “A robust longitudinal sliding-mode
controller design for autonomous ground vehicle based on fuzzy logic,”
Int. J. Veh. Auton. Syst., vol. 11, no. 4, pp. 368–383, 2013.
[18] Y. F. Peng, “Adaptive intelligent backstepping longitudinal control of
vehicle platoons using output recurrent cerebellar model articulation con-
troller,” Expert. Syst. Appl., vol. 37, no. 3, pp. 2016–2027, Mar. 2010.
[19] P. F. Toulotte, S. Delprat, T. M. Guerra, and J. Boonaert, “Vehicle spacing
control using robust fuzzy control with pole placement in LMI region,”
Eng. Appl. Artif. Intel., vol. 21, no. 5, pp. 756–768, Nov. 2008.
[20] J. Wang, L. Zhang, D. Zhang, and K. Li, “An adaptive longitudinal driving
assistance system based on driver characteristics,” IEEE Trans. Intell.
Transp. Syst., vol. 14, no. 1, pp. 1–12. Mar. 2013.
[21] K. Hakgo, K. Dongwook, and S. Insoo, “Time-varying parameter adap-
tive vehicle speed control,” IEEE Trans. Veh. Technol., vol. 65, no. 2,
pp. 581–588, Feb. 2015.
[22] J. Wahlstrom, I. Skog, and P. Handel, “Detection of dangerous cornering
in GNSS-data-driven insurance telematics,” IEEE Trans. Intell. Transp.
Syst., vol. 25, no. 6, pp. 3073–3083, Dec. 2015.
[23] K. Li, C. Tao, and Y. Luo, “Intelligent environment friendly vehicles
concept and case studies,” IEEE Trans. Intell. Transp. Syst., vol. 13,
no. 1, pp. 318–328, Mar. 2012.
[24] S. Wu, H. Chiang, and J. Perng, “The heterogeneous systems integration
design and implementation for lane keeping on a vehicle,” IEEE Trans.
Intell. Transp. Syst., vol. 9, no. 2, pp. 246–263, Jun. 2008.
[25] C. Gonzatez and E. Wood, Digital Image Processing, 3rd ed. Upper
Saddle River, NJ, USA: Prentice-Hall, 2007.
[26] F. Ahmed, A. Elsayed, and M. Wael, “Adaptive fuzzy sliding mode control
using supervisory fuzzy control for a 3DOF planar robot manipulators,”
Appl. Soft Comput., vol. 11, no. 8, pp. 4943–4953, Dec. 2011.
Jinghua Guo received the Ph.D. degrees from
Dalian University of Technology, Dalian, China,
in 2012.
From 2012 to 2015, he finished his postdoctoral
research with Tsinghua University, Beijing, China.
He is currently an Assistant Professor with Xiamen
University, Xiamen, China. He has authored more
than 20 journal papers. He has engaged in more
than five sponsored projects. His research interests
include intelligent vehicles, vision system, control
theory, and applications.
Ping Hu received the B.E. degree in mathematics,
the M.E. degree in mechanics, and the Ph.D. degree
in computing mechanics from the previous Jilin Uni-
versity of Technology, Changchun, China, in 1982,
1984, and 1993, respectively.
He was with Jilin University of Technology,
where he was an Associate Professor in 1991 and a
Professor in 1993. He is currently a Professor with
the Department of Automobile Engineering, Dalian
University of Technology, Dalian, China. His re-
search interests include automotive engineering and
mechanics.
Rongben Wang received the B.E., M.E., and Ph.D.
degrees from Jilin University of Technology, Jilin,
China, in 1970, 1991, and 1995, respectively.
He is a Full Professor with the College of Traf-
fic, Jilin University. He has authored more than 80
journal papers. He has been responsible for several
projects for National Natural Science Foundation of
China and National Aerospace Exploration Program
of China. His research interests include automatic
guided vehicles, computer vision systems, and image
processing.
