Particle swarm optimization based LQ-servo controller for congestion avoidance

Abstract

As an effective mechanism acting on the intermediate nodes to support end-to-end congestion control, Active Queue Management (AQM) takes a trade-off between link utilization and delay experienced by data packets. In this paper, a linear quadratic optimal controller was designed based on linear control theory for TCP/AQM router. The design specifications depend on choosing weighting matrices Q and R, Weighting matrices (Q and R) of the linear quadratic performance index are tuned the desired step response acquired, by using Particle Swarm Optimization (PSO) that minimizes a performance criterion The controller simulation results show the efficiency of the proposed controller.

Full text

63
Particle Swarm Optimization Based LQ-Servo
Controller for Congestion Avoidance
Sana S. Sabry1, Thaker M. Nayl2
1 college of engineering, University of information technology and communications, Baghdad, Iraq
2 college of engineering, University of information technology and communications, Baghdad, Iraq
sana.sabah@uoitc.edu.iq, thaker.nayl@uoitc.edu.iq
Abstract— The network congestion is an essential problem that leads to packets
losing and performance degradation. Thus, preventing congestion in the
network is very important to enhance and improve the quality of service. Active
queue management (AQM) is the solution to control congestion in TCP network
middle nodes to improve theire performance. We design a linear quadratic
(LQ)-servo controller as an AQM applied to TCP network to control congestion
and attempt to achieve high quality of service under dynamic network
environments. The LQ-servo controller is proposed to provide queue length
stabilization with a small delay and faster settling time. The designed controller
parameters are tuned by using the particle swarm optimization (PSO) method.
The PSO algorithm was fundamentally applied to find the optimal controller
parameters Q and R, such that a good output response could be obtained. The PI
controller is examined for comparison reasons. The MATLAB simulation result
shows that the controller is more effective than the PI in reaching zero steady-
state error with better congestion avoidance under the dynamic network
environment. Moreover, the proposed controller achieves a smaller delay and
faster settling time.
Index Terms— AQM, Network Congestion control, LQ-servo, Particle Swarm Optimization.
I. INTRODUCTION
With the fast evolution of communication networks and the internet, the large amount of generated
data requires a reliable and effective data transmission to avoid network congestion. Thus, preventing
congestion in the network is very important to enhance and improve the quality of service.
Transmission control protocol is one of the most widespread transport protocols which provides an
end to end congestion control mechanism [1]. In this protocol, the average congestion window size
increases when the packets are delivered to the receiver. Conversely, the average congestion window
size decreases during an unsuccessful data transition. However, this mechanism has significant
disadvantages such as poor utilization for the network resources and global stream synchronization [2].
To avoid these disadvantages, AQM has been proposed at gateways to improve their performance.
Several types of AQM in recent years have been proposed to control congestion in TCP networks. in
[3] random early detection (RED) was proposed which randomly dropped the packet at specific
probability before the queue buffer overflows. Then Misra in [4] obtained numerically transient
behaviour of AQM/TCP routers. This modelling has a great role in understanding and analyzing
different network congestion avoidance schemes. In [5], the linearization was used by Hollot et al. to
analyze the previously obtained non-linear model for the AQM /TCP routers. A PI controller was
proposed in [6] based on the linearized model and the control theory, the controller displayed better
properties compared to the RED controller. In [7], a linear quadratic (LQ)-servo controller was

64
proposed and the controller parameter was chosen by trial and error. In [8], an adaptive AQM based
neural network was developed and the proposed algorithm showed a good performance.
All the AQM proposed above do not control the congestion of a dynamic network environment,
where the parameters of the network change continuously.
In this paper, an improved AQM based on PSO- LQ controller is designed to control congestion and
attempt to achieve high quality of service under the various network environments.
The paper contributions are as follows:
 An LQ-servo controller is proposed to provide queue length stabilization with a small delay
and faster settling time.
 The LQ-servo controller is compared with the PI controller.
 A PSO algorithm is applied to tune the controller parameter.
 The steadiness of the controller is examined under different network environments.
This paper is structured as follows: section II describes the linearized AQM model, while the linear
quadratic controller is discussed in section III. Afterwards, section IV gives an overview about the
practical swarm optimization method. Section V shows the MATLAB simulation results. Finally,
section VI discusses the findings and conclusions of this research.
II. TCP/ AQM SYSTEM DYNAMICS
A. TCP Model
The dynamic behaviour of the TCP model has been developed by [4] using stochastic differential
equation analysis and fluid flow, and it is expressed by:
CtN
tr tW
tq
trtp
trtr trtWtW
tr
tW







)(
)( )(
)(
))((
))((2 ))(()(
)(
1
)( (1)
Where W is the congestion window size (packet).
q is the queue size (packet).
rp
TCq /: is the Round-trip time (second).
C: is the single link capacity (packets/second).
N: are TCP connections.
P: is the packet dropping probability.
t: is the time (second)
B. Linearization
For control analysis purposes, (1) was linearized about an equilibrium point to get the linearized
TCP model described in (2), see [6] for more linearization details.
)(
1
)()(
)(
2
)(
2
)(
00
0
2
2
0
2
0
tq
r
tW
r
N
tq
rtp
N
Cr
tW
Cr N
tW






(2)
Rewriting (2) in state space form leads to the next equation

65
)()(
)()()( 0
tCxty
rtButAxtx


 (3)
Where






q
W
tx


)( (4)
,)(ty is the queue length and )(tu is a packet dropping probability, as shown in Fig. 1.















rr
N
rN
A1
0
22
0 , 







0
22
2
0N
Cr
B (5)


10

C
The linearized AQM block diagram is shown in Fig.1.
FIG. 1. LINEARIZED AQM DIAGRAM AS FEEDBACK CONTROL
Using the discrete-time theory, the continuous-time system expressed by (3) can be
represented by the discrete-time system as in (5):
(k)x =(k)y
(k)u +(k)x =1)+(kx
dd
ddd C


(6)
Where
AT
e

, )(
0
dte
TAT

 and T is the sampling period.
III. THE CONTROLLER DESIGN
In this section, the LQ-servo controller is presented to make the output (queue length)
stabilize to the desired queue length with a minimum steady-state error ( ss
e) for network congestion
avoidance.

66
A. Servo Structure for the TCP Model
The main principle of the servo system is when the system model has no integrator (as the
TCP model) a feedforward integrator must be inserted as shown in Fig 2 to meet the controller
objective in reaching zero ss
e in the system response [9].
FIG. 2. SERVO SYSTEM STRUCTURE
From Fig. 2, the enhanced state-space model was obtained as described in the next equation, for
more details see [10]:
(k)x =(k)y
0,1,2,...k (k),u +(k)x =1)+(kx
dd
ddd
aug
augaug
G




Define the enhanced state as follows:







1
0



G
aug , 







G
aug
And

I
KKK 
* ,

T
eee
e
exxx
kz
kx
k321
)(
)(
)( 








,
)()( ** kKku

 (8)
Where u*(k) is the optimal control low and K is the control gain.
B. Optimal LQ-servo Controller
The optimal LQ-servo problem is to determine the *
K
matrix of )(
*ku to minimize the cost
function:
))()( )()( ( **
0kRukukQxkJ T
T 


(9)
Where,

PPRk TT  1
)( and T
P
P
is the solution matrix of the discrete algebraic
Riccatti's equation:

PSPQPP TTT  1=0 (10)

67
IV. PARTICLE SWARM OPTIMIZATION OF LQ CONTROLLER
To overcome the obstacle of selecting the appropriate controller parameters ( *
K
),
Particle Swarm Optimization was used.
The PSO algorithm was fundamentally applied to find the optimal controller parameters Q and R,
such that a good output response could be obtained.
Since Q has dimensions of 3×3, T
QQ and R is a scalar value, therefore, seven controller
parameters were defined, 332322131211 ,,,,, qqqqqq and R; so, there are seven members in an individual.
As a result, the dimension of a population is n×7 for n individuals in a population.
The optimization method is driven by using the fitness function F, defined bellow:
 T
ref qqTITAE 0
(11)
Fitness=1/ (1+ ITAE) (12)
The searching proceedings of the proposed LQ-servo based PSO controller were as follows:
Step 1: The seven parameters of the controller were assigned and the population individuals are
randomly initialized such as searching points, best positiot, velocities and gbest, position.
Step 2: For each individual, the fitness value (F) was calculated.
Step 3: Each individual’s (F) was compared with its best position . The best value among the best
position was assigned as gbest (best controller parameter).
Step 4: Position and velosity of each individual were modified.
Step 5: If the iterations number is less than the maximum, then go to Step 2. else, go to 6.
Step 6: The individual with the latest gbest value is assigned as the best controller parameter.
V. SIMULATION
In this section, the performance of the designed LQ controller is verified by MATLAB.
FIG 3. SIMULATION TOPOLOGY[11]
The designed controller is compared with the PI controller. The TCP/AQM network topology that
was used in the simulation is illustrated in Fig.3 with 60 TCP connections, bottleneck link bandwidth
is 15 Mbps, delay = 20 ms and the packet size = 500 bytes. The target queue size changes every 50
seconds with an initial value of 300, then it drops to 200 then changes to 400 and 200. The input
matrices of the continuous system model are:








4.0650242.9024
00.5388
A , 





0
480.4688
B

68

10C
Firstly, the PI controller is examined for comparison reasons. Then, the designed controller
(LQ-servo) performance is determined by adding the servo structure and the enhanced state space
model is:

010
192.7
192.7-
28.3-
178.077.12
078.077.12
0096.0
























aug
aug
G

By the proposed PSO method, the optimal value of matrices Q and R, and the control
gains were found below:

0130.00281.00615.0
]2[,
0,83650.12110.3338
0.0.12110.33330.6555
0.33380.65550.1000














K
RQ
With the following PSO parameters: population size = 90, weight factor w = 0.8, acceleration
constants 1
c=1.2 and 2
c=0.12. Fig.4 shows the system response of the LQ_servo based on PSO
compared with LQ-servo and PI controllers.
FIG. 4. SYSTEM RESPONSE WITH LQ-SERVO BASED PSO
It is seen that the LQ controller enhances the system response, and as a result, it provides better
congestion avoidance compared with the PI controller and the LQ_servo based PSO provides even
better congestion avoidance ability than the LQ_servo controller, as shown in table (1).
Rise time, overshoot and settling time were recorded for performance comparison, as illustrated
in table (1).
Finally, the performance of the LQ-servo controller was analyzed under various network
environments such as varying number of TCP connections (N), which were changed to 300. The
system response is shown in Fig.5, and the results show that the LQ-servo based on PSO controller
has the ability to bring the queue size to the desired level.

69
FIG. 5. THE PERFORMANCE OF PI & LQ-SERVO BASED ON PSO CONTROLLERS WITH N=300
TABLE 1 . SYSTEM RESPONSE FOR TCP WITH DIFFERENT CONTROLLERS
Controller Rise Time
(second) Overshoot
Peak Time
(second) Settling Time
(second)
PI 3.3 33% 8 36
LQ-SERVO 2.4 - - 8
PSO/LQ-SERVO 0.83 - - 4
VI. CONCLUSION
Congestion is a very significant problem in internet networks. In this paper, an AQM model was
proposed to solve the congestion problem. The parameters of this AQM based LQ-servo controller
were obtained by the PSO algorithm. Then, the PSO-LQ was implemented in MATLAB. The results
proved the effectiveness of the proposed controller and showed that it is more effective than the PI in
reaching zero steady-state error with better congestion avoidance under the dynamic network
environment. Moreover, the proposed controller achieved a smaller delay and faster settling time.
REFERENCES
[1] V. Jacobson, "Congestion avoidance and control," in ACM SIGCOMM computer communication review,
1988, pp. 314-329.
[2] B. Braden, D. Clark, J. Crowcroft, B. Davie, S. Deering, D. Estrin, et al., "Recommendations on queue
management and congestion avoidance in the Internet," 2070-1721, 1998.
[3] S. Floyd and V. Jacobson, "Random early detection gateways for congestion avoidance," IEEE/ACM
Transactions on networking, vol. 1, pp. 397-413, 1993.
[4] V. Misra, W.-B. Gong, and D. Towsley, "Fluid-based analysis of a network of AQM routers supporting
TCP flows with an application to RED," in ACM SIGCOMM Computer Communication Review, 2000, pp.
151-160.
[5] C. V. Hollot, V. Misra, D. Towsley, and W.-B. Gong, "On designing improved controllers for AQM
routers supporting TCP flows," in Proceedings IEEE INFOCOM 2001. Conference on Computer

70
Communications. Twentieth Annual Joint Conference of the IEEE Computer and Communications Society,
2001, pp. 1726-1734.
[6] C. Hollot, V. Misra, D. Towsley, and W.-B. Gong, "A control theoretic analysis of RED," in INFOCOM
2001. Twentieth annual joint conference of the IEEE computer and communications societies.
Proceedings. IEEE, 2001, pp. 1510-1519.
[7] S. Floyd and V. Jacobson, "On traffic phase effects in packet-switched gateways," Internetworking:
Research and Experience, vol. 3, pp. 115-156, 1992.
[8] M. Zhenwei, Q. Junlong, and Z. Lijun, "Design and Implementation: Adaptive Active Queue Management
Algorithm Based on Neural Network," in Computational Intelligence and Security (CIS), 2014 Tenth
International Conference on, 2014, pp. 104-108.
[9] K. Ogata, "Modern control engineering,(1997)," ISBN: 0-13-227307-1, pp. 299-231, 2013.
[10] M. Z. Al-Faiz and S. S. Sabry, "Optimal linear quadratic controller based on genetic algorithm for
TCP/AQM router," in Future Communication Networks (ICFCN), 2012 International Conference on,
2012, pp. 78-83.
[11] B. A. Sadek, T. El Houssaine, and C. Noreddine, "A robust PID controller for active queue management
framework in congested routers," in 2017 Intelligent Systems and Computer Vision (ISCV), 2017, pp. 1-6.