A Modified DSR Protocol Using Deep Reinforced Learning for MANETS A Modified DSR Protocol Using Deep Reinforced Learning for MANETS

Abstract

Wireless Networks like Mobile Adhoc Networks (MANETs) have been under extensive research over the past few years with congestion control as one of the most important features to ensure efficient and fair sharing of network resources among users. Machine learning (ML) has achieved a success rate in addressing large-scale and complex problems and researchers have begun to shift their attention from the rule-based method to an ML-based approach to handle the complex needs of future networks where conventional rule-based approaches tend to become inefficient and ineffective. To handle the problems in congestion control, precise notification should be generated as the backpressure transmission process. Although backpressure-based routing algorithms give optimal throughput, they typically have poor delay performance under moderate loads. This may be because packets are being sent over longer routes unnecessarily. Furthermore, the existing backpressure-based optimisation algorithms require every node to compute differential backlogs for every destination queue with the corresponding destination queue at every adjacent node. The proposed algorithm proves to be a cross-layer protocol for wireless MANET that generalises the channel access management and routing process which includes traffic management, connection maintenance and distributed scheduling for concurrent transmission. The joint congestion control method with scheduling algorithm has enhanced active radio communication network by interchanging scheduling schema with adaptation modelling and also the optimum congestion dominance and flow control model is being designed using deep reinforced learning.

Full text

Full Terms & Conditions of access and use can be found at
https://www.tandfonline.com/action/journalInformation?journalCode=tijr20
IETE Journal of Research
ISSN: (Print) (Online) Journal homepage: https://www.tandfonline.com/loi/tijr20
A Modified DSR Protocol Using Deep Reinforced
Learning for MANETS
S. Jothi Lakshmi & M. Karishma
To cite this article: S. Jothi Lakshmi & M. Karishma (2023): A Modified DSR Protocol
Using Deep Reinforced Learning for MANETS, IETE Journal of Research, DOI:
10.1080/03772063.2023.2223168
To link to this article: https://doi.org/10.1080/03772063.2023.2223168
Published online: 18 Jun 2023.
Submit your article to this journal
View related articles
View Crossmark data

IETE JOURNAL OF RESEARCH
https://doi.org/10.1080/03772063.2023.2223168
A Modified DSR Protocol Using Deep Reinforced Learning for MANETS
S. Jothi Lakshmi and M. Karishma
Department of CSE, Akshaya College of Engineering and Technology, Kinathukadavu, Coimbatore, TamilNadu, India
ABSTRACT
Wireless Networks like Mobile Adhoc Networks (MANETs) have been under extensive research over
the past few years with congestion control as one of the most important features to ensure effi-
cient and fair sharing of network resources among users. Machine learning (ML) has achieved a
success rate in addressing large-scale and complex problems and researchers have begun to shift
their attention from the rule-based method to an ML-based approach to handle the complex needs
of future networks where conventional rule-based approaches tend to become inefficient and inef-
fective. To handle the problems in congestion control, precise notification should be generated
as the backpressure transmission process. Although backpressure-based routing algorithms give
optimal throughput, they typically have poor delay performance under moderate loads. This may
be because packets are being sent over longer routes unnecessarily. Furthermore, the existing
backpressure-based optimisation algorithms require every node to compute differential backlogs for
every destination queue with the corresponding destination queue at every adjacent node. The pro-
posed algorithm proves to be a cross-layer protocol for wireless MANET that generalises the channel
access management and routing process which includes traffic management, connection mainte-
nance and distributed scheduling for concurrent transmission. The joint congestion control method
with scheduling algorithm has enhanced active radio communication network by interchanging
scheduling schema with adaptation modelling and also the optimum congestion dominance and
flow control model is being designed using deep reinforced learning.
KEYWORDS
Backpressure transmission;
deep reinforced learning;
dynamic source routing;
mobile ad hoc networks;
network congestion; optimal
backpressure; Wireless
network
1. INTRODUCTION
In recent years, Mobile Ad Hoc networks (MANETs)
have been extensively studied as an alternative to infras-
tructure networks because of their ease of deployment.
The targeted environments for ad hoc networks are typ-
ically inhospitable regions where it is dicult to set up
infrastructure or environments where the existing infras-
tructure has collapsed temporarily or permanently. The
growing development of information technologies has
ledtotheemergenceofvariousarchitecturesamong
wired and wireless communication networks. Under
wireless network technologies, Mobile Adhoc Network
(MANET)isanencouragingresearchareabecauseofits
self-conguring wireless network of mobile devices or
nodes, which communicate with each other using radio
transmissions. Unlike traditional networks, an ad hoc
networkdoesnothaveabasestationwhichactsasa
router [1].AllintermediatenodesinaMANETactas
a router and forwards packets on behalf of other nodes
until the packet is received by the destination from its
sender. MANETs rely on multi-hop transmissions among
the nodes and in the past years, a lot of research has
been encouraged due to considerable issues in rout-
ing techniques which include the large area of ooding,
greedy forwarding, at addressing and widely distributed
information, large power consumption, interference and
load balancing [2]. Hence an ecient routing protocol is
required to enhance communication in MANET.
1.1 Routing in MANET
Early MANET protocols are broadly classied into
[3] table-driven proactive routing protocols and on-
demand reactive routing protocols. The proactive pro-
tocols exchange control messages between nodes peri-
odically to maintain a consistent view of the network
even when there is no active data session. This allows
the proactive protocol to discover the route quickly at
the price of large bandwidth consumption from the over-
head in exchanging control messages. Moreover, there is
a waste of network resources because every node has to
maintain a complete view of the network even though
most routing information is never used.
In contrast, the reactive protocols establish and main-
tain the route between the source and the destination
onlyifthereisarequest[4]. In this model, the estab-
lishedrouteismaintainedaslongasthedatasession
© 2023 IETE

2 S. JOTHI LAKSHMI AND M. KARISHMA: A MODIFIED DSR PROTOCOL
is active. After a certain period, when the data ses-
sion becomes inactive, the route is removed to release
the occupied resources. Therefore, reactive protocols
consume less bandwidth than proactive protocols. Sev-
eral hybrid and hierarchical approaches have been pro-
posed [2] between these two considering time com-
plexity, energy awareness, bandwidth availability, delay-
based and other rising performance metrics. To achieve
high throughput and high-quality communications over
multi-hop wireless networks such as MANET’s utilisa-
tion of bandwidth resources is very important from every
node.
1.2 Congestion Control
Dynamic Source routing (DSR) is an on-demand reac-
tive routing technique in which the entire order of nodes
through which a packet has to be sent is determined
bythesourcenodedynamically[5]. The proposed tech-
nique is a novel method designed considering the tradi-
tional DSR as a base protocol with an additional mech-
anism to enhance the traditional DSR protocol through
ML [6,7] and identify the optimal path [4] between any
two nodes in a MANET. The additional mechanism is
explained in the later part of the article. The performance
of MANET primarily depends on the path layout prob-
lem, and the choice of a suitable set of paths ensures an
acceptable QoS. Congestion control is a signicant issue
that can ensure the allocation of network resources e-
ciently and in a justied manner among the communica-
tion network[8]. To handle congestion at every node and
acquire optimal throughput, backpressure-based routing
[9] has been extensively used and experimented within
the past years.
1.3 Delay-Based Backpressure Networks
Although backpressure-based adaptive routing algo-
rithms have been reliable towards data transmission,
they need to overcome a few limitations. The primary
issue in backpressure-based algorithms is they typi-
cally have a high end-to-end delay because of pack-
ets being sent over avoidable longer routes or routing
loops. Also, backpressure algorithms maintain dier-
ential backlogs for every destination queue based on
which the routing and scheduling decisions are made.
Due to this scenario, practical implementation requires
backpressure-based algorithms to compute dierential
backlogs for every destination queue in accordance
withthenextdestinationqueueateveryadjacentnode.
This leads to expensive computations and information
exchanges between every pair of destination queues
because of the large number of possible pairs of adjacent
nodes.
Like the back-pressure algorithm, these low-complexity
scheduling algorithms are usually also queue-length-
based. The drawback of these approaches, however, is
that the end-to-end delay of the resulting queue-length-
basedschedulingalgorithmisverydiculttoquantify
under certain cases the back-pressure algorithm can have
poor delay performance. To handle end-to-end delay
problems, the backpressure algorithm can be extended
to a xed route, where packets are forced to use an opti-
mal path. However, narrowing the choice of routes to
just one path will cause network saturation quicker than
the scenario when all routing choices are permitted. In
this article, we propose a backpressure-based data rate
adaptation model to manage congestion. In this model,
an integrated module monitors network trac and esti-
mates the load on the path and controls the data ow rate
adapting to the current network conditions.
1.4 Deep Reinforced Learning (DRL)
Reinforced Learning (RL) is always considered a pow-
erfultoolforlearningoptimalpolicyduetoitsability
to interact between an agent and the environment. In
reinforcement learning, the agent technically uses a trial-
and-error method and increases the reward obtained
from the environment [7]. In every step, the agent obtains
the required state from the environment and chooses
an appropriate action based on which the environment
determines the reward. This leads to the formulation of
a new module depending on the reward feedback. Deep
reinforcement learning (DRL) is an improved form of
reinforced learning technique to solve complex and dif-
cult problems. RL is improved further using a deep
learning mechanism. Deep learning can help RL agents
to become more ecient and improve their ability to
optimise policies. Compared to other machine learn-
ing techniques, RL does not need any dataset. In DRL,
the agent interacts with the environment to produce
its dataset. Next, DRL uses this dataset to train a deep
network[10].
Therestofthearticleisorganisedasfollows:wehave
discussed all related works that led us to our proposed
model in Section 2. Section 3 presents the proposed net-
work model. Section 4 presents the proposed machine
learning model that identies the path for optimal data
transfer. Section 5 discusses expected results and evalua-
tion metrics and nally, Conclusion and Future work are
discussed in Section 6.

S. JOTHI LAKSHMI AND M. KARISHMA: A MODIFIED DSR PROTOCOL 3
2. RELATED WORK
The Dynamic Source Routing (DSR) is a straightforward
and eective routing protocol designed for multi-hop
wireless ad hoc networks constructed with dynamic
topology. The DSR operational framework was described
by D. Johnson et al. [5], which allows the ad-hoc network
to be completely self-organising and self-conguring,
without the need for any existing network infrastructure
or administration.
The wireless channels are shared systems with limited
resources over which a large number of users compete for
resources and fair allocation of resources is an important
component in wireless network construction. A.Eryilmaz
et al[8]. showed that a combination of queue-length-
basedschedulingatthebasestationandcongestioncon-
trolimplementedeitheratthebasestationorattheend
users can lead to fair resource allocation and queue-
length stability in their article. This phenomenon can be
extended over wireless MANET at every node instead of
abasestation.
L. Bui et al.[11] compared the poor delay performance
of back-pressure algorithms in the case of xed routing
and adaptive routing and showed that maintenance of
per-neighbour queues at each node is sucient instead of
per-ow queues required by the back-pressure algorithm
in the xed routing system.
To solve the delay ineciency problems of backpressure-
based routing under low load conditions, M.Alresaini
et al., [12] proposed a hybrid technique which was
referred to as backpressure with adaptive redundancy
(BWAR) in their work. This article also explains how
the available bandwidth may be utilised more eciently
during low load conditions and introduced an energy-
ecient variant of BWAR that controls duplication of
packets and optimise power consumption.
Deep learning as a branch of machine learning has been a
growing research area and has achieved signicant atten-
tion in various domains such as robotics, computer vision
and speech recognition. In this context, many deep learn-
ing algorithms to control network trac were introduced
in recent years [13–17]. An overview of deep learning-
based intelligent routing and its eciency over conven-
tional routing strategies was discussed by Z.M.Fadlullah
et al. [6].
The cross-layer optimisation framework was designed by
Khan et al. [18], for multicast communication in Mul-
tihop Wireless Mesh Networks (MWMN). Throughput
maximisation and wireless contention are the main fac-
tors considered to decompose the optimisation prob-
lem with the set of Lagrangian variables. This frame-
work solves the power control and data routing problem
by using network coding. Interference management is
applied using the game theory process. QoS-supported
routing protocols in MANET are evaluated and a com-
prehensive survey is provided. This work mainly points
out the issues in QoS-aware routing and also provides
the solution to handle problems such as land balancing,
bandwidth utilisation, and trac management.
A two-dimensional evaluation of a exible and feasi-
ble approach, based on hop counts and trust values, to
choose the shortest path from various wireless mobile
nodesofaMANETthatmeetstherequirementsofdata
packets, was provided by X.Li et al [19]. They proposed
a trust-based reactive multipath routing protocol which
coulddiscoverdierentloop-freepathsfordatatrans-
mission. They conducted experiments to compare their
proposed work with ad hoc on-demand distance vec-
tor (AODV) routing protocol and show improvement in
packet delivery ratio (PDR) and security against the black
hole, grey hole, and modication attacks.
Wang e t a l [ 7]presentedaworkowfordesigningvar-
ious types of networking techniques and how to apply
machine learning technology in each step of network
design. This works provides a selective survey of the lat-
est advancements in design principles and their benets.
Their experiments show models trained under a specic
network environment can achieve better performance
in other environments but also exclaims the diculty
in the practical implementation of the learning model.
Optimisation algorithms, such as particle swarm opti-
misation and articial intelligence optimisation [20,21],
are addressed to enhance the performance of the routing
protocol.
Distributed Channel Assignment(DCA) and routing
strategy were designed by H.Wu et al [22]formultihop
communication. This method considered the channel
cost as the primary metric along with the interference
information between the connected devices. The chan-
nel interference and the expected transition time are
the two weighing factors to form the routing and chan-
nel assignment. Joint channel assignment and routing
methods were designed for heterogeneous networks with
interference and diversity information.
The optimal path selection was suggested by Priya
Sharma et al. [20] to increase the network utilisation

4 S. JOTHI LAKSHMI AND M. KARISHMA: A MODIFIED DSR PROTOCOL
done by the enhanced swarm intelligence-based sched-
ulerforMANET.Inthismodel,mobilenodesutilise
the query resource available using multiple threads. This
optimisation model combines the process of both Ant
Colony Optimisation (ACO) and Bee Colony Optimi-
sation (BCO). The heuristic function employed in the
optimisation is applied to search the optimal path from
the QoS parameters such as delay and packet loss. The
high overhead involved in request ooding during route
creation is one of the limiting factors of the DSR protocol.
A. Gupta et al. [23] modied the DSR protocol consider-
ing new generation wireless standards designed for low-
rate wireless personal area networks. The multicasting
approach, mobile internetwork broadcast infrastructure
techniqueusedinthismodel,minimisesthenumberof
route requests and achieved extensive improvement.
L. Ying et al. [24] proposed a scheduling backpressure
algorithm that can promise network stability and select
a collection of optimal routes based on the shortest
path information. This model explores possible paths
with average path lengths between source and destina-
tion nodes. This framework selects the dierent routes
based on network trac loads such that longer paths
are used only when required. This method had a much
smaller delay and greater network stability or optimal
throughput.
TheroutediscoverydelayintheDSRprotocolcanbesig-
nicantly reduced by maintaining the route cache. Route
cachingapproachwiththetransactionofactivepack-
ets through nodes of the MANET more than once was
proposed by Dimitri Marandin [25] to improve delay
while using the DSR protocol. This model developed a
caching strategy that can allow nodes to update their
memory cache quickly to minimise delay for short-lived
trac. This article showed that the DSR protocol can be
improved more eectively by updating the cache mem-
ory although DSR is an easy-to-implement on-demand
routing protocol.
A fast route recovery approach that handles path unavail-
ability, as well as packet collision and bad channel con-
dition, was proposed by Zhang et al. [17]andrein-
forced learning-based routing protocols for ying ad-
hocnetworkwasproposedbyLanskyetal.[10]. This
approach increases the reliability of data transmission
by applying the concept of the backup node’s reliability
in MANET. In this model, acknowledgement from the
backup node was used to avoid the misjudgment of node
mobilityandinthisapproach;theorthogonalpolarisa-
tion communication model was utilised to improve the
node capacity and network performance. It also recom-
mended that dual-polarised directional communication
amend the quality of the communication. Zhang et al
applied a novel approach to estimate the available band-
width based on the Lagrange interpolation polynomial
computation implied.
A.Mohajeretal.[26] proposed a dynamic optimisation
model to minimise the overall energy consumption of
fth-generation (5G) heterogeneous networks and pro-
vide the essential coverage and capacity. This model pro-
posed a multi-hop backhauling strategy to eectively
use the existing infrastructure of small-cell networks for
simultaneous dual-hop transmissions and their numeri-
cal results indicated considerable rates of power saving in
dierent trac models while guaranteeing the through-
put requirements.
Based on our study from all related works, we propose
anovelmodeltoimproveatraditionalDSRrouting
algorithm using a deep reinforced learning technique.
DSR protocol involves two stages: route discovery and
route maintenance. The performance enhancement of
the MANET is always based on its ability to re-establish
the path in the case of local route failure and path unavail-
ability. The traditional DSR re-establishes new paths dur-
ing path failures in its route maintenance stage. But we
propose a model where instead of choosing a random
new path, an optimal path is identied and readily avail-
able to handle such routing errors. The workow of the
proposed model is explained in the next section.
3. THE PROPOSED MODEL
In the proposed network model, DSR is the chosen foun-
dation routing protocol due to its On-Demand routing
nature at the source. The proposed modied DSR has
three main modules: Path innovation, Path investigation,
and Path preservation working in sequence as shown in
Figure 1.
Path innovation is initiated by the source node by broad-
casting a route request (RREQ) packet to all neighbour-
ingnodesandthenfollowedbyeveryreceivingnode
until the destination node is found. When the destina-
tion node receives an RREQ packet, a route reply (RREP)
packet is sent back to the source following the reverse
path of the RREQ route updating the cache memory of
all intermediate nodes.
Path investigation starts working by establishing a con-
nection with neighbouring nodes and analyses the path
based on hop count, load, available bandwidth, and

S. JOTHI LAKSHMI AND M. KARISHMA: A MODIFIED DSR PROTOCOL 5
Figure 1: Proposed flow diagram
energy at each node training the cache memory for an
optimal path.
The path preservation module supervises the path in use
and communicates with the source node for handling any
routing errors.
3.1 Path Innovation
In this module, the source node starts the route discovery
stageassameasthetraditionalDSRprotocol,itwillsend
therouterequestpacketRREQtotheneighbouringnode.
But repeating this step every time a new route is required
or due to link failures will increase the pre-routing com-
putations. This module aims to nd a more optimal route
while decreasing route discovery computations.
The proposed algorithm, to send a data packet from node
Nsto node Nd, canberepresentedasfollowsandthedata
ow can be depicted as shown in Figure 4.
Step 1: The RREQ packet containing Packet ID, Source
ID, Destination ID, and Route Table is generated by the
source node Ns.
Step 2: Node Nsveries its cache memory for the avail-
able path to Node Nd, if a reliable path exists the data
packet is directly sent to Ndandifnosuchpathexists,the
RREQ packet is broadcasted to its neighbouring nodes.
Step 3: The intermediate neighbouring node Nithat
receives the RREQ packet from Nsdoes the following:
(a) Verify its cache memory for the available route to the
destination. If available the RREP packet containing
the route table information to reach the destination
through Niis generated.
(b) If node Niidenties itself to be the destination node
Nd, then the RREP packet with identied path infor-
mation is generated.
(c) If node Nicould not identify the required path in
its cache memory, then it extends the routeinh table
with its own id in the RREQ packet and further
broadcasts to its neighbour.
Step 4: Step 3 is repeated until the Niis equal to the
destination Nd.
Step5:RREPwillbesentbackthroughthediscovered
path from one node to another until the sender node Ns
is reached.
Step 6: The source node and any node within the identi-
ed route will update their cache memory with the newly
discovered route to Nd(Figure 2).
3.2 Path Investigation
Path investigation is a learning and decision-maker mod-
ulethatchoosestheoptimalpathfromsourcetodestina-
tion at the source node and intermediate to destination at
all intermediate nodes. To decide the optimal path from
any node to the destination, the various paths identied
in the path innovation step are analysed based on their
path conditions. Once path optimisation is complete, the
connection is established along the prioritised optimal
path and the source node initiates the data packet trans-
fer and transmits the data to the next intermediate node
along the decided path. An intermediate node receives
thedatapacketandproceedsfurtheruntilthepacket
reaches the destination node.
3.3 Path Preservation
The path preservation module supervises the path in
use and informs the source node about any routing
errors. When any node experience link failure and is not
capable of transmitting the next packet due to exten-
sive load or energy drop or missing node problem the
route error (RERR) packet is generated by the node
and sent to the source node. Whenever such negative
feedbackisreceivedbythesourcenodeorifitfailsto
receive acknowledgement, then the data rate adaptation
is applied. This model uses the EWMA estimation in

6 S. JOTHI LAKSHMI AND M. KARISHMA: A MODIFIED DSR PROTOCOL
Figure 2: Path innovation flow chart for the proposed modified DSR protocol
which current transmission and cumulate average trans-
mission rate are considered to update the current trans-
mission. The dynamic rate of the transmission is reduced
whenever the congestion occurs. If there is no congestion,
then the transmission rate is increased gradually. For
each update, the current transmission rate is maintained
in the table. Along with this information, the transmis-
sion reliability in terms of successful transmission and
transmission latency is stored.
3.4 Data Rate Adaptation
The data transfer rate is dynamically controlled and gets
adapted to the network’s environmental condition based
on the following logistics. Consider any node with ‘x’ par-
ent nodes and ‘y’ child nodes in a directed graph. Then,
the total number of transmissions for a source node will
be (x+y). If the node is intermediate, then the number of
transmissions will (be x). Let ‘n’ be the number of data
packets with size ‘l’ to be transmitted within the time
interval ‘t’, then the virtual rate can be estimated as shown
in equation (1) for the source node and as in equation (2)
for the intermediate node.
VR =n×l×(x+y)×8
t(1)
VR =n×l×(x)×8
t(2)
Then the weighted virtual rate can be estimated using the
current virtual rate (VR) and the average of the previously
obtained value of the virtual rate (VRavg)as
WVR =a×VR +(1−a)×VRavg (3)
The optimal weighted virtual rate is determined using the
transfer rate, as shown in equations 4 and 5.
TR =n
i=1CPS(i)
t+n
i=1DPS(i)
t×8(4)
OptimalWVR =WVR +TR
2(5)

S. JOTHI LAKSHMI AND M. KARISHMA: A MODIFIED DSR PROTOCOL 7
where TR represents the transfer rate, CPS represents the
control packet size, DPS represents the data packet size,
and t represents the current time.
During path preservation, rate=based scheduling is
invokedtostandardisethetraclevelofeachnodeon
the neighbourhood. The rrac load (TL) of each node is
calculated as shown in equation (6) based on the amount
of incoming data (ni)andtheamountofoutgoingdata
(no).
TL =(ni×Size +no×Size)×8
Interval (6)
The calculated TL values are updated in the routing table
ofeachnodeandthetotalpathloadiscalculatedasshown
in equation (7).
Path Load =
n

i=1
TL(i)(7)
If the current node has more outgoing trac rates, then
thedatatransferrateisreducedanditiscompleted
without congestion. This estimation can be depicted in
equations (8) to (11).
Current Rate =Nt×Size ×8
interval (8)
where Ntis the amount of trac generated.
The estimated packet generation count (EPGC) can be
determined from the available bandwidth for the packet
size.
EPGC =Avai l a b l e BW
Size ×8(9)
Estimated Rate =EPGC ×Size ×8 (10)
The new data transfer rate can be assigned depending on
available bandwidth, as shown in equation (11).
New Rate =EstRate;if (currrate >avail BW
CurrentRate;otherwise
(11)
Basedonthisinformationthevariouspathidentiedin
path innovation is prioritised.
4. DRL FRAMEWORK
Due to the dynamic topology of MANET, acquiring net-
work data is generally hard but it is known that reinforced
learning (RL) can postpone the training process until all
required actions can be executed and reward values can
be estimated. RL will be a dominant model to implement
machine learning-based congestion control. It can nd
the best decision based on trial and error and quickly
react to environmental changes.
If the link failure process is predetermined then it imme-
diately initiates the route recovery process to avoid packet
loss during data transmission. The trac management
process is done by classifying the incoming and outgoing
data packets. The trac manager prioritises the real-time
data trac over elastic trac. For each packet priority is
estimated and noted in the packets itself.
The trac manager also monitors the incoming trac
level for load balancing. The trac manager monitors
the trac table to achieve load balancing and to priori-
tise the trac. The priority of the packet lies between 0
and 1. The table contains information about trac id,
trac type, estimated priority value, and incoming traf-
c route. The priority is based on the trac rate and
type.
The working mechanism of the proposed approach
involves the following. First, the wireless loss and con-
gestion loss are categorised to perform the window adap-
tationbasedonthespatialtemporalrelationship.Second
bandwidth allocation for wireless devices is determined
based on the current requirement and availability of
bandwidth.Thebestpathselectionisdonebasedonthe
Energy, Mobility, Bandwidth of the path.
Depending on the measured parameter and the goal,
thereshouldbeaproperdenitionofthecorresponding
policy, state, action, and reward for a DRL-based routing
model. Figure 3depicts the framework for DRL for the
proposed path analysis.
In the proposed model, the initial weight values are dis-
tributed as network observations and the action is to opti-
mise the available path-based reliability, data rate, and
latency and thereby reward the network with an optimal
data transmission rate.
The nal routing table is constructed in accordance with
the dened policies of the model and the optimal path is
madeavailablefortheMANETcommunication,andthe
remaining paths will be used as backup alternate routes
whenever a link failure occurs. By choosing the optimal
path at minimum to moderate load conditions, the over-
all throughput and packet delivery ratio (PDR) can be
improved at the same time the total end-to-end delay
is reduced. Meanwhile, during link failure or increasing
backlog in the selected path, the available alternate path

8 S. JOTHI LAKSHMI AND M. KARISHMA: A MODIFIED DSR PROTOCOL
Figure 3: The DRL framework for path analysis
ensures eective communication and maintains optimal
time delay for consecutive data transmissions.
5. RESULTS AND DISCUSSION
Theperformanceoftheproposedmodelcanbeevaluated
in terms of packet-related metrics such as PDR, Delay,
Network availability, and Throughput.
5.1 Evaluation Metrics
Packet Delivery Ratio: PDR may be dened as the ratio
between the number of successfully delivered packets to
the total number of packets attempted for transmission
as shown in equation 1.
PDR =No.of successful delivery
No.of delivery attempts (1)
End-to-End Delay: End-to-end delay may be dened as
the average time taken to complete the transmission of
data packets from source to destination in the network as
expressed in equation (2).
Delay =
n

i=1
(Dest Time(i)−Src time(i))
n(2)
Throughput(TP): Throughput may be termed as the rate
of successful transmission of packets from source to des-
tination, as shown in equation 3. For good designed
network the value should be high.
TP =packs delivered ×Pack size ×8
Transmission Time (3)
Network Availability (NA): Network availability may be
termed as a percentage of the ratio of uptime for data
transmission of packets from source to destination to that
of a total time interval as shown in equation 4. For good
designed network the value should be high.
NA =Up Time ×100
Total Ti m e (4)
5.2 Evaluation Results
The results achieved by the proposed protocol compared
to the original DSR protocol and other existing optimisa-
tion techniques such as Distributed Channel Assignment
(DCA), Multihop Wireless Mesh Networks (MWMN),
Ant Colony Optimisation (ACO) and Bee Colony Opti-
misation (BCO) in terms of PDR, Throughput, End-to-
End delay, and Network availability are shown in the
gures below.
In the PDR evaluation, as shown in Figure 4,theexisting
protocols achieved about 86% to 92% performance while
increasing the number of messages in the network. But
theproposedmodelsshowasignicantimprovementin
thepacketdeliveryratio.
Figure 5showsthatourmodelobtainedasignicant
improvement in throughput estimation and Figure 6
illustrates the delay performance of the traditional DSR
algorithm which is signicantly lower than the proposed
modied version scheme.
From the observations made in the evaluation results, it
can be understood that the throughput in the proposed
scheme is higher than the existing format while decreas-
ing the overall end-to-end delay. Figure 7compares the
availability of the network of the proposed model with
the existing techniques and the results show that the
proposed modied version has greater network availabil-
ity than the existing models which in turn indicates the

S. JOTHI LAKSHMI AND M. KARISHMA: A MODIFIED DSR PROTOCOL 9
Figure 4: PDR comparison rate of the proposed model with other existing models
Figure 5: Comparison of throughput with the proposed and other existing models
Figure 6: End-to-end delay comparison of the proposed model
with other existing models
right usage of network resources throughout the data
transmission time period (Tables 1–4).
Table 1: Evaluation results of PDR
Schemes PDR (in %)
Original DSR 85.78
DCA 87.24
MWMN 88.58
ACO & BCO 91.23
Proposed Model 94.03
The results achieved by the proposed modied DSR pro-
tocol compared to the original DSR protocol and other
existing optimisation techniques such as Distributed
Channel Assignment (DCA), Multihop Wireless Mesh
Networks (MWMN), Ant Colony Optimisation (ACO),
and Bee Colony Optimisation (BCO) in terms of PDR,
Throughput,End-to-Enddelay,andNetworkavailability
arediscussedinthetablesbelow.

10 S. JOTHI LAKSHMI AND M. KARISHMA: A MODIFIED DSR PROTOCOL
Figure 7: Comparison of network availability of the proposed model with other existing models
Table 2: Evaluation results of throughput
Schemes Throughput (in bits per second)
Original DSR 13345
DCA 17657
MWMN 19254
ACO&BCO 19896
Proposed Model 21389
Table 3: Evaluation results of end-to-end
delay
Schemes Delay (in seconds)
Original DSR 2.5
DCA 1.3
MWMN 0.9
ACO&BCO 0.8
Proposed model 0.6
Table 4: Evaluation results of network availability
Schemes Network availability (in %)
Original DSR 95.34
DCA 88.43
MWMN 92.76
ACO&BCO 96.59
Proposed model 98.92
The results show that the proposed model has bet-
ter throughput and network availability when compared
with other route optimisation techniques discussed in the
related works.
6. CONCLUSION AND FUTURE WORK
In this article, a time-ecient routing protocol that
works based on an on-demand reactive DSR protocol is
proposed. This protocol reactively establishes the route
between the source and the destination and maintains it
by using feedback packets for each successfully delivered
packet at the destination. The feedback packet evaluates
the route that the data packet has taken and updates the
activity at each node in the route by using the hop count
information, allowing the route to react to changes in the
network, e.g. node failures and mobility, without creating
additional control overhead on changes. It ensures e-
cient data transfer under low or moderate trac loads.
This model minimises the computation overhead of the
source node for rediscovering the routes whenever a
network failure occurs. A transmission failure due to a
node’s unavailability or a node’s failure is handled by pro-
viding alternative route information to the source node.
This model explains a deep reinforced machine learning
modelthatgivesthesourcenodecontrolofselectingthe
alternative route. It reduces network failure due to loss
of node’s energy and minimises loss of data packets. This
work explains an algorithm which helps in route selection
and the trac management functionality of MANETS.
Itisacross-layerprotocolforwirelessMANETthat
generalises the channel access management and routing
process which includes trac management, connection
maintenance, and distributed scheduling for concurrent
transmission. The components are integrated to work
together and provide better outcomes. This proposed
method can be enhanced further which aims at incor-
porating malicious node discovery and threat handling
mechanisms.
DISCLOSURE STATEMENT
No potential conict of interest was reported by the author(s).
REFERENCES
1. S. Aarthi, and S. Jothi Lakshmi, “Optimal backpressure
data transmission using deep learning,” Int. J. Sci. Res.
Comp. Sci. Eng. Inform. Techn., Vol. 8, no. 4, pp. 349–358,
July-August 2022.
2. A. R. Rajeswari, “A mobile ad hoc network routing pro-
tocols: A comparative study,” Chap. Metr. Overv.,pp.
1508–1511. DOI: 10.1109/wicom.2007.380

S. JOTHI LAKSHMI AND M. KARISHMA: A MODIFIED DSR PROTOCOL 11
3. Z. Long, and Z. He, “Optimization and implementa-
tion of DSR route protocol based on ad hoc network,”
in International Conference on Wireless Communica-
tions, Networking and Mobile Computing, IEEE, 2007,pp.
1508–1511. DOI:10.1109/wicom.2007.380
4. A.Nasipuri, and S. R.Das, “On-demand multi-path rout-
ing for mobile ad hoc networks,” IEEE ICCCN, Vol. 99, pp.
64–70, 1999.
5. D. Johnson, D. Maltz, and Y.-C. Hu. “The Dynamic
Source Routing Protocol for Mobile Ad Hoc Networks.”
http://www.ietf.org/internet- drafts/draftietfmanet- DSR-
09.txt, IETF Internet draft, Apr. 2003.
6. Z. M. Fadlullah, et al., “State-of-the-art deep learning:
evolving machine intelligence toward tomorrow’s intelli-
gent network trac control systems,” IEEE Commun. Surv.
Tutor., Vol. 19, no. 4, 4th qtr. pp. 2432–55, 2017.
7. Q. Wang, and Z. Zhan, “Reinforcement learning model,
algorithms and its application,” in Proceedings of the 2011
International Conference on Mechatronic Science, Elec-
tric Engineering and Computer (MEC), Jilin, China, 19–22
August 2011; pp. 1143–1146.
8. A. Eryilmaz, and R. Srikant. “Fair resource allocation in
wireless networks using queue-length-based scheduling
and congestion control,” Proc. IEEE Infocom., 2005.
9. L. Tassiulas, and A. Ephremides, “Stability properties of
constrained queueing systems and scheduling policies for
maximum throughput in multihop radio networks,” IEEE
Trans. Autom. Cont, Vol. 37, no. 12, pp. 1936–1948, Dec
1992.
10.J.Lansky,S.Ali,A.M.Rahmani,M.S.Yousefpoor,E.
Yousefpoor, F. Khan, and M. Hosseinzadeh, “Reinforce-
ment learning-based routing protocols in ying ad hoc
networks (FANET): A review,” Mathematics,Vol.10,pp.
3017, 2022.
11. L. Bui, R. Srikant, and A. Stolyar, “Novel architectures and
algorithms for delay reduction in back-pressure schedul-
ing and routing,” in INFOCOM, 21, IEEE, 2009,pp.
2936–2940.
12. M. Alresaini, K. L. Wright, B. Krishnamachari, and M. J.
Neely, “Backpressure delay enhancement for encounter-
based mobile networks while sustaining throughput opti-
mality,” IEEE ACM Trans. Network,Vol.24,no.2,pp.
1196–1208, April 2016.
13. G. Wi, S. Son, and K.-J. Park, “Delay-aware TDMA
scheduling with deep reinforcement learning in tactical
MANET,” in Proceeding of the 11th International Con-
ference On Information and Communication Technology
Convergence (ICTC), 2020, pp. 370–372.
14. Y. Cheng, B. Yin, and S. Zhang, “Deep learning for wireless
networking: The next frontier,” IEEE Wirel. Commun.,Vol.
2, no. 4, pp. 1–8, 2021.
15. C. She, et al., “Deep learning for ultra-reliable and low-
latency communications in 6G networks,” IEEE. Netw,Vol.
34, no. 5, pp. 219–225, 2020.
16. M. Eisen, and A. Ribeiro, “Optimal wireless resource allo-
cation with random edge graph neural networks, IEEE
trans,” Signal Process., Vol. 68, pp. 2977–2991, 2020.
17. A. Zhang, M. Sun, J. Wang, Z. Li, Y. Cheng, and C.
Wang, “Deep reinforcement learning-based multi-Hop
state-aware routing strategy for wireless sensor networks,”
App. Sci., Vol. 11, no. 10, pp. 4436, 2021.
18. A.N.Khan,M.A.Tariq,M.Asim,Z.Maamar,andT.Baker,
“Congestion avoidance in wireless sensor network using
software dened network,” Computing, Vol. 103, no. 11, pp.
2573–2596, 2021.
19. X.Li,Z.Jia,P.Zhang,R.Zhang,andH.Wang,“Trust-based
on-demand multipath routing in mobile ad hoc networks,”
IET Inf. Secur., Vol. 4, no. 4, pp. 212–232, 2010.
20. K. k. Priya sharma, “Hybrid articial bee colony and tabu
search based power aware scheduling for cloud comput-
ing,” Int. J.. Intell.l Syst. Appl., Vol. 10, no. 7, pp. 39–47, 2018.
DOI: 10.5815/ijisa.2018.07.04.
21. W. Haoxiang, “Multi-objective optimization algorithm for
power management in cognitive radio networks,” J. Ubiq-
uit. Comput. Commun. Technol., Vol. 1 (02), pp. 97–109,
2019.
22. H.Wu,F.Yang,K.Tan,J.Chen,Q.Zhang,andZ.Zhang,
“Distributed channel assignment and routing in multira-
dio multichannel multihop wireless networks,” IEEE J. Sel.
Areas Commun., Vol. 24, no. 11, pp. 1972–1983, Nov. 2006.
DOI: 10.1109/JSAC.2006.881638.
23. A. Gupta, R. Upadhyay, and U. R. Bhatt, “MIKBIT- Mod-
ied DSR for MANET,” in International Conference on
Issues and Challenges in Intelligent Computing Tech-
niques (ICICT), Ghaziabad, 2014. pp. 268–271, 2014.DOI:
10.1109/icicict.2014.6781291
24. L.Ying,S.Shakkottai,A.Reddy,andS.Liu,“Oncombin-
ing shortest-path and back-pressure routing over multihop
wireless networks,” IEEE ACM Trans. Netw. (TON),Vol.
19, no. 3, pp. 841–854, 2011.
25. D. Marandin, “Improvement of link cache performance in
dynamic source routing (DSR) protocol by using active
packets,” Next Gent. Teletra. Wire. Wirel. Advan. Netw.,
Vol. 4712, pp. 367–378, 2007.DOI:10.07/978-3-540-
74833-5_31.
26. A. Mohajer, F. Sorouri, A. Mirzaei, A. Ziaeddini, K. J. Rad,
and M. Bavaghar, “Energy-aware hierarchical resource
management and backhaul trac optimization in hetero-
geneous cellular networks,” IEEE Syst. J.,Vol.16,no.4,pp.
5188–5199, Dec. 2022.DOI:10.1109/JSYST.2022.3154162

12 S. JOTHI LAKSHMI AND M. KARISHMA: A MODIFIED DSR PROTOCOL
AUTHORS
S Jothi Lakshmi, working as associate
professor, department of CSE, Akshaya
College of Engineering and Technol-
ogy, Coimbatore. She received her Btech
degree in Information Technology from
Anna University, Tamilnadu, India in 2006
and her ME degree in Computer Science
and Engineering in 2010 and She received
her PhD degree in Information and Communication Engi-
neering from Anna University, Tamilnadu, India in 2021. The
research area includes Image Processing, Computer Vision,
Deep Learning, and Network.
Corresponding author. Email: jothiman@gmail.com
MKarishmaisaPGscholarpeursu-
inganMEatthedepartmentofCSE,
Akshaya College of Engineering and Tech-
nology,Kinathukadavu,Coimbatore.She
completed her BE degree course in CSE,
Sri Subramanya College of Engineering
and Technology, Aliated to Anna Uni-
versity, Chennai in 2016.
Email: mkarish273@gmail.com