Object identification and surveillance based on deep learning algorithms for quadcopters

Abstract

Drone technology is evolving for the applications like surveillance, observation, rescue and control crimes, military, agriculture, civil and many more purposes. But these surveillance systems are monitored by human interaction so there may be some negligence and malfunction may happen due to lack of observation. The objective of the work was development of unmanned aerial vehicle and implementation of the deep learning, image processing tools in the quad-copter based surveillance system. Development of the quad-copter was done by using necessary components such as Arms and power distribution board with help of fasteners. And the brushless dc motors are fitted with electronic speed controllers. And the suitable propellers are fitted to the motors, from which thrust is obtained. The Arduino Uno microcontroller is used as flight controller with MPU6050. Sensor fusion concept coding was used for accelerometer and gyroscope for stable direction orientations of Aerial vehicle. Multiwii platform was used to build the flight controller for achieving desire rotation of motors as well as proper directions and speed. The receiver was installed in the quad-copter for wireless control with transmitter of 2.4 GHz range. And IP camera was used, from which the surveillance visuals are taken for monitoring. Battery of 2200mah capacity of 3 cells was used for power supply of whole system. The visuals were obtained in raspberry pi, the live video stream/images are processed with the deep learning tools i.e., Open CV, Tensor flow, yolo for effective surveillance.

Full text

Object identification and surveillance based on deep
learning algorithms for quadcopters
Prashant Udapudi1*, Nagesh S2, and Rakesh Tapaskar3
1,3Department of Automation and Robotics, KLE Technological University, Hubballi, India 580031
2Dept. of Mechanical Engineering, R V College of Engineering, Bengaluru, India 560059
Abstract. Drone technology is evolving for the applications like
surveillance, observation, rescue and control crimes, military, agriculture,
civil and many more purposes. But these surveillance systems are monitored
by human interaction so there may be some negligence and malfunction may
happen due to lack of observation. The objective of the work was
development of unmanned aerial vehicle and implementation of the deep
learning, image processing tools in the quad-copter based surveillance
system. Development of the quad-copter was done by using necessary
components such as Arms and power distribution board with help of
fasteners. And the brushless dc motors are fitted with electronic speed
controllers. And the suitable propellers are fitted to the motors, from which
thrust is obtained. The Arduino Uno microcontroller is used as flight
controller with MPU6050. Sensor fusion concept coding was used for
accelerometer and gyroscope for stable direction orientations of Aerial
vehicle. Multiwii platform was used to build the flight controller for
achieving desire rotation of motors as well as proper directions and speed.
The receiver was installed in the quad-copter for wireless control with
transmitter of 2.4 GHz range. And IP camera was used, from which the
surveillance visuals are taken for monitoring. Battery of 2200mah capacity
of 3 cells was used for power supply of whole system. The visuals were
obtained in raspberry pi, the live video stream/images are processed with the
deep learning tools i.e., Open CV, Tensor flow, yolo for effective
surveillance.
Keywords: Image Processing, UAV surveillance, Convolution neural
networks, Drone design and development.
1
Introduction
The UAV (unmanned aerial vehicle) is an aircraft which can fly without a human pilot within
it. UAVs are a part of an automated airplane framework. This incorporates a transceiver base
station on the ground. Unmanned aerial vehicles has number of types, quad-copter is one of
them which is used largely in photography, surveillance and rescue operations etc. The
principal use is surveillance, which is basic for security tasks. An increase of security officers
was not an optimal solution to address the problem as general civilians would feel restricted
for their usual activities, leading to a drone-based surveillance system that does not indulge
in the day-to-day activities of the general civilians.
State of the art drone has an appreciable influence on daily life of human beings reducing
their challenges in various arena. Some of the areas of application are surveillance systems,
military applications, aerial photography and agricultural sectors. The basic core
* Corresponding author : prashant.udapudi@kletech.ac.in
© The Authors, published by EDP Sciences. This is an open access article distributed under the terms of the Creative Commons
Attribution License 4.0 (https://creativecommons.org/licenses/by/4.0/).
ITM Web of Conferences 57, 02008 (2023)
ICAECT 2023
https://doi.org/10.1051/itmconf/20235702008

technologies like ML, open CV, AI & Control systems are integrated for the required
unmanned aerial application in hazardous and inaccessible areas.
Many researchers are trying to improve the productivity of Automations and a large number
of them had made it further developed. Though the advanced automation systems have
evolved but due to few constraints some real time applications demand the old manual
systems. Automation innovation was utilized to screen and distinguish crimes in combat
zone, wild, inborn zones and deserts.
2
Literature Survey
2.1
Review on Unmanned Aerial Vehicles
The research on unmanned airborne vehicle equipped with present day technologies for civil
security applications. With miniaturization and growing capabilities of the mems devices
have resulted in real time autopilot UAV systems [1]. The demand for UAV has seen a steep
growth in the year 2018 in military applications in navy, air force and army and also expected
to rise in the coming decade [2]. New age air traffic includes UAV’s which collect the air
traffic information and distribute to the controllers for efficient tracking and monitoring of
the airborne machines [3,5].
UAVs normally referred as “drones” may be fully or in part autonomous however sometimes
controlled remotely by the human pilot [6, 7]. Mainstream improvement started out in US
military, based totally on the aircraft precept, later including helicopter-primarily based
movement to growth versatility [8].
The primary and basic requirements/functions of the military applications include targeting
the enemy base and bombing them or spraying of the fluids with UAV’s [9]. Vulnerabilities
such as hacking and spoofing of data packets sent and received by the drone is a critical issue
which can be addressed with suitable firewall mechanisms inside the kernel of its operating
system [10].
2.2
Unmanned Aerial Vehicle’s Applications
The (UAVs) Unmanned aerial vehicles commonly called as drones, the UAV domain is
known as most robust and multi-dimensional evolving technologies in present technology
research fields, because they are using in the air traffic management systems, surveillance,
agriculture, photography, inspection along with data collection and management [11, 13].
And coming to societal applications there are many advantages but within that many issues
with privacy things so many people have published papers on primary social, legal and ethical
dilemmas, in the case of automatically controlled by pilot or autonomously sensing devices
[14, 15].
2.3
Review of Image processing applications
Open CV is one of the booming technologies in image processing nowadays, initially open
cv was introduced by Intel with inbuilt libraries for video and image analysis. There are three
approaches in open cv the first one is using specification of histogram in an image with
respect to python language, second one was live edge detection of a video, which can be
possible using webcam also. And third one is live face recognition [16].
Authentication & Identification has become main problem in nowadays digital world. Face
recognition performs a sizable position in authentication & identification. In this paper,
numerous existing face detection and recognition methods are analysed and discussed. Each
2
ITM Web of Conferences 57, 02008 (2023)
ICAECT 2023
https://doi.org/10.1051/itmconf/20235702008

approached was discussed briefly & in comparison with the opposite in phrases of key
assessment parameters [17].
Detecting and spotting objects in unstructured as well as in structured environments is tough
job obligations from OpenCV and AI perspective. This paper introduced a new computer
vision based totally obstacle detection technique for cell technology and its programs [18].
Each character photograph pixel is assessed as belonging either to an obstacle based on its
appearance. The method uses a webcam that performs in real-time, and affords a binary
obstacle photograph at excessive decision. The system has been tested successfully in a
spread of environments, indoors as well as outdoors [19].
3
Methodology
# Task 1 : Study the characteristics of the Components and Procurement of Required
Components
Study the characteristics and specifications of different components used for development of
Quad-copter, selection of frame, motors, batteries and controller as for the system
requirements, which are efficient and availability of components in low-cost for the
development of surveillance system.
#Task 2 : Assembly and subassembly of the components to build the Quad-copter
Assembly of the frame will be done by four arms and fasteners then brushless dc motors will
be installed over the frame, suitable controller will be built and installed in the system with
electronic speed controllers, and as per the requirements the power source will be provided.
# Task 3 : Development of the Surveillance part
The framework comprises of the IP (Internet Protocol) Camera, which has fitted to bottom
of the quad-copter so the camera is a device to get the visuals from the top view using the
drone so this camera sends the visuals to another system using IP address and then the further
process is done using the software tools like Open CV, Tensor flow, Yolo etc.
# Task 4 : Using the proper codes and algorithms for accomplishing required rotations
of motors for getting desired orientations of quad-copter and speed
The accelerometer is a device facilitates in analysis and motion and to be vigorous within
accelerating conditions. Utilization of hardware systems associated with gyro-accelerometer
and interplay with the Arduino yield sensor that keeps up quad-copter direction. Gyroscope
allows balancing and maintaining of the quad-copter in steady mode with x, y, z, values.
Using the standards of these sensors improvement of the algorithms can be performed.
# Task 5 : Implementation of Image processing tools and algorithms and testing
The visuals are transferred to the Monitoring system through the Internet protocol facility of
camera. Then the Open CV and Tensor flow, Yolo algorithms and codes are used for the
captured visuals for feature extraction and activity detection for better surveillance.
3
ITM Web of Conferences 57, 02008 (2023)
ICAECT 2023
https://doi.org/10.1051/itmconf/20235702008

Fig. 1. Flow of the research work.
•
CNN
The CNN (Convolution neural network) is a type of deep neural network, widely used in the
applications related to the images. The networks of convolution model are inspired by the
animal visual cortex which resembles connectivity pattern between neurons of animal neural
network. Proposed architecture has input layer followed by a convolution layer with
activation function as ReLU layer then comes pooling layer and at the end fully connected
layer where architecture will give the output of visuals as shown in the Figure as well as flow
chart.
Fig. 2. CNN Architecture.
4
ITM Web of Conferences 57, 02008 (2023)
ICAECT 2023
https://doi.org/10.1051/itmconf/20235702008

Fig. 3. CNN Flowchart.
4
Design approach
4.1
Quad-copter Architecture
The below block diagram of the quad-copter design as shown in Fig 4. Power source is
connected by means of power supply system to the components of quadcopter i.e., flight
controller, sensors, motor drivers (ESCs). Each motor of the quadcopter is driven by the
ESCs. And the other components of the system are camera and wireless trans-receiver. The
Global Positioning System which fetches the location coordinates from worldwide orbiting
satellites, which is connected to Arduino flight controller for positioning. And the telemetry
data is sent back to system using RF communication along with video stream.
Fig. 4. Quadcopter Architecture.
5
ITM Web of Conferences 57, 02008 (2023)
ICAECT 2023
https://doi.org/10.1051/itmconf/20235702008

4.1.1
Wireless Transmission System
The transmission system consists of Wireless receiver module in quad-copter which is
programmed with Arduino. The wireless transmitter module which was controlled by
operator so as to provide the commands to the flight controller through the receiver. There
are many remote transmissions, which are of RF range at 900 MHz, which was allotted by
ISM Band for open-source programming.
4.2
Calculations for selection of components
4.2.1
Maximum amperes rating
Electronic Speed Controllers were used to run the brushless DC motors that are used in most
of the quad-copters. The maximum ampere of an ESC should be more than the
motor/propeller draws. As far as ESC, recommending 25 to 45% additional Amperes can
withstand by ESC.
The basic equation is,
Ampere Rating of ESCs = 1.5 x 15.6 (Motor’s max ampere rating) = 23.4
According to calculations, ESC between ranges of 20A to 30A are selected.
4.2.2
Thrust calculations
Generally, the required thrust is calculated using below mentioned equation.
Thrust required = (weight of whole system) / 4.
As per the component’s specifications ESCs, Motors will weigh about 1350 grams and frame
weigh about 975 grams and other things like battery, RC receiver, flight controller generally
weigh about 375 grams. Thrust required = 1350×2/4
= 2700/4
= 675 grams
The required thrust from each motor is 675 grams, so the actual amount of thrust produced
by each motor is known by the calculations done below mentioned formula. As per few
sources thrust calculation equation is given below.
Thrust = [(eta × P) 2 × 2 × π × r2 × air density] ^1/3 (1)
Where,
Eta = propeller hover efficiency let assume it as in range of 70 to 80 %.
P = power of shaft = current*voltage*motor efficiency
R = Propeller’s radius in meters
Density of Air = 1.22 kg/m³
Required voltage = 11.1 v
Required current (amp) = 15.6 A
Efficiency of Motor is assumed 75% = 0.75
Eta = 0.8
Then, the thrust
Thrust = [(8/10×10×15.6×0.75) ²×2×3.14× (0.127) ² ×1.22] ^0.33
= [(93.6) ²×0.1235] ^0.33
= (8760.96×0.123) ^0.33
= 871.92^0.33
= 10.016 N
Therefore, Thrust calculated Thrust = 10.016 N
6
ITM Web of Conferences 57, 02008 (2023)
ICAECT 2023
https://doi.org/10.1051/itmconf/20235702008

= 10.016 × 0.101 kg
= 1.0116 kg
= 1011.65 grams
Consequently, the thrust produced from one motor is 1011.65 grams.
Quad-copter has 4 motors, the all-out thrust produced, total thrust produced by four motors
shown in the below calculation. Overall thrust = 1011.65 × 4
= 4046.61 grams
Thrust = 4.0466 kg.
On the off chance that if the efficiency of motor is less, at that point by taking some factor
of safety is must, if the motor is of 70% proficient in the above, then 70% proficient work
framework can create thrust of motors.
Thrust = 4.0466 × 70/100
Thrust = 2.83 kg
As per calculations the thrust required to lift the quadcopter’s motors is 2.83 kg.
4.2.3
Specifications of Electrical & Mechanical Components
Below mentioned are accompanying primary electrical and mechanical modules whose
determinations are portrayed.
a)
Quad-copter frame
b)
Brushless DC motors
c)
Quad-copter Propellers
d)
Li-Po battery
a)
Quad-copter Frame
As per the requirements the frame of quadcopter should be flexible enough to nullify
the vibrations produced by the motors during the flight. For the selection of the
appropriate frame for the quadcopter the main parameters are size, weight, material
used to build the frame. And the frame is built according to below mentioned
parameters.
•
The frame has a central power distribution board at the center to mount the
electronic components and to power up the system.
•
The board has four ends where four arms can be connected to build
quadcopter.
•
And the brackets at the end of the arms to fix the BLDC motors.
Fig. 5. Glass fibre frame of the developed quadcopter.
7
ITM Web of Conferences 57, 02008 (2023)
ICAECT 2023
https://doi.org/10.1051/itmconf/20235702008

b)
DC Motors (Brushless)
In the quadcopter, four motors drive the propellers to produce the thrust.
As per the design the motors are of 1000 RPM brushless DC motors with a 3.17 mm
shaft breadth and weight of one motor is 55 grams, the maximum current drawn is
17 Amps.
Fig. 6. Brushless DC motor used for propulsion system.
c)
Propellers
The propellers are sufficiently enormous to give satisfactory lift to the quad-copter,
propellers are compact enough to fit into the selected frame.
The propellers selected for the quadcopter are made up of carbon fiber, suitable for
selected frame with vary in size of 4 to 22 inches in length and 2 to 12 degrees of
pitch.
Fig. 7. Propeller of the quadcopter.
d)
Battery
The battery gives constant voltage, extreme contemporary power for all the other
components at the quad-copter.
i) Battery Selection
The decision was to use the 2300 mAh battery as per the UAV’s flight time and other
component’s power requirements.
Fig. 8. Lithium Polymer battery (2200mAh).
8
ITM Web of Conferences 57, 02008 (2023)
ICAECT 2023
https://doi.org/10.1051/itmconf/20235702008

Fig. 9. Graph of variation in flight time w.r.t battery capacity.
Table 1. Battery Capacities with Flight Times.
Battery power (mAh)
Time of flight (min)
2,450
6.5
3,000
7.55
3,300
8.78
5,000
10.5
11,975
30.0
4.3
Sensor Modules
a)
MPU 6050 :
The MPU-6050 is a IMU sensor which has Micro Electro Mechanical System
(MEMS), which provides the Accelerometer and Gyroscope data on a single chip. The
Inertial measurement unit which monitors forces acting on accelerometer and angular
changes on Gyroscope and magnetic forces on Magnetometer. These are used in the self-
balancing robots, planes, smart phones, UAVs for positioning, motion tracking and to control
the flight.
Fig. 10. MPU6050.
b)
IP Camera :
Internet protocol camera which is abbreviated as IP camera, which is a sort of digital
video camera that captures the visuals by its lens and sends that data by means of internet of
things. These are generally used for surveillance and security purpose, similar to Closed
circuit television (CCTV) cameras but IP camera doesn’t require the local storage system
instead these need local area network for data transmission.
9
ITM Web of Conferences 57, 02008 (2023)
ICAECT 2023
https://doi.org/10.1051/itmconf/20235702008

4.4
Embedded Systems & Electronics Modules
4.4.1
Flight Controller Using Arduino Uno board
Arduino is an open-source software and hardware platform dedicated for microcontrollers
programming. Arduino products have the General public license which permits all to design
and development of Arduino boards and software distribution.
Fig. 11. Arduino Uno – Microcontroller board.
4.4.2
Electronic speed controllers
The electronic speed controller is a circuit which controls the rpm of the BLDC motors as
per the instruction of receiver data. These ESCs are commonly used in electrically powered
& RC controlled toys and crafts especially for brushes dc motors for generating three phase
electric power from voltage source.
Fig. 12. Electronic speed controller of 30A.
4.4.3
Transmitter & Receiver
•
Transmitter :
The transmitter is a device which uses radio signals to communicate its commands remotely
through predetermined radio frequency to a radio receiver. The receiver will be in the copter
and its being controlled remotely through the transmitter, the transmitter interprets the pilot’s
commands into the motor’s rotation for desired movements of copter. In some cases, there
will an option to use the required frequency of band i.e., 900MHz in a 2.4GHz radio trans-
receiver. Quad-copter Radio Transmitter transmits commands by means of channels. Every
channel has its own function being sent to the craft. Throttle, Yaw, Roll and Pitch are the
main four parameters of the data need to control the drone.
10
ITM Web of Conferences 57, 02008 (2023)
ICAECT 2023
https://doi.org/10.1051/itmconf/20235702008

Fig. 13. Radio Transmitter.
•
Receiver :
The device which has the capability of receiving the commands or signals sent by Radio
Transmitter is called as Radio Receiver. Which communicates with flight controller of the
system to control the flight of aircraft.
Fig. 14. Radio receiver and binding cable.
4.4.4
Software tools
a)
Arduino IDE
The Integrated development environment of Arduino IDE is an application platform
for many operating systems like windows, Linux, Mac etc. which can be programmed in C
as well as C++ programming languages. This platform is used to write the code for
requirements of application and then to upload that code to the suitable microcontroller
boards.
b)
Open CV
Open CV (open-source computer vision library) is an AI programming library. On
view of achieving the proper structure to computer vision applications and to enhance the use
of machine learning in commercial aspects. The open cv has more than 2500 optimized
algorithms, which are included with the comprehensive set of classic and state of art
computer vision machine learning algorithms.
c)
IP Web Cam
IP webcam tool is utilized in this work for the for short range video transmission
with or without internet. IP webcam android application is used for video transmission. It
11
ITM Web of Conferences 57, 02008 (2023)
ICAECT 2023
https://doi.org/10.1051/itmconf/20235702008

gives live video transfer which can be accessed by local or global computers. For local
correspondence it utilizes Wi-Fi connection. Cell phone and the on-board PC needs to
connect in same Wi-Fi communication. IP Webcam helps to convert mobile device into a
web cam for online stream and monitoring. To get the stream just one needs to go to browser
and type the IP address that’s it will give the live stream of the visuals. The IP address is
provided by the application itself, which will be displayed below the stream. But system
needs a flash player to watch the stream.
d)
Multiwii
Multiwii is a flight control platform that is used by many if the drone developers
and which is flexible to all type of UAVs i.e., Drones, planes etc. The flight controller will
monitor the acceleration and gyroscope data along with roll, pitch, yaw from the MPU-6050
module. Based on the real time data fetched by motion processing unit the flight controller
will controls the motors for desired flight and directions using the PID control system. Drone
flight control frameworks are numerous.
4.5
Cost Estimation
The process of cost estimation done before the development, because there may be some
wastage of components during or after the development of the
system. The segments required
for this venture are effectively accessible in the market in various sizes and expenses. Project
budget estimation is given in the Table below.
Table 2. Battery Capacities with Flight Times.
Sl.No.
Components
price per unit (in
rupees)
Total
1.
Quad-copter frame x
1
850
850
2.
BLDC Motors x 4
350
1,400
3.
ESCs x 4
470
1,880
4.
Propellers x 4
65
260
5.
Arduino Uno x 1
450
450
6.
MPU6050 x 1
150
200
7.
IP Camera x 1
1500
1,499
8.
2300mAh Li-Po
Battery x 1
1600
1,320
9.
Transmitter and
receiver x 1
3299
3,299
10.
Jumping wires x 30
90
90
11.
XT60 Connector
200
200
Total cost
10,128
5
Development
5.1
Procedure for assembly of components to build the quad-copter
The components can be assembled in any manner however below listed steps are to given to
do the particular sequence so that there are no other consequences occur to the process and
components.
•
Firstly, connect the motors to the arms of the quadcopter and fix those with screws and
nuts provided with the system so that they will not get harmed during the flight.
•
Later on, ESCs terminals are connected to the power distribution board by soldering
the process. After the connections need to decide the front and back orientations of the
12
ITM Web of Conferences 57, 02008 (2023)
ICAECT 2023
https://doi.org/10.1051/itmconf/20235702008

quadcopter, this step is crucial for the flight controller installation because based on the
orientation it will help to flight in different directions.
•
Later on, connect the ESCs to the motors with proper color coding because the
connection decides the rotation of the motors in clockwise and anti-clockwise direction, the
4 motors should be numbered i.e., the left front motor is 1st one, right front motor is 2nd one,
right rear motor is 3rd one and left rear motor is 4th one.
•
The 1st & 3rd motors rotate in clockwise direction; 2nd & 4th motors will rotate
anticlockwise clockwise direction. The motors used here are 3 phase brushless DC motors.
The three wires in ESCs are connected with three wires of these motors. There is no specific
order of connection with color coding so serially connect the three wires of esc to motor wires
which will leads to rotation of motors in clockwise this is done for 1st & 3rd motors and for
2nd & 4th motors right end and left end wires are exchanged so that they will rotate in
anticlockwise direction.
•
In the next step the ESC’s positive and negative terminals are soldered to the power
distribution board’s positive and negative terminals respectively. Then the ESCs are tied to
the arms of quadcopter using the zip ties, which will help us in reduction of hanging and
vibrations of ESCs during the flight.
•
Next step is to fix the flight controller on the power distribution board. And insulation
has to be done by a cover case.
•
Then the servo connectors of ESCs are connected to the flight controller which has a
specific order printed on the flight controller.
•
Then place the battery exactly at the center of the drone since its bit heavy and the
center of gravity should act at center of the frame.
•
Then an external connector is needed to connect the battery and power distribution
board called XT60.
•
Later on, the receiver is to be attached system, and connect the ESC’s pins to respective
channels of receiver as well as flight controller pins.
•
Affix a landing tripod to the bottom of the quadcopter which will be safety guard to
drone while landing after flight.
•
Then fix IP camera module below the quadcopter using suitable arrangements. Fix the
camera in such a way that it should be easy to install and uninstall the arrangement.
•
After all the above steps finally test the components working status, for that first switch
on the transmitter and then switch on the drone, the feedback sound from the receiver says
that they both are working well.
•
Quadcopter need to armed before the flight by the transmitter. Bring down the throttle
stick of transmitter to right side, drone will be armed, which means ready to take off.
•
After arming is done slowly raise the throttle to check the working of motors.
5.2
Circuit connection of quad-copter
Fig. 15. Circuit diagram of quadcopter.
According to the provided blueprint of system as shown in Fig 15, did the subsequent
connections,
13
ITM Web of Conferences 57, 02008 (2023)
ICAECT 2023
https://doi.org/10.1051/itmconf/20235702008

1
: ESCs : Connected the 1st ESC’s signal Pin to Arduino Digital pin 10, 2nd ESC’s
Signal Pin to Arduino Digital pin 9, 3rd ESC’s Signal Pin to Arduino Digital pin 11 and 4th
ESC’s Signal Pin to Arduino Digital pin 3.
2
: MPU-6050 : Connected MPU6050’s SDA & SCL pins to Analog 4 and Analog 5 pins
of Arduino respectively.
3
: Receiver : Connected receiver’s Channel 1 to Arduino Digital pin 4, Channel 2 to
Arduino Digital pin 5, Channel 3 to Arduino Digital pin 2 and Channel 4 to Arduino digital
pin 6.
In the last part grounded the MPU-6050, the receiver, and the ESCs. For this connected
the GND pins of components to GND pins of Arduino.
4
: Battery : Connected the 2300 mAh Li-Po Battery to the system using the XT60 cable
for power supply to whole system.
5.3
Configuration of Multiwii and Uploading the code to Flight controller
Multiwii is a flight control platform that is used by many if the drone developers and which
is flexible to all type of UAVs i.e., Drones, planes etc. The Multiwii is available in the internet
freely in zip file (open source), by unzipping it and will get the files, find the .ino format file,
open it in Arduino IDE, will get the bunch of libraries within those select the config.h file to
do the configuration of the quadcopter settings as shown in the Fig 17 i.e., quad X, quad +,
PPM values, min and max throttle setting etc. Then select the quadcopter configuration ‘X’
and then selected the sensor type and buzzers pins and so on. Then no need to change
anything in the other libraries.
Fig. 16. Configuration of Multicopter type in Arduino IDE.
To activate any configuration, need to delete the double slashes, in which the whole line will
become comment. Here the type of building drone is quad X with 4 motors.
Fig. 17. Setting of maximum and minimum throttle in Arduino IDE.
Whenever the transmitter sends the PWM signals to the ESCs the motors will rotate in
required direction with specific rpm according to the need. As and when signal pulse width
changes the PWM signal will be modulated according to the signal’s changes. The signal
14
ITM Web of Conferences 57, 02008 (2023)
ICAECT 2023
https://doi.org/10.1051/itmconf/20235702008

pulse width is measured in terms of milli seconds. Motors used in the quad-copter are having
the throttle range of 1000 µs to 2000 µs where 1000 is minimum and 2000 is maximum
throttle.
Fig. 18. Selection of IMU board in Arduino IDE.
Remove the slashes to select the MPU6050 sensor as shown in Fig 19. Then build the
communication range of PPM in between transmitter & receiver with respect to flight
controller. Transmitter and receiver used here is of 6 channels PPM signal which consists of
parameters throttle, yaw, pitch, roll. PPM and snap found.
Fig. 19. Setting of PPM signal pins in Arduino IDE.
In the PPM sum receiver, added the below given instructions.
#define SERIAL_SUM_PPM THROTTLE, YAW, PITCH, ROLL, AUX1, AUX2, AUX3,
AUX4, 8, 9, 10, 11
After all, above steps were done then the code is ready for uploading, later on uploaded the
code by connecting the Arduino flight controller to laptop/system using cable and selected
appropriate port and board.
Fig. 20. Uploading the code to flight controller using Arduino IDE.
After uploading the code, the software shows the message ‘Done uploading.’ keeping it
connected and open the file Multiwiiconf.exe, different GUIs are available there chose
according to systems using as shown in Fig 21.
Fig. 21. Testing the working of flight controller using Multiwiiconf GUI.
15
ITM Web of Conferences 57, 02008 (2023)
ICAECT 2023
https://doi.org/10.1051/itmconf/20235702008

After opening the Multiwiiconf.exe file the GUI showed some options like selection of port
com and blocks with some readings so selected appropriate com port in the GUI and clicked
on start button then the GUI showed the readings and Quadcopter orientations graphically as
shown in Fig 22.
Fig. 22. Representation of quadcopter orientations and readings in Multiwiiconf.
5.4
Binding of transmitter and receiver
Sometimes the receiver won’t receive the commands sent by transmitter because of binding
problem, so binding of the transmitter and receiver has to be done before connecting to the
system. For that, binding connector is connected to the receiver’s bat port and one ESC’s
connector connected to one channel of the receiver where ESC connected with battery as
shown in the Fig 23 then red light started blinking in the receiver so then by keep pressing
on the binding key, switched on the transmitter then blinking of receiver light stopped, this
is sign that the transmitter and receiver are bounded.
Fig. 23. Binding of transmitter and receiver.
5.5
Steps involved in surveillance system
The visuals captured by IP camera are used for feature extraction process where the deep
learning techniques helped to get some crucial data from the images. Later on, there are series
of operations has to be carried out for data extraction from the images so the if images were
passed with primary layer, then they will be sent to next layer otherwise those images would
have been rejected from the process so in this way many visuals were rejected and many were
good in providing the needed data. The figures shown below explain the steps involved in
the system.
Step 1 : Streaming the quadcopter’s surveillance video on a computer/laptop using the IP
address
Fig. 24. Flow diagram of surveillance system.
16
ITM Web of Conferences 57, 02008 (2023)
ICAECT 2023
https://doi.org/10.1051/itmconf/20235702008

Step 2: Displaying the results by running the object detection models on surveillance visuals
Fig. 25. Flow diagram of image processing system.
•
Object Detection:
In this method work has discovered that these representations permitted to examine object
location frameworks with different manners & increase new knowledge into the identifier’s
failures. So here an algorithm has worked to identify objects utilizing shape highlights. The
accompanying algorithm dependent on discovering full item (Man and vehicles). Even
though there are some variations the algorithm is robust in doing its job.
Fig. 26. Detected Objects in image.
In Fig 26 algorithms tried to distinguish thing dependent on their corresponding structure.
The right-side picture showing the identified objects with green bounding box. The left-side
image also showing the similar things i.e., Human and tree, stands with bounding box.
6
Results and Discussions
This part of the report explains about the testing and validation of each and every component
& software tool used in the project. And all the difficulties faced during the tasks performed
portrayed in detail and the safety measures taken to overcome those issues. Along with these
many other suggestions are also added in the process to enhance the system efficiency.
6.1
Flight test on field
With the intention of flight, went to free surroundings. The whole system was equipped with
power distribution board and a battery to power up. And 2.4 GHz Radio controlled transmitter
and receiver for communication between quad-copter and pilot. So, after all the above steps
before switching on the quad-copter first switch on the transmitter and then switch on the
quadcopter by connecting the battery using XT60 connector, then after switching on both
systems, armed the quadcopter. without arming the quadcopter motors won’t spin so by
keeping left stick (throttle) down and move stick to right side as shown in the Fig 27, after
arming the quadcopter the yellow light switches on in the flight controller as shown in Fig
28 this showed that the quadcopter was armed and ready to fly.
17
ITM Web of Conferences 57, 02008 (2023)
ICAECT 2023
https://doi.org/10.1051/itmconf/20235702008

Fig. 27. Arming the quadcopter by transmitter.
Fig. 28. Armed signal in flight controller.
Fig. 29. Flight test of quadcopter.
After taking off the quadcopter got crashed because to fly the quadcopter one need some
practice so since didn’t have much practice, it got crashed by hitting to iron rod nearby as
shown in Fig 30 below.
Fig. 30. Quadcopter crash during flight test.
Luckily, none of the electronic components and parts were damaged. So later on tried to fly
it by tying the four arms as shown in the Fig 31 below.
18
ITM Web of Conferences 57, 02008 (2023)
ICAECT 2023
https://doi.org/10.1051/itmconf/20235702008

Fig. 31. Tied flight test of quadcopter.
Then tried to fly it by tying the arms, so quadcopter started to work properly without any
disturbances, when raised the throttle, the quadcopter started to fly within tied area as shown
in Fig 32, Then tried to operate quadcopter by giving the roll, pitch and yaw operations.
Fig. 32. Tied Quadcopter flight with maximum throttle.
Then by operating the right stick in the transmitter in forward direction the quadcopter started
to move forward by tilting towards frontal down as shown in Fig 33 and when did reverse
the quadcopter tilted in opposite direction by operating the right stick in right direction the
quadcopter tilted towards right as shown in Fig 34 and also tried for yaw movement because
of tying the quadcopter was not able to show yaw movement effectively anyhow it was
working.
Fig. 33. Quadcopter front pitch movement.
Fig. 34. Quadcopter Right roll movement.
19
ITM Web of Conferences 57, 02008 (2023)
ICAECT 2023
https://doi.org/10.1051/itmconf/20235702008

6.2
Surveillance test
After the flight test and confirmation of quadcopter efficient flight, an IP camera was fixed
below the quadcopter using suitable arrangements and tried for surveillance of surroundings
using the quadcopter-based surveillance system and captured some pictures as shown in the
figures below which were used for image processing for security purpose. Surveillance live
capture streaming was planned to get done by 5.4 GHz wireless video trans-receiver but it
was costly, to overcome this one, tried with android mobile with good camera as an IP
Camera using an application called IP-Webcam. The figures shown below are the
surveillance images captured from drone’s IP Camera.
Fig. 35. Surveillance image captured from IP-Webcam live video.
Fig. 36. Captured image in browser from IP-Webcam live video.
6.3
Image processing test
Detection and tracking of moving items in a video stream were the first relevant step of facts
extraction in computer vision packages together with people monitoring, video surveillance,
site visitors monitoring and semantic annotation of movies The input video records were
obtained from unmanned aerial automobile. Both images based and video-based performance
measures were considered for analysis.
6.3.1
Implementation of object detection algorithms
The results of proposed of open CV and tensor flow, yolo techniques for object detection for
different video and image data collected from the surveillance drone was shown and
discussed as a comparison with the normal surveillance methods. The video data consists of
a vehicle (car) as target detected in a cluttered environment. The video was a consistently
low altitude angular video taken by UAV at Belagavi, India. The video was taken for the
purpose of surveillance. The results of the video are shown below after implementation of
algorithms of open CV and tensor flow techniques.
20
ITM Web of Conferences 57, 02008 (2023)
ICAECT 2023
https://doi.org/10.1051/itmconf/20235702008

Fig. 37. Result image of object detection.
The Fig 37 shows the sample condition of object detection using image processing techniques
between two consecutive frames. The red and blue color bounding box tracks the cars in
consecutive frames. The UAV was in regular motion tracking the car was not moving. Since
there was no reasonable noise due to regular motion pattern of drone, the efficiency of
detection techniques was high.
6.4
Outcomes
•
The use UAVs reduced the human efforts in monitoring of the intolerable activities in
the society, which in turn improves the security and surveillance activities.
•
During surveillance, the use of image processing and deep learning techniques
minimizes the human efforts and gives efficient surveillance in absence of human interaction.
•
Increase in security and quality by the use of process through image processing and
deep learning tools.
•
Exposure to UAVs design and development methods.
•
Exposure to Mechanism of Quad-copter flight system.
•
Exposure of programming language of Python and software tools like Arduino IDE,
Multiwii and other deep learning tools like Open CV, Tensor flow, Yolo
•
Exposure of Electronic components and Mechanical systems used in the UAV systems.
•
Introduction to new methods of design and development processes which helps
understand the theory better.
•
Studying the design and development methods helped for better understanding of the
methods and the gap between academic and actual application part.
6.5
Future Scope of This Research Work
The study was concentrated on Surveillance of surrounding for security purposes using UAV
and Image processing tools where these two are separate systems in this project. However,
the study may be extended in the following areas :
•
The study may be extended to build drone flight completely rich with challenges that
are ready to be tackled by future researches in this area and this work has taken a step towards
the larger goal of fully autonomous drones.
•
Using the deep learning tools system can be developed with much efficient for
applications like obstacles detection and avoiding systems from attacks of enemies, which
can be used in the border surveillance and war fields.
•
Daily life activities can be run easily by enhancement of the surveillance application
especially in crime scenes.
•
The image processing can be optimized with greater efficiency by increasing the wide
variety of data sets and reducing the processing time & enhancement of accuracy.
21
ITM Web of Conferences 57, 02008 (2023)
ICAECT 2023
https://doi.org/10.1051/itmconf/20235702008

References
1.
H. Shakhatreh, A. Sawalmeh, A. A. Fuqaha, Unmanned Aerial Vehicles: A Survey on
Civil Applications and Key Research Challenges, arXiv: 1805.00881v1 [cs.RO], (19 Apr
2018)
2.
S. Kumar, E. Kanniga, Literature Survey on Unmanned Aerial Vehicle, International
Journal of Pure and Applied Mathematics Volume 119, No. 12 (2018)
3.
M. A. R. Khan, W. Ectors, T. Bellemans, Y. Ruichek, A. u. Haque Yasar, D. Janssens,
G. Wets, Unmanned Aerial Vehicle-based Traffic Analysis: A Case Study to Analyze
Traffic Streams at Urban Roundabout, Procedia Computer Science 130 (2018)
4.
M. A. R. Khan, W. Ectors, T. Bellemans, Y. Ruichek, A. u. Haque Yasar, D. Janssens,
G. Wets, Unmanned Aerial Vehicle-based Traffic Analysis: A Case Study to Analyze
Traffic Streams at Urban Roundabout, Procedia Computer Science 130 (2018)
5.
S. A. Meshram, A.V. Malvia, Traffic Surveillance by Counting and Classification of
Vehicles from Video using Image Processing,
International Journal of
Advance Research
in Computer Science and Management Studies, November (2013)
6.
Y. Li, Y. Zhang, L. Cai, Optimal location of supplementary node in UAV surveillance
system, Journal of Network and Computer Applications Volume 140, (15 August 2019)
7.
J. D. Reagan Vivek, Mariappan. M, M. J. Rickson.M, Autonomous Quad-copter for
Surveillance and Monitoring, International Journal of Advanced Research in Computer
Engineering & Technology (IJARCET) Volume 7, Issue 4, (April 2018)
8.
A. Vacca, H. Onishi, Drones: military weapons, surveillance or mapping tools for
environmental monitoring, Transportation Research Procedia 25 (2017)
9.
Ciara Bracken-Roche, David Lyon Mark, James Mansour, Surveillance Drones: Privacy
Implications of the Spread of Unmanned Aerial Vehicles, Surveillance Studies Centre,
Queen’s University, (April 30, 2014)
10.
Caleb, N. Nimyel, Alfred, N. Albert, Security of Unmanned Aerial Vehicles: A Literature
Review and Research Directions, American Journal of Engineering Research (AJER),
(02-10-2018)
11.
B. Mishra, D. Garg, P. Narang, V. Mishra, Drone-surveillance for search and rescue in
natural disaster, Computer Communications, Volume 156, (15 April 2020)
12.
C. F. Liew, D. DeLatte, N. Takeishi, T. Yairi, Recent Developments in Aerial Robotics:
A Survey and Prototypes Overview, (30 Nov 2017)
13.
A. Bucknell, T. Bassindale, An investigation into the effect of surveillance drones on
evidence at crime scenes, Science & Justice, Volume 57, (September 2017)
14.
S. Berrahal, J. H. Kim, S. Rekhis, N. Boudriga, Deon Wilkins, And Jaime Acevedo,
Border Surveillance Monitoring Using Quad-copter Uav-Aided Wireless Sensor
Networks, Journal of Communications Software and Systems, Vol. 12, No. 1, (March
2016)
15.
H. Hildmann, E. Kovacs, Using Unmanned Aerial Vehicles (UAVs) As Mobile Sensing
Platforms (MSPs) For Disaster Response, Civil Security and Public Safety,
Https://Www.Mdpi.Com/Journals, (2019)
16.
Dr. A. Govardhan, Image Retrieval Based on Color and Texture Features of the Image
Sub-blocks, International Journal of Computer Applications, (February 2011)
17.
E Unlu, E. Zenou, N. Riviere, Deep learning-based strategies for the detection and
tracking of drones using several cameras, Springer link, (24 July 2019)
22
ITM Web of Conferences 57, 02008 (2023)
ICAECT 2023
https://doi.org/10.1051/itmconf/20235702008

18.
V. Kishore Bhanse, Face Detection and Tracking Using Image Processing on Raspberry
Pi, International Conference, (2018)
19.
S. Bhosale, Face Detection and Recognition Techniques: A Survey, International Journal
of Advanced Research in Computer and Communication Engineering Vol. 7, Issue 11,
(November 2018)
23
ITM Web of Conferences 57, 02008 (2023)
ICAECT 2023
https://doi.org/10.1051/itmconf/20235702008