An Embedded Real-Time Object Detection and Measurement of its Size

Abstract

In these days, real-time object detection and dimensioning of objects is an important issue from many areas of industry. This is a vital topic of computer vision problems. This study presents an enhanced technique for detecting objects and computing their measurements in real time from video streams. We suggested an object measurement technique for real-time video by utilizing OpenCV libraries and includes the canny edge detection, dilation, and erosion algorithms. The suggested technique comprises of four stages: (1) identifying an object to be measured by using canny edge detection algorithm, (2) using morphological operators includes dilation and erosion algorithm to close gaps between edges, (3) find and sort contours, (4) measuring the dimensions of objects. In the implementation of the proposed technique, we designed a system that used OpenCV software library, Raspberry Pi 3 and Raspberry Camera. The proposed technique was nearly achieved 98% success in determines the size of the objects.

Full text

978-1-5386-6878-8/18/$31.00 ©2018 IEEE
An Embedded Real-Time Object Detection and
Measurement of its Size
Nashwan Adnan OTHMAN
Computer Engineering
Department
Firat University
Elazig, Turkey
nashwan.adnan92@gmail.com
Mehmet Umut SALUR
Computer Engineering
Department
Harran University
Sanliurfa, Turkey
umutsalur@harran.edu.tr
Mehmet KARAKOSE
Computer Engineering
Department
Firat University
Elazig, Turkey
mkarakose@firat.edu.tr
Ilhan AYDIN
Computer Engineering
Department
Firat University
Elazig, Turkey
iaydin@firat.edu.tr
Abstract—
In these days, real-time object detection and
dimensioning of objects is an important issue from many areas of
industry. This is a vital topic of computer vision problems. This
study presents an enhanced technique for detecting objects and
computing their measurements in real time from video streams.
We suggested an object measurement technique for real-time
video by utilizing OpenCV libraries and includes the canny edge
detection, dilation, and erosion algorithms. The suggested
technique comprises of four stages: (1) identifying an object to be
measured by using canny edge detection algorithm, (2) using
morphological operators includes dilation and erosion algorithm
to close gaps between edges, (3) find and sort contours, (4)
measuring the dimensions of objects. In the implementation of
the proposed technique, we designed a system that used OpenCV
software library, Raspberry Pi 3 and Raspberry Camera.
The
proposed technique was nearly achieved 98% success in
determines the size of the objects.
Index Terms— Object Detection, Object Dimension
Measurement, Computer Vision, OpenCV, Raspberry pi 3,
Canny Edge Detection.
I. I
NTRODUCTION
Real-time object detection and measurement systems are
very vital tasks in the industrial process. Object detection is
often used in product quality stages in the industry. The
proposed system can be applied to an industrial quality control
system. Likewise, it can be utilized for various industrial
systems or for security purposes. Generally, it is identifying
objects in public area and measure dimensions of each of
them.
The competence of the proposed system has been
confirmed through utilizing real videos that taken from a
Raspberry Pi camera. The execution of this procedure has a
high computation rate and it is dependent on the resolution of
frames. The achievement of identifying objects and separate
these objects from the background is perfect [1-4].
To calculate the size of each object, firstly we need to
determine the reference object. After that, the dimensions of
the reference objects will be used to calculate the size of other
objects. We calibrate the camera according to the reference
object. The reference object always is the left-most object in
the image. Also, to calibrate your pixels per metric variable,
the reference object can be utilized and from there, calculate
the size of other objects in all frames [5]. Completely
computational procedures are assessed through a raspbian
operating system using a raspberry pi 3 running with a
frequency of 1.3 GHz [6-10]. Every processes utilize the
libraries of OpenCV [11].
We have found various papers attached to a measurement
system. Each measurement applications are utilized for various
purposes. A movable outdoor distance measurement system
was proposed and 90% precision were obtained [12]. The
distance was measured by utilizing through utilizing S3C2410.
For this purpose, a temperature reparation module was used to
improve the accuracy [13]. An ultrasonic distance
measurement system was proposed for embedding distance
measurement [14]. The sizes of objects in an image are
estimated by using computer vision methods [15].
II. T
OGY FOR REALHE PROPOSED METHODOL
-
TIME OBJECT
EMENTSDETECTION AND MEASUR
The system consists of two parts which are object
detection and object measurement. In the first part, raspberry
pi camera used to achieve the frames. In the second part,
computer vision module will be applied to the captured frames
to determine the objects, then, we will measure each object.
The detected object of the current frame immediately will be
processed to extract dimensions of objects.
In the proposed system, firstly, we need to preprocess our
image. The camera will capture a frame and the frame will
convert to grayscale to increase quickness and accuracy.
Objects are detected via canny edge detector algorithm. It is
used to detect only one object or multiple objects. By the help
of canny edge detector, the converted image will be processed.
The canny edge algorithm scans the entire image. After that,
execute dilation and erosion algorithm to close holes among
edges in the edge frame [16-18]. Figure 1, shows that the flow
chart for the recommended system.

Begin
Capture frame
Object Identification
Object Measurement
Save on local storage
See the Output on the Screen
Figure 1. Flowchart for proposed method
Figure 2, shows the input frame that used for canny edge
detection. First stage in canny edge detector algorithm is
delete the noise in the frames by applying a Gaussian filter.
The frame after converting to gray scale and apply Gaussian
filter is appeared in Figure 3.
Figure 2. Input frame
Figure 3. Grayscale and Smoothed input frame
In the compute gradient stage, we detect the edge gradient
and direction for each pixel. For the define the gradient at
every pixel of smoothened frame, Sobel operator utilized.
A complete scan of frame will be done afterward receiving
gradient magnitude and direction, to eliminate any undesirable
pixels which might not establish the edge. In this stage, just
local maxima must be considered as edges through applying
Non-maximum suppression. Non-maximum suppression
exchanges the smoothed edges in the frame of the gradient
magnitudes to sharp edges. Non-maximum suppression is
carried out to keep every local maximum in the gradient
picture, and removing the whole thing else. Figure 4, shows
that the frame after apply non-maximum suppression.
Figure 4. Input frame apply non-maximum suppression
The final stage of canny edge detector algorithm is
hysteresis thresholding. This stage selects which are every
edges are surely edges and which aren’t edges. The two
threshold values are empirically selected and their definition
will upon on the content of a given frame. This is achieved via
choosing big and small threshold values. If Edge pixels
stronger than the big threshold, it is marked such as sturdy.
Strong edges will be measured as the last edges. Also, edge
pixel will be suppressed If an edge pixels weaker than the
small threshold, and it is marked as weak edge if an edge pixel
among the big and small thresholds.
To obtain the better result and more accurate object
detection, the canny edge detection procedure has been
improved with some Morphological operations [19]. These
procedures are commonly a combination of nonlinear
procedures performed relatively on the preparation of pixels
without changing their numeral values. erosion and dilation
are the keys for morphological operations.
In this study, a morphological process is performed such as
a mixture of dilation and erosion. The opening is the initial
procedure in which erosion is followed through dilation.
Closing is the second operation in which dilation is followed
through erosion. As a mixture of these processes we are
capable to acquire superior determination for discovery edges
in-depth frame. Figure 5 shows that the frame after applying
erosion and dilation operation.
Figure 5. Input frame after apply dilation & erosion operation
To briefly summarize object measurement, after edge
detection and close any gaps between edges, we detect
contours by using an OpenCV function that is
cv2.findContours to find the shapes of the objects in the edge

map. We arrange contours from left to right. The reference
object in the frame is permanently the left one. By depending
to the reference object, we calibrate the camera and set the
value of parameter. Next, we scan every contours, begin
looping above every individual contours. After that, the
rectangle around objects will be drawn in green. So, the points
of the bounding box rectangle will draw in a small purple
rounds. After that, we can get midpoints because the bounding
box is ordered. Finally, we calculate pixels Per Metric variable
through dependence on reference object. The height-distance
in pixels will put on hD (height) variable and width distance
will put on wD (width) variable. Then, we calculated the
Euclidean distance among sets of center points.
III.
XPERIMENTAL
E
ESULTS
R
We proposed the system to measure objects in a real time
video and pictures. We prepared a few experimental setups to
test the correctness of the proposed method. The implement
the proposed system has made by the help of Python language.
Figure 6. show that the setup of the prepared system. Except
the hardware formation, the software’s required has installed.
Figure 6. Setup of proposed system
For the experiment the camera has been effectively
capturing the pictures. The proposed system applies four
operations such as record frames, find edges, find objects, and
measure size for each objects. When we run the application,
the output screen displays on the PC screen as appear in
Figure 7.
Figure 7. Display output on the screen
Figure 8, illustrates the object detection and
measurements. The size of each object in the frame which are
calculated.
Figure 8. Calculate the size of objects
In the first experiment we measured size of objects such as
white glasses, orange cup, bottle, potatoes. Table I, shows the
accuracy of proposed object measurement system for these
objects. Abbreviations in the table are as follows; AM-H:
Actual Measure-Height, PM-H: Proposed Measure-Height,
AM-W: Actual Measure-Width, PM-W: Proposed Measure-
Width.
TABLE I
. ACCURACY OBJECT MEASUREMENT FOR ONE FRAME
Name of
objects
AM-H
(cm)
PM-H
(cm)
AM-W
(cm)
PM-W
(cm)
Accuracy
(%)
White glasses 10.0 10.5 6.8 7.1 % 95.45
Orange cup 8.5 9.0 7.5 7.4 % 97.56
Bottle 15.2 14.9 7.4 7.3
% 98.23
Potatoes 4.8 5.0 7.4 7.6 % 96.82
Nevertheless, not every results are perfect, since this is due
to the seeing angle and lens deformation. By calibrate the
camera and set good width parameter, accuracy will be
increase.
In the second experiment, we set the camera above the
objects. Figure 9, shows that the results of the object detection
and measurement for another objects. And the Table II,
demonstrations the accuracy values among actual measure and
system measure.
Figure 9. Experimental result when camera above the objects

TABLE II.
ACCURACY OBJECT MEASUREMENT WHEN CAMERA
ABOVE THE OBJECTS
Name of
objects
AM-H
(cm)
PM-H
(cm)
AM-W
(cm)
PM-W
(cm)
Accuracy
(%)
Turk kurus 1.7 1.7 1.7 1.8 % 97.14
Mouse 10.0 10.1 5.2 5.7 % 96.20
Card reader 3.8 3.8 1.7 1.8 % 98.21
Peace of paper 4.6 4.7 1.2 1.3 % 96.66
tray 12.6 12.9 12.6 13.0 % 97.29
Charger 7.6 7.6 3.7 3.9
% 98.26
plaster 4.4 4.4 4.4 4.2 % 97.72
The result for error column displays very low errors. The
error rate is especially smaller when camera above the objects.
IV.
ONCLUSION
C
In this study, an powerful real time object measurement
method is proposed for industrial systems. In the offered
system, Computer Vision used to detect and measure objects.
The system can detect and measure objects in a real time
video. After the object has been detected by using canny edge
detector, the size is obtained for each object by using OpenCV
functions. We enhanced the canny edge detector algorithm
through utilizing Morphological operations. This procedure
benefits to eliminate extra noises. Furthermore, whereas
eliminating the extra noises it likewise smoothens the shape
and keeps the outline and size of each object. Thus, the
outlines of the different objects in the scene were kept.
The proposed technique works very fast and five frames
can be processed pending one second. Raspberry Pi 3 used to
implement the systems since it has very great features and low
cost embedded equipment platform.
R
EFERENCES
[1] Muthukrishnan.R and M.Radha “Edge Detection Techniques for image
Segmentation” International Journal of Computer Science &
Information Technology (IJCSIT) Vol3, No 6, Dec 2011.
[2] Geng Xing, Chen ken , Hu Xiaoguang “An improved Canny edge
detection algorithm for color image” IEEE TRANSATION ,2012 978-1-
4673-0311-8/12/$31.00 ©2012 IEEE.
[3] Chen.W, Yue.H, Wang.J, Wu.X. An improved edge detection algorithm
for depth map in painting. Optics and Lasers in Engineering 2014;55:
69–77.
[4] Moeslund T. Canny Edge detection. Denmark: Laboratory of Computer
Vision and Media Technology,AalborgUniversity. 〈http://www.cvmt.dk/
education/teaching/f09/VGIS8/AIP/canny_09gr820.pdf〉 [March 2009].
[5] OpenCV, 2016. http://opencv.org/.[Accessed December 23, 2016].
[6] I. Aydin and N. A. Othman, "A new IoT combined face detection of
people by using computer vision for security application," 2017
International Artificial Intelligence and Data Processing Symposium
(IDAP), Malatya, 2017, pp. 1-6.
[7] N. A. Othman and I. Aydin, "A new IoT combined body detection of
people by using computer vision for security application," 2017 9th
International Conference on Computational Intelligence and
Communication Networks (CICN), Girne, 2017, pp. 108-112.
[8] Raspberry Pi, 2015. https://www.raspberrypi.org.[Accessed may 4,
2015).
[9] Virginia Menezes, Vamsikrishna Patchava, M. Surya Deekshith Gupta,
"Surveillance and monitoring system using Raspberry Pi and
SimpleCV", 2015 International Conference on Green Computing and
Internet of Things (ICGCIoT), vol. 00, no. pp. 1276-1278, 2015,
doi:10.1109/ICGCIoT.2015.7380661.
[10] Ms. RenukaChuimurkar, Prof. Vijay Bagdi, “Smart Surveillance
Security &Monitoring System Using Raspberry PI and PIR Sensor”,
International Journal of Scientific Engineering and Applied Science
(IJSEAS) – Volume-2, Issue-1, January 2016 ISSN: 2395-3470
[11] G. Bradsky, A. Kaehler, “Learning OpenCV, Computer Vision with the
OpenCV Library” (Book Style), O’Reilly Media, Sebastopol, CA, 2008.
[12] Y. Jang, S. Shin, J. W. Lee, and S. Kim, “A preliminary study for
portable walking distance measurement system using ultrasonic
sensors,” Proceedings of the 29th Annual IEEE International
Conference of the EMBS, France, Aug. 2007, pp. 5290-5293.
[13] H. He, and J. Liu, “The design of ultrasonic distance measurement
system based on S3C2410,” Proceedings of the IEEE International
Conference on Intelligent Computation Technology and Automation,
Oct. 2008, pp. 44-47.
[14] D. Webster, “A pulsed ultrasonic distance measurement system based
upon phase digitizing,” IEEE Transaction on Instrumentation and
Measurement, Vol. 43, No. 4, Aug. 1994, pp. 578-582.
[15] Kainz, O., Jakab, F., Horečný, M. W., & Cymbalák, D. (2015, October).
Estimating the object size from static 2D image. In Computing and
Communication (IEMCON), 2015 International Conference and
Workshop on (pp. 1-5). IEEE.
[16] Z. Xu, X. Baojie and W. Guoxin, "Canny edge detection based on Open
CV," 2017 13th IEEE International Conference on Electronic
Measurement & Instruments (ICEMI), Yangzhou, 2017, pp. 53-56.
doi: 10.1109/ICEMI.2017.8265710
[17] I. M. O. Widyantara, N. M. Ary Esta Dewi Wirastuti, I. M. D. P. Asana
and I. B. P. Adnyana, "Gamma correction-based image enhancement
and canny edge detection for shoreline extraction from coastal
imagery," 2017 1st International Conference on Informatics and
Computational Sciences (ICICoS), Semarang, 2017, pp. 17-22.
[18] Y. y. Zheng, J. l. Rao and L. Wu, "Edge detection methods in digital
image processing," 2010 5th International Conference on Computer
Science & Education, Hefei, 2010, pp. 471-473.
[19] Morphological image analysis. Principles and applications by Pierre
Soille. ISBN 3-540-65671-5 (1999), 2nd edition (2003).