Integrating Face Recognition Security System with the Internet of Things

Full text

Integrating Face Recognition Security System With
the Internet of Things
Ibrahim Mohammad Sayem
Dept. of Computer Science and
Engineering
University of Chittagong
Chittagong, Bangladesh
ibmcsecu@gmail.com
Mohammad Sanaullah
Chowdhury
Dept. of Computer Science and
Engineering
University of Chittagong
Chittagong, Bangladesh
sana1691@gmail.com
Abstract— A perennial need for safety in the community depends on
country, city, and district. In some instances, feeling safe is required
on a 24/7 basis. A popular and cost-effective solution based on the
Raspberry Pi has the promise of being both user-friendly and cost-
effective. It pairs the Raspberry Pi to a camera module for face
recognition. It learns to detect those with granted access to the
specified area under protection. Such stored faces are the subject of
system training. If during operation the system recognizes the face in
the dataset, then the camera shows the matching name with a
confidence level possibly granting access, but alternatively it takes a
photo of the subject and sends it as an email notifications warning.
The proposed system can implement face recognition even from poor
quality images performing well over both known and unknown
datasets. Face recognition leverages techniques from the OpenCV
library and is written in the Python language.
Keywords— Raspberry Pi, Local Binary Pattern Histograms,
OpenCV, Face Recognition, Gmail Notifications.
I. INTRODUCTION
IoT is commonly the network of interconnected things or
devices which are embedded with sensors, software,
network connectivity and essential electronics that allow
them to gather and transfer data to making them responsive.
CASAGRAS(2008) stated that IoT is defined as A global
network infrastructure, linking physical and virtual objects
through the exploitation of data capture and communication
capabilities[1]. However, Adoption of the Internet of
Things is proceeding more slowly than expected, explained
in an article by Mark Patel, Jason Shangkuan, and
Christopher Thomas(2017)[2].
During the past several years, Face Recognition becomes
extremely essential to have a dependable security system
which can secure our homes, offices, industry, and property
in the safest way possible. A conventional safety system
has a lot of problems and lacking, sometimes they are
unable to provide real-time data. Moreover, Face
Recognition is one of the various popular methods of
biometric technology which can uniquely verify and
identify a person by extracting different facial features
mathematically and uses a different algorithm to recognize
a person. When contrasted to other biometric technologies
such as pattern matching, fingerprint, voice recognition, etc.
Face Recognition achieved significant attention in security
purposes. Besides, Face Recognition based security system
is more natural than other technologies.
Many research has been taking place in this field in recent
few years. Ylber Januzaja, Artan Lumaa, Ymer Januzaja
and Vehbi Ramajb(2015) presented a system Real-time
accessed control based on face recognition using Raspberry
Pi and used OpenCV libraries and python language in order
to achieve high effectiveness[3]. Although some
researchers used machine learning and deep learning based
approach to get much better result, likewise Li Xinhua and
Yu Qian(2015) addressed a system which focused on the
face recognition problem and used deep learning method,
convolution neural network, to solve the problem along
with the Sobel operator to improve the accuracy[4].
Xiang Xu, Wanquan Liu, Ling Li(2014) developed a
system Low-Resolution Face Recognition in Surveillance
Systems that has the ability to detect and recognize known
and unknown face in quite low resolution[5]. The face
recognition can be summed to feature extraction from face
images to get the feature vector first, and put it into a
trained classifier, at last, get the class and finish the
recognition task. More specifically, in our system at first,
we have to take some human faces, then we have to train
them and lastly, we can recognize known and unknown
faces where unknown faces will send via Email. We don't
have to worry about the recordings because pi security
system will send an Email and the system is also user-
friendly.
14
2018 International Conference on Machine Learning and Data Engineering (iCMLDE)
978-1-7281-0404-1/19/$31.00 ©2019 IEEE
DOI 10.1109/iCMLDE.2018.00013

The rest of the paper is structured as follows. In Section II we
briefly describe the system requirements for our application.
In Section III we describe the system architecture and
workflow and evaluate it in Section VI. We conclude the
paper in Section V.
II. SYSTEM REQUIREMENTS
The requirements to build the system are given below:
A. Hardware Requirements:
B. Raspberry Pi 3 Model B+
The main system is installed with Raspberry Pi 3 Model B+. It
is a tiny, affordable credit card size computer which is
connecting with a computer monitor and uses keyboard and
mouse. The Raspberry Pi 3 B+ is a third generation Raspberry
Pi. It requires 2 Amperes of current to power up and can
tolerate up to 5Volts. It has 4× ARM Cortex-A53, 1.2GHz
processor, 10/100 Ethernet, 2.4GHz 802.11n wireless, and
Bluetooth 4.1 Classic. It also has 1GB LPDDR2 RAM and
Broadcom Video Core IV GPU. It has different ports like
HDMI, 3.5mm analog audio-video jack, 4 USB 2.0, Ethernet,
Display Serial Interface (DSI). In our system, we used the
Camera Serial Interface (CSI) for using the Raspberry Pi
camera. It also has Micro SD card slot for entering OS
installed memory card.
C. Raspberry Pi Camera Module V2
For our system, we have chosen the Raspberry Pi Camera
Module V2. The Raspberry Pi Camera Module v2 is an
amazing quality 8 megapixel Sony IMX219 picture sensor
custom composed add-on board for Raspberry Pi, emphasizing
a fixed focus camera. It's fitted with 3280 x 2464 pixels
motionless pictures, and further hold 1080p30, 720p60 and
640x480p90 video. It appends to Pi by means of one of the
little sockets on the board upper surface.
D. Programming Languages: Python
Python is one of the most powerful programming languages in
today's world particularly in the IoT and Machine
Learning(ML) field. Mainly, Raspberry Pi supports two
languages: C++ and Python. In our system, we use Python
because it is an object-oriented, high-level, general-purpose
programming language with dynamic semantics and it is also
interpreted. It is very simple to use, the syntax is easy to learn
and therefore it reduces the program maintenance cost.
E. Software Library: OpenCV
OpenCV (Open Source Computer Vision Library) is an open
source computer vision and machine learning software library.
OpenCV was built to provide a common infrastructure for
computer vision applications and to accelerate the use of
machine perception in the commercial products. It helps in
real-time image processing. OpenCV has C++, Java and
Python and interfaces and holds various OS like Windows,
Linux, Android etc.. It was created for computational edge and
including a robust focus on real-time use. It is Formulated in
C/C++, the library can use multi-core processing. OpenCV
3.3.0 now comes with the very new Face Recognizer class for
face recognition and in our system, we use OpenCV 3.3.0.
F. Email Server: SMTP
Simple Mail Transfer Protocol(SMTP) is a TCP/IP protocol
used for sending and receiving an email which also takes care
of the whole email sending process. But at first, it needs the
correct SMTP settings, in particular, the right SMTP
address(for instance:smtp.gmail.com:587). We use port
number 587 for Raspberry Pi.
III. SYSTEM ARCHITECTURE AND WORK FLOW
Figure 1 describes the overall system architecture of the
proposed system. Moreover, the main goal is to develop a
user-friendly smart security camera using Raspberry Pi to
recognize known and unknown faces which later sends email
alerts for unknown faces. We divided the system architecture
into different parts to show the workflow of the system.
Figure 1. Overall System Architecture
15

Step 1: Raspberry Pi camera is used to take images of people
for further use and these images are saved by giving specific
id in dataset directory. We take 30 pictures of each person.
Step 2: After capturing images there is a pre-processing step
where different operations such as image resize, grayscale
conversion and image enhancement occur. Grayscale
conversion is important because seldom the color information
doesn't help us identify important edges or other features. An
RGB image consists of 3 layers R, G, B and it's a 3-
dimensional matrix so when we convert an RGB picture to
grayscale, we need to take the RGB values for every pixel.
Furthermore, Face images may suffer from illumination,
therefore, image enhancement is necessary to improve the
quality of face image which helps face recognition system to
perform effectively and accurately[6].
Step 3: The primary purpose of face detection algorithms is to
determine whether there is any face in an image or not and it
is the process that can apply to stored images or images from a
camera. Ordinarily, face detection applications use machine
learning methods for detecting human faces within images or
videos. These images might contain any distinct objects which
are not part of images such as trees, buildings and different
parts of humans etc. are ignored from the digital image. While
the process is somewhat complex, face detection algorithms
often begin by searching for human eyes. Furthermore, once
eyes detection process is complete, the algorithm might then
attempt to detect facial regions, including eyebrows, the
mouth, nose, nostrils, and the iris.
Step 4: In this phase, we extract various facial features like as
mouth, eyes, nose, eyebrows etc. from the detected face. There
are three types of feature extraction methods: Generic
methods, Feature template-based methods and Appearance-
based methods.
Step 5: There are different types of face recognition
algorithms, for example, Eigenfaces(1991), Local Binary
Pattern Histograms(LBPH)(1996) and Fisherfaces(1997)[7].
Each method has a different approach to extract the image
information and perform the matching with the input image. In
our system we use LBPH. LBPH uses 4 parameters such as
Radius, Neighbors, Grid X and Grid Y. After that, we train the
algorithm and applying the LBP operation which marks the
pixels of a picture by thresholding the 3-by-3 neighborhood of
every pixel with the focus pixel esteem and considering the
outcome as a parallel number. Then extracting the histograms
which are followed by Face Recognition.
Step 6: Machine Learning (ML) algorithms are often
categorized as supervised or unsupervised. Machine learning
is used to detect patterns in data and adjust program actions
respectively. In our system ML, first look at pictures and find
all faces in it. Furthermore, ML is able to find out the unique
features of different people and then compare the unique
features of all the people we already know.
IV. IMPLEMENTATIONS AND EXPERIMENTAL RESULTS
We use Raspberry Pi 3 Model B as our IoT core and
Raspberry Pi camera module as the system camera also, we
installed OpenCV 3 for face recognition process and all scripts
in our system are written in Python. The implementation of
the proposed system includes the following 4 stages:
1. We capture 30 images of each person by giving them
specific id and applied face detecting process.
2. On the second stage, We train the images by considering
numeric vectors and the final values will be exported to a
YML file called trainer.yml.
3. In the third stage, we will capture a fresh face on our
camera and if this person's face trained before, our recognizer
will make a prediction returning its id and confidence level of
the system.
4. In the final stage, the system will send an email if the
system is unable to recognize the face.
Figure 2: Implementation of the Proposed system
Figure 3 is the result of stage 1 where we gathered data using
the Raspberry Pi Camera module and perform the detecting
process. Images are stored with a specific id for each person.
All images are saved in .jpg format and named as “User”. We
saved all images in the “dataset” directory and during the
image training stage we take all images from “dataset”
directory and after training process is complete we stored it in
“trainer” directory.
16

Figure 3: Dataset Gathering
After the images are being trained, the system is ready to
recognize faces and if any faces are found in the trained file it
will show the name of the person as well as show the results
with a confident label, but if it is not trained it will display
unknown and save it to the unknown directory and the image
is forward through email. Figure 4 shows the result of the Face
Recognition procedure.
Figure 4: Face Recognition
Figure 5 shows the result of email alert in our system. To send
the email we use SMTP server which is a simple mail transfer
protocol and we configure Gmail SMTP server settings to
send email from Raspberry Pi account to any Gmail account.
We send only unrecognized face through email because we
need only those people faces whose are not allowed to enter
any particular area.
Figure 5: Email Alert
V. CONCLUSION AND FUTURE WORKS
Face recognition has great potential in improving security
operatives to better carry out their duties, especially in
emerging countries where this technology is not currently
extensively used. In this paper, we developed a facial
recognition system using a raspberry pi, which is smaller,
lighter and works successfully utilizing the lower control
power with a security alert message to the authorized person
utilities. This system is more convenient than the pc-based
face recognition system. This development system is
affordable, agile, and highly secure and Raspberry Pi takes
limited power and provides enough flexibility to satisfy the
requirement of different people.
However, there is still a lot of scope for improvement. We can
also use this security system by making a required
modification to the system in an area like the banking sector to
provide more security to the lockers, based on their facial
authentication and keep a record of account holders history of
data when and who is entering the lockers.
17

R
EFERENCES
[1] CASAGRAS(2014). RFID and the Inclusive Model for the Internet of
Things. EU Project Number 216803.
[2] Mark Patel, Jason Shangkuan, and Christopher Thomas(2017). Adoption
of the Internet of Things is proceeding more slowly than expected.
Mckinsey&Company article may,2017.
[3] Ylber Januzaja, Artan Lumaa, Ymer Januzaja and Vehbi Ramajb(2015).
Real-time Accessed Control Based on Face Recognition. International
Conference on Network security & Computer Science (ICNSCS-15) June 10-
11, 2015 Antalya (Turkey).
[4] Li Xinhua and Yu Qian(2015). Face Recognition based on Deep Neural
Network. International Journal of Signal Processing, Image Processing and
Pattern Recognition Vol.8, No.10 (2015), pp.29-38.
[5] Xiang Xu, Wanquan Liu, Ling Li(2014). Low Resolution Face
Recognition inSurveillance Systems. Journal of Computer and
Communications, 2014, 2, 70-77.
[6] Rok Novosel, Blaz Meden, Ziga Emersic, Vitomir Struc and Peter
Peer(2017). Face recognition with Raspberry Pi for IoT Environments. 26th
ERK International Electrotechnical and Computer Science Conference · ERK
2017.
[7] Mrutyunjanya Sahani, Subhashree Subudhi and Mihir Narayan
Mohanty(2016). Design of Face Recognition based Embedded Home Security
System. KSII TRANSACTIONS ON INTERNET AND INFORMATION
SYSTEMS VOL. 10, NO. 4, Apr. 2016.
18