Design of Face Recognition based Embedded Home Security System

Abstract

Home security has become the prime concern for everyone in present scenario. In this work an attempt has been made to develop a home security system which is accessible, affordable and yet effective.The proposed system is based on 'Remote Embedded Control System' (RECS) which works both on the web and gsm platform for authentication and monitoring. This system is therefore cost effective as it relies on existing network infrastructure. As PCA is most popular and efficient algorithm for face recognition, it has been usedin this work. Next to it an interface has been developed for communication purpose in the embedded security system through the ZigBee module. Based on this embedded system, automated control of door movement has been implemented through electromagnetic door lock technology. This helps the users to monitor the real-time activities through web services/SMS. The web service consists of either web browser command or e-mail provision. The system establishes the communication between the system and authenticated user. The e-mail received by the system from the authorized person will monitor and control the real-time operation and door lock. The entire control system is reinforced using ARM1176JZF-S microcontroller and tested for actual use in the home environment. The result shows the experimental verification of the proposed system.

1. Introduction

There is a wide range of commercial and security applications based on image processing.Butan efficient and accurate embedded access control system based on facial recognition technologyis still a challenge. This technology is being gradually adopted in several applications across the globe [1-2]. Accurate identification of the visitors to the control area is the heart of the solution of this modern technology [3-4].For the physical control of the entrance of the control area other technologies like passwords, radio frequency identification (RFID) sensors, fingerprints and face recognition methods are used in [5]. Among all the above techniques the efficient biometric authentication system is based on image processing for security issues. Face recognition is an alternative to finger print as biometric trait [6-8].

Face recognition technique in real-time embedded system is a moduleof the proposed design. Some of the work has been progressed by few reserchers in this field. For face recognition on hardware platform most of the systems implement a principal component analysis(PCA) algorithm [9-12].As it is of simple and reduced dimension it became popular[13-17]. For access control system other wireless technologies like RFID, ultra wideband(UWB) and ZigBee are used in [18].

This piece of work proposes a wireless access control system and designed a smart home environment. The physical components that are needed to be installed for functioning of virtual home security systems are: the camera unit and the Raspberry Pi unit. These components are installed in a home through ZigBee. The virtual home security system functions by intimating the house owner about any visitor who confronts the camera unit by SMS or E-mail. He/she also gets the option of interacting with the embedded device in real-time through a webpage without the necessity of an additional server. All the instructions are verified before the implementation for security and safety. These will be processed in virtual environment prior to real environment implementation. The proposed system is energy efficient, intelligent, low cost, portable and delivers high performance.

This paper is organized into five sections. Section II explains the architectural design of the proposed system. The real-time implementation of system through webpages is described in section III. Section IV contains the experimental result and discussion. Finally a summary and conclusion is drawn in section V.

 

2. Design of the Proposed System

Raspberry Pi acts as an Internet Gateway Device and has the ability to route data coming from the WSN network to internet and send data coming from internet to WSN network. It hosts an Embedded Web Server (EWS) on it for implementation of TCP/IP protocols[19-22]. By implementing TCP/IP stack on microcontroller we can control and monitor device status through the webpage. The embedded web server is used to serve web pages. The web pages can be accessed by giving the IP address provided to EWS.

Fig. 1 shows the architecture of the proposed system. The major components of door access control are: Wireless Control Unit(WCU) and Wireless Information Unit(WIU).The two units are linked to each other by radio transceivers which allows the transfer of information from the control unit. The WCU uses ZigBee technology to design wireless sensor network. Here ZigBee end devices are connected to relay driver circuit to control the electromagnetic door lock module. The WIU uses Raspberry Pi as its processing unit. Model B+ of Pi is used in this work. It contains a System on Chip (SoC) named BCM2835 which we find in many mobile phones. The SoC integrates an ARM1176JZ-F processor, a graphic processing unit named Video Core IV and 512 MB ram into a single unit. This model has 40 GPIO pins,4 USB ports and an Ethernet port. Pi doesn’t have a persistent internal memory so it is booted up from SD card. A camera unit is connected to Pi to capture images of visitors. To access and send emails to the consumer python coded algorithm is fed into it.

Fig. 1.Layout of the proposed system configuration; (a) Wireless Information Unit, (b) Wireless Control Unit.

For transmitting data via mobile networks a GPRS module is also available in WIU. This module can send alert SMS about the visitors to the concerned authorities. Data captured by smart camera is transmitted over the internet by integrating an internet gateway with ZigBee network.In a ZigBee network end devices collect and forward data to a coordinator and then ZigBee protocol data format is translated to internet protocol (IPv6) format by the gateway. The proposed design consists of following modules and is described briefly.

2.1. Face Detection Module

A face detector has to tell whether an image of arbitrary size contains a human face and if so, where it is. To enable a fast and accurate detection Viola-Jones algorithm is used[23-24]. Here a system is designed by giving input as some faces and non-faces and training a classifier that identifies the face. We have trained data and using trained data, we try to classify new data as faces or non-faces. The Viola-Jones algorithm consists of four steps: Haar features, Integral image, Adaboost and Cascading.

Haar-feature based cascade classifier for object detection has proved to be an effective object detection method. A cascade is a series of haar-like features that are combined to form a classifier. Initially, the classifier is trained by taking a lot of positive images (images of faces) and negative images (images without faces). Then features are extracted from it by using haar features.Haar features closely resembles a convolution kernel.

Each feature is a single value obtained by subtracting the sum of pixels under white rectangle from the sum of pixels under black rectangle. If a 24×24 window is moved across an image, then it results over 160000 features. Sum of pixels under white and black rectangles must be calculated for each feature. This results in huge calculation. To solve this problem ‘Integral Images’ are introduced. It simplifies a large number of pixels to an operation involving just four pixels. Among all the calculated features, most of them are irrelevant. For example, in Fig. 2 top row shows two good features. The first feature focus on the property that region of the eyes is darker than the region of the nose and cheeks. As per the second feature eyes are darker than the bridge of the nose. But application of same window on cheeks or any other places results in irrelevant features.

Fig. 2.Haar-like feature detector

So we use ‘Adaboost’ to eliminate irrelevant and redundant features and select the best features out of 160000 features. Here each and every feature are applied on all the training images and the best threshold is found for each feature which will classify the features to positive and negative. But obviously there will be errors or misclassifications. The features with minimum error rates are selected, which means they are features that best classifies the face and non-face images. Final classifier is a weighted sum of these weak classifiers. It is called weak because, it can’t classify images alone, but together it forms a strong classifier. Therefore, after applying Adaboost the total features have reduced from 160000 to 6000 features.

Most of the image regions in an image are non-face region so a technique is developed to check if a window is not a face region. If it is not a face region, then it can be discarded in a single shot. Hence, more time can be dedicated on possible face regions rather than non-face regions. This technique is called ‘Cascading of Classifiers’. Here instead of applying all 6000 features on a window, features are grouped into different stages of classifiers and applied one-by-one. 10 features out of 6000 are evaluated per sub-window. If window fails first stage it is discarded and remaining features on it are not considered. If it clears first stage, then the second stage of features is applied and the process is continued. If all the stages pass by a window, then it is a face region.

2.2. Face Recognition Module

Now-a-days for security purpose face recognition is widely used. Face recognition methods have been reported in the literature [25-29]. Face recognition with spatial misalignment and image occlusion is discussed in [25]. A new unsupervised feature selection algorithm, named clustering-guided sparse structural learning(CGSSL) is proposed in [26]. Accuracy improvement of color face recognition based on PCA was suggested in [27]. Principal Component Analysis(PCA) is found as an efficient method for face recognition.To recognize and determine patterns to reduce the dimensions of the dataset with insignificant loss of information PCA technique has been employed[28]. PCA uses an orthogonal transformation to convert a set of M face images into a set of K uncorrelated variables called Eigen Faces [29]. Based on this logic the algorithm is developed for our proposed method and described as follows:

Fig. 3 shows the Flow Chart of PCA algorithm. To implement PCA algorithm a training set consisting of total 200 images is created and loaded into a database. All face images must be of the same dimension. Each image is having a dimension 50×50. Then the face images in the training set are converted to face vectors Γi. The average face vector Ψ is then calculated. Average face vector represents the unique features of all the 200 images and the test image. After that the normalized face vectorΦi is obtained by subtracting the average face vector from each face vector.

Fig. 3.Flow chart of PCA algorithm

Next the covariance matrix C is calculated.

Where A = [Φ1Φ2……ΦN]

The dimensionality of C is very large. Higher dimensionality may slow down the system terribly or results in very huge computation and noise. So to reduce dimensionality of training set evaluate C as:-

Hence Eigenvector is calculated from the covariance matix with reduced dimensionality. Then K best eigen faces are selected such that K< 100 and represents the whole training set. The lower dimensional K eigenvectors are then converted to original face dimensionality.

Each face image is then represented as a linear combination of all K eigenvectors. The distance between input weight vector and all weight vectors of the training set are calculated.

If the distance is greater than threshold value, then it is known face else it is an unknown face.

The output is shown in Fig. 4.

Fig. 4.Face detection and Recognition Result

2.3. Zigbee Module

The proposed home security system uses the ZigBee communication technology. This technology is energy efficient, self configuring, low cost and yet gives high accuracy [30-31].In the proposed system the control module and the sink module use ZigBee platform for communication. XBee S2 module working on 2.4 GHz band has been used. This module, although working in 2.4 GHz band transmits data and receives them serially. The configuration of ZigBee module has been done through X-CTU software. In this system network it is possible to converse 8 data modules to the single sink module which is coupled to a Raspberry Pi. The private area network ID remains the same as that of developed control and sink modules. If proper configuration is done network connection between the control modules and sink node gets automatically established. Every equipment in the system is configured to send their status in every 4 seconds to the coordinators.

2.4. Lock Module

The Lock module consists of a ZigBee end device (ZED), an electromagnetic lock and a control circuit. For operating the ZED a 5v DC circuit has been used. The ZED circuit controls the electromagnetic lock through control circuit. The control circuit de-energizes the lock to open the door when “open” command is received. Otherwise the lock gets energized to lock the door as shown in Fig. 5.

Fig. 5.Wireless control unit

2.5. IOT Application Gateway

The program at the IOT application gateway executes the transformation of control information between the ZigBee and IPv6 network, as the architecture to communicate with internet protocols is lacking in the ZigBee network. The ZigBee address is transformed and data payloads are encapsulated in an internet protocol in the IoTapplication gateway.The sample packets produced by the XBee-S2 module are converted to IPv6 user datagram packets (UDP) by the application gateway and sent to the server. To control the XBee-S2 module by the server, the command packet encapsulated in UDP packets is converted to ZigBee packets by the IoT application gateway.

2.6.SMS Module

This module comprises of GPRS/GSM modem and control program.The GSM modem is connected to the Raspberry Pi via serial interface to the switching module.The SMS module acts as an interface between the embedded processor and the GSM network. The Embedded gateway stores the GSM dial-up, communication protocol and control program. The system logs into the network and makes data transfer and communication through the SMS module. The GSM/GPRS module used is SIM900A.It operates on various frequencies such as 880MHz, 900MHz, 1800MHz and 1900MHz to send SMS[32].The modem connects to microcontroller’s serial interface via an RS232 level converter circuitry. The TCP/IP stack present in modem enables the microcontroller to connect to the internet via GPRS. To communicate with home owner an SMS activation system is implemented. The module takes the AT command from a remote terminal or mobile devices and sends them to switching module via the GSM network.

 

3. Real Time Implementationthrough Webpages

The embedded web server is developed in the Linux environment supporting SLIP, TCP/IP, HTTP and AT Commands protocol and it is implemented on the Raspberry Pi development board. The dynamically generated files which could include output data from hardware resources is supported by web server’s flash file system and is called an Embedded Server Page (ESP).

3.1 Embedded Server Pages

The embedded server page is created using the following languages: HTML, JavaScript, PHP and SQLite. Users can add embedded equipment data in a page using dynamic HTML. Dynamic HTML gives access to the user to both control and observe in a secure environment. Fig. 6 shows the login page.

Fig. 6.Login page

JavaScript enables user friendly interaction with equipments to HTML. Interaction with database of home security system is handled by page scripts which are embedded in or included with HTML pages. Resizing of the data field of HTML pages and validation of the input field is done with JavaScript. Fig. 7 below shows the monitor and control interface page.

Fig. 7.Monitor and control interface

SQLite is used as embedded database to store local in application software’s such as embedded equipment data records for present and future use of the web browsers. SQLite is extensively used today by several widespread browsers, operating systems and embedded systems. It is arguably the most widely deployed database engine.

3.2. Storage of Data

The sample data to be sent to the Linux based server is encapsulated by the UDP packets produced at the gateway. The UDP packets are received on an arbitrary port and the relevant information is stored in a SQLite database using an application running on the server. There are 4 columns in the database table namely source address, time, source channel and sample data. For each UDP packet received new rows are added to the database. This helps in sorting of data by time, sensor node and sensor channel. Python programming language is used for writing programs for address, packet transformations, data transmissionand packet repetition. PHP and Java script are used for developing the Web interface.

3.3. Display on Web Interface

The UDP packet containing sample data is collected by a Linux based server and it is stored in the database. The samples collectedcan be accessed through a website hosted on the server. The database also stores the raw sample data, the sample source and the time of arrival. This helps in the organization of the samples by date and other parameters. The current data are displayed on the website as shown in the Fig. 8.

Fig. 8.Real-time data display on the web interface

 

4. Experimental Results

The monitoring of the environment for visitor entry is done by the experimental setup shown in Fig. 9. A notification email and/or SMS indicating the update regarding the visitor is sent by the WIU module to the authorized users. The user sends the response for controlling action to WIU as shown in Fig. 10 to Fig. 14 respectively.

Fig. 9.Working Experimental setup, (a) Wireless sensing unit (b) Wireless information unit having GSM modem, Raspberry pi, ZigBee coordinator, monitor, keyboard and mouse.

Fig. 10.Screen shot of "E-mail Received with Snapshot of the Visitors" from WIU

Fig. 11.Screen shot of "INBOX" received on WIU

Fig. 12.Step by step usage of the mobile notification program

The algorithm reads the subject and checks for the existence of safe range of devices wirelessly.The data is then forwarded to WCU for further processing. If all the safety and security norms are satisfied by the system,then WIU forwards controlling commands to WCU wirelessly. The WCU now performs the necessary switching action. In the given experimental setup an email with subject ON1 was sent to the Raspberry Pi account (‘rpi.home3@gmail.com’) from the authorized users account (agrawalrishabh3@gmail.com)

When the algorithm gets the command ON1 at WIU, it commands the WCU to unlock the door1. The WCU unlocks the door and immediately replies the owner by email under the subject –'Notification on Home Security System’ and the body of the email being 'Door l Unlocked'. Fig. 13 shows the screen shot of sent mail from WIU to sender indicating the details of the action performed. Any security breach or unauthorized trespass is notified by the system Fig. 15.

Fig. 13.Screen shot of "CONFORMATION MAIL" from WIU

Fig. 14.Screen shot of "CONFORMATION MAIL AFTER CLOSING THE DOOR” from WIU

Fig. 15.Screen shot of "SECURITY NOTIFICATION" from WIU

The control program runs on the server. The owner can control his doors remotely through the website. The inherent flexibility of the application is due to the fact that it can be controlled remotely using the secure website as shown in Fig. 16.Thus the home security system enables the users to effectively monitor and manage their home environment remotely.

Fig. 16.Page to monitor and control appliances

The interaction of WSN and IPv6 provides a stable and reliable web based home monitoring system which depicts that IoT technology is a reliable and cost effective platform.

 

5. Conclusion

The web based monitoring and automatic control of an interactive home security system with the GSM, ZigBee communication and Web-enabled measurement is adding a new dimension in the field of home automation. This system replaces the traditional PC with a low-cost single chip processor, which makes administrators to get parameters of different remote devices and send control information to field equipments at any time through Internet.

As GSM technology has maximum coverage it can be used as an excellent choice. Similarly a text based protocol, SMS is one of the basic component of GSM system. It can have access to modify the status of the services. The complete system is secured through a login E-mail and Webpage password based authentication. The design is completely wireless and integrated with the software. Such an authentication system is low cost, robust as well as ease of installation. It is based on ZigBee communication.

The GSM, E-mail and Web based controlled duplex communication system provides a powerful decision making device concept for adaptation to several smart home scenarios.