22_ by ides editor

Short Paper Proc. of Int. Conf. on Advances in Computer Engineering 2012

Automatic Fatigue Detection Based On Eye States Saeid Fazli, Parisa Esfehani Research Institute of Physiology and Biotechnology, University of Zanjan, Zanjan, Iran Email: fazli@znu.ac.ir Electrical Engineering Department, University of Zanjan , Zanjan, Iran Email: p_esfehani@yahoo.com Abstract— In this paper, we propose a method for automatics fatigue detection based on eye-tracking. In the present paper we use CCD camera for capturing image from drivers ‘face. The proposed method consists of four steps. In the first step we detect face area using YCbCr color-space, and then crop the image according to calculated maximum and minimum values of vertical and horizontal. based on the obtained results, generally eyes located in the two-fifth to three-fifth of the upper region of face, So we can crop image in this area, and estimate eyes region. In the third step determine exact location of eyes and track them whit use changing pixels of white to black. In the last step counter the number of white and black pixel. In the initial frame we posit driver is awaked, and consideration these number as threshold, so in another frame compare number of white and black pixel whit threshold and determine is driver sleepy or not..

experimental results in section 6. Conclusion is discussed in last section. II.

In the past two decades several studies have been performed in this field. There are various techniques for doing this work; some of them use physiological information, another one check vehicle behavioral and last one investigate driver behavioral. Among these methods the last one is better than the others. When driver fatigue occurs, visual behaviors can be easily observed from changes in their facial features especially from their eyes. It is indicated that the change regularity of eye states have high relativity with the driver’s mental states [5]. Zheng pei calculated the ratio of eye closing during a period of time. The ratio can reflect driver’s vigilance level [6]. Wenhui Dong proposed a method to detect the distance of eyelid, then judged the driver’s status by this kind of information [5]. Nikolaos P used front view and side view images to precisely locate eyes [7]. Edge detection and gray-level projection methods were also applied for the eyes location by Wen-Bing Horng [8]. Zutao Zhang located the face by using Haar algorithm and proposed an eye tracking method based on Unscented Kalman Filter [9]. Abdelfattah Fawky presented a combination of algorithms, namely wavelets transform, edge detection and YCrCb transform in the eye detection [10]. Qiang Ji depended on IR illumination to locate eyes [11]. In [12], the authors used the symmetric property of faces to detect facial area on an image. Then, they used pixel difference to find the edges on the facial region to locate the vertical position of the eyes. Since the edge detection they used cannot clearly mark edges, it is not easy to locate accurate ocular locations. The authors used thresholding to improve the location of the ocular places. A fier finding the approximate eyes positions, a concentric circle template was designed to locate the exact eyes locations, and the template was used to track eyes in the following images. Face symmetry is an obvious feature for an upright face. However, it usually fails to locate the correct face [13].

Index Terms— Introduction, previous worked , overal flowchart, face detection, eyes detection and localization, fatigue detection, experimental results, conclusion, references

I. INTRODUCTION In recent years issue of vehicle safety has special significance and recently many investigations have been conducted on it. Intelligent Driver Monitoring Systems is one this items that has been considered in vehicle safety. As this systems with intelligent recognize accidental conditions, try to help and warning to drivers. One of the most important equipment for smart car is Driver Fatigue detection system that has great importance in restrain road and fatal accidents. The National Highway Traffic Safety Administration (NHTSA) of USA estimates that there are annually about 100,000 crashes in USA that are caused by fatigue and result in more than 1500 fatalities and 71,000 injuries [1]. Some studies have demonstrated that the driver drowsiness accounts for 16% of all crashes and over 20% of the crashes in the highways [2]. Thus, the driver fatigue assessment remains to be a big challenge to meet the demands of future intelligent transportation systems [3]. Developing a system that actively monitors the driver’s fatigue level in real time (and produces alarm signals when necessary), is important for the prevention of accidents, and this is the main motivation of our paper [4]. The reminder of this paper is organized as follows: Section 2 provides a report survey of related researches on driver fatigue detection. In section 3 described a system for face detection. Explain proposed system for eye detection in section 4. Section 5 described a new method of Fatigue Detection by measuring the number of black pixels in eye region. Shows © 2012 ACEEE DOI: 02.ACE.2012.03. 22

PREVIOUS WORKED

III. OVERAL

FLOWCHART

The purposed algorithm is based on changes of eyes state. It considers 6 steps that show in Fig.1. At first we received a video film of drivers’ face so we should convert it to consecutive frames of image, and these images are input for our worked. 77

Short Paper Proc. of Int. Conf. on Advances in Computer Engineering 2012 input image to detect Pixels that appear to be skin. Working in this color Space Chai and Ngan have found that the range of Cb And Cr most representatives for the skin–color Reference map are [13]: 80 Cb 120 and 133 Cr 173 oSo, we check Cb and Cr for each pixel and if it being in above limitation, amount of this change to1 else change to 0. After that we have a black and white image that face detect in it, now should found left and right boundaries of it. In each column of image added amount of whole pixels whit (2) [15]: (2) That PV is vertical curve and F(x,y) is input image and it size is M*N(M represented to row and N represented to column).here we have two sudden changes that they are exactly left and right boundaries of face. For horizontal boundaries we can use the same method whit the difference that here we should calculated amount of all pixels in each row whit (3) and last found max and min difference of these: (3) The result of face detection and boundaries shows in Fig.2.

Figure1. Flowchart of our driver fatigue detection system

Figure2: a (input image), b(black and whit face detection), c(face detection)

IV. FACE DETECTION

V. EYES DETECTION AND LOCALIZATION

There are different techniques for face detection, such as: Face detection use various color-spaces like: HIS space, YCbCr space and face detection use some basic future of face like distance between two eyes, distance between eyes and mouth. As regards in this work we have a frontal image from drivers’ face so using color-spaces is more suitable and according to experimental results YCbCr color-space has the better results for face detection. The input image is in the RGB color-space, so at first we should change it to YCbCr and then determine each pixel is the skin pixel or not. The formulas used to convert per pixel from RGB to YCbCr color space is shown in (1). In this space Y didn’t change between various skin color. Chai and Ngan [14] have developed an algorithm that exploits the spatial

According to the researches on faces’ image,if divided face to five equal parts, eyes located in two-fifth and three-fifth areas. So for estimated the border of eyes area we can cut out the picture from one-fifth line until three-fifth line. Fig.3 show results for this step:

(1)

Figure3: a (open eye detecting), b (open eye detecting black and white), c (closed eye detecting), d (closed eye detecting black and white)

Short Paper Proc. of Int. Conf. on Advances in Computer Engineering 2012 4. 3. Pixels related on iris and eyelash(black pixels) Whereas driver sleeps her/his eyes are closed completely, So the number of whit pixels (related on whit area of eye) Decreased and tended to zero, also the number of black pixel decreased but not zero because of eyelash existed. So counter the number of white pixels and compare them whit threshold has more accuracy and verity. For doing this work, in first step converted image from RGB to YCbCr and eliminated this area. In second step change this RGB image to gray level and in this image countered number of white and black pixels. Whereas, initial impose is that driver doesn’t sleepy, so in the first image eyes are open and the number of countered pixels get for open eyes and we can calculated threshold for closed eyes. If the number of white pixels in next frames is less than threshold the eyes are closed. Now if found eyes closet for 5 consecutive frames, so deriver recognized fatigue and alarm system is on.

Now, for obtaining exact position of eyes and track them in each frame, use from sudden changing of pixels. For more speed and accuracy we divided image in two and limited our searching in one part. In this selected area just eyebrows and eyes exist, so for extracting eyebrows from image and find uo and down points of eyes, if moving in order one column the first changing pixel from whit to black is the upper bound of eyebrow and second changing shows the upper bound of eye. For lower bound, first changing pixel from black to white is lower bound of eyebrow and the second one is lower bound of eye. Result changing is show in Fig.4. in this picture horizontal axis is coordinate of pixels in one column and vertical axis show amount of each pixel in this column.

VII. EXPERIMENTAL RESULTS The experimental result for each step is shown in tabal1. We were written all codes in MATLAB. Whit one camera fixed on in front of car, we afforded our database. It is consisted of some videos that take in natural driving conditions from six people whit the same position in car and then converted these videos to 100 images for each people. The proposed method has been tested on 30 images of per driver. The whole input image format is 720*1280 and they are in RGB color-space.

Figur4: diagram of pixel changing

These two points show coordinate of upper and lower bound of eyes in selective region. for indicating eyes’ position exactly, reduce amount of upper coordinate from five(x1=x15) and add amount of lower pixel whit five(x2=x2+5). The results of tracking eye show in Fig 5.

TABLE I: THE EXPERIMENTAL RESULTS

VIII. CONCLUSION

In the present research we have new method for fatigue detection based on the number of whit pixels in the eye area. At first recognized face region whit using YCbCr color-space. After that cut face area from initial image and estimated eyes area. Next step is found exact localized of eyes basic of changing amount of pixels from whit to black and against. At last in this region countered whit pixels and check them whit threshold and detection fatigue. If driver was drowsy alarm system is on.

Figur5: a (open eye estimation), b (exact location of open eye), c (closed eye estimation), d (exact location of closed eye)

VI. FATIGUE DETECTION 1. The known area consisted of three parts: 2. 1. Pixels depended on face skin and eyelids 3. 2. Pixels depended on white region inside of eye © 2012 ACEEE DOI: 02.ACE.2012.03. 22

Short Paper Proc. of Int. Conf. on Advances in Computer Engineering 2012 [10]. Abdelfattah Fawky, Sherif Khalil, and Maha Elsabrouty, “Eye Detection to Assist Drowsy Drivers,” IEEE. pp. 131-134, 2007. [11]. Qiang Ji, Zhiwei Zhu, and Peilin Lan, “Real-Time Nonintrusive Monitoring and Prediction of Driver Fatigue,” IEEE Transaction on vehicular technologyvol. 53, no. 4, pp. 657662,2004. [12]. M. Eriksson and N. Papankolopoulos, “Eye-Tracking for Detection of Driver Fatigue,” Proceedings of the International Conference onIntelligent Transportation Systems, Boston, MA, pp. 3 14-3 19, November 1997. [13]. Jorge Alberto, Marcial Basilio, Gualberto AGUILAR Torres, Gabriel Sanchez Perez, L. Karina Toscano Medina, Hector M. Perez Meana “Explicit Image Detection using YCbCr Space Color Model as Skin Detection “. Applications of Mathematics and Computer Engineering. [14]. D. Chai, and K.N. Ngan, “Face segmentation using skin-color map in videophone applications”. IEEE Trans. on Circuits and Systems for Video Technology, 9(4): 551-564, June 1999. [15]. Jie Tang, Zuhua Fang, Shifeng Hu, Ying Sun “Driver Fatigue Detection Algorithm Based on Eye features” 2010 Seventh International Conference on Fuzzy Systems and Knowledge Discover (FSKD2010). [16]. Yong Du, Peijun Ma, Xiaohong Su, Yingjun Zhang “ Driver Fatigue Detection based on Eye state analysis” Proceedings of the 11th Joint Conference on Information Sciences (2008). [17]. Mandalapu Sarada Devi, Dr Preeti R Bajaj “Driver Fatigue Detection Based on Eye Tracking “ , First International Conference on Emerging Trends in Engineering and Technology

REFERENCES

[1]. Q. Ji, Z. Zhu, P. Lan, “Real-time nonintrusive monitoring and prediction of driver fatigue,” IEEE Transactions on Vehicular Technology 53 (2004) 1052–1068 [2]. B. Fasel, J. Luettin, “Automatic facial expression analysis,” a Survey, Pattern Recognition 36 (2003) 259–275 [3]. H. Cai, Y. Lin, A “roadside ITS data bus prototype for future intelligent highway, “ IEEE Transactions on Intelligent Transportation Systems 9 (2008)344-348. [4]. Guosheng Yang, Yingzi Lin, Prabir Bhattacharya “A driver fatigue recognition model based on information fusion and dynamic Bayesian network, “ Information Sciences 180 (2010) 1942–1954. [5]. Wen-Hui Dong, Xiao-Juan Wu, “Driver Fatigue Detection Based on the Distance of Eyelid,” Proc. IEEE Int. Workshop VLSI Design & Video,Tech., pp. 28-30 , 2005. [6]. Zheng Pei, Song Zhenghe, and Zhou Yiming, “Perclos-based recognition algorithms of motor driver fatigue,” Journal of China Agricultural University, pp. 104-109, 2004. [7]. Nikolaos P, “Vision-based Detection of Driver Fatigue,” Proc. IEEE Internetional Conference on Intelligent Transportation, 2000. [8]. Wen-Bing Horng, Chih-Yuan Chen, Yi Chang, et al, “Driver Fatigue Detection Based on Eye Tracking and Dynamic Template Matching “ Proc. of the 2004 IEEE International Conference on Networking, Sensing & Control, pp. 7-12, 2004. [9]. Zutao Zhang, Jiashu Zhang, “A New Real-Time Eye Tracking for Driver Fatigue Detection,” Proc.2006 6th International Conference on ITS Telecommunications, pp8-11,2006.