Hand Gesture Recognition Approach:A Survey

IJIRST –International Journal for Innovative Research in Science & Technology| Volume 3 | Issue 10 | March 2017 ISSN (online): 2349-6010

Hand Gesture Recognition Approach: A Survey Mr.V.K.Jadeja C.U.Shah College of Engineering and Technology

Mr. R.H.Davda C.U.Shah College of Engineering and Technology

Mr.C.R.Patel C.U.Shah College of Engineering and Technology

Mr.J.N.Solanki C.U.Shah College of Engineering and Technology

Mr.M.R.Zala C.U.Shah College of Engineering and Technology

Abstract Gesture recognition enables humans to communicate with the machine and interact naturally without any mechanical devices.

Till now human can interact with computer through input devices like mouse and keyboard or technologies like Graphical User Interface, Command Line Interface, touch user interface, motion tracking interface. The use of gestures to convey information is an important part of human communication. Hand gesture recognition is widely used in many applications, such as in computer games, machinery control and thorough mouse replacement. Hand gestures can be classified into two categories: static and dynamic. Hand gesture recognition approach works on the three main phases i.e. object detection, tracking of object, and gesture recognition. In this survey paper different hand gesture recognition methods and issues are explained.

Keywords: Hand Gesture Recognition, Gesture Recognition methods, SVM _______________________________________________________________________________________________________ I.

INTRODUCTION

Non-verbal communication includes various gestures of body parts, mostly face and hand. In human being the oldest method of communication is gesture [3]. In early ages the way of communication for men to share the information of food, source of water, information about their enemy, request for help etc. Though gestures are used widely for different applications. This includes human-robot interaction, sign language recognition, interactive games, vision-based augmented reality etc. Another major application of gestures is found for making the passengers aware about the safety features by the airhostess [3]. For communication by the people at a visible, but not audible distance and for the physically challenged people (mainly the deaf and dumb) gesture is the only method. Another area where hand gesture recognition has been successfully used is human robot interaction. Keyboard and mouse can be used in 2D world, but the control of a robot needs a 3D space. Hand gesture is most suitable for such purposes. Different algorithms and features are used by different researchers for recognition purpose.

Body language is an important way of interaction among humans, adding emphasis to voice messages or even being complete messages by itself [1]. Gesture is a form of non-verbal communication using various body parts, mostly hand and face. Gesture recognition systems could be used for improving human-machine interaction. This type of interfaces would allow human user to control remotely through hand postures a wide variety of devices. Gestures are used widely for different applications like humanrobot interaction, sign language recognition, interactive games, vision-based augmented reality, making passengers aware about the safety features during driving etc. For communication at the people at a visible, but not audible distance, gesture is the only method [2].

Hand gesture can be classified in two types i.e. Static and Dynamic gestures. Static gesture is also called posture which refers only a single image corresponding to a single command. Stop sign is an example of static gesture. Static gesture is simple and need less computational power. Sequence of postures is called as gesture which is called as dynamic gesture [2]. Dynamic gesture is intended to change over a period of time and it is complex. A waving hand means goodbye is an example of dynamic gesture.

The rest of this paper is organized as follows. Section II presents overview of gesture recognition. Section III presents approaches of gesture recognition. Gesture Recognition methods are discussed in section IV. Section V presents various applications of gesture recognition. Section VI present different issues related to hand gesture recognition. Finally, conclusion is presented in section VII. II. SYSTEM OVERVIEW Hand gesture recognition system has four different phases to find out the gesture. They are data acquisition, hand segmentation and pre-processing, feature extraction and finally the recognition [3]. The hand image is captured by suitable input device or take existing video as input. The image is segmented to locate the hand from the (cluttered) background and other parts of the body and thereafter the image is processed to remove noises, to detect edges/ contours, to normalize for generating the simplest and desired model [3]. The features are extracted from the segmented and pre-processed image for recognition. Any gesture

All rights reserved by www.ijirst.org

197

Hand Gesture Recognition Approach: A Survey (IJIRST/ Volume 3 / Issue 10/ 035)

recognition system works according to following steps first acquiring the input image from camera, filtering, segmentation, feature extraction and classification [3].

Following figure 1 shows the basic gesture recognition steps.

Fig. 1: Basic steps of Gesture Recognition [4]

III. GESTURE RECOGNITION APPROACHES Hand Gesture recognition approaches can be classified into various categories.

Data glove based approaches: Data Glove, based approach uses a glove-type device which could detect hand position, movement and finger bending. In this approach user require to wear a glove like device, which uses sensors that can sense the movements of hands and fingers, and pass the information to the computer. These approaches can easily provide exact coordinates of palm and finger’s location and orientation, and hand configurations [15] [16].

Vision based approaches: In this approach user not require to wear anything. Instead the system requires only camera (s), which are used to capture the images of hands for interaction between human and computers. Vision based approach is simple, natural and convenience [16].

Color glove based approaches: Color glove based approaches represent a compromise between data glove based approaches and vision based approaches.

Marked gloves or colored markers are gloves that worn by the human hand [18] with some colors to direct the process of tracking the hand and locating the palm and fingers [18], which provide the ability to extract geometric features necessary to form hand shape [18].

IV. GESTURE RECOGNITION METHODS The task of assigning a feature vector or a set of features to predefined classes in order to recognize any gesture is called as classification. After analysis and modelling of input hand image is done, to recognize the gesture, classification methods can be used. There are number of classification methods that can be used for recognizing gesture from which some of them can be describe below.

Support Vector Machine (SVM) SVM is a non-linear classifier which is reported as producing superior classification results compared to other methods. The idea is to non-linearly map the input data to some high dimensional space, where the data can be linearly separated, thus providing great classification (or regression) performance [8]. One of the bottleneck of SVM is the large number of support vectors used from the training set to perform classification (regression) tasks. To performing linear classification, SVMs can efficiently perform non-linear classification using what is called the kernel trick, implicitly mapping their inputs into high-dimensional feature space.

Dynamic time warping (DTW) It has long been used to find the optimal alignment of the signals. The DTW algorithm calculates the difference between each possible pair of points out of two signals in term of their associated feature values [6] [8]. By using these distances they calculate a cumulative distance matrix and find the least expensive path through this matrix. This path represents the ideal warp- the synchronization of the two signals which causes the feature distance between their synchronized points to be minimized. The signals are normalized and smoothed before the distances between points are calculated [6]. DTW has been used in various

All rights reserved by www.ijirst.org

198

Hand Gesture Recognition Approach: A Survey (IJIRST/ Volume 3 / Issue 10/ 035)

fields, such as speech recognition, data mining, and movement recognition. DTW mainly focused on speeding up the algorithm whose complexity is quadratic in the length of the series.

K-Means The k-means problem is to determine k points called centers so as to minimize the clustering error, defined as the sum of the distances of all data points to their respective cluster centers [6]. This classification finds statistically similar groups in multispectral space. The most commonly used k-means algorithm is Lloyd’s k-means and Mac-Queens k-means algorithm.

Hidden Marko Model (HMM) A HMM is defined as a set of states of which one state is the initial state, a set of output symbols, and a set of state transitions [6] [8]. Each state transition is represented by the state from which the transition starts, the state to which transition moves, the output symbol generated, and the probability that the transition [8]. The state transitions represent the probability that a certain hand position transitions into another: the corresponding output symbol represents a specific posture and sequence of output symbols represent a hand gesture. The HMM with the highest forward probability determines the user’s most likely gesture.

V. APPLICATIONS This section includes applications of vision based hand gesture recognition.

Sign Language Sign language is the natural way of communication of hearing and/or speech impaired people. Various vision based gesture recognition methods have been embedded into sign language interpreters [20]. Usually, a capture device is used to find and track hands and record the shapes and trajectories of hands, which are represented by feature vectors. After being matched to corresponding signs, the feature vectors are compared against a grammar library to determine whether the signs make sense in a grammar context.

Robot control Through the use of gesture recognition, “robot control with the wave of a hand” of various devices is possible. The signal must not only indicate the desired response, but also which device to be controlled. The system consists of a robot unit, a video or infrared camera affixed to the robot unit for capturing hand images, a gesture recognition unit and a gesture databases. It is also possible to use train robots to learn new gestures in an online or interactive manner [19].

Television Control Hand postures and gestures are used for controlling the Television device. In a set of hand gesture or particular count of finger are used to control the TV activities, such as turning the TV on and off, increasing and decreasing the volume, muting the sound, and changing the channel using open and close hand [21].

Games Gesture is used for computer games. Using gesture we can easily interact with computer. In video game using gesture track and control the player’s movement or recognize the position of players. Using gestures control the movement of avatars in a virtual world, and play station.

VI. ISSUES IN GESTURE RECOGNITION This section includes issues generated in gesture recognition method.

Change in illumination When there is a change in the illumination condition, the system fails to recognize properly. Even if there is variation in lighting condition between the training dataset and inputs, some system fails to recognize.

Difficult background Most of the work can be done in uniform background for recognizing hand. But in real-time gesture recognition, uniform background is not desirable or available.

Skin-like-colored objects Sometimes objects with similar color that of human skin may be present in the environment and this leads to confusion of the recognition systems.

All rights reserved by www.ijirst.org

199

Hand Gesture Recognition Approach: A Survey (IJIRST/ Volume 3 / Issue 10/ 035)

Special hardware A number of special hardware, like Range camera, 3D depth sensor, Data gloves have been used.

VII. CONCLUSION Hand gesture recognition finding its application for non-verbal communication between human and computer. With the increase in applications, the gesture recognition system demands lots of research in different directions. Hand gesture can be recognized easily, and actions performed depends on gesture movement are the primary focus of many researchers. In this paper, various approaches of hand gesture recognition can be given. Also different gesture recognition methods can be discussed in this work.

Issues in hand gesture recognition can also discuss.

REFERENCE [1] [2] [3] [4] [5] [6] [7] [8] [9]

[10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21]

Narendra V. Jagtap,Prof. R. K. Somani, Prof. Pankaj Singh Parihar, “Hand Gesture Recognition System for Human- Computer Interaction with Web-Cam”, in IJSRD –International Journal for Scientific Research & Development| Vol. 1, Issue 9, 2013 | ISSN (online): 2321-0613.

Arpita Ray Sarkar, G. Sanyal and S. Majumder, “Hand Gesture Recognition System : A Survey”, International Journal of Computer Applications (0975 – 8887) Volume 71– No.15,May 2013.

Surve Pranjali, Ubale V.S, “Hand gesture recognition system: A Survey” , in International journal of Inventive Engineering and Science (IJIES), ISSN:2319-9598,Volume-3,Issue-3, February 2015.

Dr. K.S.Kinage and Samara Mutha, “Study on Hand Gesture Recognition”, in International Journal of Computer Science and Mobile Computing, 2015.

Surve Pranjali, Prof. Ubale V.S, “Recognition of Hand Gesture using PCA, Image Hu’s Moment and SVM” ,in IPASJ International journal of Electronics and communication (IIJEC), ISSN 2321-5984, Volume 3, Issue 4, April 2015.

Sukhdip Singh, Yogita Bhardwaj, “Hnad Gesture Recognition Techniques: A Review” , in National conference on Innovative Trends in Computer Science Engineering (ITCSE), ISSN:2349-7688, April 2015.

Mokhtar M. Hasan, Noor A. Ibraheem and Rafiqul Z.Khan, “Comparative study of Skin Color based Segmentation Techniques”, International Journal of Applied Information Systems (IJAIS), Volume 5, No. 10, August 2013.

Anupam Agrawal, Siddharth S. Rautaray,” Vision based hand gesture recognition for human computer interaction: a survey”, Springer/Science Business Media Dordrecht, 2012.

Gashree R N , Stafford Michahial , Aishwarya G N ,Beebi Hajira Azeez , Jayalakshmi M R ,and R Krupa Rani, “Hand Gesture Recognition using Support Vector Machine”, in International Journal Of Engineering And Science (IJES), Volume 4, Issue 6, Pages PP.42-46, June – 2015, ISSN (e): 2319 – 1813 ISSN (p): 2319 – 1805.

Sushmita Mitra and Tinku Acharya, “Gesture Recognition: A Survey”, in IEEE Transactions on systems, VOL. 37, NO. 3, MAY 2007.

E. Ohn-Bar and M. M. Trivedi, “Hand gesture recognition in real time for automotive interfaces: A multimodal vision-based approach and evaluations,” IEEE Trans.Intelligent Transportation Systems, vol. 15, no. 6, pp.2368–2377, Dec 2014.

W. Heng, A. Kl¨aser, C. Schmid, and C.-L. Liu, “Dense trajectories and motion boundary descriptors for action recognition,” Intl. Journal of Computer Vision, vol. 103, no. 1, pp. 60-79, May. 2013.

P. Doliotis, A. Stefan, C. McMurrough, D. Eckhard, and V. Athitsos, “Comparing gesture recognition accuracy using Color and depth information,” in ACM Conf. Pervasive Technologies Related to Assistive Environments, 2011.

Jayesh s. sonkusare, Nilkanth B. chopade, Ravindra sor and Sunil L. tade, “A review on Hand Gesture Recognition System”, in International Conference on Computing Communication Control and Automation, 2015.

Harshith.C, Karthik.R.Shastry, Manoj Ravindran,M.V.V.N.S Srikanth, Navee Lakshmikhanth “Survey on various Gesture Recognition Techniques for Interfacing Machines based on Ambient Intelligence” (IJCSES), Vol.1, No.2, November 2010.

PragatiGarg, Naveen Aggarwal and SanjeevSofat, 2009.Vision Based Hand Gesture Recognition, World Academy of Science, Engineering and Technology 49, pp. 972-977 Laura Dipietro, Angelo M. Sabatini, and Paolo Dario,2008. Survey of Glove-Based Systems and their applications, IEEE Transactions on systems, Man and Cybernetics, Part C:Applications and reviews, vol. 38(4), pp. 461-482, doi: 10.1109/TSMCC.2008.923862 Mokhtar M. Hasan, and Pramod K. Mishra, 2012. Hand Gesture Modeling and Recognition using Geometric Features: A Review, Canadian Journal on Image Processing and Computer Vision Vol. 3, No.1.

C. Lee and Y. S. Xu, “Online, Interactive Learning of Gestures For Human/Robot Interfaces,” In Proceedings of the IEEE International Conference on Robotics and Automation, vol. 4, pp. 2982-2987, 1996.

H. Birk and T. B. Moeslund, “Recognizing Gestures From the HandAlphabet Using Principal Component Analysis,” Master Thesis, Laboratory of Image Analysis, Aalborg University, Denmark, 1996.

Shiguo Lian Wei Hu,kai Wang “Automatic User State Recognition for Hand Gesture Based Low-Cost Television Control System” IEEE Transaction paper, 2014.

All rights reserved by www.ijirst.org

200