Activity Recognition From IR Images Using Fuzzy Clustering Techniques

INTERNATIONAL JOURNAL FOR TRENDS IN ENGINEERING & TECHNOLOGY VOLUME 5 ISSUE 1 – MAY 2015 - ISSN: 2349 - 9303

Activity Recognition From IR Images Using Fuzzy Clustering Techniques Savitha Suman

D. Pamela

Karunya University, Department of EIE, sonia446.savitha@gmail.com

Karunya University, Assistant Professor, Department of EIE pamela@karunya.edu

Abstract— Infrared sensors ensures that activity recognition is possible in the day and night times. It is used especially for activity monitoring of older adults as falls are more prevalent at night than the day. This paper focus on an application of fuzzy set techniques and it is capable of accurately detecting several different activity states related to fall detection and fall risk assessment and it also includes sitting, standing and being on the floor to ensure that elderly residents gets the help they need quickly in case of emergencies. Fall detection and fall risk assessment is used for an aging in place facility for the elderly people. It describes the silhouette extraction process, the image features , and the fuzzy clustering technique.

Index Terms— Activity labeling, Fuzzy clustering, Image moments, Infrared camera .

——————————  ——————————

1 INTRODUCTION Activity recognition is done on vision sensors under normal illumination and low lightning conditions that indicate the severe fall risk of older adults .Since nocturnal activities are an important aspect of an independent lifestyle it will create a potential problem. This shows the need for surveillance techniques that can be implemented in the absence of light or under negligible lighting conditions. Fall detection and fall risk assessment has given much more importance and dynamic infrared sensors are also involved.

Fuzzy clustering techniques are mostly unsupervised methods that can be used to organize data into groups based on similarities among the different data items . All the clustering algorithms do not rely on assumptions common to conventional mean, median, average, standard deviation methods etc., and it undergoes statistical sharing of data, and therefore they are useful in situations where little prior understanding exists. The potential of clustering algorithms to reveal the underlying structures in data can be exploited in a wide avariety of applications, including classification of an activities, processing of an image, recognizing of a pattern, modeling and identification.

continuously monitor elderly persons as they perform their day-today activities, maintaining their privacy by using silhouettes instead of raw images for further analysis. It has been shown previously that silhouettes addresses the privacy concerns of elderly persons participating ,and increases their willingness to accept video monitoring systems in their households [1]. From these silhouettes, image moments are extracted, which are then clustered to produce fuzzy labels in the basic activity categories.

Clustering itself can be concluded as a fuzzy concept[2].

Depending on the implemented clustering algorithm the criterion function to be optimized changes, and the nature and shape of the clusters vary.

While clustering was employed in some of the above mentioned techniques and silhouettes were extracted in others, combination to segment activities are used nowhere. By using fuzzy clustering techniques in identifying sit-to-stand frames using image moments on visible light data has inspired the work.

Images (from any source)

2 OVERVIEW In this system, background subtraction techniques using mixture of Gaussian models with texture features are used on the raw image data to separate the foreground from the background and the resulting silhouette are taken as inputs to the automatic activity segmentation ————————————————

Pre-processing & Silhouette Extraction

Activity State Identification

 Author name is currently pursuing masters degree program in electric power engineering in University, Country, PH-01123456789. E-mail: author_name@mail.com  Co-Author name is currently pursuing masters degree program in electric power engineering in University, Country, PH-01123456789. E-mail: author_name@mail.com (This information is optional; change it according to your need.)

Extraction Of Image Moments

Fuzzy Clustering Of Image Moments

Fig .1 Block diagram of an algorithm

system. It is to build an automated video surveillance system to

The paper is organized as follows. Silhouette extraction and moment description is used for clustering present in section 3.

Section 4 describes the fuzzy clustering techniques used for activity

6

INTERNATIONAL JOURNAL FOR TRENDS IN ENGINEERING & TECHNOLOGY VOLUME 5 ISSUE 1 – MAY 2015 - ISSN: 2349 - 9303 analysis and Section 5 briefly describes the experimental setup and results using the standard web cameras under normal illumination.

The conclusions and future work present in Section V6.

element is indicated between 0 and 1 and it is the membership range. In order to reduce the number of samples and thus to minimize the amount of input signal, used for fuzzy clustering.

Algorithm:

3 METHODOLOGY 3.1 Silhouette Extraction Silhouette extraction is a background change detection technique whose accuracy depends on how well the background is modeled.

The background subtraction method implemented using color and texture features employs shadow removal for greater accuracy binary morphological operations are used to fill up holes and remove noise from the extracted silhouettes [3]. After obtaining the silhouettes from the image steps, the next step in the algorithm is extracting image moments as shown in the block diagram. Image moments are applicable in a wide range of applications such as pattern recognition and image encoding.

One of the most important and popular set of moments is the set of Hu moments [4]. This Hu moments consists of seven central moments taken around the weighted image center. In particular, the first three Hu moments are more robust than the other Hu moments in the presence of noise and were used in this analysis. scale and rotation invariant in Hu moments, makes them extremely robust and applicable in different scenarios [5]. However, they are nonorthogonal in nature; i.e., their basis functions are correlated, making the information captured redundant. In contrast, the Zernike orthogonal moments comprise image moments with higher performance in terms of noise resilience, information redundancy and capability of reconstruction.

1.

3.2 Image Moments For Activity Classification The image moments using here for the classification of an activities is Zernike moments [6]. These moments are briefly described below:

2.

3.

4.

Zernike Moments: Zernike moments are the mappings of an image onto a set of complex Zernike polynomials. Since Zernike polynomials are orthogonal to each other, these moments can represent the properties of an image with no redundancy or overlap of information between the moments. These moments are significantly dependent on the scaling and translation of the object in an Region Of Interest.

Fix c = number of clusters &initialize the iteration counter t = 1.

For all the data points and for each of the clusters we have to initialize the membership matrix U. (The initialization is explained further in this section.)

Do. Compute the cluster centers using

5.

Compute the covariance matrices for each of the clusters as in

6.

Update the partition matrix

7.

Increment the iteration counter t.

8.

Until || Îź (t) â&#x2C6;&#x2019;Îź (t â&#x2C6;&#x2019; 1) || < Đ&#x201E; or t >

where Đ&#x201E; is the

maximum permissible error and number of iterations specified.

is the maximum

Here, Îź(t) is the vector of all centers, and the distance norm employed to determine convergence is the standard Euclidean distance. An important point to be noted is that it is essential to initialize the membership values to random values but with the means equal to 1/c (where c is the number of clusters) and standard deviation equal to 1 so that the algorithm converges in a faster rate.

For ensuring equal importance to each of the moments Standard essential is used. Otherwise, the algorithm would focus on the moments with the highest range. It is worth noting that since we constrain the determinant of the co-variance matrix to be 1, we impose restrictions on the size of the clusters, and as a consequence, the identified ellipsoidal clusters have to be of similar size [10].

4 FUZZY CLUSTERING Fuzzy clustering techniques are used of partition data on the basis of their closeness or similarity using fuzzy methods. As opposed to the hard clustering, each element can belong to a certain cluster with varying degrees of membership.

The Gustafson-Kessel [7] fuzzy clustering technique was implemented on the Zernike image moments . With applications in several ďŹ elds such as image processing, recognizing the patterns, identiďŹ cation of systems, and classiďŹ caion it has become very popular clustering algorithm [8]. The Gustafson-Kessel clustering technique is that it is well suited for the ellipsoidal clusters produced by the moments . This clustering technique is an extension of the fuzzy c-means algorithm in which each cluster has its own unique co-variance matrix which makes it robust and it is more applicable for various data sets which contains ellipsoidal clusters of different orientations and sizes [9]. As the basic approach for clustering is well known it is summarized for completeness.

A membership function indicates the membership degree of a particular element regarding as an event. The level of influence of an

Parameters of the Gustafsonâ&#x20AC;&#x201C;Kessel Algorithm: As for the FCM algorithm (except for the norm including matrix A, which is automatically adapted) same parameters must be specified to the number of clusters c, termination tolerance and fuzzy exponent m. Cluster volume đ?&#x203A;&#x2018; is the additional parameter. Without any basic information, for each cluster đ?&#x203A;&#x2018; is simply fixed at 1. The main disadvantage of this setting is that due to the constraint Gustafson Kessel algorithms, clusters of equal volumes can be found.

5 EXPERIMENTAL SETUP AND RESULTS â&#x20AC;&#x201C; WEB CAMERAS WITH INFRARED LIGHT 5.1 Web Cameras Using Infrared Illumination:

7

INTERNATIONAL JOURNAL FOR TRENDS IN ENGINEERING & TECHNOLOGY VOLUME 5 ISSUE 1 – MAY 2015 - ISSN: 2349 - 9303 An image sequence database was at a resolution of 640 × 480 with IR camera . We can recognize activities even with the degradation in silhouette quality. In the visible spectrum, these images are completely black. IR lighting was used with a wavelength of 850nm; in total, there were 216 individual IR LED's distributed between the two lamps with a total power draw of approximately 20w. There is no IR filter on the camera lens.

The possible activities practiced at night and included them in our data collection: walking, standing without hand motions, a hand motion, sitting down and standing up, sitting on a sofa, going to bed, and getting up from bed. The data collection also contained the following situations on the bed: sleeping (lying on the bed), being sleepless (flipping with some movements, i.e., with a little toss and turns), sitting on the bed, and transitioning from sitting to lying on the bed. In addition, four abnormal activities (falling) were included: walking in the room and falling to the ground due to loss of balance, slipping when trying to get up from a chair, falling when trying to get up from a bed, and falling out of the bed when sleeping. The frame rate is 3 frames/s.

on the floor activity label.

All the analyzed pixels can be given to the Fuzzy Clustering algorithm . In this the images will be processed in four steps:

5.2 Experimental Results: Preliminary experiments were conducted to establish the input parameters and best features are used for these data. Several participants performed from a variety of activities. Silhouettes are extracted from the raw image sequences, and the moment features were computed. The GK clustering technique requires the number of clusters to be specified as an input parameter. By demonstrating the clustering of the Zernike moments using the GK algorithm with the number of clusters initialized to the number of activities will be yielded [6]. Since single camera images are used here, the activities of walking and standing cannot be differentiated in general; thus, they are grouped together as ―upright‖ frames for the purpose of activity recognition.

The clustering results of one data sequence using an input of three clusters. The clustering results with the X-axis that indicates the frame number in the sequence and the Y -axis that indicates the cluster number after hardening the membership matrix. These results have been color coded for display purposes and black-colored points indicate the areas where the points are cowardly clustered and it is evident that the two clusters obtained represent the ―sit‖ and ―on the floor‖ kind of activities, without any prior information, we couldn't identify which cluster indicates which activity. This scenario had involved a participant who performs several actions in an unlit room.

The three activities performed in the scenario below were night time activities of moving around in the room (upright), sleeping on the bed, and then falling onto the floor. After fuzzy clustering and partition hardening activities are well separated . However, in this scenario, the activity ―fall‖ is equivalent to ―on the floor‖ since no other parameter has been taken into consideration which could differentiate between the two activities. The detection of the transition frames as well as identifying the activity state using prototype matching. Using prototype matching, the activity states were identified and the sample results are taken. In the raw images, the person has been circled in yellow to distinguish the person from the background for visualization. When compared with the standard illuminated data silhouettes are fairly noisy and their shapes are quite different. However, the results show strong clustering of the image moments obtained from the same activity states. This makes activity analysis possible, even in the dark using fuzzy clustering. Some of the color-labeled image frames with blue indicating an upright frame, red signifying on the bed activity label, and pink representing

(iii) Edge Detection: It is used to detect a wide ranges of edges in images.

(1) Pre-processing (2) Silhouette extraction (3) Activity classification (4) Fuzzy clustering (1)Pre-processing: Pre-processing is used to find the foreground detection and it is done in three process. (i) Gray image: Gray image is also known as infrared images. (ii) Binary image: It is a digital image that has only two possible values for each pixel. Typically two colors used for a binary images are black and white though any kind of colors were used. The color used for the object in the image is the foreground color while the rest of the image is the background color.

Fig.2 Gray Image

Fig.3 Binary Image

Fig.4 Edge Detection

(2) Silhouette Extraction: Silhouette extraction is used for detecting the subtraction of the background. The dark shape and outline of someone will be visible against in lighter background and especially in dim light. (i) Image Dilation: It is developed by binary images. Binary image is to gradually enlarge the boundaries of regions of foreground pixels. (ii) Image Filling: It is used for closing the regions while keeping

8

INTERNATIONAL JOURNAL FOR TRENDS IN ENGINEERING & TECHNOLOGY VOLUME 5 ISSUE 1 – MAY 2015 - ISSN: 2349 - 9303 initial region sizes.

 Fig.5 Image Dilation

 Fig.6 Image Filling

(3) Activity Classification: Activity Classification is done by extracting the image moments by Hu moments . Hu moments is used for activity classification.

Hu Moments: Feature moments and normalized central moments can be applied on an image to calculate the seven invariant moments defined by Hu moments in terms of centralized moments for purpose of shape recognition.

The function to be used directly by the user ―A‖ is a 2D matrix representing an image. Inside this function another function cent moments (p, q, A) is called normalized moments. Based on normalized central moments, Hu moments will give seven moment invariants: • • • • • • •





M1=n20+n02 M2=(n20-n02)^2+4*n11^2 M3=(n30-3*n12)^2+(3*n21-n03)^2 M4=(n30+n12)^2+(n21+n03)^2 M5=(n30-3*n21)*(n30+n12)*[(n30+n12)^2 3*(n21+n03)^2]+(3*n21-n03)*(n21+n03)*[3*(n30 +n12)^2-(n21+n03)^2] M6=(n20-n02)*[(n30+n12)^2-(n21+n03)^2] +4 *n11 *(n30+n12)*(n21+n03) M7=(3*n21-n03)*(n30+n12)*[(n30+n12)^2-3*(n21 +n03)^2]-(n30+3*n12)*(n21+n03)*[3*(n30+n12)^2(n21+n03)^2]



Output of Hu moments of different activities:





First image of Hu moments: M= 1.8648 2.8531 0.0248 0.0289 0.0008 0.0487 -0.0000 Second image of Hu moments: M= 0.4353



9

0.1240 0.0065 0.0030 0.0000 0.0000 0.0011 Third image Hu moments: M= 0.1946 0.0041 0.0002 0.0001 -0.0000 0.0000 0.0000 Fourth image Hu moments: M= 0.3562 0.0824 0.0053 0.0009 0.0000 0.0002 0.0000 Fifth image Hu moments: M= 0.4206 0.1203 0.0108 0.0014 0.0000 0.0004 0.0000 Sixth image Hu moments: M= 0.3802 0.1113 0.0027 0.0006 0.0000 0.0001 0.0000 Seventh image Hu moments: M= 0.3600 0.0993 0.0016 0.0007 0.0000 0.0002 0.0000 Eighth image Hu moments: M= 0.3829 0.0804 0.0091 0.0028 0.0000 -0.0006 0.0000

INTERNATIONAL JOURNAL FOR TRENDS IN ENGINEERING & TECHNOLOGY VOLUME 5 ISSUE 1 – MAY 2015 - ISSN: 2349 - 9303 

Figure 8 shows the clustering results with the X axis indicating the frame number in the sequence and the Y axis indicating the cluster number after hardening the membership matrix. The results have been color coded for display purposes. In both the figures, the points colored red represent frames of a person sitting on the couch and the blue colored points represent the image frames indicating the person on the floor. Black colored points indicate the areas where the points are densely clustered.

Ninth image Hu moments: M= 0.2601 0.0324 0.0020 0.0002 0.0000 0.0000 0.0000  Tenth image Hu moments: M= 0.2424 0.0241 0.0013 0.0001 0.0000 0.0000 -0.0000  Eleventh image Hu moments: M= 0.2986 0.0456 0.0021 0.0012 0.0000 0.0002 0.0000

Fig.8 GK on Zernike Moments and clustering results into membership results by frame number.

6 CONCLUSION AND FUTURE WORK Fuzzy clustering methods have been employed in detecting activity frames in different environments, both controlled as well as uncontrolled . A classifier was constructed and the Zernike Moments were obtained by using clustering results and membership results by frame number.

In the future work , the simulation and implementation of detecting the different activities i.e., sitting, standing, on the floor etc., is displayed on the LCD in processors.

(4) Fuzzy Clustering: Fuzzy cluster algorithm is used to identify and to classify the Zernike moments. The fuzzy sampling algorithm extracts the most relevant aspects of the Zernike moments , without loss of important information. In this work, the clustering processes is used to indicate the value and location of each cluster, without requiring the generation of functions or rules corresponding to each cluster. It is a great advantage since it requires less computational effort. It applies the fuzzy clustering algorithm to the GK on Zernike moments, that are clustered to 4 clusters.

REFERENCES

Fig.7 GK on Zernike Moments and clustering results into 2 clusters.

[1] J. Keller and I. Sledge (2007), ―A cluster by any other name,‖ in Proc. IEEE Proc. Fuzzy Inf. Process,427–432.

[2] G. Demiris, D. Parker, J. Giger, M. Skubic, and M. Rantz (2009), ―Older adults’ privacy considerations for vision based recognition methods of eldercare applications,‖ Technol. Health Care, 17(1), 41–48.

[3] A.K. Jain (1989), Fundamentals of Digital Image Processing.

Englewood Cliffs, NJ, USA: Prentice-Hall.

[4] A.El Maadi and X. Maldague (2007), ―Outdoor infrared video surveillance: A novel 3dynamic technique for the subtraction of a changing background of IR images,‖ Elsevier,Infrared Physics Technology 49, 261–265.

[5] M. K. Hu (1962), ―Visual pattern recognition by moment invariants,‖ IRE trans.Inf.Theory, 8,179-187.

[6] T. Banerjee, J. M. Keller, M. Skubic, and C. C.Abbott(2010), ―Sitto-stand detection using fuzzy clustering techniques,‖ in Proc. IEEE transaction on Fuzzy Systems / Computer Intel, 13(4), 1–8.

[7] D. E. Gustafson and W.C. Kessel (1979) , ―Fuzzy clustering with a fuzzy covariance matrix,‖ in proc. IEEE Annu. Conf. Decision Control, San Diego, CA, USA, 761-766.

[8] M. J. Lesot and R. Kruse (2006), ―Gustafson-Kessel-like clustering algorithm based on typicality degrees,‖ presented at the Int. Conf.

Inf. Process. Manag. Uncertainity, Paris, France.

The GK clustering technique requires the number of clusters to be specified as an input parameter. Using the GK algorithm it is shown that clustering the Zernike moments with the number of clusters initialized to the number of activities outputs gives best results. Since only images of one camera are used here, the activities of walking and standing cannot be differentiated in general so, they are grouped together as ―upright‖ frames for the purpose of activity recognition.

It shows the clustering results of one data sequence using an input of 2 clusters.

10

INTERNATIONAL JOURNAL FOR TRENDS IN ENGINEERING & TECHNOLOGY VOLUME 5 ISSUE 1 – MAY 2015 - ISSN: 2349 - 9303 [9] R. Babuska, P.J. van der Veen, and U.Kaymak (2002), ―Improved covariance estimation for gustafson- Kessel clustering,‖ in Proc.

IEEE Int. Conf.Fuzzy Syst.,1081-1085.

[10] S. Theodoridis and K. Koutroumbas (1999), Pattern Recognition.

San Diego, CA, USA.

11