12_ by ides editor

Full Paper Proc. of Int. Conf. on Advances in Computer Engineering 2012

Daily Life Object Management System in Intelligent Space Weerachai Skulkittiyut1, Katsuhiro Mayama2, and Makoto Mizukawa1 1

Functional Control Systems Engineering, 2 Computer Engineering and Computer Science Graduate School of Engineering, Shibaura Institute of Technology, Tokyo, Japan Email: m710502@shibaura-it.ac.jp to observe the environment situation. Initially, the space was straightforward and uncomplicated. Two sets of cameras (vision sensor) and a computer equipped with 3D tracking software was used to recognize the human tracking and positioning. A Distributed Intelligent Network Devices (DINDs), an intelligent device, have sensing, processing and networking functions. The information acquired by each DIND is shared among DINDs through the network communication system. Finally, intelligent space able to perceive an environment situation based on the accumulated information.

Abstract—In this research, we propose a daily life object management system for user in intelligent space environment. We proposed two convenience applications which are searching an object and automatic object restoring to the storing place. The system functionalities include identification of the object, monitoring and tracking the object’s position, real-time recording of the object information and building historical database. We also provide an interface for retrieving and analyzing historical database. In additions, we introduce human posture and position tracking system to allow a system perceives human posture and position information. To improve the accuracy of positioning , we also offer Robotic Technology Case (RT-Case) which provides object tracking capability in living space based on RFID technology, and Robotic Technology Shelf (RT-Shelf) which represents perceivable storage space based on ultrasonic technology. Index Terms—daily life object management system, robotic technology (RT) equipment, intelligent space.

I. INTRODUCTION A. The Intelligent Space Robot Technology (RT) use is widely expanding from automation in the industries to other areas such as rescue robot, surgery robot, entertainment robot and etc. Intelligent Space or Smart home is one of the technological trends of RT application that invoke our daily life. The Intelligent Space consists of service robots, environment, objects and residents. Robots aid and support the residents with providing service.

Figure 2.DIND Concept [1]

K. Ukai et al proposed the RT-Ontology including RT-Service to bring the object and tsuide-task together [2]. However, the user needs to specify the destination manually. B. Object Managemt System in Intelligent Space In a real world like living room, there are a large number of prioritized objects that we often use in everyday life [3]. This information is valuable both for human and robot. To management (including perceive, handle, identify) an object for the individual is easy and uncomplicated, however, for the intelligent space it is still being an issues. In 2007, Mori T. et al proposed the indoor object location estimation method in active Radio-Frequency Identification (RFID) sensor embedded environment [4], which provides the ability to access and manage objects in the environment. This research mainly focuses on improving localization object based on RFID Received Signal Strength Indication (RSSI). Fukui R. et al proposed the intelligent Container for logistical support robot system [5]. The container concept is a mediator between human’s request and robots’ capabilities. The intelligent Container includes put on and pick up the object. Moreover, the intelligent Container provide accessible user interface to display the device information to the user. However, the intelligent Container is standalone system and

Figure 1. Intelligent Space [1]

Intelligent space [1] concept was first proposed by Hashimoto Laboratory of University of Tokyo in 1995. Its goal was to support the resident within it. The Intelligent Space is room or areas that equipped with sensors, which enable the system © 2012 ACEEE DOI: 02.ACE.2012.03. 12

Full Paper Proc. of Int. Conf. on Advances in Computer Engineering 2012 rather obvious system. C. Our Purposed System In this paper, we propose a daily life object management system including its framework following RoiS standard. Main application services are searching and automatic object restoring. The system provides several functionalities including identification of the object, monitoring and tracking the object’s location (position), real-time recording of the object information, and recording information into the database (historical database). The system is also planned to provide the interface for retrieve and analyze object log in the future. The objects information including frequency of use, position, and status will be tracked recorded and represented (e.g. tagged by IDs) systematically. To prevent combination problem for whole objects in the entire house, we limit the space to the small area. In this research, we selected the RT-Shelf which represent storage space and RTCase to represent object container and achieve the object management system for those spaces.

Figure 3. Automatic Object Restoring Concept

This paper separated to five sections. Section II describes a framework and associated standard. In Section III, we discuss the system architecture overview. In section IV, we present the experimental implementation and results. Finally, we conclude the paper. II. OBJECT MANAGEMENT SYSTEM FRAMEWORK A. Robotic Interaction Service Object management system framework needs to design based on the complete robotic system framework. Therefore, we choose the framework defined in Robotic Interaction Service (RoIS) standard [6] from Object Management Group (OMG) as the entire robotic system framework. In the RoIS framework, every RT functional elements including hardware and software are abstracted as a Human-Robot Interaction (HRI) Component that is managed by HRI Engine as shown in Figure 3. Service Application sends commands based on the event message from HRI Engine by using the RoIS interface. To select commands, service application can query information. For these operations, service application may need to know the functions provided from HRI Engine. Hence, RoIS standard mentions not only interface between service application and HRI Engine but also profiles to represent 30 © 2012 ACEEE DOI: 02.ACE.2012.03.12

Figure 4. Robotic Interaction Service framework

functions of HRI Engine for service application. After receiving commands form service application, HRI Engine decides the robotic resource that can execute commands and dispatches them to appropriate robotic resource. B. Object Management System Functions To provide daily life object position to other RT functional modules working on the RoIS framework, we need to handle object management system as a HRI Component. We named this HRI Component as Goods Localization Component. HRI Components need to provide functions that can help robotic service scenario decision by service application. Hence, we defined Goods Localization Component as a HRI Component that provides following two functions. (1) Goods location change event notification (2) Providing goods latest position “Location” of (1) sentence equal to the place including table, shelf, etc. Therefore, as an example, first function can realize mail delivery notification to users. In addition, service application can provide automatic objects transportation service based on the first function. On the other hand, Second function can realize service for finding objects. C. Object Management System Interface Goods localization component notices event by using required interface named as Goods_Localized as shown in Figure 4. This interface is provided modules that need event message from goods localization component. On the other hand, goods position query is realized by Goods_Localization interface. This interface is provided by goods localization component and is designed as the interface that inherit RoIS_Common interface. RoIS_Common interface includes five methods that all HRI Components need to provide in common.

Figure 5. Goods localization component interface

Full Paper Proc. of Int. Conf. on Advances in Computer Engineering 2012 A. Object Positioning and Identification Object identification and positioning is one of the useful information. In this research, we assume that an environment has been separated into living space (area) and storing space. The living space represents the area where the object provides its function or usage to the resident. In the other hand, storing space represents the area where the object store when we finish using it. To represent these two different spaces, we proposed Robot Technology Case (RT-Case) which used for contains and carry objects and Robot Technology Shelf (RT-Shelf) which represents perceivable storage space. In living space, an ultrasonic positioning system, which provides accurate positioning solution, will be used to locate, and when the RT-Case enters the shelf, a near field RFID will be activated and used for positioning instead. The ultrasonic positioning system consists of transmitter and receiver. A transmitter emitted periodic ultrasonic signals to receivers mounted across the ceiling. Using Time of Arrival (TOA) and multi-lateration technique, we are able to measure a distance and produce object position accurately. The problems of using this ultrasonic technique are the requirement of the large number of receivers across the ceilings and their precise placements which needed quite sensitive alignments. With accurate information, the tri-lateration technique will give an accurate, unique answer, for example, the single point at the intersection of the circles (Figure 8A). With inaccurate information, the circles will not intersect at a point (Figure 8B). In this case, estimation of the position is found by looking for the point that simultaneously minimizes the distance to all circles, for example, by using Least Square Estimation (LSE) [7].

D. Object Management System Internal Structure Goods localization component is composed of goods localization sensor layer and localization software layer as shown in Figure 5. Additionally, we divided localization software into three layers. First is measurement. Measurement layer includes software that makes position data from sensor data. Second is aggregation. Aggregation layer includes software that output higher accuracy position or different meaning position from multiple position data. Last is transformation layer that realizes coordinate system transformation. Localization software includes measurement layer certainly and probably includes aggregation and transformation layer.

Figure 6. Goods localization component internal structure

III. OBJECT MANAGEMENT SYSTEM DESIGN The system consists of three main layers. The top layer is an application layer, which provides an application to the user. Second layer is a software layer, which is related to data and information processing, analyzing and deciding and hardware layer for acquiring the raw data. Software layer can be separated into two main parts. First part is object identification and positioning which collect and manage object information. Second part is human positioning and human posture recognition. In addition, object log has been integrated to collect and create the historical data.

Figure 8. Intersection Point

The constraints are the equations of the spheres with radii ri, (1) This linear system is simply written in matrix form (2)

Figure 7. Object Management System Overview

ÂŠ 2012 ACEEE DOI: 02.ACE.2012.03.12

Full Paper Proc. of Int. Conf. on Advances in Computer Engineering 2012 This linear system (2) has (n-1) equations in three unknowns. Therefore, theoretically only four receiver (n=4) are essential to determine the unique position in 3D; provided no more than two receivers are co-linear. Once the system has been liberalized, and since the distance ri is only approximate, the problem requires the determination of such that

 Robotic Technology Shelf (RT-Shelf) RT-Shelf is a Robot Technology furniture. The main function of the RT-Shelf is similar to traditional shelf for storing an object. To extend a perception, we assign a RFID tag to the shelf space to identify the storing status. Working with the RT-Case, which augmented with RFID Reader, the system can observe and identify the events when a new object is stored or retrieved from the shelf (storing space).  Robotic Technology Case (RT-Case) A main purpose of RT-Case is storing and containing an object. RT-Case reduces the difficulty of robot manipulation. We attached two kinds of positioning sensor; an ultrasonic sensor for positioning in living space and RFID reader for positioning in the storing space. E. Application Service  Automatic Restoring Object This service shows usability of the object information to produce some useful service. Previously, we specify the location for storing the object manually. However, our service provides automatic restoring of the object. The system use the general object information and object historical information in object log such as frequency of use or object category to specify the default storing place. In addition, human information is used as ticker events. For example, a default place of the book is book shelf (user often stores the book on the shelf) and now no human stay in the space (space is available), then the book is restored to shelf automatically by the robot.

(3) (4) B. Human Positioning and Posture Recognition This module is used for tracking and recognizing human posture. The information that we are interested is the position and posture. The posture categories to sitting, standing and lying [8]. The system uses these data to determine that what and when the service should be executed. For example, if the system detects that there is no individual in space, the system will be able to provide automatic object restoring service. C. Object Log Data is separated to current and historical database. In addition, a query interface (GUI) is provided for query, and make some report based on this historical data.  General Information of Object The General Information of object includes ID (primary key), name, type (category of the object), shape information (shape, height, width, and length), owner, etc. These data can be categorized into static data, which unchanged and non static data, which can be changed and modified.  Historical Information of Object When the object status or object position was significantly changed, the updated data including timestamp were recorded into object information log module. For example, when the key is moved from the RT-Shelf to the table, the new position with the timestamp will logged to the object historical information database.

Figure 11. Automatic Restoring Object Service

 Searching Object When user lost a key, user could query to see a current position of the object “key” (using an object ID) via the web interface that we provided (Figure 9). Figure 9. Object Log Module including Query Interface

D. RT-Shelf and RT-Case

Figure 10. A) RT-Shelf equipped with RFID tag and B) RT-Case with Ultrasonic Sensor and RFID Reader

Figure 12. Searching Object Service

Full Paper Proc. of Int. Conf. on Advances in Computer Engineering 2012 In 3D positioning (configuration as Figure.13) We obtain the relative distance between four receivers and transmitter, then calculate the position in (X, Y, Z) using a tri-lateration technique, which mentioned in section III. For example, four distance measurements between transmitter and receivers are 2,430 mm, 2,572 mm 2,588 mm and 2,574 mm, we can estimate the positioning X = 600, Y = 450 and Z = 154. In the experiment, we position 10 points and each point 10 times. The results in table I are the average error between the actual (X, Y, Z) and estimated (X, Y, Z).

IV. EXPERIMENTAL RESULT A. Living Space 3D positioning As we mention, Ultrasonic sensor is used for RT-Case positioning. We used ultrasonic sensor called Hexamite Ultrasonic Positioning System (HX19) [9]. HX19 ultrasonic sensor consists of transmitter (T), receiver (R) and synchronizer (position computing module). We install the receivers on the ceiling with fixed position and attach the transmitter on the RT-Cases (movable). Finally, 3D position is calculated based on tri-lateration technique based on least square approximate.

TABLE I. 3D POSITIONING AVERAGE ERROR

The results in Table I show that the 3D positioning error based on least square estimation is lower than 5 cm. However, the precision of the reference node (Receiver) positioning which is setting parameter is effect to the accuracy of the transmitter position. With 5 cm of the reference node error setting, the positioning error can enlarge to 19 cm error in transmitter positioning. Hence, we need to calibrate the reference node (receiver) positioning during installation to get the best positioning solution.

Figure 13. 3D Positioning Environment

We conducted two experiments for ultrasonic distance measurement. First experiment, we measured a distance and covering area using ultrasonic sensor at various range and direction to assure the accuracy of ultrasonic sensor. The experimental result has been shown in Figure 14. The result in Figure 14A shows that a maximum error during 10 cm to 2.8 m is about 5.5 mm. Ultrasonic covering area is calculated from the covering angles and height. At height = 2.5 m, themeasured error of distanced isrelatively low until 20 degrees. After 20 degrees, the error was dramatically increased. Hence, the covering area (X) is about 1 m (θ = 20, h = 2.5 m).

B. Storage Space RFID Detection and Positioning As shown in Figure 15, object management system for RT-Shelf is composed of three hardware, ID package builder module, ID/position converter module and RFID tag. ID package builder module represents storage relationship by making ID package. (1) RT-Case and stored objects (2) Case storage space and RT-Case By sending ID package, system can notice four event, RT-Case storage, RT-Case takeoff, object storage and object takeoff.

Figure 15. 3D Positioning Environment Figure 14. A: distance error with distance computation between 10 and 280 cm B: distance error at h = 2.5 m with θ is between 0 to 30 degrees

Full Paper Proc. of Int. Conf. on Advances in Computer Engineering 2012 CONCLUSIONS In this paper, we proposed the object management system, its framework which based on RoIS standard and provided two sample services: automatic object restoring object and searching object. The system is divided in to three layers: application layer, software layer and hardware layer. In hardware layer, RT-Case and RT-Shelf is conducted for object identifying, tracking and positioning. In software layer, the information that we interested includes human and object information. Moreover, the system provides object log, which consists of general and historical information. Finally, object management service is generated based on this information. Figure 16. ID package builder module for RT-Shelf

REFERENCES

ID package builder module makes ID package by reading RFID tag attached on an object and case storage space. As shown in Figure 16, RFID reader of ID package builder module is put in front side of each case and detects space RFID tag automatically. In addition, ID package builder module can send object storage and takeoff event message by manual object identification by users. ID package is converted into 3D position data by ID/position converter module. As shown in Figure 17, event is correctly detected.

[1] Joo-Ho Lee and Hideki Hashimoto, “Intelligent Space”, IEEE/ RSJ International Conference on Intelligent Robots and Systems, 2000, pp.1358–1363. [2] K. Ukai, Y. Ando, and M. Mizukawa, “Investigation of user RT-service generation system design”, Fuzzy Systems 2009, IEEE Int. Conf. on FUZZ-IEEE 2009, pp. 1486-1491, 2009. [3] Young Sang Choi, Travis Deyle, and Charles C. Kemp, “A list of household objects for robotic retrieval prioritized by people with ALS”, ICORR 2009., IEEE International Conference on Rehabilitation Robotics, pp 510-517, 2009. [4] Taketoshi MORI, Chetaphan SIRIDANUPATH, Hiroshi Noguchi, Tomomasa SATO, “Active RFID-Based Object Management System in Sensor-Embedded Environment”, FGCN2007 Workshop: International Symposium on Smart Home (SH07), Korea, pp.25-30, December 2007. [5] Rui FUKUI, Masayuki SHODAI, Taketoshi MORI, Tomomasa SATO, “Development of an intelligent Container Prototype for a logistical Support Robot System in Living Space”, IEEE/RSJ International Conference on Intelligent Robots and Systems, pp 3397-3402. [6] Robotic Interaction Service (RoIS), http://www.omg.org/spec/ RoIS/ [7] W. Murphy and W. Hereman, “Determination of a Position in Three Dimensions using Trilateration and Approximate Distance”, Report MCS-95-07, 1990. [8] Weerachai Skulkittiyut, Makoto Mizukawa, “Human Posture Recognition using Top-view Vision for Intelligence Space Environment”, ICCAS2010 pp.1687-1690, Oct 2010. [9] Hexamite Ultrasound, Ultrasonic Positioning System, http:// www.hexamite.com

Figure 17. ID Package Experimental Result