JAMRIS 2016 Vol 10 No 1

Page 1

VOLUME  10 N°1 2016 www.jamris.org pISSN 1897-8649 (PRINT) / eISSN 2080-2145 (ONLINE)

JOURNAL OF AUTOMATION, MOBILE ROBOTICS & INTELLIGENT SYSTEMS

Editor-in-Chief

Associate Editors:

Janusz Kacprzyk

Jacek Salach (Warsaw University of Technology, Poland) Maciej Trojnacki (PIAP, Poland)

(Polish Academy of Sciences, PIAP, Poland)

Statistical Editor:

Advisory Board:

Małgorzata Kaliczynska (PIAP, Poland)

Dimitar Filev (Research & Advenced Engineering, Ford Motor Company, USA) Kaoru Hirota (Japan Society for the Promotion of Science, Beijing Office) Jan Jabłkowski (PIAP, Poland) Witold Pedrycz (ECERF, University of Alberta, Canada)

Language Editor: Grace Palmer (USA)

Typesetting: Ewa Markowska, PIAP

Co-Editors: Roman Szewczyk (PIAP, Warsaw University of Technology) Oscar Castillo (Tijuana Institute of Technology, Mexico) Marek Zaremba (University of Quebec, Canada) (ECERF, University of Alberta, Canada)

Executive Editor: Anna Ładan aladan@piap.pl

Webmaster: Piotr Ryszawa, PIAP

Editorial Office: Industrial Research Institute for Automation and Measurements PIAP Al. Jerozolimskie 202, 02-486 Warsaw, POLAND Tel. +48-22-8740109, office@jamris.org Copyright and reprint permissions Executive Editor The reference version of the journal is e-version. Printed in 300 copies.

Editorial Board: Chairman - Janusz Kacprzyk (Polish Academy of Sciences, PIAP, Poland) Plamen Angelov (Lancaster University, UK) Adam Borkowski (Polish Academy of Sciences, Poland) Wolfgang Borutzky (Fachhochschule Bonn-Rhein-Sieg, Germany) Chin Chen Chang (Feng Chia University, Taiwan) Jorge Manuel Miranda Dias (University of Coimbra, Portugal) Andries Engelbrecht (University of Pretoria, Republic of South Africa) Pablo Estévez (University of Chile) Bogdan Gabrys (Bournemouth University, UK) Fernando Gomide (University of Campinas, São Paulo, Brazil) Aboul Ella Hassanien (Cairo University, Egypt) Joachim Hertzberg (Osnabrück University, Germany) Evangelos V. Hristoforou (National Technical University of Athens, Greece) Ryszard Jachowicz (Warsaw University of Technology, Poland) Tadeusz Kaczorek (Bialystok University of Technology, Poland) Nikola Kasabov (Auckland University of Technology, New Zealand) Marian P. Kazmierkowski (Warsaw University of Technology, Poland) Laszlo T. Kóczy (Szechenyi Istvan University, Gyor and Budapest University of Technology and Economics, Hungary) Józef Korbicz (University of Zielona Góra, Poland) Krzysztof Kozłowski (Poznan University of Technology, Poland) Eckart Kramer (Fachhochschule Eberswalde, Germany) Rudolf Kruse (Otto-von-Guericke-Universität, Magdeburg, Germany) Ching-Teng Lin (National Chiao-Tung University, Taiwan) Piotr Kulczycki (AGH University of Science and Technology, Cracow, Poland) Andrew Kusiak (University of Iowa, USA)

Mark Last (Ben-Gurion University, Israel) Anthony Maciejewski (Colorado State University, USA) Krzysztof Malinowski (Warsaw University of Technology, Poland) Andrzej Masłowski (Warsaw University of Technology, Poland) Patricia Melin (Tijuana Institute of Technology, Mexico) Fazel Naghdy (University of Wollongong, Australia) Zbigniew Nahorski (Polish Academy of Sciences, Poland) Nadia Nedjah (State University of Rio de Janeiro, Brazil) Duc Truong Pham (Birmingham University, UK) Lech Polkowski (Polish-Japanese Institute of Information Technology, Poland) Alain Pruski (University of Metz, France) Rita Ribeiro (UNINOVA, Instituto de Desenvolvimento de Novas Tecnologias, Caparica, Portugal) Imre Rudas (Óbuda University, Hungary) Leszek Rutkowski (Czestochowa University of Technology, Poland) Alessandro Saffiotti (Örebro University, Sweden) Klaus Schilling (Julius-Maximilians-University Wuerzburg, Germany) Vassil Sgurev (Bulgarian Academy of Sciences, Department of Intelligent Systems, Bulgaria) Helena Szczerbicka (Leibniz Universität, Hannover, Germany) Ryszard Tadeusiewicz (AGH University of Science and Technology in Cracow, Poland) Stanisław Tarasiewicz (University of Laval, Canada) Piotr Tatjewski (Warsaw University of Technology, Poland) Rene Wamkeue (University of Quebec, Canada) Janusz Zalewski (Florida Gulf Coast University, USA) Teresa Zielinska (Warsaw University of Technology, Poland)

Publisher: Industrial Research Institute for Automation and Measurements PIAP

If in doubt about the proper edition of contributions, please contact the Executive Editor. Articles are reviewed, excluding advertisements and descriptions of products. All rights reserved © Articles

1

JOURNAL OF AUTOMATION, MOBILE ROBOTICS & INTELLIGENT SYSTEMS VOLUME 10, N° 1, 2016 DOI: 10.14313/JAMRIS_1-2016

CONTENTS 3

43

Invisible Leash: Object-Following Robot Elisabeth Frink, Daniel Flippo, Ajay Sharda DOI: 10.14313/JAMRIS_1-2016/1

Design of Piezoelectric Gravimeter for Automated Aviation Gravimetric System Igor Korobiichuk, Olena Bezvesilna, Andrii Tkachuk, Tetyana Chilchenko, Michal Nowicki, Roman Szewczyk, Jan Studzinski DOI: 10.14313/JAMRIS_1-2016/6

8

Comparison of Two Nonlinear Predictive Control Algorithms for Dissolved Oxygen Tracking Problem at WWTP Robert Piotrowski DOI: 10.14313/JAMRIS_1-2016/2

17

Mathematical Modelling and Feasibility Study of the Biomimetic Undulating Fin of a Knife Fish Robot Ajith Anil Meera, Attadappa Puthanveetil Sudheer DOI: 10.14313/JAMRIS_1-2016/3

48

Recognition of Hand Posture Based on a Point Cloud Descriptor and a Feature of Extended Fingers Dawid Warchol, Marian Wysocki DOI: 10.14313/JAMRIS_1-2016/7

58

AJVM – JAVA Virtual Machine Implemented in Actionscript 3.0 Arkadiusz Janik, Jakub Krawczyk DOI: 10.14313/JAMRIS_1-2016/8

26

Design Optimization of the Biomimetic Undulating Fin of a Knife Fish Robot Ajith Anil Meera, Attadappa Puthanveetil Sudheer DOI: 10.14313/JAMRIS_1-2016/4

34

State of the Art in Predictive Control of Wheeled Mobile Robots Patryk Harasim, Maciej Trojnacki DOI: 10.14313/JAMRIS_1-2016/5

2

Articles

69

A New Integer Linear Programming and Quadratically Constrained Quadratic Programming Formulation for Vertex Bisection Minimization Problem Pallavi Jain, Gur Saran, Kamal Srivastava DOI: 10.14313/JAMRIS_1-2016/9

Journal of Automation, Mobile Robotics & Intelligent Systems

I

L

VOLUME 10,

:O

-F

N∘ 1

2016

R

Submi ed: 10th November 2015; accepted: 21st December 2015

Elizabeth Frink, Daniel Flippo, Ajay Sharda DOI: 10.14313/JAMRIS_1-2016/1 Abstract: This study looked at developing a simple concept for an object-following robot, meant to follow a person in an agricultural se ng. A KIPR Link was used as the framework, and simple mathema cal rela onships were derived to control the mo on of the robot. The work described is meant as a proof-of-concept, not as a full-scale development. Reasonable results were obtained, indicating that the robot could successfully follow the object. Keywords: object-following, agriculture, KIPR Link, unmaned robot

1. Introduc on Unmanned robots are of signi icant interest currently. There is a push from the NSF (National Science Foundation) to accelerate the development of robots that can work beside or cooperatively with people [2]. In the past, several different approaches have been used to achieve unmanned control. These approaches are summarized in the following sections. The goal of this work is to develop a person-following method that will work in many settings, where obstacles and equipment are always changing. Agriculture could use this method to haul equipment around the farm or ield while the user is able to use his or her hands for other tasks. Outside of agriculture, this method could also be used for handicapped persons who need help carrying multiple items, such as groceries. Additionally, any manual laborers who have a large amount of equipment to carry around could bene it from this person-following method. 1.1. Path Learning Path learning has been applied in several techniques. Araujo and Ameida [1] uses a trial-and-error method, with learning, to navigate a path. Spatial resolution is chosen using a game-theoretic cell-splitting criteria. This method is able to avoid obstacles and with each repeat of the same path or environment, its navigation becomes quicker (less learning necessary). This does not it as a direct solution for the problem, but this method is of particular interest as an addition for future work, allowing a unmanned robot to navigate obstacles while following a person. 1.2. Image-Based Visual Servoing Multiple projects [3, 7, 8] have used image-based visual servoing, or a modi ication of the method. This

method uses the image Jacobian to determine velocity vectors. It is necessary to know points in the environment in this method, but a modi ication of the method, Image Based Visual Navigation developed by [7], does not need this information. Instead, motion vectors from consecutive images are obtained, and compared to the desired motion vectors. Both methods are very involved mathematically, but a simpler solution is desired for this application. 1.3. Person Following Hu et al tracks the size of a human torso, along with a fuzzy PID, in order to follow the human [4]. This method is limited in its ability to keep up with the human, but is able to discern the target human from other humans in the environment and reliably follow the target at slower speeds. If it were not for the limited ability to keep up with a human, this could be a potential method for the application. Future work will involve including additional vision processors. 1.4. Feature-Matching Geometrical features of strategically placed landmarks (distinct geometric images) with localization and line tracking algorithms are used to navigate indoors in work by Hu et al [9]. The robot from this work can achieve a speed of 18 cm/s and it is able to accurately discern its location from the landmarks. In order for a robot to follow another robot, geometrical and color-coded patterns can be used, as shown by [6]. Then covariance matching allows the following robot to correctly identify its target robot/pattern. It is not reasonable to place geometric features strategically in an agricultural ield, so this method would be less suitable for the desired application. 1.5. Project Objec ve This paper will describe a proof of concept for a simple object-following robot, where the object could be carried by or attached to a human. The concept will be referred to as �InviLeash�. The feasibility of this concept will be illustrated with preliminary results. As far as the authors are aware, an object-following approach of this simplicity has not been developed yet.

2. Methods The concept behind InviLeash is straightforward. The robot will follow a single-color ball, using the process illustrated in ig. 1 and described in later sections. Initial calibration requires acquiring an image of the ball at the desired distance and determining the 3

Journal of Automation, Mobile Robotics & Intelligent Systems

VOLUME 10,

N∘ 1

2016

image coordinates of the ball. The desired distance was selected as a reasonable following distance of 45.7 cm. After initial calibration, a loop is iterated, acquiring images, determining the current ball coordinates, comparing to desired, determining direction vector and translating to motor motion.

Fig. 2. Object-following robot prototype

x

z Fig. 1. Concept map for object following

F o r w a r d

2.1. Proof of Concept InviLeash uses the KIPR platform from the KISS Institute for Practical Robots [5]. This platform was selected because it has ”blob detection” built in, making it ideal for a proof of concept. It also is an open source platform and has a built-in motor-control library. The KISS IDE environment for Windows was used for programming. More information on the KIPR Link and the KISS IDE can be found in the KIPR Link Manual or the KIPR Link C Standard Library (provided with KISS IDE download). 2.2. Design The KIPR module with USB camera was mounted to a four wheel drive remote control car. The servo motors of the car were controlled by the on-board KIPR motor controllers. Fig. 2 shows the stripped car with module and camera mounted. One motor drives the front axle, another motor drives the rear axle, and a third motor turns the wheels on the front axle. Fig. 3 shows the direction conventions for the robot, as a top view. Forward is a negative z-value, reverse is positive. A negative x-value is left and right is positive. A ping-pong ball was used as the ball to follow. In order to avoid confusion with other objects in the 4

Fig. 3. Direc on conven ons (top view) vicinity, the ball was painted pink (a distinct and contrasting color not found in the environment). Using the camera channel settings on the KIPR touchscreen, the camera was set to detect the pink color. For each image that the camera acquires, a box is identi ied around the largest ”blob” of pink (it is assumed that this is the ball, as there is no other pink in the environment) using the built-in features of the KIPR. The coordinates of the box and the length of the horizontal sides are used to determine the ball’s location in the image. Fig. 4 shows the parameters for the desired ball image location (white box) and the actual ball image location (shaded box). It was determined through iteration that the horizontal width of the blob provided more repeatable results than the vertical height of the blob. Determining Z-Posi on The z-distance of the ball can be estimated simply using the width of the blob. The relation between z-distance and blob width was calibrated using a second-order polynomial. Fig. 5 shows

Journal of Automation, Mobile Robotics & Intelligent Systems

a1

N∘ 1

2016

position between 0 and 2047 ticks) for the required travel distance. For the z-direction motors, a position command of 250 ticks results in 50.8 cm travel distance for the vehicle used here.

a3

d3 a2 aw

d1 d2

VOLUME 10,

dw

Fig. 4. Size parameters of desired blob (white) and actual blob (shaded) the results for six trials each at four different ball distances. The resulting calibration equation it to all 24 data points is �

Fig. 6. Calibra on of ball distance

= −0.0006đ?‘Ž +0.1009đ?‘Ž −5.6627đ?‘Ž+117.7 (1)

where a is the width of the ball in the image, in pixels and z dist is the z-distance of the ball from the robot, in cm. From this, the distance to travel, in cm, can be determined by subtracting the 45.7 cm desired distance. It is likely that the calibration is not linear because of the non-linear optical properties of a camera lens. �

=�

− 45.7

(2)

Determining X-Posi on The inal calculations are for the x-direction. Using a 1 , a 3 , d 1 , d 3 , and blob width, the difference in horizontal center of the blob vs horizontal center of the desired position is calculated, then the y-component of the distance is removed and only the x-component of the distance is considered.

đ?‘Ľ

= ((đ?‘Ž +đ?‘Ž )/2−(đ?‘‘ +đ?‘‘ )/2)/(0.03667đ?‘Ž+0.59667) (4) where x dist is the x-component of the distance from actual center to desired center (as described above) and the other parameters are as shown in ig. 4.

Determining X-Motor Command The calibration for the x-direction motor command would depend on how far the x-direction motors would travel, so to simplify this portion of the method a set motor command was used for x-direction. If the robot needs to move right the x-motor command would be +50 ticks, and if the robot needs to move left the x-motor command would be -50 ticks. Fig. 5. Calibra on of ball distance

Determining Z-Motor Command The servo motor command for z-direction must also be calibrated. The relation between motor command and traveled distance is almost linear, but deviates from linear at the two extremes of distance (near and far). The calibration data is shown in ig. 6, with a linear and a 2nd order polynomial it. The 2nd order polynomial was selected as a better it. The equation is �

= 0.0002(�

) + 0.3208(�

) − 4.9257 (3) where z trav is the required travel distance in cm, and z motor is the necessary servo motor command (a

Mo on Finally, the three motor commands (one for x-direction, two identical for the z-direction) are sent and then the motors are shut off after the commanded positions are reached. From this point, the loop is repeated. 2.3. Results Overall, the InviLeash had satisfactory performance, considering its limitations (discussed later). The average response and standard deviation from six trials is summarized in tab. 1. It can be noted that as the ball moves further from the camera (in the direction of decreasing desired travel), the standard deviation increases. This is likely related to the image resolution of the system. Fig. 7 and ig. 8 show a visual representation of the results. The average response was promising. 5

Journal of Automation, Mobile Robotics & Intelligent Systems

Tab. 1. Actual Travel Results Desired Travel [in.] 30.5 15.2 0 -15.2

VOLUME 10,

N∘ 1

2016

3. Applica ons

Actual Travel [cm] Avg StDev 22.3 3.9 13.2 1.5 -4.2 5.4 -11.0 8.8

Future work will focus on applying this technique to agricultural applications, such as a helper robot to follow a worker around with equipment or tools. The robot will incorporate path-learning so that it can also function without user interaction. Obstacle avoidance will also be necessary, as well as rugged terrain navigation ability. This technique could also be used in industrial settings, to tow trailers or to haul pallets around a warehouse. This platform could link with sensors such as LIDAR for local steering, or could be linked with GPS to supplement or replace navigation information in locations where GPS information is limited. Additionally, it is envisioned that color-lighted wands or balls could be used as the object to follow, so that the robot can distinguish between two different people. Along the same lines, a serial pulse code could be employed.

4. Conclusions Fig. 7. Repeated Response

Although this is a very simple platform, the potential for future work is abundant. The initial results were promising regarding the ability to follow an object. There are limitations in the platform, including processing speed which limits the speed of following, and in-consistent identi ication of the target, but there are plans to minimize the limitations. Employing a different computing platform such as Arduino or LabVIEW myRIO may help improve processing and following speed. Additionally, using a distinct lashing light as the target may help in consistent identi ication of the target.

AUTHORS Fig. 8. Average Response

Limita ons and Problems The results of the InviLeash were limited by some important factors, summarized in tab. 2. The largest issue was with the limitations in image resolution. Regardless of the camera, the KIPR system has a set resolution of 160x120. This greatly degrades the accuracy of the system. In future work, a different platform will be used, such as openCV, an open source computer vision program.

6

∗

Corresponding author

REFERENCES

Tab. 2. Issues with Proof of Concept Issue

Cause

Solution

Distances are approximate

Limited image resolution on KIPR Small screen on KIPR, no readout to computer screen

Use a different platform, such as openCV Use a different platform, such as openCV

Limited troubleshooting

Elizabeth Frink – Kansas State University, Manhattan, KS, e-mail: enfrink@ksu.edu. Daniel Flippo∗ – Kansas State University, Manhattan, KS, e-mail: dk lippo@ksu.edu, www: http://www.bae.ksu.edu/people/faculty/ lippo/ index.html. Ajay Sharda – Kansas State University, Manhattan, KS, e-mail: asharda@ksu.edu, www: http://www.bae.ksu.edu/people/faculty/sharda/ index.html.

[1] R. Araujo and A. de Ameida, “Mobile robot pathlearning to separate goals on an unknown world”. In: Intelligent Engineering Systems Proceedings, Budapest, 1997. [2] N. S. Foundation. 2014.

“National robotics initiative”,

[3] C. Gao, X. Piao, and W. Tong, “Optimal motion control for ibvs of robot”. In: Proceedings of the 10th World Congress on Intelligent Control and Automation, Bejing, China, 2012.

Journal of Automation, Mobile Robotics & Intelligent Systems

VOLUME 10,

N∘ 1

2016

[4] C.-H. Hu, X.-D. Ma, and X.-Z. Dai, “Reliable person following approach for mobile robot in indoor environment”. In: Proceedings of the 9th International Conference on MAchine LEarning and Cybernetics, Boading, 2009. [5] KISS. “Kiss hardware/software”, 2014. [6] H. J. Min, N. Papanikolopoulos, C. Smith, and V. Morellas, “Feature-based covariance matching for a moving target in multi-robot following”. In: 19th Mediterranean Conference on Control and Automation, Corfu, Greece, 2011. [7] L. O’Sullivan, P. Corke, and R. Mahony, “Imagebased visual navigation for mobile robots”. In: 2013 IEEE International Conference on Robotics and Automation, Karlsruhe, Germany, 2013. [8] L. Pari, J. Sebastian, A. Traslosheros, and L. Angel, “Image based visual servoing: Estimated image jacobian by using fundametal matrix vs analytic jacobian”, Image Analysis and Recognition, vol. 5112, 2008, 706–717. [9] F. Wen, K. Yuan, W. Zou, X. Chai, and R. Zheng, “Visual navigation of an indoor mobile robot based on a novel arti icial landmark system”. In: Proceedings of the 2009 IEEE International Conference on Mechantronics and Automation, Changchun, China, 2009.

7

Journal of Automation, Mobile Robotics & Intelligent Systems

VOLUME 10,

N째 1

2016

Comparison of Two Nonlinear Predictive Control Algorithms for Dissolved Oxygen Tracking Problem at WWTP Submitted: 18th June 2015; accepted: 25th September 2016

Robert Piotrowski DOI: 10.14313/JAMRIS_1-2016/2 Abstract: The wastewater treatment plant is classified as a complex system due to its nonlinear dynamics, large uncertainty in the disturbance inputs, multiple time scales in the internal process dynamics and multivariable structure. Aeration is an important and expensive activity that is carried out during wastewater treatment plant operation. A precise aeration control in biological processes for all the operating conditions is necessary in order to guarantee adequate biological conditions for microorganisms. Therefore, the most important is to operate the biological processes so that a minimal energy was consumed and minimal DO concentration demand was applied. The paper proposes a two advanced control systems to track the dissolved oxygen reference trajectory. A decentralized and multivariable nonlinear predictive control algorithms are designed and compared. Simulation tests for the case study wastewater treatment plant are presented. Keywords: aeration, dissolved oxygen, wastewater treatment, mathematical modelling, nonlinear system, predictive control

1. Introduction

1.1. General information

8

Progressive development of technology and human population growth caused the amount of produced sewage to increase. Years of experience and scientific research, gave rise to the idea about wastewater treatment plant (WWTP). It is a complex, nonlinear biological-chemical-physical system with strong interactions between processes. Processes are difficult to control due to large disturbances such as inflow and load, nonlinearity, time-varying and complexity with strong coupling of the process variables. The popular treatment technology used in the field of WWTPs is the biological. Most of the municipal WWTPs use activated sludge processes. Activated sludge refers to a mass of microorganisms where help clean up wastewater. This is a process in which air or oxygen is forced into sewage liquor to develop a biological floc which reduces the organic content of the sewage. Activated sludge is used where high removal of organic pollution is required. For effective wastewater treatment, an activated sludge requires a continuous supply of oxygen/air. Aeration is used in different parts of a WWTP, especially is applied to biological processes (e.g. nitrification, denitrification

and phosphorus removal). Additionally, energy consumption of aeration processes has the major share in energy bill at a WWTP. Dissolved oxygen (DO) concentration is the most important control parameter in the biological WWTP. Unfortunately, in industrial practice, the control of DO is usually carried out using a simple feedback loop (e.g. linear PI algorithm with constant parameters) that does not aim at all at combining the tracking functions with the energy cost optimization. Good control performance for all the operating conditions cannot be achieved. Furthermore, the lack of on-line measurements at the WWTP, may result in limited use of advanced control systems.

1.2. Survey of related works

As mentioned earlier, control of DO is very important and difficult activity in the activated sludge processes. As the DO dynamics is nonlinear and typically WWTP operates under high variability of the influent quality and pollutant parameters. Works related to control of DO have a long history. Different control technologies have been researched over the last years, e.g. adaptive controller [2], on/off controller with genetic algorithms [11], multivariable PID controller [22], nonlinear multiregional PI controller [23], fuzzy controller [2]. The second group of DO control strategies are algorithms, where beside DO measurement, additionally ammonia nitrogen (NH4), nitrate (NO3) and phosphate (PO4) measurements are included for control system design [21,17]. An interesting and comprehensive review of DO control can be found in [1]. The model predictive control (MPC) technology is an attractive method for dynamic optimizing control. The MPC optimizer enables for direct incorporation of the constraints in the control problem into the optimization task at each time step, what is a great advantage of this technology. During the last years, many industrial applications of control systems have utilized the MPC technology [20]. The MPC algorithm was also applied for DO control at the WWTP. The linear MPC algorithm was proposed in [12] and its application to benchmark simulation WWTP. In [5] the hybrid nonlinear predictive controller was applied. The binary control signals were reformulated to nonlinear programming task at every sampling time. Interesting control results were obtained. In [7] the MPC system of DO control based on self-organizing radial basis function neural

Journal of Automation, Mobile Robotics & Intelligent Systems

VOLUME 10,

Air

Influent from mechanical part

Anaerobic section

Anoxic section

Internal recycle 2

Aerated section

N째 1

2016

Blowers station

Secondary Effluent settler

Internal recycle 1

Waste sludge

External recycle

Fig. 1. Scheme of biological WWTP at Kartuzy

network model was designed and examined for the benchmark simulation WWTP. In [16] a decentralized nonlinear hierarchical predictive control system was proposed. Biological reactor to be coupled with an aeration system and in order to track prescribed DO trajectories. The upper level controller (UCL) prescribes trajectories for airflows desired to be delivered into the aerobic biological reactor zones. The lower level controller (LLC) forces the aeration system to follow those set point trajectories. This controller was synthesized as a hybrid (discrete and continuous manipulated variables) nonlinear MPC. Authors of the paper [6] derived a nonlinear hybrid predictive control algorithm for the LLC. Dedicated operators were used to derive genetic algorithms, thus allowing for efficient handling of the switching constraints and nonlinear hybrid system dynamics. In [15] a nonlinear multivariable hierarchical MPC was applied to design the ULC. This controller extends to plants with several aerobic zones supplied by an aeration system of limited capacity. This controller handles the airflow distribution between the zones. The constraint on the airflow that can be delivered by the aeration system is then active. The ULC produces the airflow demands as set points for the LLC. In summary, the aim of the paper is the design of the hybrid nonlinear control system to perform an efficient control quality of DO concentration for different operating conditions. This paper further develops the results presented in [16,15]. Two advanced hierarchical nonlinear predictive control systems are designed and compared. Only DO and airflow measurements are applied for control systems design. In this paper, as opposite for previous research works [16,15] dynamic of aeration system is omitted and treated as a static system, which provides the amount of air required by the control system. Simulation tests for case study plant are presented.

2. Description of the WWTP

Kartuzy WWTP is typical system with a continuous flow throughout the plant. This system consists of two parts. One is the mechanical where solids, mineral and insoluble organic pollutants are removed. The second is the biological part that is composed

of the activated sludge process. The activated sludge method is applied for biological wastewater treatment processes. Only a biological part is considered in the paper. The structure of biological part for the case study plant is shown in Fig. 1. The advanced biological treatment with nutrient removal is accomplished in the activated sludge reactor designed and operated according the University of Cape Town (UCT) process. The first zone where the phosphorus is released is anaerobic. The second zone where the denitrification process is conducted is anoxic. The internal recirculation 2 of mixed liquor originates from the anoxic zone. The returned activated sludge from the bottom of the clarifiers and the internal recirculation 1 from the end of the aerobic zone (containing nitrates) are directed to the anoxic zone. The last part of the reactor (aerobic) is aerated by a diffused aeration system. This zone is divided into four compartments of various intensity of aeration. The anaerobic, anoxic, and four aerobic tanks have volumes of 800, 800, 700, 1760, 860, and 1150 m3, respectively. Oxygen is supplied to aerated tanks by the aeration system (blowers, pipes, throttling valves and diffusers). Wastewater is mixed with activated sludge, which helps during the process of wastewater treatment. Different aeration methods are applied: high-purity oxygen aeration, mechanical aeration, and diffused aeration [13]. The last technique is used at the Kartuzy WWTP. The aeration system delivers air to each of the aeration tanks. The wastewater and activated sludge are separated into two parallel secondary settlers. The volume of each secondary settler is approximately 1600 m3. The activated sludge is internally recirculated from the anoxic tank to the anaerobic zone and from the last aerobic zone to the anoxic tank. These recirculations are typically set to 45 to 100% and 210% of influent waste. Additionally, the wastewater is recirculated from the secondary settlers to the anoxic tank (45 to 100% of influent waste). The excess waste sludge is removed, chemically stabilized, and stored.

3. Control structures

It is industry practice that simple technology is used to control of DO: manual control, rule-based control and PI controller with fixed parameters. High quality of Articles

9

Journal of Automation, Mobile Robotics & Intelligent Systems

VOLUME 10,

Airflow trajectory Qair DO concentration trajectory S o

. . .

Nonlinear MPC1 Qair, 1 ref

. . .

4.1. Model of biological WWTP The most popular mathematical description of biological processes at WWTP is a series defined by Activated Sludge Models (ASMs) proposed by International Water Association (IWA). The models (ASM1, ASM2, ASM2d, ASM3) were presented and summarized in [10]. A critical reviews of activated sludge modelling for seven most commonly used models were presented in [8]. In the paper the biological processes are modelled by ASM2d model. ASM2d consists of 21 state variables and 20 kinetic and stoichiometric parameters. Values of those parameters are equal to their default values at 20°C [9]. ASM2d model was calibrated based on real data sets from the Kartuzy WWTP. Additionally, data from the plant permitted to define the quality of load: chemical oxygen demand (COD), total nitrogen concentration (TN) and total phosphorous concentration (TP). Verification of the modelling results was satisfactory and so they were used for control purposes.

4.2. Model of DO concentration

A dynamics of dissolved oxygen for the j-th aeration tank is described by the following nonlinear differential equation:

(1)

So, 4 ref

DO concentration reference trajectory

Nonlinear MPC4

Dynamic model of DO concentration So

Airflow reference trajectory Inflow

ref . . . Qair, 4

MODEL OF BIOLOGICAL WWTP

Fig. 2. Structure of the control system 1

So, 1 ref

Airflow trajectory Qair DO concentration trajectory S o

Articles

So, 4 ref

DO concentration reference trajectory

Nonlinear Multivariable MPC Qair, 1 ref

ref . . . Qair, 4

Dynamic model of DO concentration So

Airflow reference trajectory Inflow

MODEL OF BIOLOGICAL WWTP

Fig. 3. Structure of the control system 2 10

. . .

2016

4. Mathematical models

control cannot be obtained by conventional and linear control methods. MPC technology is an attractive optimizing method for advanced control of dynamic nonlinear systems. The nonlinear MPC algorithm uses the DO model for prediction. The predictive control adjusts its operation in advance, before there are changes in the output of the control system. The two structures of the new control systems are illustrated in Figs. 2–3. The controller’s objective is to force the So,j (controlled variable) into zones indicated by prescribed references So,jref and, at the same time, to minimize the associated electricity cost by blowing air. The controller’s objective is to force the So,j (controlled variable) into zones indicated by prescribed references So,jref and, at the same time, to minimize the associated electricity cost by blowing air. The manipulated variables in the control systems are airflows to the each of the aerobic tanks. Decentralized control system (Fig. 2, control system 1), based on information about DO concentration calculates control trajectories of the airflows, taking into account its constraints on minimum and maximum value as well as the rate of change in one step prediction. Respiration R is the disturbing variable that affects DO concentration. All aeration tanks are connected to a single aeration system. Hence, four independently operating controllers may have a problem with distributing required amounts of air correctly (e.g., for large inflow to the WWTP and/ or a sudden increase in concentration of pollutants in wastewater flow). Therefore, a multivariable, nonlinear MPC is also designed (Fig. 3, control system 2).

So, 1 ref

N° 1

Journal of Automation, Mobile Robotics & Intelligent Systems

VOLUME 10,

where So – dissolved oxygen concentration, kLa – oxygen transfer function, Qair – airflow, R - respiration, So,sat = 8.637 g O2 /m3 – dissolved oxygen saturation concentration, Ko = 0.2 g/m3 – Monod’s constant. The function, kLa(Qair), describes the oxygen transfer and depends on the aeration actuating system and sludge conditions. Different approaches to modelling this function are presented in literature. In this paper, the linear model is applied:

(2)

where α = 0.208 1/m3 [18]. The respiration R is an important parameter to biological processes taking place in aerobic zones. The respiration varies with time, depends on the biomass concentration, and describes oxygen consumption by the microorganism. This variable can be calculated using the ASM2d model [9]; however, to determine R, another 18 nonlinear differential equations in the ASM2d model are required. Because of the complexity of the ASM2d model, respiration R is treated as an external disturbance signal. Respiration can be measured by a respirometer. In some research, respiration was assumed to measure and be used for monitoring and control of biological processes. However, dedicated measuring equipment is very expensive; hence, this variable is rarely being measured. Therefore, respiration has to be estimated to be used for control. Different approaches based on the augmented Kalman filter were taken for joint estimation of respiration and oxygen transfer functions [14]. Another approach was presented in literature [19]. A sequential algorithm with the Kalman filter was proposed and investigated. The approach presented in [4] is used in this paper. It is based on point estimation of respiration at the appropriate time instant valid for a particular prediction horizon sufficient to be taken as constant.

5. Predictive controllers design

where

N° 1

.

2016

(4)

In (4), it is important to note that there is a onestep delay. This has no practical significance because of the variability of the aforementioned slow respiration in relation to the rate dissolved oxygen concentration changes. Moreover, it is possible to treat the temporary estimate as a constant prediction of respiration over a suitably selected length of prediction horizon. Because (4) contains measurements, respiration (i.e., differentiation of the measurement noise present in the measurement of dissolved oxygen) occurs during estimation. Results of previous studies [4] indicate that, provided a typical signal is within acceptable noise levels, its effect on the quality of estimates can be omitted (see Figs. 4–7). In Figs. 4–7 the examples trajectories of the DO concentration and respiration R are presented, with (Fig. 4 and 6) and without (Fig. 5 and 7) measuring noise. Knowing the measurement errors characteristic of devices for measuring the DO concentration, the standard deviation of measurement equal 0.1 g O2/m3 was assumed. Results confirm the assumption of low impact of measurement noise in the measurement of DO concentration on the quality of estimate of respiration R. In addition, simplification of the model and the resulting inaccuracies are eliminated by feedback mechanism which is an integral part of the control system.

5.1. Control system 1

The nonlinear MPC performance function for j-th aeration tank is defined as:

A model of DO concentration is needed to design an MPC (see (1)). For the j-th aeration tank, the following discrete model can be formulated:

(3)

where k and T are the discrete time instant and dissolved oxygen sampling interval. At the time instant ( k + 1)T , the estimate Rˆ j ( k ) of R j ( kT ) can be obtained by discretizing (3), solving the resulting discrete time equation for the unknown ∆ respiration value R j ( k ) = R j ( kT ), and substituting the expression S o , j ( k ) , S o , j ( k + 1) with∆the measurements S om, j ( k ) , S om, j ( k + 1) where S o , j ( k ) = S o , j ( kT ) . Hence:

(5)

The first term in (5) represents the tracking error. The second and third terms describe rates of changes of the control input over Hp (prediction horizon) while the fourth term represents the control cost. The weights α, β and χ are tuning knobs used to achieve a desired compromise between the tracking error, the intensity of switching the blowers, and the cost of the energy used for pumping air. min max Let Qair , j and Qair , j be the minimum and maximum value of the airflow, respectively. The constraints Articles

11

Journal of Automation, Mobile Robotics & Intelligent Systems

VOLUME 10,

N° 1

2016

3

2.8

DO concentration So [g O2 / m3 ]

DO concentration So [g O2 / m3 ]

2.6 2.4 2.2 2 1.8 1.6 1.4 1.2

2.5

2

1.5

1

1 0.8 0

0.2

0.4

0.6

0.8 1 1.2 Time [day]

1.4

1.6

1.8

0.5 0

2

1

0.9

0.9

0.8

0.8

0.7

0.7 0.6 0.5 0.4 0.3

0.6

0.8 1 1.2 Time [day]

1.4

1.6

1.8

2

1.8

2

0.6 0.5 0.4 0.3 0.2

0.2

0.1

0.1 0 0

0.4

Fig. 5. DO concentration in aeration tank – with measurement noise

Respiration R [mg / m3 h]

Respiration R [mg / m3 h]

Fig. 4. DO concentration in aeration tank – without measurement noise

0.2

0.2

0.4

0.6

0.8 1 1.2 Time [day]

1.4

1.6

1.8

0 0

2

0.2

0.4

0.6

0.8 1 1.2 Time [day]

1.4

1.6

Fig. 6. Respiration in aeration tank – without measurement noise

Fig. 7. Respiration in aeration tank – with measurement noise

on minimum and maximum values of control action at each prediction step are defined as follows:

are taken from the measurements. This results in the optimised Qair , j (k k ) , ... , Qair , j k + H p = 1 k DO trajectories over the prediction horizon.

where

and

.

(6)

(

)

5.2. Control system 2

The MPC performance function is written as:

max Moreover, ∆Qair , j is the maximum value of the rate of change. The constraints are given by:

(7)

12

where . At the time instants kT the nonlinear MPC solves its optimisation task by minimising the performance function (5) with respect to the aeration flows subject to the constraints (6) and (7). The DO concentrations S o , j (k + i k ) at the aerobic zones predicted over Hp are calculated by using the discretized models (3). The respirations R j ( k + i − 1) in these models are replaced by their predictions that are calculated according to (4) based on the DO measurements at the aerobic zones. The initial conditions Articles

(8)

The first term in (8) represents the tracking error. The second and the third term describe the rate of change of the control input over Hp, while the fourth term represents the control cost. The weights δ, ϕ, and γ were calculated based on simulation tests. min max The constraints on Qair , j and Qair , j are given by: min Qair ,j ≤

∑

j∈{1,2,3,4}

max Qair , j ( k + i − 1| k ) ≤ Qair , j ; i = 1,  , H p

(9)

VOLUME 10,

1.8

1.4

1.6

1.2

Respiration R2 [mg/m3 h]

Respiration R1 [mg/m3 h]

Journal of Automation, Mobile Robotics & Intelligent Systems

1.4 1.2

2016

1

0.8

1 0.8

0.6

0.6

0.4

0.4

0.2

0.2 0

N° 1

0

0.2

0.4

0.6

0.8

1

1.2

Time [day]

1.4

1.6

1.8

0

2

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

1.6

1.8

2

Time [day]

Fig. 8. Respiration in aeration tank 1

Fig. 9. Respiration in aeration tank 2

1.4 0.9 0.8

1

Respiration R4 [mg/m3 h]

Respiration R3 [mg/m3 h]

1.2

0.8 0.6 0.4 0.2 0

0.7 0.6 0.5 0.4 0.3 0.2 0.1

0

0.2

0.4

0.6

0.8

1

1.2

Time [day]

1.4

1.6

1.8

2

0

0

0.2

0.4

0.6

0.8

1

1.2

Time [day]

1.4

Fig. 10. Respiration in aeration tank 3

Fig. 11. Respiration in aeration tank 4

where

The amount and composition of the influent wastewater to WWTP is varied during the day. Their variability is modelled by four parameters (disturbances): inflow Qin, COD, TN and TP. The input disturbances were as follows: Qin (between 2200-3500 m3/h) and COD (between 700-1200 mg/dm3) were time-varying; TN (equal 90 mg/dm3) and TP (equal 10 mg/dm3) were constants over time. Their values and variability correspond to the real values of the influent wastewater for case study WWTP. First, it examined the effect of the length of the prediction horizon Hp and the length of the prediction step T, on control quality and computation time. Control errors were increase by shortening Hp and lengthening T. As a result of numerical analysis, simulation parameters of predictive controllers were as follows: Hp =10 steps and T=5 min. Simulation results are shown for four aeration tanks. Most important parameter disturbing the process of aeration is the respiration R (Figs. 8-11). It refers to the rate of oxygen consumption by bacteria as a result of biochemical reactions. These results show that the respiration disturbance is time-varying, reflecting the varying load of the WWTP (inflow and load). Control results for DO tracking at Kartuzy WWTP are illustrated. The different range of DO changes are set, which corresponds to the optimal conditions of aeration wastewater. In Figs. 12, 14, 16 and 18 the set

min 3 max 3 Qair , j = 0 m h and Qair , j = 5000 m h .

max The constraints on ∆Qair , j are defined as follows:

max  Qair , j (k + i k ) − Qair , j (k + i − 1 k ) ≤ ∆Qair j ∈{ 1,2,3,4} ,j  (10) ;  max i = 1,  , H p  Qair , j (k − 1 k − 1) − Qair , j (k k ) ≤ ∆Qair , j

max 3 where ∆Qair , j = 5000 m h . The nonlinear MPC generates at time instant k, the 4 ref ref control sequence, , Qair , j ( k ) ,..., Qair , j ( k + H p − 1 j =1 based on a discretised nonlinear model (3) with the predicted trajectory of R j ( k ) over k ∈[k , k + H p − 1] (4) by minimizing the performance function (8) with respect to {Qair , j (k ),...., Qair , j (k + H p − 1)}4j =1 subject to the constraints (9)-(10).

{

}

6. Simulation tests and comparative analysis

In this section the proposed two novel control systems (see section 5) were tested by simulation, based on real data records from the case study Kartuzy WWTP. The commercial simulation package Simba [18] was applied to modelling biological processes at a WWTP (ASM2d model). Matlab environment was applied to implementing two advanced control strategies. The Sequential Quadratic Programming (SQP) solver was applied to solve the nonlinear MPC optimisation task.

Articles

13

Journal of Automation, Mobile Robotics & Intelligent Systems

VOLUME 10,

2

DO concentration So,1 [g O2 / m3 ]

DO concentration So,1 [g O 2 / m3 ]

1.6 1.4 1.2

Soref,1

1 0.8 0.6 0.4

S o ,1

0.2 0

0.1

0.2

0.3

0.4 0.5 0.6 Time [day]

0.7

0.8

0.9

1.4 1.2 1

S oref,1

0.8 0.6 0.4

S o ,1 0

0.1

0.2

0.3

0.4 0.5 0.6 Time [day]

0.7

0.4 0.5 0.6 Time [day]

0.7

0.8

0.9

1

Fig. 13. DO concentration in aeration tank 1 – control system 2

2

2 DO concentration So,1 [g O2 / m3 ]

DO concentration So,1 [g O 2 / m3 ]

1.6

0

1

1.8 1.6 1.4 1.2

S

1

ref o ,1

0.8 0.6 0.4

S o ,1

0.2

1.8 1.6 1.4 1.2 1

S oref,1

0.8 0.6 0.4

S o ,1

0.2

0

0.1

0.2

0.3

0.4 0.5 0.6 Time [day]

0.7

0.8

0.9

0

1

0

0.1

0.2

0.3

0.8

0.9

1

Fig. 14. DO concentration in aeration tank 2 – control system 1

Fig. 15. DO concentration in aeration tank 2 – control system 2

point Soref and DO tracking So for control system 1 are presented. In Figs. 13, 15, 17 and 19 the results for control system 2 are illustrated. For both control algorithms was calculated Root Mean Square (RMS) error, given by:

longer to complete required calculations. The control system 2 is characterized by a larger RMS error. For this system, RMS error does not exceed 0.05 in value (see Table 1). Control results could be even better, but not for constraints included in the predictive control systems (see [6]–[7] and [9]–[10]). The average time to solve one predictive optimization task for control system 2 is longer (much larger number of decision variables). However, this time is small enough to carry out the calculation on-line with appropriate prediction horizon, required by the dynamics of the plant and the disturbances rate of change. The main advantage of control system 1 is less computations effort. In addition, in case of failure of one of the controllers the other three controllers can still control the supply of oxygen the individual aerated tanks. In some cases, for example increased inflows of sewage with a high concentration of pollutants, set points trajectories of DO concentration can vary significantly and control system 1 may have a problem with the proper distribution of the required amount of air (it is connected with the fulfillment of constraints, see [6]–[7]). This situation can cause a deviation of the control trajectories. Better quality control for control system 2 takes into account the needs of all aerobic tanks and the possibility of aeration system and therefore is able to find a compromise in the air the division.

(S

RMS =

o

− S oref n

)

2

(11)

where n denotes the number of samples. The control results of RMS error are summarized in Table 1. Table 1. DO tracking error Control system Control system 1

Control system 2

14

1.8

0.2

Fig. 12. DO concentration in aeration tank 1 – control system 1

0

2016

2

1.8

0

N° 1

1

RMS error – aeration tank

0.143

0.044

2

0.110

0.041

3

0.128

0.028

4

0.134

0.049

The control results, for two control systems, show on a good tracking performance by using nonlinear advanced control strategies. It can be seen to follow the DO trajectory with good accuracy. The control system 2 demonstrates better quality control, but takes Articles

Journal of Automation, Mobile Robotics & Intelligent Systems

VOLUME 10,

2

DO concentration So,1 [g O2 / m3 ]

DO concentration So,1 [g O 2 / m3 ]

1.6 1.4 1.2

Soref,1

1 0.8 0.6 0.4

S o ,1

0.2

1.8 1.6 1.4 1.2 1

0

0.1

0.2

0.3

0.4 0.5 0.6 Time [day]

0.7

0.8

0.9

0.6 0.4

0

1

S o ,1 0

0.1

0.2

0.3

0.4 0.5 0.6 Time [day]

0.7

0.4 0.5 0.6 Time [day]

0.7

0.8

0.9

1

Fig. 13. DO concentration in aeration tank 1 – control system 2

2

2 DO concentration So,1 [g O2 / m3 ]

1.8 DO concentration So,1 [g O 2 / m3 ]

S oref,1

0.8

0.2

Fig. 12. DO concentration in aeration tank 1 – control system 1

1.6 1.4 1.2

S

1

ref o ,1

0.8 0.6 0.4

S o ,1

0.2 0

2016

2

1.8

0

N° 1

1.8 1.6 1.4 1.2 1

S oref,1

0.8 0.6 0.4

S o ,1

0.2

0

0.1

0.2

0.3

0.4 0.5 0.6 Time [day]

0.7

0.8

0.9

0

1

Fig. 14. DO concentration in aeration tank 2 – control system 1

AUTHOR Robert Piotrowski – Gdansk University of Technology, Faculty of Electrical and Control Engineering, Narutowicza 11/12, 80-233 Gdansk. E-mail: robert.piotrowski@pg.gda.pl.

[3]

[4]

[5]

[6]

REFERENCES

[1] Åmand L., Olsson G., Carlsson B., “Aeration control – a review”, Water Science & Technology, vol. 67, no. 11, 2013, 2374–2398. DOI: 10.2166/ wst.2013.139. [2] Belchior C.A.C., Araújo R.A.M., Landeck J.A.C., “Dissolved oxygen control of the activated sludge wastewater treatment process using stable

0.1

0.2

0.3

0.8

0.9

1

Fig. 15. DO concentration in aeration tank 2 – control system 2

7. Conclusions Control of dissolved oxygen at a WWTP is important for economic and process reasons. The paper has addressed an important and difficult control problem. A novel approach to the dissolved oxygen concentration tracking has been presented. Two nonlinear predictive control systems have been designed and compared. Its properties and tracking performance have been investigated by simulation based on real data sets from Kartuzy case study plant. Promising results have been observed.

0

[7]

adaptive fuzzy control”, Computers and Chemical Engineering, vol. 37, no. 10, 2012, 152–162. DOI: 10.1016/j.compchemeng.2011.09.011. Błaszkiewicz K., Piotrowski R., Duzinkiewicz K., “A Model-Based Improved Control of Dissolved Oxygen Concentration in Sequencing Wastewater Batch Reactor”, Studies in Informatics and Control, vol. 23, no 4, 2014, 323–332. Chotkowski W., Brdys M.A., Konarczak K., “Dissolved oxygen control for activated sludge processes”, International Journal of Systems Science, vol. 36, no. 12, 2005, 727–736. DOI: 10.1080/00207720500218866. Cristea S., de Prada C., Sarabia D., Gutiérrez G., “Aeration control of a wastewater treatment plant using hybrid NMPC”, Computers and Chemical Engineering, vol. 35, no. 4, 2011, 638–650. DOI: 10.1016/j.compchemeng.2010.07.021. Duzinkiewicz K., Brdys M.A., Kurek W., Piotrowski R., “Genetic hybrid predictive controller for optimised dissolved oxygen tracking at lower control level”, IEEE Transactions on Control Systems Technology, vol. 17, no. 5, 2009, 1183-1192. DOI: 10.1109/TCST.2008.2004499. Han H.-G., Qiao J.-F., Chen Q.-L., “Model predictive control of dissolved oxygen concentration based on a self-organizing RBF neural network”, Control Engineering Practice, vol. 20, no. 4, 2012, 465– 476. DOI: 10.1016/j.conengprac.2012.01.001. Articles

15

Journal of Automation, Mobile Robotics & Intelligent Systems

16

[8] Hauduc H., Rieger L., Oehmen A., van Loosdrecht M.C.M., Comeau Y., Héduit A., Vanrolleghem P.A., Gillot S. (2013). “Critical review of activated sludge modeling: state of process knowledge, modeling concepts, and limitations. Biotechnology and Bioengineering 110(1), 24-46. DOI: 10.1002/bit.24624. [9] Henze M., Gujer W., Mino T., Matsuo T., Wentzel M.C., Marais G.v.R., van Loosdrecht M.C.M., “Activated Sludge Model no. 2d, ASM2d, Water Science & Technology, vol. 39, no. 1, 1999, 165–182. DOI: 10.1016/S0273-1223(98)00829-4. [10] Henze M., Gujer W., Mino T., van Loosdrecht M.C.M., Activated sludge models ASM1, ASM2, ASM2d and ASM3. Scientific and Technical Report no. 9, IAWQ, London 2000. [11] Holenda B., Domokos E., Rédey A., Fazakas J., “Aeration optimization of a wastewater treatment plant using genetic algorithm”, Optimal Control Applications and Methods, vol. 28, no. 3, 2007, 191–208. DOI: 10.1002/oca.796. [12] Holenda B., Domokos E., Rédey A., Fazakas J., “Dissolved oxygen control of the activated sludge wastewater treatment process using model predictive control”, Computers and Chemical Engineering, vol. 32, no. 6, 2008, 1270–1278. DOI: 10.1016/j.compchemeng.2007.06.008. [13] Keskar P., “Control and optimization of unit operations – aeration and DO controls” [in:] Instrument Engineers’ Handbook – Process Control and Optimization, B.G. Lipták (ed.). CRC Press, Boca Raton, 2006, 1484–1506. [14] Lindberg C. F., Control and estimation strategies applied to the activated sludge process. PhD thesis 1997, Uppsala University, Sweden. [15] Piotrowski R., “ Two-Level Multivariable Control System of Dissolved Oxygen Tracking and Aeration System for Activated Sludge Processes”, Water Environment Research, vol. 87, no. 1, 2015, 3–13. DOI: http://dx.doi.org/10.2175/1061430 14X14062131178916. [16] Piotrowski R., Brdys M.A., Konarczak K., Duzinkiewicz K., Chotkowski W., “Hierarchical dissolved oxygen control for activated sludge processes”, Control Engineering Practice, vol. 16, no. 1, 2008, 114–131. DOI: 10.1016/j.conengprac.2007.04.005. [17] Rieger L., Jones R.M., Dold P.L., Bott Ch.B., “Ammonia-Based Feedforward and Feedback Aeration Control in Activated Sludge Processes”, Water Environment Research, vol. 86, no. 1, 2014, 63–73. DOI: 10.2175/106143013X1359652451 6987. [18] Simba,User’s guide, 2015. Available: http:// www.ifak-system.com/en/environmental -simulation/simba/. [19] Suchodolski T., Brdys M.A., “Piotrowski R., Respiration rate estimation for model predictive control of dissolved oxygen in wastewater treatment plant”. In: The 11th IFAC Symposium on Large Scale Systems: Theory and Applications, Gdańsk, July 23–25, 2007, Poland, IFAC Publisher, Elsevier, vol. 11, no. 1, 2008, Austria – Laxenburg. Articles

VOLUME 10,

N° 1

2016

[20] Tatjewski P., Advanced control of industrial processes: structures and algorithms, Springer-Verlag, London 2007. [21] Vrečko D., Hvala N., Stražar M., “The application of model predictive control of ammonia nitrogen in an activated sludge process“, Water Science & Technology, vol. 64, no. 5, 2011, 1115–1121. DOI: 10.2166/wst.2011.477. [22] Wahab N.A., Katebi R., Balderud J., “Multivariable PID control design for activated sludge process with nitrification and denitrification”, Biochemical Engineering Journal, vol. 45, no. 3, 2009, 239– 248. DOI: 10.1016/j.bej.2009.04.016. [23] Zubowicz T., Brdys M.A., Piotrowski R., “Intelligent PI controller and its application to dissolved oxygen tracking problem”, Journal of Automation, Mobile Robotics & Intelligent Systems vol. 4, no. 3, 2010, 16–24.

Journal of Automation, Mobile Robotics & Intelligent Systems

VOLUME 10,

N° 1

2016

Mathematical Modelling and Feasibility Study of the Biomimetic Undulating Fin of a Knife Fish Robot Submitted: 27th August 2015; accepted 9th November 2015

Ajith Anil Meera, Attadappa Puthanveetil Sudheer DOI: 10.14313/JAMRIS_1-2016/3 Abstract: Bio-mimetic underwater robotics is an emerging area of research, which has the potential to substitute the conventional energy inefficient mode of underwater propulsion using thrusters. In this paper, the mathematical modelling of the undulating fin is done and the effect of various parameters of the mechanism design on the available workspace is studied. The mathematical beauty is revealed, for the curves representing the mechanical constraint and the family of undulating waves. The feasibility of a wave to be generated by the mechanism was analyzed. Keywords: biomimetic design, propulsion, fins, mechanism, underwater robotics, five bar mechanism, fish robot

1. Introduction Nature provides us the best and robust solutions for underwater propulsion through living organisms under water subjected to evolutionary optimization. This notion has made man enthusiastic about observing the marine life and replicating it in the underwater vehicles. This has opened up a new realm for research called bio-mimetic underwater robotics. As a result, wide range of underwater robots were designed and tested for better propulsive efficiency around the globe. Nanyang Technological University’s NKF-1 Knife fish robot, Squid type vehicle from Osaka University, MIT’s Robotuna, Festo’s Airacuda and Stingray robot are some of them. Number of papers have been published based on these works. The experimental results and conclusions of the swimming knife fish robot with undulating fin is dealt in [1], while the hydrodynamic analysis and forward thrust estimation for the robot propelled by two undulating side fins is discussed in [2]. The empirical results of braking performance of the two undulating side fin robot is done in [3], while the efficiency of usable area of a mechanism was quantified in [4]. The modelling of the motion of a biomimetic underwater vehicle was done in [5]. Developments in the field of experimental biology have expedited the growth of bio-mimetic robots. The experimental results for hovering and swimming of electric ghost knife fish, which could be used to mimic the real fishes is done in [6]. The disturbances and fluid patterns caused by the undulating fin under

water is discussed in [7] and [8]. The role of counter propagation waves used by fishes for propulsion and maneuverability is revealed in [9] and [10]. However, the above literature does not explore the modelling and parametric study of the fin mechanism in depth. In this paper, the mathematical model of the fin mechanism is formulated and the feasibility of the waves to be generated by the mechanism is studied. The robot was tested for different waveforms and was found to make a smooth and eco-friendly swimming along with the real fishes.

2. Mechanism

The fin mechanism consist of a series of five bar mechanism with the non-crank member as a slider or a flexible membrane as introduced in [1].

2.1. Kinematic Design

The mechanism to generate a sinusoidal profile on the fin consists of cranks of length R placed at a fixed distance L between each other.

Fig. 1. Mechanism for sinusoidal wave generation In Fig. 1, A and D represent the servo heads of the adjacent servo motors rotating about an axis perpendicular to the plane of the paper. Each of the cranks are made to move in a circular arc with the respective servo head as the center, as given in [4]. Intervals AB and DC represent the cranks of the servo motors and BC represents a mechanical slider. The angle q made by the crank with the horizontal varies sinusoidal with respect to time. The phase difference between the adjacent crank angles is fixed as b. N number of servo motors are mounted with all the servo heads lying on the line joining A and D together creating an undulation on the fin. The equation of motion for ith crank is described by (1), where qm represents the maximum inclination of the cranks with the horizontal, f the frequency and t the time.

(1)

17

Journal of Automation, Mobile Robotics & Intelligent Systems

Fig. 2. Sinusoidal wave made by fin with N = 9, θm = 45⁰ and β = 45⁰ at t = 0 s Fig. 2 represents the sketch of the sine wave generation by the fin mechanism, while Fig. 3 shows the undulating fin made using a stretchable membrane.

Fig. 3. Undulating fin with a flexible membrane

2.2 Mechanical Constraints: The distance between the adjacent crank ends varies with respect to time. If the distance between the crank ends is less than a minimum value Cmin, the slider hits. If the distance is more than the maximum value Cmax, the slider detaches. Due to this mechanical constraint offered by the sliders, the distance between the crank ends are restricted to be within a range. The minimum and maximum stretchable lengths represents the mechanical constraint for a flexible membrane.

3. Parametric Study on Workspace:

The effect of parameter selection of the mechanism on the available workspace is studied in this section. The interdependency of the parameters is studied so that the mechanism can generate the sine waves.

3.1. Curves of Fixed Link Length

18

The workspace of the adjacent crank angle is a property of the mechanical design of the fin. The parameters affecting the workspace design are L, R and Cmin and Cmax. Proper selection of these parameters is essential to optimize the workspace. Since the servo motors are able to deliver –90° to 90° rotations with position control, the workspace design was done within a domain of –  £ qi £  and –  £ qi+1 £  only. Consider a five bar linkage represented by Fig. 1. with equal crank lengths and let the coupler BC be rigid with a fixed length C. From Fig. 1. the position of B at any time relative to A is (R cos qi, R sin qi) and the position of C relative to A is (L + R cos qi+1, R sin qi+1). Evaluating the distance BC and equating it to C yields the expression given by (2), which on further simplification yields (3). Articles

N° 1

2016

(2)

(3)

3.1.1. Taylor Series Approximation: The Taylor series approximation was done on equation (3) and the powers higher of θi and θi+1 were neglected, resulting in equation (4). (4)

The error of approximation is dependent on L/R and independent of C. For different values of L/R, workspace plot for both (3) and (4) were plotted and was observed that the curves representing (4) completely lies inside the curves represented by (3) in the domain selected for analysis. 1.5

Original Curve Approximate Curve

1 0.5

θi+1

The fin is represented by the line joining the ends of the cranks. These lines represent the side view of the plane of the sliders kept between the crank ends. The ends of the cranks together impart a sinusoidal waveform to the fin. Hence each of the sliders kept between the crank ends, together form an approximate sine wave. The length of each slider varies as per the motion of the corresponding adjacent cranks.

VOLUME 10,

0

−0.5 −1 −1.5 −1

0

θi

1

Fig. 4. Plot for L=5, R=3 and C=6 Moreover, the error was found to increase as the curve passes through points with either of the crank angles closer to 90° as shown in Fig. 4 (axes measured in radians). However, the approximated curves represent the original curve with a small error for the regions closer to the origin. Since the approximated curves are always found to be inside the original curves, no point on the curve represented by equation (4) would violate the original mechanical constraint. Hence, replacing the original equation with an approximated equation is equivalent to offering more constraints to the workspace, without violating the original constraint at any point. The advantage of neglecting the higher order terms in the expansion of the cosine series is that it leads to a simple general form of a second degree polynomial curve with properties that can easily be analyzed through mathematical modelling. Hence, throughout the paper, the equation governing the adjacent crank angles was selected as given by equation (4).

Journal of Automation, Mobile Robotics & Intelligent Systems

VOLUME 10,

3.1.2. Nature of the Curves The curve represented by equation (4) is that of a general second degree equation of the form given by (5).

Hence, the standard equation of the hyperbola is obtained after transformation as given by equation (6) for C > L and equation (7) for C < L.

and D = h2 – ab.

(6)

D= > 0 and for (4). Hence, it can be concluded that if L = C, the equation (4) represents a pair of straight lines and if L ¹ C it represents the equation of a hyperbola.

3.1.3. Standard Form of Hyperbola The equation (4) represents a family of hyperbola inclined to the axes for L ¹ C. To formulate the standard equation of the hyperbola, the axes are rotated about an axis perpendicular to the paper such that the transformed axis is coincident with the major axis of the hyperbola, as shown in Fig. 5.

(7)

The hyperbolas represented by C > L and C < L have mutually perpendicular axis. Moreover, the curve represent a pair of straight lines at C = L. The inclination of the transverse axis of the hyperbola with respect to the θi axis is represented by α, given by (8), which is dependent on L/R and is independent of C. The conjugate axis of the hyperbola is perpendicular to the transverse axis obtained. (8)

3.1.4. Asymptotes of the Hyperbola The asymptotes of the hyperbola play a crucial role in the workspace design of the adjacent cranks. The equation representing the pair of asymptotes is given by (9).

Fig. 5. Rotation about z axis The transformation matrix for rotation about z-axis is represented by

2016

(5)

To determine the nature of the curve, two parameters D and D have to be evaluated, where D=

N° 1

(9)

.

Hence, it can be concluded that the asymptotes are dependent on L/R and is independent of C. Moreover, is an asymptote which is independent of the

Hence the transformation is represented by

1.5 1

θi+1

0.5 0

−0.5 −1 −1.5

(a)

−1

1

(b)

(c)

Fig. 6. Pair of asymptotes for L/R = 1.5, L/R = 1, L/R = 0.5, respectively Articles

19

Journal of Automation, Mobile Robotics & Intelligent Systems

VOLUME 10,

link lengths of the mechanism. The other asymptote is = 0 for L = R. The asymptote has a negative slope for L > R and a positive slope for L < R. Both the asymptotes pass through the origin for all cases. This result is illustrated in Fig. 6. The axes of all the graphs in the paper are measured in radians.

N° 1

2016

It can be concluded that the area is defined by the region bounded by the boundary curves represented by (10) and (11). Hence, the workspace of the adjacent crank angles is defined as the area bounded by and the boundary curves in the domain . It represents the region in which the design of the lengths of the linkages constrain the motion of the adjacent crank angles. Increase in the workspace corresponds to a lesser constrain over the adjacent crank angles. Therefore, in order to increase the flexibility in the motion of the adjacent cranks, the workspace has to be optimized.

3.2. Workspace for Variable Link Length

Equation (4) represents the variation of the adjacent crank angles with equal crank lengths constrained to move with all the links of fixed length. Considering the non-crank member to be a membrane with , and plotting all the intermediate curves yields a region corresponding to the workspace of the adjacent crank angles. The workspace is represented by an area bounded by the curves represented by equations (10) and (11) and is equivalent to relation (12).

1.5

1.5

1

1

1

0.5

0.5

0.5

0

θi+1

1.5

θi+1

θi+1

3.2.1. Parametric Study for the Workspace It can be observed from (4) that the equation of the hyperbola defining the motion of the 4 bar linkage depends on two ratios, L/R and C/R. This implies that the hyperbola remains the same even if the mechanism is scaled up or down. This result allows to scale (10) the mechanism in order to increase or decrease the size of the robot. Parametric study for the hyperbola therefore constitutes a methodology involving the two ratios. Dif ferent cases of the parametric study are explored in which L/R is fixed and then C is varied covering all the (11) possible ratio ranges. In the study, the L/R ratio is chosen to be either less than 1 or greater than 1. A feasible value for L and R is selected randomly such that the L/R falls in the selected range. The value of C is varied from a value less than min (L, R) to a value more than max (12) (L, R). This results in three different cases of the range

0

−0.5

−0.5 −1

−1

−1

−1.5

−1.5

−1.5

−1

0

θi

1

−1

(a)

0

θi

1

1.5

1.5

1

1

1

0.5

0.5

0.5

θi+1

1.5

0

−0.5

−0.5

−0.5

−1

−1

−1

−1.5

−1.5

0

1

−1

Fig. 7. Effect of parameter selection on workspace for L≠R Articles

0

1

0

θi

1

(c)

0

0

−1

−1

(b)

(d)

20

0

−0.5

−1.5

(e)

−1

0

θi (f)

1

Journal of Automation, Mobile Robotics & Intelligent Systems

of C, excluding the condition of equality of C with L or R. The workspace plot for all the three cases were plotted. The three cases were repeated for the other range of L/R ratio. Fig. 7(a, b, c) represents the 3 cases of the workspace for . The parameters chosen were, L = 3, R= 4, Cmin = 2 and Cmax = 5. Fig. 7a: Cmin < C < L, Fig. 7b: L < C < R, and Fig 7c: R < C < Cmax . The asymptotes of the hyperbolas can be observed to be as in equation (8), and is independent of C. Both the asymptotes have a positive slope. Fig. 7 (d, e, f) represents the three cases of the workspace for > 1. The parameters chosen were, R = 3, L = 4, Cmin = 2 and Cmax = 5. Fig. 7d: Cmin < C < R, Fig. 7e: R < C < L, Fig. 7f: L < C < Cmax. The asymptotes of the hyperbolas can be observed to be as in equation (8), and is independent of C. One asymptote has a negative slope. Fig. 7 represents the effect of selection of the slider properties on the total workspace of the mechanism for a chosen L and R. Therefore, the slider parameters have to be chosen such that the workspace is optimum. The transverse axis of the hyperbola changes beyond C = L, i.e. after Fig. 7a for L < R and after Fig. 7e for L > R. It is in agreement with the results of the section 3.1.3. This property of the curve is crucial in the slider selection in the later section. The equality of C and R does not serve any characteristic property changes in the curves, while the equality of L and R give rise to different properties for C < L and C > L. Fig. 8 represents the two cases of the workspace for L = R, C < L and C > L, respectively. The parameters

1

θi+1

0.5 0

−0.5 −1 −1

0

θi

1

(a)

1.5 1

θi+1

0.5 0

−0.5 −1 −1.5

N° 1

2016

selected were L = R = 3. The asymptotes are in agreement with the results of 3.1.3. Hence, it can be concluded that the asymptotes corresponding to the ratio divide the entire domain

and

into two regions,

 < 1 and  > 1. The characteristic of the hyperbola in a region is its axis. The two regions have hyperbolas with axes mutually perpendicular to each other. The conjugate axis of hyperbolas in a region acts as the transverse axis in the other region and vice versa. The L/R ratio decides the inclination a of the transverse axis with the qi axis as discussed in 3.1.3. The region corresponding to a slider can fall in any of the two regions or both. Hence the selection of Cmin and Cmax determines the region in which the family of hyperbolas might belong.

4. Motion Planning for Undulation of Fins

The knife fish robot propels under water by creating an undulation of the anal fin through the controlled and combined motion of the servo motors kept in series along the fish body. The robot propagates through the generation of propagating sinusoidal waves of constant or linearly increasing amplitudes.

4.1. Constant Amplitude

Consider the motion of the ith and i + 1th crank as described in the section 2.1.

(13)

(14)

Where đ?‘– = 1, 2, 3,‌.đ?‘ and N is the number of servo motors used. Simplifying the equation of motion of the adjacent crank angles, (13) and (14) yields (15). (15)

1.5

−1.5

VOLUME 10,

−1

0

θi

1

(b)

Fig. 8. Effect of parameter selection on workspace for L=R

Therefore, a single equation (15) describes the relationship between any two adjacent crank angles in the mechanism. It can be noted that the curve is independent of the frequency of oscillation of the fin and i. Hence, for a constant qm and b, the same curve represents the joint angle trajectories of all the adjacent cranks.

4.1.1. Nature of the Curve The equation (15) is that of a general second degree equation as represented by (5). Here D = –qm2 (sin b)2 šâ€‰0 and D = –(sin b)2 < 0. Hence, it can be concluded that (15) represents the equation of an ellipse if b šâ€‰0, p and it represents a pair of straight lines if b =0 or p. 4.1.2. Standard Form of Ellipse Equation (15) represents a family of ellipses in  (b) clined to the axes for b šâ€‰0, p. Following the sim- ilar steps as discussed in 3.1.3, the standard equation of the ellipse is derived for and , as given by (16) and (17), respectively. Articles

21

Journal of Automation, Mobile Robotics & Intelligent Systems

VOLUME 10,

(16)

(17)

Equation (16) represents the standard form of the ellipse with its major axis inclined at 45â ° to the θi axis, and equation (17) corresponds to an ellipse with its major axis inclined at –45â ° to the θi axis. The axes of the family of ellipses are the same, except that the major axis of (16) is the minor axis of (17) and vice versa. The ellipse becomes a circle for .

N° 1

2016

(19)

The points on the ellipse having tangents parallel to the qi axis are (qmcos b, qm) and (–qmcos b, –qm) is evaluated from the slope of the ellipse given by (19). Hence, a common tangent to all the ellipse exists, which is parallel to the qi axis. The common tangent is independent of b and is represented as qi+1= qm and qi+1= –qm. Similarly, the points on the ellipse having tangents perpendicular to the θi axis are (qm, qmcos b) and (–qm, –qmcos b). Therefore, the common tangent is qi = qm and qi = –qm and is independent of b. Hence, it can be concluded that all the ellipses have four common tangents which is independent of b. Therefore, the family of ellipses are bounded inside a square of length 2qm with (0, 0) as the centre. Fig. 9. and Fig. 10. demonstrate the result.

Fig. 10. Joint trajectories for various β Fig. 9. Bounding square for constant amplitude wave 4.1.3. Area Enclosed by the Ellipse The area enclosed by an ellipse is A = pab and is given by (18), where a and b are the half lengths of major and minor axis of the ellipse.

22

(18)

4.1.4. Bounding Square Sum of squares of major and minor axis of the ellipse represented by (16) yield a constant value of 8qm2 which is independent of b. Therefore, it can be concluded that for a constant qm the lengths of the major and minor axis of the ellipse lies on a circle with radius . This special property of the family of ellipses bounds the length of major and minor axes of the ellipses. The result can be visualized as a growing ellipse with major axis inclined 45â ° to the horizontal. As the b increases, the length of major axis decreases and the length of minor axis increases such that both fall on the circle. To inspect on the bounding nature of all the ellipses, the tangent to the family of ellipses were studied. Articles

4.2 Linearly Increasing Amplitude Consider the fin to be operated with a linearly increasing wave described for the ith and (i+1)th cranks as described by (20) and (21).

(20)

(21)

Where đ?‘–=1, 2, 3,‌đ?‘ and N is the number of servo motors used. On simplification, it yields equation (22) that describes the joint angle trajectories.

(22)

It represents a family of ellipses and, unlike in the case of constant amplitude wave generation, the equation of the curve is dependent on i. Hence in order to represent the joint angle trajectories of the curves of a linearly increasing amplitude wave, N–1 number of equations are necessary, as shown in Fig. 12.

Journal of Automation, Mobile Robotics & Intelligent Systems

VOLUME 10,

4.2.1. Nature of the Curve The equation (22) is that of a general second degree equation as represented by (5). Hence, Δ and D are evaluated as and

D =   . Hence, equation (22) represents an ellipse if β ≠ 0, π and a pair of straight lines if β = 0 or π.

N° 1

2016

a of the minor axis of the ellipse with the θi axis is given by (25).

(25)

Unlike the case for constant amplitude, the inclination a is not a constant, but is dependent on і and b. The minor axis is along the θi axis for b = 0. The inclination of the ellipse goes on decreasing as і increases for a constant b. 4.2.3. Area Enclosed by the Ellipse The area enclosed by the ellipse is given by equation (26).

A = i(i + 1)pqm2 sin b (26)

Fig. 11. Bounding rectangle for i=5 4.2.2. Standard Form of the Ellipse The conversion formula for the ellipses of the form ax2+ 2hxy + by2 +c = 0, into the general form is given by (23), where k2 = 4h2 + (a – b)2.

(23)

Hence the standard form of the ellipse representing linearly increasing amplitude is given by (24), where

.

(24)

Unlike the case of a constant amplitude, the lengths of both major and minor axis are dependent on i and b as demonstrated in Fig. 11. The inclination

Fig. 12. Joint trajectories for linearly increasing amplitude wave for N=7

4.2.4. Bounding Rectangle The sum of the squares of major and minor axis of the ellipse represented by (24), is a constant for a particular and is independent of b. Hence, the ellipses are bounded for a particular i, similar to the constant amplitude case. The boundaries are defined by the i value and the boundaries increases as i increases. The points on the ellipse parallel to the θi axis are obtained as (iqm cos b, θm (i + 1)) and (–iqm cos b, –θm (i + 1)), from the slope of the ellipse represented by equation (27).

(27)

Hence the common tangents are qi+1= qm (i + 1) and qi+1= –qm (i + 1) and are independent of b. Similarly, the points on the ellipse where the tangents are perpendicular to the θi axis are (iqm, (i + 1)qm cos b) and (– iqm, –(i + 1)qm cos b). Hence, the common tangents are qi = iqm and qi = –iqm which are independent of b. All the four common tangents to the ellipse are independent of b and dependent on i. Therefore, the family of ellipses are bound inside a rectangle when a pair of adjacent cranks are considered, as illustrated in Fig. 11. As the adjacent cranks corresponding to higher amplitude portion of the wave is considered, the size of the bounding rectangle goes on increasing. The size of the bounding rectangle is 2iqm by 2(i + 1)qm centred at origin as shown in Fig. 13.

Fig. 13. Increase of size of bounding rectangle with i Articles

23

Journal of Automation, Mobile Robotics & Intelligent Systems

5. Feasibility of an Undulation by the Mechanism Section 3 discusses about the workspace created by a mechanism, while section 4 discusses about the joint trajectory of an undulation. An undulation is said to be feasible only if the joint trajectory completely falls inside the workspace described by the mechanism in [1]. If any portion of the joint trajectory falls out of the workspace, the undulation will remain incomplete in those regions. Therefore, to produce an undulation, the ellipse must be in the region bounded by the Cmin and Cmax hyperbolas. For the feasibility of a linearly increasing amplitude wave, the outermost ellipse has to be inside the workspace. Hence, for both waves, the ellipse must be inside the hyperbolas completely. The hydrodynamic effects are not considered in this study.

5.1. Parameters of the Sliders

The minimum and maximum lengths attainable by the slider plays a crucial role in workspace design. Consider that a workspace similar to any one of that of Fig. 7 was chosen, then it is impossible to make an ellipse centered at origin and of any b and qm to be in the workspace. Some part of the ellipse will always be outside the workspace. Therefore, a combination of these workspaces have to be adopted. The workspace will accommodate an ellipse only if Cmin < L and Cmax > L. The length of the slider should be such that it is able to take a length less than L, then extend and cross the length L. For the workspace of Fig. 14, the undulation is impossible. Choosing a slider that can extend to 7 units as shown in Fig. 15, solved the problem.

VOLUME 10,

N° 1

2016

5.2. Confirmation of Ellipse Inside Hyperbolas To have an undulation feasible by the mechanism, any point on the ellipse has to yield a positive sign with the Cmin hyperbola and a negative sign with the Cmax hyperbola, given that Cmin < L < Cmax. Solving equations (12), (13) and (14) for i = 1, yield (28) for all values of t.

(28)

The equality condition of the equation (28) yields the intersection point of the ellipse and the hyperbola. Hence, the condition (28) should be checked prior to the wave generation using the mechanism.

5.3. Optimal and Feasible Workspace Design

To design an optimal mechanism that can generate a given wave, it is necessary to find the hyperbola touching the ellipse. This optimal solution will ensure that the capability of the membrane to stretch, is fully utilized in generating the wave as shown in Fig. 16. It eliminates the wastage in useful workspace by finding the best fitting hyperbola. The objective function for workspace optimization of a given wave is the minimum distance between the ellipse and hyperbola, which is to be minimized. Since both the curves are inclined with respect to the horizontal at different angles, both the equations (4) and (15) are difficult to solve. It urges the necessity for an optimization algorithm to solve the equations.

Fig. 14. L = 5, R = 5, Cmin = 3, Cmax = 5, β = 45⁰, qm = 45⁰

Fig. 16. Optimal mechanism design for β =45⁰, qm =45⁰

6. Conclusion and Outlook

Fig. 15. L = 5, R = 5, Cmin = 3, Cmax = 7, β = 45⁰, qm = 45⁰ 24

Articles

The mathematical modelling of the fins subjected to the mechanical constraint offered by the sliders was done. The effect of the parameter selection of the fin mechanism on the total available workspace was studied in light of the types of undulations that the robot is supposed to take. The feasibility of an undulation by the fin is modelled mathematically. The nature

Journal of Automation, Mobile Robotics & Intelligent Systems

of the curves representing the constraints and the undulation were studied for optimizing the available workspace. The analysis begins with the Taylor series approximation for simplification of the equation representing the mechanical constraint offered by the sliders. Error due to this approximation remains throughout the analysis. The feasibility study was done without considering any hydrodynamic effects. An algorithm to optimize the objective function is to be proposed to reduce the computational complexity in solving the two second degree equations, with both the curves touching each other. The efficiency of the fin mechanism to incorporate more number of undulations should be defined such that it precisely reflects the robot’s maneuverability. The wave corresponding to the maximum thrust generation is to be found and incorporated in the workspace. Energy efficiency of the fin is to be evaluated to check for its better performance than thrusters. Further, this technology can be incorporated in the marine drives, as it is expected to be more energy efficient than the thrusters. Future underwater robots can be made with Median Paired Fin Propulsion System, and can be employed for oceanographic researches, underwater surveillance, deep sea mining, swarm robotics etc.

ACKNOWLEDGMENT

This research was supported by National Institute of Technology Calicut, Kerala, India by providing the laboratory facilities and funds for fabrication and experimentation of the robot. This support is gratefully acknowledged.

AUTHORS

Ajith Anil Meera* – Robotics Interest Group, Department of Mechanical Engineering, National Institute of Technology, Calicut, Kerala, India-673601. Tel. +918547267258. E-mail: ajitham1994@gmail.com.

VOLUME 10,

N° 1

2016

[3] Sugimori S., Miki H., Yamamoto R., et al., “Braking Performance of a Biomimetic Squid-Like Underwater Robot”, Japan Journal of Bionic Engineering, 2013, vol. 10, no. 3, 265–273. DOI: 10.1016/ S1672-6529(13)60222-X. [4] Low K.H., “Modelling and parametric study of modular undulating fin rays”, Mechanism and Machine Theory, 2009, vol. 44, no. 3, 615–632. DOI: 10.1109/ICMA.2007.4303527. [5] Morawski M., Malec M., Szymak P., Trzmiel A., “Analysis of Parameters of Traveling Wave Impact on the Speed of Biomimetic Underwater Vehicle”, Solid State Phenomena, vol. 210, 273–279, Oct. 2013, DOI: 10.4028/www.scientific.net/ SSP.210.273. [6] Curet O. M., Lauder G.V., MacIver M.A., Ruiz-Torres R., “Kinematics of the ribbon fin in hovering and swimming of the electric ghost knifefish”, Journal of Experimental Biology, 2012, vol. 216, no. 5, 823–834. DOI: 10.1242/jeb.076471. [7] Bale R., Pal A., Bhalla S., et al., “Undulating fins produce off-axis thrust and flow structures”, Journal of Experimental Biology, 2014, vol. 217, no. 2, pp. 201–213. DOI: 10.1242/jeb.091520 [8] Curet O.M., Neelesh A. Patankar, and M.A. MacIver Anup A. Shirgaonkar, “The hydrodynamics of ribbon-fin propulsion during impulsive motion”, Journal of Experimental Biology, 2008, vol. 211, no. 21, pp. 3490–3503. DOI: 10.1242/ jeb.019224. [9] Patankar N.A., Lauder G.V., MacIver M.A., Curet O.M., “Aquatic manoeuvering with counter-propagating waves: a novel locomotive strategy”, Journal of Royal Society Interface, 2010, vol. 8, no. 60, 1041–1050. DOI: 10.1098/rsif.2010.0493 [10] Nevelnb I.D., Rothc E., Mitchelld T.R.T., et al., “Mutually opposing forces during locomotion can eliminate the tradeoff between maneuverability and stability”, PNAS, 2013, vol. 110, no. 47, 18798–18803. DOI: 10.1073/pnas.1309300110.

Attadappa Puthanveetil Sudheer – Department of Mechanical Engineering Robotics\Mechatronics Lab National Institute of Technology, Calicut Kerala, India-673601 Tel. +919961450987 E-mail: apsudheer@nitc.ac.in *Corresponding author

REFERENCES

[1] Low K.H, Willy A., “Biomimetic Motion Planning of an Undulating Robotic Fish Fin”, Journal of Vibration and Control, 2006, vol. 12, no. 12, 1337– 1359. DOI: 10.1177/1077546306070597. [2] Wang S., Dong X., Shang L.-J., et al., “Design and Kinetic Analysis of a Biomimetic Underwater Vehicle with Two Undulating Long-fins”, Acta Automatica Sinica, 2013, vol. 39, no. 8, 1330–1338. DOI: 10.1016/S1874-1029(13)60049-X.

Articles

25

Journal of Automation, Mobile Robotics & Intelligent Systems

VOLUME 10,

N° 1

2016

Design Optimization of the Biomimetic Undulating Fin of a Knife Fish Robot Submitted: 27th August 2015; accepted 9th November 2015

Ajith Anil Meera, Attadappa Puthanveetil Sudheer DOI: 10.14313/JAMRIS_1-2016/4 Abstract: Median Paired Fin Propulsion used by fishes like knife fish, has the potential to replace current energy inefficient underwater propulsion systems. This paper quantizes the efficiency of a fin mechanism based on its ability to incorporate a large number of undulations. Characteristic ellipses of a mechanism were defined, based on which two algorithms were defined to measure the capabilities of the mechanism. The available workspace of the mechanism was optimized to accommodate the undulation requirements of the robot based on the formulations. Further, the distortion effect on the waveform was identified and the optimization was redefined to obviate its possibility while in operation. Keywords: mechanism, underwater robot, propulsion, optimization, biomimetic design, fish robot

26

robot with two undulating fins by [7] and mechanical properties of the undulating proposer by [8]. The computational fluid dynamic analysis of the fish robot was done in [9], while [10] quantified the thrust efficiency of the robotic fish. The modelling of the motion of a biomimetic underwater vehicle was done in [11]. The potential of the MPF mode of propulsion system is immense in underwater locomotion, which makes it necessary to optimize the design. This work mainly focusses on the design optimization of the undulating fin so that the fin is capable to generate more number of undulations without any wastage in membrane extension. This work quantify the efficiency of the fin mechanism in generating more waves.

2. Mechanism

The mechanism to generate sinusoidal waveform on the fin membrane consists of a series of five bar mechanism with a flexible coupler, arranged adjacent to each other as proposed in [4].

1. Introduction

2.1. Kinematic Design

The propulsion systems adopted by fishes and other underwater organisms might not be optimal, but they have evolved over a long period, improving and adapting on the energy utilization for locomotion. Median Paired Fin (MPF) propulsion system used in knife fish, cuttlefish etc. is one such propulsion in which the fish generates a sinusoidal waveform on its fins creating forward thrust for locomotion. Researchers have identified the potential of such systems as a replacement for the contemporary thruster based propulsion systems which are energy inefficient in nature. Underwater robotics is a well explored field of research, while the bio-inspired underwater robotics is a less explored one. Researches have led to the development of a variety of underwater robots that emulate a real fish. The fish robot named RoboTuna, replicating the real fish with caudal and pectoral fins was developed by Barrett [1]. A soft robotic fish was developed by MIT [2] that works with fluidic elastomer actuators. Nanyang Technological University’s Knife fish robot, Squid type vehicle from Osaka University, Festo’s Airacuda and Stingray robot are some other examples of such robots. The research on MPF propulsion robot include modelling and parametric study on knife fish robot by [3] and [4], kinematic modelling and dynamic analysis on the undulating fin by [5], braking performance of the squid like robot by [6], kinematic analysis of

Fig. 1 represents the mechanism, where A and D denote the center of the adjacent servo heads, BC represents the slider or flexible membrane, AB and DC represents the cranks. A number of servo motors are arranged in the line joining A and D such that the distance between the adjacent servo heads is L. The crank length is denoted by R and is same for all servos. The minimum and maximum lengths of the slider or flexible membrane is denoted by Cmin and Cmax.

Fig. 1. Mechanism to generate sinusoidal waves

2.2. Workspace The flexible membrane in the five bar mechanism allows the cranks to rotate independent of each other within a certain limit. The restriction over the independent movement of the cranks depends on the allowable extension of the membrane. The membrane begins to slack once the distance BC is less than Cmin. Similarly, the membrane begins to break once the dis-

Journal of Automation, Mobile Robotics & Intelligent Systems

VOLUME 10,

tance BC is more than Cmax. Therefore, there exists an available workspace for the fin mechanism as given in [3]. The boundary of the workspace is represented by two hyperbolas, one representing the Cmin hyperbola and the other, the Cmax hyperbola. Equation (1) represent the boundary hyperbola where θi and θi+1 are the crank angles of the іth and (і+1)th servo. The equation represents an approximate boundary of the workspace for lower values of θi and θi+1. The error in the equation increases with the increase in values of the crank angles. (1)

The region between both the curves represented by C = Cmin and C = Cmax is the total available workspace of the mechanism. 1.5

1

(3)

(4)

(5)

Here i=1,2,3,….N where N is the number of servo motors used. Simplification of (2) and (3) yields (4), which represents the equation of a family of ellipses bounded inside a square for a constant θm. Here θm and β represents the maximum amplitude and the phase difference between the adjacent crank angles, respectively. Similarly, for linearly increasing amplitude undulations, the equations are given by (5) and (6), which on simplification yields the family of ellipses bounded inside a rectangle and is represented by equation (7).

−0.5 −1 −1

0

θi

1

Fig. 2. Workspace of the mechanism The joint trajectories of the adjacent crank angles making up the total available workspace for the operation of the fin where L = 5, R = 5, Cmin = 3 and Cmax = 8 is represented by Fig. 2., where the axes are in radians. The dimensions of the robot, taken in the paper, is chosen randomly for the purpose of illustration.

3. Undulation

The robot propels under water by creating undulations on the fin that pushes water to either side, creating a net thrust in the forward direction. The undulations used by most of the aquatic organisms for propulsion are constant amplitude undulation and linearly increasing amplitude undulation. The sinusoidal waveform is thus generated on the fin for propulsion. The fabricated model of the undulating fin of the knife fish robot is shown in Fig. 3.

(6)

(7)

4. Feasibility of an Undulation by the Mechanism

An undulation is said to be feasible by a fin mechanism if the ellipse corresponding to the undulation completely lies inside the pair of hyperbola representing the mechanical constraints offered by the membrane. The result is illustrated in Fig. 4 for L = 7, R  =  5, Cmin  =  5, Cmax  =  10, β  =  50⁰ and θm  =  55⁰, where axes are in radians. 1.5

1 0.5

θi+1

θi+1

0

2016

(2)

0.5

−1.5

N° 1

0

−0.5 −1 −1.5

−1

0

θi

1

Fig. 4. Feasible undulation

4.1. Constraints over Undulation

Fig. 3. Undulating fin Consider the motion of the ith and i+1th crank of a constant amplitude undulating fin.

The mechanical limitations of the five bar linkage mechanism with an extendable coupler, offers several constraints to the undulation motion produced by the knife fish robot. It imposes a restriction over the maximum amplitude of the wave generated and also over the phase difference between the sinusoidal motion of the adjacent cranks of the robot. These restrictions need to be taken care of before the generation of the wave. The phase difference of the wave has to be comArticles

27

Journal of Automation, Mobile Robotics & Intelligent Systems

VOLUME 10,

The characteristic ellipses are found by iterating the values of θm and β , and checking for the intersection of the ellipse and the hyperbola, for a particular workspace. Initially a very low value of θm closer to 0⁰ is chosen and β is varied from 0 to until the ellipse touches the hyperbola. If no intersection is detected maximum allowable phase difference is . Since the undulations made by β =  does not generate a sinusoidal profile on the fin, this region is not considered in the paper. The value of θm is given a small increment and the full range of β is inspected for the touch of ellipse with the hyperbola. The ellipses corresponding to this iteration can be imagined to be bounded within a square. The algorithm for the case of a constant amplitude undulation can be visualized as an expanding square centered at origin, having a set of ellipses contained within each of the growing squares. The ellipses in the squares are such that they fall inside the boundary curves represented by the Cmin and Cmax hyperbolas. The growing boundary is a rectangle for a linearly increasing amplitude undulation, and the ellipses are the joint trajectories of the last pair of cranks. All the ellipses fall inside their corresponding bounding rectangle and the Cmin and Cmax hyperbolas. The visualization of the algorithm for constant amplitude wave for L = 5, R = 5, Cmin = 3 and Cmax = 7 is represented in Fig. 5, where axes are in radians.

5. Characteristic Ellipses of the Workspace

The term area ratio that tried to denote the usable area of the workspace was introduced by Low [3]. The ratio merely serves as a proper measure for the usable area since a square that fits in the workspace doesn’t represent the actual usable workspace which is the area between the curves. Hence it serves as an imperfect measure of the usable area which is easily visible from the low values of the area ratio obtained. Moreover, it does not consider the possibility of optimizing the maximum amplitude. This incomplete representation of the usable area in the workspace by a square was redefined by introducing two new curves characterizing the workspace. The curves being the first touch ellipse for constant amplitude approach and the same for constant phase difference approach.

6.2. First Touch Ellipse

6. Constant Amplitude Approach

The algorithm gives βmax =  for lower values of θm. There exists a value of θm beyond which βmax is no longer   , but a lower value. The ellipse corresponding to the first touch on any of the pair of hyperbolas is

The amplitude is kept constant and the phase difference is varied to locate the intersection of the ellipse and the hyperbola.

1.5

1

1

0.5

0.5

θi+1

1 0.5

θi+1

θi+1

1.5

0

0

−0.5

−0.5

−1

−1

−1

−1

0

θi

1

−1.5

−1

0

θi

1

−1.5

1.5

1

1

0.5

0.5

0.5

0

θi+1

1.5

1

−0.5

0

−0.5

0

−1

−1

−1

−1.5

−1.5

0

(d)

1

−1

0

θi (e)

Fig. 5. Illustration of algorithm by constant amplitude approach Articles

1

−0.5

−1.5

θi

0

θi (c)

1.5

−1

−1

(b)

θi+1

θi+1

(a)

28

0

−0.5

−1.5

2016

6.1. Algorithm

promised to optimize the maximum amplitude of the wave. This leads to the introduction of more number of servo motors to obtain a nearly full wave. Thus, the length of the robot increases. Selection of the maximum amplitude closer to 90⁰ is possible, since one of the asymptotes of the hyperbolas is same as the axis of the ellipse. This property ensures the growth of the ellipse along the asymptote of the hyperbolas, making the undulation possible. The operational range for β will be very low if the maximum amplitude of the wave is selected closer to 90⁰.

1.5

N° 1

1

−1

0

θi (f)

1

Journal of Automation, Mobile Robotics & Intelligent Systems

VOLUME 10,

called the first touch ellipse. This ellipse marks the boundary of unconstrained selection of β. The value of β for the undulation is constrained as θm increases beyond this ellipse. Hence, the first touch ellipse of a workspace represents the boundary for unconstrained undulation. The first touch ellipse may represent the intersection of the ellipse with Cmin or Cmax hyperbola depending on the workspace.

7.2. Constant Amplitude Undulation

The algorithm defined in section 6.1 and 7.1 for L = 5, R = 7, Cmin = 3, Cmax = 9 for a constant amplitude undulation is shown in Fig. 7 and Fig. 8, where axes are in degrees. Points A and C represent the first touch ellipses for the two approaches, while points B and D represent the second touch ellipses. The ellipses represented by A and C are the characteristic of the workspace with respect to constant amplitude undulation. Hence, the first touch ellipses of both the approaches are the characteristic ellipses and it serves as a basis of comparison between the workspaces. Ellipse with conditions at point A represents the boundary for unconstrained phase difference and ellipse with conditions at point C represents the boundary for unconstrained amplitude. The size of these ellipses vary depending on the workspace.

The constraints over β selection increase beyond the first touch ellipse. The ellipse may touch the other pair of hyperbola for the first time depending on the workspace. This ellipse that represents the first touch on the second pair of hyperbola is called the second touch ellipse. The β selection is over-constrained beyond the second touch ellipse. The ellipses tend to be congested along the common asymptote of the hyperbolas as shown in Fig. 5.

7. Constant Phase Difference Approach

The phase difference is kept constant and the amplitude is varied to check for the touch of ellipse and hyperbola.

100

7.1. Algorithm

Max β

60

B

40 20 0 0

20

1

1

0.5

0.5

θi +1

1

0

−0.5

−0.5

−1

−1

−1

0

θi

1

−1.5

−1

0

θi

1

−1.5

1.5

1

1

0.5

0.5

0.5

θi +1

1.5

1

0

−0.5

−0.5

−1

−1

−1

0

θi (d)

1

−1.5

100

−1

0

θi

0

θi

1

0

−0.5

−1

80

(c)

1.5

−1.5

−1

(b)

θi +1

θi +1

(a)

0

60

0

−0.5

−1

θm

1.5

0.5

−1.5

40

Fig. 7. Constant amplitude approach

1.5

θi +1

θi +1

A

80

A very low value of β is chosen initially and the value of θm is varied until the ellipse first intersects the hyperbola. This procedure is iterated for all β and the maximum allowable θm was found. The algorithm can be visualized as an ellipse that grows along the common asymptote of the hyperbola, with the lengths of major and minor axes growing linearly with an increase in θm . The ellipse takes the shape of a circle as β approaches the maximum value. The visualization of

0

2016

the algorithm for constant phase difference wave for L = 5, R = 5, Cmin = 3 and Cmax = 7 is shown in Fig. 6, where axes are in radians.

6.3. Second Touch Ellipse

1.5

N° 1

1

(e)

−1.5

−1

0

θi

1

(f)

Fig. 6. Illustration of algorithm by constant phase difference approach Articles

29

Journal of Automation, Mobile Robotics & Intelligent Systems

Max θm

C

80 70

D

60 50

20

40

β

60

80

Fig. 8. Constant phase difference approach

7.3. Linearly Increasing Amplitude Undulation The algorithm defined at section 6.1 and 7.1 for L = 5, R = 7, Cmin = 3, Cmax = 9, N = 8 for a linearly increasing amplitude undulation is shown in Fig. 9 and Fig. 10, where axes are in degrees. The ellipse under consideration is the one corresponding to the last pair of adjacent cranks (cranks 7 and 8 in this case); the outermost ellipse of the joint trajectory. The axes of the ellipses change with θm and hence the ellipse does not grow about the common asymptote of the hyperbolas. Hence, the amplitude might not take its maximum value initially for the constant phase difference approach. It gets restricted to a value below , unlike the case of a constant amplitude wave. 100

A

Max β

80

B

60 40 20 0 0

20

40

θm

60

80

100

C

Max θm

80 70

D

60 50 40 0

20

40

β

60

80

100

Fig. 10. Constant phase difference approach

8. Efficiency of the Mechanism for Undulation

30

The area contained between the hyperbolas does not represent the efficiency of a mechanism for undulation since the constant amplitude undulation has a fixed axis ellipse and hence some of the area within the workspace remains unused by the undulation. Moreover, different types of undulations use up the workspace in different manner. Hence, there is a necessity for the introduction of an efficiency term, deArticles

The unconstrained phase difference efficiency, hp is defined as the ratio of the area of the first touch ellipse from constant amplitude approach to the total available workspace area. The unconstrained amplitude efficiency, ha is defined as the ratio of the area of the first touch ellipse from constant phase difference approach to the total available workspace area. A higher value of ha represent the better capability of a mechanism to incorporate waves of higher amplitude, without any constraints on the phase difference. Similarly, a higher value of ha represent the capability to generate waves of higher phase difference without any constraints on the amplitude selection. As per the requirement of the swimming performance of the robot, an optimum value of both the efficiencies can be selected. It can be noted that β = for the first touch ellipse for constant amplitude approach and, for the first touch ellipse for constant phase difference approach. Area of first touch ellipse (constant amplitude approach): Area of first touch ellipse (constant phase difference approach): Area of total available Workspace for servos: A3 = π2. (8)

(9)

The maximum possible efficiency is 78.54%.

8.2. Linearly Increasing Amplitude Undulation

Fig. 9. Constant amplitude approach 90

2016

8.1. Constant Amplitude Undulation

40 0

N° 1

fined based on the undulation to be generated. The usable area of the workspace created by a mechanism for an undulation was quantized by introducing two efficiencies, one based on the first touch ellipse on constant amplitude approach and the other on constant phase difference approach.

100 90

VOLUME 10,

Unlike the case of constant amplitude undulation, the unconstrained amplitude selection might not be possible in some mechanisms. Hence, the first touch ellipse for constant phase difference approach has a restricted θm even for a very low β. Similar to the constant amplitude undulation, β =  for the first touch ellipse for constant amplitude approach. Area of first touch ellipse (constant amplitude approach): A1 = i(i+1) πθm2. Area of first touch ellipse (constant phase difference approach): A2 = i(i+1) πθm2 sin β . Area of total available Workspace for servos: A3 = π2. (10)

9. Optimal Workspace Design

(11)

It is necessary that the robot is capable to perform a variety of undulations with different amplitudes and phase differences, for the optimum performance of a fish robot. An increase in the workspace of the mecha-

Journal of Automation, Mobile Robotics & Intelligent Systems

1.5

1

θi+1

0

50 0 6

L R

−0.5

2016

100

4

0.5

2 0 0

C R

5

10

Fig. 12. Workspace Optimization Surface

−1 −1.5

N° 1

It can be concluded that, for achieving a hp for a selected L/R > 1, the optimum slider lengths are fully defined from equation (13) and (14). The blue colored circle in Fig. 11 represents the inscribed circle for the set of all waves corresponding to amplitude of 50⁰. The design of the slider for L/R = 3.3 is computed using the algorithm mentioned above and is represented by the red colored hyperbolas in Fig 11. Hence, this design procedure ensures the feasibility of the undulations below a design amplitude. The membrane is chosen based on this approach. Hence, the design procedure of the undulating fin is such that it considers all the undulations that the mechanism is supposed to produce, without any wastage in elongation.

θm

nism increases the variety of undulations the fish is capable to perform. Moreover, the experimental results [4] of a knife fish robot, shows an optimal value for the phase difference in order to obtain maximum velocity in the forward direction. The robot should be capable to perform these optimum undulations. The ellipses in section 6 are stepwise bounded by a square corresponding to the chosen θm . The optimum workspace design corresponds to the touch of the Cmin hyperbola with the top and bottom sides of the square and the touch of the Cmax hyperbola with the circle corresponding to the maximum phase difference for a given amplitude. Hence, the optimum design for a given hp is such that one pair of hyperbola touches the bounding square and the other pair touches the circle inscribed in the bounding square as shown in Fig. 11, where axes are in radians. The design ensures an unrestricted amplitude choice below the design value and an unrestricted phase difference choice below   .

VOLUME 10,

−1

0

θi

1

Fig. 11. Optimum workspace For such an optimal design of the mechanism, the ellipse for constant amplitude approach first touches both the pairs of hyperbolas simultaneously. This eliminates the existence of a second touch ellipse. Hence, in an optimum workspace design, the second touch ellipse should coincide with the first touch ellipse. This selection eliminate the wastage of the membrane extension and reduce the need for high quality membrane. The workspace design procedure starts with the selection of hp as per the requirement. Corresponding to a hp , there exists a bounding square and an inscribed circle for constant amplitude undulation. The pairs of hyperbolas are selected such that one pair touches the bounding square and the other touches the inscribed circle. The equation of the inscribed circle is given by equation (12).

(12)

(13)

(14)

Solving the hyperbola, inscribed circle and the bounding square gives equations (13) and (14) that represent the optimal workspace design.

The surface for equation (13) for >  and equation (14) for <  is shown in Fig 12. In case of a constant L/R, θm decreases for an increasing until =  and then increases with . Similarly, for a constant , θm decreases for an increasing until =  and then increases with . The plateau surface corresponding to the lower values of L/R and higher values of C/R and vice versa in the Fig 12 represents the saturated θm region. Cmin and Cmax should be chosen such that they lie on the opposite sides of the line –  = 0 and on the intersection of the curves of constant θm plane and constant L/R plane as per the design requirement. Optimizing θm ensures the complete flexibility in selection of the phase difference. The increase in elongation of the slider/membrane as θm increases can be observed from Fig. 12. It can be visualized as the widening up of the two curves from their line of intersection as shown in Fig. 12. Hence choosing a lower θm will reduce the need for higher elongation of the membrane/slider.

10. Distortions in the Generated Wave

The analysis done so far is with the assumption that a sine wave profile is generated on the fin membrane when the adjacent cranks are rotated in a specific pattern as mentioned in the beginning. However, there are some special cases where a sine wave profile might not exactly be generated, maybe for a specific period of time in each cycle. This factor has not been considered by any previous research in this field. Consider a mechanism with L<R, during the operation of the fin, there is a possibility that at any instant, the mechanism takes a form similar to the one Articles

31

Journal of Automation, Mobile Robotics & Intelligent Systems

(15)

Hence, for L < R the family of ellipses should lie inside three pair of hyperbolas. The optimized workspace will correspond to the touch of the distortion hyperbola to the top and bottom of the bounding square, and the touch of inscribed circle to the Cmax hyperbola. Design of   to generate a wave should obey condition (16) to prevent distortion effects during operation.

(16)

Fig. 13. Distortion in sine wave

32

Hence, the optimization procedure will include selection of the maximum amplitude of operation of the mechanism, determination of minimum value of from (16), and determination of and from (13) and (14). Therefore, to decrease the size of the robot, can be selected less than 1 for a maximum amplitude of up to 81.03°. The error of 9.87° is due to the error in Taylor series approximation. The optimization for the set of all waves below 60°amplitude for unrestricted phase difference is given by Fig. 14, where  < 1. The pink curve represents the distortion hyperbola for  =0,583. It can be noted that the entire green region is inside the region bounded by three pair of hyperbolas, two representing the mechanical constraints due to the membrane and one representing the distortion of the sine wave. Articles

N° 1

2016

1.5 1 0.5

θi +1

in Fig. 13 where crank end B of the ith servo crosses the crank end of (i+1)th servo. In such cases, the slider/ membrane curves back and forms distortions in the sine wave at that instant. This phenomenon may lead to vortex formation or turbulence in water at that instant and consumes more power to propagate under water. Also it may lead to the instability of the robot giving non uniform forward thrust. Hence, it is necessary to preclude such distortions through optimal design. Such a distortion happens when, L+ R cos θi + 1 ³ R cos θi  or  ³cos θi – cos θi + 1. The Taylor series approximation of the expression reduces to equation (15) which is the expression for a hyperbola.

VOLUME 10,

0

−0.5 −1 −1.5

−1

0

θi

1

Fig. 14. Optimization in case of distortion

11. Conclusion and Outlook The algorithms that help to compare the workspace of various fin mechanisms were introduced. The characteristic ellipses were defined for a mechanism and were used to evaluate two new efficiency terms. These parameters were used to compare the efficiency with which the mechanism is capable to incorporate wide variety of undulations. In light of the results from the analysis, the workspace was optimized to reduce wastage of elongation of the membrane. The fin was designed based on the optimization technique employed to evaluate the parameters of the mechanism as per the requirements of the robot. A distortion effect on the sine wave generated was identified and the optimization was redefined to get rid of such distortions in the design stage itself. The optimization was done to incorporate all types of undulations, and not specifically for the most thrust generating wave. This work can be extended in different ways. The hydrodynamic force analysis on the fin can be done with the optimum design. The wave corresponding to the maximum thrust generation is to be found and the optimization algorithm could be modified to incorporate the wave. The equations governing the interdependency of the wave generated and the stability of the robot is to be found. Energy efficiency of the fin could be evaluated to check for its better performance than thrusters. Further, this technology can be incorporated in the marine drives as it is expected to be more energy efficient than the thrusters. Future underwater robots may be made with this kind of Median Paired Fin propulsion system which could be used for oceanographic researches, underwater surveillance, swarm robotics etc.

ACKNOWLEDGMENT

This research was supported by National Institute of Technology Calicut, Kerala, India by providing laboratory facilities and funds for fabrication and experimentation of the robot. This support is gratefully acknowledged.

AUTHORS

Ajith Anil Meera* – Robotics Interest Group, Department of Mechanical Engineering, National Institute of Technology, Calicut, Kerala, India-673601. Tel. +918547267258. E-mail: ajitham1994@gmail.com.

Journal of Automation, Mobile Robotics & Intelligent Systems

VOLUME 10,

N° 1

2016

Attadappa Puthanveetil Sudheer – Department of Mechanical Engineering Robotics\Mechatronics Lab National Institute of Technology, Calicut Kerala, India-673601. Tel. +919961450987 E-mail: apsudheer@nitc.ac.in *Corresponding author

REFERENCES

[1] Triantafyllou M.S., Yue D. K. P., Grosenbaugh M. A., Wolfgang M. J., Barrett D. S., “Drag reduction in fish like locomotion”, Journal of Fluid Mechanics, 1999, vol. 392, 83–212. DOI: 10.1017/ S0022112099005455. [2] Onal C.D., Rus D., Marchese A. D., “Autonomous Soft Robotic Fish Capable of Escape Maneuvers Using Fluidic Elastomer Actuators”, Soft Robotics, 2014, vol. 1, no. 1, 75–87. doi:10.1089/soro.2013.0009. [3] Low K.H, “Modelling and parametric study of modular undulating fin rays”, Mechanism and Machine Theory, 2009, vol. 44, no. 3, 615–632. DOI:10.1016/j.mechmachtheory.2008.11.009 [4] Low K.H, Willy A., “Biomimetic Motion Planning of an Undulating Robotic Fish Fin”, Journal of Vibration and Control, 2006, vol. 12, no. 12, 1337–1359. DOI: 10.1177/1077546306070597. [5] Lincheng Shen, Tianjiang Hu, Guangming Wang, “Kinematic Modeling and Dynamic Analysis of the Long-based Undulation Fin”, Icarv, 5-8 Dec. 2006, 1–6. DOI: 10.1109/ICARCV.2006.345111. [6] Sugimori S., Miki H., Yamamoto R., et al., “Braking Performance of a Biomimetic Squid-Like Underwater Robot”, Japan Journal of Bionic Engineering, 2013, vol. 10, no. 3, 265–273. DOI: 10.1016/ S1672-6529(13)60222-X. [7] Wang S., Dong X., Shang L.-J., et al., “Design and Kinetic Analysis of a Biomimetic Underwater Vehicle with Two Undulating Long-fins”, Acta Automatica Sinica, 2013, vol. 39, no. 8, 1330–1338. DOI: 10.1016/S1874-1029(13)60049-X. [8] Patankar N. A., Lauder G. V., MacIver M. A., Curet O. M., “Mechanical properties of a bio-inspired robotic knifefish with an undulatory propulsor”, Bioinspiration and Biomimetics, April 2011, vol. 6, no. 2. Doi:10.1088/1748-3182/6/2/026004. [9] Lamas M. I., J. D. Rodriguez, C. G. Rodriguez, “CFD Analysis of Biologically Inspired Marine Propulsors”, Brodogradnja, 2012, vol. 63, no. 2, 125–133. [10] Tianmiao Wang, Guanhao Wu, and Jianhong Liang Li Wen, “Quantitative Thrust Efficiency of a Self-Propulsive Robotic Fish: Experimental Method and Hydrodynamic Investigation”, IEEE/ASME Transactions on Mechatronics, June 2013, vol. 18, no. 3, 1027–1038. DOI: 10.1109/ TMECH.2012.2194719 [11] Morawski M., Malec M., Szymak P., Trzmiel A., “Analysis of Parameters of Traveling Wave Impact on the Speed of Biomimetic Underwater Vehicle”, Solid State Phenomena, vol. 210, 273–279, Oct. 2013, DOI: 10.4028/www.scientific.net/SSP.210.273. Articles

33

Journal of Automation, Mobile Robotics & Intelligent Systems

VOLUME 10,

N° 1

2016

State of the Art in Predictive Control of Wheeled Mobile Robots Submitted: 10th January 2016; accepted 25th January 2015

Patryk Harasim, Maciej Trojnacki DOI: 10.14313/JAMRIS_1-2016/5 Abstract: The paper is concerned with the problem of tracking control of wheeled mobile robots (WMRs) using predictive control systems. Various kinematic structures of WMRs important from the point of view of motion control are discussed. A hierarchical approach to the problem of motion control of this kind of robots is presented. The problems of trajectory tracking and path following control of WMRs are briefly discussed. The methods of predictive control of WMRs are described in detail and the following aspects relevant to predictive control are considered: kinematic structures of robots, slip of wheels and its compensation, assumed constraints, methods of optimization of the objective function, problems of model nonlinearity, linearization and discretization, stability of the control system and use of the state observers. Keywords: wheeled mobile robot, motion control, predictive control, model-based control, optimization

1. Introduction

34

Wheeled mobile robots (WMRs) are vehicles designed to help humans with handling repetitive tasks, sometimes in hazardous or difficult-to-access environments. They find practical applications in various domains including: military, anti-terrorist, manufacturing, civil engineering, logistics and transport, agriculture, space exploration, healthcare and in other fields of science and technology [48]. The important task which WMRs have to perform is possibly most accurate realization of motion in changeable conditions, the changes being associated with a robot (e.g. displacement of mass of cargo), environment (e.g. type of ground and terrain inclination), but ultimately resulting from wheel-ground interaction. Important problems associated with motion of WMRs are slip phenomenon and ground deformation (in case of soft grounds like soil, sand, etc.). Especially the slip of wheels has significant influence on robot motion and requires adequate control strategy. The WMRs should also operate correctly in case of noisy or delayed input signals, temporary lack of signals from control system (e.g. in case of teleoperation) or sensors (e.g. from Global Navigation Satellite Systems) and permanent failure of selected sensors or other devices. Model-based predictive control (MPC) is advanced control technique (usually understood as any tech-

nique more advanced than a standard PID control) which was tremendously successful in practical applications in recent decades, exerting great influence on directions of development of industrial control systems as well as on research in this area [37]. From the point of view of the WMRs, the predictive control is attractive technique which main advantage is connected with realization of accurate motion in previously mentioned changeable conditions. Despite numerous works regarding particular solutions of predictive control of WMRs, there is lack of publications describing in a comprehensive way current state of the art, that is, taking into account: kinematic structures of robots, slip of wheels and its compensation, assumed constraints, methods of optimization of objective function, problems of model nonlinearity, linearization and discretization, stability of the control system and use of the state observer. Therefore the objective of this work is detailed review of work associated with predictive control of the WMRs while taking into consideration the mentioned aspects. This review is mainly limited to the most recent works, that is, from the 2010–2015 period, with few exceptions. The paper focuses on the problem of direct control of the WMR movement, that is, excluding some issues related to path or trajectory planning and obstacle avoidance using MPC. However, on the occasion of the review it was noticed that often tackled issue in the research works is the problem of path or trajectory planning of the WMRs using the MPC algorithms.

2. Wheeled Mobile Robots and Motion Control In general, it is possible to distinguish the following kinematic structures of the WMRs: • differentially driven (e.g. three-wheeled Pioneer 2-DX robot with two non-steered driven wheels and a castor), • skid-steered (e.g. six-wheeled IBIS robot [47] by PIAP with all wheels non-steered and independently driven), • car-like (e.g. four-wheeled robot with two steered and driven wheels and two non-steered free wheels), • omnidirectional robot (e.g. three-wheeled robot with non-steered mecanum wheels). From the point of view of motion control of the WMRs, important phenomenon is the slip of wheels. Neglecting slip of wheels is reasonable when robot moves with small speeds and accelerations, which results in longitudinal slips of limited magnitude. In turn, neglecting of side slips can be justified when robot

Journal of Automation, Mobile Robotics & Intelligent Systems

moves with small speeds, has steered, caster or mecanum wheels and turning radius is large with respect to velocity of motion. However, it is worth emphasizing that in case of wheeled robots with all non-steered wheels, which is the most popular design type in commercial solutions so far (see e.g. [23, 47]), the slip of wheels always takes place during change of direction of motion. Robots like that are called skid-steered mobile robots and are objects of research, for example, in the works [25, 39]. One of the most common tasks in the WMRs control is that a robot should move from the known initial position to the desired goal position. The control systems performing this task often have a hierarchical structure. The highest layer of the control system is responsible for global path planning in which complete knowledge about robot environment is usually assumed. The next layer performs local path planning, which enables avoidance of any previously unknown obstacles detected by robot environmental sensors during motion. The lowest layer of the control system, which is analyzed in this paper in detail, is responsible for trajectory tracking or less often path following. The trajectory tracking control (or simply tracking control) is the kind of control where the chosen point of a robot has to move on desired motion trajectory. Similar problem but less often analyzed is path planning control in which the chosen point of a robot has to move on desired path. Main difference between those problems is that a trajectory is parametrized by time, which allows to compute all desired motion parameters of the robot. The path following is considered in applications in which spatial errors are more critical than temporal errors. Both kinds of problems are solved using the model predictive control technique, for instance, in article [29]. Tracking control (or path following) of the WMRs using a conventional PID controller is sensitive to variable conditions of operation such as change of transported mass or change of forces of motion resistance. Therefore, control systems of the WMRs are designed so as to guarantee high accuracy and stability of motion in variable conditions. The objective like that requires implementation of complex control techniques such as robust, adaptive or predictive control or their combinations. The control systems governing WMRs motion often use artificial intelligence (or soft computing) techniques which include artificial neural networks, fuzzy logic systems, evolutionary algorithms, etc., and various combinations of these techniques like neural-fuzzy control. Stability of the control systems of the WMRs is usually studied using Lyapunov method whose advantage is that as one of few methods it can be applied in case of non-linear systems.

3. Predictive Control Methods

3.1. General Idea of Predictive Control

In order to make the considerations which follow easier, short description of the predictive control algorithm is provided. The description is based mainly on the work [37]. General principle of predictive control is calculation at each sampling instant k of the algorithm (i.e. at time

VOLUME 10,

N° 1

2016

t = kΔt, where Δt is the controller sampling period) optimal control inputs (signals) sequence u(k + p|k), for p = N1, ..., N and N1 ≥ 1, so as to minimize differences between the desired future outputs (set-points) trajectory ysp(k + p|k) ≡ yd(k + p|k) and predicted controlled outputs y(k + p|k) trajectory over the assumed horizon of prediction N. Prediction is performed based on: model of controlled object with assumed model of disturbances (uncontrolled inputs) and models of constraints, measurements of current and past outputs together with past values of control inputs, known or assumed controlled outputs [37]. Main advantage of this class of algorithms is that effects of control are satisfactory in the presence of object constraints and delays in control system. In comparison to the classic PID algorithm, the control error is defined as a difference between desired trajectory, and predicted trajectory. It follows that the MPC algorithms can respond to set point changes without need of occurrence of control error resulting from actual measurement, only based on the predicted output values. Important consequence of this approach is possibility of correct operation of the control system in case of temporary unavailability of the measured quantities. This is particularly important in case of autonomous control of vehicles using satellite navigation and loss of signals from satellites, e.g. as a result of motion in a tunnel, through terrain with trees or in a city with dense concentration of tall buildings. The following symbols and terminology are introduced for the purpose of carrying out further analysis: • ysp(k + p|k) ≡ yd(k + p|k) – desired future trajectory, • y(k) – output measurement, • d(k) = y(k) – y(k|k–1) – unmeasured disturbance, defined as a difference between measured and predicted output value for a current sampling instant, • y(k + p|k) – predicted controlled output trajectory calculated at a current sampling instant k for sampling instant k + p, • y0(k + p|k) – free component of the predicted output trajectory, corresponding to the situation where the control inputs are kept constant over the entire prediction horizon, i.e., u(k + p – 1) = u(k – 1), dependent on current control signals, • Δy(k + p|k) – forced component of the predicted output trajectory, dependent on future control signals, • u(k + p) – control inputs trajectory, • Δu(k + p) – sequence of control inputs changes, where vectors of control inputs have dimension nu whilst vectors of controlled outputs and their predictions have dimension ny. Control outputs are determined over control horizon Nu ≤ N by minimizing the selected objective function which describes the control quality over the prediction horizon. For this purpose, the following quadratic function is usually used:

(1)

where Λ(p), Ψ(p) – matrices of weights with positive coefficients. Articles

35

Journal of Automation, Mobile Robotics & Intelligent Systems

VOLUME 10,

N° 1

2016

model of the object or difficulties in its implementaIn equation (1) the first sum concerns the differtion (large industrial objects). In case of WMRs with ence beetween desired and predicted output trajecknown model of dynamics it is rarely used. tory. The second sum represents magnitude of future control signal increments, which can be regarded as Generalized Predictive Control. In GPC algorithm a the measure of control energy. Optimization of this model of object in the following form of a difference function consists in minimization of the control error, equation is used: taking into account energy consumption. Optimization can be conducted by means of ana lytical method (in case of linear model) or by means of (4) numerical calculations. Use of linear models is particularly recommended, because the optimal sequence where: A, B and C are polynomial matrices, v(k) is a of control values is obtained in a simple and unique vector of white noises with zero mean, z–1 denotes the way. In practical considerations also constraints of operator of a unit time delay, denotes the backwardoutput signals, control signals and increments of condifference operator. trol signals should be taken into account, which makes Model like that can be transformed using the Bézout the process of optimization more complicated. identity to the form useful for the objective function. Values predicted for the given time instant k usualObject transfer function can be obtained based on ly differ from the measured value for that time instant known model or by conducting object identification. by d(k) = y(k) – y(k|k–1). For the linear object model, according to the superModel Predictive Control with State-space Equaposition principle it is possible to treat the output sigtions. MPCS algorithm uses linear model of object dynal y(k + p|k) as a sum of free and forced trajectories: namics in state space of the form: (2) (5) The forced trajectory depends only on future increments of control signals, which results from the following dependency:

(3)

From this dependency, it follows that knowing matrix of robot dynamics Mp it is possible to determine sequence of control signals based on the optimization problem (most often quadratic programming).

3.2. Predictive Control Algorithms

There exist many variants of the predictive control algorithm, which differ in form of the object model and the method of solution of the optimization problem. Types of the predictive control algorithm include [37]: • DMC (Dynamic Matrix Control) – algorithm based on the object model in a form of step response, • GPC (Generalized Predictive Control) – algorithm based on the object model in the form of difference equations, • MPCS (Model Predictive Control with State-space Equations) – algorithm which uses linear model of object dynamics in state space, • NMPC (Nonlinear Model Predictive Control) – method in which nonlinear dynamics model of the controlled object is used, • FMPC (Fuzzy Model Predictive Control) – method based on fuzzy nonlinear dynamics model of the controlled object, • NNPC (Neural Network Predictive Control) – algorithm with modelling or optimization based on artificial neural networks. 36

Dynamics Matrix Control. DMC algorithm is often used in case of lack of knowledge of mathematical Articles

where: x(k) is a state vector, A, B, C and D are respectively state (system) matrix, input (control) matrix, output matrix and feedforward matrix. Model like that is used in majority of the reviewed works concerning predictive control. The advantage in comparison to the above mentioned algorithms is no of need of storing data about previous values of control signals. Vector of optimal increments of control signals can be determined using transformations described in works [28, 37, 46]. It is also analogous for the other mentioned earlier algorithms based on linear model. Two types of the MPCS algorithm can be distinguished: • Algorithm with Measured State, in which entire state vector is available for measurements, • Algorithm with Estimated State, in which state vector is estimated using state observer or Kalman filter. Nonlinear Model Predictive Control. In NMPC algorithms, nonlinear model of dynamics of controlled object is used. There exist various approaches to this problem. Often model linearization is used in the neighborhood of the operating point, which ultimately boils down to use of algorithms relying on linear model. In contrast, the NMPC uses advanced methods of objective function optimization, which was discussed in the works: [3, 7, 10, 19, 28, 29, 31, 34, 38, 44]. Fuzzy Model Predictive Control. FMPC method relies on fuzzy nonlinear object model. Example of this approach is described in [33]. In this case, the approaches based on linearization at the operating point or nonlinear optimization methods are possible.

Neural Network Predictive Control. NNPC method uses algorithms with optimization based on arti-

Journal of Automation, Mobile Robotics & Intelligent Systems

ficial neural networks, and relevant examples are in works [11, 43].

4. Predictive Control of Wheeled Mobile Robots 4.1. Wheeled Mobile Robots and Slippage

In the present work, kinematic structures of the robots that appear as objects of research in the reviewed articles were analyzed. It was noticed that, in the works concerning predictive control, differentially driven robots and three-wheeled omnidirectional robots are predominant. The works [6, 7, 12, 22, 24, 28, 31, 32, 43] concern two-wheeled robots, and work [6] concerns inverted pendulum robot. In turn, works [2, 5, 10, 20, 21, 29, 38] describe three-wheeled omnidirectional robots, whereas [1, 4, 18, 35], three-wheeled robots with two fixed driven wheels and castor. Works [13, 17, 27, 30] cover research involving fourwheeled robots of car-like steering type, the work [11] is concerned with differentially driven four-wheeled robot with two wheels driven and two castors, whereas works [9, 45, 46] describe four-wheeled omnidirectional robot. Apart from the works [3, 42], no other works were specifically concerned with predictive control of skid-steered robots. In the work [42] however, authors do not take into account model of robot dynamics, neither they explicitly mention that robot wheels are not steered. In turn, authors of the paper [3] introduce a simplified model of the robot reducing it to a twowheeled version. It should be pointed out, that in case of robots like that, because slip of wheels is inherent property of their motion during turning, advantages of the predictive control associated with high accuracy of the realized motion become the most important. As far as slip of wheels is concerned, in a couple of reviewed works, that is, [13–15, 26, 27, 30, 36], information of modeling slip phenomenon and control of the object in conditions of slip of wheels was explicitly stated. In several works, topic of methods of compensation of this kind of disturbances for three-wheeled omnidirectional robots was discussed. Detailed description of controller structure for the case of compensation of friction results for robots like that can be found in works: [2, 5, 9, 28]. In particular, in work [2] example description of model transformations is contained, where in a separate matrix dependencies of measurable (or modelled) disturbances are given. After taking into account total number of reviewed publications about predictive control it can be noticed, that slip of wheels and its compensation are rarely discussed problems. It follows from the mentioned earlier deficiency of works on skid-steered mobile robots. Advantages of predictive control can be revealed also in case of robot control with large delays, e.g. during teleoperation of planetary rovers. In such cases, use of other methods of control would lead to large errors and be hazardous for the robot and its surroundings.

4.2. Constraints of Controlled Objects and Optimization Criteria

In modeling of every real controlled object, physical constraints should be taken into account. The constraints may be associated with:

VOLUME 10,

N° 1

2016

• minimum and maximum values of control, state and output signals of the robot (e.g., magnitudes of control voltages, rates of change of signals, etc.), • control and prediction horizons at which calculations can be conducted in real time, • sampling frequency (length of sampling period). The constraints can significantly complicate the procedure of optimization in predictive control. In the literature two main approaches connected to this problem are followed in general: • introduction of limits for control signals and use of methods analogous to restriction of the integrating action (anti-windup) in the PID controller, while keeping simple methods of optimization (the solution is usually not optimal), • solution of the optimization problem using more advanced numerical methods (not always possible). In the reviewed literature concerning predictive control of WMRs, in the example works [2, 5, 11, 17, 18, 21, 22, 28, 44, 46] authors mainly use the second approach, because it is more promising for more effective solution of the problem. Many works concerning predictive control, like [4, 9, 11, 12, 17, 19, 22, 29, 32–34, 36, 38, 41, 43–46], take up the topic of optimization of the objective function. This topic is especially elaborated in case of nonlinear algorithms of predictive control. Another factor that imposes use of sophisticated numerical methods is the need of taking into account hardware limitations (computer performance).

4.3. Linearization, Uncertainty and Discretization of the Model

In case of nonlinear control objects, important problem from the point of view of predictive control is determination of their mathematical models. In order to enable implementation of well-known algorithms for linear models, linearization of the model in the neighborhood of operating point is carried out. Approaches consisting in linearization of the model at each step of the algorithm, or calculation of the free trajectory based on nonlinear model and carrying out remaining calculations using the linearized model, are also known in theory. In work [4] comparison of methods of modeling of WMRs with linearization and using nonlinear optimization is presented. Other works concerned with the problem of linearization of the model of WMR to use it in predictive control include [2, 6–9, 15, 18, 22, 24, 28, 32, 42, 43, 45, 46]. An important problem is determination of discrete model of the object based on the model with continuous time. Worth attention is the method from work [28], where discretization based on solution of the system state equation is used. Its advantage is accurate reflection of system dynamics. Other works concerned with determination of the model of WMRs with discrete time include [5, 12, 19, 36].

4.4. Stability of the Control System

Another important problem is ensuring control system stability in order to guarantee correct operation of the controller. In [1] example of approach to stability study based on Lyapunov function is given. Other works on Articles

37

Journal of Automation, Mobile Robotics & Intelligent Systems

this problem include [3, 22, 31, 42]. One may notice that despite its significance, the problem of control system stability is rarely considered in the works concerning predictive control of wheeled mobile robots. It probably follows from difficulty of this kind of analysis for more complex objects and control systems.

4.5. Estimation of the State Vector

For development of the predictive control algorithm, the mathematical model of an object is required. In case of system model in state space with discrete time, information about actual state of the system is necessary. This information can be obtained by: • measurement of physical quantities that belong to the state vector (e.g., angular velocities of robot wheels, driving torques), • estimation of state variables of the system based on knowledge of approximate mathematical model and measurements of input (e.g., voltage control signals for drives) and output quantities of the object (e.g., velocity and orientation of the robot platform). Because rarely there exists a possibility of measurement of all state variables, often methods of state vector estimation are used, like for example in works [4, 12, 13, 16, 27, 30, 40]. Often used methods of state vector estimation include state observer and Kalman filter. State observer. Model of the state observer can have the form resulting from the object model, that is [37]:

(6)

where denotes the estimate of the state vector x(k) evaluated on the basis of information available at the previous sampling instant k – 1, while L is a gain matrix which defines observer dynamics. For the observer, the state reconstruction error is defined as: (7)

38

For this error to tend to zero with time, the form of L matrix should be assumed such that the asymptotic stability of the system is guaranteed. The problem of determination of L matrix of the observer boils down to the basic problem of synthesis of a control system – changing positions of the poles of the transposed system. It follows that it is possible to determine the state observer if and only if the object described with the model in state space is observable. The possibility of influencing object dynamics by introduction of the observer adds to it yet another important property – possibility of stabilization of the control system [37]. The state observer may be complemented with model of a measurable or possible-to-model disturbance. After determination of the observer, synthesis of the control system with predictive controller can be carried out based on the estimated object model. The example of a system like that is described in [13]. Kalman filter. In case of the Kalman filter, the following object model can be assumed [37]: Articles

VOLUME 10,

N° 1

2016

(8)

where wx(k) and wy(k) are respectively vectors of nonmeasurable disturbances of the system and of the measurement. The disturbances in general are assumed as probabilistic signals having character of white noise. The idea of the Kalman filter is determination of the optimal state estimate by minimization of the objective function [37]:

(9)

The algorithm of determination of the filter model can rely on the observer equation. Examples of determination of system state based on the Kalman filter are provided in works [12, 13].

5. Conclusion

In the present paper, the state of the art concerning predictive control system for motion control of the WMRs was described in detail. Results of this review are presented in a compact way in Table 1. The table contains information about particular works including: authors, year of publication, reference, number of wheels and type of investigated vehicle, and description of the analyzed problem. The works are given in the order of publication (years). All mentioned works contain description of vehicle dynamics or kinematics model (except for [34] where, however, a reference is made to model from another paper), results of simulation and/or empirical research, therefore these information is not included in the table. After analysis of data presented in the table one can notice that: • differentially driven (mainly three-wheeled) and omnidirectional robots are predominant, • skid-steered robots are rarely analyzed but more often slip of wheels or friction compensation are taken into consideration, • drive units of the WMRs are hardly ever taken into account in robot model, • the problem of model identification of the WMRs is very rarely analyzed, • very popular is the approach of description of model of the WMR in state space, • in recent years more and more often the nonlinear MPC methods, state observer and Kalman filter are used, • stability of predictive control systems and methods of tuning of the MPC controllers are seldom investigated, • the problems of objective function optimization, model linearization or discretization as well as signal constraints are often considered, • path or trajectory planning and obstacle avoidance problems are sometimes analyzed together with trajectory tracking (or less often path following) control, • in selected works the problems of robot formation control or simultaneous tracking control and stabilization are analyzed.

Journal of Automation, Mobile Robotics & Intelligent Systems

VOLUME 10,

N° 1

2016

Table 1. Selected works concerning predictive control of WMRs and ground vehicles Authors

Year

Ref.

Armesto et al.

2015

[4]

Bature et al.

2015

[6]

Garcia et al.

2015

[13]

Kanjanawanishkul

2015

[21]

Lucet et al.

2015

[30]

Nascimento et al.

2015

[34]

Yang et al.

2015

[43]

2/differentially driven

Yu et al.

2015

[44]

–/–

Barreto et al.

2014

[5]

Deng et al.

2014

[11]

Farrokhsiar & Najjaran

2014

[12]

Li et al.

2014

[28]

Teatro et al.

2014

[38]

Zarghami et al.

2014

[45]

Zarghami et al.

2014

[46]

Cartade et al.

2013

[8]

Guillet et al.

2013

[16]

4/car-like

Amoozgar & Zhang

2012

[1]

2012

[27

3/differentially driven

Lenain & Thuilot

Wheels no./ vehicle type

3/differentially driven

2/inverted pendulum 4/car-like

Analyzed issues Nonlinear MPC, model in state space, tracking control, linearization, Kalman filter, optimization

Model in state space, linearization, identification methods, tracking control

Model in state space, Kalman filter, path planning, state observer, delays

3/omnidirectional Model in state space, path following, virtual vehicle, constraints 4/car-like

Model in state space, slip of wheels, state observer, tracking control, adaptive control, dynamic stabilization

–/omnidirectional Nonlinear MPC, robot formation control, controller tuning, optimization

Nonlinear MPC, model in state space, hybrid chaotic optimization, tracking control, extreme learning machine, artificial neural network

Nonlinear MPC, model in state space, path following, stability, constraints, optimization

3/omnidirectional Model in state space, friction compensation, tracking control, discretization, constraints 4/differentially driven

Model in state space, constraints, optimization, tracking control, artificial neural network

2/differentially driven

Nonlinear MPC, model in state space, feedback linearization, tracking control, discretization, constraints

2/differentially driven

Model in state space, Kalman filter, stochastic control, motion planning, statistical linearization, discretization, constraints, optimization

3/omnidirectional Nonlinear MPC, motion planning and tracking, obstacle avoidance, optimization

4/omnidirectional Model in state space, tracking control, delays, linearization, optimization 4/omnidirectional Model in state space, tracking control, delays, linearization, constraints, optimization 4/car-like

4/car-like

Slip of wheels, robot formation control, adaptive control, state observer, linearization

Model in state space, slip of wheels, robot formation control, adaptive control, state observer, linearization Model in state space, tracking control, state tracking, kinematic approach, stability

Model in state space, slip of wheels, tracking control, adaptive control, Kalman filter, observer Articles

39

Journal of Automation, Mobile Robotics & Intelligent Systems

VOLUME 10,

N° 1

2016

Table 1 cont. Selected works concerning predictive control of WMRs and ground vehicles

40

Wheels no./vehicle type

Analyzed issues

3/differentially driven

Model in state space, slip of wheels, discretization, tracking control, optimization

Authors

Year

Ref.

Ma et al.

2012

[31]

Panathula et al.

2012

[36]

Araújo et al.

2011

[2]

González et al.

2011

[15]

Hach et al.

2011

[17]

Hedjar et al.

2011

[18]

Maurović et al.

2011

[32]

Conceição et al.

2010

[9]

Kanjanawanishkul

2010

[20]

Kanjanawanishkul et al.

2010

[22]

Wang et al.

2010

[41]

Argomedo et al.

2009

[3]

4/skid-steered

Brezak et al.

2009

[7]

Lee & Yoo

2009

[26]

2/differentially driven

Conceição et al.

2008

[10]

Li et al.

2008

[29]

Pacheco et al.

2008

[35]

Xie & Fierro

2008

[42]

2007

[24]

Klančar & Škrjanc Articles

2/differentially driven

3/omnidirectional

–/differentially driven 4/car-like

3/differentially driven

2/differentially driven 4/omnidirectional

3/omnidirectional, 2/differentially driven

2/differentially driven

3/differentially driven

4/car-like

3/omnidirectional

3/omnidirectional

3/differentially driven 4/skid-steered

3/differentially driven

Nonlinear MPC, tracking control, stabilization, time-varying system, T-S fuzzy model, constraints, stability

Model in state space, tracking control, linearization, constraints, stability

Model in state space, slip of wheels, robust MPC, tracking control, linearization, constraints, stability

Model in state space, slip of wheels, path tracking, optimization, constraints Nonlinear MPC, tracking control, approximation, obstacle avoidance, artificial potential field, constraints

Model in state space, tracking control, linearization, optimization Model in state space, friction compensation, tracking control, linearization, optimization

PhD thesis regarding many problems associated with predictive control of WMRs, including robot formation control and path following Model in state space, path following, path replanning, obstacle avoidance, optimization, linearization, constraints

Model in state space, motor model, path tracking, electromagnetism-like optimization mechanism

Model in state space, slip of wheels, discretization, tracking control, open-closed system, quadratic optimization, stabilization, stability Model in state space, linear/nonlinear control, discretization, linearization, comparison of tracking controllers

Model in state space, nonlinear MPC, slip of wheels, tracking control Model in state space, nonlinear MPC, tracking control, optimization Model in state space, nonlinear MPC, path following, tracking control, optimization, constraints

Path planning, tracking control, identification, optimization, artificial potential field Model in state space, tracking control, linearization, stabilization, stability, optimization, constraints Model in state space, tracking control, linearization, state feedback, constraints

Journal of Automation, Mobile Robotics & Intelligent Systems

AUTHORS Patryk Harasim – Warsaw University of Technology, Faculty of Mechatronics, Warsaw, 02-525, POLAND, p.harasim27@gmail.com.

Maciej Trojanacki* – Industrial Research Institute for Automation and Measurements (PIAP), Warsaw, 02486, POLAND, mtrojnacki@piap.pl. *Corresponding author

REFERENCES

[1] Amoozgar M.H., Zhang Y.M., “Trajectory tracking of Wheeled Mobile Robots: A kinematical approach”. In: Proceedings of 2012 8th IEEE/ASME International Conference on Mechatronic and Embedded Systems and Applications, MESA 2012, 275–280. DOI: 10.1109/MESA.2012.6275574. [2] Araújo, H.X. et al., “Model predictive control based on LMIs applied to an omni-directional mobile robot”. In: IFAC Proceedings Volumes, 2011, 8171– 8176. [3] Argomedo F.B. et al., “Constrained Model Predictive Control of a skid-steering mobile robot”. In: 2009 European Control Conference, ECC’09, August 2009, Budapest, Hungary, 4653–4658. [4] Armesto L. et al., “Duality-Based Nonlinear Quadratic Control: Application to Mobile Robot Trajectory-Following”, IEEE Transactions on Control Systems Technology, vol. 23, no. 4, 2015, 1494–1504. [5] Barreto S., J.C.L. et al., “Design and implementation of model-predictive control with friction compensation on an omnidirectional mobile robot”, IEEE ASME Trans. Mechatron., vol. 19, no. 2, 2014, 467–476. DOI: 10.1109/TMECH.2013.2243161. [6] Bature A.A. et al., “Identification and model predictive position control of Two Wheeled Inverted Pendulum mobile robot”, J. Teknol., vol. 73, no. 6, 2015, 153–156. DOI: 10.11113/ jt.v73.4467. [7] Brezak M. et al., “Experimental comparison of trajectory tracking algorithms for nonholonomic mobile robots”. In: IECON Proceedings (Industrial Electronics Conference), 2009, 2229–2234. DOI: 10.1109/IECON.2009.5415188. [8] Cartade P. et al., “Adaptive and predictive control of a mobile robots fleet: Application to offroad formation regulation”. In: Proceedings – IEEE International Conference on Robotics and Automation, 2013, 1836–1842. [9] Conceição, A.G.S. et al., Predictive control of an omnidirectional mobile robot with friction compensation. In: Proceedings - 2010 Latin American Robotics Symposium and Intelligent Robotics Meeting, LARS 2010. 30–35 (2010). [10] Conceição A.S. et al., “A nonlinear model predictive control strategy for trajectory tracking of a fourwheeled omnidirectional mobile robot”, Optim. Control Appl. Methods., vol. 29, no. 5, 2008, 335– 352. DOI: 10.1002/oca.827.

VOLUME 10,

N° 1

2016

[11] Deng J. et al., “Trajectory tracking of mobile robots based on model predictive control using primal dual neural network”. In: Xu S. and Zhao Q. (eds.), Proceedings of the 33rd Chinese Control Conference, CCC 2014, IEEE Computer Society 2014, 8353– 8358. DOI: 10.1109/ChiCC.2014.6896401. [12] Farrokhsiar M., Najjaran H., “Unscented model predictive control of chance constrained nonlinear systems”, Adv. Robot., vol. 28, no. 4, 2014, 257–267. DOI: 10.1080/01691864.2013.867815. [13] Garcia O. et al., “Design and simulation for path tracking control of a commercial vehicle using MPC”. In: Branco K.C. et al. (eds.), SBR-LARS Robotics Symposium and Robocontrol (SBR LARS Robocontrol), 2014 Joint Conference on Robotics, 2015, ISBN 9781479967117, 61–66. DOI: 10.1109/SBR.LARS.Robocontrol.2014.23. [14] Ghasemi M. et al., “Finite-time tracking using sliding mode control”, J. Frankl. Inst., vol. 351, no. 5, 2014, 2966–2990. DOI: 10.1016/j. jfranklin.2014.02.001. [15] González R. et al., “Robust tube-based predictive control for mobile robots in off-road conditions”, Robot. Auton. Syst., vol. 59, no. 10, 2011, 711–726. DOI: 10.1016/j.robot.2011.05.006. [16] Guillet A. et al., “Off-road path tracking of a fleet of WMR with adaptive and predictive control”. In: IEEE International Conference on Intelligent Robots and Systems, 2013, 2855–2861. DOI: 10.1109/ IROS.2013.6696760. [17] Hach O. et al., “Avoiding steering actuator saturation in off-road mobile robot path tracking via predictive velocity controls”. In: IEEE International Conference on Intelligent Robots and Systems, 2011, 4072–4077. DOI: 10.1109/ IROS.2011.6095007. [18] Hedjar R. et al., “Approximated nonlinear predictive control for trajectory tracking of a wheeled mobile robot”. In: First International Conference on Robot, Vision and Signal Processing, RVSP 2011, 296–299. DOI: 10.1109/RVSP.2011.21. [19] Heonyoung L. et al., “Nonlinear model predictive controller design with obstacle avoidance for a mobile robot”. In: 2008 IEEE/ASME International Conference on Mechatronics and Embedded Systems and Applications, MESA 2008, 494–499. [20] Kanjanawanishkul K., Coordinated Path Following Control and Formation Control of Mobile Robots. PhD Thesis. Universität Tübingen 2010. [21] Kanjanawanishkul K., “MPC-Based path following control of an omnidirectional mobile robot with consideration of robot constraints”, Adv. Electr. Electron. Eng., vol. 13, no. 1, 2015. DOI: 10.15598/ aeee.v13i1.1228. [22] Kanjanawanishkul K. et al., “Path following with an optimal forward velocity for a mobile robot”. In: IFAC Proceedings Volumes (IFAC-PapersOnline), 2010, 19–24. [23] Kasprzyczak L. et al., “Robot for monitoring hazardous environments as a mechatronic product”, Journal of Automation, Mobile Robotics, and Intelligent Syststems, vol. 6, no. 4, 2012, 57–64. Articles

41

Journal of Automation, Mobile Robotics & Intelligent Systems

42

[24] Klančar G., Škrjanc I., “Predictive trajectory tracking control for mobile robots”. In: Proceedings of EPEPEMC 2006: 12th International Power Electronics and Motion Control Conference, 2007, 373–378. DOI: 10.1109/EPEPEMC.2006.283188. [25] Kozlowski, K., Pazderski, D., “Practical stabilization of a skid-steering mobile robot-A kinematicbased approach”. In: Proc. IEEE 3rd Int. Conf. on Mechatronics, 2006, 519–524. DOI: 10.1109/ ICMECH.2006.252581. [26] Lee J.-H., Yoo W.-S., “An improved model-based predictive control of vehicle trajectory by using nonlinear function”, J. Mech. Sci. Technol., vol. 23, no. 4, 2009, 918–922. DOI: 10.1007/ s12206-009-0312-9. [27] Lenain R., Thuilot B., “Mobile robot control on uneven and slippery ground: An adaptive approach based on a multi-model observer”. In: IEEE International Conference on Intelligent Robots and Systems, 2012, 1141–1148. DOI: 10.1109/ IROS.2012.6385533. [28] Li R. et al., “Nonlinear model predictive control for WMR with input constraint”. In: Processing of 2014 International Conference on Multisensor Fusion and Information Integration for Intelligent Systems, MFI 2014. DOI: 10.1109/MFI.2014.6997679. [29] Li X. et al., “Nonlinear model predictive control of an omnidirectional mobile robot”. In: Intelligent Autonomous Systems 10, IAS 2008, 92–99 . [30] Lucet E. et al., “Dynamic path tracking control of a vehicle on slippery terrain”, Control Eng. Pract., vol. 42, 2015, 60–73. DOI: 10.1016/j. conengprac.2015.05.008. [31] Ma M.-M. et al., “Tracking control and stabilization of wheeled mobile robots by nonlinear model predictive control”. In: Chinese Control Conference, CCC, 2012, 4056–4061. [32] Maurović I. et al., “Explicit model predictive control for trajectory tracking with mobile robots”. In: IEEE/ASME International Conference on Advanced Intelligent Mechatronics, 2011, 712–717. DOI: 10.1109/AIM.2011.6027140. [33] Nagata A. et al., “Model predictive obstacle avoidance control for omni-directional mobile robots based on fuzzy potential method”. In: 2014 European Control Conference, 352–357. DOI: 10.1109/ECC.2014.6862356. [34] Nascimento T.P. et al., “Nonlinear Model Predictive Formation Control: An Iterative Weighted Tuning Approach”, Journal of Intelligent & Robotic Systems, vol. 80, no. 3–4, 2015, 441–454. DOI: 10.1007/ s10846-015-0183-5. [35] Pacheco L. et al., “Local model predictive control experiences with differential driven wheeled mobile robots”. In: Proceedings of 2008 IEEE International Conference on Automation, Quality and Testing, Robotics, AQTR 2008 – THETA 16th Edition, 377–382. DOI: 10.1109/ AQTR.2008.4588858. [36] Panathula C.B. et al., “Model predictive traction control for robots on slippery 3D terrains”. In: Proceedings of the American Control Articles

VOLUME 10,

N° 1

2016

Conference, 2012, 4257–4262. DOI: 10.1109/ ACC.2012.6315090. [37] Tatjewski P., Advanced control of industrial processes: structures and algorithms, Springer Science & Business Media 2007. [38] Teatro T.A.V. et al., “Nonlinear model predictive control for omnidirectional robot motion planning and tracking with avoidance of moving obstacles”, Can. J. Electr. Comput. Eng., vol. 37, no. 3, 2014, 151–156. DOI: 10.1109/CJECE.2014.2328973. [39] Trojnacki M., “Dynamics model of a fourwheeled mobile robot for control applications – a three-case study”. In: Intelligent Systems’ 2014, chapter 10, 2015, 99–116. DOI: 10.1007/978-3-319-11310-4_10. [40] Tsoeu M.S., Esmail M., “Unconstrained MPC and PID evaluation for motion profile tracking applications”. In: IEEE AFRICON Conference, 2011. DOI: 10.1109/AFRCON.2011.6072037. [41] Wang Y. et al., “A model predictive control strategy for path-tracking of autonomous mobile robot using electromagnetism-like mechanism”. In: Proceedings – International Conference on Electrical and Control Engineering, ICECE 2010, 96–100. [42] Xie F., Fierro, R., “First-state contractive model predictive control of nonholonomic mobile robots”. In: Proceedings of the American Control Conference, 2008, 3494–3499. [43] Yang Y. et al., Predictive Control Strategy Based on Extreme Learning Machine for Path-Tracking of Autonomous Mobile Robot. Intell. Autom. Soft Comput. 21, 1, 2015, 1–19. [44] Yu S. et al., “Nonlinear model predictive control for path following problems”, Int. J. Robust Nonlinear Control., vol. 25, no. 8, 2015, 1168–1182. DOI: 10.1002/rnc.3133. [45] Zarghami M. et al., “Fast and precise positioning of wheeled Omni-directional robot with input delay using model-based predictive control”. In: Xu S. and Zhao Q. (eds.) Proceedings of the 33rd Chinese Control Conference, CCC 2014, 7800–7804. DOI: 10.1109/ChiCC.2014.6896302. [46] Zarghami M. et al., “Model-based predictive control of wheeled omni-directional robots considering nonlinear dynamical constraints and input delay”. In: 2014 13th International Conference on Control Automation Robotics and Vision, ICARCV 2014, 1379–1385. DOI: 10.1109/ ICARCV.2014.7064517. [47] Mobile robots for counter-terrorism (PIAP), http://www.antiterrorism.eu. [48] SPARC Multi Annual Roadmap For Robotics in Europe Horizon 2020 Call ICT – 2016 (ICT - 25 & ICT - 26), http://sparc-robotics.eu.

Journal of Automation, Mobile Robotics & Intelligent Systems

VOLUME 10,

N° 1

2016

Design of Piezoelectric Gravimeter for Automated Aviation Gravimetric System Submitted: 10th January 2016; accepted 25th January 2015

Igor Korobiichuk, Olena Bezvesilna, Andrii Tkachuk, Tetyana Chilchenko, Michał Nowicki, Roman Szewczyk DOI: 10.14313/JAMRIS_1-2016/6 Abstract: The article presents and describes the structure of a new piezoelectric gravimeter of automated aviation gravimetric system (AGS), which has a higher accuracy (1mGal) than other types of gravimeters known to date and automated signal processing. It also describes the operation principle of a piezoelectric gravimeter and presents its mathematical model. There have been determined conditions under which the need to use lowpass filter in automated AGS is prevented. Keywords: piezoelectric gravimeter, aviation gravimetric system, gravitational acceleration, piezoelectric effect

1. Introduction Information about gravitational field of the earth is used in gravimetry, seismology, aviation and space technology, geology, archeology etc. In classical physics, gravitational field is described only by potential and strength. Strength of gravitational field coincides with gravitational acceleration. The most convenient way to determine features of earth’s gravitational field, that is gravitational acceleration gz, is to use aviation gravimetric system (AGS) in which gravimeter is a sensitive element. By means of AGS gravimetric information in remote areas of the globe can be obtained much faster and at lower cost than by means of other gravimetric facilities or systems [1–7]. The effectiveness of AGS is mainly provided by a choice of sensitive element of the systems, i.e. a gravimeter. Today, the most famous AGS gravimeters are: quartz gravimeter GAL-S, developed by Air gravimetric Laboratory at the Institute of Physics of the Earth of the Russian Academy of Sciences; quartz gravimeter GI 1/1, developed by Ramensky Instrument Engineering Plant; string gravimeter “Graviton-M”, developed by State Scientific Production Enterprise “Aerogeofizika”, Bauman Moscow State Technical University and All-Union Scientific Research Institute of Geophysics; quartz gravimeter “Chekan-AM” developed by OJSC Concern “CSRI “Elektropribor”; magnetic gravimeters MAG-1 and GT1A, developed in CJSC “Gravimetric Technologies” and “Canadian Micro Gravity” respectively; gyroscopic gravimeters PIGA 16, PIGA 25, developed at Massachusetts Institute of Technology. The accuracy of these gravimeters is insufficient (3-10 mGal). The speed is slow as well. They are not automated. Results are processed on

Earth after flight in a time-consuming process. Besides, these gravimeters measure vertical acceleration h along with gravitational acceleration. High-accuracy measurement of h is a complex scientific and technical challenge and requires the use of additional filters. Automated gyroscopic gravimeters gz, have been developed and investigated in NTUU “KPI”. However, their accuracy is insufficient (2 mGal) [5, 8–13]. Therefore, improvement of accuracy and speed of gravitational acceleration measurement ​​by creating a new gravimeter of automated AGS is a relevant scientific and technical challenge. The objective of the article is to describe the structure and features of a new piezoelectric gravimeter (PG) of automated aviation gravimetric system which has a higher accuracy and speed that other gravimeters known today.

2. Single Channel Piezoelectric Gravimeter Description

A new piezoelectric gravimeter (PG) of automated aviation gravimetric system has been developed at Instrument Engineering Department of KPI under supervision of the honored master of sciences and engineering of Ukraine, Prof. Dr.-Ing. O.M. Bezvesilna [14]. This gravimeter has an accuracy of 1 mGal. The sensitive element of PG (Fig. 1) consists of a piezoelectric element (PE) 5, which operates on the compression-stretching deformation, insulators 7 at the ends of PE and inertial mass (IM) 6. A sensitive element is elastically attached to the basis 8 by a screw 10 with purpose to increase reliability and durability of the structure. Due to cable 11 PG is connected to an operational amplifier. Piezoelectric element 5 is a multi-layer structure (piezoelectric packet) consisting of layers of crystalline lithium niobate.

Fig. 1. Structure of PG: 1 –PE; 2 – insulators; 3 – IM; 4 – basis; 5 – hermetic enclosure; 6 – screw; 7 – inlet cable 43

Journal of Automation, Mobile Robotics & Intelligent Systems

VOLUME 10,

Let us consider PG oscillation system that includes the following elements: MI (m), stiffening element (k) and damping element (n). Air creating resistance to motion of IM is a damping element. Stiffening element is characterized by elastic properties of PE [5]. According to Newton’s second law, IM is under action of force which equals to: G = mgz (1) where G is gravity affecting IM; m is weight of IM.

N° 1

2016

Due to delayed action of PG structure, first two components will further be neglected. The equation of PG motion in terms of installation on Earth, would be

(6)

In terms of installation of PG on an aircraft the component

should be considered. Let’s consider the equation of automated AGS motion to determine anomalies of gravitational acceleration Δg [1, 7]:

Fig. 2. PG oscillation system In motion IM is also affected by PE elastic force, damping force and inertial force of action of aircraft vertical acceleration. According to Hooke’s law, PE elastic force equals to: (2)

where Fef is PE elastic force; k is elasticity coefficient depending on PE properties. IM damping force Frf:

(3)

where n is damping (friction) coefficient. Giving the (1), (2) and (3) we get:

The final equation of PG motion taking into account inertial force

would look like:

44

Articles

where fz is an output signal; v is aircraft velocity; r is radius of aircraft location; e is ellipsoid compression; φ is latitude; k is aircraft course; ω3 is angular rotational rate of Earth; h is height of aircraft over ellip is vertical airsoid; h is vertical aircraft velocity; h craft acceleration; γ0 is reference value of gravitational acceleration. Let’s rewrite equation of AGS motion as

where f z is AGS output signal; Eotvos correction;

(4)

(7)

– elevation correction;

reference value of gravitational acceleration;

(8)

–

m/s2 – equatorial reference value of gravitational acceleration. Output signal of AGS gravimeter after calculation and introduction of corrections and E, A and γ0 can be written as:

(5)

(9)

Journal of Automation, Mobile Robotics & Intelligent Systems

VOLUME 10,

where is error due to impact of vertical aircraft acceleration. Paper [1, 15, 17] presents analytical expressions of useful signal spectral densities  (ω) and vertical aircraft acceleration  (ω) and their characteristics (Fig. 3).

N° 1

2016

where m is weight of inertial mass; sp, Ep and hp are area, material modulus of elasticity and height of piezoelement, respectively. It is low-frequency filtration with a cutoff frequency of 0.1 rad/s through which can be separated from gz with an accuracy of 1 mGal. Besides, other components of perturbations with predominant frequency of more than 0.1 rad/s are also removed from PG output signal. These perturbations includes translational vibration acceleration with predominant frequency of 3140 rad/s; angular accelerations with predominant frequency of more than 0.1 rad/s [16–19].

3. Double Channel Piezoelectric Gravimeter Description

Fig. 3. Graph of useful signal spectral density (2) and vertical aircraft acceleration (1) [17] As shown in Fig. 3, automatic pilot system significantly reduces the effects of vertical acceleration at frequencies lower than 0.1 rad/s; spectral density of vertical acceleration tends to infinity at frequencies above 0.1 rad/s. It is therefore necessary to ensure filtering process for PG output signal with frequency of 0.1 rad/s. The most common method of constructive implementation of this process in known gravimeters is to use low-pass filters with a cutoff frequency of 0.1 rad/s. However, operation of filter electronic components becomes unstable within terms of PG usage, thereby changing its cutoff frequency. Ultimately, filter lets the interference pass to PG output or doesn’t let the part of signal pass at all. Therefore, low-pass filter as a separate unit of a gravimeter significantly reduces, primarily, reliability and accuracy of the device. There is a suggestion regarding PG output signal filtering technique which consists in creation of an element of a new gravimeter of automated AGS in the form of piezoelectric packet, the natural oscillation frequency ω0 of which equals to the highest frequency of gravitational accelerations which may be measured against interference – 0.1 rad/s:

(10)

As already noted sensitive element of a new PG [14] is a single-channel and constructed as a set of piezoplates. Frequency of sensitive element natural oscillations is 0.1 rad/s. Moreover, PG piezoplates serve both as a sensitive element and a low-pass filter, separating useful signal of gravitational acceleration from high-frequency interference (it is mostly vertical aircraft acceleration). However, the structure of a new PG does not provide for instrumental error compensation resulted from effects of changes in temperature, humidity and pressure of environment that are significant in extreme conditions, associated with gravimetric measurements on an aircraft. Therefore, development of a new double-channel PG was proposed to compensate instrumental errors. For this sensitive element is made of two channels with one piezoelement in each. Piezoelements of both channels are identical and made in the form of piezoplates and inertial mass, which are fixed one over another. Piezoelement 3 of one channel is placed with piezoplates down, and piezoelement 4 of another channel is placed with piezoplates up. Outputs of piezoplates of both channels are connected to inputs of adder unit 5, the output of which is connected to an input of a gravimeter output signal calculation system 2 (Fig. 4).

Fig. 4. Block diagram of double-channel PG Double-channel PG works as follows. Piezoelements of both channels are affected by gravitational acceleration , vertical aircraft acceleration and instrumental errors resulting from residual nonidentity of identical structures of piezoplates and masses, and from the influence of changes in temperature, huArticles

45

Journal of Automation, Mobile Robotics & Intelligent Systems

N° 1

2016

midity and ambient pressure. If these effects are projected on PG measuring axis Oz and given that piezoelement 3 of one channel works in compression and piezoelement 4 of another channel works in tension, we get:

Andrii Tkachuk – Zhytomyr State Technological University, 10005 Zhуtomуr, Ukraine. E-mail: andrew_tkachuk@i.ua.

Michał Nowicki* – Industrial Research Institute for Automation and Measurements, Jerozolimskie 202, 02-486 Warsaw, Poland. E-mail: mnowicki@mchtr.pw.edu.pl.

where u1 is output electric signal of piezoplates of one channel; u2 is output electric signal of piezoplates of another channel; m is weight of inertial mass in each channel; k is piezoelectric constant. Output electric signals u1 and u2 of piezoplates of both channels are summed in adder unit 5:

where is output signal of adder unit 5. Output signal of adder unit 5 is supplied to gravimeter output signal processing system 2. In the final result we get an output signal T of PG output signal processing system 2, which contains double gravitational acceleration signal. It contains no measurement errors resulting from influence of vertical acceleration and instrumental errors . Thus, accuracy of double-channel PG would be significantly higher.

4. Conclusions

The paper considers a new piezoelectric gravimeter of automated AGS, which has higher accuracy (1 mGal) and speed (fully automated) than gravimeters known to date. It also describes the operation principle of a piezoelectric gravimeter and presents its mathematical model. It has been found that it is possible to set piezoelectric gravimeter natural frequency of rad/s and avoid the need to use low-pass filter in automated AGS due to selection of design parameters for a sensitive element of piezoelectric gravimeter. The paper analyzes the prospects for further improvement of accuracy of a new piezoelectric gravimeter by implementing the process of instrumental error compensation resulting from effects of changes in temperature, humidity and pressure of environment that are significant in extreme conditions, associated with gravimetric measurements on an aircraft.

AUTHORS

Igor Korobiichuk* – Institute of Automatic Control and Robotics, Warsaw University of Technology, Boboli 8, 02-525 Warsaw, Poland. E-mail: kiv_Igor@list.ru.

Olena Bezvesilna – National Technical University of Ukraine “Kyiv Polytechnic Institute”, Avenue Peremogy, 37, Kyiv, Ukraine, 03056. E-mail: bezvesilna@mail.ru 46

VOLUME 10,

Articles

Tetyana Chilchenko – Zhytomyr State Technological University, 10005 Zhуtomуr, Ukraine. E-mail: xvulunka@mail.ru.

Roman Szewczyk – Institute of Metrology and Biomedical Engineering, Warsaw University of Technology, Boboli 8, 02-525 Warsaw, Poland. E-mail: r.szewczyk@mchtr.pw.edu.pl *Corresponding author

REFERENCES

[1] Bezvesilna O.M., Aviation gravimetric systems and gravimeters, Zhytomyr: ZSTU, 2007, 604 p. (in Ukrainian). [2] Korobiichuk I., Bezvesilna O., Tkachuk A., Nowicki M., Szewczyk R., Shadura V., “Aviation gravimetric system”, International Journal of Scientific & Engineering Research, vol. 6, ni. 7, July 2015, 1122–1127. [3] Krasnov A.A., Nesenyuk L.P., Peshekhonov V.G., Sokolov A.V., Elinson L.S., “Integrated marine gravimetric system. Development and operation results”, Gyroscopy and Navigation, vol. 2, issue 2, April 2011, 75–81. DOI: 10.1134/ S2075108711020052. [4] Forsberg R. , Olesen A.V., Einarsson I., “Airborne gravimetry for geoid determination with Lacoste Romberg and Chekan gravimeters”, Gyroscopy and Navigation, vol. 6, issue 4, 1 October 2015, 265–270. DOI: 10.1134/ S2075108715040069 [5] Bezvesilna O.M., Tkachuk A. H., Piezoelectric gravimeter of aviation gravimetric system, Zhytomyr: ZSTU, 2013, 240 p. (in Ukrainian). [6] Bezvesilna O.M., Ostapchuk O.O., Tymchyk G.S., Automated laser ballistic gravimeter, Zhytomyr: ZSTU, 2014, 180 p. (in Ukrainian). [7] Bezvesilna O.M., Tkachuk A.H., “Corrected gyrocompass synthesis as a system with changeable structure for aviation gravimetric system with piezoelectric gravimeter”, Aviation, vol. 18, no. 3, July 2014, 134–140, DOI:10.3846/16487788.20 14.969878. [8] Bezvesilna O.M., Tkachuk A.H., “Аviation gravimetric system”, Assembly in mechanical engineering, instrument. Journal, no. 6, 2014, 32–37. (in Russian). [9] Williams S., MacQueen J.D., “Development of a versatile, commercially proven, and cost-effec-

Journal of Automation, Mobile Robotics & Intelligent Systems

VOLUME 10,

N° 1

2016

tive airborne gravity system”, The Leading Edge, vol. 20, issue 6, June 2001, 651–654. DOI: DOI: 10.1190/1.1439018 [10] Cai S., Zhang K., Wu M., Huang Y., “Long-term stability of the SGA-WZ strap down airborne gravimeter”, Sensors, vol. 12, issue 8, August 2012, Switzerland, 11091–11099. DOI: 10.3390/ s120811091. [11] Wang W., Luo C., Xue Z., et al., “Progress in the development of Laser Strapdown Airborne Gravimeter in China”, Gyroscopy and Navigation, vol. 6, issue 4, 1 October 2015, 271–277. DOI: 0.1134/S2075108715040161. [12] Jentzsch G., Schulz R., Weise A., “A well-known principle in a new gravimeter: The automated Burris Gravity Meter”, AVN Allgemeine Vermessungs-Nachrichten, vol. 122, issue 5, 1 January 2015, 168–175. [13 Baumann H., Klingele E.E., Marson I., “Absolute airborne gravimetry: a feasibility study”, Geophysical Prospecting, 2012, no. 60, 361–372. [14] Bezvesilna O. M., Tkachuk A.H., Piezogravimeter, 2012, Ukraine. Pat. 99084 (in Ukrainian). [15] Bezvesilna O. M., Koval A.V., Two gyro gravimeter of automated aviation gravimetric system, Zhytomyr: ZSTU, 2013, 252 p. (in Ukrainian). [16] Bezvesilna O. M., Tkachuk A.H., Aviation Gravimetric system for measuring the gravity anomalie, 2014, Ukraine. Pat. 105949 (in Ukrainian). [17] Korobiichuk I., Nowicki M., Szewczyk R., “Design of the novel double-ring dynamical gravimeter” Journal of Automation, Mobile Robotics and Intelligent Systems, vol. 9, nr 3, 2015, 47–52. DOI: DOI: 10.14313/JAMRIS_2-2015/23. [18] Bezvesilna O.M., Tkachuk A.H., Kozko K.S., “System for airborne gravimetry”, European Applied Sciences Journal, no. 5, 2013, 37–40. [19] Bezvesilna O.M., Tkachuk A. H., Chepyuk L.O., Ostapchuk O.O., “Electromechanical gravimeter”, The Advanced Science Journal, no. 7, 2014, 45–48, DOI: 10.15550/ASJ.2014.07.045.

Articles

47

Journal of Automation, Mobile Robotics & Intelligent Systems

VOLUME 10,

N° 1

2016

Recognition of Hand Postures Based on a Point Cloud Descriptor and a Feature of Extended Fingers Submitted: 23rd December 2015; accepted 3rd February 2016

Dawid Warchoł, Marian Wysocki DOI: 10.14313/JAMRIS_1-2016/7 Abstract: Our work involves hand posture recognition based on 3D data acquired by the KinectTM sensor in the form of point clouds. We combine a descriptor built on the basis of the Viewpoint Feature Histogram (VFH) with additional feature describing the number of extended fingers. First, we extract a region corresponding to the hand and then a histogram of the edge distances from the palm center is built. Based on quantized version of the histogram we calculate the number of extended fingers. This information is used as a first feature describing the hand which, together with VFH-based features, form the feature vector. Before calculating VFH we rotate the hand making our method invariant to hand rotations around the axis perpendicular to the camera lens. Finally, we apply nearest neighbor technique for the posture classification. We present results of crossvalidation tests performed on a representative dataset consisting of 10 different postures, each shown 10 times by 10 subjects. The comparison of recognition rate and mean computation time with other works performed on this dataset confirms the usefulness of our approach. Keywords: hand posture recognition, depth cameras, Kinect, point cloud, Viewpoint Feature Histogram

1. Introduction

48

Nowadays, people tend to use gesture-based computer interfaces which are present in mobile device applications, computer games, control systems used in television sets, etc. Gesture recognition is therefore one of the most important problems of human-computer or human-robot interaction. Currently available gesture recognition methods are relatively primitive, compared to vision of mammals [4], and offer satisfactory reliability only in controlled laboratory environment. This makes vision based recognition algorithms an important and challenging research area. This paper presents an approach to recognize hand postures, which are often referred to as static hand gestures, based on three-dimensional depth data in a form of point clouds. 3D representation of visual information is more natural to humans than images obtained from standard 2D cameras. It is because we have two eyes that enable us to estimate our distances from particular objects. Recently introduced Microsoft’s KinectTM sensor, used mainly to control computer games, became

a low-cost device that can be used to depth data acquisition. Another worth mentioning 3D imaging devices are time-of-flight cameras which are now becoming more and more affordable to the mass market. Growing popularity of these devices caused researchers’ interest in hand gesture recognition using depth cameras. However, most often the depth data is used only for hand segmentation or as an auxiliary information included in feature vectors of classified objects [11], [12], [19], [21]. Depth data combined with color data is used by [20] to classify postures based on Average Neighborhood Margin Maximization Transformation approximated by Haarlets (i.e., Haar wavelet-like features). Another approach to hand posture recognition based on depth data is presented by Keskin et al. [7] where KinectTM data is used to obtain hand skeletons by the Mean Shift Local Mode Finding algorithm. Then a skeleton fitting method is used with the random decision forests to classify depth pixels into hand parts. Skeletal data is also utilized by Jiang et al. [3] along with depth data by calculating histograms of points’ distances from the hand joints. The mean, variance and symmetry of the histograms are used as features to recognize hand gestures. Two of the most interesting methods of depthbased segmentation data are presented by Oprisescu et al. [12] and Dominio et al. [2]. In the first of these works the region growing algorithm is used. The growing stops at the boundaries of the region detected using three thresholds: (i) the depth distance of the current point from the seed (the point nearest the camera), (ii) the depth distance of the current point from its neighborhood, and (iii) the luminance threshold based on the intensity time-of-flight image. In [2] palm is separated from fingers and forearm using the largest circle (or ellipse) that can be fitted in the palm region. A center of palm is found by performing Gaussian blur on the binarized arm image and searching for the point with sufficiently large value that is close enough to the point nearest the camera. Proposed by us segmentation method, described in Subsection 3.1, is based on this approach. In the discussed work depth information is also used in the extraction of the classified objects’ features. The combined descriptor includes the following feature sets: (i) the distances of the fingertips from the palm center, (ii) the distances of the fingertips from a plane fitted on the palm samples, (iii) the curvature of the contour of the hand region, and (iv) the shape of the palm region. The authors’ another work [10] includes the addition of features extracted from the Leap Mo-

Journal of Automation, Mobile Robotics & Intelligent Systems

tion data. This device provides 3D information as a set of relevant hand points and some hand pose features. The Leap Motion-based descriptor contains three features: (i) the position of the fingertips, (ii) the palm center, and (iii) the hand orientation. Viewpoint feature histogram (VFH) has been introduced by Rusu et al. [16]. It is a descriptor of 3D objects in a point cloud form. Particular VFH components describe geometry and viewpoint of the surface robustly to large surface noise and even missing depth information. In [5] an approach to recognition of dynamic hand gestures based on VFH and time-of-flight data has been described. Our later work includes the recognition of dynamic hand gestures as well as hand postures using both time-of-flight and KinectTM data [6]. In these works a modification of VFH calculation has been proposed, which consists in dividing the observed scene into smaller cuboidal cells and calculating VFH for each of them. This method increases distinctiveness of the descriptor, especially for objects with subtle differences in shape, which has been proved by experiments resulting in significantly higher recognition rates for a divided scene. The approach is not fully rotation-invariant since while rotating the cloud some of its parts move between neighboring cells. Therefore, a transformation of the hand area should be applied before its division into cells and VFH calculation. We propose a fast way of hand rotation based on the most protruding hand point location. The feature vector contains one additional feature encoding the number of extended fingers (including thumb) as a discrete value. A different approach to gesture recognition using the number of extended fingers is presented in [9]. The fingers are counted, identified and then the algorithm tracks the fingertips while they arrange to form a posture. In our method, the finger information is not crucial and is used only for discriminatory purposes that can be achieved by properly adjusting the importance of this feature with respect to other features. The contributions of this paper are: (i) introducing the extended fingers feature, (ii) proposing some modifications to the hand segmentation and rotation methods that result in faster and easier to implement recognition algorithm, (iii) comparing the proposed algorithm (in terms of recognition rate and time) with the approaches from two significant and recent works. The paper is organized as follows: Section 2 discusses the VFH descriptor; Section 3 describes every step of our proposed posture recognition system; In

Fig. 1. Values of the surface shape component of the VFH

VOLUME 10,

N° 1

2016

Section 4 we present the experimental results; Section 5 contains the conclusions and the plans for the future work related to this subject.

2. Viewpoint Feature Histogram

VFH is the global descriptor of a point cloud – a data structure representing a multidimensional set of points in a clockwise coordinate system [17]. The system’s x-axis is horizontal and is directed to the left, the y-axis runs vertically and faces up, the z-axis coincides with the optical axis of the camera and is turned towards the observed objects. VFH consists of two components: a surface shape component and a viewpoint direction component. They describe geometry and viewpoint of surface created by clouds. The descriptor is able to detect subtle variations in the geometry of objects even for untextured surface, which has been shown experimentally [16]. The first component consists of values Ɵ, cos(α), cos(Φ) and d measured between the gravity center pc and every point pi belonging to the cloud (see Fig. 1). nc is the vector with initial point at pc with coordinates equal to the average of all surface normals. ni is the surface normal estimated at point pi. The angles Ɵ and α can be described as the yaw and pitch angles between two vectors while d denotes the Euclidean distance between pi and pc. The vectors and angles shown in Fig. 1 are defined as follows:

(1)

(2)

(3)

(4)

(5)

(6)

where dot denotes the scalar product and cross denotes the vector product. The viewpoint component consists of a histogram of the angles that the viewpoint direction makes with each normal. The method of VFH calculation has one parameter nn denoting the number of points belonging to local neighborhood used to estimate the surface normals. Default histograms consist of 45 bins for each feature of the surface shape component and 128 for the viewpoint component (308 bins in total). Computational complexity of VFH is quadratic with respect to the number of cloud points. The more detailed descriptions of VFH calculation are presented in [15], [18] (PFH and FPFH descriptors) and [16]. In our experiments we process point clouds and calculate VFH using PCL library described in [17]. Articles

49

Journal of Automation, Mobile Robotics & Intelligent Systems

VOLUME 10,

N° 1

2016

3.1. Hand Segmentation Proposed by us method of recognition begins with the segmentation of the hand. It is based on the approach described in [2] with some modifications introduced in order to make the algorithm faster and simplify it. The first step of the hand segmentation is the extraction of the arm region (hand with forearm) from the depth map DM (see Fig. 3b) which is our input data. To this end, the pixel with the least depth value (closest to the camera) is localized and marked as na and its depth in [m] is denoted as dna. Then all the pixels with the depth value greater than dna + Tdepth are rejected by setting them to 0. As a result we obtain depth map representing arm that is shown in Fig. 3c. In our experiments we set Tdepth to 0.1 [m] according to the empirical observations and taking the example from [2]. Subsequently, we threshold DA in order to obtain a binary image BA (see Fig. 3d):

where DAzi is the depth value of i-th DA pixel and BAi is the i-th BA pixel. Note that if Tdepth value is too large, BA may contain some torso or hand pixels adjacent to the hand parts. However, if the value is too small, there may be no forearm parts in BA. Both cases may result in improper hand segmentation. To remove defects resulting from measurement errors of the imaging device, we propose to perform morphological operations on BA preparing it to the next step of the segmentation process. First, the jagged edges are smoothed using opening and closing with a 3×3 structuring element. Then, small gaps within the arm region, that are not larger than the rectangular structuring element of size 35×35 pixels, are filled. This size turned out sufficient for the data acquired by KinectTM in good lighting conditions. The next step of the hand segmentation is the selection of the central hand point. For this purpose, we apply the Gaussian filter on BA with the kernel of a large size depending on the depth value of na: fex = fey = roundo(100/dna) pixels, where roundo is a function rounding the value to the nearest odd integer. With such calculated kernel size the hand is wholly embraced by the kernel. The resulting grayscale image has a global maximum (the brightest point) Co at the object’s center (see Fig. 3e). We must, however,

Fig. 2. Architecture of the proposed posture recognition system. Blocks represent functions; labels, next to arrows, represent input/output data

3. Posture Recognition System Proposed by us posture recognition system is shown as a diagram in Fig. 2. It consists of the following steps: hand segmentation, conversion of the depth map to the point cloud, rotation and downsampling of the point cloud, extraction of the features, and classification. Our method imposes the following requirements to the person showing postures: (i) the hand should be the object closest to the camera, (ii) the hand should be situated at some distance, greater than a specified parameter Tdepth, from the other body parts, (iii) at least a small part of the forearm should be visible. Note that all these requirements are satisfied almost every time if the postures are shown in a natural way and if there are no visible unwanted objects situated closer to the camera than the user’s hand. The requirement (i) results from the fact that we exploit depth information only.

(a)

(b)

(c)

(d)

(7)

(e)

(f)

(g)

(h)

Fig. 3. Segmentation and rotation of the hand: (a) Color image (not used in our method); (b) Depth map DM; (c) Depth map with the arm region only (na is marked as a red dot); (d) Binary arm image BA (fa is marked as a blue dot); (e) Output of the Gaussian filter on BA (Ch is marked as a green dot); (f) BA with circle Cc (Cf is marked as a brown dot); (g) Binary hand image BH; (h) Rotated and downsampled point cloud 50

Articles

Journal of Automation, Mobile Robotics & Intelligent Systems

ensure that Co belongs to the hand (not the wrist or the forearm region). Therefore, we create a set M containing every local maximum not smaller than 85% of the Co value. Let us denote by Dm,na the Euclidean 2D distance (on the xy plane) from the maximum point m to na and by mv the value (brightness level) of m. As a central hand point Ch we select a maximum from the set M for which the difference Dm,na – mv is minimal. Thus, the hand center is not too far from the point of the least depth na, and it has similar (if not the same) brightness as the global maximum Co. In [2] the point Ch is selected by minimization of Dm,na, without using mv. Our approach to selection of Ch turned out to be more precise which was confirmed by the experiments. After the calculation of the central hand point, the palm region of the hand is roughly estimated by the largest circle that can be fitted on it. The circle with the center Ch is iteratively enlarged, starting from the radius equal to one pixel, until the hand area inside it, denoted as hcarea, is less than 95% of the circle area carea. Then the circle is moved to a neighboring position maximizing the hcarea, and finally the radius is increased once more. After this step, we obtain the updated central hand point Cf with coordinates Cfx,Cfy and the central palm circle Cc with radius RCc. The last segmentation stage is the rejection of the pixels corresponding to the forearm as well as the eventual unwanted objects. We begin with the pixels not belonging to the arm. Such fragments might not be filtered during the first segmentation step because of not using the color-based skin detection in contrast to [2]. They are usually not adjacent to the hand pixels, and we exploit this fact by applying a region growing algorithm with the seed point Cf. The output of this operation is the arm region without the unwanted fragments that were previously isolated from the arm pixels in the BA image. Note that if the algorithm requirements (i) and (ii), mentioned at the beginning of this section, are satisfied, we are sure that, at this point, BA contains only the pixels belonging to the arm region. The corresponding pixels are also rejected in the depth map DA. The main difference in relation to the original segmentation method is the separation of the forearm from the hand. Dominio et al. [2] propose to first rotate the arm in a way that its main axis, extracted using PCA (Principal Component Analysis), coincides with y-axis and faces up. Then, every arm pixel with x coordinate smaller than Cfy-RCc is considered as forearm pixel and therefore discarded. The direction of the main axis computed in this way is not very precise because it depends on the position not only of the hand, but also of the forearm (which can be different). The approach proposed by us consists in discarding the forearm region first and performing the rotation of the hand later (for the feature extraction purposes), without the forearm pixels. The hand rotation is described in Subsection 3.3. For the separation of the forearm pixels we use region growing method, as for the previous operation. In this case, as a seed point we use the pixel with the largest depth value (furthest to the camera) marked as fa. In almost ev-

VOLUME 10,

N° 1

2016

ery case this pixel belongs to the forearm. The region growing is applied for BA image with drawn circle Cc (see Fig. 3f). The region boundaries are the edges of the forearm and the lower part of Cc. The designated pixels are considered to belong to the forearm region and are therefore rejected forming a binary hand image BH. In some rare cases, which may be the consequence of showing the posture while not following the requirement (ii), fa may belong to a hand region situated outside the circle Cc. Such a small part is then rejected and the forearm region remains. To give the algorithm another chance to remove the proper pixels, the region growing is repeated when the number of removed points in the last operation is smaller than 5% of the whole arm area in BA. As a new seed point, the arm pixel closest to fa is selected provided it lies outside Cc. An example of a segmented hand is presented in Fig. 3g.

3.2. Conversion to the Point Cloud

In order to calculate VFH, each depth map has to be converted into the point cloud format. For this purpose, we define the set H of hand pixels belonging to binary image BH: H = {x ∈ BH∣ x = 1}. The conversion is applied for each DA pixel if H contains a pixel of the same coordinates. Therefore, the created point cloud consists only of points belonging to the hand, without forearm and isolated objects. The coordinates of cloud points: , , and were set with respect to the DA pixels’ depth value based on the perspective projection equations and KinectTM camera’s parameters:

(8)

(9)

(10)

where DAwidth is the number of depth map columns; DAheight is the number of depth map rows; fl is the KinectTM infrared camera’s focal length; psx and psy are the pixel dimensions, width and height, respectively. The values of KinectTM camera’s parameters were taken from http://kinectexplorer.blogspot.com and set as follows: fl = 4.73 mm, psx = psy = 0.0078 mm. After the conversion clouds observed from different angles, look realistic compared to corresponding real life objects.

3.3. Rotation and Downsampling of the Point Cloud

Since the approach of calculating VFH for cells, which is explained in Subsection 3.4, is not invariant to hand rotations around the z-axis, the point cloud has to be properly rotated before its division into cells. To this end, we calculate 3D distance (exploiting depth information) between the central point Cf and each hand pixel Hi. Then we choose the point for Articles

51

Journal of Automation, Mobile Robotics & Intelligent Systems

(a)

(b)

VOLUME 10,

N° 1

2016

(c)

Fig. 4. Point clouds of postures and their divided bounding boxes (marked by red edges): (a) nine cells; (b) three horizontal cells; (c) three vertical cells which the distance is maximal and denote it as Mf. This point indicates the position of the most protruding hand point which is usually the longest finger’s tip when at least one finger is extended in the posture. The point cloud is then rotated in a way that the vector Mf - Cf has the same direction as the y-axis. The obtained cloud is redundantly dense. We do not need such a large number of points to compute a representative VFH. For this reason the cloud is downsampled which reduces the number of points and, therefore, speeds up the process of histograms calculation. This operation is performed by the so-called voxel grid filter which creates a 3D voxel grid over the input point cloud data. Every point situated within each voxel (i.e., 3D cuboid) is approximated by its centroid. The voxel dimensions Vx× Vy× Vz are the parameters of the filter. We decided to use cubic voxels and set the dimensions as follows: Vx× Vy× Vz =0.00439 m, which in our previous work [6] turned out to be the optimal value in terms of postures recognition rate. An example of a rotated and downsampled point cloud is shown in Fig 3h.

3.4. Extraction of the Features

52

For the classification purposes we collect the feature set based on the VFH descriptor and one additional feature of extended fingers, denoted as Fvfh and Fef , respectively. In order to calculate VFH, we must define the bounding box, i.e., the cuboidal area around the point cloud. Its walls are determined by the points situated furthest in each main direction of the coordinate system: Bxmin, Bxmax, Bymin, Bymax, Bzmin, Bzmax. The bounding box size is thus matched to the cloud size embracing it entirely with no space between each furthest point and the box wall. To increase the distinctiveness of the descriptor, we divide bounding boxes into several cells of equal sizes and VFH is calculated for each of them. Such a modification is inspired by the method of dividing image into smaller blocks in order to calculate histogram of oriented gradients (HOG) descriptor [1]. It seems reasonable to try to divide areas of interest into different numbers of small regions. Koszowski [8] showed that the number of blocks significantly affects the recognition rate in the case of the HOG-based hand posture recognition. Therefore, our Articles

experiments involve the division of point cloud area into: (i) nine cells, (ii) three horizontal cells, and (iii) three vertical cells (see Fig. 4). To avoid problems with high dimensionality, Fvfh does not include full histograms. Instead, VFH calculated for every descriptor value is represented by its mean and standard deviation. We use the two VFH values: ∣d∣ and cos(Φ) since our previous work [6] indicates that this combination is the best in terms of the hand posture recognition and adding new values or replacing some of them does not yield better results. The experiments undertaken in the mentioned work and other previous works also led us to the conclusion that VFH calculated for the entire point cloud (without the division into cells) was significantly less distinctive, and in this case the postures recognition most often resulted in misclassification, even for the seemingly dissimilar hand shapes. For these reasons we decided to omit the experiments with different VFH values as well as with the VFH calculated for the entire cloud. The Fvfh feature set has a cardinality of Nc· Nv· Nhs , where Nc is the number of cells, Nv is the number of VFH values, and Nhs is the number of histograms representative values (in our case mean and standard deviation). Every feature is normalized to the range [0-1], with maximum and minimum values determined on the basis of the training data, and the whole set is added to the feature vector F. The additional feature is the number of extended fingers (including thumb). To extract it, a histogram of the edge distances from the central hand point Cf is calculated. For each hand pixel Hi we determine its 2D distance from the central point dHi = ∣∣Hi – Cf ∣∣ as well as the angle γHi between the vector Hi – Cf and the vector perpendicular to the y-axis with the opposite direction. The calculated set of angles is then quantized with the uniform quantization step Δ=6° and assigned to the set γ’. All Hi pixels with the same discrete value γ’Hi are grouped together dividing H as follows: H = {C1, C2, ... , C60}, where Cj denotes the angular range, i.e., the group of Hi pixels for which γ’Hi = j, and j ∈{1,2,…,60}. For each group Cj the maximum dHi value is chosen and added to the histogram L:

(11)

Journal of Automation, Mobile Robotics & Intelligent Systems

VOLUME 10,

N° 1

2016

Table 1. Mean accuracies obtained by 5-folds cross validation tests and mean running times of our approach compared to authors’ method from [2]. The tests were performed on the computer (i) - PC with Intel® Core™ Quad, 2.4 GHz CPU Recognition rate [%]

Method

Mean running time [ms]

Fvfh: 9 cells + Fef

95.4

center. Local maxima corresponding to the tips of the extended fingers are marked as red dots

Fvfh: 3 vertical cells + Fef

91.1

Thus, L contains the distances of a subset of the hand edge pixels. Then we find every local maximum of L that is not smaller than the threshold ft equal to 145% of the Cc radius. These points represent the tips of the extended fingers. In our method, a maximum is a point that is greater than its left neighbor and not smaller than its right neighbor (the first and the last point are also considered as neighbors). The threshold ft ratio is set to 145% after some experimentation, so that the algorithm can properly identify extended fingers (even relatively short thumbs). Fig. 5. contains the histogram calculated for the posture presented in Fig. 3. The obtained number of extended fingers is normalized to the range [0–1] and added to the feature vector F.

Fvfh: 3 horizontal cells (without Fef)

88.9

56

99

114

99

114

98.5

104

Fvfh: 3 horizontal cells + Fef

Fig. 5. Histogram of the edge distances from the hand

3.5. Classification

For the classification of postures the k-nearest neighbors technique with k=1 was utilized. Each hand posture was represented by the feature vector F = [Fvfh , Fef]. We noticed that Fef feature was too important for the classifier to treat it like Fvfh features whose number is usually much greater. Thus, while calculating the distance of classified object to each pattern from the training set, we multiply the Fef by a weight ief to increase its importance in classification. The distance Do,p from the object o to pattern p is therefore calculated in the following way: (12) where Nvfh is the number of Fvfh features. The best ief value, in terms of recognition rate, depends on Nvfh. We thus performed cross-validation experiments with different number of cells and VFH values which resulted in changing Nvfh. This let us roughly estimate, (a)

(b)

(c)

(d)

(e)

(f)

(g)

Fvfh: 9 cells (without Fef)

Fvfh: 3 vertical cells (without Fef) Four combined depth features (Dist. + curv. + elev. + area) [2]

Three combined depth features (Dist. + curv. + area) [2]

Two combined depth features (Dist. + curv.) [2]

98

82.6

83.4

57

through logarithmic regression, the following dependency allowing to calculate ief:

(13)

We also observed that larger ief values did not cause much worse recognition results. We may then assume that ief should be greater than or equal to the value determined by equation (13). Later experiments were performed with ief estimated as above. As one can observe, Do,p is calculated based on the Euclidean distance. We also examined the city-block distance as well as k-nearest neighbors classifier with different values of k but the results were not better.

4. Experiments

Our method was evaluated on a dataset first introduced in [14] and used in later works: [13], and [2]. This is a set of 10 postures from which 8 is a subset of American Sign Language finger alpha(h)

(i)

(j)

Fig. 6. Postures included in the dataset used for the evaluation of our recognition method Articles

53

Journal of Automation, Mobile Robotics & Intelligent Systems

Table 2. Mean accuracies obtained by template matching tests and mean running times of our approach compared to authors’ and classic methods provided by Ren et al. [13]. The tests were performed on the computer (ii) - PC with Intel® Core™ Quad, 2.66 GHz CPU Method

Recognition rate [%]

Fvfh: 9 cells + Fef

71.3

Fvfh: 3 vertical cells + Fef

88.3

Fvfh: 3 horizontal cells + Fef

Fvfh: 9 cells (without Fef)

Fvfh: 3 horizontal cells (without Fef) Fvfh: 3 vertical cells (without Fef)

Thresholding decomposition + FEMD – authors’ method [13]

Near-convex decomposition + FEMD – authors’ method [13]

Shape context without bending cost – classic method [13] Shape context with bending cost – classic method [13] Skeleton matching – classic method [13]

93.8 49.8

77.9 69

Mean running time [ms]

46

45

93.9

4 001

93.2

750

83.2

12 364

79.1

26 777

78.6

2 445

bet. Hands are shown on a cluttered background, in a various orientation around the z-axis differing up to about 135 degrees. They also differ in scale and articulation. The postures were shown 10 times by 10 subjects (1000 executions in total). Fig. 6 contains an example of each posture from the dataset.

4.1. Evaluation of the Segmentation Method

2016

4.3. Evaluation of the Extended Fingers Counting

The approach to count the extended fingers was also evaluated. 97.6% of all the postures have their fingers properly counted. Analyzing the failures we observed that 62.5% of them were related to the posture G1 (clenched fist). In most cases the thumb does not appear on the background of the fingers but on the left side of the fist. It is also not clenched at all or bent only a little. Sometimes even the forefinger is not entirely clenched which appears unnatural even for a human. Increasing the threshold ft prevents such mistakes. However, it also causes the situations where relatively short extended thumbs are not counted, esTab. 3. Confusion matrix for our algorithm with the three horizontal cells case and 5-fold cross-validation tests. The maximum possible value in every cell is 100. For clarity, the cells with value 0 are shown as empty

G1 G2 G3

The next step was the evaluation of the rotation method. The visual analysis of the rotated clouds showed that the point Mf did not always correspond to the actual longest extended finger because of little Articles

N° 1

differences between the lengths of some fingers and the camera measurement errors. We checked all 100 realisations of the posture G6. Mf corresponded to the middle finger in 93 postures, to the ring finger in 4 postures, and to the index finger – in 3 postures. We also checked 100 realisations of the posture G7. Mf corresponded to the little finger in 83 postures, to the thumb – in 15 postures, and 2 postures were improperly segmented (the forearm pixels were not rejected). These inaccuracies do not become a significant problem if we use sufficiently large training dataset and the k-nearest neighbors classifier. In such cases, for some posture classes we have 2 or 3 variants of rotated postures according to the finger to which Mf corresponds. In most cases the classifier is able to properly find the nearest neighbour among each variant leading to proper classification, which can be seen on the recognition rates presented later in this section.

The segmentation method was evaluated first. We visually analyzed generated binary hand images BH for every posture file from the database. Only 5 (out of 1000) images were improperly segmented. In each case some hand parts were deleted and in four cases the forearm parts were present. These errors occur when the furthest arm point fa does not belong to the forearm. It is hard to determine whether this is the consequence of not following the algorithm requirement (i) or (ii) by the person showing postures, or this results from a measurement error of the camera.

4.2. Evaluation of the Rotation Method

54

VOLUME 10,

G1

G2

G3

97

2

1

G4

G5

G6

G7

99

G8

G10

1 100

G4

99

1

G5

1

97

2

2

98

G6 G7

94

6

G8

2

98

G9

1

G10

G9

1

99 99

Journal of Automation, Mobile Robotics & Intelligent Systems

pecially when they are too close to the palm region. The remaining 37.5% of failures result from the segmentation errors or from arranging the extended fingers too close to each other so they appear on the image as nearly adjacent. In such cases an error would occur if the tips of these fingers belong to the same angular range Cj of the quantized histogram L. In 70.8% of all the miscounts, the obtained number of fingers differs by 1 from the actual number. For each remaining failure, the difference is equal to 2 and is related to the posture G1 (not entirely clenched forefinger and thumb).

4.4. Comparison of Recognition Rates and Mean Running Times

The main evaluation of our method consists in the comparison of recognition rates and mean running times with the results obtained in [2] and [13] for the same dataset. The comparison is shown in Table 1 and Table 2. The results from Dominio et al. [2] were obtained for different combinations of their proposed depth features. In addition to their own method, Ren et al. [13] provide results for three classic shape recognition algorithms: (i) shape context with bending cost, (ii) shape context without bending cost, and (iii) skeleton matching. Our comparison also includes those scores. Note that Ren et al. [13] exploit a black belt, worn by all the people showing postures, in the hand segmentation. Our method does not require to wear such markers and therefore the belt information is not used. The recognition rates shown in Table 1 were measured in 5-fold cross-validation tests. We divided the dataset to five subsets: one used as a training set and the remaining four as a test set. The division was performed in a way that subjects from the test set were not present in the training set. All the gestures of the first two subjects created the first test set, the gestures of the next two subjects created the second one, and so on. Thus, the ability of the recognition system to generalize over unseen subjects was verified. Dominio et al. [2] performed exactly the same cross-validation tests. The ief value was set according to equation (13). Our best score, obtained for the three horizontal cells, is 98%. It is slightly lower than for the algorithm described in [2] where the best results, obtained for three or four combined depth feature sets, are 99%. In [13] the recognition rates are measured using template matching method where the first gesture realization of each class is used for creating its template. Therefore, the training set consists of only ten realizations. For comparison purposes, we decided to additionally perform such tests despite the fact that our recognition system was designed to be trained on much larger data. In our tests first gesture realizations from person number 5 were used as templates since they turned out to be the most representative. Using such a small training set does not provide representative Fvfh features for each class. Therefore, we multiplied by 1.5 the ief value, calculated according to equation (13), in order to put more emphasis on the extended fingers feature. Our best score, obtained for the three horizontal cells, outperforms every classic algorithm and the authors’ method near-convex de-

VOLUME 10,

N° 1

2016

composition + FEMD and is lower by 0.1% in the case of thresholding decomposition + FEMD. To prove the usefulness of the extended fingers feature, we also present the results obtained without it. For the three horizontal cells case the recognition rate is less by 9.1% according to the cross validation tests and less by 18.7% according to the template matching tests, compared to the case when the additional feature is applied. As one can see, the best obtained recognition rate is slightly higher than the correctness of counting fingers. It is due to the fact that if the number of incorrectly detected fingertips does not much differ from the actual number (what has been observed in every case during our experiments), the classifier is still able to correctly guess the posture class. We can assume that the results for the horizontal cells are significantly higher than for the vertical cells because in the first case the method is less dependent on the hand rotations around the y-axis while it depends more on the hand inclinations around the x-axis. Analyzing the postures from the chosen dataset, we can observe that the rotation around the y-axis is a little more variable than the second one. That explains the better results obtained for the three horizontal cells. Table 3 is the confusion matrix for our algorithm and the three horizontal cells case. The worst result has been obtained for the posture G7 which was confused with G8 six times. This can be explained by the high similarity between them as well as the same number of extended fingers. The interesting fact is that the posture G3 has been recognized correctly in 100% cases, despite that there are three postures with two extended fingers in the dataset. It shows that the VFH descriptor is able to very well differentiate G3 from G7 and G8 for which the Fef feature has the same value. In this case, the upper cell of the point cloud bounding box seems to be the most discriminant because, after the clouds rotation, for G3 it includes two extended fingers while for G7 and G8 only one extended finger is present in this cell and another one is located in the middle or the lower cell. In the terms of a recognition program’s mean computation time, our method turned out to be very fast compared to others. The time refers to the entire recognition process of a single posture. Obviously, this score depends on the computer, on which the program is running. We performed experiments on two computers: (i) PC with Intel® Core™ Quad, 2.4 GHz CPU (released in 2007) and (ii) PC with Intel® Core™ Quad, 2.66 GHz CPU (released in 2009). Ren et al. [13] used a PC with the same CPU as in computer (ii). The mean running time differences between our method and the methods presented in this paper are huge. Our recognition algorithm is about 16 times faster than thresholding decomposition + FEMD and about 87 times faster than near-convex decomposition + FEMD. It is worth noting that the usage of much larger training sets, as in cross-validation tests, significantly improves the recognition rates. In authors’ methods from [13] it will cause much greater running times because the calculation of Finger-Earth’s Move Distance, that have to be done in each template Articles

55

Journal of Automation, Mobile Robotics & Intelligent Systems

comparison, is time-consuming. For our method, the difference between processing times for test sets containing 10 depth maps and test sets containing 800 depth maps is below 0.5 ms. Dominio et al. [2] performed tests on a computer with the same CPU as in computer (ii). In this case the mean running time of our method is almost twice lower. We also compared the segmentation and the rotation stages’ mean running time. The original method takes about 75 ms while our modified version – 47 ms, which shows the advantage of our modifications. The computation time of the extended fingers feature extraction is very short (not greater than 1 ms on the computer (i) and (ii)). Our recognition system is able to run in real-time at about 17 frames per second on the computer (i) and 21 frames per second on the computer (ii). We were able to achieve 35 frames per second while running the program on a modern laptop with Intel® Core™ i7, 2.5 GHz CPU. In order to achieve greater speed, it is possible to apply multithreading to the implementation of our algorithm since some tasks can be performed concurrently, e.g., creating a histogram of the edge distances, converting depth map hand samples to point cloud, and determining the position of the most protruding hand point (see Fig. 2).

5. Conclusions and Future Work

56

In this paper we propose the depth-based posture recognition method using the Viewpoint Feature Histogram and the extended fingers feature. The accuracy of our approach was tested on a challenging dataset with the usage of demanding cross-validation tests in which the classifier recognized postures shown by unseen subjects. The mean running time of the recognition program was also measured. The relatively good results confirm the usefulness of the proposed algorithm. The recognition system proposed by us can be applied to control and interact with computers and robots. For example, the user is able to give specific commands to the robot showing his hand in a specific configuration. Although the recognition of postures by the system does not depend on rotation around z-axis, the algorithm provides an information about how much the hand is rotated. It may be used, e.g., for setting some application or system parameters in a way that recognized gesture corresponds to particular parameter and its rotation angle represents the parameter value. The extended fingers feature turned out to be very helpful because of the low computation time of its extraction and the accuracy improvement caused by its application. This feature is obviously not autonomous because of its possible ambiguity (there may be many postures with particular number of extended fingers). However, it can significantly increase the distinctiveness of the descriptor if its importance parameter is properly chosen. It can be observed that the extended fingers feature and the VFH features complement each other because the first analyzes the hand contours while the second examines dependencies within the hand surface. Our method fails to correctly count the number of Articles

VOLUME 10,

N° 1

2016

fingers if they are adjacent to each other in a presented posture (i.e., the letter ‘H’ of American Sign Language finger alphabet). In such a case each group of adjacent fingers would be counted as one. However, obtaining the actual number of extended fingers is not an end in itself. The method’s purpose is to search for sufficiently large local extrema in the shape of the hand posture and to exploit this information in the distinction of posture classes. The future work in the subject could consist in altering the VFH descriptor. So far only the modification of its calculation was proposed, which proved to be very effective. Searching for other dependencies among the cloud points or the surface normals, that better describe the hand shape, is reasonable.

ACKNOWLEDGEMENTS

The authors thank dr Tomasz Kapuscinski and dr Mariusz Oszust from Rzeszow University of Technology for their valuable suggestions.

AUTHORS

Dawid Warchoł* - Rzeszów University of Technology, Department of Computer and Control Engineering, Faculty of Electrical and Computer Engineering, Wincentego Pola 2, Rzeszów, Poland, 35-959, tel.: (17)8651592. E-mail: dawwar@prz.edu.pl. Marian Wysocki - Rzeszów University of Technology, Department of Computer and Control Engineering, Faculty of Electrical and Computer Engineering, Wincentego Pola 2, Rzeszów, Poland, 35-959, tel.: (17)8651583. E-mail: mwysocki@prz.edu.pl. *Corresponding author

REFERENCES

[1] N. Dalal, B. Triggs, “Histograms of oriented gradients for human detection”, IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Diego: IEEE, 2005, 886–893. DOI: 10.1109/CVPR.2005.177. [2] F. Dominio, M. Donadeo, P. Zanuttigh, “Combining multiple depth-based descriptors for hand gesture recognition”, Pattern Recognition Letters, vol. 50, 2014, pp. 101–111. DOI: 10.1016/j. patrec.2013.10.010. [3] F. Jiang, C. Wang, Y. Gao, S. Wu, D. Zhao “Discriminating features learning in hand gesture classification”, IET Computer Vision, vol. 9, 2015, no. 5, 673–680. DOI: 10.1049/iet-cvi.2014.0426. [4] P. Garg, N. Aggarwal, S. Sofat, “Vision based hand gesture recognition”, World Academy of Science, Engineering and Technology, vol. 49, 2009, no. 1, 972–977. [5] T. Kapuściński, M. Oszust, M. Wysocki, “Recognition of signed dynamic expressions observed by ToF camera”, Signal Processing: Algorithms,

Journal of Automation, Mobile Robotics & Intelligent Systems

[6]

[7]

[8]

[9]

[10]

[11]

[12]

[13]

[14]

[15]

[16]

[17]

[18]

Architectures, Arrangements, and Applications (SPA), Poznań, 2013, 291–296. T. Kapuściński, M. Oszust, M. Wysocki, D. Warchoł, “Recognition of hand gestures observed by depth cameras”, International Journal of Advanced Robotic Systems, 2015. DOI: 10.5772/60091. C. Keskin, F. Kirac, Y.E. Kara, L. Akarun, “Real time hand pose estimation using depth sensors”. In: IEEE International Conference on Computer Vision Workshops, Barcelona, Spain, 2011, 1228– 1234. DOI: 10.1109/ICCVW.2011.6130391. G. Koszowski, “Two-pass algorithm for hand pose gesture recognition”. In: XIX National Conference of Discrete Processes Automation, Zakopane, Poland, 2014. Y. Li, “Hand gesture recognition using Kinect”, Software Engineering and Service Science (ICSESS), Beijing, 2012, 196–199. DOI: 10.1109/ ICSESS.2012.6269439. G. Marin, F. Dominio, P. Zanuttigh, “Hand gesture recognition with Leap Motion and Kinect devices”. In: IEEE International Conference on Image Processing (ICIP), Paris, 2014, 1565-1569. DOI: 10.1109/ICIP.2014.7025313. J. Molina, A. Escudero-Viñolo, A. Signoriello, M. Pardàs, C. Ferràn, J. Bescós, F. Marqués, J. M. Martínez, “Real-time user independent hand gesture recognition from time-of-flight camera video using static and dynamic models”, Machine Vision and Applications, vol. 24, 2013, no. 1, 187–204. DOI: 10.1007/s00138-011-0364-6. S. R. Oprisescu, “Automatic static hand gesture recognition using ToF cameras”. In: European Signal Processing Conference, Bucharest, 2012, 2748–2751. Z. Ren, J. Junsong, Y. Meng, Z. Zhang, “Robust part-based hand gesture recognition using Kinect sensor”, IEEE Transactions on Multimedia, vol. 15, 2013, no. 5, 1110–1120. DOI: 10.1109/ TMM.2013.2246148. Z. Ren, J. Yuan, Z. Zhang, “Robust hand gesture recognition based on finger-earth mover’s distance with a commodity depth camera”. In: 19th ACM International Conference on Multimedia, Scottsdale, 2011, 1093–1096. R. B. Rusu, N. Blodow, M. Beetz, “Fast point feature histograms (FPFH) for 3D registration”. In: IEEE Conference on Robotics and Automation, Kobe, 2009, 3212–3217. DOI: 10.1109/ROBOT. 2009.5152473. R. B. Rusu, G. Bradski, R. Thibaux, “Fast 3D recognition and pose using the Viewpoint Feature Histogram”. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Taipei, 2010, 2155–2162. DOI: 10.1109/ IROS.2010.5651280. R. B. Rusu, S. Cousins, “3D is here: Point Cloud Library (PCL)”. In: IEEE International Conference, Shanghai, 2011, 1–4. DOI: 10.1109/ICRA.2011. 5980567. R. Rusu, Z. C. Marton, N. Blodow, “Learning informative point classes for the acquisition of object

VOLUME 10,

N° 1

2016

model maps”. In: Conference on Control, Automation, Robotics and Vision, Hanoi, 2008, 643-650. DOI: 10.1109/ICARCV.2008.4795593. [19] D. Uebersax, J. Gall, M. Van den Bergh, “Real-time sign language letter and word recognition from depth data”. In: IEEE International Conference on Computer Vision Workshops (ICCV Workshops), Barcelona, 2011, 383–390. DOI: 10.1109/ICCVW. 2011.6130267. [20] M. Van den Bergh, L. Van Gool, “Combining RGB and ToF cameras for real-time 3D hand gesture interaction”. In: IEEE Workshop on Applications of Computer Vision (WACV), Kona, 2011, 66-72. DOI: 10.1109/WACV.2011.5711485. [21] Y. Wen, H. Chuanyah, Y. Guanghui, W. Changbo, “A robust method of detecting hand gestures using depth sensors”. In: IEEE International Workshop on Haptic Audio Visual Environments and Games (HAVE), Munich, 2012, 72–77. DOI: 10.1109/HAVE.2012.6374441.

Articles

57

Journal of Automation, Mobile Robotics & Intelligent Systems

VOLUME 10,

N° 1

2016

AJVM – Java Virtual Machine Implemented in ActionScript 3.0 Submitted: 5th January 2016; accepted: 2nd February 2016

Arkadiusz Janik, Jakub Krawczyk DOI: 10.14313/JAMRIS_1-2016/8 Abstract: This paper describes the concept and implementation details of AJVM – state-of-the art Java Virtual Machine (JVM) implemented in ActionScript 3.0. Action Script is an objective programming language that supports compilation to Java bytecode. In the presented solution there has been a novel idea utilized – to use the other Virtual Machine’s execution environment (ActionScript Virtual Machine) to build Java Virtual Machine. The subset of features specified in JVM Specification v.2 supported by AJVM has been chosen in a way which enables the machine to be used in many practical applications both in the commercial context as well as in science. As the architecture of AJVM is modular, the extension of its features in the future will not cause any difficulties. The implementation of AJVM in ActionScript 3.0 which is executed by ActionScript Virtual Machine (AVM) makes it possible to use Java code in applications written in ActionScript 3.0. It spawns many new opportunities considering that AVM is a part of FlashPlayer – commonly used multimedia player available in the form of plugins for the majority of modern web browsers, including mobile versions. Keywords: JVM, Action Script, bytecode

1. Introduction Traditional, commercial implementations of JVM Specification aim to increase the performance of a Virtual Machine (VM). As a result VMs are usually implemented in low level programming languages, tightly integrated with Operating System (OS) and hardware (Just In Time compilers that compile bytecode into native code of the hardware platform). The goal of AJVM is to execute Java applications in a variety of computers equipped mainly with a web browser, to increase code reusability by making it possible to use Java libraries in Flash applications and to provide convenient platform for end-users to observe and understand concepts behind JVM. On top of that the solution extends Flash platform with new features: multi-threading, blocking operations, generic classes as well as a new concept of executing bytecode.

1.1. Basic Terms

58

There are two major types of virtual machines: emulators and interpreters [9]. The emulator is a

solution allowing to execute (in an isolated environment) the whole OS and other software designed for a specific platform due to hardware virtualization (VMWare Workstation or Oracle xVM VirtualBox). The interpreter is software capable of executing binary, precompiled code which is an output of a built-in compiler defining its own architecture as of a virtual device [7]. In the further part of this paper the term virtual machine will be used to denote the interpreter.

1.2. Flash Platform and ActionScript 3.0

Adobe Flash technology (previously known as Macromedia Flash) is a multimedia platform enabling new features in web pages: animations, video streaming, interactivity commonly used to implement games, advertisements as well as more sophisticated and complex applications. Flash applications are published as SWF files (Shockwave Flash Object). Usually there is a single file per application even though it may contain many libraries and multimedia resources. In Flash 5 the concept of actions introduced in Flash 4 was extended and, for the first time, Action Script term was used in the context of a programming language [10, 11]. Flash 7 was released together with the second version of ActionScript programming language: such features as type-control (during compilation) and inheritance based on classes were introduced. Flash Players 9 was released with ActionScript 3.0 – the language was redesigned significantly and, to support downgrade compatibility, there are two virtual machines in the player: AVM1 (to support ActionScript 1.0 and 2.0) and AVM2 (to support ActionScript 3.0) [14]. The number of new features have been introduced to the player including performance optimization (hardware acceleration for DirectX and OpenGL), type-control (during code compilation and execution), separation of class-based and prototypebased inheritance, using packages, namespaces and regular expressions, new bytecode format, support for E4X format and others. To sum up: ActionScript 3.0 is object-based, imperative programing language with strong type-control, compiled to bytecode being executed on AVM2 virtual machine, single-threaded (driven by events triggered by Flash Player) so not supporting blocking methods, with automatic memory management (Garbage Collector), not supporting generic classes, not supporting anonymous classes. In the further part of the paper any references to Action Script will refer to Action Script 3.0.

Journal of Automation, Mobile Robotics & Intelligent Systems

2. Related Work There are two major aspects of the contribution of our work: Implementation of Java Virtual Machine in a nonstandard environment Emulation of Java platform inside Flash Player. In the opinion of the authors, there are no solutions other than AJVM that handle both: Action Script and JVM. This makes AJVM a unique system capable of emulating Java platform inside Flash Player. There are number of publications on the subject of non-standard JVMs including the ones mentioned below: Jamiga – the goal of the project is to execute Java applications on Amiga computers [16]. JC – the property of the JVM is a novel approach to executing bytecode. All Java classes are translated on-the-fly to source code in C language and then compiled to native code thus enabling performance similar to traditional Just In Time Compilers (JITs) [17]. Squawk – implementation of JVM for Java ME (Micro Edition) for embedded systems and small, mobile devices. All elements, except for low-level modules supporting I/O operations and OS specific code, were implemented in Java (including Garbage Collector) [15]. GNU Classpath – Java Standard Library distributed under GNU license providing a great base to build own JVMs [20]. JOP – Java Optimized Processor is a hardware implementation of JVM with predictable execution time for embedded real-time systems. Due to the small size of the processor used, it can be implemented in a low cost FPGA (Field-Programmable Gate Array). For low volume systems, the flexibility of an FPGA can be of more importance though slightly more expensive than conventional processors. The processor was designed in VHDL programming language (Very high speed Hardware Description Language). The processor executes bytecode directly, without necessity to compile in-time nor to parse/interpret class files. Using FPGA allows JOP to dynamically declare stack size (which is consistent with JVM –stack-based VM rather than register-based) [21]. Sable VM – is a highly portable and efficient Java virtual machine, using state-of-the-art interpretation techniques. Its goals is to be reasonably small, fast, and compliant with the various specifications (JVM specification, JNI, invocation interface, etc.) [18].

2.1. Emulators of Other Platforms Implemented in the Flash

Even though ActionScript 3.0 is relatively new language there have been several emulators implemented so far: FC64 – Flash Commodore 64 Emulator – FC64 is a low-level, fully functional emulator of Commodore 64 allowing a user to execute applications designed for Commodore 64 as well as to write code in BASIC programming language [22]. FlashZXSpectrum48k – Sinclair ZX Spectrum Emulator} – FlashZXSpectrum48k is a solution similar to FC64. The difference is that it emulates Sinclair ZX Spectrum platform.

VOLUME 10,

N° 1

2016

AminNes – Flash NES Emulator – Flash NES Emulator is an emulator of Nintendo Entertainment System – 8 bits gaming console equipped with 2kB RAM and 2KB video memory. The emulator supports MOS 6502 and guarantees the highest quality of rendered video. Flip8 – CHIP-8 Flash Emulator – Flip8 is an emulator of CHIP-8. CHIP-8 is a virtual machine designed in 70s used to interpret programming language called CHIP-8. It used to be installed on a graphical calculators. Flip8 is an emulator able to execute bytecode including 35 different operations [19].

3. Implementation of JVM in ActionScript 3.0

ActionScript Java Virtual Machine (AJVM) was implemented in Action Script [13]. It implements a subset of features described in the specification of JVM v.2.0. The sections included below present more information on the architecture and implementation problems the authors of this article had to solve.

3.1. Virtual Machine vs Standard Library

The crucial part of the implementation of any Java Virtual Machine is a standard library – the implementation of core Java classes described by JVM specification [4]. Even execution of the simplest Hello Word Java application requires hundreds of core Java classes to be present and loaded into VM. The solution described in the paper contains following elements in terms of a standard library: AJVM library – ActionScript 3.0 library containing executable files of AJVM, ready to be used in any Flash or Flex projects. Flex is a set of components allowing a developer to write RIA applications. It is being compared to technologies such as XUL, JavaFX or Silverlight [12]. ART standard library – an equivalent of Java standard library included in any JVM distributions (such as JRE System Library for JavaSE). A standard library is specific for VM implementation as it expects native implementation of selected methods in the VM. For the purpose of AJVM we implemented a standard library called ART (ActionScript Runtime) which is based on GNU Classpath. As the authors focused on the selected features of JVM, the ART is not a complete implementation of the standard library though it is possible to extend the implementation in future.

3.2. High Level Overview of AJVM

AJVM is available as an object created by the programmer’s code inside his/her Flash application (in the similar way as for Jython interpreter for Python programming language). Consequently, the programmer’s code has a full control over AJVM. Moreover, several isolated instances of AJVM within the single Flash host application can be created. Due to the features of the Flash platform usually the code of Flash host-application and the code of AJVM are distributed in a single SWF file whereas Java classes that are part of ART and Java classes of the programmer’s code will be distributed as JAR archives (or single *.class files) located on the same web server. Articles

59

Journal of Automation, Mobile Robotics & Intelligent Systems

Fig. 1. High level overview of AJVM

60

Figure 1 presents a high level overview of AJVM. As mentioned before, the programmer’s Flash code has a full control over AJVM particularly in the below areas: Starting and stopping AJVM instances – instances of AJVM are independent objects created, started and stopped in moments which are arbitrarily chosen during the lifetime of host Flash application. Configuring classpath – similarly to traditional JVMs, also AJVM needs classpath [3] to be defined to know where to look for compiled Java classes. Unlike traditional JVMs, AJVM does not have access to the file system thus it needs to download class files from a web server. AJVM adds a prefix with the name of its own web server to compose URLs used to locate class code. For instance, if one assumes that AJVM is used in Flash application available under http://domain.server.com/apps/test/index.html and SWF file is available under http://domain.server.com/apps/test/application.swf. Flash host application defined the classpath as java/rt;java/classes and then executed Java class pl.edu.agh.test.Example. It means that AJVM tries to load the class from one of the locations below (until the first successful try). http://domain.server.com/apps/test/java/rt/pl/ edu/agh/test/Example.class http://domain.server.com/apps/test/java/classes/pl/edu/agh/test/Example.class AJVM supports dynamic, on-the-fly addition of new entries to the class path. Specifying main class – Each JVM starts the execution of the user’s code following the method: public static void main(String args[]); defined in the class specified as JVM’s argument. AJVM also expects the name of the main class to be provided (as one of the mandatory elements of the configuration of AJVM). Configuring lifecycle of VM – As mentioned before Flash is a single-threaded environment. The programmer’s code is executed solely in response to events triggered by Flash Player (a mouse click, a keyboard event etc.). It is expected that an event handler gives control back to the Flash Player as soon as possible to guarantee smooth execution. As the consequence, AJVM has to work in very short time slots divided by pauses long enough to handle other no Java events and to render the next frame of Flash animation. A developer of a Flash appliArticles

VOLUME 10,

N° 1

2016

cation defines by themselves (usually statically, when compiling the application) frequency of refreshing the application’s view in a web browser. Flash Player takes all available steps to support the requested FPS (Frames Per Second) parameter. We implemented similar logic for AJVM – a programmer using AJVM defines (as one of the parameters of AJVM configuration) how many milliseconds per frame AJVM can execute before giving the control back to Flash application. AJVM monitors its execution time and breaks the execution of the bytecode immediately when the requested threshold is met. In future we plan to add adaptive mechanism inside AJVM so that it dynamically changes the parameter depending on complexity of Java and/or Flash code and available computational power (for instance to slow down execution of Java code during dynamic animations of a Flash application). Implementing native methods – some methods delivered with Java standard library are native methods. Similarly, user’s methods can also be native. AJVM uses native code to implement ART thus enabling users to declare and implement own native methods – written in ActionScript 3.0. As a result the user’s applications can communicate with Flash applications. Configuring logger – one of the goals of implementing AJVM was to provide platform that can be used by users to experiment with and learn about architecture of VMs. Monitoring is crucial thus we equipped AJVM in easy to use logger supporting four levels: SEVERE, WARNING, INFO and DEBUG.

3.3. Executing Bytecode in AJVM

One of innovations of AJVM is a fully objective approach to VM’s major functionality which is executing bytecode. The overview on the standard location of bytecode execution module in a typical JVM’s architecture can be found in the Figure 2. In Java the bytecode is one of many attributes of a method. In JVM specification bytecode instructions are identified as numbers (0x00 till 0xca). An execution module of a traditional JVM’s works on binary instructions kept in a method area memory and represented in an unchanged way comparing to the compiled bytecode. The only exception is replacing constant pool elements with symbolic references (and, of course JIT compilation to native code). There are

Fig. 2. Location of bytecode execution module in a standard JVM’s architecture

Journal of Automation, Mobile Robotics & Intelligent Systems

Fig. 3. Simplified version of syntactic parse tree for Java classes in AJVM

VOLUME 10,

N° 1

2016

typical graph representation of classes and relations between them. There are following known limitations: Indirect references – Apart from direct relations between a class and its methods (or between a method and its arguments) a traditional syntactic parse tree stores relations between some objects purely by their identifiers. For instance: to retrieve any attributes of a class com.test.B one has to take string “com.test.A” and look for a graph of the class with such a name. Such an approach requires checking whether the object with such a name exists and this is time consuming. Bytecode in a binary representation – A part of a syntactic parse tree – methods’ bytecode – was not parsed at all. Inside instructions of a method there are references to classes, other methods and fields (which are also elements of syntactic parse tree), which remain in a binary representation as long as the bytecode is executed. The above syntactic parse trees can be used to build even more effective representation shown in Figure 3. All relations between represented classes, methods, fields and even external types can be resolved immediately after the first step of class loading process (parsing the class representation) thus making execution of classes’ bytecode more efficient. Building object-oriented representation of the bytecode – As presented on the diagram from the previous section we decided to use object-oriented representation also for the bytecode. In a traditional approach the machine code is represented as a single-dimensional sequence of instructions executed one-by-one. In practice, instruction pointer very often changes in a more complex way due to branch instructions, conditional or unconditional jumps etc. The place in the bytecode that the control should be transferred to (after a branch instruction) is calculated as an offset relative to the current instruction pointer. It means that executing a branch instruction requires calculating the new address and reading the bytecode instruction from that address. The approach

mobile implementations of JVMs utilizing XIP concept (eXecution In Place) thus not copying bytecode to the method area but referring to *.class files whenever the instruction is being executed [6]. There are following implications of the above mentioned models: Necessity to interpret the instruction’s code every time the instruction is executed – in Java’s bytecode one needs to interpret the instruction to understand how many bytes after the instruction are the instruction’s arguments, which is necessary to obtain the beginning of the next instruction. Necessity to derefer pointers to constant pool every time the instruction is executed – the instruction’s arguments can refer to classes, methods or fields addressed as entries in the constant pool (for instance: new, anewarray instructions which allocate objects or arrays, invokevirtual instruction which calls a method etc.). The above approach may lead to a significant overload. For instance: getting a value of a non-static field of an object requires following steps to be executed: Retrieve (from bytecode and using information from an entry in a constant pool) an address of a data structure with the description of the field, Retrieve (from the field’s description) an address of the data structure with the description of the field’s class, Retrieve an offset of the field using information about its class and its fully qualified name, Retrieve from the data structure representing the object of the field the value of the field (using the offset of the fields and a data type of the field). The above mentioned steps are not actioned for JIT compilation (JIT) [8]. The low performance of the standard approach to the bytecode execution and the access to constant pool data motivated us to look for another approach in which the following steps were taken: The construction of objective representation of Java class – Construction of syntactic parse tree, which is an object representation of elements in class file Fig. 4. A sample, full graph of classes and relations between them (see Figure 3). The representation is not a

Articles

61

Journal of Automation, Mobile Robotics & Intelligent Systems

VOLUME 10,

N° 1

2016

Fig. 5. Sample instruction execution graph shown in the context of objects it uses

62

we have taken is using a novel idea: after parsing the field instruction stores the reference to an object of class, there is an instruction execution graph being JVMField class, allowing it to load the current value built for each non-abstract method (see Figure 5). of a field. Operation codes are mapped on classes of respecThis is a decentralized approach: each of more tive instructions. Whenever the instruction needs to than 200 bytecode’s instructions has their implemenbe executed, the object’s method is executed with the tation (in the execute() method) thus the conceptual instruction’s arguments. Each object has the knowlschema of AJVM (see Figure 8) differs significantly edge (stored as reference to) about objects reprefrom the traditional implementations of JVM. senting classes, fields and methods that are required The bytecode execution module in AJVM is virtualby the instruction. Similarly, there are references to ized. The module is created by instructions of loaded a consecutive instruction (or to consecutive instrucbytecode expressed as objects. It means that AJVM tions for branch instructions) stored in the object. neither contains nor uses binary representation of The method stores the reference to the first instrucbytecode. There are following advantages of such an tion from its method whereas the bytecode stores the approach: reference to its method so that it knows where to give One-time interpretation of bytecode – Binary control back when returning. representation of bytecode is read by AJVM once, Delegation of executing an instruction to its when the class is loaded and the instruction graph is object representation – Using an objective nature of built. ActionScript 3.0 language one can create full hierarNovel implementation of reflection – All classchy of classes for Java instructions. A part of such a es, methods, fields and their attributes have objective hierarchy is presented in Figure 6. representations thus the implementation of reflection Each instruction implement method execute() declared in Instruction interface (design pattern Command). An argument of the method is object FrameExecutionContext, which represents a single stack frame. The object’s diagram can be found in Figure 7. FrameExecutionContext object provides access to computational stack and current values of local variables thus allowing an instruction to execute (itself) in the current context. After execution in the way specific for the instruction (for instance: pop two values from the stack, add them and push on to stack for iadd instruction) JVM has to set up the next instruction to be executed. As a result, conditional instructions can control their thread’s execution. An instruction can use additional references passed when the class was parsed and the graph was built. For instance: class JgetFig. 6. A part of a diagram showing objective hierarchy of Java fieldInstruction, which is representing getinstructions Articles

Journal of Automation, Mobile Robotics & Intelligent Systems

VOLUME 10,

N° 1

2016

does not require referencing native code. Clear and flexible architecture – Inspection of the instruction graph as well as dependencies between classes make possible to extend AJVM in the future with interesting features such as dynamic code transformation or visualization of code execution which can be used to analyze and optimize code performance.

4. Additional Remarks on AJVM Implementation 4.1. Implementing Exception Handlers Fig. 7. A part of a diagram of the most important classes of AJVM execution module

Fig. 8. Bytecode execution module in AJVM

One of the attributes of a compiled class is an exception handlers table. The table is used to store information about the class of an exception being handled, the first instruction being protected by the handler, the offset of the last instruction being protected and the offset of the first instruction of the exception handling procedure. In AJVM there are no instruction offsets at all, as they are replaced by the instruction graph. Consequently exception handlers have to be defined in an objective way. Each method has a collection of objects representing exception handlers (the list may be empty for methods without try-catch blocks). An object representing an exception handler stores references to all instructions being protected (as it is impossible to represent the set as the first instruction and the offset). The idea is presented in Figure 9.

4.2. Garbage Collection

Fig. 9. An objective representation of an exception handler table for a method

We solved the problem of Automatic Memory Management (AAM [5]) using features of ActionScript 3.0 which is equipped with AMM. Flash VM (AVM2) is equipped with Garbage Collector thus the easiest way of solving GC related problems is to: • Resign from our own heap in AJVM – our JVM does not use heap at all. • Encapsulate each and every Java object inside native ActionScript objects – it is possible due to representing each Java object as an instance of class JVMObject defined in ActionScript 3.0 allocated on Action Script’s AVM2 heap and managed by AVM2. Each Articles

63

Journal of Automation, Mobile Robotics & Intelligent Systems

Java object is encapsulated in ActionScript 3.0 object. Each Action Script’s JVMObject stores a reference to the object representing the class of the Java object and a data structure to store the Java object’s state (value of all fields from the object’s class). • Utilizing AVM2 Garbage Collector – as Java objects inside AJVM are represented by ActionScript’s object they are managed by AVM2’s Garbage Collector. The only limitation is not to store any additional, external references to these objects even though it may be useful for the purpose of VM monitoring etc.

4.3. Multithreading

The traditional implementation of JVMs provides multithreading through the mapping of Java threads onto native threads (on the Operating System level) [1, 2]. AJVM does not have access to low level OS threads as it is limited by Flash Player it is run in and which executes ActionScript 3.0’s code just inside the eventhandler’s thread. Consequently, it was necessary to implement the multithreading model of our own design at the level of application. We had to use an objective representation of a thread as a part of VM. An instance of AJVM holds the collection of instances of JVMThread class. New objects are added to the collection every time a bytecode instruction executes java.lang.Thread.start() method, and are removed while returning from java.lang.Thread.run() method. As soon as the collection is empty the AJVM stops. AJVM works in cycles and each cycle is about executing an amount of bytecode’s instructions of each active thread. The length of AJVM’s cycle (measured as a period of time as opposed to a number of instructions) is AJVM configuration item. The available time of each cycle is split between all threads accordingly to their priorities. The simplest way to meet the above requirements is to implement RoundRobin algorithm with the handling of priorities. Each AJVM cycle consists of/requires the following steps: Calculate the possible time of the end of the cycle endTime (current time + configured cycle time). If endTime is reached, give the control back to event handling procedure (thus finishing the cycle). For each thread in the collection: Execute method step() a number of times basing calculations on the priority of threads. If the thread ends(execution stack is empty), remove the thread from the collection. Go back to step 2. Obviously, the above algorithm does not apply to “blocked” or “waiting for monitor” (monitorenter instruction) threads.

4.4. Native Interface

64

Native interface of AJVM allows a user to provide their implementation of Java classes as functions of ActionScript 3.0. It is done during VM initialization thanks to the method which binds a native method (its class, name and signature) with ActionScript 3.0 object that represents the closure of the function (Callback design pattern). Figure 10 demonstrates sample implementation and the registration of native method sqrt) in VM’s case. Articles

VOLUME 10,

N° 1

2016

Fig. 10. A sample code used to register a native method in AJVM Arguments of the function are used to pass following elements from native interface of AJVM to the user’s method: • The context of the current execution frame in the execution stack frame – the reference to the case of FrameExecutionContext class that gives the access to properties of the current Java thread as well as to the stack and the local area of the stack frame. It allows the native method to use information that is forbidden for Java code (for instance to implement reflection with the use of the objective representation of classes and methods). • The values of arguments passed to Java method. args contains 0 or more arguments passed to the native method by the Java code. Primitive types are mapped onto corresponding types of ActionScript 3.0 whereas objects are represented by their encapsulations (JVMObject) – in the native code one can easily get their classes and access fields (also private) in a way similar to the one used in the reflection. • Callback that returns from native method – even though keyword return is present in ActionScript 3.0, employing it for returning from function would significantly reduce the capabilities of native methods. Each native implementation would have to give control back to AJVM soon after getting it as Flash Player uses a single thread. Implementing return functionality in the described way allows a developer to implement blocking methods (for instance input/output operations). A thread calling a native method is blocked until the callback is executed. In practice it means that even void methods (not returning anything) should contain onReturn(null) call at the end.

5. Test Cases

The output of the implementation part of our work includes an ActionScript 3.0 library implementing the most important features of Java Virtual Machine. The quality of the Virtual Machine can be measured with use of many indicators including the two most important: compatibility with JVM specification version 2 and performance. As was mentioned at the beginning of this paper, the full compliance with JVM specification was not our objective. Instead, we focused on those parts of Standard Java Library that allow a developer to create the majority of typical Java applications. Neither high performance was the goal of AJVM. However, it is worth comparing the implementation of AJVM with the most popular JVM – Oracle HotSpot.

Journal of Automation, Mobile Robotics & Intelligent Systems

VOLUME 10,

N° 1

2016

ecution time increases linearly in a function of array size for both: AJVM and HotSpot virtual machines.

5.3. Test Case 3 – Sorting of Linked List of java.lang. Comparable Objects

Fig. 11. Execution time of the test 1 – performing fixed point arithmetic

The test was to call java.util.Collections.sort() method from the standard library to sort linked list (java.util.LinkedList) containing objects implementing compareTo() method. The test validates pointer operations and virtual methods (unlike sorting arrays of primitive types). The results can be found in Figure 13.

The following testing platform was used: (VM: Java HotSpot(TM) 64-Bit Server VM (build 21.0-b17, mixed mode, library: Java(TM) SE Runtime Environment (build 1.7.0-b147). AJVM (plugin) – VM: AJVM, reference version, library: ART, reference version, Flash Player: 11.2.202.233, ActiveX plugin, Internet Explorer 8. All tests have been executed on Intel(R) Core(TM) i7-2630QM @ 2.0 GHz, 8 GB RAM, System Windows 7 64-bit.

Fig. 13. Execution time of the test 3 – sorting linked lists of comparable objects

5.1. Test Case 1 – Fixed Point Arithmetic

The test was to iterate through an array of integers and perform several calculations on each element of the array. The results are presented in Figure 11. Test results leave no doubts with regards to the differences between commercial HotSpot VM and experimental, research AJVM. Due to Just In Time compilation used in HotSpot the code is compiled to native representation and executed on a physical processor rather than on a virtual machine. As a result, calculations are done significantly faster.

5.2. Test Case 2 – Sorting of Floating Point Numbers

One can see an interesting observation here: even though both tests; test 2 and test 3 are about sorting of a collection, the increase of execution time in test 3 is significantly greater than in test 2. The reason is that in test 3 objects are sorted rather than numbers which results in more object operations (compare() method calls). And pointer operations are less efficient in AJVM than in HotSpot.

5.4. Test case 4 – Network Connections and Object Deserialization

The test was to call java.util.Arrays.sort() method of the standard library to sort an array of double numbers. The results can be found in Figure 12.

The aim of the test was to establish a network connection with a remote server with the use of java.net. Socket so as to obtain some data and deserialize it. The results can be found in Figure 14.

Fig. 12. Execution time of the test 2 – sorting arrays of fixed point numbers

Fig. 14. Execution time of the test 4 – transferring data through network connection and deserializing it

Again HotSpot VM beats AJVM and again the major factor is JIT implemented in AJVM. Another important factor is that source code for arithmetic in HotSpot is written in C which makes development optimization techniques possible. It is worth mentioning that ex-

Interestingly, the difference between execution time between AJVM and HotSpot is less visible and do not exceeds 35%. The reason is that in the test external, system resources are used (network connection) intensely. In other words: it is not so important Articles

65

Journal of Automation, Mobile Robotics & Intelligent Systems

how elements of a virtual machine are implemented as most operations are done on a level of OS. In other words, a proportion of input/output operations to other operations is higher than in previous tests.

5.5. Test Case 5 – Massive Multithreading

The test consisted of launching the number of concurrent threads, executing them and waiting for the last one to end. The results are illustrated in Figure 15.

Fig. 15. Execution time of the test 5 – executing significant amount of concurrent threads The grey bar in the Figure 15 for HotSpot indicates missing results – it was impossible to start so many threads. The major factor that contributes to smaller execution time in AJVM comparing to HotSpot is a different threading model used in both solutions. HotSpot, like many other VMs maps Java threads to native OS threads. As a result thread creation is time-consuming operation. When there is a huge amount of threads OS may behave unstable. AJVM emulates multithreading programmatically – creating and starting a new thread is as expensive as creating one more Java object.

5.6. Test Cases Summary

66

The conclusions drawn from the executed test cases are as follows: Tests 1, 2 and 3 leave no doubts with regards to the differences between commercial HotSpot VM and experimental, research AJVM. Due to Just In Time compilation used in HotSpot the code is compiled to native representation and executed on a physical processor rather than on a virtual machine. As a result, calculations are done significantly faster. Test 4 shows more balanced results due to the significant influence of input/output operations in comparison with the execution time. Input/output operations are less CPU demanding thus AJVM is more effective. The results prove clearly that in some applications AJVM can be used with success. Test 5 provides solid evidence in favour of the existence of domains on which AJVM is more robust than traditional VMs. The benefit of AJVM is the way thread creation is implemented. Having said that it must be emphasized that efficient creation of a thread does not mean that all thread operations are more effective on AJVM. As it was mentioned before test 5 focuses only on thread creation rather than on measuring all thread-related operations (creation, sleeping, waking up, thread scheduling etc.). Articles

VOLUME 10,

N° 1

2016

5.7. Samples of AJVM Utilization This section demonstrates sample applications of AJVM. It describes how AJVM can be used to execute sample Internet applications – multiuser, text chat room.

Fig. 16. The architecture of chatroom application executed on AJVM The architecture shown in Figure 16 contains the following elements: • HTTP server – any web server to provide static files via http protocol. • Chatroom server – server implemented in Java for real-time, multi user chat room (Java Object Serialization is used). • Chatroom client – client application written in Flash using Flex components. AJVM is applied to implement networking functionality (object (de) serialization and transfer) as well as implement business logic (logging in, authorizing and validating data). It is worth mentioning that the above sample uses Java code shared between the client and server applications: the same classes are loaded by HotSpot JVM (server side) and AJVM (client side). Once SWF file (Flash application) and Java classes are loaded, HTTP protocol is no longer used and there is a single network communication (TCP) established.

6. Conclusions

Authors of this paper successfully implemented Java Virtual Machine in ActionScript 3.0. As a result it was possible to use elements of existing elements of a virtual machine’s infrastructure (FlashPlayer) to build your own solution. It allowed authors to focus on selected modules of a virtual machine (like implementation of a novel concept of bytecode execution engine) and to use existing ones which are beyond the scope of interest (GC, threading model etc.). Selection of an environment that supports Java bytecode (Action Script 3.0 and Flash Player) spawns many opportunities considering the fact that Flash Player is commonly used in most of modern web browsers. As shown in the previous section, AJVM can be successfully used in implementing various applications, including Internet ones. There are many advantages of using AJVM including: High reusability of classes

Journal of Automation, Mobile Robotics & Intelligent Systems

Client and server elements of an Internet application usually use the same data structures, similar validation rules (for input data) as well as some common elements of business logic. In a traditional solution many of these common features are implemented twice (often using different implementation platforms): on each side. It is a developer’s responsibility to guarantee consistency of both parts of the application. The use of AJVM enables developers to use the same Java code at the client’s and server’s side. It is a huge time-saver and the way to reduce a number of errors. Providing lightweight implementation of Java Virtual Machine together with an application Even though using Java applets meet the previously mentioned criteria (high reusability of classes), it is worth mentioning that Flash Player is much lighter and more popular than Java plugins for web browsers. AJVM makes it possible to execute Java code by users who do not intend to install Java Runtime Environment neither additional plugins to web browsers. End users do not have to be even aware of JVM running on their computers. AJVM is extremely lightweight – the size of compiled AJVM is less than 100kb. Eliminating limitations of traditional clients Client applications are usually running in singlethreaded environment and thus use only a single thread (designed to support user’s interface). Good examples are JavaScript, HTML5 and Flash. AJVM allows to use (on a client side) additional computational models: multi-threading, generics etc. Of course, the performance of reference implementation of AJVM requires ActionScript 3.0 code to be responsible for such tasks as: UI rendering, multimedia streaming, and CPU intensive computations. Executed test cases shows clearly that the current implementation is missing optimization techniques which speed up bytecode execution, particularly Just In Time compilation. Consequently, the efficiency of JVM is lower comparing to traditional, commercial VMs (represented by HotSpot in test cases executed). The difference of efficiency ranges from ~35% for test cases using intensively external resources (less operations on VM-level comparing to a number of operations on OS-level) to ~12 800% for others. The exception is some specific situations (described in test case 5) which reveals dominance of AJVM. The future work on AJVM includes the extension of the compliance with the reference specification of JVM (for instance providing bytecode verification during class loading). One should also remember that in order to seriously think about practical application of AJVM it should be made compliant with Java 7 or at least Java 6 standard. Another significant work to be done is further development of Java Standard Library delivered with JVM (to provide full compliance with Oracle implementation or full compliance with GNU Classpath). Another part of the research will concern increasing the performance. There are many options including Just-In-Time compilation. Also, the novel bytecode execution model opens new possibilities related to

VOLUME 10,

N° 1

2016

on-the-fly code transformation, code injection during runtime, and more efficient and precise code profiling. As it was mentioned a couple of times, AJVM is designed in a modular way so that the level of dependency on external VM (Flash Player) can be reduced to some extent. For instance, it is possible to implement own Garbage Collector module and replace the one provided by Flash container. However, due to module dependencies, replacing a module may result in being forced to replace others as well. GC module mentioned above is a good example. In the current implementation objects creation and management is controlled by Flash at all. In order to replace GC both domains would have to be delivered.

Source code of AJVM can be accessed here: http://galaxy.uci.agh.edu.pl/~ajanik/AJVM_source_ files.zip

AUTHORS

Arkadiusz Janik* – AGH University of Science and Technology, Faculty of Computer Science, Electronics and Telecommunications, Department of Computer Science, al. Mickiewicza 30, 30-059 Kraków, Poland. E-mail: arkadiusz.janik@agh.edu.pl Jakub Krawczyk – AGH University of Science and Technology, Faculty of Computer Science, Electronics and Telecommunications, Department of Computer Science, al. Mickiewicza 30, 30-059 Kraków, Poland. *Corresponding author

REFERENCES [1] Lindholm T., Yellin, F., The Java Virtual Machine Specification, 2nd Edition, Addison-Wesley, 1999. [2] Gosling J., Joy B., The Java Language Specification, Addison-Wesley, 1996. [3] Chan P., Lee R., The Java Class Libraries: An Annotated Reference, Addison-Wesley, 1997. [4] Naughton P., Morrison M., The Java Handbook, Osborne/McGraw-Hill, 1996. [5] Venners B., Inside the Java 2 Virtual Machine, McGraw-Hil, 2000. [6] Downing T., Meyer J., The Java Virtual Machine, O’Reilly Media, 1997. [7] Craig I., Virtual Machines, Springer, 2005. [8] Stark R., Java and the Java Virtual Machine: Definition, Verification, Validation, Springer, 2001. [9] Barrio V., Fernandez A., Study of the techniques for emulation programming, Universidad Politecnica de Catalunya, 2001. [10] Braunstein, R., ActionScript 3.0 Bible”, Wiley, 2010. [11] ActionScript Virtual Machine 2 (AVM2) Overview, Adobe Systems Incorporated, 2007. [12] Gassner D., Flash Builder 4 and Flex 4 Bible, Wiley, 2010. Articles

67

Journal of Automation, Mobile Robotics & Intelligent Systems

[13] Elst P., Object-Oriented ActionScript 3.0, friendsofED, 2007. [14] Adobe Flash Player Technology Breakdown. http://www.adobe.com/products/player\_census/ flashplayer/tech\_breakdown.html accessed on 1st Oct 2015. [15] Simon D., Cifuentes C., The squawk virtual machine: Java on the bare metal, ACM, 2005. [16] JAmiga VM homepage, 2014. http://jamiga2.blogspot.com/ accessed on 15th July 2015. [17] JC Virtual Machine homepage, 2013. http://jcvm.sourceforge.net/ accessed on 15th July 2015. [18] Pickett C., Verbrugge C., Return Value Prediction in a Java Virtual Machine, VPW2, 2004. [19] Flip8 – CHIP-8 Flash Emulator homepage, 2014. http://sourceforge.net/projects/flip8/accessed on 21st Sep 2015. [20] GNU Classpath homepage, 2014. http://jcvm.sourceforge.net/ accessed on 21st Sep 2015. [21] Schoeber, M., “JOP: A Java Optimized Processor for Embedded Real-Time Systems”, VDM Verlag Dr. Müller, 2008. [22] FC64 – Flash Commodore 64 Emulator homepage, 2014. http://codeazur.com.br/stuff/fc64\_final/accessed on 2nd Apr 2015.

68

Articles

VOLUME 10,

N° 1

2016

Journal of Automation, Mobile Robotics & Intelligent Systems

VOLUME 10,

N° 1

2016

A new Integer Linear Programming and Quadratically Constrained Quadratic Programming Formulation for Vertex Bisection Minimization Problem Submitted: 11th July 2015; accepted: 10th February 2016

Pallavi Jain, Gur Saran, Kamal Srivastava DOI: 10.14313/JAMRIS_1-2016/9 Abstract: Vertex Bisection Minimization problem (VBMP) consists of partitioning a vertex set V of graph G = (V, such that E) into two sets B and B′ where ∣B∣ = vertex width (VW) is minimized where vertex width is defined as the number of vertices in B which are adjacent to at least one vertex in B′. It is an NP-complete problem in general. VBMP has applications in fault tolerance and is related to the complexity of sending messages to processors in interconnection networks via vertex disjoint paths. In this paper, we have proposed a new integer linear programming (ILP) and quadratically constrained quadratic programming (QCQP) formulation for VBMP. Both of them require number of variables and constraints lesser than existing ILPs and QCQP. We have also implemented ILP and obtained optimal results for various classes of graphs. The result of the experiments with the benchmark graphs shows that the proposed model outperforms the state of the art. Moreover, proposed model obtains optimal result for all the benchmark graphs. Keywords: Vertex Bisection Minimization, Integer Linear Programming, Quadratic Programming

1. Introduction Vertex Bisection Minimization problem (VBMP) consists of partitioning a vertex set of a graph G = (V, E) where |V| = n into two sets B and B′ where ∣B∣ =  such that vertex width (VW) is minimized where vertex width is defined as the number of vertices in B which are adjacent to at least one vertex in B′. Mathematically, VW = ∣{u∈ B:∃v ∈ B’ ∧ (u,v) ∈ E(G)}∣ [1]. It can also be defined as a graph layout problem. Graph layout problems are a class of combinatorial optimization problems whose goal is to find a layout of an input graph G to optimize a certain objective function [2]. A linear layout or layout of an undirected graph G = (V, E) is the bijective function ϕ: V → [n] = {1,2,…, n} [2, 3]. Set of all layouts is denoted by ϕ(G). Vertex Bisection Minimization Problem consists of finding a layout Φ* ∈ Φ(G) of a graph G = (V, E) which minimizes the vertex width (VW) where VW = δ( , Φ, G) for a layout j where, δ(i, ϕ, G) = ∣{u ∈ L(i, ϕ, G):∃v ∈ R(i, ϕ, G) ∧ (u,v) ∈ E(G)}∣, L(i, ϕ, G) = {u ∈ V : ϕ (u) ≤i} and R(i, ϕ, G) = {u ∈ V : ϕ (u) >i} [2,3]. Vertex Bisection Minimization Problem is relevant to fault tolerance and is related to the

complexity of sending messages to processors in interconnection networks via vertex disjoint paths [2]. Brandes and Fleisher [1] proved that Vertex Bisection minimization problem is NP-complete in general but vertex bisection minimization is polynomially solvable for trees and hypercubes. Fraire et al. [4] have proposed two ILP models and one QCQP for VBMP. In this paper, we have proposed a 0-1 ILP model and two QCQP models which requires fewer number of variables and constraints than the one proposed by [4]. When solving a problem using Integer Programming (IP) where all the involved variables are integer variables, we first need to develop an IP formulation involving: (a) an objective function which must be either maximized or minimized, (b) definition of the variables used and (c) a set of the associated constraints [5]. In ILP, the objective function and the constraints are restricted to be linear. 0-1 integer linear programming is a special case in which unknowns are binary [6]. A large number of problems have been formulated as ILP such as Travelling salesman, Vertex Cover and other covering problems, Set Packing and other Packing problems, Max Bisection Minimization problem, Vertex Separation Minimization problem, Cutwidth Minimization Problem, bandwidth minimization problem, MinLA, etc. [5–10]. Quadratically constrained quadratic programming (QCQP) is an optimization problem in which both the objective functions and the constraints are quadratic functions [5]. In this paper, Section 2 describes the 0-1 integer linear programming model for VBMP. Section 3 is devoted to the results of VBMP obtained using ILP for various classes of graphs such as Small graphs (http://www.optsicom.es/cutwidth), 2-dimensional ordinary meshes, 2-dimensional cylindrical meshes and 2-dimensional toroidal meshes. We have also proposed two quadratically constrained quadratic programming formulation for VBMP which is described in Section 4. Section 5 is devoted to conclusions.

2. 0-1 Integer Linear Programming for VBMP

We first define the variables. These variables take value 0 or 1.

2.1. Variable xi

This binary variable indicates whether the vertex is placed in B or B′.

69

Journal of Automation, Mobile Robotics & Intelligent Systems

VOLUME 10,

2.2. Variable vij

This variable indicates that for an edge (i,j) whether ′ or not. vij = 1 if i is placed in B and j is placed in B′ otherwise 0. It is defined as follows:  ,

2.3. Variable zi

∀

Variable zi indicates that whether the vertex i contribute to the vertex width or not. zi = 1 if vij = 1 for some j ∈ {1,2,…, n}\i where n is the order of graph G otherwise 0, ∀i ∈ V (G). It is defined as follows:

2.4. ILP Formulation The following integer linear formulation is proposed for VBMP.

programming

zi, xi, vij ∈ {0,1}

3. Computational Experiments for VBMP

This section describes the computational experiments performed to test the efficacy of 0-1 integer linear programming formulated by us for VBMP. We have solved 0-1 ILP using CPLEX 12.6 using multiple threads on an Intel Xeon 12 core CPU. We have carried out experiments on the benchmark graphs in [4] which consists of Small graphs, Grid, Trees and Harwell-Boeing graphs. These instances are available at www.optsicom.es/vsp/#instances. Table 1-4 shows the optimal results achieved and the time taken to obtain the result. As can be seen in the table, the results are obtained in a very short time for all the graphs.

4. Quadratic Programming Formulation

In the QP formulation, xi is defined in the similar manner as in Section 2. Variable vi indicates that whether the vertex i ∈ V(G) contribute to the vertex width or not. It is defined as follows:

(1)

70

(3)

(4)

(5)

(6)

Constraint (1) guarantees that ∣B∣ = . Constraints (2) – (4) represents that . If xi = 0, then vij = 0 using constraint (2). If xi =1 and xj = 0, then from using constraints (2) and (3) vij may be 0 or 1 but constraint (4) guarantees that vij = 1 not 0. If xi =1 and xj = 1, vij = 0 using constraint (3). Constraints (5) – (6) represents zi = ⋁(i,j)∈E(G) vij . If vij = 0 for all (i, j) ∈ E(G), where i is fixed, then constraint (5) gives zi = 0 or 1 but constraint (6) gives that zi = 0 not 1. If vij = 1 for some (i, j) ∈ E(G), where i is fixed, then constraint (5) gives zi = 1 and equation (6) also satisfies it. Let n and m Articles

4.1. {0,1} Quadratic Programming Formulation for VBMP

2016

be the order of graph G and the number of edges in G respectively. It can be observed that there are 2(n+m) 0-1 variables and 8m+n constraints. In the ILP1 and ILP2 proposed by Fraire et al. [4], number of variables and constraints are O(mn2) and O(mn2), respectively.

(2)

N° 1

(1)

(2)

(3)

In above quadratic programming, equation (1) guarantees that ∣B∣ = . Equation (2) and (3) represents that vi = ⋁(i,j)∈E(G)xi .  . If xi = 0, then vi = 0 or 1 from equation (2) but since left hand side of equation (3) is 0, therefore equation (3) gives that vi = 0 not 1. If xi = 1 and xj = 0, then from equation (2) vi = 1. If xi = 1 and xj = 1, then from equation (2) vi = 0 or 1. If xi = 1 and xj = 0 for some (i,j)∈E(G), then equation (3) gives that vi = 1. If xi = 1 and xj = 1 for all (i,j)∈E(G), then equation (3) gives that vi = 0. Let n and m be the order of graph G and the number of edges in G respectively. It can be observed that there are O(n) 0-1 variables and O(n.Δ(G)) constraints while in Fraire et al. [4] number of variables and constraints both are equal to O(m.n2).

Journal of Automation, Mobile Robotics & Intelligent Systems

VOLUME 10,

N째 1

2016

Table 1. Small Graphs Graphs

Optimal Result

Time (ms)

Graphs

Optimal Result

Time (ms)

p17_16_24

3

57

p59_20_23

2

47

p19_16_19

2

55

p61_21_22

2

55

p18_16_21

p20_16_18

p21_17_20

p22_17_19

p23_17_23

p24_17_29

p25_17_20

p26_17_19

p27_17_19

p28_17_18

p29_17_18

p30_17_19

p31_18_21

p32_18_20

p33_18_21

p34_18_21

p35_18_19

p36_18_20

p37_18_20

p38_18_19

p39_18_19

p40_18_32

p41_19_20

p42_19_24

p43_19_22

p44_19_25

p45_19_25

p46_19_20

p47_19_21

p48_19_21

P49_19_22

P50_19_25 p51_20_28

p52_20_27

p53_20_22

p54_20_28

p55_20_24

p56_20_23

p57_20_24

p58_20_21

2

2

2

2

2

3

2

2

2

2

1

2

2

2

3

2

2

2

2

2

2

4

1

3

2

3

2

2

2

2

2

2

60

46

83

39

84

57

91

51

46

58

51

78

63

57

54

33

64

90

39

71

66

60

63

66

42

82

61

61

52

36

52

49

4

103

2

45

2

3

2

3

2

2

87

69

63

68

47

55

p60_20_22

p62_21_30

p63_21_42

p64_21_22

p65_21_24

p66_21_28

p67_21_22

p68_21_27

p69_21_23

p70_21_25

p71_22_29

p72_22_49

p73_22_29

p74_22_30

p75_22_25

p76_22_30

p77_22_37

p78_22_31

p79_22_29

p80_22_30

p81_23_46

p82_23_24

p83_23_24

p84_23_26

p85_23_26

p86_23_24

p87_23_30

p88_23_26

p89_23_27

p90_23_35

p91_24_33

p92_24_26

p93_24_27

p94_24_31

p95_24_27

p96_24_27

p97_24_26

p98_24_29

p99_24_27

p100_24_34

2

3

53

61

5

200

2

77

2

3

2

3

2

3

3

60

86

70

62

53

55

61

5

198

3

81

2

2

2

64

48

61

4

155

3

53

3

3

47

55

6

175

1

49

2

2

1

2

59

58

74

40

3

108

3

71

2

3

3

2

2

82

60

57

45

49

3

115

2

65

2

2

2

2

3

61

55

69

72

94

Articles

71

Journal of Automation, Mobile Robotics & Intelligent Systems

VOLUME 10,

N° 1

2016

Table 2. Grids Graphs

Optimal Result

Time (ms)

Graphs

Optimal Result

Time (ms)

Grid3

3

53

Grid4

4

82

Grid 7

7

348

Grid5

5

247

Grid6

6

258

Table 3. Trees Graphs

Optimal Result

Time (ms)

Graphs

Optimal Result

Time (ms)

1rot_Tree_22_3_rot1.mtx

2

94

2rot_Tree_22_3_rot1.mtx

2

57

1rot_Tree_22_3_rot3.mtx

2

61

2rot_Tree_22_3_rot3.mtx

2

52

1rot_Tree_22_3_rot2.mtx

1rot_Tree_22_3_rot4.mtx

1rot_Tree_22_3_rot5.mtx

2

100

2

87

2

3rot_Tree_22_3_rot1.mtx

55

2

3rot_Tree_22_3_rot2.mtx

72

2

3rot_Tree_22_3_rot5.mtx

91

22

52

2rot_Tree_22_3_rot2.mtx

2

2rot_Tree_22_3_rot4.mtx

68

2

2rot_Tree_22_3_rot5.mtx

67

2

3rot_Tree_22_3_rot3.mtx

95

2

3rot_Tree_22_3_rot4.mtx

67

2

68

Table 4. Harwell-Boeing Graphs Graphs

Optimal Result

Time (ms)

Graphs

Optimal Result

Time (ms)

bcspwr01

3

98

bcspwr02

2

92

bcsstk01

12

1909

can24

4.2. {-1,1} Quadratic Programming Formulation for VBMP xk, vk ∈ {-1, 1}

–1

72

(1)

(2)

(3)

xi = 1 if i ∈ B otherwise xi = -1

(4)

(5)

Equation (1) and (2) guarantees the bipartition of the vertex set. If xi = 1 and xj= 1, (3) gives vi= 1 or Articles

4

88

-1. If for all (i, j) ∈ E(G) where i is fixed, xj = 1 then vi = -1 using (4) and (5) also satisfies it. If xi = 1 and xj = -1, (3) gives vi = 1 or -1. (4) also implies vi= 1 or -1 and (5) ensures that vi= 1. If xi = -1 and xj = 1, (3) gives vi = -1. (4) and (5) also satisfies it. If xi= -1 and xj= -1, (3) gives vi= -1. (4) and (5) also satisfies it. Let n and m be the order of graph G and the number of edges in G respectively. It can be observed that in this formulation also there are O(n) {-1,1} variables and O(n.Δ(G)) constraints.

5. Conclusions

In this paper, a 0-1 Integer linear programming model and two quadratically constrained quadratic programming models for the Vertex Bisection problem have been proposed. These formulations involve fewer number of variables and constraints than the earlier ILPs and QP which have been proposed in the literature. We have also experimented with the ILP for the Vertex Bisection Minimization problem. It can be observed that, optimal results are achieved very quickly for all the benchmark graphs that were experimented on.

Journal of Automation, Mobile Robotics & Intelligent Systems

VOLUME 10,

N° 1

2016

ACKNOWLEDGMENTS

We thank Dr. Volkmar Sauerland, ChristianAlbrechts-Universitat zu Kiel for CPLEX runs. The first author also acknowledges UGC for MANF fellowship [MANF-2014-15-JAI-UTT-38830].

AUTHORS

Pallavi Jain*, Gur Saran, Kamal Srivastava – Department of Mathematics, Dayalbagh Educational Institute, Agra, India. Tel: +91-9690000126. E-mail: pallavijain.t.cms@gmail.com. *Corresponding author

REFERENCES

[1] Brandes U., Fleischer D., “Vertex Bisection is Hard, too”, Journal of Graph Algorithms and Applications, vol. 13, no. 2, 2009, 119–131. DOI: 10.7155/jgaa.00179. [2] Diaz J., Petit J., Serna M., “A Survey of Graph Layout Problems”, ACM Computing Surveys, vol. 34, no. 3, 2002, 313–356. DOI: 10.1145/568522.568523. [3] Petit J., “Addenda to the Survey of Layout Problems”, Bulletin of the European Association for Theoretical Computer Science (EATCS), no. 105, 2011, 177–201. [4] Fraire H., David J., Villanueva T., et al., “Exact Methods for the Vertex Bisection Problem“. In: Recent Advances on Hybrid Approaches for Designing Intelligent Systems, ed. by: O. Castillo, P. Melin, W. Pedrycz, J. Kacprzyk, chapter 40, series: Studies in Computational Intelligence vol. 547, 2014, 567–577. DOI: 10.1007/978-3-319-05170-3_40. [5] Vazirani V.V., Approximation Algorithms, Springer 2003. DOI: 10.1007/978-3-662-04565-7. [6] Diaby M., “The Travelling Salesman Problem: Linear Programming Formulation”, WSEAS Transactions on Mathematics, no. 6, 2007, 745–754. [7] Frieze A., Jerrum M., “Improved Approximation Algorithms for Max k-cut and Max Bisection“, Algorithmica, vol. 18, no. 1, 1997, 67–81. DOI: 10.1007/BF02523688. [8] Duarte A., Escudero L.F., Marti R., et al., “Variable Neighborhood Search for the Vertex Separation Problem“, Computers and Operations Research, vol. 39, no. 12, 2012, 3247–3255. DOI: 10.1016/j. cor.2012.04.017. [9] Loces M.C.L., Garcia N.C., Huacuja H.J.F., et al., “A New Integer Linear Programming Model for the Cutwidth Minimization Problem of a Connected Undirected Graph“, In: Recent Advances on Hybrid Approaches for Designing Intelligent Systems, ed. by: O. Castillo, P. Melin, W. Pedrycz, J. Kacprzyk, chapter 40, series: Studies in Computational Intelligence vol. 547, 2014, 509–517. DOI: 10.1007/978-3-319-05170-3_35. [10] Luttamaguzi J., Pelsmajer, M., Shen, Z., Yang, B., Integer Programming Solutions for Several Optimization Problems in Graph Theory. Technical report, DIMACS 2005.

Articles

73

Turn static files into dynamic content formats.

Create a flipbook
Issuu converts static files into: digital portfolios, online yearbooks, online catalogs, digital photo albums and more. Sign up and create your flipbook.