Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N° 3
September 2007
JOURNAL of AUTOMATION, MOBILE ROBOTICS and INTELLIGENT SYSTEMS
Editor-in-Chief Janusz Kacprzyk (Systems Research Institute, Polish Academy of Sciences, Poland)
Webmaster: Tomasz Kobyliński tkobylinski@piap.pl Marketing office: Małgorzata Korbecka-Pachuta mkorbecka@piap.pl Proofreading: Urszula Wiączek Secretary: Agnieszka Sprońska Subscription: sub@jamris.org Copyright and reprint permissions - Executive Editor Editorial Office: Industrial Research Institute for Automation and Measurements PIAP Al. Jerozolimskie 202, 02-486 Warsaw, POLAND Tel. +48-22-8740109, office@jamris.org
Co-Editors: Dimitar Filev (Research & Advenced Engineering, Ford Motor Company, USA)
Kaoru Hirota (Interdisciplinary Graduate School of Science and Engineering, Tokyo Institute of Technology)
Witold Pedrycz (ECERF, University of Alberta, Canada)
Roman Szewczyk (PIAP, Warsaw University of Technology)
Executive Editor: Anna Ładan aladan@piap.pl Associate Editors: Mariusz Andrzejczak, Katarzyna Rzeplińska-Rykała
Program Committee: Chairman: Janusz Kacprzyk (Polish Academy of Sciences, Poland) Zenn Bien (Korea Advanced Institute of Science and Technology, Korea) Adam Borkowski (Polish Academy of Sciences, Poland) Wolfgang Borutzky (Fachhochschule Bonn-Rhein-Sieg, Germany) Oscar Castillo (Tijuana Institute of Technology, Mexico) Chin Chen Chang (Feng Chia University, Taiwan) Jorge Manuel Miranda Dias (University of Coimbra, Portugal) Bogdan Gabryś (Bournemouth University, UK) Jan Jabłkowski (PIAP, Poland) Stanisław Kaczanowski (PIAP, Poland) Tadeusz Kaczorek (Warsaw University of Technology, Poland) Marian P. Kaźmierkowski (Warsaw University of Technology, Poland) Józef Korbicz (University of Zielona Góra, Poland) Krzysztof Kozłowski (Poznań University of Technology, Poland) Eckart Kramer (Fachhochschule Eberswalde, Germany) Andrew Kusiak (University of Iowa, USA) Mark Last (Ben–Gurion University of the Negev, Israel) Anthony Maciejewski (Colorado State University, USA) Krzysztof Malinowski (Warsaw University of Technology, Poland) Andrzej Masłowski (PIAP, Poland)
Tadeusz Missala (PIAP, Poland) Fazel Naghdy (University of Wollongong, Australia) Zbigniew Nahorski (Polish Academy of Science, Poland) Antoni Niederliński (Silesian University of Technology, Poland) Witold Pedrycz (University of Alberta, Canada) Duc Truong Pham (Cardiff University, UK) Lech Polkowski (Polish-Japanese Institute of Information Technology, Poland)
Alain Pruski (University of Metz, France) Leszek Rutkowski (Częstochowa University of Technology, Poland) Klaus Schilling (Julius-Maximilians-University Würzburg, Germany) Ryszard Tadeusiewicz (AGH University of Science and Technology in Kraków, Poland)
Stanisław Tarasiewicz (University of Laval, Canada) Piotr Tatjewski (Warsaw University of Technology, Poland) Władysław Torbicz (Polish Academy of Sciences, Poland) Leszek Trybus (Rzeszów University of Technology, Poland) René Wamkeue (University of Québec, Canada) Janusz Zalewski (Florida Gulf Coast University, USA) Marek Zaremba (University of Québec, Canada) Teresa Zielińska (Warsaw University of Technology, Poland)
Publisher: Industrial Research Institute for Automation and Measurements PIAP
If in doubt about the proper edition of contributions, please contact the Executive Editor. Articles are reviewed, excluding advertisements and descriptions of products. The Editor does not take the responsibility for contents of advertisements, inserts etc. The Editor reserves the right to make relevant revisions, abbreviations and adjustments to the articles.
All rights reserved ©
1
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N° 3
September 2007
JOURNAL of AUTOMATION, MOBILE ROBOTICS and INTELLIGENT SYSTEMS VOLUME 1, N° 3
CONTENTS ARTICLES 3 69
DEPARTMENTS INTERVIEW about European education and collaboration with professor Piotr Tatjewski Original Structure Could Be Reintroduced
STATE OF THE ART A SuFET based sensor for nano-microscope R. Sklyar
21 Player and stage at PJIIT robotics laborn P. Ośmiałowski
71
IN THE SPOTLIGHT 72
29 A direct algorithm of possibilistic clustering with partial supervision D. A. Viattchenin
EVENTS
39 Methods and systems of leak detection in long range pipelines M. Turkowski, A. Bratek, M. Słowikowski
47 Presenting a technique for registering images and range data using a topological representation of a path within an environment F. Ferreira, L. Davim, R. Rocha, J. Dias, V. Santos
56 A conception of expert system for MIG-29 aircraft K. Butlewski, P. Golański
59 Architecture of Mobile Robotics Platform planned for Intelligent Robotic Porter System - IRPS project A. Woloszczuk, M. Andrzejczak, P. Szynkarczyk
64 Influence of a real radio channel parameters on the quality of the OFDM signal in mobile applications. Estimation of parameters of the channel as an element of improvement of the transmission quality K. Bożek
2
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N° 3
September 2007
STATE OF THE ART
A SUFET BASED SENSOR FOR NANO-MICROSCOPE Received 19th March; accepted 20th April.
Rostyslav Sklyar
Abstract: A superconducting field-effect transistor (SuFET) based transducer (sensor) with carbon nanotubes (CNT) or pickup coil kind of input circuit for the nerve and neuron impulses, DNA recombination signals, flows of biochemical molecules, micro- and nanoscopy, and biosusceptibility has been designed. A nanoSuFET with a high-temperature superconducting channel is introduced into the nerve fibre or brain tissue for transducing their signals in both directions. Pickup coils are implanted into an organism in order to obtain the natural or artificially excited biosignals from the organs and tissues. The range of picked up signals varies from 0.6 nA to 10 µA with frequencies from 20 to 2000 Hz. The output signal lies in the range of (5÷5)V, 17 3 (7÷0)·10 /cm molecules, and (2÷10) pH. The sensitivity -4 of this micro- or nanoscope can be estimated as HJ = 10 4 (A·m/ÖHz) with SNR equal to 10 . The sensitivity of an advanced first-order biogradiometer is equal to 3fT/ÖHz. The smallest resolvable change in magnetic moment detected by this system in the band 10 Hz is 1 fJ/T. Keywords: biosignals, SuFET, nanoFET, biosusceptibility, nanoscope, operating conditions
1. Introduction – Different sensors and diagnostic systems The sensor converts information or energy from the measurement quantity into another quantity, especially an electrical quantity. This “intermediate” signal may be processed in some way [1]. There are sensor and transducers for electrical quantities especially for current, voltage and magnetic quantities. A biosensor is a device that incorporates a biologically active layer as the recognition element and converts the physical parameters of the biological interaction into a measurable analytical signal [2]. Thanks to advances in nanotechnology, microelectromechanical systems, molecular diagnostics, and several other technologies, biosensors are now being developed to detect everything from the first chemical signature of cancer to the presence of anthrax. The technology behind biosensors sounds perfectly simple. Typically, there is a probe on the tip of the sensor that makes contact with blood or tissue. Because of its particular electrical, metallic, and chemical composition, the sensor is able to "react" in a telltale manner when it comes into contact with the target. That reaction is then magnified, quantified, illuminated with various stains and dyes, and then matched against the ever-expanding libraries of gene sequences and protein structures of diseases [3].
There are a number of methods and devices for transducing different biosignals (BSs) into recordable or measurable information. The transfer of nerve impulses (NIs) is the main data flow, which carries sensory information to the brain and control signals from it and the spinal cord to the limbs. That is why detecting currents between neighbouring neurons and ionic currents in the nerve fibres is an important area of research. Electric-field control of physical properties is highly desirable from fundamental and technological viewpoints because it does not introduce any chemical or microscopic structural disorder in the pristine material. This is also the basis of FETs, in which accumulation, depletion, and inversion layers are formed at the interface [4]. Moreover, the complex view on BSs requires further stages of precise processing in order to decode the received or control information. There are different kinds of transducers/sensors for picking up NIs: room-temperature and superconducting, external and implantable. Development of such devices is increasing the penetration into bioprocess while simultaneously simplifying the exploitation of the measuring systems in order to bring them closer to a wide range of applications. For this reason the magnetometer with a room-temperature pickup coil (PC) (without using the toroidal core) for detecting signals, which can clearly be detected in higher frequency range, was developed in order to simplify the superconducting quantum interference device (SQUID) system. The PC is set outside the cryostat and is connected to the input coil of the SQUID [5] or a channel of superconducting FET (SuFET) [6]. At frequencies higher than 20 Hz the noise decreases as 1/f function and then reaches a high flat noise at 2kHz as 200 fT/ÖHz. On the other hand, implantable into nerve fiber transducers are evolving from the ordinary Si-chip microelectronics devices [7] into superconducting and nanodevices [8, 9]. The growing variety of biosensors can be grouped into two categories: implantable and external. Because external sensors employ widely used contact mechanisms like needles and lasers, this branch has advanced faster than implantables. For this reason a working interface between the living tissue of individual neurons and the inorganic compounds of silicon chips were developed. There also was provided the link between ionic channels of the neurons and semiconductor material in a way that neural electrical signals could be passed to the silicon chip. Once there, that signal can be recorded using the chip's transistors. What's more, the neurons can also be stimulated through the capacitors. This is what enables the two-way communications. The project tested the Articles 3
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N° 3
September 2007
device by stimulating the neurons and recording which ones fired using standard neuroscience techniques while tracking the signals coming from the chip [10]. The recent achievements in nanoelectronics can be regarded as a further step in the progress of BS transduction. They give us the possibility to create the most advanced and universal device on the basis of known micro systems. Such a sensor/transducer is suitable for picking up BSs- NIs, electrically active (ionised) molecules and the base-pair recognition event in DNA sequences- and transforming it into recognizable information in the form of electric voltage, or a concentration of organic or chemical substances. Moreover, this process can be executed in reverse. Substances and/or voltages influence BSs, thereby controlling or creating them (BSs). Also the new fiber-based optical microfluidic detectors are designed for nano-volume sample measurement. The silicon-glass microdetectors were fabricated by use of micromechanical techniques [11]. The known development of scanning-probe magnetic microscopes is based on circulating Josephson currents in a SQUID loop to produce microwave images with a spacial resolution of about 30 Âľm [12]. A SQUID-based microwave microscope has some distinct advantages over earlier microwave systems: the spacial resolution is limited only by the size of the SQUID loop. A mice biomagnetic measurement system using a dc SQUID magnetometer for a comparative magnetoencephalogram study of transgenic and wild mice has been developed [13]. A magnetometer with PC of 1.0 mm diameter and 0.7 mm lift-off distance to improve spatial resolution and magnetic field sensitivity was adopted. The magnetocardiogram of a wild mouse was measured as the initial application of the system. The bipolar peak positions in a magnetic field map separated by 5.6 mm were clearly detected with the maximum amplitude of 88 pT. The advent of semiconductor devices with nanoscale dimensions creates the potential to integrate nanoelectronics and optoelectronic devices with a great variety of biological systems. Moreover, the advances in nanotechnology are opening the way to achieving direct electrical contact of nanoelectronic structures with electrically and electrochemically active subcellular structures-including ion channels, receptors, and transmembrane proteins such as bacteriorhodopsin. Direct electrical interfacing at the biomolecular level opens the possibility of monitoring and controlling critical biological functions and processes in unprecedented ways, and portends a vast array of possibilities such as new classes of prosthetic devices, medical monitoring devices, medical delivery systems, and patient monitoring systems, as well as other applications [14].
other [1]. Proceeding from the previously mentioned difficulties, including superconducting element of the sensor/transducer into an electric current could be the solution to the problem. Electronic or ionic currents in conductors or axons respectively, passing through the SuFET's channel induce the output voltage on its gate [8]. Whereas the sensor element can deliver a weak signal, the transmitted signal should generally have a high signal level, and perhaps suitable values, in order to reach superior units undisturbed and to simplify the following calculations. Therefore, the sensor signal should be generally pre-processed. Thereby several important tasks could be realized, such as signal amplification, scaling, linearization, conversion, and conjunction with other components in a chain, parallel, or closed-loop structure [15]. As an electrical signal, the BS has two components: electrical potential or voltage and ionic currents. The first component is sufficiently developed and does not require penetration into the substances of BS propagation. The marketable progress in transducing of the second component began when the necessary instrumentation for measurement of micro and nano dimensions had been created. The main informational flux from organs of the senses to motor nerves is transmitted through nerve fibres, which consist of a myelin shield with axons as a core. Recent research results suggest that such an arrangement is similar to a transmission line [16]. The nerve impulse in motor nerve of a frog is equal to 2 nA [17]. Synaptic currents between first order neighbouring neurons into in vivo or brain slice preparations have an order of 50 pA [18]. The nerve impulses passing through the fibre could be unambiguously defined by detecting the matching ionic current(s) or its superposition. Such a technique seems optimal because even precise voltage measurement could not give a current value according to the Ohm law. First of all, nerve fibre must be separated from a living organism for resistance of fibre measurement and, secondly, this resistance may vary in time. The implantable microelectrodes for neural applications are based on thin-film polymer foils with embedded microelectrodes for both recording and stimulation. Applications for these biomedical micro devices will include stem cell research, cancer cell characterization, drug discovery, treatments for neurological disorders, and neuroprosthetic devices [19]. A. The NIs propagation, measurement and simulation The signal conducted along the axon is a transient. This difference in potential is the result of ionic gradients due to ionic concentration and is modified by the ionic flow that produces ionic currents. These ionic currents give rise in turn to longitudinal currents closing local ionic current circuits that allow the regeneration of the membrane potential changes in a different region of the axon. This process is a true propagation instead of the conduction phenomenon occurring in wires.
2. Sensors and diagnostic instruments: the input signals and flows Electrical current may be measured by measuring the related magnetic field by means of the Hall effect or the Faraday effect into optoelectronic devices. The measurand as well as the reference value are often converted into quantities of either the same or different physical nature, before they are actually compared with each 4 Articles
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N째 3
September 2007
magnetic field could barely be detected (Fig.2). The magnetic signal is produced predominately by the total axial current ibio, enclosed by the toroid while the voltage action potential in air expresses the charge on the membrane. The magnetic trace should then be proportional to the first derivative of the electric trace, as observed in Fig. 3.
Fig. 1. Longitudinal section of an axon showing a few lines of current flow. The propagation of the NI in the cylindrical geometry of the axon and the consequences of the geometry of the axon on the current circulation across the membrane and along the axon is show in Fig. 1 [20]. The generation of the action potential has been carried out in an axon with an axial wire.
Fig. 4. Test of the single-fiber model. Propagation of action potentials in an isolated normal fiber. Development of such devices is increasing the penetration into bioprocess while simultaneously simplifying the exploitation of the measuring systems in order to bring them closer to the wide range of applications. Since the field and spatial resolution are highly diminished as the distance between the sample and the sensor increases, the key to this technique is to bring the sensor as close as possible to the sample. The cryogenic positioning system consists of three different main components and is very bulky. SQUID magnetometer systems do not provide the spatial resolution necessary to study the generation of the magnetic activity or injury currents at tissue and cellular scales. Synchronous impulse transmission and the formation of ''condensed'' pulse states are found [22]. Electric impulses with a delay of 0.5 ms are presented to the system, and the numerical results show that, for increasing coupling, the impulses tend to adjust their speed and become synchronized (Fig. 4). Cell polarity is critical in various cellular processes ranging from cell migration to asymmetric cell division and axon and dendrite specification. Additionally, Par-3 directly associates and recruits the p75 neurotrophin receptor to the axon-glial junction, forming a complex necessary for myelination. Together, these results point to a critical role in the establishment of cell polarity for myelination [23]. B. Transducing and simulation of the neuron activity Cultures of neurons can be grown on microelectrode arrays, so that their spike and burst activity can be monitored by a low noise multichannel integrated circuit for recording neuronal signals. These activity patterns are quite sensitive to changes in the environment, such as chemical exposure, and hence the cultures can be used as biosensors [24]. The signalling from the chip through a pair of neurons and back to the chip is thought to be the as a fundamental pathway of future neurochips (Fig. 5) Articles 5
Fig. 2. A nerve action potential propagates from proximal to distal (left to right in the figure). The wide and narrow arrows around the nerve represent the magnetic and electric field, respectively; the arrows on the nerve axis are equivalent dipole sources.
Fig. 3. The magnetic (solid lines) and electric (dashed lines) signals recorded from a frog sciatic nerve immersed in Ringer solution. The magnetic fields were recorded with a SQUID magnetometer with a room-temperature PC [21]. At the closest PC-to-nerve separation of 15 mm, the nerve
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N° 3
September 2007
[25]. The activity of the fundamental element of such future hybrids was recorded by joining a silicon chip with an excitatory chemical synapse between a pair of identified neurons [26].
a)
b)
Fig. 5. The neuron-transistor coupling of the postsynaptic neuron: a) The weak response (noise, left scale) is the postsynaptic response (upper). The strong response is an action potential, right scale (lower). b) Record of microelectrode in neuron (upper). The presynaptic stimulus elicits a postsynaptic action potencial (lower).
and systems of cultured neurons. In the experiments with retina, the amplitudes of extra cellular neuronal signals are typically in the range (50–500) µV with the frequency spectrum in the range (20÷2000) Hz (Fig. 6). Both the low signal amplitude and the low frequency range are challenging for an IC design. A low noise preamplifier and proper shaping of the frequency passband are required to optimise the signalto-noise ratio. In most low frequency applications, high precision switched-capacitor filters play the dominant role. The nervous system uses two basic types of formats for encoding information. The parameters of many sensory (and some premotor) signals are represented by the pattern of activity among an array of neurons each of which is optimally responsive to a different parameter value [28]. This type of code is commonly referred to as a place code. Motor commands, in contrast, use rate coding: the desired force of a muscle is specified as a monotonic function of the aggregate rate of discharge across all of its motor neurons. Generating movements based on sensory information often requires converting signals from a place code to a rate code. Sensory maps or place codes are ubiquitous throughout the brain. The individual neurons that make up sensory maps respond maximally to different “preferred” values of the sensory parameter being encoded, and their responses decrease if the actual stimulus either exceeds or falls short of that particular favourite parameter value. Thus, only by examining the responses of a population of such receptors is it possible to deduce what the stimulus actually looks, sounds, feels, smells, or tastes like.
Fig. 6. Analog data from two electrodes recorded simultaneously by the system. This plot indicates the richness of the information contained in the analog data. The spikes with different amplitudes on the same electrode (B1 and B3) were likely produced by different neurons. The spikes in time coincidence on the two electrodes (A1 and B3) are likely produced by a single neuron. Modern neurophysiological experiments often require high-density multi-electrode readout systems for recording signals simultaneously from many neurons. Multichannel readout systems are built mostly in hybrid technologies using off-the-shelf components. With such techniques readout systems with up to several tens of channels have been fabricated [27]. The development described in this paper is driven mainly by a project to readout signals from live retina tissue. Similar readout systems can be used in the study of slices of brain tissue 6 Articles
Fig. 7. A simulation of three models, illustrating the relationship between the output and both the location, and the amount of activity: A - Simulation of the summation, averaging, and summation-saturation models. Each curve represents the output value that occurs for “microstimulation” of a unit at one of five sites in the input map, as a function of the number of pulses of microstimulation delivered to that site; B - Comparison with actual microstimulation data from the superior colliculus. Microstimulation was conducted at a single site for two different pulse frequencies and for varying durations. The neural representation of motor commands shares some similarities with sensory systems. To move a body part farther or faster requires a monotonic increase in motor neuron activity. This can occur by increasing the activity level of a given set of motor neurons, by
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N째 3
September 2007
recruiting additional motor neurons, or both (Fig. 7). The amplitude of the evoked movement initially varies with the number of pulses, then saturates. This behaviour is most similar to the summation-saturation model.
Fig. 8. Genosensor technology. A - An atomic force microscope image of a 500-base-long (~250 nm) RecA nucleoprotein filament (black arrow) localized at a homologous sequence on l =DNA scaffold molecule [34]; B - Expression levels of cell clusters in a microarray are propor-tional, over a fourfold range, to the amount of plasmid DNA [35]. The implantable electronic circuits for neural recording and stimulation in freely behaving animals that could provide prosthetic connections to replace or augment damaged pathways in the nervous system have been developing [29]. A Neurochip creates an artificial connection between two sites by using action potentials recorded on one electrode to trigger electrical stimuli delivered to another. The chip made the recordings and stimulations through two implanted electrodes, thus providing a constant connection between the two sites in the brain. C. Analytical signal of DNA Recent advances in automated DNA synthesis and the convenient site-specific labelling of synthetic oligonucleotides with suitable functional moieties, coupled with advances in microelectronics, have accelerated the development of biosensors for the analysis of specific gene sequences (Fig. 8). A DNA biosensor (or genosensor) employs an immobilised DNA as the recognition element [2]. Electrochemical DNA biosensors rely on the conversion of the base-pair recognition event into a useful electrical signal. Current DNA detection methods based on sequencing by synthesis rely on optical readouts; however, a direct electrical detection method for this technique is not available. Application of the label-free electrical detection method, charge perturbation detection, applied to sequencing by synthesis, could be used for direct electrical detection of enzymatically catalysed DNA synthesis by induced surface charge perturbation to detect any enzymatic DNA or RNA synthesis as well as other biochemical reactions based on similar principles [30].
D. Non-invasive biomedical instruments SQUIDs have been a key factor in the development and commercialisation of ultrasensitive electric and magnetic measurement systems. In many cases, SQUID instrumentation offers the ability to make measurements where no other methodology is possible. Although SQUID electronics have the capability to operate well above 1 MHz, most applications tend to be at lower frequencies. Specific examples of input circuits and detection coil configuration for different applications and environments, along with expected performance, are described [31]. In particular, anticipated signal strength, magnetic field environment (applied field and external noise), and cryogenic requirements are discussed. Finally, a variety of applications with specific examples in the areas of electromagnetic, material property, nondestructive test and evaluation, and biomedical measurements are reviewed. Since the field and spatial resolution are highly diminished as the distance between the sample and the sensor increases, the key to this technique is to bring the sensor, (held in at cryogenic temperatures), as close as possible to the sample. It has been shown that the best combination of spatial resolution and field sensitivity for a specific SQUID geometry occurs when the diameter of the pickup coil is approximately equal to the sample-tosensor distance [32]. These systems do not provide the spatial resolution necessary to study the generation of the magnetic activity or injury currents at tissue and cellular scales. In excitable tissue extracellular potentials, transmembrane potentials, or action currents are interrelated. The extracellular potentials are typically recorded using microneedles arrays. However, the insertion of microneedles influences the measurement results and is impractical to achieve submillimeter (nano) special resolution. The approach to record the action currents using SQUID microscopy allows us to obtain more detailed information on the generation of the magnetocardiogram (MCG). High-resolution biomagnetic imaging provides insights that will improve existing mathematical models of biological tissue.
3. The components for superconducting nanosensors CNT-based nanobiosensors may be used to detect DNA sequences in the body. These instruments detect a very specific piece of DNA that may be related to a particular disease. The use of nanotube-based sensors will avoid problems associated with the current much-larger implantable sensors, which can cause inflammation [33]. The devices can be administered transdermally. CNT chemical sensors for liquids can be used for blood analysis (for example, detecting sodium or finding pH value). Implantable sensors can be useful in health assessment. Because of their small size and less power consumption, they are highly suitable as implantable sensors. One way of compensating the loss of photoreceptors is by bypassing the destroyed photoreceptors and artificially stimulating the intact cells in the neighbourhood. Another possible area related to the application of CNTs that can be investigated is cochlear implants related to hearing problems. Articles 7
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N° 3
September 2007
Superconducting nanowires are unusual in that they newer show zero resistance, although resistance does exponentially upon cooling [34]. A new class of metallic devices based on DNA molecules is promising due to the self-assembly properties of DNA. As the resistance of the devices is controlled by the spatial profile of superconducting phase within the leads, there is the potential for applications. These include local magnetometry (as is widely done with conventional SQUID) and the imaging of phase profiles created by supercurrents – in essence a superconducting phase gradiometer. Traditional materials have been pushed to their limits, which means that entirely new materials (such as high-kappa gate dielectrics and metal gate electrodes), and new device structures are required [35]. Entirely new device structures (such as nanowire or molecular devices) and computational paradigms will almost certainly be needed to improve performance. The development of new nanoscale electronic devices and materials places increasingly stringent requirements on metrology. A. The Available FET's Variants Applicable as SuFET Application of the SuFET's modifications such as CMOSuFET (low Tc) [36] and coplanar SuFET (high Tc) [37] broadens the range of requirements, which are being satisfied by the SuFET based transducer (SuFETTr). Lowdimensional semiconductor nanostructures and organic molecules, which offer unique possibilities such as extremely low power dissipation, quantum effects, surface sensitivity and low synthesis cost, could be the building blocks for next- generation electronics [38]. Moreover, an organic superconductivity of carbon molecules, known as bucky balls, which can act as superconductors at relatively warm temperatures, raises hopes for loss-free organic electronics and their practical applications in biosensors, including organic ones.
Fig. 10. A - SAMFET structure. A highly doped Si substrate is used as the gate electrode, a thermally grown SiO2 layer acts as gate insulator, the gold source electrode is deposited by thermal evaporation, and the active semiconducting material is a two-component SAM of alkanedithiols mixed with 4,49-biphenyldithiol or 5,59-terthiophenedithiol. The drain contact is defined by shallow angle shadow evaporation of gold. The active region of the device is magnified; B - Transfer charac-teristics at room temperature of two SAMFETs (drain-source voltage of 20.5 V). The control corresponds to a “pure” 4,49-biphenyldithiol SAM, whereas the second one is based on a twocomponent SAM (4,49-biphenyldithiol to alkanedithiol ratio is 1:100). The inset shows the current at a gate and drain-source voltage of 20.5 V as a function of the mixture ratio.
Fig. 9. An organic SuFET device and its electrodes. High densities of electrons and holes have been induced by gate doping in a FET geometry (Fig. 9). At low temperatures, the material turns superconducting with a maximum transition temperature of 117 K in holedoped C60/CHBr3 [39]. The increased spacing between the C60 molecules increases the density of states, and the resulting increase of TC is well documented in alkali metaldoped bulk samples (A3C60). The observation of gate-induced hole doping of C60 resulting in a TC of 52 K suggests that significantly higher TC's could be anticipated in suitably “expanded” C60 crystals. Indeed, here is reported a raise in TC to 117 K with such methods.
Fig. 11. OFET of organic single cristals with bottom contact configuration A - the schematic structure of the fabricated OFET; B - transfer characteristics of the single cristal FET device at a source-drain voltage Vd of -30 V.
8
Articles
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N° 3
September 2007
B. Organic FETs with a superconducting potential Vertical self-assembled monolayer (SAM) FET (SAMFET) action, i.e. conductance modulation through a third electrode, in devices consisting of only several electrically active molecules has been reported [40]. The channel length of 10 to 20 Ă… is defined by the thickness of a SAM rather than lithography (Fig. 10). Moreover, the peak conductance at low temperatures is quantized in 2 units of 2e /h, indicating that the conductance of single molecules is modulated. Organic FETs (OFETs) are of great interest for future electrolitic applications due to their flexibility. Up to now, a lot of fabrication processes or device configurations of OFETs have been reported [41]. Most of them were based on thin film technique such as vapour deposition. In general, the thin films for example, amorphous, poly crystalline, polymeric, and so on, consist of grains, and therefore, charge carriers behave as hopping conduction. This decreases the field-effect mobility of devices. Thus, annealing the organic thin films or surface preparations of substrate grows grains, and consequently these devices have few boundaries of grains between source and drain electrodes.
Fig. 13. In-plane gate FET on hydro-genated single-crystal diamond surface: A - Schematic diagram of an in-plaine FET; B - Drain-source resistance RDS, channel resistance RCH, and depletion region width as a function of the gate voltage VGS. A simple but powerful method to observe directly the regions in which the carriers are exhausted or injected by electric fields has been proposed [4]. As shown in the Fig. 12, the source (S)-drain (D) current in thin film transistors (TFT) is measured as a function of the film thickness with a bottom-contact configuration. The fabrication and characterization of in-plane gate transistors on hydrogenated single-crystal diamond surfaces have been reported [43]. A wire structure is designed in such a way that the gate voltage modulated the conductance of the channel (Fig. 13). The combination of the biocompatibility and tissue equivalence of diamond, as well as the metal-free surface of these inplane devices, in particular, may open interesting applications for diamond in bioelectronics. C. NanoFETs with Superconducting CNT Channel (nano SuFETs) The first such devices were fabricated in 1998. The CNT was simply laid on the gold electrodes and was held by weak van derWaals forces. In addition to increasing the gate capacitance, it is essential that each CNT FET is gated independently by its own gate so that complex integrated circuits can be built [44]. The next generation of CNTFETs with top gates was fabricated by dispersing single-walled (SW) CNTs (SWCNTs) on an oxidized wafer (Fig. 14). The intrinsic transconductance of CNTFETs in which CNTs were grown by chemical vapour deposition was measured at a drain voltage of -1V was 8.7 mS for a CNT Articles 9
Fig. 12. Thin film (TFT) FET: A - The source- drain current was measured in situ at various gate voltages; B - shows channel current (ISD) versus voltage (VSD) for the 20 nm thick pentacene TFT FET. A quite simple and easy fabrication technique of OFETs with single crystals was proposed, and it demonstrated that the OFETs exhibit high mobility and good device characteristics (Fig. 11). The presented technique makes the fabrication of high performance OFETs easy and serves as a potential way to make organic integrated circuits. Also N-channel and ambipolar OFETs with a few tens of nanometer channel length were fabricated and characterized [42].
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N° 3
September 2007
with a diameter of 1.5 nm (Fig. 15). Estimated intrinsic transconductance was 20 mS when parasitic resistance was taken into account. Measured and intrinsic transconductance per unit channel width were 5800 mS/mm and 13000 mS/mm, respectively.
This is considerably larger than those for state-ofthe-art Si-MOSFETs. This transconductance can be improved drastically by decreasing parasitic resistance and the performance of CNT FETs will advance further by improving the CNT quality and the device structures [45].
Fig. 16. A 4 by 4 crossed NW-FET array. A - with four horizontal NWs (I1 to I4) as inputs and four vertical NWs (O1 to O4) as signal outputs; B - conductance versus gate voltage of a single cNW-FET.
Fig. 14. Schematic representation of top-gated CNTFET: A - The SWCNT played the role of the “channel,” while the two metal electrodes functioned as the “source” and “drain” electrodes. The heavily doped silicon wafer itself was used as the “gate” (back-gate); B - Transfer characteristics of such a CNTFET.
Fig. 17. A self-assembled SWNT-FET: A - schematic representation of the electrical measurement circuit; B - drain-source current versus gate voltage for different values of drain-source bias. The importance of being able to address nanoscale elements in arrays goes beyond the area of nanocomputing and will be critical to the realization of other integrated nanosystems such as chemical/biological sensors. A regular crossed-nanowire FET (cNW-FET) array (Fig. 16.A) that consists of n-input í and m-output Vout n NWs, in which outputs are the active channels of FETs and the inputs function as gate electrodes that turn these output lines on and off [46]. Conductance versus applied NW input gate voltage data (Fig. 16.B) shows that each of the four cNW-FET elements could be turned off with gate voltage of 1V to 2 V, whereas the off-diagonal elements remained unaffected for the same input voltage. When NW are superconducting and, as a result, all cNW-FETs are functioning in SuFET mode, such output voltage Vout will show the changes in their conductance. The realization of a self-assembled SW CNT FET operating at room temperature promotes such strategy as realistic for construction CNT-based electronics (Fig. 17). The assembly process was guided by the information encoded in DNA molecules and homologous genetic recombination [47]. Using a variant of nano-patterning, a self-assembling polymer could also create tiny, finger-shaped silicon protrusions sticking up from the underlying substrate [48]. These fingers would constitute the "channel" in a transistor through which electrons flow- but one in which electrons flow vertically instead of across a chip, as in
Fig. 15. CNTFETs with CNTs grown by chemical vapor deposition combined with low-resistance ohmic contacts and top gates: A - an AFM image of an individual CNT bridging the 1-mm gap between the source and drain electrodes. The gate electrode (LG 210 nm) is formed on the CNT. Inset: Schematic structure of the CNTFET; B - CNTFET's channel current IDS as a function of VGS at VDS = -1V. 10 Articles
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N째 3
September 2007
today's devices (Fig. 18). The gate to turn the transistor off and on could encircle the silicon finger. The geometry might prevent electrons from "tunneling," or leaking, through the channel when the transistor is in the off state, a constant threat when feature sizes become very small.
Fig. 18. Self-assembled nanocrystal FET. A layer of self-assembled silicon nanocrystals is inserted into an otherwise standard device.
Fig. 19. Schematic of the electrolyte gate measurement. A water-gate voltage Vwg applied to a silver wire in the pipette is used to establish the electrochemical potential in the electrolyte relative to the device. For -0.9V < Vwg< 0.9 V, the leakage current between the water and the Au electrodes/SWNT was negligible (less than 1 nA); the electrolyte then functions as a well-insulated liquid gate. The excellent device characteristics of SW CNT transistors in salty water also indicate that they may be ideal for biosensing applications. Since a SW CNT has dimensions comparable to typical biomolecules (e.g. DNA, whose width is approximately 2 nm), they should be capable of electrical sensing of single biomolecules [49]. A charged molecule near the SW CNT will act as an effective gate, changing the conductance of the tube. The large transconductances indicate that the signal from single molecules should be readily observable (Fig. 19). D. The geometry of PC Since the magnetic biosignals are so small, rejection of all disturbances is of extreme importance. Biological tissues also generate disturbances. The vulnerability to the external magnetic noise can be reduced greatly by a proper design of the flux transformer, which collects the signal and transmit it to the transducer [50]. Two designs, one axial with two oppositely wound superconducting coils, and the other planar of figure-of-eight geometry are insensitive to a spatially uniform background field but respond to changes in an inhomogeneous field as generated by the nearby organ or tissue. To avoid the problems of wire-wound gradiometers, which have only a modest balance, planar integrated sensors offer an elegant solution. The main advantage of flat thin-film gradiometers is their compact structure and
excellent dimensional precision, providing a good intrinsic balance. The ability to resolve separate sources within the brain or heart should be an important factor for all designs [51]. Different existing instruments can be compared by the dependence of spatial resolving power of the instrument on them. In designing a new system, given the physical constraints of the design, these methods may be used to optimise the system for resolving sources under the experimental conditions anticipated. The vector current sources are the usual equivalent current dipole sources that are the most frequently utilized source models for biomagnetic phenomenon. The design parameters were the radius (or other geometrical constraints) of the array, the number of sensors, and the spacing of the sensors. Three basic gradiometers are possible [51]. With regards to gradiometer parameters, the type of gradiometer-planar, axial, or normal-and the base line of the gradiometer are important considerations. Finally, the dependence on the sources involves the relative orientation of the sources, the strength of the sources, the number of active sources, and the signal-to-noise ratios (SNR). An axial first-order gradiometer is formed by coupling the input coil of a flexible high-temperature superconducting flux transformer inductively to a directly coupled SQUID magnetometer [52]. The tape of the transformer is bent such that the two outer pickup loops of the transformer are facing each other while perpendicular to the magnetometer plane resulting in a gradiometer baseline of 35 mm. A superconducting shield is mechanically adjusted to reduce the gradiometer response to uniform fields applied perpendicularly to both the magnetometer plane and the plane of the transformer pickup loops, by a factor of typically 7000.
4. Implantable Sensors: Design and Performance Data In general, single sensor systems can only provide partial information on the state of the environment, while multisensor systems combine related data from multiple similar and/or different sensors. The goal of using multisensor systems is to provide synergetic effects that enhance the quality and availability of information about the state of the measurement environment [15]. A number of sensing organs of different physical quantities and environmental conditions are discovered in living beings. On the other hand, output signals from these organs- nerve impulses- can be picked up by stateto-the-art techniques. Further signal processing will define the connection between sensed quantity and output data of such biological sensors. As a result they could be applied in addition to the known artificial sensors for intelligent sensing. Electrical properties of hybrid structures consisting of arrays of nanowire FETs integrated with the individual axons and dendrites of live mammalian neurons have been reported [53]. Arrays of nanowire-neuron junctions enable simultaneous measurement of the rate, amplitude, and shape of signals propagating along individual axons and dendrites. The configuration of nanowire-axon junctions in arrays, as both inputs and outputs, makes Articles 11
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N掳 3
September 2007
possible controlled studies of partial to complete inhibition of signal propagation by both local electrical and chemical stimuli. In addition, nanowire-axon junction arrays were integrated and tested at a level of at least 50 "artificial synapses" per neuron. Interfacing of nerve cells and field-effect transistors is determined by current flow along the electrical resistance of the cell-chip junction [54]. A spectral power -14 2 density of the junction is (5路10 )V /Hz and can be interpreted as Nyquist noise of the cell-chip junction with a resistance of 3 MOhm by measuring the fluctuations of extracellular voltage with a low-noise transistor. The thermal noise allows us to elucidate the properties of cell adhesion and it sets a thermodynamical limit for the signal-to-noise ratio of neuroelectronic interfacing. A. Methods of picking up the biosignals: non-invasive and implantable, electro/magnetic- and biosensors Voltage potentials of the living organism and its organs are measured by both implantable and external electric field probes of high sensitivity [55]. Information on organ activity is obtained by measuring biomagnetic signals. For such purposes a multi-channel high temperature superconducting interference (high Tc SQUID) system for magnetocardiography (MCG) and magnetoencephalography (MEG) of humans, with high magnetic field resolution has been developed [56, 57]. The known amperometric techniques of biosignals involve the Renview bight realising method [17], and the second method of "biosensors typically rely on an enzyme system which catalytically converts electroche-mically non-active analytes into products which can be oxidized or reduced at a working electrode which is main-tained at a specific potential with respect to a reference electrode" [58]. The same method is applicable also to potentiometic measurements "that can measure substrates, inhibitors or modulators of the enzyme". The Renview method requires extra stimulating of the isolated nerve fibre and the other method needs additional reagents and applied voltage. There are two feasible courses for impulse detection. The first one is measuring the magnetic flux around the fibres by PC [59]. The second one is to let the impulses pass through the electronic device with minimal impediments. B. Devices for matching of biosignal to electronic element (circuit) Creative integration of microchip technologies and nanostructures is feasible. By tuning the dielectrophoretic frequency within a microdevice, nanoparticles can be manipulated with the same precision as cells because a one-to-one correspondence exists between a given alternating current frequency and a nanoparticle interaction or biological event. Multiple biological events could be probed simultaneously provided that their corresponding frequencies are distinct. Combined with electroporation, electrokinetics also enables inclusion of molecular complexes inside the cells. Alternatively, functionalized nanoposts can be used to impale cells and relay information from the cell interior to nanoelectronic circuits. By merging the fields of 12 Articles
microfluidics, electrokinetics, and cell biology, microchips are capable of creating tiny, mobile laboratories. The challenge for the future of designing a nano-interface in a microfluidic chip to probe a living cell lies in seamlessly integrating techniques into a robust and versatile, yet reliable, platform [60]. A planar FET can be configured as a sensor by modifying the gate oxide (without gate electrode) with molecular receptors or a selective membrane for the analyte of interest. Binding of a charged species then results in depletion or accumulation of carriers within the transistor structure. An attractive feature of such chemically sensitive FETs is that binding can be monitored by a direct change in conductance or related electrical property, although the sensitivity and potential for integration are limited. The so-called floating gate architecture combines a complementary metal oxide semiconductor (CMOS)-type n-channel FET with an independent sensing area for recording extracellular signals from electrogenic cells was presented [61]. This concept allows the transistor and sensing area to be optimised separately. The noise level of the devices was smaller than of comparable non-metallised gate FETs. The potential of NW nanosensors with direct, highly sensitive real-time detection of chemical and biological species in aqueous solution has been demonstrated [9]. A silicon NW (SiNW) solid state FET, whose conductance is modulated by an applied gate, is transformed into a pH nano-sensor (Fig. 20) by modifying the silicon oxide surface with 3-aminopropyltriethoxysilane (APTES).
Fig. 20. A pH nanosensor: A - a silicon NW (SiNW) solid state FET; B - measurements of conductance as a function of time and solution pH. The report shows how the scientists fabricate FETs from CNT with the precise electrical properties and any variable band-gap desired. In parallel studies of CNT, researchers have been working to improve the electrical characteristics of individual nanotube transistors [62]. C. Design of a SuFET based either implantable or noninvasive transducer of the BSs The method of combining the bioelectric nature of NIs and synaptic currents between neighbouring neurons with body-temperature PC and zero resistance input of the SuFET device in order to obtain most advantageous biosensor/transducer was recently advanced [8].
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N° 3
September 2007
The SuFET is used as a zero-resistance ammeter, which converts drain currents into gate voltages [6]. Transducing the vertical magnetic field from the BSs by the PC that is wrapped around the nerve fibre or DNA sequence is executed similarly to Fig. 2, but the PC is in nano-dimension [63]. By using the said superconducting magnetometer with a room- temperature PC (SIM) [6] it is possible to create the implantable transducer [8]. D. New nano-microscope and biosusceptometer There was described the eddy current microscopy of thin conducting samples using a SQUID-based magnetic flux microscope in which the sample and SQUID are cooled in liquid nitrogen [64]. If the technique could be extended to the imaging of room temperature samples, it could be used to diagnostic purposes. To achieve a high spacial resolution and flux resolution, this microscope uses a small SQUID directly as a magnetic sensor, rather than a superconducting pickup loop coupled to SQUID. A time-varying field not only induces local eddy currents but also produces currents, which circulate around structures with closed loops. The ability to detect small regions of nonmagnetic conducting materials, and distinguish them from ferromagnets, superconductors, and paramagnets at the microscopic level, gives the magnetic flux microscope broad capabilities in materials analysis. For such testing to be of practical use, it will be necessary to develop systems, which allow the microscopic magnetic imaging of room temperature samples. In order to realize a short distance between the cooled SQUID and the room temperature sample, a microscope- type SQUID has been developed [65]. In this system, the diameter of the sample is 5 mm. Corresponding to this sample size, a directly coupled thin 2 film gradiometer with two 5×5 mm pickup coils is used, where the gradiometer is chosen to reduce the spatially uniform external noise. The distance between the sample and the SQUID is approximately 1.5 mm, which is short enough for a sample size of 5 mm. The SQUID and the sample are surrounded with three layers of a permalloy magnetic shield in order to reduce the disturbance from the environmental magnetic noise. A low temperature magnetic susceptometer was constructed by combining an Oxford Instruments Heliox probe sorption pumped 3He cryostat with a Quantum Design model 5000 DC SQUID sensor, giving an effective temperature range from 300 mK to 4.2 K [66]. The smallest resolvable change in magnetic moment detected -12 -1 by this system was found to be Delta~10 J·T . 1) In vivo micro- and nanoscope for scanning and sounding A new design for scanning or sounding micro- or nanoscopes that combines a simple mechanical arrangement with a miniature SIM's PC. The microwave imaging process (Fig. 21) is shown on a prototype sample: a normal conducting ring of self-inductance Li and resistance Ri. The quantity L0 is the PC selfinductance and MIi represents the magnetic coupling of the PC through a mutual inductance M to an external circuit carrying a current Ii in a tissue [67]. If wT>>RiLi, then Ri may be ignored and Ii(wT)» - Ij(wT)M/Li.
induced UHF solid eddy currents
EC/PC
Ï Î conducting organ or tissue
Ï Î gu ide of EC/PC
IJ ~
UHF H -field
Fig. 21. Schematic diagram of SIM based nano-microscope. For nonzero drain voltages the SuFET absorbs lowfrequency power of the average Josephson current IJ and re-emits this power at extremely high frequencies [6]. When the perturbation is small we can write:
I J (wT ) =
2eI 0 æ VDS ö - VGS ÷ ç hwT è 2 ø
(1)
where Io - the critical Josephson current; VDS and VGS - are the drain-to-source and gate-tosource voltages in SuFET respectively. wT - is closely related to the small signal transconductance of the SuFET. After defying the factor K J = 2eI 0 hwT Eq.(1) becomes simple:
ö æV I J (wT ) = K J ç DS - VGS ÷ ø è 2
(2)
As a result, the decreasing of the SuFET channel's current is defined by the value of losses for eddy currents Ii(wT) in the tissue: (3) DI J (wT )
æV öæ M = I J (wT ) - I i (wT ) = K J ç DS - VGS ÷ç 1Li è 2 øç è
ö ÷ ÷ ø
Thus, by monitoring the change in DIJ(wT) as a function of PC position, we make use of the factor M/Li dynamics of the tissue to obtain a micro- or nanowave-screening image. Because IJ flows around the SIM's PC(s), it produces a time varying MF in the envelope of the PC(s) which can serve as a local probing field (Fig. 21). By adjusting IJ, one can continuously adjust the magnetic field. To achieve a high spatial resolution, our nano-microscope uses a small ambient temperature PC(s) directly as a magnetic sensor, rather than a SQUID's pickup loop coupled to a cooled SuFET. Sounding and scanning closer to the tissue improves the spatial resolution, thanks to the complete penetration into the measuring process. Other than the SuFET itself, there are no other microwave components, sources, or detectors. This is particular advantageous at very high frequencies where components are difficult to construct. In conclusion, we note that the ability to detect small regions of nonmagnetic Articles 13
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N° 3
September 2007
conducting materials, gives the magnetic flux nanomicroscope broad capabilities in material analysis. Since the developed system allows the micro- and nanoscopic of room and tissue temperature samples, such testing will be of practical use for clinical diagnostic. The sensitivity of this instrument HJ can be estimated by considering the noise source INJ of SuFET according to Ref. [6]. The digital value of HJ for PC with the diameter of 0.1 µm and inductance 1 µH, and INJ=10-11 (A·Hz) is equal to: HJ=INJL/µ0Seq=10-4 (A·m/ÖHz). This means that with a SIM exciting signal Hsign=(VDS/2-VGS)/ 3 /µ0SeqwT=(VDS/2-VGS)·10 » 1 (A·m) a magnetic SNR in a band less than 10 Hz will be: Hsign/HR(D10Hz)= =(VDS/2-VGS)/ INJLwT. For the said values, SNR of the described nano-microscope will be equal to HSNR= 7 4 =(VDS/2-VGS)·10 » 10 . Typically our magnetic images are taken at about 8 pixels/s. 2) In vivo biosusceptometer of organs and tissues There are several commercial SQUID biosusceptometers available [68]. They have been specially designed for high-sensitivity measurements on small HTS samples at very small fields (below 1 mT) in the temperature range from 4.2 K to room temperature. When operated in AC mode the magnetometer measures the real and imaginary components of the AC susceptibility. Note that without the ability to make highly sensitive AC measurements, it would not be otherwise possible to distinguish between intrinsic and defect determined properties. In the AC mode the noise-limited sensitivity was estimated from measurements on a niobium foil of known mass to be 10-12·Am2 ÖHz. A technique had been previously developed, based on magnetic field measurements using a SQUID sensor, to localize in three dimensions steel needles lost in the human body. In all six cases that were treated until now, the technique allowed easy surgical localization of the needles with high accuracy. Despite the importance of needle localization, the most prevalent foreign body in the modern society is the firearm projectile (bullet), generally composed of lead, a paramagnetic material, thus not presenting a remnant magnetic field as steel needles do. On the other hand, since lead is a good conductor, eddy current detection techniques can be employed, by applying an alternating magnetic field with the aid of excitation coils. The primary field induces eddy currents on the lead, which in turn generate a secondary magnetic field that can be detected by a magnetometer, and give information about position and volume of the conducting foreign body. A theoretical study for the development of a localization technique for lead bullets inside the human body was presented [69]. The main difficulties in the measurement of biosusceptibility are irregular shapes of the investigated objects, which are difficult to be formalized, and this susceptibility is inhomogeneous. Following, the measurement in vivo, the PC should be maximally penetrative and flexible to follow the object's form and miniature to have the high-resolution ability. Simultaneously, the angle between the outside- magnetized coil- and an inside PC should be adjusted due to the amplification of the signal from some particular area. Also to raise the 14 Articles
sensitivity to deeply, laid down in vivo areas, it is necessary to use the Helmholtz coils for the creation of the laminar magnetic fields. U out 7 5 4 • • • •
~
6 1
2 3
U in
~
R in
Fig. 22. A simplified circuit for a SIM based biosusceptometer. Taking into account all the described requirements it is advancing the configuration of the measuring system that shown in Fig. 22. The part of living object 1 is excited by a quasi DC or AC magnetic field H(r) 2, generated by Helmholtz coils 3, is penetrated into the investigating subject 4. The magnetization M(r) of an organ or tissue 4 is detected by the in vivo PC 5 of SIM 6 that creates a corresponding output voltage Uout. PC has been handling in any direction of the subject's irregular form by a biochemically inert rod or manipulator 7. For the biological tissues the magnetization M(r) in the point r is defined by the value of magnetic susceptibility c(r) as: M(r)=c(r)×H(r) (4)
where H(r) is defined by the supplied current IHl and the transforming factor of the coils KHl as: H(r)= KHl× ×IHl=H(r)=UinKHl/Rin. The value of M(r) with the frequency f and energy resolution of SuFET EJ will be defined as [6]:
M (r ) =
eI 0VGS zc pm0 S eq E J wT f
(5)
where zC - the impedance of PC; -7 µ0 - the permeability of free space, µ0=4×10 henry/meter; -30 EJ=10 (J/Hz); Seq- equivalent area of PC. From Eq. (4) and Eq. (5):
c (r ) =
eI 0VGS zc Rin pm0 S eq E J wT U in K Hl f eI 0 zc Rin pm0 S eq E J wT K Hl
(6)
If we define K c =
as a factor of biosusceptibilitymeter, the value of c(r) will be calculated simple:
c (r ) = K cVGS U in
(7)
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N° 3
September 2007
c(r)=KcVGS/Uinf. For the said values of the SIM's circuit, -1 KHl=10(m ), and exiting frequency 10 Hz, c(r) will be 4 equal to c(r)=5×10 ×VGS/Uin. The shape of PC(s) should be designed according to the specific conditions and aims of the BM measurements. These PCs can be as gradiometer of different order [51], also with the compensation coils [59]. The biogradiometer of first with the coils of an equal area has a factor of efficiency h=2 and for a second order- h=6. For the typical reference of a PC's radius to the radius 2 of wire as the value of a/c=10 , the previous sensitivity -6 3 of biogradiometer is defined as: SH=4×10 hEJ/a . For the PC's radius 0.001 (m) the sensitivity of an advanced firstorder biogradiometer is equal to: 3fT/ÖHz. The smallest resolvable change in magnetic moment detected by this system in the band 10 Hz is: (EJ/SH=) 1 fJ/T. The sensitivity of this biosusceptibilitymeter can be increased by using of PCs as gradiometers of a second and high order (with h=6 and more). As a result, it appears the possibility to refuse both of the shielding room and cryogenic system for PC(s). At the same time, the design of in vivo PS(s) varying from nano-micro dimensions to the large shapes of conducting material, which are assembled around the functioning organs, or regions of the living tissue has been under investigation.
The creation of a permissive environment for axonal regrowth was described using a synthetic biological nanomaterial that self assembles in vivo, with components that break down into beneficial building blocks and produce no adverse effects on the nerve fibre. This discovery, by reducing or overcoming the first two obstacles and possibly more, allows for the reconnection of disconnected parts of the nerve fibre [73]. Thin films of carbon nanotubes deposited on transparent plastic can serve as a surface on which cells can grow and these nanotube films could potentially serve as an electrical interface between living tissue and prosthetic devices or biomedical instruments. Both cell types were placed on ten-layer-thick “mats” of single-walled carbon nanotubes (SWNTs) deposited on transparent plastic. This enabled to use a microscope to position a tiny electrode next to individual cells and record their responses to electrical pulses transmitted through the SWNTs [74].
6. The main arrangements of SuFETTrs Among the variety of the above-presented FET devices there are majority of them, mainly modifications of nanoFETs, which allow simultaneous processing of a number of BSs directly or from the PC [8]. There are two factors that make simultaneous processing possible. First of all, the sizes of nanoFETs and nanoPCs are in the same order as the transmitting substances of BSs, such as axons, neurons, and the DNA spiral. Secondly, the cNW-FET array (Fig. 16) is, in itself, multi-input. The remaining part of FET devices is applicable for serial connection to the said mediums. In addition, some of these FETs can be arranged in the chain in order to transduce the BSs into different physical and chemical quantities and vice versa. A. Serial Connection of BSs to SuFETTr
5. Connection of Su(CNT)FETs with the BS's medium Microdevices with electroplated wire traces were etched with well-defined edges. These devices are implanted in living bodies to connect nerve tissue with electronics to record nerve cell activities or restore lost functions by stimulation of nerve cells. Electroplating of gold meets the requirements for producing neural implants with low-ohmic wire traces, because this technique allows the microfabrication of gold layers with a thickness of several micrometers and lateral dimensions in the same range. Hence the mechanical stability of the electroplated gold is sufficient for chronic implantation of the structures [70]. The implantable microelectrodes for neural applications are based on thin-film polymer foils with embedded microelectrodes for both recording and stimulation [71]. Very efficient attachment mechanisms are those in which patterned surface structures interact with the profile of the substrate. This general trend is quantitatively explained by applying the principles of contact mechanics, according to which splitting up the contact into finer subcontacts increases adhesion. This principle is widespread in the design of natural adhesive systems and may also be transferred into practical applications [72]. Recently, strong evidence has been presented that the adhesion of gecko setae is caused by van der Waals interaction, rejecting mechanisms relying on capillary adhesion. Elements of contact mechanics have also been applied to this problem and it was predicted that arrays with smaller setae endings should result in greater adhesive strength. An extensive microscopical study of biological surface devices has been combined with the theory of contact mechanics based on molecular adhesion.
í V out Fig. 23. Schematic of SuFETTr in the serial connection. IDS, ìA 0,5 -
t
í
-5
I
0,2 0
I
5 Vg, V
VDS(bias), V G(bias)=const
t
Vout
Fig. 24. Operational pattern of a serial SuFETTr. The organic or nano SuFET device shown, for example in Fig. 17, is connected with the source of BSs through the said micro or nano contacts according to Fig. 23. In such case the current of í, passing through the SuFET's Articles 15
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N° 3
September 2007
channel arouses Vout on the gate with some constant bias voltages VDS , VG (Fig. 24). Transfer function (TF) of the device defined by its numerical order as follows:
K DNA =
é 10 ù Vout 7éBù Þê ú = 2 ×10 ê A ú i A 0 , 5 m ë û ë û
(8)
The operational pattern of the transducer for one input BS into cNW-FET is shown in Fig. 26. The input BS changes conductance of the FET's channel which influences output voltage Vg. TF of this device will be similar to the previous one:
As the channel of the majority of cited FETs is not superconducting in the present stage of development, it is possible to define the sensitivity threshold (ST) by the channel's resistance R: DNA U noise = 4kTR
K cNW =
Vout é 1B ù éBù Þ ê -7 ú @ 107 ê ú VsupQ + i ë10 A û ë Aû
(11)
(9)
where k - the Boltzmann constant, T - absolute temperature of FET's channel. The noise voltage of SuFETs is obtained analogous to the Johnson noise value for the resistive circuit as [6]: (EN) =4kTSuFET gnoise/gdn and gdn=I0/VC 2
(10)
where gnoise is the ratio of the kinetic energy of the JJ link to thermal energy and VC -characteristic voltage. Ratio gnoise is proportional to the normalized gate charge gG/C (gG is the total gate charge reflected in the channel). The peak currents range of BS from 5 to 10 µA [21] give a maximal output voltage Vout on absciss axis -5 to 5 V also with the necessity of some its reducing it slightly by changing VDS(bias) of the FET's channel. A current, which elicits an action potential in the neuron, is 0.6 nA [25] and will stimulate Vout of the transducer equal to 12 mV. B. Parallel Connection of BSs to SuFETTr An example of connecting the axons of nerve fibre to multi-input cNW-FET [46] array is shown in Fig. 25. The output voltage of the transducer according to Fig. 16.B lies in the range of 1 V (linear part of the curves).
Similarly the value of Vout can be obtained for organic, chemical and DNA SuFETTrs. The carrier density thus obtained is a function of the distance x from the interface (Fig. 11). The large carrier density at VG=0 V in small x region is due to the charge transfer from the Au electrodes to the pentacene molecules [4]. Carrier density n(x) is a function of the distance (x) from the interface at various gate voltages (VG). According to the Eq.(4) deviation of carrier density n(x) in the range of maximal Vout=±15 V is equal to 17 3 (7¸0)×10 /cm . Chemical SuFETTr converts the changes in pH through Q of the channel (similarly to pH nanosensor Fig. 20.A [9]) also into output signal Vout. In the scale of Q from 10 to 400 nS a pH (2 to 10) is transformed into variations of Vout from 0 to 1 V (Fig. 26). The potential applied during immobilisation of thiolmodified DNA lies in a different potential window from -0.7 to +0.7 V [2]. The corresponding current í=VinQ, to this voltage which is put into the FET channel, according to Fig. 26 vary from 0.4 to 1.8 nA. Clean gold monolayers are stable in the potential range from -400 to +1400 mV in dilute sulfuric acid solutions, thus allowing electrochemical applications. The careful selection of the terminal functionalities of the monolayers and the proper surface chemistry allows a tremendous flexibility in biosensor design. ST of the biosensor/transducer in the first approximation (generally) depends on the conductance Q of the superconducting channel. An absolute sensitivity of the transducer with n parallel inputs in the said frequency range derives also from the Johnson noise: cNW U noise = n 4kT / Q
(12)
í
V out
which for cNW-FET varies in the range 50 to 150 nS [46] -7 and gives the order of ST 10 V/ÖHz. Also for the noise voltage of parallel SuFETs based transducer is: (EN) (n)=4nkTSuFET gnoise/gdn 2
Fig. 25. Schematic of SuFETTr in the parallel connection. (13) Q, nS 400 -
7. Results t 200 -
í Vsup=const
0
I
1
Vg, V
t
Vout
Fig. 26. Operational pattern of a parallel SuFETTr.
Application variety of the novel superconducting, organic and CNT FETs allows us to design transducers of BSs (electronic, nerve, DNA, etc.) that transduce them into different quantities, including electric voltage, density of chemical and biomolecules. On the other hand, the said BSs can be controlled by the applied electrical signals, or bio and chemical mediums. The described SuFETTrs designed on the basis of organic and nano SuFETs are suitable for describing the wide range of BS dynamical parameters (see Table 1).
16
Articles
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N° 3
September 2007
Following the columns of the table, it should be noticeable, that serial connection of the external PCs allows us to gain some integrated signal, i.e., the whole sensing or control electronic or NI, which spreads along the number of axons of the nerve fibre; the amount of ions passing through the PCs and the generalized BS
passing through one or both spirals of DNA. When SuFET channel(s) of are implanted into the tissue or process we can acquire more precise data about the frequency distribution of NIs, volume distribution of ionized molecules and detecting activity of individual nucleoteds.
Table 1. Dependence of the received BS parameters on the mode of SuFETTr's functioning. Mode Medium Current Molecules DNA external òi=1 cont. or sens. imp. òBSs®bio and chem. molec. propagation of BS along DNA's spirals Serial implantable i=i(f1)+i(f2)+...+i(fN) external di/dt, di/dx Parallel implantable åi=1 network or 1 fibre åBSs=åbio and chem. molec. 4 nucleoteds® 4 outputs
variation of BSs® åBSs=1 type of molec. concentr. of molec. decoding the BSs of space and lengh nucleoted recognition dynamic on both spirals
The described biosusceptometer or nano-microscope designed on the basis of SuFETTrs is suitable for investigating both the structure of organic objects and their comparing analysis (see Table 2). Following the strings of the table, it should be noticeable, that investigations of biological surfaces are performing according to the surface integrals for a biosusceptometer
and nano-microscope modulus respectively. The surface gradients are acquiring by finding of the difference between the respective values of I1 or I2. The same is applying to the investigations of biological volumes V1 and V2 as the double and triple integrals respectively. The next two strings are explaining the bounds on the possible spreading of the said method.
Table 2. Dependence of the received structure parameters on the mode of functioning- a biosusceptometer or nano-microscope. Mode Medium Surface Volume biosusceptometer modulus I1=òò¦ (x, y, z)ds s V1=òò¦ (x, y)dxdy s
nano-microscope modulus I2=òò¦ (x, y, z)dxdy s V2=òòò¦ (x, y, z)dv v
biosusceptometer gradient DI1=I' 1 -I" 1 DV1=V' 1 -V" 1
nano-microscope gradient DI2=I'2 -I" 2 DV2=V' 2 -V" 2
Structule level
Investigation of sheath (envelopes) of organs Investigation of homogeneous organs or tissues
Differential Investigation of Comparing investigation investigation the twin the inside structure of of the organ's (pair) organs or tissues the organs and tissues or tissue's areas Investigation of inhomogeneous organs or tissues Comparing investigation Differential investigation of the homogeneous the inhomogeneous twin organ's or tissue's areas (pair) organs or tissues
Object (body) level
Exploitation of the parallel input to SuFETTr allows determination of space and time dynamics of BSs in the nerve fibre and DNA spiral(s) and also the amplification of output signal Uout by multiplying the concentration of molecules according to a number of input BSs. After the implantation of parallel SuFET(s), the averaging or summation of this dynamic among the whole neural network, nerve fibre or DNA spiral(s) is possible.
8. Conclusions The invented biotransducer has the following fundamental improvements upon existing ones: a) the sign of the output voltage permits the determination of the direction of the input bioflow passing through a single SuFET device;
b) situating the reference electrode outside the living organism makes precise measurement possible; c) the capability to regulate the proportion of axons, neurons or flows that are being investigated to the untouched ones- either the whole cross section of the fibre or flow, or any part of them; d) the possibility to substitute the SuFET device or to adjust its ratings to comply with the conditions of the measurement process without repeatedly destroying nerve fibre or flow vessel; e) the transducer could create conversion in both directions, respectively in passive and active modes; f) the combination of biocompatibility and tissue equivalence in both the diamond and protein-based (organic) FETs makes them naturally fit for implantation; Articles 17
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N° 3
September 2007
g) the possibility to compose a converging device by changing the instrument - from a current sensor to biosusceptometer, from a flowmeter to nanomicroscope, etc. - by switching-over (modifying) of SuFET's working mode or transferring from the natural ambient conditions to applying of the exciting magnetic field. The reviewed variety of FETs shows the varying extent of readiness for them to be exploited them in SuFETTr of BSs. The most appropriate for such an application are the ordinary solid-state SuFET modifications and novel CNT based SuFETs. The organic SuFETs are not amply developed, but this work is being carried out in a number of directions. At the same time, the PCs, which are necessary for the external sensor with respect to the transducing medium (solid-state conductor, nerve fibre, flow of ions and DNA spiral), and corresponding lowohmic wire traces for connecting PCs to the FET's channel are sufficiently developed, even at nano dimensions. The preliminary calculations confirm the possibility of broadening the SuFETTr's action from magnetic field to the biochemical medium of BSs. The main parameters of such BSs can be gained by applying the arrangement of the SuFETTr(s) to the whole measurement system. Two directions of SuFETTr function enable decoding of the BS by comparing the result of its action on some process or organ with an action on them of the simulated electrical or biochemical signal after their reverse transducing through the SuFETTr(s). Furthermore, this decoded signal will provide a basis for creating feedback and feedforward loops in the measuring system for more precise and complete influence on the biochemical process.
[6] [7]
[8]
[9]
[10]
[11]
[12]
[13]
[14]
[15]
[16]
AUTHOR Rostyslav Sklyar – Independent researcher, Verchratskogo st. 15-1, Lviv 79010 UKRAINE; tel./fax: +380 322 762432/769613, e-mail: sklyar@tsp.lviv.ua.
[17] [18]
References [1] Weiss H., “Electrical measurement and instrumentations - today and tomorrow”, Measurement, 1993, no. 12, pp. 191-210. Lucarelli F., Marrazza G., Turner A. P. F. et al., “Carbon and gold electrodes as electrochemical transducers for DNA hybridisation sensors” (Review), Biosensors and Bioelectronics, 2004, no. 19, pp. 515-530. S. Herrera, I've got you under my skin, Red Herring, 2003. Available at: http://www.redherring.com/insider/ 2003/02/biosensors020503.html. M. Kiguchi, M. Nakayama, K. Fujiwara et al., “Accumulation and Depletion Layer Thicknesses in Organic Field Effect Transistors”, Jpn. J. Appl. Phys., 2003, vol. 42, Pt. 2, issue 12A, L1408-L1410. A. Kandori, D. Suzuki, K. Yokosawa et al., “A Superconducting Quantum Interference Device Magnetometer with a Room-Temperature Pickup Coil for Measuring Impedance Magnetocardiograms”, Jpn. J. Appl. Phys., 2002, vol. 41, Pt. 1, pp. 596-599. [19]
[2]
[20] [21]
[3]
[22]
[4]
[23]
[5]
[24]
R. Sklyar, “Superconducting Induction Magnetometer”, IEEE Sensors J., 2006, no. 6, pp. 357- 364. P. Fromherz, “Electrical Interfacing of Nerve Cells and Semiconductor Chips”, CHEMPHYSCHEM, 2002, no. 3, pp. 276-284. R. Sklyar, “A SuFET Based Either Implantable or NonInvasive (Bio)Transducer of Nerve Impulses”, Proceedings of the 13th International Symposium on Measurement and Control in Robotics - ISMCR'03, Madrid, Spain 2003, pp. 121-126. Y. Cui, Q. Wei, H. Park et al., “Nanowire Nanosensors for Highly Sensitive and Selective Detection of Biological and Chemical Species”, Science, 2001, no. 293, pp. 1289-1292. P. Fromherz, Vassanelli S., and Greeff N. G., NACHIP Project, Reference: IST-2001-38915, 2006. Available at: http://www.biochem.mpg.de/mnphys/europroject/ project.html. S. Bargiel, A. Górecka-Drzazga, J. A. Dziubana et al., “Nanoliter detectors for flow systems”, Sens. Act. A, 2004, no. 115, pp. 245-251. R. C. Black, F. C. Wellstood, E. Dantsker et al., Microwave Microscopy Using a Superconducting Quantum Interference Device”, Applied Physics Letters, 1995, no. 66, pp. 99-101. Y. Ono, A Ishiyama, “Development of biomagnetic measurement system for mice with high spatial resolution”, Applied Physics Letters, 2004, no. 85, pp. 332334. M. A. Stroscio, M. Dutta, “Integrated biological-semiconductor devices”, Proc. IEEE, 2005, no. 93, pp. 17721783. O. Kanoun, H.-R. Tränkler, “Sensor Technology Advances and Future Trends”, IEEE Trans. Instrum. Meas., 2004, no. 53, pp. 1497-1501. C. Hanisch, “Nervensache“, Bild der Wissenschaft, 1999, no. 2, pp. 70-74. I. Tasaki, Nervous transmission, Charles C. Thomas Publ., Springfield IL USA, 1953. C. Wyart et al., “Constrained synaptic connectivity in functional mammalian neuronal networks grown on patterned surfaces”, Journal of Neuroscience Methods, 2002, no. 117, pp. 123-131. K. C. Cheunga, Ph. Renaudb, “BioMEMS for medicine: On-chip cell characterization and implantable microelectrodes”, Solid-State Electronics, 2006, no. 50, pp. 551557. F. Bezanilla, The Nerve Impulse, 2004. Available at: http://pb010.anes.ucla.edu. J. P. Wikswo, J. P. Barach, and J.A. Freeman, “Magnetic Field of a Nerve Impulse: First Measurements”, Science, 1980, no. 208, pp. 53-55. S. Reutskiy, E. Rossoni, and B. Tirozzi, “Conduction in bundles of demyelinated nerve fibers: computer simulation”, Biol. Cybern., 2003, no. 89, pp. 439-448. J. R. Chan, Ch. Jolicoeur, J. Yamauchi, “The Polarity Pro-tein Par-3 Directly Interacts with p75NTR to Regulate Myelination”, Science, 2006, no. 314, pp. 832-836. J. J. Pancrazioa, G. W. Gross, “Measuring synchronization in neuronal networks for biosensor applications”, Biosensors and Bioelectronics, 2004, no. 19, pp. 675-683.
18
Articles
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N° 3
September 2007
[25]
[26]
[27]
[28] [29]
[30]
[31]
[32]
[33]
[34]
[35]
[36]
[37]
[38]
[39]
[40]
[41]
[42]
M. Jenkner, B. Muller, and P. Fromherz, “Interfacing a silicon chip to pairs of snail neurons connected by electrical synapses”, Biol. Cybern., 2001, no. 84, pp. 239-249. R. A. Kaul, N. I. Syed, and P. Fromherz, “NeuronSemiconductor Chip with Chemical Synapse between Identified Neurons”, Phys. Rev. Lett., 2004, no. 92, 038102 (4 pages). W. Dąbrowski, P. Grybos, and A. M. Litke, “A low noise multichannel integrated circuit for recording neuronal signals using microelectrode arrays”, Biosensors and Bioelectronics, 2004, no.19, pp. 749-761. J. M. Groh, “Converting neural signals from place codes to rate codes”, Biol. Cybern., 2001, no. 85, pp. 159-165. A. Jackson, J. Mavoori, E. E. Fetz, “Long-term motor cortex plasticity induced by an electronic neural implant”, Nature, 2006, no. 444, pp. 56-60. N. Pourmand, M. Karhanek, H. H. J. Persson et al., “Direct electrical detection of DNA synthesis”, PNAS, 2006, no. 103, pp. 6466-6470. R. L. Fagaly, “Superconducting quantum interference device instruments and applications”, Rev. Sci. Instrum., 2006, no. 77, 101101 (45 pages). L. E. Fong, J. R. Holzer, K. K. McBride et al., “HighResolution Room-Temperature Sample Scanning Superconducting Quantum Interference Device Microscope Configurable for Geological and Biomagnetic Applications”, Rev. Sci. Instrum., 2005, no. 76, 053703-(1-9). N. Sinha, J. T.-W. Yeow, “Carbon Nanotubes for Biomedical Applications”, IEEE Trans. Nanobiosc., 2005, no. 4, pp. 180-195. D. S. Hopkins, D. Pekker, P. M. Goldbart et al., “Quantum Interference Device Made by DNA Templating of Superconducting Nanowires”, Science, 2005, no. 308, pp. 1762-1765. E. Vogel, “Technology and metrology of new electronic materials and devices”, Nature Nanotechnology, 2007, no. 2, pp. 25-32. J. F. Jiang, Q. Y. Cai, H. M. Jiang et al., “High-performance complementary metal-oxide-superconductor field effect transistor (CMOSuFET) current-mode operational amplifier”, Supercond. Sci. Technol., 1996, no. 9, pp. A66-A70. Sh. Suzuki, H. Tobisaka, and Sh. Oda, Electric properties of coplanar high-Tc superconducting field-effect devices, Jpn. J. Appl. Phys., 1998, no. 37, Pt. 1, pp. 492-495. B. Yu, M. Meyyappan, “Nanotechnology: role in emerging nanoelectronics”, Solid-State Electronics, 2006, no. 50, pp. 536-544. J. H. Schön, Ch. Kloc, and B. Batlogg, “High-temperature superconductivity in lattice-expanded C60”, Science, 2001, no. 293, pp. 2432-2434. J. H. Schön, H. Meng, and Z. Bao, “Field-effect modulation of the conductance of single molecules”, Science, 2001, no. 294, pp. 2138-2141. K. Nakamura, M. Ichikawa, R. Fushiki et al., “Organic field-effect transistor of (thiophene/phenylene) co-oligomer single crystals with bottom-contact configuration”, Jpn. J. Appl. Phys., 2004, no. 43, pp. L100-L102. T. Jung, B. Yoo, L. Wang et al., “Nanoscale n-channel and ambipolar organic field-effect transistors”, Appl. Phys. Lett., 2006, no. 88, 183102 (3 pages).
[43]
[44]
[45]
[46]
[47]
[48] [49]
[50]
[51]
[52]
[53]
[54]
[55]
[56]
[57] [58] [59]
[60]
[61]
[62]
J. A. Garrido, C. E. Nebel, and R. Todt, “Fabrication of in-plane gate transistors on hydrogenated diamond surfaces”, Appl. Phys. Lett., 2003, no. 82, pp. 988-1000. P. Avouris, J. Appenzeller, R. Martel et al., “Carbon nanotube electronics”, Proc. of the IEEE, 2003, no. 1772-1784. F. Nihey, H. Hongo, Y. Ochiai et al., “Carbon-nanotube field-effect transistors with very high intrinsic transconductance”, Jpn. J. Appl. Phys., 2003, no. 42, Pt. 2, L1288-L1291. Z. Zhong, D. Warmg, and Y. Cui, “Nanowire crossbar arrays as address decoders for integrated nanosystems”, Science, 2003, no. 302, pp. 1377-1379. K. Keren, R. Berman, E. Buchstab et al., “DNA-templated carbon nanotube field-effect transistor”, Science, 2003, no. 302, pp. 1380-1382. G. Stix, “Nano Patterning”, Scientific American, February 2004. S. Rosenblatt, Y. Yaish, J. Park et al., “High performance electrolyte gated carbon nanotube transistors”, Nano Lett., 2002, no. 2, pp. 869-872. O. V. Lounasmaa, J. Knuutila, R. Salmelin, SQUID Technology and Brain Research, Physica B, 1994, no. 197, pp. 54-63. E. R. Flynn, “Factors Which Affect Spatial Resolving Power in Large Array Biomagnetic Sensors”, Rev. Sci. Instrum., 1994, no. 65, pp. 922-935. M. Bick, K. E. Leslie, R. A. Binks et al., “Axial high-temperature superconducting gradiometer with a flexible flux transformer”, Applied Physics Letters, 2004, no. 84, pp. 5347-5349. M. Voelker, P. Fromherz, “Nyquist Noise of Cell Adhesion Detected in a Neuron-Silicon Transistor”, Physical Review Letters, 2006, no. 96, 228102 (4 pages). F. Patolsky, B. P. Timko, G. Yu et al., “Detection, Stimulation, and Inhibition of Neuronal Signals with High-Density Nanowire Transistor Arrays”, Science, 2006, no. 313, pp. 1100-1104. K. T. Ng et al., “Noise and sensitivity analysis for miniature e-field probes”, IEEE Trans. Instrum. Meas., 1989, no. 30, pp. 27-31. H. Itozaki et al., “Multi-channel high Tc SQUID”, IEICE Trans. on Electron., 2160 E77-C (1994), 1185-1190. (Invited paper) O. V. Lounasmaa et al., SQUID technology and brain research, Physica B 197 (1994) 54-63. A. Sharma and K. R. Rogers, “Biosensors, Meas. Sci. Technol., 1994, no. 5, pp. 461-472. G. L. Romani, S. J. Williamson, and L. Kaufman, “Biomagnetic instrumentation”, Rev. Sci. Instrum., 1982, no. 53, pp. 1815-1845. B. P. Helmke, A. R. Minerick, “Designing a nano-interface in a microfluidic chip to probe living cells: Challenges and perspectives”, PNAS, 2006, no. 103, pp. 6419-6424. S. Meyburga, M. Goryllb, J. Moersb et al., “N-channel field-effect transistors with floating gates for extracellular recordings”, Biosens. Bioelectr., 2006, no. 21, pp. 1037-1044. Ph. G. Collins, M. S. Arnold, and Ph. Avouris, “Engineering carbon nanotubes and nanotube circuits using electrical breakdown”, Science, 2001, no. 292, pp.706-709. Articles 19
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N° 3
September 2007
[63]
[64]
[65]
[66]
[67]
[68]
[69]
[70]
[71]
[72]
[73]
[74]
B. A. Korgel, “Materials science: self-assembled nanocoils”, Science, 27th February 2004, no. 303, pp. 1308-9. R. C. Black, F. C. Wellstood, E. Dantsker et al., “Eddy Current Microscopy Using a 77-K Superconducting Sensor”, Appl. Phys. Lett., vol. 64, 1994, issue 1, pp. 100-102. K. Nikawa, T. Kobayashi, K. Tanabe et al., “Recent Topics in High-Tc Superconductive Electronics” (Invited Review Paper), Jpn. J. Appl. Phys. 44 (2005) 7735-7749. D. Read, I. Terry, S. R. Giblin, “Low Temperature Magnetic Susceptometer Based Upon a DC Superconducting Quantum Interference Device”, Rev. Sci. Instrum. 77 (2006) 103906 (3 pages). R. C. Black, F. C. Wellstood, E. Dantsker et al., “Microwave Microscopy Using a Superconducting Quantum Interference Device”, Appl. Phys. Lett., vol. 66, 1995, issue 1, pp. 99-101. T. J. Jackson, M. N. Keene, C. E. Gough, “A SQUID Magnetometer for Low Field DC Magnetization and AC Susceptibility Measurements”, Meas. Sci. Technol., 1992, no. 3, pp. 988-991. C. H. Barbosa, “Localization of Firearm Projectiles in the Human Body Using a Superconducting Quantum Interference Device Magnetometer: A Theoretical Study”, Rev. Sci. Instrum., vol. 75, 2004, issue 6, pp. 20982106. M. Gross, D. Altpeter, T. Stieglitz et al., “Micromachining of flexible neural implants with low- ohmic wire traces using electroplating”, Sens. Act. A: Physical, vol. 96, 2002, no. 2, pp. 105-110. K. C. Cheunga, Ph. Renaudb, “BioMEMS for medicine: on-chip cell characterization and implantable microelectrodes”, Solid-State Electronics, vol. 50, 2006, issue 4, pp. 551-557. E. Arzt, S. Gorb, and R. Spolenak, “From micro to nano contacts in biological attachment devices, PNAS, 2003, no. 100, pp. 10603-10606. R. G. Ellis-Behnke, Y.-X. Liang, S.-W. You et al., “Nano neuro knitting: peptide nanofiber scaffold for brain repair and axon regeneration with functional return of vision”, PNAS, 2006, no. 103, pp. 5054-5059. A. V. Liopo, M. P. Stewart, J. Hudson et al., “Biocompatibility of native and functionalized single-walled carbon nanotubes for neuronal interface”, Journal. of Nanoscience and Nanotechnol., vol. 6, 2006, no. 5, pp. 1365-1374.
20
Articles
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N° 3
September 2007
PLAYER AND STAGE AT PJIIT ROBOTICS LABORATORY Received 4th July; accepted 16th Julyl.
Paweł Ośmiałowski
Abstract: Player/Stage/Gazebo is an Open-Source Software project designed for robotics research that provides infrastructure for distributed access to robotics equipment both real and simulated. Founded by Brian Gerkey, Richard Vaughan, Andrew Howard, Kasper Stoy and Nate Koenig it soon became popular among roboticists community. Project itself gained lots of contributors and many related software projects were started that support or make use of Player/Stage/Gazebo software. At PJIIT Robotics Laboratory we decided to deploy this software for educational purposes as well as for our Virtual Robotics Laboratory project where it plays significant role as an integration platform. Since it is Open-Source Software we were able to contribute new modules and also release fixes for bugs that we have found. This paper deals with our experiences with using and maintaining Player and Stage software. Keywords: Robotics software, simulation software, opensource software, robotics programming, robotics laboratory
1. Introduction Player/Stage/Gazebo[1] is an open-source Software project that consists of three main software parts that make it successful robotics framework. It can be downloaded for free from projects home page (http://playerstage.sourceforge.net). These parts are: Player - the core of whole framework, which contains message-passing server and drivers for most popular robotics equipment; it can be extended by plugins pieces of software that use Player as communication interface. Stage - a plugin for Player that acts as a 2D simulation device, which simulates existence of real hardware in predefined 2D workspaces, called worlds. Gazebo - a simulation plugin for Player that works with 3D workspaces. All this suite of programs and com panion software conforms the spirit of UNIX operating system [2] and Open-Source movement [3]. In this paper we will focus on Player and Stage parts only, as only these parts were used in PJIIT student's projects so far. Examples of these projects will be also described shortly.
many popular languages (C, C++, Java, Lisp, Scheme, Python, Ada, Octave/Matlab) that are required to build client-side programs. Also Player server plugins may act as the clients for other Player servers, which is useful for building more sophisticated communication topologies. Player may be considered as a communication bus, to which client-side programs can connect in order to communicate with robotics hardware through drivers also connected to that bus. Server configuration file describes what drivers are connected to communication bus in certain Player server instance. Although many drivers are built-in to Player itself, new drivers for the new hardware can be written in C++ language using Player API. These drivers are called plugins. Stage is an example of the most sophisticated Player plugin that provides access to simulated hardware. Each driver provides at least one predefined interface (consequently, Stage simulator provides many different interfaces). Every interface describes the kind of offered data with their internal structure and syntax of commands that associated device can accept (except readonly devices that do not accept any commands, only provide data like camera images, sonar readings, laser scans, power source status, etc.). For example, Video4Linux driver provides camera interface, which accepts no commands while data offered by this interface are: image (as an matrix of integer numbers), compression information (if JPEG compression is used or not), width, height and colour depth of the image.
2. Architecture Whole framework makes use of client-server architecture where Player itself plays the role of the server while client programs communicate with it to access data (sensors readings, camera images, odometry or GPS position clues and so on) and to send commands to actuators (robot motors, gripper, pan-tilt-zoom module on camera and so on). There are programming libraries available for
Fig. 1. Stage is the most sophisticated plugin for Player - it opens its own window in which progress in simulation can be observed; also a user can make changes in simulation by moving objects using a mouse. Articles 21
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N째 3
September 2007
There are some drivers (both built-in and plugins) that aren't associated with any particular hardware. These drivers provide useful functionality for whole Player infrastructure. Since communication between drivers is also possible through the Player's communication bus, one driver can take data in one form and offer different representation of them for connected clients or other drivers. Also one driver can take data directly from other Player server (it acts as a Client then) and then it can offer data processed by itself to other drivers or clients connected to Player server in which this driver was started. Note that communication between connected clients using Player communication bus is not possible; they should use other means of communication if they need it. client 1 client 2 client 3
camera driver
image compress driver
motors driver
write and read-only devices. For example, configuration query can be sent using laser interface to ask for geometry of laser device (where it is situated on the robot and what is it's shape and size, information typically used by client-side graphic visualization programs that tries to restore current situation in users application window). The same query can be sent using position2d interface to ask for - geometry of whole movable robot. Also configuration query can be sent to change some parameter of the device (even if it's read-only). For example, laser scans resolution can be changed that way. Some of the drivers that are not directly assigned to hardware can do heavy computations. If Player is intended to run on robot's onboard computer it must be considered that it can be too slow to run some of mentioned drivers that Player can provide. For example, there are few drivers that do advanced image processing. They are linked against OpenCV [17] (Open-Source Computer Vision library), which provides programming functions for image processing that needs CPU-consuming computations. Mentioned earlier amcl driver (Adaptive Monte-Carlo Localization) needs GNU Scientific Library [18] (gsl), which also provides functions that do heavy computations.
Fig. 2. Player communication bus with connected clients and drivers. As an example of a driver that is not associated with any certain hardware we may consider cameracompress that takes image from camera driver and offers the same image JPEG compressed, which can be useful for distant communication purposes. Another example is the amcl driver that implements Adaptive Monte-Carlo Localization algorithm. It takes data from other drivers using position2d interface which offers odometry position clues, laser interface which offers laser scans of environment nearby, map interface which offers 2D map of the whole current workspace and provides position2d interface to return guessed robot position within given map. In facts, position2d interface has two roles: to provide current position (absolute or odometric) and to command position changes which effects in robot motors activity. There are two kinds of position change commands accepted by position2d interface: position commands (for example move to given distance from current position) and velocity commands (for example move forward with given speed). Which kind of commands is accepted depends on associated device capabilities. Since real robots accept only velocity commands, a driver called vfh (Vector Field Histogram) can be used, which takes position commands using position2d interface and recalculates them to velocity commands that are later sent to position2d interface provided by some robot's driver. This driver also uses laser interface to read laser scans necessary to obstacle avoidance during the movement. Note that metric system of units is used in Player for distance measurements. There are three kinds of data travelling through communication bus: sensor readings, commands and configuration queries. First two were explained above. Configuration queries are sent to certain driver, which responds to it instantly. They are accepted both by read22 Articles
3. Client-side programs Player software package is shipped with example client-side applications. Although desired way of using robotics devices with Player is to write a program implementing control behaviours that does not require interaction with human operator, first program that most users learn first is playerv (PlayerViewer) - graphics user interface for controlling interactively all the devices accessible by Player server to which it will be connected. To control a device or to just read data from it a subscription should be made using Devices menu. The way subscribed device is represented on the window work area depends on interface provided by its driver. See figure 3 for an example situation.
Fig. 3. PlayerViewer (playerv) connected to a Player server that works on Pioneer P2DX robot on-board computer. As we can see, there are several devices subscribed that provide following interfaces: position2d (red box), ptz (blue and green lines, pan-tilt-zoom driver for the Sony EVID-30 camera), laser (blue field represents free space seen in front of the robot) and two different devices that provide blobfinder interface.
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N째 3
September 2007
Unfortunately, playerv cannot manage with camera interface; therefore another client program called videoplayer [15] was created which is an example of many contributed applications. It opens a window in which image from camera interface is constantly redrawn (see figure 12). It is also able to check if an image is JPEG compressed and run decompress routine to manage with it. Later Player developers have made another program that does similar thing called playercam, but videoplayer is still in use due to its simplicity and clear source code which is used as an example of how camera interface data can be acquired in programs for further image processing. Another interesting client-side program shipped with Player software package is playernav. It is designed to be a frontend for the planner interface (yet another interface that playerv cannot manage with). It displays map of the current workspace taken from given device that provides map interface. A user first has to place (using mouse) robot symbol within this map to give playernav a hint where the robot currently is and what is its current pose. Then it is possible to set the target position, which will be sent to the device that provides planner interface. Example driver that provides such interface is wavefront - a path-planning device, which, after receiving new target position a robot should approach, computes list of waypoint positions, that makes a patch to the target position. During these computations, wavefront helps itself by using a map of the workspace taken from a device that provides map interface. Having this list of waypoint positions, wavefront sends commands to actuator device using position2d interface. Since majority of known position2d actuator devices accept only velocity commands, typically wavefront sends position commands to the vfw device which changes them to desired velocity commands as described earlier. Commonly used device that provides map interface is called mapfile - it reads bitmap image from .pgm or .png file and interprets it as a workspace map with given resolution (for example 10cm per pixel). Each black pixel denotes place occupied by an obstacle, while white pixels denote free space.
Despite of that, the most important role at PJIIT Player infrastructure plays in Virtual Robotics Laboratory [16] where it integrates all communication. Player as an integration platform for Virtual Robotics Laboratory The purpose of Virtual Robotics Laboratory is to provide remote access through the Internet to its robotics resources for students and researchers from other educational institutes. At the beginning of the project, very first topic to consider was the choice of integrated communication platform [4]. Although we could choose to create our own communication software or to use existing solutions like CORBA, we wanted to conform standards of software used in robotics research laboratories around the world. We've realized that the most popular Open-Source Software solution is the Player server, which was already used by individual PJIIT students for their projects. Currently our Virtual Laboratory offers access to two Pioneer P2DX robots (one of them is adapted to work 24 hours 7 days a week, the other is on duty at users request). Each is equipped with ring of ultrasonic sonars, SICK LMS200 laser, camera Sony EVID-30 with pan-tiltzoom module, and onboard computer (PC/104+ compatible) with Video4Linux-compatible PCI framegrabber device. One of them is also equipped with gripper. To observe current situation, there are few cameras situated on top of the workspace. Two of them are mounted on movable tractor devices controlled by Player's plugin running on PC computer, which access these devices through RS232C serial port. This plugin driver was created in our laboratory and provides position2d interface making remote operator able to change position of given camera. 4.1.
4. Player and Stage at PJIIT Player and Stage are widely used in PJIIT Robotics Laboratory. It is used both for educational purposes (during classes) and for research projects. The success of this robotics framework corresponds to overall success of OpenSource Software in field of educational research at PJIIT.
Fig. 5. One of D-link cameras mounted on a movable tractor. Each PC computer and all onboard PC/104+ computers working in our Virtual Laboratory has started at least one Player server instance. The main server working at address vlab.pjwstk.edu.pl at TCP port 6665 offers public access to cameras and full access to movable tractor devices. Other devices (for example robot actuators) are available through special authorization proxy that we have designed to make sure only registered users who Articles 23
Fig. 4. Playernav in action; black rectangle is the map of the workspace available in our Virtual Robotics Laboratory.
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N째 3
September 2007
have full responsibility on the way they use laboratory resources can have full access. These users can also access our Control Panel web service available at address https://cp.vlab.pjwstk.edu.pl. Users, that use this control panel, can view the history of their access attempts and current condition of every device provided by our Virtual Laboratory. Supervisor user can also disconnect all other users, which leads to stop all robots movement.
Fig. 7. Snapshot from Stage simulation of one of experiments described by Janusz Matkowski in his work [5].
Fig. 6. Virtual Laboratory Control Panel started in the Firefox web browser. Currently our laboratory uses Player version 1.6.5 with our set of bugfixes. Corresponding client software and programming function libraries for both UNIXcompatible and MS Windows systems are available to download. We're providing source code, binaries for MS Windows and a portage tree for automatic installation in Gentoo Linux operating system [20], which is the most supported system by us. 4.2 Student projects Janusz Matkowski used the first time Stage simulator at PJIIT in his graduate studies research [5]. In his work he has described approach to artificial intelligence opposing classic AI ideas (this approach was earlier presented by Rodney A. Brooks [6]). Instead of heavy symbolic processing, paradigm of embodied intelligence relies on physical implementation (in particular environment) as a key to achieve truly intelligent agents. Aside from describing theory behind this approach he has demonstrated in simulation (figures 7 and 8) how sophisticated behaviours can emerge from appropriate combination of agent morphology and controller. Another example of students' project is Karol Yamazaki's PaGo (Point-and-Go) [7]. The main goal was to create new means for mobile robot navigation based on image analysis. Yamazaki's program is using Player to acquire live image from robot's camera. Image analysis is used for pointer object detection and space positioning. This pointer object (yellow bar) role is to show a robot where it should go (see figure 9). OpenCV [17] library functions were used for image analysis.
Fig. 8. Snapshot from experiment shown in previous figure after few minutes of simulation process.
Fig. 9. PaGo at work.
24
Articles
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N째 3
September 2007
Potential Field Method (PFM) as means for navigation is used in Michal Dendewicz and Lukasz Hrynakowski project [8]. The method is not a new idea, but is still considered interesting, so they wanted to focus on it and present its main assumptions on an operative model. They used Pioneer P2DX robot equipped with SICK LMS200 laser device and onboard computer running Player server. In several words, PFM main assumption focuses on imaginary forces acting on a robot. It can be compared to an electron behaviour in an electromagnetic field. In their case, mobile robot is the electron, and the electromagnetic field is emitted by obstacles situated in the robot's workspace (figure 10). Although PFM is very effective and pretty simple, it may suffer from local optima problem. Our students in their work have described their challenges in avoiding it.
Fig. 10. Example potential field. 4.3. Our improvements and bugfixes During the years of experience with Player and Stage software we have made number of improvements and bugfixes. First changes in Player's code were made in parts responsible for FireWire cameras operation. In our Robotics Laboratory we are using Imaging Source's DFK 41F02 high resolution FireWire camera that we wanted to use as a live image source for Player. Trying to do it we have realized that whole Player driver for FireWire camera devices should be totally rewritten. After buying three D-link's DCS-5300W cameras we had to write a completely new Player driver for them. It provides two interfaces: camera for the live image and ptz for pan-tilt-zoom module available in these cameras. We have also discovered and fixed few bugs in Video4Linux driver for cameras connected to the standard PCI framegrabbers. Although Player developers try to be up to date with providing drivers for the most popular robots, there are still many devices that need new drivers to be written. Example is small Hemisson robot by K-Team. PJIIT Robotics Laboratory uses two such robots. There are Player drivers for other K-Team products built into Player server (Khepera robot, and robot controllers based on REB/Kameleon board). Hemisson uses communication protocol similar to other K-Team robots; therefore we were able to base our work on Khepera driver source code written by Toby Collett from University of Auckland
Robotics Group. During this work, we did totally change serial port communication part of the driver code using new functions based on source code of minicom [19] program (popular Open-Source communication software). During the time we were using Player, many changes has taken place in related Open-Source Software infrastructure. When the new version line (4.x) of gcc compiler suite that is used to compile Player was started, some smaller parts of the code had to be rewritten. Also new, more restrictive version of GNU C library (glibc) made some hidden errors in memory management to show up. We have proposed set of patches that fix both problems for Player version 1.6.5 and Stage 2.0.0a. Also we have released portage tree for Gentoo Linux [20] that provides building guidelines for Player 1.6.5, Stage 2.0.0a and Gazebo 0.5.2, all including our bugfixes. For MS Windows users we have ported client-side programming library using MinGW [21] development environment. We had to replace usage of BSD sockets (used for TCP communication in Linux and other UNIXcompatible systems) with Winsock. Having client-side library ported to Windows we were able to release playerv and videoplayer compatible with Player 1.6.5 as regular Windows applications [22]. Soon we have realized that instead of using two different models of communication sockets, one for MS Windows, another for other systems, we can use one portable solution that works the same way almost everywhere. We have used SDL suite of highly portable programming libraries [23] that also provides its own communication sockets in a programming library called SDL_net. That way we have released SDL-style version of client-side programming library and using this we were able to release new playerv and videoplayer as regular Windows application, this time compatible with Player 2.0.4 [24]. During the works on Virtual Robotics Laboratory project we have released two simple programs that constantly monitor state of our infrastructure [9]. The playercheck program checks every 30 seconds if given Player server started on the same host responds properly (it tries to read list of available devices). If not, the process of that Player server instance is killed. Since we're starting our server instances in infinite loop, new Player server will be started in these circumstances. Another monitoring program is called lowpower. It is started on a robot which Player's driver provides power interface. It reads constantly voltage and whenever it goes below defined threshold, whole system is going to shutdown. This protects at least file system from being damaged during unexpected halt of onboard computer. Although C, C++ and Java are of the most popular programming languages in PJIIT, students in our Robotics Laboratory use also other languages for their programming works. Recently, Octave [10] - Open-Source Software interpreter of Matlab-compatible programming language became popular, mostly during the classes. We have started to use Cameron Morland's octplayer [11] - Player's client-side programming library for Octave. Soon we have extended it by adding more interfaces (laser, camera, map, localization). Also we have rewritten it totally to become compatible with latest Player 2.0.4 (June 2007). Articles 25
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N째 3
September 2007
r.robot = client_create("vlab.pjwstk.edu.pl", 6665); r.index = 1; laser = proxy_create("laser", "r", r); n = 0; while (1) n++; do for i = 1:5, if (client_read(r)) error("client_read returned an error!\n"); end end scans = laser_val(laser); len = length(scans); until (l > 359) x = cos((pi / 180.0) * (((1:len) - 1.0) / 2.0)) .* scans(1:len); y = sin((pi / 180.0) * (((1:len) - 1.0) / 2.0)) .* scans(1:len); clg(); plot(x, y, "@"); if (n > 200) n = 0; closeplot(); purge_tmp_files(); end end closeplot(); purge_tmp_files(); client_destroy(r); Listing 1. Example Octave script that constantly plots scans from laser device.
Fig. 11. Output of a script presented in listing 1. Another programming language popular in our Robotics Laboratory is Scheme. We have developed client-side programming library called guileplayer [12], which is intended to use with guile [13] - the most popular Open-Source Software interpreter of Scheme. Writing this library we tried as hard as possible to conform the spirit of Scheme language, therefore we couldn't have used automatic bindings generators (like swig [14]). We had to do everything from scratch, but finally we can admit that it was worth it. Fig. 12. The same situation as in figure 11 seen by the robot's camera (it cannot see 180 degrees as laser do, so this image shows less information about obstacles than laser scans presented earlier).
26
Articles
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N째 3
September 2007
(load-from-path "guileplayer.scm") (define main-loop (lambda (client position sonar turn-counter) (cond ((not (player-client-signalled?)) (player-client-read-for-sure client) (main-loop client position sonar (cond ((> (player-sonar-scan-count sonar) 5) (cond ((< (player-sonar-scan-n sonar 2) 1.0) (player-position-set-cmd-vel position '(0.0 0.0 -1.0) player-enable ) (+ turn-counter 1) ) ((< (player-sonar-scan-n sonar 5) 1.0) (cond ((> turn-counter 10) (player-position-set-cmd-vel position '(0.0 0.0 -1.0) player-enable ) ) (else (player-position-set-cmd-vel position '(0.0 0.0 1.0) player-enable ) ) ) (+ turn-counter 1) ) (else (player-position-set-cmd-vel position `(,(player-sonar-scan-n sonar 3) 0.0 0.0) player-enable ) 0 ) ) ) (else (display (player-sonar-scan-count sonar)) (newline) turn-counter ) )) ) (else (display "going to quit...") (newline) ) ) )) (define client (player-client-create player-null "localhost" 6665)) (player-client-connect client) (define position (player-position-create client 0)) Articles 27
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N° 3
September 2007
(player-position-subscribe position player-all-access-mode) (define sonar (player-sonar-create client 0)) (player-sonar-subscribe sonar player-read-access-mode) (player-position-enable position player-enable) (player-client-trap-signal SIGINT) (player-client-trap-signal SIGTERM) (main-loop client position sonar 0) (player-position-set-cmd-vel position '(0.0 0.0 0.0) player-enable) (player-position-enable position player-disable) (player-position-unsubscribe position) (player-position-destroy position) (player-sonar-unsubscribe sonar) (player-sonar-destroy sonar) (player-client-disconnect client) (player-client-destroy client) Listing 2. Example Scheme script that implements very simple obstacle avoidance behavior. It uses sonar ring to detect distance to the obstacles.
5. Future work Currently we are preparing for upgrade of whole Virtual Laboratory infrastructure to be compatible with Player 2.0.x. Also we are intended to help in developing new version line (2.1) of Player, which is supposed to be more portable and able to run also natively also on MS Windows. Since PJIIT Robotics Laboratory students build their own robots, there is unavoidable need for writing Player drivers for them. A driver will be also required by a new global positioning device that is currently under construction and will be installed in our Virtual Robotics Laboratory.
[7]
[8]
AUTHOR Pawel Osmialowski - Polish-Japanese Institute of Information Technology, ul. Koszykowa 86, 02-008 Warszawa, Poland, e-mail: newchief@ai.pjwstk.edu.pl.
References [1] Brian Gerkey, Richard T. Vaughan and Andrew Howard. "The Player/Stage Project: Tools for Multi-Robot and Distributed Sensor Systems". In Proceedings of the 11th International Conference on Advanced Robotics (ICAR 2003), Coimbra, Portugal, June 2003, pages 317-323. Eric Steven Raymond. "The Art of Unix Programming", on-line book available at http://www.faqs.org/docs/ artu, 2003. Sam Williams. "Free as in Freedom: Richard Stallman's Cruscade for Free Software", on-line book available at http://www.faifzilla.org, 2002. Paul Ośmiałowski. "Implementation of distributed robotics framework and robotics hardware adaptation in the Virtual Robotics Laboratory", PJIIT, Master's thesis, Warsaw, Poland, June 2006. Janusz Matkowski. "Paradygmat Inteligencji Ucieleśnionej" [Paradigm of Intelligence Materialized], PJIIT, Master's thesis, Warsaw, Poland, October 2003. Rodney A. Brooks. "Elephants Don't Play Chess", MIT Artificial Intelligence Laboratory, Cambridge, Massachusetts, 1990.
[9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] [24]
Karol Yamazaki. "PaGo. System nawigacji oparty na analizie orientacji 3D obiektu" [PaGo. The navigation system based on orientation analysis of 3D object], PJIIT, Warsaw, Poland, 2006 Michal Dendewicz, Lukasz Hrynakowski. "Potential Field Method for Mobile Robot Navigation", PJIIT, Warsaw, Poland, 2006 http://vlab.pjwstk.edu.pl/files/PJIIT/vlabrelated http://www.gnu.org/software/octave http://cns.bu.edu/~cjmorlan/robotics/octplayer http://sourceforge.net/projects/guileplayer http://www.gnu.org/software/guile http://www.swig.org http://king.net.pl/playercontrib/videoplayer http://vlab.pjwstk.edu.pl http://sourceforge.net/projects/opencvlibrary http://www.gnu.org/software/gsl http://alioth.debian.org/projects/minicom http://www.gentoo.org http://www.mingw.org http://vlab.pjwstk.edu.pl/downloads http://www.libsdl.org http://vlab.pjwstk.edu.pl/files/PJIIT/SDL
[2]
[3]
[4]
[5]
[6]
28
Articles
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N° 3
September 2007
A DIRECT ALGORITHM OF POSSIBILISTIC CLUSTERING WITH PARTIAL SUPERVISION Received 18th May; accepted 26th June.
Dmitri A. Viattchenin
Abstract: Fuzzy clustering plays an important role in intelligent systems design and the respective methods constitute a part of the areas of automation and robotics. This paper describes a modification of a direct algorithm of possibilistic clustering that takes into account the information coming from the labeled objects. The clustering method based on the concept of allotment among fuzzy clusters is the basis of the new algorithm. The paper provides the description of basic ideas of the method and the plan of the basic version of a direct possibilistic-clustering algorithm. A plan of modification of the direct possibilistic-clustering algorithm in the presence of information from labeled objects is proposed. An illustrative example of the method's application to the Sneath and Sokal's two-dimensional data in comparison with the Gaussian-clustering method is carried out. Preliminary conclusions are formulated. Keywords: clustering, fuzzy tolerance, fuzzy cluster, membership degree, allotment, typical point, labeled object, and partial supervision
coefficients, the matrix of dissimilarity coefficients or the matrix of object attributes, should be divided into c fuzzy clusters. Namely, the grade µli , 1 £ l£ c, 1 £ i £ n, l to which an object xi belongs to the fuzzy cluster A should be determined. For each object xi , i=1,...n the grades of member-ship should satisfy the conditions of a fuzzy partition:
åm l =1
c
li
= 1, 1 £ i £ n; 0 £ mli £ 1, 1 £ l £ c
(1)
In other words, the family of fuzzy sets P(X)={Al½l=1,c,c £ n} is the fuzzy partition of the initial set of objects X=(x1 ,...xn) if condition (1) is met. Different authors proposed objective function-based fuzzy clustering algorithms, which are considered by Hoeppner, Klawonn, Kruse and Runkler [4]. If, on the other hand, condition
åm l =1
c
li
³ 1, 1 £ i £ n; 0 £ mli £ 1, 1 £ l £ c
(2)
1. Introduction Some remarks on fuzzy approach to clustering are considered in the first subsection. The second subsection includes a brief review of partially supervised fuzzy clustering methods and the aims of the paper. 1.1 Preliminary remarks In general, cluster analysis refers to a spectrum of methods, which try to divide a set of objects X=(x1 ,...xn) into subsets, called clusters, which are pair wise disjoint, all non empty and reproduce X via union. Heuristic methods, hierarchical methods, optimization methods and approximation methods are used as approaches to the cluster analysis problem solving. Clustering algorithms can also in general be divided into two types: hard versus fuzzy. Hard clustering assigns each object to exactly one cluster. In fuzzy clustering, founded upon fuzzy set theory [19], a given pattern does not necessarily belong to only one cluster, but can have varying degrees of memberships in several clusters. In heuristic methods of fuzzy clustering different researchers proposed hierarchical methods of fuzzy clustering and optimization methods of fuzzy clustering. These algorithms are described in [15]. The most widespread approach in fuzzy clustering is the optimization approach and the traditional optimization methods of fuzzy clustering are based on the concept of fuzzy partition. The initial set X=(x1,...xn) of n objects represented by the matrix of similarity
is met for each object x1 1 £ i £ n, then the corresponding family of fuzzy sets C(X)={Al½l=1,c,c £ n} is the fuzzy coverage of the initial set of objects X=(x1 ,...xn). The concept of fuzzy coverage is used mainly in heuristic fuzzy clustering procedures. A possibilistic approach to clustering was proposed by Krishnapuram and Keller [5]. The concept of possibilistic partition is the basis of possibilistic clustering methods and membership values µli , 1 £ l£ c, 1 £ i £ n can be interpreted as a typicality degree. For each object xi , i=1,...n the grades of membership should satisfy the conditions of a possibilistic partition:
åm l =1
c
li
> 0, 1 £ i £ n; 0 £ mli £ 1, 1 £ l £ c
(3)
So, the family of fuzzy sets Y(X)={Al½l=1,c,c £ n} is the possibilistic partition of the initial set of objects X=(x1,...xn) if condition (3) is met. The possibilistic approach to clustering was developed by Łęski [6], Zhang and Leung [20], Yang and Wu [18] and other researchers. This approach can be considered as a way in the optimization approach in fuzzy clustering because all methods of possibilistic clustering are objective function-based methods. Heuristic algorithms of fuzzy clustering display high level of essential clarity and low level of a complexity. Some heuristic clustering algorithms are based on a definition of a cluster concept and the aim of these algoArticles 29
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N° 3
September 2007
rithms is cluster detection conform to a given definition. Mandel [8] notes that such algorithms are called algorithms of direct classification or direct clustering algorithms. Direct heuristic algorithms of fuzzy clustering are simple and very effective in many cases. Fuzzy clustering can be considered as a technique of representation of the initial set of objects by fuzzy clusters. The structure of the set of objects can be described by some fuzzy tolerance, that is - a fuzzy binary intransitive relation. So, a fuzzy cluster can be understood as some fuzzy subset originated by fuzzy tolerance relation stipulating that the similarity degree of the fuzzy subset elements is not less than some threshold value. An outline for a new heuristic method of fuzzy clustering is presented by Viattchenin in [16], where concepts of fuzzy a-cluster and allotment among fuzzy a-clusters were introduced and a basic version of direct fuzzy clustering algorithm was described. The basic version of direct fuzzy clustering algorithm requires that the number c of fuzzy a-clusters be fixed. Some modifications of the basic version of the algorithm for different parameters of classification can be elaborated [17]. The version of the algorithm, which is presented in [16] is called the D-AFC(c)-algorithm. Note at this point that the name of AFC-algorithm was used for the fuzzy clustering algorithm, which was proposed by Dave in [3]. The allotment of elements of the set of classified objects among fuzzy clusters can be considered as a special case of possibilistic partition. That is why the D-AFC(c)-algorithm can be considered as a direct algorithm of possibilistic clustering. 1.2 A problem of partially supervised fuzzy clustering Partially supervised fuzzy clustering plays a unique role in discovering structure in data realized in the presence of labeled patterns. The circumstance is very useful in speech recognition systems and for elaboration of the robot vision systems. Some other problems related to robotics and automation can be successfully solved on the basis of partial-supervised clustering methods. A priori knowledge about belonging of some objects can be very useful for classification in many cases. This fact was the basis of an approach to fuzzy clustering with partial supervision. Algorithms of fuzzy clustering with partial supervision were proposed by Pedrycz in [9]. Numerical experiments show that knowledge concerning membership of a small portion of the patterns significantly improve clustering results in such a sense that the partition matrix detects a real structure existing in the data set. Moreover, the speed of convergence of the scheme has been improved. These facts are demonstrated by Pedrycz in [10]. Different researchers developed the idea of partial supervision in fuzzy clustering. For example, Bensaid, Hall, Bezdek, and Clarke proposed an original semi-supervised modification of the FCM-algorithm [1]. The method is well suited to problems such as image segmentation. In particular, the procedure was effectively applied to magnetic resonance images segmentation [1]. Very interesting and important results in the area of fuzzy clustering with partial supervision are presented by Bouchachia and Pedrycz in [2]. Articles
The main goal of the present paper is consideration of the modification of the D-AFC(c)-algorithm in the case of the presence of labeled objects. For this purpose, an outline of the method of possibilistic clustering based on the concept of allotment of elements of the set of classified objects among fuzzy clusters is presented. A mechanism of partial supervision for the method is proposed and a modification of the algorithm is described. The illustrative examples of application of the proposed method to the Sneath and Sokal's two-dimensional data in comparison with the basic version of the algorithm and the Li and Mukaidono's GCM-algorithm are given. Concluding remarks are stated and perspectives of research work are considered.
2. Outline of the method Basic concepts of the method and a plan of the basic version of the algorithm are considered in the first subsection. A mechanism of partial supervision for the method and a modification of the algorithm are proposed in the second subsection. Basic concepts Let us recall the basic concepts of the fuzzy clustering method based on the concept of allotment among fuzzy clusters, which was proposed in [16]. The concept of fuzzy tolerance is the basis for the concept of fuzzy a-cluster. That is why definition of fuzzy tolerance must be considered in the first place. Let X=(x1,...xn) be the initial set of elements and T : X ´ X ® [0,1] some binary fuzzy relation on X=(x1 ,...xn) with µT=(xi , xj) Î [0,1], "xi , xj Î X being its membership function. Definition 2.1. Fuzzy tolerance is the fuzzy binary intransitive relation, which possesses the symmetricity property µT (xi , xj)= µT (xj , xi), "xi , xj Î X , and the reflexivity property µT (xi , xj)=1, "xi Î X . (5) (4) 2.1
The notions of powerful fuzzy tolerance, feeble fuzzy tolerance and strict feeble fuzzy tolerance were considered in [16], as well. In this context the classical fuzzy tolerance in the sense of Definition 2.1. was called usual fuzzy tolerance and this kind of fuzzy tolerance was denoted by T2. So, the notions of powerful fuzzy tolerance, feeble fuzzy tolerance and strict feeble fuzzy tolerance must be considered too. Definition 2.2. The feeble fuzzy tolerance is the fuzzy binary intransitive relation, which possesses the symmetricity property (4) and the feeble reflexivity property µT (xi , xj) £ µT (xi , xi), "xi , xj Î X . This kind of fuzzy tolerance is denoted by T1. (6)
30
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N° 3
September 2007
Definition 2.3. The strict feeble fuzzy tolerance is the feeble fuzzy tolerance with strict inequality in (6): µT (xi , xj) < µT (xi , xi), "xi , xj Î X . This kind of fuzzy tolerance is denoted by T0. Definition 2.4. The powerful fuzzy tolerance is the fuzzy binary intransitive relation, which possesses the symmetricity property (4) and the powerful reflexivity property. The powerful reflexivity property is defined as the condition of reflexivity (5) together with the condition µT (xi , xj) < 1, "xi , xj Î X , xi ¹ xj . This kind of fuzzy tolerance is denoted by T3. Fuzzy tolerances T1 and T0 are subnormal fuzzy relations if the condition µT (xi , xi) < 1, "xi Î X. is met. The fact was demonstrated in [12]. The kind of the fuzzy tolerance imposed determines the nature of the implied of fuzzy clusters, as demonstrated in [13]. However, the essence of the method here considered does not depend on the kind of fuzzy tolerance. That is why the method herein is described for any fuzzy tolerance T. Let us consider the general definition of fuzzy cluster, the concept of the fuzzy cluster's typical point and the concept of the fuzzy allotment of objects. The number c of fuzzy clusters can be equal the number of objects, n. This is taken into account in further considerations. Let X=(x1 ,...xn) be the initial set of objects. Let T be a fuzzy tolerance on X and a be a-level value of T, a Î (0,1]. Columns or lines of the fuzzy tolerance matrix 1 n 1 n are fuzzy sets {A ,..., A }. Let {A ,..., A } be fuzzy sets on X, which are generated by a fuzzy tolerance T. Definition 2.5. The -level fuzzy set (8) (7)
sense (3) from the methodological positions. In other words, if columns or lines of fuzzy tolerance 1 n T matrix are fuzzy sets {A ,..., A } on X then fuzzy 1 n clusters { A(a ) ,..., A(a ) } are fuzzy subsets of fuzzy sets 1 n {A ,..., A } for some value a, a Î (0,1]. The value zero for a fuzzy set membership function is equivalent to nonbelonging of an element to a fuzzy set. That is why values of tolerance threshold are considered in the interval (0,1]. Definition 2.6. Let T is a fuzzy tolerance on X, where X 1 n is the set of elements, and { A(a ) ,..., A(a ) } is the family of l l fuzzy clusters for some a Î (0,1]. The point t e , for Î Aa which l t el = arg max mli , "xi Î Aa xi
(10)
is called a typical point of the fuzzy cluster A(la ) , a Î (0,1], l Î [1, n]. Obviously, a typical point of a fuzzy cluster does not depend on the value of tolerance threshold. Moreover, a fuzzy cluster can have several typical points. That is why symbol e is the index of the typical point. Definition 2.7. Let of fuzzy clusters for some value of tolerance threshold a, a Î (0,1], which are generated by some fuzzy tolerance T on the initial set of elements X=(x1 ,...xn). If condition a Rz ( X ) = { A(la ) | l = 1, c,2 £ c £ n, a Î (0,1]} be a family
åm l =1
c
li
> 0, "xi Î X l
(11)
fuzzy a-cluster or, simply, fuzzy cluster. l l l 1 n So A(a ) Í A , a Î (0,1], A Î { A , K , A } and µij is the membership degree of the element x1 Î X for some l fuzzy cluster A(a ) , a Î (0,1], l Î [1, n] . Value of a is the toleran-ce threshold of fuzzy clusters elements. The membership degree of the element xi Î X for l some fuzzy cluster A(a ) , a Î (0,1], l Î [1, n] can be defined as a
A(la ) = {( xi , m Al ( xi )) | m Al ( xi ) ³ a , xi Î X , l Î [1, n]} is
ì l ïm Al ( xi ), xi Î Aa mli = í , (9) ï0, otherwise î l = {xi Î X | m Al ( xi ) ³ a }, a Î (0,1] where a a-level Aa l of a fuzzy set A is the support of the fuzzy cluster A(la ) . l = Supp ( A(la ) ) is met for each fuzzy So, condition Aa l cluster A(a ) , a Î (0,1], l Î [1, n] . Membership degree can be interpreted as a degree of typicality of an element to a fuzzy cluster. The value of a membership function of each element of the fuzzy cluster in the sense of definition 2.5 is the degree of similarity of the object to some typical object of fuzzy cluster. So, fuzzy clusters in the definition 2.5 are different from fuzzy clusters in the
is met for all A(a ) , l = 1, c, c £ n , then the family is the allotment of elements of the set X=(x1 ,...xn) among l fuzzy clusters { A(a ) , l = 1, c,2 £ c £ n} for some value of the tolerance threshold a, a Î (0,1]. a It should be noted that several allotments Rz (X ) could exist for some tolerance threshold a, a Î (0,1]. That is why symbol z is the index of an allotment. The condition (11) requires that every object xi , i=1,n must be assigned to at least one fuzzy cluster A(la ) , l = 1, c, c £ n with the membership degree higher than zero. The condition 2 £ l£ c requires that the a number of fuzzy clusters in Rz ( X ) must be more than two. Otherwise, the unique fuzzy cluster will contain all objects possibly with different positive membership degrees. Obviously, the definition of the allotment among fuzzy clusters (11) is similar to the definition of the possibilistic partition (3). Moreover, each column R(l) , l=1,...,c of the allotment matrix Rc´n=[µli], i=1,...,n l=1,...,c can be considered as a possibility distribution on X. So, the allotment among fuzzy clusters can be considered as the possibilistic partition and fuzzy clusters in the sense of definition 2.5 are elements of the possibilistic partition. However, the concept of allotment will be used in further considerations. The concept of allotment is the central point of the method. But the next concept introduced should be paid attention to, as well. Articles 31
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N° 3
September 2007
Definition 2.8. Allotment
among n fuzzy clusters for some tolerance threshold a, a Î (0,1] is the initial allotment of the set X= (x1 ,...,xn). In other words, if initial data are represented by a matrix of some fuzzy T then lines or columns of the matrix are fuzzy sets Al Í X, l=1,n and level fuzzy l sets A(a ) , l = 1, n, a Î (0,1] are fuzzy clusters. These fuzzy clusters constitute an initial allotment for some tolerance threshold and they can be considered as clustering components. Thus, the problem of fuzzy cluster analysis can be defined in general as the problem of discovering the unique allotment R*(X), resulting from the classification process, which corresponds to either most natural allocation of objects among fuzzy clusters or to the researcher's opinion about classification. In the first case, the number of fuzzy clusters c is not fixed. In the second case, the researcher's opinion determines the kind of the allotment sought and the number of fuzzy clusters can be fixed. If some allotment a Rz ( X ) = { A(la ) | l = 1, c, c £ n, a Î (0,1]} corresponds to the formulation of a concrete problem, then this allotment is an adequate allotment. In particular, if condition
RIa ( X ) = { A(la ) | l = 1, n, a Î (0,1]} of the set of objects
usly, if w=0 in conditions (14) and (15) then conditions (12) and (13) are met. a The adequate allotment Rz ( X ) for some value of tolerance threshold a, a Î (0,1] is a family of fuzzy clusa ters which are elements of the initial allotment Rz ( X ) for the value of a and the family of fuzzy clusters should satisfy either the conditions (6) and (7) or the conditions (14) and (15). So, the construction of adequate l a allotments Rz ( X ) = { A(a ) | l = 1, c, c £ n, a Î (0,1]} for every a, a Î (0,1] is a trivial problem of combinatorics. Several adequate allotments can exist. Thus the problem consists in the selection of the unique adequate allotment R*(X) from the set B of adequate a allotments, B = {Rz ( X )} , which is the class of possible solutions of the concrete classification problem and depends on the parameters the classification problem. The selection of the unique adequate allotment R*(X) a from the set B = {Rz ( X )}of adequate allotments must be made on the basis of evaluation of allotments. The criterion a ( X ), a ) = å F1 ( Rz l =1 c
1 nl
åm i =1
nl
li
-a ×c ,
(16)
UA l =1
c
l a
= X,
where c is the number of fuzzy clusters in the allotl l a a ment Rz ( X ) and nl = card ( Aa ), A(a ) Î Rz ( X ) is the number of elements in the support of the fuzzy cluster A(la ) , can be used for evaluation of allotments. The criterion a F2 ( Rz ( X ), a ) = åå ( mli - a ) , l =1 i =1 c nl
(12)
(17)
and condition (13) l m card ( Aa Ç Aa ) = 0, "A(la ) , A(m a ) , l ¹ m, a Î (0,1]
are met for all fuzzy clusters A(la ) , l = 1, c of some ala ( X ) = { A(la ) | l = 1, c, c £ n, a Î (0,1]} then lotment Rz the allotment is the allotment among fully separate fuzzy clusters. However, fuzzy clusters in the sense of definition 2.5 can have an intersection area. This fact was demonstrated in [17]. If the intersection area of any pair of different fuzzy cluster is an empty set, then condition (13) is met and fuzzy clusters are called fully separate fuzzy clusters. Otherwise, fuzzy clusters are called particularly separate fuzzy clusters and w={0,...,n} is the maximum number of elements in the intersection area of different fuzzy clusters. Obviously, for w=0 fuzzy clusters are fully separate fuzzy clusters. So, the conditions (12) and (13) can be generalized for a case of particularly separate fuzzy clusters. Condition a l ) ³ card ( X ), "A(la ) Î R z ( X ), å card ( Aa c
can also be used for evaluation of allotments. Both criteria were proposed in [14]. Maximum of criterion (10) or criterion (11) corresponds to the best allotment of objects among c fuzzy clusters. So, the classification problem can be characterized formally as determination of the solution R*(X) satisfying a R * ( X ) = arg a max F ( Rz ( X ), a ) , Rz ( X )ÎB
(18)
where B = {Rz ( X )} is the set of adequate allotments corresponding to the formulation of a concrete classification problem and criteria (16) and (17) are denoted a by F ( Rz ( X ), a ) . The criterion (16) can be considered as the average total membership of objects in fuzzy clusters of the a allotment Rz ( X ) minus a · c. The quantity a · c regularizes with respect to the number of clusters c in the a allotment Rz ( X ) . The criterion (17) can be considered as the total membership of objects in fuzzy clusters a of the allotment Rz ( X ) with an appreciation through the value a of tolerance threshold. The condition (18) a must be met for the some unique allotment Rz ( X ) Î B(c) . Otherwise, the number c of fuzzy clusters in the allotment sought R*(X) is suboptimal. Detection of fixed c number of fuzzy clusters can be considered as the aim of classification. So, the adequate a allotment Rz ( X ) is any allotment among c fuzzy clusters in the case. There is the D-AFC(c)-algorithm:
a
(14)
l =1
a a Î (0,1], card ( R z ( X )) = c ,
and condition (15) l m card ( Aa Ç Aa ) £ w, "A(la ) , A(m a ) , l ¹ m, a Î (0,1] ,
are generalizations of conditions (12) and (13). Obvio32 Articles
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N° 3
September 2007
1. Calculate a-level values of the fuzzy tolerance T and construct the sequence 0<a0<a1<...<al<...<aZ£1 of a-levels; 2. Construct the initial allotment R Ia ( X ) = { A(la ) | l = 1, n}, a = al for every value al from the sequence 0<a0<a1 <...<al<...<aZ £ 1; 3. Let w := 0; l 4. Construct allotments R a z ( X ) = { A(a ) | l = 1, c, c £ n}, a = al which satisfy conditions (14) and (15) for every value al from the sequence 0<a0<a1<...< al<...<aZ £ 1; 5. Construct the class of possible solutions of the classia fication problem B(c) = {R z ( X )}, a Î{a 1 , K , a Z } for the given number of fuzzy clusters c and different values of the tolerance threshold a, a Î {a1,...,aZ} as follows: a ( X ), a Î{a 1 , K , a Z } the if for some allotment R z a condition card ( R z ( X )) = c is met, a ( X ) Î B (c ) then R z else let w := w+1 and go to step 4. 6. Calculate the value of some criterion F ( R a z ( X ), a ) for a every allotment R z ( X ) Î B(c) ; 7. The result R*(X) of classification is formed as follows: if for some unique allotment R a z ( X ) from the set B(c) the condition (18) is met, then the allotment is the result of classification else the number c of classes is suboptimal. a ( X ) = { A(la ) | l = 1, c, a Î (0,1]} The allotment Rz among the given number of fuzzy clusters and the corresponding value of tolerance threshold a, a Î (0,1] are the results of classification.
a fication problem B(c) = {R z ( X )}, a Î{a 1 , K , a Z } for the given number of fuzzy clusters c and different values of the tolerance threshold a, a Î {a1,...,aZ} as follows: a ( X ), a Î{a 1 , K , a Z } the if for some allotment R z a condition card ( R z ( X )) = c is met and for every labeled object xL(j)=xi , j=1,c,i a ( X ), Î{1,...,n} the condition m li ³ y lj , A(la ) Î R z l=1,...,c is met, a ( X ) Î B (c ) then R z else let w := w+1 and go to step 4. 6. Calculate the value of criterion F ( R a z ( X ), a ) for every a ( X ) Î B (c ) ; allotment R z 7. The result R*(X) of classification is formed as follows: if for some unique allotment R a z ( X ) from the set B(c) the condition (18) is met, then the allotment is the result of classification else the number c of classes is suboptimal.
The proposed modification of the D-AFC(c)-algorithm can be called the D-AFC-PS(c)-algorithm. Obviously, the modification does not differ significantly from the basic version of the clustering procedure.
3. An illustrative example The Sneath and Sokal's two-dimensional data and results of their processing by the GCM-algorithm are considered in the first subsection of the section. Results of three numerical experiments with the proposed procedure are presented in the second subsection. 3.1. The Sneath and Sokal's data In this section an artificial data set is used for testing of the proposed clustering procedure. These data originally appear as Table 1 in [11] and are shown here in Fig. 1. Li and Mukaidono applied their GCM-algorithm [7] to this data set for the number of classes c=2. The results of the GCM application are presented also in Table 1. Table 1. The Sneath and Sokal's data set and the results of its processing by the GCM-algorithm. Numbers of objects, i 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
2.2. A mechanism of partial supervision Let us consider a subset of labeled objects XL={xL(1),...,xL(c)} and XL Ì X. A condition card(XL)=c must be met for the subset. Let the membership grades yl(j) , l=1,...,c, j=1,...,c correspond to each labeled object xL(j) Î XL , j=1,...,c as follows: if xi Î XL and xi = XL(j), the values of yl(j) are given by researcher. So, detection of fixed c number of fuzzy clusters can be considered as the aim of classification and each labeled object must be assigned to a unique fuzzy cluster. Moreover, for each labeled object xi = XL(j) its membership value µli , l=1,...,c, i=1,...,n in the sought allotment R*(X) must be greater than a priori determined membership grade yl(j) Î (0,1]. There is a seven-step procedure of classification: 1. Calculate a-level values of the fuzzy tolerance T and construct the sequence 0<a0<a1<...<al<...<aZ £ 1 of a-levels; 2. Construct the initial allotment R Ia ( X ) = { A(la ) | l = 1, n}, a = al for every value al from the sequence 0<a0<a1 <...<al<...<aZ £ 1; 3. Let w := 0; l 4. Construct allotments R a z ( X ) = { A(a ) | l = 1, c, c £ n}, a = al, which satisfy conditions (14) and (15) for every value al from the sequence 0<a0<a1<...< al<...<aZ £ 1; 5. Construct the class of possible solutions of the classi-
ˆ1 , x ˆ2) xi = ( x ˆ1 ˆ2 x x 0 0 1 2 3 2 2 1 5 6 7 5 7 6 6 8 4 3 5 4 3 2 1 0 5 5 6 3 3 2 1 1
The data,
Membership grades obtained from the GCM-algorithm µ1i µ2i 1.00 0.00 1.00 0.00 1.00 0.00 0.97 0.03 0.84 0.16 0.98 0.02 0.99 0.01 1.00 0.00 0.05 0.95 0.01 0.99 0.00 1.00 0.07 0.93 0.00 1.00 0.01 0.99 0.01 0.99 0.00 1.00 Articles 33
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N° 3
September 2007
Obviously, two well-separated classes can be distinguished. In particular, the Gaussian membership function is sharper than the membership function obtained from the FCM-algorithm. The fact was shown by Li and Mukaidono in [7].
for all attributes x , t=1,...,m. So, each object can be t considered as a fuzzy set xi , i=1,...n and µxi (x )Î[0,1], i=1,...,n, t=1,...,m are the corresponding membership functions. After application of the normalized Euclidean distance (20) 2 1 m e( x i , x j ) = å m xi ( x t ) - m x j ( x t ) , i, j = 1, n , m t =1
t
(
)
to the matrix of normalized data
a fuzzy intolerance I=[µI(xi , xj)], i,j=1,...,n is obtained. The matrix I=[µI(xi , xj)], i,j=1,...,n is the matrix of pair wise dissimilarity coefficients. The matrix of fuzzy tolerance T=[µT(xi , xj)], i,j=1,...,n is obtained after application of the complement operation µT(xi , xj)=1- µI(xi , xj), "i,j=1,...n (21)
t ˆ the matrix of X n´m = [ m xi ( x )], i = 1, K , n, t = 1, K , m
to the matrix of fuzzy intolerance I=[µI(xi , xj)], i,j=1,...,n. The results of the data set processing by the D-AFC(c)algorithm must be considered in the first place. The matrix of the allotment between two fuzzy clusters is presented in Table 2. Table 2. Results of the Sneath and Sokal's data set classification obtained from the D-AFC(c)-algorithm for c=2. Numbers of objects, i 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Membership grades µ1i 0.88214887 1.00000000 0.74826988 0.78754085 0.73483496 0.78754085 0.70537217 0.63556551 0.00000000 0.00000000 0.00000000 0.00000000 0.00000000 0.00000000 0.00000000 0.00000000 µ2i 0.00000000 0.00000000 0.00000000 0.00000000 0.64644661 0.00000000 0.00000000 0.00000000 0.70537217 0.74826988 0.64644661 0.82322330 1.00000000 0.85268609 0.74826988 0.74826988
Fig. 1. The Sneath and Sokal's two-dimensional data set. A diagram can illustrate the matrix of the fuzzy partition. Membership functions of two classes are presented in Fig. 2.
Fig. 2. Membership functions obtained from the GCMalgorithm. Membership values of the first class are represented in Fig. 2 by ¾ and membership values of the second class are represented by ¢. The results, obtained from the GCM-algorithm will be very useful for the following considerations. 3.2. Experimental results Let us consider results of application of the proposed D-AFC-PS(c)-algorithm to the Sneath and Sokal's data t set. The matrix of attributes is the matrix X m´n = [ xi ], i=1,...,n,t=1,...,m, where n=16 and m=2. So, the t value xi is the value of the t-th attribute for the i-th object. The data can be normalized as follows:
xit =
xit , i = 1, K , n , max xit i
(19)
i = 1, K, n, t = 1, K , m 34 Articles
By executing the D-AFC(c)-algorithm for two classes we obtain the following: the first class is formed by 8 elements and the second class is composed of 9 elements. The fifth element belongs to both classes. The allotment R*(X), which corresponds to the result, was obtained for the tolerance threshold a = 0.57508171. The value of the membership function of the fuzzy cluster, which corresponds, to the first class is maximal for the second object and is equal one. So, the second object is the typical point of the first fuzzy cluster. The membership value of the thirteenth object is equal one for the second fuzzy cluster. Thus, the thirteenth object is the typical point of the second fuzzy cluster. Membership functions of two classes of the allotment are
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N° 3
September 2007
presented in Fig. 3 and values, which equal zero, are not shown in the figure.
Table 3. Results of the Sneath and Sokal's data set classification obtained from the D-AFC-PS(c)-algorithm in the first experiment. Numbers of objects, i 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Membership grades µ1i 0.60471529 0.70537217 0.52038065 0.64644661 0.74826988 0.88214887 1.00000000 0.85268609 0.00000000 0.00000000 0.00000000 0.64522112 0.49913270 0.62732200 0.64644661 0.46966991 µ2i 0.00000000 0.00000000 0.00000000 0.49913270 0.50000000 0.00000000 0.00000000 0.00000000 0.78754085 0.85268609 1.00000000 0.60471529 0.64644661 0.52038065 0.00000000 0.00000000
Fig. 3. Membership function obtained from the D-AFC(c)algorithm for two classes. Note, that the Gaussian membership function is sharper than the membership function, which is obtained from the D-AFC(c)-algorithm, but the essential interpretation of the results which obtained are from the DAFC(c)-algorithm is better than in the case of GCMalgorithm. The results are illustrated also in Fig 4. Supports of two fuzzy clusters are distinguished in Fig. 4 and typical points are denoted by ¢.
By executing the D-AFC-PS(c)-algorithm for the set of labeled objects XL = {x5 = xL(1) , x9 = xL(2)} with their membership functions y1(5)=0.6 and y2(9)=0.6 we obtain the following: the first class is formed by 13 elements and the second class is composed of 8 elements. Five objects are elements of both classes. The allotment R*(X), which corresponds to the result, was obtained for the tolerance threshold a = 0.46966991. Membership functions of two classes of the allotment are presented in Fig. 5.
Fig. 4. Supports and typical points of fuzzy clusters obtained from the D-AFC(c)-algorithm for two classes. Let us consider the results of experiments with the proposed D-AFC-PS(c)-algorithm. The first experiment was made for the set of labeled objects XL = {x5 = xL(1) , x9 = xL(2)} with their membership functions y1(5)=0.6 and y2(9)=0.6. Results of the first experiment are presented in Table 3. Fig. 5. Membership function obtained from the D-AFCPS(c)-algorithm in the first experiment. The value of the membership function of the first fuzzy cluster is equal one for the seventh object. The seventh object is the typical point of the first fuzzy cluster. The membership value of the eleven object is equal one for the second fuzzy cluster. Thus, the eleventh object is the typical point of the fuzzy cluster, which corresponds to the second class. Fig. 6 illustrates supports of fuzzy clusters and their typical points.
Articles
35
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N° 3
September 2007
classes of the allotment are presented in Fig. 7.
Fig. 6. Supports and typical points of fuzzy clusters obtained from the D-AFC-PS(c)-algorithm in the first experiment. The second experiment was made for the set of labeled objects XL = {x7 = xL(1) , x5 = xL(2)} with their membership functions y1(7)=0.8, y2(5)=0.8. The results are presented in Table 4. Table 4. Results of the Sneath and Sokal's data set classification obtained from the D-AFC-PS(c)-algorithm in the second experiment. Numbers of objects, i 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Membership grades µ1i 0.00000000 0.63556551 0.00000000 0.00000000 0.60471529 0.74826988 0.85268609 1.00000000 0.00000000 0.00000000 0.00000000 0.00000000 0.00000000 0.00000000 0.54261462 0.00000000 µ2i 0.54261462 0.55805826 0.57508171 0.70982524 0.82322330 0.70982524 0.64522112 0.00000000 0.76429774 0.74826988 0.60471529 1.00000000 0.82322330 0.85268609 0.74826988 0.64522112
Fig. 7. Membership function obtained from the D-AFCPS(c)-algorithm in the second experiment. Fig. 8 illustrates supports of fuzzy clusters and their typical points.
Fig. 8. Supports and typical points of fuzzy clusters obtained from the D-AFC-PS(c)-algorithm in the second experiment. The third experiment was performed for the set of labeled objects XL = {x5 = xL(1) , x16 = xL(2)} with their membership functions y1(5)=1.0 and y2(16)=1.0. The results of the experiment are presented in Table 5. Obviously, the labeled objects will be typical points in the sought allotment in this case. Table 5. Results of the Sneath and Sokal's data set classification obtained from the D-AFC-PS(c)-algorithm in the third experiment. Numbers of objects, i 1 2 3 4 5 6 Membership grades µ1i 0.70982524 0.73483496 0.70537217 0.85268609 1.00000000 0.85268609 µ2i 0.00000000 0.00000000 0.00000000 0.00000000 0.00000000 0.00000000
By executing the D-AFC-PS(c)-algorithm for the set of labeled objects XL = {x7 = xL(1) , x5 = xL(2)} with their membership functions y1(7)=0.8 and y2(5)=0.8 we obtain the following: the first class is formed by 6 elements and the second class is composed of 15 elements. Five objects are elements of both classes. The allotment R*(X) was obtained for the tolerance threshold a = 0.52859548. The value of the membership function of the first fuzzy cluster is maximal for the eighth object. So, the eighth object is the typical point of the fuzzy cluster, which corresponds to the first class. The membership value of the twelfth object is equal one for the second fuzzy cluster. The twelfth object is the typical point of the second fuzzy cluster. Membership functions of two 36 Articles
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N° 3
September 2007
Numbers of objects, i 7 8 9 10 11 12 13 14 15 16
Membership grades µ1i 0.74826988 0.60471529 0.70537217 0.64522112 0.50000000 0.82322330 0.64644661 0.70982524 0.64522112 0.00000000 µ2i 0.00000000 0.00000000 0.00000000 0.00000000 0.00000000 0.64522112 0.74826988 0.78754085 0.82322330 1.00000000
Membership values of the first class are represented in all figures by ¾ and membership values of the second class are represented in all figures by ¢. Results of experiments show that the results of classification depend on the set of labeled objects and their a priori membership functions.
4. Concluding remarks Some remarks to the results of numerical experiments are made in the first subsection. Perspectives on future investigations are outlined in the second subsection. 4.1. Discussion of the experimental results In conclusion it should be said that the concept of fuzzy cluster and allotment have an epistemological motivation. That is why the results of application of the fuzzy clustering method based on the allotment concept can be very well interpreted. Moreover, the fuzzy clustering method based on the allotment concept depends on the set of adequate allotments only. That is why the clustering results are stable. The D-AFC-PS(c)-algorithm of possibilistic clustering is proposed in the paper. The algorithm is based on the mechanism of partial supervision. Numerical experiments show that a result of the D-AFC-PS(c)-algorithm application to the data set depends on the choice of the labeled objects and on their a priori membership functions. The D-AFC-PS(c)-algorithm can be applied directly to the data given as the matrix of tolerance coefficients. This means that it can be used with the objects by attributes data, by choosing a suitable metric to measure similarity or it can be used in situations where objects by objects proximity data is available. The results of application of the D-AFC-PS(c)-algorithm to the Sneath and Sokal's data set show that the D-AFC-PS(c)-algorithm is a precise and effective numerical procedure for solving classification problem in the case of the presence of labeled objects. 4.2. Perspectives Given membership functions can be different for different labeled objects. A problem of choosing of the membership function values for the labeled objects must be investigated. Moreover, the method can be extended for the case of presence of a few labeled objects for every class in the sought allotment. These perspectives for investigations are of great interest both from the theoretical point of view and from practical one as well. ACKNOWLEDGEMENTS I am grateful to Dr. Jan W. Owsinski and Prof. Janusz Kacprzyk for their useful remarks during the paper preparation. I would like to thank the Director of the Systems Research Institute of the Polish Academy of Sciences for the possibility of conducting the investigations and the Mianowski Fund and Siemens for financial support. I also thank Mr. Aliaksandr Damaratski for elaborating experimental software.
By executing the D-AFC-PS(c)-algorithm for the set of labeled objects XL = {x5 = xL(1) , x16 = xL(2)} with their membership functions y1(5)=1.0, y2(16)=1.0 we obtain that the first class is formed by 15 elements and the second class is composed of 5 elements. Four objects are elements of both classes. Membership functions of two classes are presented in Fig. 9. The allotment R*(X) was obtained for the tolerance threshold a = 0.500.
Fig. 9. Membership function obtained from the D-AFCPS(c)-algorithm in the third experiment. Fig. 9 illustrates supports of fuzzy clusters and their typical points.
Fig. 10. Supports and typical points of fuzzy clusters obtained from the D-AFC-PS(c)-algorithm in the third experiment. Articles 37
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N° 3
September 2007
AUTHOR Dmitri A. Viattchenin - United Institute of Informatics Problems of the National Academy of Sciences of Belarus, Surganov St. 6, 220012 Minsk, Belarus, e-mail: viattchenin@mail.ru.
[18]
[19] [20]
References [1] Bensaid A.M., Hall L.O., Bezdek J.C., and Clarke L.P., “Partially supervised clustering for image segmentation”, Pattern Recognition, vol. 29, no. 5, 1996, pp. 859-871. Bouchachia A. and Pedrycz W., “Enhancement of fuzzy clustering by mechanisms of partial supervision”, Fuzzy Sets and Systems, vol. 157, issue 13, 2006, pp. 1733-1759. Davé R.N., “Use of the adaptive fuzzy clustering algorithm to detect lines in digital images”, Intelligent Robots and Computer Vision, 1989, vol. 1192, pp. 600-611. Höppner F., Klawonn F., Kruse R. and Runkler T., Fuzzy Cluster Analysis: Methods for classification, data analysis and image recognition, Chichester: Wiley Intersciences, 1998. Krishnapuram R. and Keller J.M., “A Possibilistic Approach to Clustering”, IEEE Transactions on Fuzzy Systems, vol. 1, 1993, pp. 98-110. Łęski J.M., “Robust possibilistic clustering”, Archives of Control Sciences, vol. 10, 2000, pp. 141-155. Li R. and Mukaidono M., “Gaussian clustering method based on maximum-fuzzy-entropy interpretation”, Fuzzy Sets and Systems, vol. 102, 1999, pp. 253-258. Mandel I.D., Clustering analysis, Moscow: “Finansy i Statistica” Publishing House, 1988, (in Russian). Pedrycz W., “Algorithms of fuzzy clustering with partial supervision”, Pattern Recognition Letters, vol. 3, 1985, pp. 13-20. Pedrycz W., “Fuzzy sets in pattern recognition: methodology and methods”, Pattern Recognition, vol. 23, 1990, pp. 121-146. Sneath P.H.A. and Sokal R., Numerical Taxonomy, San Francisco: Freeman, 1973. Viattchenin D.A., “On Projections of Fuzzy Similarity Relations”, Proc. of the 5th International Conference on Computer Data Analysis and Modeling CDAM'1998, Minsk, Belarus, 2003, vol. 2, pp. 91-94. Viattchenin D.A., “Remarks on Kinds of Fuzzy Clusters”, Proc. of the International Conference “Statistical Data Analysis of Quality of Life”, Wroclaw, Poland, 1999, pp. 69-79. Viattchenin D.A., “Criteria of Quality of Allotment in Fuzzy Clustering”, Proc. of the 3rd International Conference on Neural Networks and Artifical Intelligence ICNNAI'2003, Minsk, Belarus, 2003, pp. 91-94. Viattchenin D.A., Fuzzy methods of automatic classification, Minsk: Technoprint Publishing House, 2004, (in Russian). Viattchenin D.A., “A new heuristic algorithm of fuzzy clustering”, Control & Cybernetics, vol. 33, 2004, pp. 323-340. Viattchenin D.A., “Parameters of the AFCmethod of fuzzy clustering”, Bulletin of The Military Academy of The
Republic of Belarus, no. 4, 2004, pp. 51-55, (in Russian). Yang M.-S. and Wu K.-L., “Unsupervised possibilistic clustering”, Pattern Recognition, 2006, vol. 39, pp. 5-21. Zadeh L.A., “Fuzzy Sets”, Information and Control, vol. 8, 1965, pp. 338-353. Zhang J.-S. and Leung Y.-W., “Improved Possibilistic CMeans Clustering Algorithms”, IEEE Transactions on Fuzzy Systems, 2004, vol. 12, pp. 209-217.
[2]
[3]
[4]
[5]
[6] [7]
[8] [9]
[10]
[11] [12]
[13]
[14]
[15]
[16]
[17]
38
Articles
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N° 3
September 2007
METHODS AND SYSTEMS OF LEAK DETECTION IN LONG RANGE PIPELINES Received 6th June; accepted 10th July.
Mateusz Turkowski, Andrzej Bratek, Marcin SĹ&#x201A;owikowski
Abstract: No matter how carefully the pipeline is designed and built, there is always a probability of leaks. Pipeline leak detection systems play therefore a key role in minimization of the occurrence of leaks probability and their impacts. As the vast assortment of technologies is available today, the paper provides the background information to facilitate the choice of leak-detection system. The paper deals with external methods based on external measurements and with some internal methods based on flow and pressure measurements in the pipeline: pressure wave detection and volume balance method; then gradient and analytical methods were described. Additionally, the discussion of error sources and final conclusions are presented. Keywords: leak detection, leak localization, integrity of pipelines, transport pipelines, pipelines safety
Fig. 2. Consequences of inattentive diggings [3]. Degradation of the pipeline material may occur as a result of stresses alterations that are caused by changes of the pressure and the deformations of the pipeline caused by the soil dislocations, which lead to the fatigue and forming of the micro-gaps. During operation periodical inspections of the pipeline condition are carried out with the use of so-called intelligent pistons, it enables to detect and localize damages or leaks at a very early stage of development. Despite of that the occurrence of leaks is rather inevitable. They are caused by oversight or by underestimating progress of the specific defects during the tests. However, considerably more often damages are caused by people operations: accidental - careless diggings (Fig. 2), building or repair works in the proximity of the pipeline, or conscious - i.e. illegal stealing of the transported media. Also possible terrorist attacks cannot be neglected. The leak brings always large and various losses: suspending of the product or material transport, the cost of the damage reparation and loss of transported product. In case of explosive or/and flammable or/and dangerous to environment media (e.g. petroleum and other oil products), the leak causes hazard for safety of the people and the equipment (including the pumping installation), as well as an environmental contamination. Events like that induce high social and financial costs, which are proportional to the intensity and duration of the leak. The recultivation costs of the polluted ground may reach several millions euro. If a leak happened, then its effects can be minimized only by extremely fast detection and localization of the leak point and quick dispatcher reaction (stopping pumArticles 39
1. Introduction Construction of the transport pipeline is always very costly. Therefore all stages of its building (beginning with a project, and finishing with the tests and commissioning) and operating must fulfill the regulations and recommendations of numerous standards and regulations [1], which purpose is to provide long-lasting operation of pipeline system. The damages may be caused by corrosion or erosion (Fig. 1) of external and internal walls, inhomogeneity or crack of the welds, indentation of the walls and material defects. Counteraction may consist in installing cathodic protection systems (earlier passive, but nowadays predominantly active, under surveillance of the telemetry system).
Fig. 1. Numerical visualization of the leak caused by erosion [2].
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N째 3
September 2007
ping, closing the valves, organizing provisional damage repair etc.) Damages of pipelines transporting natural gas are not neutral for environment as well. Methane, the main component of natural gas - is one of the greenhouse gases. Moreover, natural gas mixed with atmospheric air in amount 4-15% becomes very sensitive for sparkling or flame, causing an explosion of this mixture - as is shown in the Fig. 3.
The flow rate in liquid pipelines is usually constant, close to the nominal flow rate, limited by pumps efficiency and by pressure drop along the pipeline. In gas pipelines the flow rate is a function of gas demand, on which gas supplier does not have much influence. Gas demand is variable during twenty-four hours cycle (rising within a day) and during the year (rising within low temperature periods). Table 1. The influence of various factors on pipeline behavior and leak detection systems in liquids and gas pipelines. Nr 1 Criterion complication level of the pipeline system inertia of liquid compressibility of medium elastic deformations of the pipeline caused by changes of pressure variability of flow
Liquid not large
Gas large (except wide-ranging transit-pipelines) small large
Fig. 3. Burning pipeline a water curtain is applied. Photo by M. Turkowski. Particularly dangerous leaks of a gas pipeline happen in the winter. Under frozen crust of the earth gas may penetrate in long distance, and an explosion may happen in quite unexpected places. That is why for over 30 years research in the domain of leak detection and localization systems is carried out all over the world. Leak detection problems in single segments of pipeline in the steady state are generally solved. Up to now, however, a universal reliable system, which can work in every operating condition, for every pipeline system and during transients does not exist. Therefore, the issue of leak detection will be probably topical for a long time. Complications came from fact that individual pipelines differ from each other, and in each case one has to place special emphasis on totally different circumstances, which are crucial for given detection system. The examples of such differences between gas and liquid pipelines are presented in table 1. Configuration of liquid pipeline is usually quite simple - most often we have a single pipeline, starting at the supplier and ending at the recipient of medium, the branches are rather rare. A uniform gasification of an area requires however much more complicated configurations. It is clearly shown on the maps at the websites of gas and liquid pipelines operators [4, 5]. The inertia of a fluid has not great influence on the gas pipelines because of the low gas density (unless sudden and big changes of demand or very high pressures and diameters of the pipe), whereas the inertia can never be omitted in case of liquid pipelines. The medium compressibility has great influence in the case of gas pipelines and it is less important for liquid pipelines. Liquid compressibility joined with elastic deformations of pipeline lead however to sustained (tens of minutes) transient states after technological operations (start of pumping, valves operations), which may cause a water hammer phenomenon. Compressibility is less significant for stationary flows. 40 Articles
2 3
large small
4
large
small
5
steady, near to nominal (except the transient states)
various between day and night and depending on the season constant flow
6
continuity of work presence of the second phase
interrupted flow large influence
7
small influence
Liquid pipeline can be switched on and off periodically, depending on contracts between the supplier and the recipient. On the contrary, a gas pipeline once started should be operated continuously for all operating time (often for several dozen years), excepting possible failures. Presence of gas inclusions in liquid have a fundamental effect on liquid compressibility (1% of gas inclusions in form of uniformly spaced tiny bubbles can multiply compressibility of such mixture even several hundreds times), whereas similar content of liquid aerosol in gas may be neglected from the point of view of detection system. The table 1 refers also to plant technological pipelines, they are however much shorter than long-range transport pipelines, and because of that the influence of some factors may be omitted.
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N° 3
September 2007
2. Tasks and general classification of the leak detection and localization methods Detection of the gas pipelines leaks system should comply three following tasks: Leak detection. Alarm generation. Leak localization. Estimation of the flow rate of leaking medium. Very important factor, which decides on usefulness of the method and detection algorithm, equivalent to sensibility for real damages, is also resistance to disturbances. That means non-generation of false alarms, i.e. caused by technological operation (starting/stopping compressor/ pump, change of supply tank, change of the type of transported medium, closing/opening valves, changing receiving tanks). The methods of leak detection may be divided into two general categories [6, 7]: Direct (external) - the detection is done from outside the pipe through application of specialized sensors or visual observation. Indirect (analytical, internal) - the detection is based on measurements and analysis of flow parameters (mainly pressure and fluid flow rate/velocity, sometimes temperature and density). Indirect methods may be divided into three categories: Based on detection of the acoustic wave caused by the leak. Based on mass balance concept, taking into consideration accumulation. Analytical - based on mathematical model and measurement of an object acquired from telemetry or SCADA system - these methods consist in pipeline modeling in real time, and continuous comparison of the model with the object.
(Fig. 4). One of them is a continuity circuit monitoring condition of the cable (checking possible physical damages). The second one - the alarm circuit - is a normally open circuit, shorting only in the case of a leak. There is a possibility to use various mechanisms. If transported medium is conductive, the short happens naturally, because the conductive liquid facilitates current flow between the cables. For hydrocarbons a special polymer is used as insulation. The polymer degrades in the presence of the hydrocarbons allowing the cables to touch. Measurement of voltage drops in the circuits enables to pinpoint the position of the leak. UZ continuity circuit
UZ
continuity circuit
alarm circuit
Alarm
no leak
Alarm
alarm circuit
with leak
short circuit induced by leak
Fig. 4. Cable leak sensor. A concentric cable with a hydrocarbons permeable insulation can also be used. If hydrocarbons penetrate a cable after leak, the impedance of the cable changes locally. An electromagnetic pulse is sent down to detect an impedance change. The change of impedance alters the echoes returning to the detection system and triggers an alarm. An optical wave-guide situated along a pipeline is in fact an intrinsic detector reacting for local temperature changes. As a result of adiabatic expansion (Joule-Thompson effect), drop in temperature appears in pipelines at the leak points. Optical wave-guide let also determine the temperature profile along the pipeline that makes possible to localize the position of the leak. This method cannot be used for liquids. Cable sensors are suitable rather for use in short plant piping and are not practical for long-range pipelines.
3. Direct methods Conventional direct methods are so-called network rounds combined with visual observation of the area and, if possible, sensing with portable detectors of transported medium. Better results may be obtained with help of trained dogs that can trace amounts of leaking medium. The use of the helicopters can give faster but less accurate information. Acoustic methods [8] are based on detection of noise generated by leak. They request the installation of acoustic sensors (specialized microphones with wide band of transmission) along the pipeline. The noise is then analyzed to identify what causes it - leak or maybe another phenomenon. During system activation the background noise is analyzed, which makes easier the identification of noise caused by leaks. Acoustic methods enable leaks 3 3 detection as small as 4 dm /h for liquids and 40 dm /h for gases, the sensors must however be spaced at short distance one from another - not greater that several hundred meters. Moreover, in cases of large turbulent flows, the leaks effect may be affected by background noise generated by the turbulence. Cable sensors are composed of two circuits [8]
4. Methods based on pressure wave detection This is one of the indirect methods. Fig. 5 presents how gas pressure and flow rate along the pipeline are changing after a leak has happened.
Fig. 5. Gas pressure and flow rate after the leak. At the initial instant (t1) at the leak point a rapid Articles 41
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N° 3
September 2007
pressure drop occurs, which propagate to both sides of the pipeline. The flow rate just upstream the leak point increases, downstream this point decreases. The pressure wave propagates with the sound speed. The gas mass flow rate increases upstream the leak point (instant t2), whereas downstream this point it increases on a while, and then goes back to the primary value. After several minutes the new state of the pipeline can be observed (instant t3). Upstream the leak point the mass flow rate increases by the value of flow rate of leaking gas, and downstream returns it to the primary value. Pressure gradient along the pipeline increases upstream the leak point, and downstream the pressure gradient returns to its initial value. Method based on pressure wave detection [9] and [10] consists in pressure measurements at the selected points distributed along the pipeline. If quick pressure change caused by acoustic (pressure) wave appears, one have to verify, whether similar change will take place at subsequent points, and whether it appears in the time resulting from the acoustic velocity and the distance between measurement points. Sufficient quick pressure transducers (precision is not most important) must be installed at several kilometers intervals (Fig. 6). To ensure precise synchronization of time measurement usually the satellite navigation system GPS is used. It generates an accurate time information based on atomic clocks installed at system satellites. This method is relatively fast (the leak detection and localization time is about few seconds.
method makes it possible to detect and localize the leak, but it is impossible to assess directly the leak intensity unless the amplitudes of pressure waves will undergo further analysis.
Fig. 7. Leak detection system based on pressure wave detection, acc. to [11].
5. A method based on mass balance concept This concept of leak detection is based on mass conservation principle. The total quantity (mass, flow rate, volume) of fluid entering and leaving a network must be balanced by the inventory variation (accumulation) inside the network. This is apparently the simplest and most natural method. However, changes of liquid quantity, which depends on pressure and temperature, should be carefully considered. These changes are especially crucial for gas pipelines. Very important are also uncertainties and drift of flow, temperature and pressure transducers. To apply this method is necessary to calculate stepby-step current value of certain variable t(t) at regular sample intervals. For gas net with single input and n outputs (gas receiving stations), the variable t(t) for standard conditions can be written as t(t)=DVn,in(t)Â S DVn,out(t)ÂDVn,a(t) i=1 n
Fig. 6. Leak localization method based on pressure wave detection. In the Fig. 6 the abscissa represents distance z from the beginning of pipeline 0; pressure sensors are installed at points zi. Axis of ordinates represents acoustic wave propagation time tl as a function of the distance from the leak point zl. Fig. 7 presents the scheme of such system according to [11]. Leak point is calculated as intersection point L of A-L and L-B lines. Point A denotes the time of pressure wave front transition to pipeline beginning and point B - transition time of pressure wave to pipeline end. The accuracy of leak localization is about (2 - 3)cT0 , where T0 is signal-sampling period. Therefore, signal-sampling period should be as short as possible, at least few times per second. Elimination of the stochastic noise becomes possible by the use of various types of analogue or digital filters, and especially correlation methods [9]. If the leak is not detected immediately (as a result of temporary system failure, the leak will remain undetected forever. This 42 Articles
(1)
This variable t(t) may be defined as a corrected flow unbalance term at time t. This is the difference between the gas volume entering the pipeline DVn,in(t) and the volume that flowed out from the pipeline
S DV i=1
n
n,out
(t)
(where n is number of outputs, that is gas stations located along the pipeline), reduced by the volume of gas accumulated in the pipeline Vn,a(t); index n denotes standard conditions. Expression Vn,a(t) represents changes of gas content in a pipeline. It depends on pressure, temperature and gas composition. Therefore, it can be written (without taking into account the pipe elastic strain) as follows DVn,a(t)=Vgrn pTn pnTZ (2)
and should be calculated step by step from the average temperature and pressure values for every pipe section
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N° 3
September 2007
(branch). Vg is the geometrical volume of the pipeline, p and T - absolute pressure and temperature in the pipeline; index n as formerly, denotes standard conditions. Parameter t(t) usually fluctuates around the non-zero mean value m, due to drift of measurement instruments and gas meters errors (which depend on the flow rate) or as a result of changes of conditions (e.g. temperature along a pipeline). These fluctuations can be characte2 rized by a s variance. Let us denote Dm the instantaneous deviations from average value. To generate a leak alarm the following cumulative sum is examined: a(t)=a(t-1)+ Dm Dm t(t)-m2 s 2 (3)
are installed. They enable to calculate pressure gradients. The coordinate of the leak point xu is then equal xu=L 1+ tanGs DtanGk -2
(4)
The flow rate of the leak can be calculated from the formula 1– q1=q0 1– 1– DGk Gs DGp Gs (5)
The alarm is generated when the sum a(t) exceeds the given value. This value should be determined experimentally and can be changed, i.e. during technological operations producing greater disturbances of flow parameters larger acceptable value a(t) is set. To eliminate false alarms caused by slow drift of measurement transducers, one modifies m value very slowly, with the use of measurement data from periods without leaks. For the reason of relative simplicity this method is often applied even for complicated pipe networks. However, direct localization of leak point is impossible; one only can localize a section (between two measurement points) in which a leak happened.
The situation changes in some extent in the case of gas pipelines, that is shown in Fig. 9 1 S p p
90°+ + u u
S
S 1
p
Fig. 9. Pressure drop along the gas pipeline before and after leak. Research carried out into gas pipelines [14] shows that at the beginning of a pipeline pressure drop are negligible (Fig. 9). There also is not a decrease of gas flow rate downstream the leak point because the pipeline ends with a gas regulating station. Such stations are very tolerant to changes of input pressure, so gas flow rate is only a function of gas demand. Pressure gradient downstream the leak point changes are not noticeable. In situation like that one can approximately calculate the leak point coordinate from the formula [14]: xu=Dp(L,t) cosGs sin(90°–Gs–DGp) sinDGp (6)
6. Gradient method Contemporary analytical methods are developed from gradient methods [12, 13, 7]. Fig. 8 shows the pressure drop along the liquid pipeline before and after leak.
1
S
in
p k 1 u
in
out
out
1
in
in
out
out
2
Pure gradient methods are effective only in steady flow conditions, without disturbances caused by change of operating conditions.
Fig. 8. Pressure drop along the liquid pipeline before and after leak. Liquid is transported from a tank T1 to tank T2. In non-leak state the pressure gradient along the pipeline is constant, equal to GS (blue line). When leak begins, after new conditions were established, one can observe following phenomena: The pressure drops both at the pipeline input and output, Increase of pressure gradient upstream the leak point, Decrease of pressure gradient downstream the leak point, Increase of flow rate upstream the leak point, Decrease of flow rate downstream the leak point. Usually along a pipeline every several kilometers are situated shut-off valves assemblies, where pressure transducers integrated in the telemetry or SCADA system
7. Analytical methods Basis of the analytical methods is modeling in the real time the phenomena inside a pipeline with the use of suitable mathematical models (modeling flow velocity as a function of pressure or pressure as a function of velocity), and comparison of the values calculated from the model with values acquired from the real pipeline. When the differences between the measured and calculated values (so called residua) are high, the alarm is generated and procedures of leak localization are activated. Both static and dynamic models are acceptable, however it must be proceeded by careful analyze. For gas the static model based on conservation of mass principle (which can be written as continuity equation) and known from hydraulics formulas for pipelines [15] has a following form: rwA=rqv=qm=const (7) Articles 43
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N° 3
September 2007
pD qm= 4
2
2 p2 1 – p2
2ZRT ln
p1 lL + p2 2Dp
(8)
and the sound speed of a liquid in a pipeline is given by formula commonly known [15] as Żukowski-Allevy formula: c= EC D EC r(x) 1 + b ER
where qm – stream of mass, kg/s p1 and p2 – absolute pressure at input and output of the gas pipeline, Pa D – inner diameter of the gas pipeline, m Z – compressibility factor, dimensionless R – gas constant, J/(kgK) T – absolute temperature, K L – length of modeled gas pipeline section, m l – linear losses factor. A dynamic model for gas pipeline can be expressed by a set of partial differential equations, which may be derived from conservation of mass and conservation of momentum principles [7]
(14)
where EC - the elasticity coefficient of a liquid (Pa), ER - the direct elasticity coefficient of a pipeline material (Young's modulus) (Pa), b - the pipeline wall thickness (m), D - the pipeline inner diameter (m), l(x) - the pipe friction factor, a - the angle of inclination of a pipeline segment. The mathematical model should however also reconstruct as accurately as possible the static and dynamic characteristics of all other elements of a pipeline system as well as interactions between these elements [17]. No matter if a model is static, dynamic, for liquid or gas, it should be completed with procedure of pipe friction factor l calculation. It is a function of roughness k and the Reynolds number Re. In cases of more complex pipeline system the Moody formula can be very useful for its simplicity. It has a form 1 ù é æ 103 ö 3 ú ê ÷ l = 0.0055 1 + 10 ç 10 k + ç ê Re ÷ è ø ú ú ê û ë
A ¶p ¶q + =0 c2 ¶t ¶z
(9) 2
1 ¶q ¶p lc qïqï g sina =p + 2 A ¶t ¶z 2DA2 p c
(10)
The resolution of dynamic model demands large processor capacity and the calculations are time consuming. It is possible to check if the dynamic model is really necessary by calculating the value of the term related to the change of momentum in time, therefore
(15)
1 ¶qm 1 ¶(rqv) ¶(rw) D(rw) (11) = » = A ¶t A ¶t ¶t Dt Comparing it with other terms of the equation. If its value is negligible, the simple static model (7,8) can be used. For liquid pipelines, because of larger transported masses, usually dynamic model [6] has to be applied. In order to describe the liquid pipeline dynamics for the modeling purposes the pipeline is arbitrarily divided into sections at xi points where measuring transmitters were installed on the real pipeline. It enables the direct comparison of the variables values obtained by simulation with the real ones, recorded in the real pipeline. Additionally, each pipeline section between xi and xi+1 points has to be divided into shorter, equal parts Dxj. Every segment fulfils a set of partial differential equations as a result of the law of mass and momentum conservation. In the case of a leak-proof pipeline (that is a pipeline for which neither mass decrement nor momentum decrement are observed) and taking into account that w(x,t) << c, where c is the sound speed, these equations, according to [6] can be written as ¶w(x,t) 1 ¶p(x,t) =0 + ¶x E ¶t
When large residua (differences between the model and the real object) occur, the localization procedure is activated. There are few methods. Usually a leak is introduced in the model and by successive approximations of its localization and intensity is matched to get compatibility of a model with a real pipeline again, or in another words minimize the residua. The use of gradient method is also possible for leak localization as far as the measurement system allows the pressure gradient determination.
8. Error sources for various methods of leak detection, and how to reduce them In case of analytical methods a fundamental source of errors are uncertainties of liquid properties measurement or calculation. True liquid properties (density, flow resistance) may differ from properties used to calculation. Uncertainties of density calculation or measurement and of pipe diameter measurements have a direct influence on model precision. Moreover, uncertainties of viscosity, diameter and pipeline roughness assessment affects l indirectly. Gas inclusion in liquid has a radical effect on an elasticity module of liquid - pipeline system, downgrading vitally model precision. Gas bubbles in circa 1% quantity of whole volume can increase even several hundreds times compressibility of such mixture. Liquid
(12)
(13) l(x)r(x) ¶p(x,t) ¶w(x,t) w(t)êw(t)ç + r(x) = -r(x)gsina 2d ¶x ¶t 44 Articles
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N° 3
September 2007
phase inclusions in gas are not so important, although they should not be disregarded in case of large quantity of liquid phase in gas. Next errors source is due to the fact that the temperature along the pipeline is not constant. Temperature of the ground changes, influencing the density and viscosity. The temperature depends on ambient temperature and other conditions such thermal conductivity of the ground (wet - dry), so from the weather. Computed density and pipe friction coefficient values for temperature measured at pipe ends may then differ from real values. Obvious source is uncertainty, especially instability (drift) of measurement transducers. Data entered into a model are always charged by some errors - so data correction is a necessity. The criterion of correction is the residua minimization. The data should be corrected as often as possible (always, when we know that there is no leak and in steady state). In each step one should introduce only little correction. Usually the recursive formulas are applied for this purpose. Although physically the corrected parameter is usually the pipe friction factor l, in fact all slowly varying factors (i.e. instrumentation drift) are corrected.
9. Summary Every detection method has its advantages and disadvantages. For example, sensitivity of analytical methods increases dramatically together with increase of the flow rate in a pipeline, as a consequence of square relationship between the pressure drop and the flow rate, which is shown in the Fig. 10.
Fig. 10. Effects caused by the same leak for various flow rates as pressure drop changes. At larger values of flow rate effects of leak (pressure change Dp2), caused by an leak of value Dq, are distinct, while at small flow rate this effect (Dp1), caused by identical leak, is significantly smaller and the threshold of detection ability, and detection errors could achieve dozens % [14]. For low flow rates probably the method based on mass balance concept would be better. However, with the use of this method the precise localization of leak is difficult. Methods based on pressure (acoustic) wave detection demands special measurement and instrumentation solutions (hydrophones) destined only for this purpose, while analytical methods can use measurement instruments already installed in the pipeline for operating purposes. The advantage of pressure wave method (in comparison with analytical or mass balance method) is however little
requirements for precision of measurement equipment. Moreover, the particular information about pipeline parameters (accurate diameters, roughness, etc. (that for old pipes are hard to obtain) is not necessary. On the other side the pressure wave system needs high frequency of sampling - the precision of the method is then greater. As the acoustic wave methods have greater sensibility to disturbances they have been successfully implemented in long transmission pipelines without branches along the pipeline, pumps or compressors stations [11]. For more complex pipelines systems with branches, pumps or compressors the method at the present level of development and state of the art does not give satisfactory effects because of false alarms generation. However, consecutive researches are carried out in this field with the use of correlation techniques. Acoustic methods have another serious defect: if one overlooks the beginning of damage, one will never find it out by this method. For the reasons described above the most effective systems of leak detection and localization can be built through joining various described above concepts, operating in parallel. At present there are a lot of commercial systems of leak detection and localization. Before one takes a selection decision it is worth to formulate a set of control questions to the system supplier, which depends on configuration and operating conditions of a given pipeline. For example, the most important technical questions are: What leak intensity can be detected using the given method? How much time passes between the leak and detection of it? Is the system able to localize leak point, and if yes with what precision? Is the system able to determine leak intensity, with what precision? Is the system able to detect earlier existing leaks or only the new ones? Does the system function in pipeline section between two closed valves or in case of lack of flow in the pipeline? Does the system function for more complicated networks (branched configurations, with many of medium input and output points), or for simple pipe sections only? Does the system function during transients technological operations i.e. turning on/off compressor/pump, opening/closing the valves, switching supply/receiving tanks from one to another, or only in steady state? How often false alarms are generated by the system (especially in described over conditions)? Could the system be installed retrofitting) on already existing pipeline, and which additional equipment is necessary? Next group of questions is connected with economical point of view: How high is the cost of the complete system installation (including necessary additional instrumentation)? Operating costs, calibration and tuning frequency, etc. Articles 45
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N° 3
September 2007
Because of every system has some disadvantages, it is worthwhile to install two systems working in parallel, complementing one another. For example, analytical method for large flows could be supplemented by mass balance method in cases of small flows. It applies particularly pipelines for transmission of expensive or dangerous (because of its explosive properties, or because of possibility environmental contamination) media. Since 2005 authors of this paper carry out research on relatively difficult cases (from the point of view of leak detection system). Examples for liquid pipelines are transient states of any kind, transport of two or more media of different properties through the same pipeline one bath after another, or leak detection during pipeline standstill. Examples for gas pipelines are pipeline systems with complicated, branched structure and com pressor stations. Results of these researches will be published soon. ACKNOWLEDGEMENTS The paper is a result of research, financed within the Multi-Year Programme PW-004 entitled “Development of innovativeness systems of manufacturing and maintenance 2004-2008”.
[10]
[11]
[12]
[13]
[14]
[15]
AUTHOR Mateusz Turkowski* - Professor at Institute of Metrology and Measuring Systems, Warsaw University of Technology, ul. św. A. Boboli 8, 02525 Warszawa, Poland, email: m.turkowski@mchtr.pw.edu.pl. Andrzej Bratek and Marcin Słowikowski - PIAP Industrial Research Institute for Automation & Measurements, Warsaw, Poland. * corresponding author
[16] [17]
einen Prozesrechnes”, Regelungstechnik, 1977, nr 3 (25), pp. 69-74. Sobczak R., “Lokalizacja nieszczelności w rurociągach metodą śledzenia czół fal ciśnienia” [Localization of the detections in the pipelines through front faces method] Przemysł Chemiczny, no. 6/04, edition JCR. Technology and Qualifications Acoustic Systems Incorporated Whithorn Dr. Houston, Texas, USA www.wavealert.com. R. Sobczak, “Detekcja wycieków z rurociągów magistralnych cieczy” [Detection of the leakages from fluid arterial pipelines], Nafta. Gaz, 2001, no. 2, pp. 97-107 R. Sobczak, “Gradientowa metoda lokalizacji nieszczelności w rurociągach magistralnych” [The gradient method of leakages' detection for arterial pipelines], Przemysł Chemiczny, nr 10/02 edycja JCR. Turkowski M., “Metody detekcji i lokalizacji nieszczelności rurociągów” [Detection and localization of pipelines' leakages methods], Proceedings of the conference Technika opomiarowania gazu dziś i jutro TOp-Gaz, Rogów, www.common.pl/aktu_rogow.html, 19-21.09. 2005, pp. 179-194. M. Mitosek M., Mechanika płynów w inżynierii środowiska [Fluid mechanics in environmental engineering], Oficyna Wydawnicza PW, Warszawa, 1997. L. Bilman, R. Isermann: „Leak detection methods for pipelines”, Automatica, vol.23, no. 3, s. 381-385, 1987. Sobczak R., Turkowski M., Bratek A., Słowikowski M., Report of research project nr 3 T09D 033 27 entitled Opracowanie systemu detekcji nieszczelności w rurociągach magistralnych ograniczającego zagrożenie środowiska wyciekiem paliw ciekłych i gazowych [Developing leakages' detection system restricting natural environment for arterial pipelines], Warszawa, 2006.
References [1] Michałowski Witold S., Trzop S., Rurociągi dalekiego zasięgu [Long-range pipelines], edition V, Warszawa: Odysseum Foundation, 2006. Peters J., “Guidance on erosion in pipework”, Flow Tidings, issue 38, summer 2003. Gros M., Leak and shock detection. Cybernetics, Marseilles. Presentation at the expert meeting, Brussels, Jan. 2006. Web site of company PERN “Przyjaźń” SA www.pern.com.pl. Web site of Gas Transmission Operator “Gaz-System” SA www.gaz-system.com.pl . Bilman L., Isermann R.: „Leak detection methods for pipelines”, Automatica, vol.23, 1987, no. 3, pp. 381385. Kowalczuk Z., Gunawickrama K., “Detekcja i lokalizacja wycieków w rurociągach przemysłowych” [Detection and localization of leakages in industrial pipelines], Chapter 21 of the monograph edited by J. Korbicz and J.M. Kościelny, Warszawa: WNT Publishing House, 2002. Glen N.F., A review of pipeline integrity systems, Report No 2005/257, National Engineering Laboratory, East Kilbridge, 2005. Siebert H., Isermann R.: “Leckerkennung und Lokalisierung bei Pipelines durch on- line Korrelation mit
[2] [3]
[4] [5] [6]
[7]
[8]
[9]
46
Articles
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N° 3
September 2007
PRESENTING A TECHNIQUE FOR REGISTERING IMAGES AND RANGE DATA USING A TOPOLOGICAL REPRESENTATION OF A PATH WITHIN AN ENVIRONMENT Received 15th June; accepted 20th July.
Filipe Ferreira, Luis Davim, Rui Rocha, Jorge Dias, VĂtor Santos
Abstract: This article presents a novel method to utilize topological representation of a path that is created from sequences of images from digital cameras and sensor data from range sensors. Leading the robot around the environment during a familiarisation phase creates a topological representation of the environment. While moving down the same path, the robot is able to localise itself within the topological representation that has been previously created. The principal contribution to the state of the art is that, by using a topological representation of the environment, individual 3D data sets acquired from a set of range sensors need not be registered in a single, [Global] Coordinate Reference System. Instead, 3D point clouds for small sections of the environment are indexed to a sequence of multi-sensor views, of images and range data. Such a registration procedure can be useful in the construction of 3D representations of large environments and in the detection of changes that might occur within these environments. Keywords: sensor feature integration, binary data, Bernoulli mixture model, dimensionality reduction, robot localisation, change detection
1. Introduction In previous work we have been primarily interested in developing a method for place recognition by attempting to recognise previously identified places in the environment using features from multiple sensors. This document applies the same technique for place recognition to solve the problem of registering pairs of 3D point-cloudsets collected by a robot as it moves in large indoor surroundings. This method of registration of point clouds is currently being applied to the creation of 3D environment-visualization tools and to detect changes that might have taken place in the environment. 1.1 Geometrical and Topological Maps To be useful for robot navigation, maps must contain information about the topology of the environment. The maps represent the layout of the environment and information of the connectivity of the environment [1], [2], [3]. Some maps can include metric information. Gridbased (geometrical) maps are a popular means of representing environments, not least, because, they easily allow the incorporation of uncertainty in the position of objects in the map (and naturally of the robot) [4], [5]. In other cases other representations of the environment, such as vector-based (geometrical) maps using lines or planes have been used, one of the first reported examples
being the work of Drumheller [6]. Topological maps and geometric maps have been compared in a number of works. These comparisons range from the very definition of when does a topological map begin to be a geometric one, to the relative advantages of each type, vis-Ă -vis scalability, computing costs and other relevant criteria. Lately, there has been debate on the type of map that is [generally] more effective for achieving localization and imparting autonomy to mobile robots placed within large environments. There has been some support for an approach that incorporates (memory consuming) metric information at certain levels and retains only, [topological] layout and connectivity information at others. The Spatial Semantic Hierarchy or the SSH by Ben Kuipers [7] is one such approach that attempts to provide a complete environment representation approach. The framework, described as a "model of knowledge of large-scale space consisting of multiple interacting representations", is presented as hierarchies, each of which perform some abstraction of the perception and interaction of the robot with the environment. More recent methods maintain fewer hierarchical levels, with [unconnected] grid-based geometrical maps at the lower level and with a high-level topological map containing the connectivity information of the whole environment [8]. In such approaches, the two maps are closely linked since the topological map is actually constructed from the grid-based map. In previous work performed by the authors of this paper was found that the knowledge of the expected sequence of observations to be found along a particular path in the environment is usually sufficient for localization (within this path). Thus, in order to get from a point A to another point B, in an environment, the robot only needs to be told of the sequence of observations it will expect to sense as it proceeds from A to B, Fig. 1 top-right. This is in contrast to the usual situation wherein the robot is provided with a complete map of the environment and localization is defined as the procedure that maximises the consistency of the observations with the [entire] map at any instant. As stated earlier, the aim of the current work is to register 3D point clouds by first registering the robot within the topological representation created during the familiarisation run Fig. 2, bottom-left. The environment is represented in the form of a line graph where each node of the graph represents a place, at which the environment was sampled, images and scans collected. The sequence of sensor views, thus collected and known as the Reference Sequence, represents a topological path Articles 47
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N째 3
September 2007
through the environment. During subsequent runs through the environment, the position of the robot is recovered within this Reference Sequence. Once this position within the Reference Sequence is known, the position in the environment that corresponds to the 3D point cloud can be obtained Fig. 2, bottom-right, and the point cloud data registered.
Fig. 2. A depiction of the topological path representation problem, at top. The robot is led through the environment on the Familiarisation run, bottom-left. The 3D point clouds must be registered in the environment, bottom-right. (a) A Topological "path" represented as a Reference Sequence. The experiments in this article were performed on a Segway Robotic Mobility Platform, the RMP 200. Two SICK Laser Range Finder (LRF) and two Firewire cameras have been added to the platform to gather data from the environment. One LRF points upwards, taking vertical sections of the environment as the robot moves forward and a forward-facing LRF that provides a horizontal section through the part of environment in front of the robot. The forward-looking camera, Camera 1, looks in the direction of robot motion while the lateral camera, Camera 2 is mounted at a sufficient height to view posters and other texture appearing on the walls of the building. The cameras are capable of taking VGA-sized images and the SICK laser range finder provides a set of 361 range measurements taken through a 180-degree interval. Features from three sensors, i.e. the forwardfacing laser range finder and from the camera images were used for place recognition. 1.2 Features from Ranged Data Range based methods have been frequently used to index places in indoor environments, first with ultrasound sensors and later with laser range scanners. In an attempt to obtain more reliable features in the environment, many range-sensor based methods extract lines and other primitive features from the laser scan. Cox [9], attempts to match points extracted in the laser range scan with the lines in the map. Drumheller [6] describes a multistage algorithm in which line segments are constructed from data from ultra-sound sensors. Using interpretation trees, the extracted lines are then matched with a given map that is composed of line segments. The extraction of lines from the laser scan
(b) The Topological map of the environment.
(c) The Geometrical map of the environment. Fig. 1. A hierarchy of Geometric, Topological and Reference Sequence Maps that can be used for mobile robot navigation. The grid map [at the bottom] includes the information that can be captured by the sensors and the Reference Sequence contains only a path within the topological map. This comparison of hierarchies has been based on Sebastian Thrun's Hybrid maps [8] and parts b) and c) of the figure are taken from [8], Fig 14. 48 Articles
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N° 3
September 2007
continues to be a popular approach in the robust segmentation of laser scan data, see [10] and [11] for recent reviews of popular line-extraction algorithms. Representing places only in terms of lines (and corners) provides a limited amount of information. Many places in the environment are found to have similar representations and these methods do not scale up easily to larger environments. Other publications have described the expansion of the set of features extracted from range scans to include, for example, trees, kerbs and more. Manandhar and Shibasaki in [12] extract roads, buildings, tunnels and other outdoor features by modelling 3D range data. In indoor environments too, composite landmarks including lines and other simpler features have been used, [13]. A different approach, which eschews segmentation into simple primitive features, favours the description of a section or sections of the 2D Laser scan in some reduced variable space. This is the approach used, for example, in [14] where each feature extracted from the laser range scan is given a symbol and each scan is described in the form of a string for example mMmMmMmMmDCm. The string alphabet, in this case (M)axima, (D)iscontinuity, (m)inima, (c)onnection), depends on the features extracted from the laser scan. Other methods use 'sections' of the laser range scan so as to minimise the effect that changes in one part of the scan will have representation of the place. In our work, we have used multiple types of features from the laser range scan, namely 1) wall-like (line) features, 2) scan region properties and 3) scan contour properties in the form of a vector that characterises 2D discontinuities in the plane of the scan using Hu moments [15]. 1.3 Local Features from Digital Images The use of cameras on mobile robots has become widespread over the last few years. Cameras can be viewed as high bandwidth sensors and images can have large redundancy. It is â&#x20AC;&#x153;expensiveâ&#x20AC;?, in terms of memory and computational costs, to store every raw image that is associated with a place or with an object. Also, certain regions in an image are known to be [more] stable with viewpoint and lighting changes. For this reason, local image features have been used to perform scene and object recognition. The use of local image descriptors based on these stable regions is characterized by two steps: 1) the selection of points of interest and 2) their characterisation. The selection must be repeatable (even with changes in the conditions in which the images are taken) and the characterisation must employ properties that must, again, be tolerant to changes in the viewpoints, lighting and other conditions. Local-image features based on local image gradients are an important class of vision features. Baker, in [16], attempts to create a generalised descriptor for local image features and the introduction to his thesis provides a perspective on the development of gradient based methods. The stability and repeatability of points extracted at local Maxima (or Minima) in gradient images that have been repeatedly smoothed using operators, has been known for some time [17] [18], and research in the field
finally culminated in the Scale-Space theory proposed by Lindeberg [19]. Building vector descriptors to represent each such feature ensures the uniqueness of the extracted features. In work that combined the lessons of Scale-Space with the reliable characterisation of features, Lowe [20] describes the use of gradient histograms taken at various points close to some point of interest. These features were called Scale Invariant Feature Transforms, SIFT. The work described in this thesis is based on local image features that are based on the SIFT features and to which some modifications that allow us to create the descriptors faster have been made. The procedure for creation of the feature database has been modified in order to simplify the creation of features for image sequences. Since their introduction, SIFT features have been widely applied, among others, to object recognition [21], [22], in the panoramic assembly of images [23] and in image retrieval [24]. Various researchers have used this descriptor in new applications and modifications on the original procedure have appeared (see Weighted Gradient Orientation Histograms [25], Modified SIFT [1], PCA SIFT and Global SIFT). In this work we have utilised SIFT features to characterise images obtained from the two Firewire cameras. Fast matching of many hundred features is achieved through the use of KDTrees, which can be constructed quickly (see [26] for an fast open-source implementation).
2. Representing a topological path using a Reference Sequence The problem of localization within a reference sequence is akin, in some ways, to the problem of supervised sequential learning [27]. In the same work, Dietrich identifies three important issues that must be addressed in the case of sequential machine learning, namely specification of a loss function, feature selection and computational efficiency. The Reference Sequence contains the sensor information that is gathered as the robot moves on its familiarisation run through the environment. In order to improve the results of the localisation algorithm, distinctive places could be maintained in the original Reference Sequence such that they improve the results of localisation and represent the environment in the form of a compact Reference Sequence. The remainder of this section will be devoted to attempting to classify places and then define what makes places "distinct" and how a compact Reference Sequence with distinct places might be created. Further details of the method can be found in [28]. View Classification using a Bernoulli Mixture Model Having to reduce the dimension of data from a sensor or multiple sensors in order to recover the original class is a common problem in mobile robot localisation. To solve this problem, methods seek to apply a model to explain the data, allowing the identification of data that is significant (and which must be used during classification). Articles 49
2.1
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N° 3
September 2007
On the other hand, a data-driven approach will attempt to extract these correlations. Methods that reduce the dimension of features with continuous values are common in many perception fields including face recognition, speech recognition etc. Among these approaches, Mixture Models are a common solution to modelling data that is thought to a follow non-parametric distribution. Mixture models assume that there exists a finite number of distributions which, when mixed together in a particular proportion, result in a distribution that best describes the data. Sajama and Orlitsky in [29] demonstrate the use Mixture models composed of Gaussian, Bernoulli and exponential distributions as a solution to the classification problem. To a greater or lesser extent these clustering or classification methods seek to identify features that are more correlated with members of their own group than with members from another group. Mclachlan and Peel [30] provide a good reference to the general topic of Finite Mixture Models. Articles by Kaban [31] and Wang [32] provide a healthily different viewpoint and go some way to demonstrated the usefulness of binary features. In [32] the context in which a word is used in a sentence is converted into multiple binary features. Similarly [33] and [34] seek to model some training data as a sample of sets of binary features taken from a population of binary features, each distributed according to a mixture of Bernoulli distributions. The application of binary features to the classification of images and text has motivated us to apply the approach to classifying other types of features, such as barcode/data-matrix features. While classification of a few binary properties might be easy, using many, correlated features is very difficult. In applications such as mobile robot localisation, image retrieval and robot localisation methods typically make use of a large number of features. In the application to mobile robot localization, we have employed up to 16000 binary features to allow the robot to recover its position within the environment. The binary features from each of the objects in the pilot set are represented within a Feature Incidence Matrix (FIM), V. Each row i, of the FIM corresponds to a feature Yi and each column j, to an object, Vj, from the reference sequence (each entry in the FIM might be represented as Yi,j where the first subscript indicates the feature and the second subscript, the object). Yi,j takes value 1 if feature Yi appears (is present in the code) in object Yj,0 otherwise. Y1,1 Y1,2 ¼ Y1,K Y2,1 Y2,2 ¼ Y2,K (1)
Inferences made under this assumption would be biased toward certain objects in the FIM and in practice, some of the features are highly correlated while others are less. In such circumstances we need to employ methods that deal with the correlation between the features. As stated earlier, this thesis describes our use of Mixtures of Bernoulli Distributions to model the binary FIM. Mixture models assume that there exist a finite number of parametric distributions, which, when mixed together in a particular proportion, result in a distribution that best describes the data we wish to characterize. In this case we model any code that is observed Vobs as N a vector of binary features {0,1} , which is obtained from a particular mixture of Bernoulli distributions, as in (2). If V represents the complete set of objects as collected during the classification of the pilot set, Vk is a single object with an index k within the pilot set, V, described by multiple features, Vobs is a single object that must be classified and F is a single [named] Feature. Z is a Hidden or incomplete data in a Mixture Model, ai represents the mixture component coefficient or component Prior probability and Qi is a single component of the mixture model with the named features Õ K represents the product operator while Sk=1, represents the sum operator with the index k varying from 1 to K. P(Vobs½Q)=åai P(Vobs½Qi) i=1 M
(2)
where Q denotes the parameters of the distribution of the objects that compose our Mixture Model. These parameters include the M component vectors, the Qis, and the proportions in which these are mixed, the ais. Each ai represents the prior probabilities of the component i in the mixture model, subject to the constraint Siai=1. The term P(Vobs½Qi), can be determined using (3) where each Qi is a multivariate vector of Bernoulli probabilities each of whose N components indicate the probability of success for a particular feature. P(Vobs½Qi)= ÕN (1- jQi) j=1Qi j
Vobs
(1- jVobs)
(3)
To obtain the mixture parameters that explain a particular FIM V, consisting of K observations it is assumed that the objects are independent and the likelihood of the mixture satisfying the FIM is expressed thus (4). P(V½Q)= Õ P(VK½Q)=L(Q½V) i=1 K
(4)
V=
YN,1 YN,2 ¼ YN,K A simple metric for verifying the similarity between a new object to be classified, and another in the pilot set, could be the number of binary features in each object that are unchanged. This metric would assume that the individual features on each object are independent. Unfortunately, given that features arise in groups and persist/ disappear as a result of the behaviour of the object and its exposure to the environment, the assumption of independence between the features does not refiect reality. 50 Articles
The optimisation task to find the mixture that best explains this V can be expressed as in (5), i.e. find the value of Q that best satisfies the distribution of features in V. Q*=argmaxQL(Q½V) (5)
¼
¼
¼
¼
The preferred method of solving the Mixture Model problem is the Expectation Maximisation algorithm. McLachlan ([30], page 19) states "…it will be seen that conceptualisation of the mixture model …(hidden data + component distributions)… is most useful in that it allows the Maximum likelihood estimation of the mixture distribution to be computed via a straightforward appli-
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N° 3
September 2007
cation of the EM algorithm". The EM algorithm proceeds in two stages: the Expectation stage attempts to reach the best value for the missing data Z, by keeping the parameters of the Mixture model constant (6), while the subsequent Maximization stage attempts to optimise the components and mixing parameters themselves by using the values of the "missing data" obtained in the expectation step just performed (7), (8). The method then alternates between the two steps until some termination criteria are satisfied. zki = ai = Qi = ai P(VkïQi) S aj Pj(VkïQj) M j=1 K
(6)
Sk=1 zki K Sk=1 zki Vk K
(7)
(8)
Sk=1 zki K
and limited computing power or communication bandwidth. Compact representations of the Reference Sequence are also desired when long paths through the environment must be traversed, resulting in faster localisation. As shown in Fig. 3, the original, sampled Reference Sequence might be broken down into a number of smaller sequences resulting in a compact representation of the topological path. A compacting procedure for the Reference Sequence should keep two types of Views: 1 Essential Views defined as those Views in the Reference Sequence at which the robot motion behaviour is altered. 2 Non-Essential Views, which, as the term might suggest are those views, that, while not essential, might improve the topological path representation. Non-Essential views should increase the probability of identifying Essential Views and should also increase the probability of correctly detecting that the robot is lost.
This termination criterion is usually a lack of change in the mean error when the Mixture Parameters are applied to the original data. In the case of such applications, where the parameters of the Mixture models are required for the purpose of classification, the process is usually stopped quite early, when the reduction in the Mean Error is not significant. Mixture models used for classification make use of both, the Mixture parameters and the posterior probabilities over the components, the Z are used to evaluate the likelihood in the space of the objects in the reference sequence as in (9) where P(Vk) represent the prior probabilities on each index k. P(kïVobs)= Sj=1 P(Vk) zki aj P(VobsïQj) M
(9)
Sk=1 Sj=1 P(Vk) zkj aj P(VobsïQj) K M
Fig. 3. The original Reference Sequence is now decomposed into a series of sub sequences, which together make up the Compact Reference Sequence. While the Essential views depend on the behaviours that are adopted along the path, and on any markers laid down by the user, the Non-Essential views refiect the information content of that part of the environment. The problem of selecting the Non-Essential Views entails selecting the most successful sub-set of Views from a single sequence. Such a set of parameters will depend on the length chosen between the Views of the HMM and (a related quantity) the number of Non-Essential Views chosen between a pair of Essential Views. Since exact methods could not be found to solve the problems of selecting the distinct views Non-Essential Views were researched within this work. These approximate methods might use cost functions that seek to maximise one or more of the above-mentioned factors. For example, a strategy for addressing the first and second factors mentioned above would aim to select the earlier single view that has the greatest [total] chance of getting selected. The selection of the "Next" non-essential View is not completely straight forward since the Prior probability P(k) in (9) implicitly infiuences the Place recognition using the Bernoulli Mixture Model. Since the information in the compact Reference Articles 51
The Maximum Likelihood Estimation approach is used to obtain the matching object, the index k*,in V that best describes the object to be matched, Vobs. P(k=k*ïVobs)=max P(kïVobs) k
(10)
Identifying distinct views In order to match multiple, consecutive, current views by using the context in which these are observed, the movement of the robot along the sampled views can be modelled as a Finite State Machine. The order in which the views were collected in the reference sequence, should, if all goes well in the subsequent runs, be repeated as the robot retraces its path on a latter occasion. It is conceivable that in certain cases, by identifying distinctive places, better results that were obtained, using the original Reference Sequence. These improvements could lead to better localisation from more suitable prior and place-transition probabilities. Alternatively, by identifying these distinct places a more compact topological representation of the path that does not necessarily provide inferior localization capabilities could be created. Shorter, more compact Reference Sequences are desirable in applications involving communication between robots/persons having different capabilities
2.2
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N° 3
September 2007
Sequence is not uniformly distributed in time or distance, a conventional Markov chain (with regular observations) cannot be used to model the hidden states of the Hidden Markov Model. Instead, the distance between the places must be modelled by modelling the duration (or time) between consecutive places. The principal feature of the HMMs that takes into account the duration of the transition between states is that there some mechanism is found to model the "selftransition" probability of a state (place) to itself. Such variants of HMMs are becoming quite popular in the modelling of phoneme information in speech recognition and synthesis [35]. The conventional way of modelling duration is by building higher order Bayesian networks or by breaking the Markov assumption of HMMs to give Hidden Semi-Markov models. A simpler method that has been suggested to account for the duration for which the system remains in the same state is the Post-Processing model by [36]. In this approach, the original Viterbi algorithm is maintained but at every step the probability is updated to refiect the probability that the transition actually respects or is in concordance with the expected transition duration, according to (11), where represents the estimate of the pro-
In the second step these local metric maps are temporally ordered using the information contained in the Topological map. The robot has been previously led through the environment during the environment familiarisation phase during which the Reference Sequence and the data from the sensors was collected. During subsequent runs, the sensor Views are compared to the Views of the Reference Sequence to estimate the location of the robot within the Reference Sequence according to the method summarised in Fig. 4, bottom-right.
(a) LRF Scans registered with inertial Data to give local metric Maps
bability of staying in state j for a time lj given an expected duration of T and a is a constant. N lj log¦=log¦+aå log pj( ) T j=1
(11) (b) Registration of Local Maps with the Reference Sequence Fig. 4. A loose reconstruction of the environment is created from the 2D scans extracted by the vertically facing, rear LRF using a 2-step process. In the first step, small 3D local metric maps are reconstructed from the 2D LRF scans using data from the inertial sensor that is attached to the sensor set-up. In the second step, local 3D metric maps from the first step are registered within the Reference Sequence, the topological representation of the path from the Environment Familiarisation run. A schematic of the method used to perform place recognition within the Reference Sequence is shown. The results of localisation along a run through a long corridor of around 120 meters length are shown in Fig. 5. The place in the Reference Sequence are shown in the Y coordinate and the places at which localisation was attempted in the second run is plotted along X coordinate. The scatter plot shows the place in the Reference Sequence against which the observed views were registered. Typical 3D reconstructions for Point cloud data along the Reference Sequence and the Observation Sequence are indexed to this scatter plot. The Fig. 6 shows two examples of registered sections of the environment with the screen shots at right indicating the reconstructions for the Reference Sequence and those at left representing the reconstructions for the subsequent run.
This approach has been employed in the current work, since it allows large variations in the distances between nodes without adding to the computational cost of the Viterbi Algorithm.
3. Feature Registration and Localisation of the Robot Building on and extending previously existing modules within the Carnegie Mellon Robotic Toolkit, CARMEN, have developed support for the Segway RMP 200 robot. Sensor data is recorded in standardised file formats within CARMEN. The sensor data from the log files is indexed to the XML file representing the Reference Sequence through the "distance-covered" variable. By recovering the correct View within the Reference Sequence, the sensor data corresponding to the position of the robot in the environment is obtained. The range data from the vertically facing LRF is extracted corresponding to the place in the Reference Sequence and consecutive scans registered using the accompanying inertial information. Reconstruction of the 3D environment from data collected from the vertically facing LRF is performed in a twostep process. In the first step, consecutive individual planar scans are registered with each other by applying the transformation that the LRF suffered between consecutive scans. The transformations required to perform this registration are obtained from an inertial sensor attached to the sensor platform. The registered scans create small sections of reconstructed 3D data each within its own local coordinate system, and known as local metric maps.
52
Articles
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N째 3
September 2007
Fig. 5. The scatter graph plots the views in the observation sequence that was matched against views in the Reference Sequence. Sections of Rendered Point Cloud data for the same matched places in the topological path are also shown.
Fig. 6. The screen shots at left depict local metric maps rendered at places that were matched against equivalent places in the Reference Sequence shown on the right. A visual comparison of the screen shots reveals the occur-rence of real and spurious changes in the environment. Articles 53
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N° 3
September 2007
4. Conclusions The localization of the robot in the Reference Sequence by reducing the dimensionality of the sensor data has been demonstrated in an earlier publication. Further improvements in place recognition have been made in order to create compact sequences. Localisation within the topological path represented in the Reference Sequence is found to be a promising way to initiate procedures to detect changes in the environment and to represent sections of the 3D point clouds in this representation of the topological path. We are currently developing local image features that can be extracted from the 3D point cloud data that can also be added to the set of features to aid the localisation of the robot along the topological path. Work has also begun to develop algorithms that might autonomously flag the presence of changes in the environment through a comparison of the a pair of point cloud data sets taken of the same place within the environment. The algorithms will attempt to fuse local metric maps containing 3D data and flag the differences that impede correct fusion. Such algorithms will be applied to detect the presence of people and the occurrence of environment-change events that may pose risks to users of the infrastructure. ACKNOWLEDGEMENTS The first author is a recipient of a research scholarship from the Faculdade de Ciências e Tecnologia, University of Coimbra (FCTUC) under the BACS (FP6-IST-27140) project. Part of this work was performed within the PhD program of the first author, funded by the University of Aveiro, Portugal. The second author is a recipient of a research scholarship from the FCTUC under the IRPS (FP6-IST-45048) project. The authors have utilised the CMU-promoted CARMEN toolkit and wish to thank the development community of this project. Special thanks go out to the developers of the 3D visualisation tool [37]. [4]
[5]
[6]
[7]
[8]
[9]
[10]
[11] [12]
[13]
[14]
AUTHOR Filipe Ferreira*, Jorge Dias, Luis Davim, Rui Rocha Institute of Systems and Robotics, University of Coimbra, 3030-290 Portugal; e-mail: cfferreira@isr.uc.pt. Vítor Santos - Department of Mechanical Engineering, University of Aviero 3810-193 Portugal. * corresponding author
[15] [16] [17] [18]
References [1] H. Andreasson and T. Duckett, “Topological Localization For Mobile Robots Using Omni-Directional Vision And Local Features”. In: 5th IFAC/EURON Symposium on Intelligent Autonomous Vehicles, Instituto Superior Tenico, Lisboa, Portugal, July 2004. IFAC/EURON. N. Vlassis, Y. Motomura, and B. Kröse, “Supervised linear feature extraction for mobile robot localization”. In: Proc. IEEE Int. Conf. on Robotics and Automation, 2000, pages 2979-2984. A. Tapus, N. Tomatis, and R. Siegwart, “Topological Global Localization and Mapping with Fingerprints and Uncertainty”. In: 9th International Symposium on [19] [20]
[2]
[21]
[3]
[22]
Experimental Robotics, ISER, 2004. A. Elfes, “Sonar-based real-world mapping and navigation”, IEEE Journal of Robotics and Automation, vol. 3, issue. 3, June 1987, pp. 249-265. R. Rocha, Building Volumetric Maps with Cooperative Mobile Robots and Useful Information Sharing. PhD thesis, University of Porto, Department of Engineering, 2005. M. Drumheller, “Mobile robot localisation using sonar”, IEEE Transactions On Pattern Analysis And Machine Intelligence, vol. 9, issue 2, March 1987, 325332. B. J. Kuipers, “The Spatial Semantic Hierarchy”, Artificial Intelligence, no. 119, 2000, pp. 191233. Available at: http://www.cs.utexas.edu/users/qr/papers/ Kuipers-aij-00.html. S. Thrun, “Learning metric-topological maps for indoor mobile robot localization”, Artificial Intelligence, issue 99(1), 1998, pp. 21-71. I.J Cox. Blanche-an experiment in guidance and navigation of an autonomous robot vehicle. IEEE Transactions on Robotics and Automation, vol. 7, issue 2, April 1991, 193-204. V. Nguyen, A. Martinelli, N. Tomatis, and R. Siegwart, “A comparison of line extraction algorithms using 2d laser rangefinder for indoor mobile robotics”. In: International Conference on Intelligent Robots and Systems, 2005. D. Sack and W. Burgard, “A comparison of methods for line extraction from range data”. In: IAV 2004, 2004. D. Manandhar and R. Shibasaki, “Feature extraction from range data”. In: ACRS 2001 - 22nd Asian Conference on Remote Sensing, Singapore, November 2001, volume 2, pages 1113-1118. Z. Xiang, V.M. F. Santos, and J. Liu, “Locating And Crossing Doors And Narrow Passages For A Mobile Robot”. In: First Int. Conf. on Informatics in Control, Automation and Robotics (ICINCO), 2004. J, Fellers, S, Lee, N, M. Amato, “Localization based on Visibility Sectors using Range Sensors”. In: Procceedings of the IEEE Int. Conference Robot. Autom. (ICRA), January 2000, pages 3505-3511. R. C. Gonzalez and R. E. Woods, Digital Image Processing, Addison-Wesley Pub. Co, 2002. S. Baker, Design and Evaluation of Feature Detectors. PhD thesis, Columbia University, 1998. J. J. Koenderink, “The Structure Of Images”, Biological Cybernetics, vol. 50, no. 5, 1984, pp. 363-370. A.P. Witkin, “Scale-space filtering”. In: Proc. 8th Joint conference on Artificial Intelligence, Karlsruhe, W. Germany, 1983, pages 1019-1023. T. Lindeberg, Scale-Space Theory in Computer Vision, Kluwer Academic Press, 1994. D. G. Lowe, “Object Recognition from Local ScaleInvariant Features”. In: Proc. of the International Conference on Computer Vision, Corfu, 1999, pages 11501157. A. Pope and D. G. Lowe, “Probabilistic Models of Appearance for 3-D Object Recognition”, International Journal of Computer Vision, vol. 40, no. 2, 2000, pp.149167. D. G. Lowe, “Local Feature View Clustering for 3D Object Recognition”. In: Proceeding of the IEEE Conference on
54
Articles
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N° 3
September 2007
[23]
[24]
[25]
[26]
[27]
[28]
[29]
[30] [31]
[32]
[33]
[34]
[35]
[36]
[37]
Computer Vision and Pattern Recognition, Kauai, Hawaii, 2001, pages 682-688. M. Brown and D. G. Lowe, “Recognising Panoramas”. In: 10th International Conference on Computer Vision (ICCV 2003), October 2003. Y. Ke, R. Sukthankar, and L. Huston, “An efficient partsbased near-duplicate and sub-image retrieval system”. In: MULTIMEDIA'04: Proceedings of the 12th annual ACM international conference on Multimedia, New York, USA, 2004. ACM Press, pages 869-876. D. M. Bradley, R. Patel, N. Vandapel, and S. M. Thayer, “Real-Time Image-Based Topological Localization in Large Outdoor Environments”. In: Proceedings of the 2005 IEEE Conf. on Intelligent Robotics and Systems, August 2005. M. B. Kennel, KDTREE 2: Fortran 95 and C++ software to efficiently search for near neighbors in a multi-dimensional Euclidean space, 2004. T. G. Dietterich, Machine learning for sequential data: A review. Technical report, Oregon State University, 2002. F. Ferreira, V. Santos, and J. Dias, “A topological path layout for autonomous navigation of multi-sensor robots”, International Journal of Factory Automation, Robotics and Soft Computing, no. 1, 2007, pp. 203-215. Sajama and Alon Orlitsky, “Supervised dimensionality reduction using mixture models”. In: Proceedings of the 22 nd International Conference on Machine Learning, Bonn, Germany, 2005. G. McLachlan and D. Peel, Finite Mixture Models, John Wiley and Sons, 2000. A. Kaban and M. Girolami, “Initialized and Guided EMClustering of Sparse Binary Data with Application to Text Based Documents”. In: Proceedings of the 15th International Conference on Pattern Recognition (ICPR'00), volume 2, 2000. X. Wang and A. Kaban, “Finding Uninformative Features in Binary Data”. In: Sixth International Conference on Intelligent Data Engineering and Automated Learning (IDEAL '05), 2005. A. Juan and E. Vidal, “Bernoulli Mixture Models for Binary Images”. In: International Conference on Pattern Recognition (ICPR'04), volume 3, 2004, pages 367-370. J. García-Hernández, V. Alabau, A. Juan, and E. Vidal, “Bernoulli mixture-based classification”. In: A. R. Figueiras-Vidal et al., ed., Proc. of the LEARNING04, ISBN 84-688-8453-7, Elche (Spain), October 2004. J. Pylkkönen and M. Kurimo, “Duration modeling techniques for continuous speech recognition”. In: 8th International Conference on Spoken Language Processing (Interspeech 2004), Jeju Island, Korea, 2004, pages 385-388. B. Juang, L. Rabiner, and S. Levinson M. Sondhi, “Recent developments in the application of hidden markov models to speaker-independent isolated word recognition”. In: IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP '85, vol. 10, 1985, pages 9-12. M. Kaess, R.C. Arkin, and J. Rossignac, “Compact encoding of robot-generated 3D maps for efficient wireless transmission”. In: IEEE Intl. Conf. on Advanced Robotics, ICAR, Coimbra, Portugal, June 2003, pages 324-331. Articles 55
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N° 3
September 2007
A CONCEPTION OF EXPERT SYSTEM FOR MIG-29 AIRCRAFT Received 16th June; accepted 12th July.
Krzysztof Butlewski, Piotr Golański
Abstract: This paper presents the concept of the MiG-29 aircraftdedicated diagnostic expert system. Subsequent stage of evaluation expert system from knowledge acquisition to software implementation has been described. As implementation tools a combination of expert systems shells CLIPS and LabWindows data acquisition libraries has been chosen. The whole system has been integrated in BorlandC++ environment. Keywords: diagnostics, expert systems, expert systems shells, CLIPS, Mig-29 aircraft
2. Problem Analysis Architecture of typical expert system is presented in fig. 2. This architecture consists of following components: inference engine, knowledge base, user interface, working storage, hardware interface. A knowledge base is an implementation of knowledge of domain expert, created by a knowledge engineer. An inference engine acts on knowledge base and generates a solution of problem, defining by user. User interacts with the system using the User Interface (usually Graphical User Interface - GUI). To the structure proposed in [9] is added a Hardware Interface, necessary for exchange data between the system and on board devices of an aircraft.
1. Introduction This issue is concerned on the problems of exploit new information technologies for creation computer diagnostics system software for some type of aircrafts exploit in the polish Air Force (Fig.1). The Computer diagnostic system likewise any information system [1], can be created with the aid of various languages and various developing environments. Most often, it takes advantage of imperative programming language like C or Pascal. However, since many years in literature citation, you can find an example of adaptation the expert systems shells for creation a diagnostic software [3, 8, 11, 12]. Expert systems shells are founded on syntax and semantics of declarative language like Lisp [7] or Prolog [4]. The essential difference between expert system and traditional problem solving system inheres in coding method. In traditional approach this method is coded in the form of algorithm and corresponding data structures. In expert system's approach the solving method is coded in data structures exclusively. This approach will be presented here in context of aircraft diagnostics system.
Domain expert
User
System Engineer
expertise
User Interface
Knowledge Engineer
Inference Engine
Working Storage
encoded expertise Knowledge Base Hardware Interface
Fig. 2. Architecture of aircraft's diagnostic expert system. Diagnostic System
The first step for the expert system creation is a knowledge acquisition in the particular domain of knowledge. In literature citation we can find different methods of knowledge acquisition. For example in [2] authors presents a diagnostic expert system, that knowledge acquisition has been founded on measurement date from the experiment, executed on the object, for which the system has been dedicated. In our case, three sources of domain knowledge exist: 1) the Russian technical documentation of diagnostic device currently applied for aircraft, 2) the result of verification of the above documentation, 3) knowledge of the diagnostic device operators.
Fig. 1. Aircraft's computer aided diagnostics. In the next paragraphs subsequent stage of evaluation expert system from knowledge acquisition, across diagnostic model definition and tools specifications to software implementation will be presented.
56
Articles
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N° 3
September 2007
Founded on this knowledge a diagnostics model has been defined [6]: M=(Q, S , d , q0 , F) where: Q - finite set of states, S - finite entrance alphabet, q0 - initial state (q0 ÎQ), d - transient function d : Q ´ S ® Q , F - set of final state (FÌ Q). Finite set of states is the set of stages defining diagnostic procedures consisted of accomplishment of required connections by operators and measurements or generating appropriate signals by diagnostic device on a specific input and output of an aircraft. Finite entrance alphabet consists of two elements: Z - correct and N - failed. Transient function d, in dependence of actual state and result of checking, transfers an automaton to appropriate state. An exemplary fragment of a model is presented in Fig. 3. This model has form of a directed graph, that nodes are appropriate states of an automaton, identified by the name (e.g. P-008-00). The states are related by the transient function . A value of this function depends on result of a checking function test (e.g. test 7). Especially when checking not occurred, from one to only one state transient is possible. This situation appears when either stimulation signal is generated or failed message is displayed (eg. message „block BNO-3PP failed” in state P-012-00). After modelling stage it is possible to come to implementing the system by means of existing expert systems building tools. (1)
Most essential elements of an expert system are: a set of data declarations depending on problem, named knowledge base and an independent on problem (but depending on data structures) program named inference engine. Knowledge base can be represented as: logical, net, production or frame models [10]. Inference engine can realize one or both of two inference techniques: forward chaining and backward chaining. From an aircraft diagnostic model presented in Fig. 3 arises, that knowledge base of an expert system for an aircraft has to be modelled by production rules and his inference engine has to realize forward chaining. As a result of searching for implementing tools, a CLIPS expert systems shells has been chosen. By means of this tool system can be formalized and implemented. Taking advantage of CLIPS language [5], nodes and edges of the conceptual graph have been defined. Fig. 4 presents a definition of a node in a form of structure deftemplate with three slots: First slot nazwa identifies node of graph thus state of automaton. In slot wynik, test result for given state is stored. Slot parent is an auxiliary field, permitting to implement test path. P-004-00 P-005-00 P-006-00 P-007-00 P-008-00 P-009-00 P-010-00 P-011-00 P-012-00 P-013-00 P-014-00 P-015-00 P-135-00)) slot wynik (type SYMBOL) (allowed-symbols NIEZDATNY ZDATNY) slot parent (type FACT-ADDRESS SYMBOL) (allowed-symbols no-parent))) (deftemplate MAIN: :parametr (slot nazwa (type SYMBOL) (allowed symbols
Fig. 4. A definition of automaton state in CLIPS. A definition of two edges of graph in a form of CLIPS rules is presented in Fig. 5. (defrule MAIN: :od5do7 ?node <- (parametr (nazwa P-005-00) (wynik ZDATNY) (nazwa P-007-00) (wynik (test 7))))
3. System implementation P-007-00 test 7 to P-135-00
=> (duplicate ?node
Z
Z P-008-00 test 8
N P-012-00
(defrule MAIN: :od5do6 ?node <- (parametr =>
(nazwa P-005-00) (wynik NIEZDATNY)
~BNO 3PP
(komunikat NIESPRAWNY-BNO-1PP) (duplicate ?node (nazwa P-006-00)))
N P-005-00 test 5
Z P-009-00 test 9
N N N P-011-00 test 11
Fig. 5. Two edges of conceptual graph in CLIPS. Given rule is activated if all of its conditional elements are satisfying. In this case, if system is in the state P-00500 and the test result of this state are CORRECT then od5do7 rule is activated. When the rule is chosen, the new state P-007-00 is created and function test with the parameter 7 is calling. Result of this function is returned in slot wynik. In some cases a message is generated (e.g. BNO-1PP DAMAGE). In Fig. 6 software architecture of proposed diagnostic expert system is presented. For implementation test function a C DAQ libraries of LabWindows are applied. DAQ library permits to data exchange with on board devices of MiG-29 aircraft. Furthermore, standard GUI interface is removed from the open source CLIPS application and in this place a customized user interface libraries from Articles 57
Z N P-010-00 test 10
Z Z
N
P-014-00 test 14
Z ~BNO 1PP
P-006-00
Z to P-015-00
N
P-013-00 test 13
to P-135-00
Fig. 3. An aircraft diagnostics model.
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N° 3
September 2007
LabWindows environment are inserted. Knowledge base is stored in a form of text files that include structures and rules written in the Lisp language. Knowledge base rules are activated and fired by inference engine embedded in a diagnostic module. This module, including CLIPS source files, is compiled in BorladC++ 5 environment. The whole system is compiled and consolidated in BorlandC++ environment. Diagnostic Module Inference Engine (CLIPS) DAQ Library (LabWindows)
[3]
[4]
[5]
Knowledge Base (*.clp files) GUI (LabWindows) AIRCRAFT
[6]
[7]
Fig. 6. Architecture of Aircraft diagnostic Expert System. [8]
4. Conclusion A presentation of conception of an expert system application to MiG-29 aircraft diagnostics has been a goal of this issue. This conception is founded on advantages of CLIPS expert systems shells combined with LabWindows libraries. The whole system has been integrated in BorlandC++ environment. This solution is tested on simple MiG-29 diagnostic procedure. Positive results of this work let suppose, that given idea allows building diagnostic expert system for MiG-29 aircraft. Obviously to obtain complete system, big effort of coding knowledge base in Lisp notation is needed. Additionally, after modifications, this system can be applied for checking technical condition of other Russian-made aircrafts owned by Air Force in Poland. ACKNOWLEDGEMENTS The authors would like to express his thanks to P. Weryński, Air Force Institute of Technology, for his aircraft terminology consultation. [9]
[10]
[11] [12]
identified model of diagnosed object], Proceedings of 8th International Conference Airplanes and Helicopters Diagnostics, AIRDIAG'2005, 27-28.10.2005, Warsaw, Air Force Institute of Technology. Gartner D., Sheppard J., W.: An Experiment in Encapsulation in System Diagnosis. Test Technology and Commercialization - Conference Record, AUTOTEST-CON'96, pp. 468-472. Gatnar E., Stąpor K.: Prolog - język sztucznej inteligencji [Prolog - the language of artificial intelligence], PLJ Publishing House, Warszawa 1991. Giarratano J.C.: CLIPS - User's Guide, http://www.ghg. net/clips/download/documentation/userguide.pdf, 2002. Homenda W.: Elementy lingwistyki matematycznej i teorii automatów [Essentials of mathematical linguistics and automation theory] Oficyna Wydawnicza Politechniki Warszawskiej, Warszawa 2005. Jurkiewicz Z., Lao M. J.: Język programowania LISP [Programming language LISP] PWN, Warszawa 1990. Liu S.T., Kelly G.E.: Rule-Based Diagnostic method for HVAC Fault Detection. Proceedings of Building Simulation'89, pp. 319-324. Merritt D.: Building Expert Systems in Prolog, http:// dcis.nohyd/ernef.in/courses/AAI/CD-1/ExpertSystems InProlog/xsip_book.pdf, 2000. Pospiełow D.A.: Sprawocznik Iskusstwiennyj intielekt. Kniga 2. Modieli i mietody. [Artificial intelligence. A guide. Book 2. Models and methods], Radio i Swjaz', Moskwa 1990. Weissman G., “Tworzenie systemu ekspertowego” [Expert system creation], Software 10/96, pp. 24-26. CLIPS'94, Third Conference on CLIPS Proceedings (Electronic Version) http://www.ghg.net/clips/ download/documentation/3cpp.pdf, Lyndon B. Johnson Space Center, September 12-14, 1994.
AUTHOR Krzysztof Butlewski and Piotr Golański* - Air Force Institute of Technology, ul. Księcia Bolesława 6, 01-494 Warszawa, skrytka poczt. 96, Poland; e-mails: krzysztof.butlewski@itwl.pl, piotr.golanski@itwl.pl. * corresponding author
References [1] Beynon-Davies P.: Inżynieria systemów informacyjnych [Information Systems Development. An Introduction to Information Systems Engineering, Third Edition, Macmillan Press Ltd, 1998], Pol. trans. by Małgorzata Szadkowska-Rucińska, WNT Publishing House, Warszawa 1999. Borowczyk H., Kwieciński R.: Tworzenie bazy reguł ekspertowego systemu diagnostycznego z wykorzystaniem zidentyfikowanego modelu diagnozowanego obiektu [Creation of base of diagnostic system rules applying
[2]
58
Articles
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N째 3
September 2007
ARCHITECTURE OF MOBILE ROBOTICS PLATFORM PLANNED FOR INTELLIGENT ROBOTIC PORTER SYSTEM - IRPS PROJECT Received 5th June; accepted 16th July.
Adam Woloszczuk, Mariusz Andrzejczak, Piotr Szynkarczyk
Abstract: The aim of this paper is to present the Intelligent Robotic Porter System with respect to architecture of Mobile Robotic Platform. The set of requirements for Mobile Robotic Platform was defined and analysed. Main engineering problems that occurred during the development of Mobile Robotic Platform's architecture were specified and described. Problems were confronted with PIAP's proposed solutions for technical issues at this stage of the project. Keywords: mobile robots, autonomous navigation, mobile platform, teleoperation
The IRPS project will demonstrate capabilities of the LIMS module integrated with the mobile platform on the move with nominal speed of 5 m/s. This concept will be developed and implemented as a key component for robotic applications for quality and dimensional control, ambience monitoring inspection and advanced navigation. Applications will be deployed for operation in large areas and within congested human environments. Operation within a network of co-operative robots will be considered.
2. IRPS structure and modules 1. Project overview The Intelligent Robotic Porter System (IRPS) is an th international project within the 6 Framework Progth ramme, 6 Call IST (Information Society Technologies). Project is scheduled for 01.01.2007 to 31.12.2009. The purpose of the project is to provide mobile robotics platforms with advanced mapping and navigation systems based on a very accurate 3D-measurement technology. The purpose of the mobile platform will be to transport disabled persons within the area of public airport, which is structured, however variable environment. Implementation of navigation system, working in real time, able to define objects in the robot's workspace is essential to the project. The 3D measurement technology, named LIMS (LIDAR Imaging and Measurement System, LIDAR - Light Detection and Ranging) based on three eye-safe laser systems originated in Israeli Aerospace Industries - Lahav Division meets application requirements. It was shown, in feasibility study, that highly accurate measurements could be achieved in real time, on the move and in presence of sparse objects and vibrations. LIMS will be used to recover the position of the robot related to the environment by using natural landmarks, through the comparison of the actual 3D map with previous 3D maps already stored in system memory. This operation will be complemented with reflectance and video data to increase information and robustness. Since the positioning and mapping will not rely on absolute positioning systems, the application can be used indoors as well as outdoors environments. The main objective of the project is to develop and integrate an accurate 3D sensing system as a modular component for robotic platforms, enabling fast dense mapping of large areas, populated with sparse objects. The IRPS will consist of three main modules (subsystems). The LIMS will be hosted on the mobile platform. Both LIMS and MRP communicate with MCC, which receives data from LIMS and generates high-level commands for MRP. The high level commands received by MRP on-board PC are converted to low level commands (desired speed, desired wheel turning angle) for the low level controller and then executed. LIMS scans and maps the area of operation. Currently acquired maps are compared to 3D maps stored in system memory. The position of robot is calculated and appropriate commands are sent to MRP on-board PC. LIMS operates in one of two modes: Data Acquisition Mode and Navigation Mode. In Data Acquisition Mode operator controls robot. LIMS maps the area and stores the 3D maps in system memory. These maps will be the reference used when operating in Navigation Mode. When in Navigation Mode, robot executes commands generated by MCC and compares current local 3D maps with those acquired in Data Acquisition Mode to determine its position. MRP internal communication will be provided by CANbus. Communication with MCC will be via Radio-Ethernet or IP at the early stages of the project, and ultimately via Wi-Fi or Wi-max. The IRPS sub-systems are described below. The Mobile Robotic Platform (MRP) At the early stage of the project (prototype V1), a relatively small sized robot will host the LIMS and onboard hardware. The main objective of MRP V1 will be operation in laboratory conditions as the test platform for LIMS and other essential components. Ultimate prototype (V3) will be fully customized for the needs of disabled passengers and will be adapted to operate within the crowded area of public airport. The MRP mechanics will be based on electric cab.
Articles
59
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N째 3
September 2007
The Mission Control Centre (MCC) The MCC is in charge of collecting commands from handling agents or the user. The MCC operator uses the various applications available in the MCC to task the robot and plan the route based on the complexity of the environment. In general, the MCC oversees the operation of MRP and LIMS. At the early stages, the MCC will consist of an external PC wired with the MRP via IP. Also, the use of Radio-Ethernet is considered. Along with the project development, the MCC will be constantly improved, until it evolves into fully operational control station. The LIDAR Imaging and Measuring System (LIMS) LIMS is the 3D eye-safe laser scanner dedicated to the observation and detection of known structured environment changes, based on the accurate measurement of the platform position on the move. Systems relies on the previously acquired 3D maps stored in memory which are being dynamically compared to currently acquired maps, so that the robot position can be determined. Fig.1 represents IRPS system structure, with three main sub-systems specified. Note communication between sub-systems via Radio-Ethernet or IP, because this is the diagram for the first IRPS prototype. Later prototypes will communicate with MCC via Wi-Fi or Wi-Max as stated in end-user requirements.
Table 1. General requirements for Mobile Robotic Platform. Type of requirement number of seats mass speed range turning radius additional cargo space maximum dimensions: constraints: - dimension of the lift - metal detector gate dimensions: communication warning system: horn, lights additional requirements
Description 1 (2 in a row) max. 500kg per m less than 17km/h not less than 30 km (2h of continuous drive) max. 2m only for hand luggage only and batteries the narrowest dim. 1m 71 cm wireless, allowed on the airpor horn, lights no reconfiguration on board, only software update by control room operator
The IRPS will be safe for the passenger and environment thanks to the LIMS Change Detection function as well as a set of ultrasonic sensors operating as the redundancy for the LIMS. Passenger will be able to communicate with the system through user-friendly HMI. HMI will be also equipped with safety features such as joystick for emergency-manual steering, emergency stop button and passenger detection.
4. MRP technical problems 4. 1 Communication: Internal The communication inside the MRP should be divided into two layers - communication between low level control system and actuators and communication between high level control system and low level control system. The communication protocol between high and low level control should be optimised for sending steering information. External The communication between MCC and MRP should be broadband because of the huge amount of data send between them. It should be wireless and use the band that is allowed on the airport. The protocol used should be universal as different applications will use it for sending data - information from sensors, steering commands, information dedicated to localization and map building as well as navigation and path planning. 4. 2 Power supply Demand of LIMS It has not yet been stated if LIMS is equipped with its own power supply. Assuming that LIMS does not have it, the additional on-board set of batteries is required. The additional power supply has to be capable of supplying LIMS with up to 1 kW of peak power. Other payload Apart from LIMS and executive mechanisms, MRP host other modules that need to be supplied with power. Onboard PC and ultrasonic sensors need to be provided with power. Executive mechanism Executive mechanism will consist of steering servo
Fig. 1. IRPS structure [1].
3. Requirements for MRP General requirements for IRPS have been stated basing on the analysis of environmental conditions, passenger needs and safety issues. The set of requirements was obtained during interaction with consortium partners and survey of end - user community. These requirements apply to the final demonstration prototype, adapted to operate within the crowded area of public airport. The preliminary requirements gathered during the first phase of system specification are listed in Table 1. Further, requirements that are more detailed are about to be developed during the project.
60
Articles
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N째 3
September 2007
and braking actuators that should be supplied with enough power to operate along the full range of MRP. User requirements for range Overall, power supply for MRP should meet user requirements concerning range. All the on-board modules should be constantly supplied with power during two hours of continuous drive. 4. 3 Executive mechanics Directional control MRP motion system should be optimised for manoeuvrability, speed and quick reaction so that it would meet requirements of safe operation within the area of public airport. Speed control Main problem concerning speed control is the reduction of cab's velocity. The braking should be smooth enough not to disturb the passenger and fast enough to stop the cab before hitting the potential obstacle. 4. 4 Physical requirements Dimensions As stated in point 3, requirements concerning physical properties of MRP strictly affect the selection of electric cab used as MRP. Due to the environmental conditions of public airport, the dimensional constraints for MRP may prove to be hard to meet. Weight Electric cab together with LIMS, on-board hardware, power supply, passenger and his hand baggage should not exceed the weight permissible for the airport structure and lifts. Additional power supply for on-board modules is the main factor increasing MRP overall weight. Payload MRP should withstand applied load. Again, the main problem is the additional power supply for the on-board modules that will significantly increase the load of the MRP. Also, the overall mass of LIMS has not yet been determined. Payload cannot exceed electric cab's structural strength. Passenger seat and cargo space Sufficient space for passenger and space for his hand baggage should be provided on the MRP. Passenger should feel comfortable while travelling on MRP and his hand baggage should be safely transported. The problem is to optimise MRP size so it would meet physical requirements (the more compact, the better) as well as it would satisfy the passenger.
tion will have two instances - one in MCC and one on MRP, the communication between them will be done with the use of IP. This protocol will be used for communication between MCC and MRP. The internal MRP communication between external and on-board instances of high-level control will be executed through Ethernet bus with IP. The communication between high-level and low-level controllers will be implemented by CAN-bus and CAN-open protocol. The CAN standard is widely used in automotive and mobile robots industry and is easy to implement. 5. 2 Power supply Electric cab's motor will use its own batteries provided by manufacturer. Depending on MRP modules power consumption, the power for on-board hardware will be supplied by electric cab's batteries or additional batteries. If the on-board modules are to be supplied from cab's batteries, PIAP team will provide power outputs (for example 24V) from cab's batteries to supply them. In case of installing additional power supply on MRP, its weight and dimensions are essential to platform mechanics. Therefore, the use of lithium polymer batteries is considered. Their price is high, comparing to gel batteries, however their small dimensions and light weight together with relatively large capacity makes them considerably the best choice for supplying MRP modules with power [3]. The lithium polymer batteries are compact, so that they can be grouped into sets dedicated to supply specified modules. The use of lithium polymer batteries would save space and mass of the MRP. Currently, all consortium partners are working on estimating the power consumption of their modules, so that the question of additional batteries could be resolved.
Fig. 2. Example of lithium polymer 50Ah/30V batteries [6].
5. Proposed solutions 5. 1 Communication The communication issues apply to MRP-MCC communication as well as internal MRP communication. After the analysis of available communication types was stated that IRPS would use Wi-Fi standard for remote communication with MCC. The use of Wi-max standard is taken into consideration. Both of the communication standards are allowed to use on the airport and the existing WiFi infrastructure can be applied for the system [2]. As all the applications responsible for path planning and naviga5. 3 Executive mechanism Executive mechanism will consist of system responsible for turning the wheels of the cab to desired angle (directional control) and speed control system responsible for moving the cab with desired speed. Executive mechanics will be operated by low-level controller, which receives commands (desired wheel turning angle, desired speed) generated by high-level controller. Servomotor will be implemented as wheel turning system. Current angular position of wheels from servo encoder will be dynamically compared to desired angular Articles 61
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N° 3
September 2007
position of wheels. Servomotor's shaft will be coaxial with steering column and geared with it. Steering wheel will be removed to provide the comfort of passenger. Use of actuator-based braking system operating in speed-feedback loop is taken into consideration. The braking actuator either push the brake pedal or be built into cab's brake system, therefore it would directly operates jaw brake shoes. 5. 4 Physical requirements The structural strength of airport facilities is large enough to withstand the load of 500 kg per square meter. Anticipated overall weight of MRP with passenger and his hand luggage will not exceed 400 kg, so there is no risk to affect the airport facility structure. Additionally, if LiPo batteries will be used, the overall weight of MRP, as well as its payload, will be significantly reduced. The narrowest dimension, resulting from detector gate width is the problematic issue affecting the selection of electric cab. Initially, before end-user requirements were formed, the use of Melex electric car was considered (see Fig. 3). Melex is the leading manufacturer of electric cabs in Poland. Melex cabs are suitable to operate within the public transport areas, however their dimensions are relatively large. Unfortunately, none of the Melex models will be able to pass through the detection gate with dimensions specified by end-user. After specifying the requirements, the selection of cab was verified and the Klingler electric cab model “bed pusher” was taken into consideration (see Fig. 4).
This model is more compact and more manoeuvrable compared to Melex; furthermore it maintains reasonable payload capacity. The on-board modules can be installed in front of the cab, and the cargo space will be on the rear side of the cab. After purchasing selected model compliant with IRPS requirements, PIAP will adapt the electric cab's construction with respect to passenger needs and physical properties of the modules provided by consortium partners. The cab's construction mechanical adaptation will consist of installing additional mountings for on-board modules, handles for passenger and baggage basket or box.
6. Proposed MRP architecture After the analysis of main technical problems and confronting them with proposed solutions [4,5], the hardware structure block diagram was elaborated (Fig.5 see next page). The diagram represents main functional blocks of IRPS (hardware aspect). The area of diagram marked with EXECUTION annotation consists of main responsibilities of PIAP in the course of elaborating mobile robotic platform for IRPS.
7. Summary In this article, the IRPS project and its main technical challenges were described. The survey made for end-users helped in definition of their requirements and translation it into the technical constraints that were shown within. The technical problems that were defined by the project team were listed and shortly described. The proposed solutions for the problems that had been developed were formulated and shown accordingly. Finally, the MRP architecture that fulfils the preliminary user requirements was proposed. ACKNOWLEDGEMENTS This work was done with support of EU (6FP) under IRPS (Intelligent Robotic Porter System) project No IST-045048.
Fig. 3. Melex cab - first selection for MRP [7].
AUTHOR Adam Woloszczuk*, Mariusz Andrzejczak, Piotr Szynkarczyk - Industrial Research Institute for Automation and Measurements PIAP, Al. Jerozolimskie 202, Warsaw, Poland, e-mail: awoloszczuk@piap.pl. * corresponding author
References [1] [2] [3] IRPS community internal materials. Telecommunications Act of 1996, Pub. LA. No. 104-104, 110 Stat. 56 (1996). Oman H., “New battery types for space vehicles”, Aerospace and Electronic Systems Magazine, IEEE, vol. 17, issue 4, April 2002, pages 34-40. Bräunl T., Embedded Robotics: Mobile Robot Design and Applications with Embedded Systems, Springer 2006. Nehmzow U., Mobile Robotics: A Practical Introduction, Springer 2006. www.grc.nasa.gov www.melex.com.pl www.klingler-ag.ch
[4] [5] [6] [7] [8]
Fig. 4. Klingler "bed pusher" - alternative to Melex [8]. 62 Articles
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N째 3
September 2007
Fig. 5. IRPS hardware block diagram [1].
Articles
63
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N° 3
September 2007
INFLUENCE OF A REAL RADIO CHANNEL PARAMETERS ON THE QUALITY OF THE OFDM SIGNAL IN MOBILE APPLICATIONS. ESTIMATION OF PARAMETERS OF THE CHANNEL AS AN ELEMENT OF IMPROVEMENT OF THE TRANSMISSION QUALITY Received 14th February; accepted 25th April.
Konrad Bożek
Abstract: The article deals with issues related to transmission of a signal with the OFDM multitone modulation in a real radio channel. Following short characteristics of the OFDM modulation, the article includes an analysis of typical radio channels from the angle of amplitude and phase on the basis of delay profiles referred to in the literature. The study contains results of simulated receipt of the OFDM signal in the multipath conditions. On the basis of diagrams of constellation of the modulation symbols, the author presented differences in the quality of the signal before and after the phase correction and improvement in the quality due to application of cyclic prefix. Influence of noise as well as the fact that the terminals are non-stationary on operation of the frequency characteristics estimation algorithm was analyzed. At the end, an example was presented of application of a linear model of estimation of phase characteristics of a channel with the shift in frequency occurring in mobile systems as a consequence of the Doppler effect being compensated. Keywords: OFDM, radio transmission, multipath, radio channel estimation
2. OFDM modulation Thanks to dynamic development of the signal digital processing technologies, it is possible to use multi-tone modulations, including the OFDM (Orthogonal Frequency Division Multiplexing) modulation, which is the subject matter of this study. Since the information is contained in modulated subcarriers, the number of which ranges from several dozen through several thousand, it is possible to demodulate each subcarrier individually, depending on the system. Thanks to the latter fact, under circumstances where, as a consequence of multipath, a portion of subcarriers will be suppressed or interfered by other system, the other subcarriers may be properly demodulated. If appropriate corrective redundancy codes are applied, it will be possible to correctly receive all the information under the above circumstances. a Relative power density [dB]
1. Introduction The sending of information over a distance is one of fundamental issues that mankind has been developing through centuries. A precondition of information reaching the recipient is the existence of certain carrier called a transmission medium. Nowadays, at the age of fast development of technology, the most broadly used medium is electromagnetic waves propagating in the surrounding space or appropriate wave-guides. The electromagnetic wave spectrum is very broad. In nature, electromagnetic waves occur within a range from very low frequencies (energy sent through electric power network) through 20 frequencies exceeding 3·10 Hz (cosmic radiation). Frequencies of electromagnetic waves used in contemporary terrestrial mobile applications usually come within the range from 30 MHz through 30 GHz. When waves propagate in inhomogeneous space, which inhomogeneity consists of the structuring of the land, buildings, trees, etc., phenomena deteriorating the conditions of signal transmission occur, which include: reflections, diffractions or interferences conducive to origination of multipath in the radio channel. In addition, a Doppler effect occurs under circumstances where one terminal is in motion with respect to the other. Notwithstanding the above phenomena, the utility signal received is always accompanied by noise that limits the available sensitivity of the receiver. 64 Articles
Delay [microseconds]
b
Relative power density [dB]
Delay [microseconds]
c
Relative power density [dB]
Delay [microseconds]
Fig. 1. Delay profiles for various types of terrain: a) flat area (non-built), b) mountain area, c) built-up area (city) source [2]. Multipath radio signal occurs in practically every mobile system. Its consequences can be analyzed in three areas. In terms of space, multipath causes origination of
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N째 3
September 2007
inhomogenous distribution of electromagnetic field in the propagation space; in terms of time it causes intersymbol interferences, and selective fading in terms of frequency. Figure 1 presents exemplary profiles of radio channel delays for various types of terrain. On the basis of the said profiles, we can calculate relative amplitudes of signals reaching the receiver through different paths. We still have to determine the phases of the signals. Let us assume that the system operation frequency amounts to 870 MHz (wavelength 0.345m). Thus, if we know the channel delay profile to an accuracy of several hundred nanoseconds, we might determine a difference in the propagation paths to an accuracy of no more than several dozen meters. This is much more than the carrier wave length; therefore it is justifiable to assume random distribution of phases for individual components of the delay profiles. Figures 2 and 3 present delays calculated on the basis of the data of profiles, based on an assumption of random distribution of phases, the amplitude and phase characteristics of a channel for different types of terrain. The values on the abscissae axis identify the subcarrier number of the given OFDM signal. a unnormalized channel amplitude characteristic
a
unnormalized channel phase characteristic
phase
normalized frequency
unnormalized channel phase characteristic
b
phase
normalized frequency
c
unnormalized channel phase characteristic
phase
normalized frequency
normalized frequency
b
unnormalized channel amplitude characteristic
Fig. 3. Radio channel phase characteristics for: a) non-built area, b) mountain area, c) built-up area (city).
3. Methods of the improvement of radio transmission quality A cyclic prefix is added to the OFDM modulation symbols in order to protect them against inter-symbol interferences, as well as to ensure orthogonality (independence) between individual subcarriers. The protection of the signal is fully effective when differences in delays of individual paths of signal propagation do not exceed the duration of the cyclic prefix. Figure 4 presents the structure of the OFDM symbol with the cyclic prefix attached. Copy of bottom symbol part to front
normalized frequency
c
unnormalized channel amplitude characteristic
direct signal n-1
n
n+1
normalized frequency
reflected signal (delayed)
n-1
n FFT evaluation interval
n+1
Fig. 2. Radio channel amplitude characteristics for: a) non-built area, b) mountain area, c) built-up area (city).
Fig. 4. OFDM symbol with cyclic prefix enclosed. Picture shows the mechanism of the protection agains intersymbol interferences. Articles 65
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N° 3
September 2007
The key element of procedure of preparation of an OFDM signal received for demodulation is to perform a phase correction on the basis of estimation of phase characteristics of a radio channel. In the formula below, R0(k) means discreet Fourier transform (DFT) of the synchro-nization symbol received. DFT of the original synchro-nization symbol, which the receiver knows beforehand, is marked as R0(k). The following relation exists between these two transforms: R0(k) = R0(k)·H0(k)+DR0(k) (1)
a
signal constellation on a given subcarrier
Where H0(k) is the channel frequency response, and DR0(k) an error of the transform related to presence of noise in the signal received during transmission of the synchronization symbol. Energy of a noise accompanying the utility signal per modulation symbol is associated with error DR0(k), as follows: EN = åDR0 (k) 2 k
signal constellation on a given subcarrier
b
(2)
If we know DFT of the broadcast (original) and received symbol, we can set an estimation of characteristics of a channel: H0(k) = R0(k) = H0(k)+DH0(k) R0(k) (3)
In the above formula, DH0(k) is an error of the estimator caused by presence of noise in the received signal. This is a stochastic error, and, therefore, it cannot be eliminated through a correction. However, we can minimize its value using the method of averaging estimations calculated on the basis of several synchronization symbols. When we know the estimation of transmittance DH0(k), on the basis of DFT of the nth information symbol received Rn(k) we set estimation Rn(k) DFT of an appropriate original symbol Rn(k): Rn(k) = Rn(k) H0(k) (4)
signal constellation on a given subcarrier
c
Figures 5, 6 and 7 present distribution of the points of constellation of one of subcarriers of the OFDM signal received on the IQ plane in different field conditions and different configurations of the signal protection elements. The phase equalizer, the effect of operation of which is visible in figures 5b, 6b and 7b, rotates the signal so that the points of its constellation are located in an appropriate quarter of the complex IQ coordinate system. In absence of cyclic prefix, as a consequence of intersymbol interferences, the signal constellation points located in individual quarters of the IQ plane are somehow dispersed. A definite improvement in the quality of the signal thanks to application of the cyclic prefix is conspicuous in figures 5c and 7c. In one of the discussed cases, which were a mountainous area, the difference in the paths of propagation exceeds an allowable value resulting from the fixed length of cyclic prefix. Therefore, even though the signal is protected with a cyclic prefix the intersymbol interferences still occur. (Fig. 6c). 66 Articles
Fig. 5. OFDM sub carrier constellation with propagation in non-built area: a) without cyclic prefix and correction, b) with a correction only, c) with correction and cyclic prefix. signal constellation on a given subcarrier
a
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N掳 3
September 2007
b
signal constellation on a given subcarrier
c
signal constellation on a given subcarrier
signal constellation on a given subcarrier
c
Fig. 7. OFDM sub carrier constellation with propagation in built-up area (city): a) without cyclic prefix and correction, b) with a correction only, c) with correction and cyclic prefix. In the case of non-stationary radio channel, the transmittance estimation error is increased by additional component DH'n(k) resulting from changes in the parameters of the channel over time. Therefore: Hn(k) = H0(k)+ DH'n(k) (5)
Fig. 6. OFDM sub carrier constellation with propagation in mountain area: a) without cyclic prefix and correction, b) with a correction only, c) with correction and cyclic prefix. signal constellation on a given subcarrier
a
An error of non-stationary channel grows in pace with increase in the number of estimations of transmittances that are subject to averaging and may be minimized through creation of a channel variability model. In mobile applications based on the OFDM system, individual sub carriers are quadrature modulated (QPSK). Therefore correction of the frequency characteristics of the channel may be limited to a correction of its argument (phase). j0(k)= arg[H0(k)] (6)
When applying a linear (first order) model of esti-mation of transmittance argument acknowledging a linear phase reverse over time, for the nth symbol of modulation, the estimated value is: jn(k) = j0(k)+n路dj(k) Where n is the number of the symbol. signal constellation on a given subcarrier
(7)
b
Using the linear model of estimation of argument of channel frequency characteristics, we can describe the Doppler effect occurring in mobile system. Shift of a component of the signal spectrum by Df as a consequence of effective motion of the receiver with respect to the transmitter at a velocity the component of which parallel to the propagation direction amounts to u consists of the following dependence: Df = - f 路 u c (8)
where f is a frequency with which the given component of the spectrum was transmitted, and c is the velocity of propagation of electromagnetic wave in vacuum.
Articles
67
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N° 3
September 2007
Let us assume that the Doppler shift in the k-th sub carrier of the OFDM signal amounts to Dfk, and duration of the modulation symbol amounts to t. Thus, the estimation of the transmittance argument of m-th signal path will be as follows: (9) u jm,n(k) = jm,0(k)+2pDfk ·n·t = jm,0(k)-2p·fk · m ·n·t c It would be still better to apply a model of higher order, the second one for example, which would acknowledge alteration of speed of a car moving with a constant acceleration. However, to increase the order of the estimation leads to increased amount of calculation related to the setting the said order, which enforces making a compromise between the channel parameters fluctuation immunity and the degree of complexity of channel estimation algorithms.
4. Conclusion In conclusion, I would like to emphasize great importance of the mechanisms of estimation of radio channel parameters in the process of receipt of broadband signals, which is proven by the results of the simulation performed. Due to limited volume of the article, I have given up a thorough analysis of the problem and presented it in a possibly simple form. People interested in the problem of channel correction parameters should refer to the literature where it is discussed in more detail.
AUTHOR Konrad Adam Bożek, M.Sc. Eng - Industrial Research Institute for Automation and Measurements PIAP, Intelligent Mobile Systems Division, Al. Jerozolimskie 202, 02-486 Warsaw, Poland, kbozek@piap.pl.
References [1] S. Haykin, Systemy telekomunikacyjne [Communication systems], translated by B. Galiński, G. Hahn, WKŁ Publishing House, Warszawa,1998. K. Wesołowski, Systemy radiokomunikacji ruchomej [Mobile radio communication systems], WKŁ Publishing House, Warszawa,1998. J. Heiskala, J. Terry, OFDM Wireless LANs: A Theoretical and Practical Guide, SAMS, Indianapolis, USA, 2001. H. Schulze, Ch. Lüders, Theory and Applications of OFDM and CDMA, John Wiley & Sons Ltd, West Sussex, England, 2005. M. Engels, Wireless OFDM Systems, Norwell, USA, Kluwer Academic Publishers 2002. L. Hanzo, C.H. Wong, M.S. Yee, Adaptive Wireless Transceivers, John Wiley & Sons Ltd , West Sussex, England, 2002. A. R. S. Bahai, B. R. Saltzberg, Multi-Carrier Digital Communications, Kluwer Academic Publishers, New York, USA, 2002. R.van Nee, R. Prasad, OFDM for Wireless Multimedia Communications, Artech House Publishers, Boston-London, 2000.
[2]
[3] [4]
[5] [6]
[7]
[8]
68
Articles
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N° 3
September 2007
Original Structure Could Be Reintroduced
Focus on new
about European education and collaboration with professor Piotr Tatjewski Piotr Tatjewski – is Professor at the Faculty of Electronics and Information Technology of Warsaw University of Technology. He is Director in the Institute of Control and Computation Engineering, and also Head of Control & Systems Division. Academician, member of Automation & Robotics Committee of the Polish Academy of Sciences. He also worked for Technische Universität in Hannover, City University in London, University of Birmingham. Tatjewski is most know from his research on control and on-line optimisation of the multilayer structures. From 1993 to 1997 Professor Tatjewski managed the EU Structural Fund Project named TEMPUS S-JEP 07181 – Information Technology for Control and Decision – Support Curriculum Development. The three-year-long project connected 16 technical universities from all European Union countries and had a budget of 700 000 ECU*. He wrote more than 80 articles and papers. He published also few books, like acclaimed Advanced Control of Industrial Processes: Structures and Algorithms, and lately with M. Brdys Iterative Algorithms for Multilayer Optimizing Control. J: Therefore, incorporated changes have been necessary to modernize the actual teaching standards? There had to be a readjustment of teaching standards to the demands of the Bologna process, and adjustments of study programmes at individual universities will follow. First, the Bologna process introduced a partitioning of the studies into lower (bachelor) and higher (master) levels, and second, it concerns completely differently the question of teaching standards. The old-type Polish standards contained certain number of subjects, treated as single or multiple courses, which should to be lectured at each university, college or academy. Currently European standards of teaching impose on the schools not only a demand of certain contents of the curriculum, but first of all duty of developing definite skills, which students should posses after completion of their education. Not before that we can test graduates from this point of view – if they fulfil standards or not. However, new teaching standards are not based on a concept of a subject or a course. They use a notion of teaching contents, which enclose the knowledge and, first of all, skills demanded from a student. The new standards are rather general directions how education should be organized, defining general requirements and a basis for a curriculum (study programme). According to the valid regulations, this basic contents do not need to cover more than 40% of the final curriculum. Every university must define its 100% curriculum autonomously, providing that its structure and 40% of its contents is in accordance with the standards. As you can see the main duty in constructing a programme of studies depends on a school. By the way, it is a challenge for the State Accreditation Commission. The accreditation process will have to be run in a different way than before, checking consistence of formal aspects of the teaching program with the teaching standards will be far not enough.
JAMRIS: Professor, let's talk about teaching of automation in Poland. What future this field has? Accusations that Polish universities do not groom students to work in in a present-day company appears quite often. You are the expert, who prepared a proposal of new standards for this branch of study. What can I say? First of all automation and robotics has light and unthreatened future, because equipment using automatic solutions is everywhere – in every house, in every vehicle, every office, every factory, only specifics and scale are different. The second is that contemporary automation is extremely close to information technologies, which means that installations of present day automation are designed by use of information technology (IT) hardware and software solutions to a great degree. At present day automation and robotics engineer has to be also an IT specialist. Therefore education of students' – in both fields has very much in common. The studies in automation and robotics should be based to an increasing degree on IT, or even can be Professor is an enthusiast of sport, the Baltic Sea and classical music. J: So, we have replacement of finished with a formal specialization e.g. responsibility now: individual work of in computer science. On the contrary, the student became equally – or even more – important the studies can be in computer science but with speciathan the topic of lecture. It is not enough to learn lization from the field of automation and robotics, as it lectures by heart. It can be a little revolution in many can be met for example in our department. Even on the Polish schools, especially in those less known. job market – if you look at graduate students all over the Yes, indeed, but it is not the main difference. Individual country – graduates in automation and robotics have no students' work always played a dominant role at technitroubles with finding a well-paid job, to a large extent cal universities. But before, the standards shaped major because they usually have a good background in IT. Interview 69
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N° 3
September 2007
part of each individual curriculum. Arranging the whole programme merely by fulfilling standards was possible. J: That is, universities have to be more competitive now? Yes, they have. They may more individualize and specialize programmes of the studies – to match them with the structure and specific expertise of the staff or with the profile of the local job market. For example, there are areas with a need for specialists in control theory, in particular in decision support theory, which is a part of modern automation. It is possible to shape the teaching programme according to that situation. In such a rather non–industrial region as Warsaw, specialists knowing IT and analysis of dynamic systems from the point of view of decision support techniques are important. An example: in my Institute is Professor Toczyłowski's research group, since many years engaged with problems of energy market in Poland. They create aggregate mathematical models of electrical energy production and transmission, decision planning, simulation of market's behaviour etc. These models are optimised, and on the basis of those e.g. decision–making rules, proposals for energy market organization are rising. J: You repeatedly stressed researches concern of multilayer systems of processes control and optimisation. From the time of my master thesis I am interested in this very general subject matter, that Prof. Findeisen formerly practiced with great success. Later professor Krzysztof Malinowski led this group, and further I have had my own team. We are working on multilayer control and optimisation methods, mainly for industrial processes. Multilayer control consists in decomposition of the initial, control task – which is naturally economical, e.g. to produce a prescribed quantity of petrol at minimal costs. Overall goals like that are decomposed to a set of simpler, partial sub-goals of different nature implemented at different control layers in the multilayer control hierarchy. Two basic control layers are: the direct control layer responsible for stabilization of technology objects at prescribed working points by means of feedback controllers, and the set-point optimisation layer which task is to define on-line short-term optimal working points (set-points) for the feedback controllers of the lower layer. Next higher layers of the hierarchy are concerned with mid-term and longterm production management and planning. The multilayer control system, for years a common practice in industry, is significantly simpler to be operated by humans and more resistant to breakdowns. Lower layers can work without the higher ones if these are out of order - of course, for not too long time, until a failure will be removed and original system structure will be reintroduced. However, an engineer at technical university is interested mainly in two lower layers: direct feedback control and optimisation of working points. My research group – known as Control Engineering Group – is working on this type of problems. In the last years we were mainly concerned with development of advanced control techniques, in particular modern predictive control algorithms for both linear and non-linear processes, and with deve70 Interview
lopment of set-point optimization algorithms, including those precisely tuned to work with predictive controllers. J: You were also a leader of the international project – like TEMPUS. I was a contractor and coordinator of a rather large project within the EU TEMPUS program in the 1990's, concerned with modernization of the curricula in the area of control and decision support. In this threeyear long project several technical universities took part: four from Poland (from Warszawa, Kraków, Gliwice and Rzeszów) and eight from the European Union countries. Generally, EU decided to support educational systems in East European countries, including Poland, long before they became the EU members – the TEMPUS program was elaborated to achieve this goal. It financed exchange of academic staff and students and purchasing equipment for laboratories. In effect, at four technical universities in Poland control laboratories were modernized, and a number of new courses was devised. Moreover, in Warszawa we introduced then, in 1990's, the two-stage model of studies at our Department (the first experience of that type at Polish technical universities). This model is being introduced currently at other technical universities as a result of the Bologna process. J: Therefore, after all that finally we educate engineers at the world level? We were never ashamed of the educational level at Warsaw University of Technology. I graduated in 1970's and many of my course colleagues' work at highly respected universities and laboratories all over the world. European or American universities have never questioned diplomas of Warsaw University of Technology. Our graduates were and are very welcome at western universities to do PhD studies. What counts in the world is the reputation of the university, not the country. In my judgement, not very good opinion about Polish education in general, which can be met now more and more often, has been caused by a freemarket type policy which was taken in 1990's in development of private high education schools, without providing an appropriate accreditation system from the very beginning. As a result of that, but also of insufficient financial support of state universities, general educational level in Poland has decreased. On the other hand, what happens at secondary schools providing candidates for university education makes our life also harder. I must confess that I was in dismay when during last session of our Department Council the information was given how many pupils will take the secondary school final examination (matura) in mathematics and physics on expanded level in this year. J: How many? About 10% in mathematics, circa 6% in physics. Do not forget, that final exams in mathematics and physics at the expanded level are usually required for admittance to a technical university, e.g. are necessary to be enrolled to the Warsaw University of Technology. Warsaw, 13th July 2007 * ECU former EU currency replaced by EURO.
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N° 3
September 2007
InFocus the Spotlight on new n Dance, dance, dance HRP-2 – (Humanoid Robotics Project) is the robot produced by Kawada company (Japan). Scientists from Tokyo University took the robot in their hand and taught it a Japanese folk dance called Aizu-Bandaisan. Traditional dances are vanishing from people's memory and such moving, interactive library seems to be optimal solution. The main problem in research like this is a balance. Imitation of hands and arms motions is easily for robots, but they have serious difficulties in moving legs. Dance Aizu-Bandaisan is based on motions of the hand and the trunk. The next step is faster dancing. Source: http://www.kawada.co.jp/global/ams/hrp_2.html n Mechanical perch Perch (Perca fluviatilis L.) have appeared an excellent model of AUV. Autonomous Underwater Vehicles are irreplaceable for deep-sea researches, exploitation of the wrecks, as well as in sea rescue. MIT's scientists under Prof. Ian Hunter noticed that perch swims differently from other fishes. Perch moves forward perpetually, without pulling back part of the trunk. Researchers inspired by perch's way of swimming constructed fin's model from polymer conducting electricity, as a computer control casting. Constructors want to understand principles of the nature to inspire them. The purpose is design automatic moving robots. n Inspector robot Inspecting and neutralizing robot is able to travel in difficult terrain and to overcome its uneven surface (smooth movements on stairs) and high obstacles. Inspector was created for support for counter terrorist operations, hazardous environments missions, as well as for protection and supervision of the buildings. Can cooperate with other robot. The manipulator is able to lift 30 kg on extended arms and 60 kg on folded arms. Constant spatial orientation of the object placed in the gripper, irrespective of the movement of other manipulator arms, enables precise manipulation of hazardous devices. Special driving system reduces recoil effect when firing the pyrotechnical disrupter or in case of explosion of the load placed in the gripping device or its vicinity. n Mechanical cuddly Followed of Canadian seal PARO (personal robot) can demonstrate feelings, moreover – it is much more clean and patient than original. Its hair has antibacterial qualities. Takanori Shibata created PARO for relax and stimulation. Robot seems to have its own character. PARO behaves like living animal turning to side from the voice comes, squeals when one strokes it, yells tremendously when one hits it and closes its eyes when one touches its mouth. It works everywhere where pets are forbidden like hospitals and senior's houses.
Source: Japan's National Institute of Advanced Industrial Science and Technology
Source: http://bioinstrumentation.mit.edu/
Source: www.antiterrorism.eu
In the spotlight
71
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N° 3
September 2007
Events autumn – winter 2007
4-7 September
AIM'07 – The 2007 IEEE/ASME International Conference on Advanced Intelligent Mechatronics, Zurich, Switzerland. http://aim2007.ethz.ch/
4-7 September
MESA'07 – 3. ASME/IEEE International Conference on Mechatronics and Embedded Systems and Applications, Las Vegas, Nevada, USA. http://www.asmemesa.org/mesa07/index.html
9-12 September
IEEE Region 8 EUROCON'07 Conference – Warsaw University of Technology, Warsaw, Poland. http://eurocon2007.isep.pw.edu.pl/
10-12 September ROBOCOMM'07 – The 1th International Conference on Robot Communication and Coordination, 2007, Athens, Greece. http://www.robocomm.org/
10-14 September ECAL'07 – 9. European Conference on Artificial Life, Lisbon, Portugal.: Workshop on Multiagent Systems in Biology and Robotics is included. http://www.ecal2007.org/
15-17 September Beijing, China. MIV '07 – The 7th WSEAS International Conference on Multimedia, Internet & Video Technologies. http://wseas.org/conferences/2007/china/miv/ SMO'07 – 7th WSEAS Int. Conf. on Simulation, Modelling & Optimalization http://wseas.org/conferences/2007/china/smo/ SSIP '07 – The 7th WSEAS International Conference on Signal, Speech and Image Processing http://www.wseas.org/conferences/2007/china/ssip
19-21 September ECMR 2007 – European Conference on Mobile Robots. Freiburg, Germany http://ecmr07.informatik.uni-freiburg.de/
24-26 September 20th International Conference on Parallel and Distributed Computing Systems. Las Vegas, Nevada, USA. http://www.isca-hq.org/ISCAconf.html
see next page >
72
Events
Journal of Automation, Mobile Robotics & Intelligent Systems
VOLUME 1,
N° 3
September 2007
25 October
AmiRE 2007 – 4th International Symposium on Autonomous Minirobots for Research and Edutainment. Buenos Aires, Argentina. http://www.amire2007.fit.qut.edu.au/
10-12 October
BVAI 2007 – 2nd International Symposium on Brain, Vision and Artificial intelligence. Naples, Italy. http://clava.cib.na.cnr.it/BVAI2007/
15-17 October
ROBOCOMM 2007 – 1st International Conference on Robot Communication and Coordination. Athens, Greece. http://www.robocomm.org/
17-20 October
ICCAS 2007 – International Conference on Control, Automation & Systems. Seoul, Korea. http://2007.iccas.org/
22-25 October
CORES'07 – 5th International Conference on Computer Recognition Systems. Wrocław, Poland. Special session on medical applications. http://cores.pwr.wroc.pl/
28-30 October
AUTONOMICS 2007 – First International Conference on Autonomic Computing and Communication Systems. Rome, Italy.
Events
73