Scientific Journal of Control Engineering October 2014, Volume 4, Issue 5, PP.150-156
Model and Algorithm Research of Multi-Sensor Information Fusion Zhiliang Zhu 1,2#, Jing Hu 3, Yan Shen 1, Shaoming Chen 4 1. College of Electrical & Information Engineering, Hunan University, Changsha Hunan 410082, China 2. College of Physic & Electronic Information Engineering, Wenzhou University, Wenzhou Zhejiang 325000, China 3. School of Management, Zhejiang Industry and Trade Vocational College, Wenzhou Zhejiang 325000, China 4. Education Equipment Engineering Technology Research Center of Zhejiang Province, Wenzhou Zhejiang 325000, China #
Email: zlzhu@hnu.edu.cn
Abstract As the precondition of intelligent control for automatic system, multi-sensor information fusion technology is one of the key technologies of auto-control area. Aiming at the fusion model and algorithm, the research development of multi-sensor information fusion technology was well introduced and concluded in detail. Finally, the future research work tendencies were pointed out. Keywords: Multi-Sensor Information Fusion; Auto-Control
INTRODUCTION With the rapid development of microelectronic technology, signal detection and processing technology, computer technology, network communication technology and control technology, multi-sensor system for the complex application background appear in a large number. Because the information acquired by various kinds of sensors may have different characteristics: time-varying or time invariant, real-time or non-real time, fast or slowly varying, fuzzy or determined, accurate or not complete, reliable or unreliable, mutual support or complementary, also may conflict, the system must make full use of the multiple sensors information which is redundant and complementary in space and time according to a combination of criteria, in order to obtain consistency on the observation environment description and explanation. Multi-sensor information fusion refers to the integrated treatment of multiple sensors to provide united expression of the external environment. It is the fusion of the complementarity, redundancy, real-time and low cost of the information. So it can reflect the environmental characteristics completely and accurately, and helps to make the right judgments and decisions, ensures the speediness, accuracy and stability of the system. The ultimate objective of multi-sensor information fusion is to improve the performance of the whole system by using the common or joint operation advantage of multiple sensors. Multi-sensor information fusion technology will bring the following advantages to system[1]: (1) Improve the reliability and robustness of the system. (2) The coverage extension of time. (3) Extend monitoring range. (4) Enhance data credibility. (5) Reduce the reaction time. Information fusion function can be summarized as: expand the space search range, improve target detectability, improve detection performance; improve the spatial or temporal resolution, increase the dimension of feature vector of the target, reduce the uncertainty of information, improve the confidence of information system; enhance the ability of fault tolerance and adaptive of system; followed by the reduction of fuzzy degree reasoning and the - 150 http://www.sj-ce.org
improvement the decision-making ability, so that the performance of the whole system is greatly improved. Fundamentally speaking, the results derived from the redundancy and complementarity of information. Therefore, the multi-sensor information fusion can often get the results that the single sensor is difficult to obtain, and the performance will have a qualitative leap. From the principle of speaking, these ideas can be further performed to equipment, system integration. The fusion model and algorithm is an important research content in multi-sensor information fusion. Aiming at the model and algorithm, the research progress of multi-sensor information fusion technology are introduced and summarized in detail in this paper, and finally the prospects of the future development direction.
1 INFORMATION FUSION MODEL In recent years, people have proposed many kinds of information fusion model[2]. The common point or central idea is multistage processing in information fusion process. In 1980s, the typical functional model are Intelink, Boyd control loop (OODA loop); the typical data model is JDL. In 1990s the waterfall model and the Dasarathy model developped, Mark Bedworth integrated several model and proposed a new hybrid model in 1999.
1.1 Intelink UK Intelink describe the information processing as a ring structure[3] , as shown in figure 1. It consists of 4 stages: a) acquisition, including the initial intelligence data from sensors and artificial information source; b) sorting, association the relevant intelligence reports, some data consolidation and compression processing will be made at this stage, for use in the next stage of fusion; c) evaluation, in the stage of fusion and analysis of intelligence data, at the same time, analysts also directly delegate tasks of information gathering; d) distribution, in this sent the fusion of information to the user (usually military commander), in order to make the decision of action, includes the next step of acquisition work. Acquisition
Distribution
Arrangement
Assessment
FIGURE 1 UK INTELINK
1.2 JDL model In 1984, JDL model was proposed by the data fusion joint command laboratory of USA Department of Defense[4], after a gradual improvement and application, this model has become a standard of USA defense information fusion system. JDL model divided the data fusion into 3 stages: first stage as the target optimization, location and recognition; second stage for situation assessment, construction situation map according to the first levels of processing information; third stage for threat assessment, explains second stage treatment results according to the possible actions, and analyses the advantages and disadvantages of actions. Process optimization is a repeated process, and can be called the fourth stage, it will monitor the system performance in the fusion process, identify potential increased information sources.
1.3 Boyd control loop The Boyd control loop[5] (OODA) is first applied to the military command processing, now has been widely used in information fusion. It consists of 4 stages: first stage as observation, acquisition of target information; second stage for orientation, determine the direction, recognize situation; third stage for decision, making reaction the plan distribution behaviour , as well as logistics management and planning; fourth stage for action, implementation - 151 http://www.sj-ce.org
plan, and the decision making efficiency problem is considered in practical only in this stage. The Boyd control loop makes the problem of feedback iterative characteristics very obvious. The advantage of OODA model is that it enables each stage ring form a closed loop, show the cyclic data fusion. As can be seen, the data level transferring to next fusion stage decreasing along with the continuous fusion stage,. But the shortcoming of OODA model is the lack of capacity of influence to other stages by decision and action stage, and each stage is sequential execution.
1.4 Waterfall model The waterfall model is put forward in 1994 by Bedworth [6], and now is widely used in the British defense information fusion system; it emphasizes the function of the lower level, concrete frame as shown in figure 2. Its sensing and signal processing, feature extraction and pattern processing correspond to JDL level first, and the situation assessment and decision making respectively correspond to JDL level 2, 3, 4. As can be seen, although fusion process of waterfall model divided into the most detailed, but it was not clear feedback process, which should be regarded as its main shortcoming. Decision making Situation assessment Pattern processing Feature extraction
Signal processing Signal acquisition
FIGURE 2 WATERFALL MODEL
1.5 DASARATHY model The Dasarathy model includes 5 fusion levels [7]: TABLE 1 DASARATHY MODEL Input
Output
Description
Data
Data
Data level fusion
Data
Feature
Feature selection and extraction
Feature
Feature
Feature level fusion
Feature
Decision
Pattern recognition and processing
Decision
Decision
Decision level fusion
As shown in table 1: So as you can see, the waterfall model makes a clear distinction about the bottom function, while JDL model makes a clear division of middle functions, and the Boyd loop has a detailed explanation of the high level processing. Intelink covers all the processing level, but do not describe in detail. While the Dasarathy model is based on the fusion task or function to the construction, so it can effectively describe all fusion behavior.
1.6 Mixed model Mixed model integrated the cycle characteristics of Intelink and feedback iterative characteristics of Boyd control loop[8], and the definition of waterfall model is applicated, which is associated with JDL and Dasarathy model ,specific framework was shown in figure 3. The feedback can be seen clearly in a mixed model. The model retains the Boyd control loop to make the cycle characteristic of information fusion processing clear. The description of main processing tasks in the model has better precision of reproduction. In addition, the position of fusion behavior is also more easily find in the model. - 152 http://www.sj-ce.org
Decision making
Soft decision fusion
Hard decision fusion
Relationship management Decision Pattern processing Feature extraction
Control Dire ctio nal
Acti on
Resource allocation
Observation
Signal processing Sensor data fusion
Sensor
Sensor management
FIGURE 3 MIXED MODEL
2 INFORMATION FUSION ALGORITHM Fusion is a form of framework, whose goal is to integrate different source information to obtain high quality useful information by using mathematical methods and technical tools. Mathematical tools is the most basic and multiple function in information fusion, all input data will be effectively described in a public space, properly integrated at the same time, and finally output in a suitable form and the data. Commonly used methods of multi-sensor information fusion can basically be summarized as random and artificial intelligence, the application of these methods can perform data layer, feature layer and decision layer fusion in different levels, and accurately, fully understand and describe the measured object and environment. Can be foreknow, new concepts and technology such as neural network and artificial intelligence will play a more and more important role in multi-sensor information fusion. In the fusion technology and calculation method, the multi-sensor information fusion method mainly has: the weighted average, Calman filtering, Bayesian estimation, statistical decision theory, the Demspter-shafer evidence reasoning, production rule; while the calculation method mainly includes: the fuzzy set theory, neural network, rough set theory.
2.1 Weighted average The weighted average is the most simple and most intuitive method, which is suitable for dynamic environment. The method weighted average the redundant information provided by a group of sensors, and use value as the final result of fusion.
2.2 Calman filtering Calman filter is used for real-time fusion of low level redundant dynamic multi-sensor data. The method use the statistical properties of measurement model to recursive determine data fusion estimation of optimal statistical significance[9]. If the system dynamics model is linear, and the system noise and sensor noise is white noise model of Gauss distribution, provide the only optimal estimation of statistical significance for fusion data, the recursive characteristics of Calman filter make the system data processing does not need a large amount of data storage and computation.
2.3 Bayesian estimation - 153 http://www.sj-ce.org
Bayesian estimation is a commonly used method for multi-sensor information fusion in static environment of low layer. The information described as a probability distribution, suitable for uncertainty that has additive Gauss noise. When the sensor group has consistent observation coordinate, the sensor data can be fused by direct method. In most cases, multiple sensors describe the same object from different coordinate frames, then the sensor measurement data should be fused indirectly using Bayesian estimation. The problem of indirect method is to find the rotation matrix and translation vector which is consistent with multiple sensor readings[10].
2.4 Multiple Bayesian estimation Durrant Whyte express the task environment representation as multi-sensor system model of uncertain geometric object set, and put forward the multiple Bayesian estimation method[11]. Each sensor in the system is represented by the useful static description ability of these objects. Multiple Bayesian estimation take each sensor as a Bayesian estimation, combine the associated probability distribution of each individual object into a posterior probability distribution function, and provide the final value of multi-sensor information fusion by minimizing the likelihood function of joint distribution function.
2.5 Statistical decision theory Literature[12] using statistical decision theory (SDT) to propose two step generalized method for the fusion of multi-sensor redundancy positioning information. Sensor noise is modeled as a probability distribution of possible variety "e-Contaminated". The sensor model "e-Contaminated" is used to increase the robustness of the decision making process, by separating the distribution function to determine the separation coefficient of ￥, to represent heavy tailed deviations caused by possible untrue sensor readings. Compared with the multiple Bayesian estimation, uncertainty of statistical decision theory is additive noise, thus has uncertainty of wider adaptation. The observed data of different sensors must pass an integrated robust test for consistency; the data of consistency validate is fused by robust extreme decision rules.
2.6 Demspter-shafer evidence reasoning Dempster-Shafer evidence reasoning is extended Bayesian method[13]. In the Bayesian method, specifies the characteristics of all the lack information of the environment as an equivalent prior probability. When the number of useful additional information or unknown premise of sensor is greater than the number of known premise, the known premise probability becomes unstable, it is the obvious disadvantage of Bayesian method. In the Dempster-Shaller method, it can be avoided by don’t specify the prior probability of unknown precondition .
2.7 Fuzzy logic In multi-sensor system, information of the environment provided by each information source has a certain degree of uncertainty, the uncertain information fusion process is a process of uncertainty reasoning. Literature[14] using fuzzy logic to fuse image analysis and target recognition. Fuzzy logic is multiple valued logic, it allows direct representation the uncertainty in the multi-sensor information fusion process in reasoning process, by specifying a real between 0 to 1 to represent the truth degree, which is equivalent to the premise of implicit operator. If using a systematic modeling approach to integrate the uncertainty in the process, it will produce consistent fuzzy reasoning.
2.8 Neural network Neural network determine the classification standard based on similarity of current system accepted sample, this determination method mainly performance in the network weight distribution, and also can obtain knowledge to get the uncertainty reasoning mechanism by using a specific learning algorithm of neural network. Study of neural network provides a good method for multi-sensor integration and fusion modelling. Foreign scholars have done some pioneering work in multi-sensor integration and information fusion by using the neural network. Literature [15] proposed a fault-tolerant adaptive reconfiguration method when a sensor failure in multi-sensor system based on neural network. Multi-sensor integration and fusion based on neural network has the following characteristics: a unified internal knowledge representation, fusing network sensor information through learning method to obtaine - 154 http://www.sj-ce.org
network parameters (such as the connection matrix, node offset vector), and can convert knowledge rules to digital form; easy to establish knowledge base to use external environment information, easy to realize automatic knowledge acquisition and associative inference. The complex and uncertain environment will fused to accurate signal which system could understand through learning and reasoning.
3. SUMMARY Information fusion is the new direction of system science based on the cross, comprehensive and extension between modern information technology and multidisciplinary. Due to its broad application prospects in military and civil fields, it has been highly concerned by many domestic and foreign scholars and relevant departments, the system review of fusion system modeling, fusion algorithm, existing problems and the idea to solve these problems is given in this paper. Although the multi-sensor information fusion technology has been developed greatly at present, but there are still many problems: how to reduce the uncertainty of sensor information, reduce the error rate of information fusion, improve the real-time fusion, optimal information fusion algorithm, rational allocation of sensor resources. In addition, the multi-sensor information fusion method in dynamic environments is also a very meaningful research direction.
REFERENCES [1]
Pan Q, Yu X, Cheng Y M, et al. Essential methods and progress of information fusion theory[J]. Acta Automatica Sinica, 2003, 29(4): 599-615. (in Chinese)
[2]
Wu Xia, et al. Research Actualities and Problems on Multi-sensor Target Recognition System Model [J], Journal of Astronautics, 2010, 31(5): 1413-1420. (in Chinese)
[3]
Lin K, Chen M, Rodrigues J P C, et al. System Design and Data Fusion in Body Sensor Networks[J]. Telemedicine and E-Health Services, Policies, and Applications: Advancements and Developments, 2012, 1: 1-25
[4]
Cilla R, Patricio M A, Berlanga A, et al. A Data Fusion Perspective on Human Motion Analysis Including Multiple Camera Applications[M]//Natural and Artificial Computation in Engineering and Medical Applications. Springer Berlin Heidelberg, 2013: 149-158
[5]
Lan H, Pan Q, Yang F, et al. Design and implementation of algorithm-level testbed for multi-sensor information fusion[C]// Control Conference (CCC), 2013 32nd Chinese. IEEE, 2013: 8546-8551
[6]
Bedworth M, O'Brien J. The Omnibus model: a new model of data fusion?[J]. IEEE Aerospace and Electronic Systems Magazine, 2000, 15(4): 30-36
[7]
Faouzi N E E, Leung H, Kurian A. Data fusion in intelligent transportation systems: Progress and challenges–A survey[J]. Information Fusion, 2011, 12(1): 4-10
[8]
Hong Zhaoyi, Gao Xunzhang, Li Xiang. Research on Temporal-Spatial Information Fusion Model Based on DS theory[J]. Signal Processing, 2011, 27(001): 14-19. (in Chinese)
[9]
Cui P, Huang X. Multi-sensor information fusion algorithm based on federal Kalman filter and its application[J]. Electric Machines and Control, 2001, 5(3): 204-207
[10] Wang Yaonan, Li Shutao. Multi sensor information fusion and its application: A Survey[J]. Control And Decision, 2001, 16(5): 518-522.(in Chinese) [11] Jiang M L, Liu S Y. Fault Diagnosis Approach Study of Bayesian Networks Based on Multi-characteristic Information Fusion[J]. Zhongguo Jixie Gongcheng(China Mechanical Engineering), 2010, 21(8). (in Chinese) [12] Wang J L, Zhang J Y. Multisensor Target Identification Based on Mass Function of Statistical Evidence and D-S Evidence Theory [J]. Chinese Journal of Sensors and Actuators, 2006, 19(3): 862-864. (in Chinese) [13] Xu C F, Geng W D, et al. Review of Theory and Applications of Dempster—Shafer Evidential Reasoning Method [J]. Pattern Recognition and Artificial Intelligence, 1999, 12(4): 424-430. (in Chinese) [14] Wu X P, Zheng Z S, Fu Y. Fault diagnosis method based on fuzzy logic and evidence theory[J]. Journal of Naval University of Engineering, 2012, 1: 002. (in Chinese) [15] Chen M, Zhu W T, et al. The application of information fusion technology based on BP neural network in water monitoring[J]. Microcomputer Information, 2010 (10): 15-17. (in Chinese) - 155 http://www.sj-ce.org
AUTHORS 1
3
nationality. He obtained his Bachelor
nationality, He is a PhD student of
degree in Electrical Engineering and
Hunan university, and his research
Automation
of
interests focus on embedded systems,
Electronic Science and Technology of
machine learning and computer vision.
China (UESTC) in 2005; and Master in
He obtained his Bachelor degree in
Zhiliang Zhu (1982- ), male, the Han
from
University
Yan Shen (1985- ), male, the Han
Signal and Information Processing of Chongqing University of Posts and Telecommunications in
communication engineering from Hunan Normal University in 2008. Email: shenyan0712@hnu.edu.cn
2008. Currently, being a lecture of Wenzhou University, he is pursuing his Ph. D degree in Hunan University, China. His research interest focuses on the field of intelligent robot control. Email: zlzhu@hnu.edu.cn
- 156 http://www.sj-ce.org