Proc. of Int. Conf. on Advances in Computer Science 2010
Approximate Solution to Managing Uncertainty using Contribution and Agent Learning Jaya Sil Department of Computer Science and Technology Bengal Engineering and Science University, Shibpur Howrah-711103, India js@cs.becs.ac.in Abstract -The concept of contribution of agents has been introduced in the paper to handle impreciseness in problem domain. Initially, experts assign the agents to solve the tasks based on their qualifications while contribution of an agent with respect to a specific task is determined using belief measures. To solve an assigned task, agents communicate with others and the respondents are clustered using Kohonen’s network, resulting reallocation of agents. Progress of each task thus improves and using revised back propagation learning algorithm, approximate solution has been generated after one epoch. The system has been implemented using Java Socket programming technique in Linux platform. Index Terms—contribution, learning, coordination
I. INTRODUCTION A multi-agent system (MAS) [1,2] is an autonomous system consisting of multiple agents that collaborate and cooperate to accomplish a common goal. Designing of MAS [3] is extremely challenging in a imprecise domain where sharing of knowledge, improvement of skill through learning, managing available resources and communication are essential. RoboCup tournament [4] is used as a common, unified approach for AI research in general and multi-agent research [5] in particular. RoboCup initiative especially the simulation league has created interest among the researchers [6] but except a few cases it has not been reported in such an extent so that researchers can access the information and apply in other domains beyond RoboCup. M.Tembe et. al.[7] focus on two of the RoboCup research challenges – team-work and multi-agent learning and attempting to extract general lessons from their experience in building an agent team for RoboCup. But the model cannot recognize the individual team member dynamically who is unable to fulfill its role in the team. Holonic multiagent systems (HOLOMAS) [8] were rigorously applied to meeting stability, autonomy and co-operation issues quite long time back. Learning of agents, another important issue has been addressed by genetic programming through message communication to achieve group behavior [9,10]. Pheromone learning concept was also introduced in [11] for self-organizing the agents and modulate their behavior in a decentralized manner, suitable for managing resources and deadlines. However, in earlier works, problems of designing MAS were addressed in isolated ways, therefore lacks in system coordination. In the paper, an integrated approach has been proposed to handle uncertainty in the domain knowledge to achieve near exact or approximate solution. Contribution [12] of agent with respect to a © 2010 ACEEE DOI: 02.ACS.2010.01.36
197
particular task is introduced here and quantified using belief measures [13]. The problem is decomposed into different tasks and represented by if-then rules while agents are initially assigned by the experts for a specific task based on their qualification(s) forming number of modules. The contribution of assigned agents may not be sufficient and therefore, individual agent using a simple protocol communicates with others, described in the paper. Agents having similar qualifications are clustered using Kohonen’s competitive learning rule and representative of each clusters are reassigned. Progress of each task thus improves and using revised back propagation learning algorithm, approximate solution has been generated after one epoch. More exact solution might be achieved depending on the availability of specialized agents. The paper has been classified into five sections. In section II, formation of the modules, agent communication and computation of contribution of agents are described while in section III, learning scheme is explained. Application of the proposed method has been illustrated in section IV and finally conclusions are summarized in section V. II. PROBLEM FORMULATION A. Contribution of Agents Each problem consists of several modules having multiple tasks and agents while knowledge of the problem domain is encoded into if-then rules. The modules are connected forming a hierarchical tree structure depending on the flow of information for achieving the solution of the problem where root node represents goal of the problem. Contribution of the k-th agent CTk is calculated using (1), describes probability of the k-th agent to performing task T with qualification Qk={q1k , q2k , . . . qnk }; where qik (i= 1... n), the i-th qualification of the k-th agent denoted by the propositions of the rule, representing task T. CT k = P(TQk) = P(T, Qk) / P(Qk).
(1)
k
In (1), P(T, Q ) is the probability of joint occurrence of the events T and Qk while P(Qk) is calculated using (2). P(Qk) = Bel(Qk) = Bel(q1k∪q2k ∪…..∪qnk) n = ΣBel(qik ) - ΣBel(qik ∩qjk ) …+ (-1)n+1Bel(q1k∩q2k i=1 i<j (2) ∩…∩qnk )
Proc. of Int. Conf. on Advances in Computer Science 2010
where Bel(Qk) = degree of belief of the k-th agent with qualification Qk and Bel(qik) = belief measure of individual element of Qk. B. Agent Communication In the paper, modal operator K( , i) [14] is used as a basic communication unit, where indicates the sender agent and I, (i=1. . n) representing qualification of the sender agent. Agents communicate with each other by transferring modal operators using Java Socket Programming. If an agent cannot contribute sufficiently (contribution less than one) it broadcasts requests to other agents for cooperation. The agents possess at least one required qualification as mentioned in the modal operator respond. III. AGENT LEARNING A. Clustering of Agents In the paper, using Kohonen’s competitive learning rule [14] agents are clustered depending on their strength of qualification and representative of each cluster are reassigned to the tasks to achieve the respective sub-goals. The neural network consists of n number of input nodes representing various qualifications of the agents. Each component of input pattern vectors corresponding to belief value of a particular qualification of an agent. k= 1…p, the number of instances where k is the number of agents are to be clustered applied as input training pattern to the network. Each output node represents a particular task Tj (j=1 … r and r≤p), which may not be solved by the contribution of the initially assigned agents. During training, agents are clustered into different output nodes, which determine involvement of new agents for respective tasks. B. Progress of Tasks After selection of new agents to performing different tasks, structures of the modules are modified, that exhibits more interconnection between different modules. Since more agents are now involved to perform a particular task than earlier there might be improvement towards achieving the sub-goal. Back propagation learning algorithm [15] is applied to improve the progress of each sub-problem where number of nodes in the input layer equal to different qualifications (say, n) possessed by the total number of agents (say, p), number of hidden nodes equal to the number of tasks (say, r) for solving that sub-problem and output layer consists of a single node representing the subgoal, presented in Fig. 1. Therefore, p number of training patterns where each input pattern corresponds to belief value of qualification of each agent (qik, where i=1... n and k=1…p) and target value equal to one is applied to the network. During training, between input and hidden layer binary weights (vjn) are assigned corresponding to each input pattern vector. The weight is equal to one provided the particular qualification of the input pattern is required to performing the task, otherwise zero. If the particular qualification matches with at least one antecedent of the rule, representing the task that implies the qualification is © 2010 ACEEE DOI: 02.ACS.2010.01.36
198
required to perform the task. Ramp activation function (a), is used at the hidden layer to limit the output of the hidden node (zj) equal to one as described in (3). It is assumed that the weights between input and hidden layer are not updated but only assigned during training of each input pattern. Initial weight (w1r(0)) between hidden and output layer represents sum of initial contributions of the agents towards performing the task j for achieving the sub-goal. Bipolar sigmoidal activation function (b) is used at the output layer while output of the network (y(k)) for k-th training pattern is described by (4). After training, the error (e) at the output node is obtained using (5) and the progress of task T is computed using (6). zj =η a(f) = a (∑vji Qi)
i y(k) = ηb (g) = ηb(Σwj (0). zj) j e = ∑(1 - y(k))2 k wT (1) = wj (0) + e. zj
(3) (4) (5) (6)
where i=1.. n, j = 1...r, k= 1.... p and η is learning constant experimentally set to 0.6.
Fig. 1. Back Propagation Network
IV. APPLICATION Say, an automotive manufacturer might buy seats from one company, brake system from another, air conditioning from a third, electrical systems from a fourth and manufacture only the chassis, body and the powertrain in its own facilities. The suppliers of major subsystems in turn purchase much of their content from other companies. As a result, the production line that turns raw materials into a vehicle is represented by a hierarchical tree structure consisting of different modules each corresponds to different subsystems for achieving respective sub-goals. Each module can tailor its operation by decomposing the sub-problem into different functions represented as tasks depending on the specialty of the functional agents and opinion of the domain experts. The tasks are order acquisition, logistics, scheduling, resource, dispatching, transportation and plant management, which vary from module to module. The decomposition of a manufacturing enterprise and the decentralization of control results a natural domain for the use of agent technology. Say, seven specialized agents are initially assigned to execute seven different tasks individually. Assume agent A7 is initially assigned for performing the task, namely comfort of the seats in the manufacturing of seats
Proc. of Int. Conf. on Advances in Computer Science 2010
subsystem module. But the contribution of agent A7 is not sufficient for solving the task and so other agents are involved to execute the same task. Therefore, agent A7 broadcasts modal operator mentioning required qualifications to execute that task and receives respond from A3, A4, A5 and A1 agents. Say, Kohonon network consists of eleven input nodes (no. of qualifications of the agents) and seven output nodes (no. of tasks). Input patterns (belief measure of qualifications of the responding agents like A3, A4, A5 and A1) are applied to train the network and agent A4 and A1 are clustered at the output node representing the task comfort of the seats. After reassignment of the agents, strength of the tasks is computed using back propagation learning algorithm. For clarity the subsystem manufacturing of seats is considered only, which consists of a single task comfort of the seats. Therefore, eleven input nodes, one hidden node and one output node form the network. The sole connection weight between the hidden node and the output node is initialized by the initial contribution of agent A7 (say, 0.3) and learning rate is taken as 0.15 to train the network. Three training patterns corresponding to agent A1, A4 and A7 are applied and after one training epoch the algorithm terminates and the connection weight between the hidden and output node determines strength or progress of the task comfort of the seats. CONCLUSIONS In the paper the problem is represented using modular structure, which is easy to modify and fault tolerant. Contribution of agents with respect to task is quantified to measure the performance of the agent executing that task. Using unsupervised learning technique agents are clustered and reassigned. Back propagation learning algorithm is revised and measures progress of the work after one pass.
© 2010 ACEEE DOI: 02.ACS.2010.01.36
199
REFERENCES
[1] C. Boutilier, Multiagent Systems: Challenges and opportunities for decision-theoretic planning, AI Magazine, pp.35-43, 1999. [2] E.E. Mangina, S.D.J. McArthur and J.R. McDonald, COMMAS (condition monitoring multiagent systems), Journal of Autonomous Agents and Multiagent Systems, vol.. 4, No. 3, pp. 279-281, 2001. [3] N.R. Jennigs, K. Sycara, M. Wooldridge: A roadmap of agent research and development, Journal of Autonomous Agents and Multi-Agent Systems, No. 1 pp. 7-38, 1998. [4] H.S. Nwana and D.T. Ndumu, A perspective on software agents research, Knowledge Engineering Review, vol. 14, No. 2, pp. 125-142, 1999. [5] M. Asada, H. Kitano, I. Noda and M. Veloso, “RoboCup today and tomorrow-what we have learned,” AI, vol.. 110, No. 2 pp. 193-214, 1999. [6] V. Krishna and V.C. Ramesh, Intelligent Agents for negotiations in market games, Part I, IEEE Trans. on Power Systems, No. 13 , pp. 1103-1108, 1998. [7] M. Tambe, J. Adibi, Y. Alonaizon, A. Erdem, G. Kaminka, S. Marsella, and I. Muslea, Building Agent teams using an explicit teamwork model and learning. AI, vol. 110, pp. 215240, 1999. [8] A. Koestler, The ghost in the Machine; Hutchinson & Co., London, 1967. [9] Y. Shoham and K.L. Brown, Multi Agent Systems, Algorithmic, Game Theoretic and Logical Foundations, Cambridge University Press, 2009. [10] K.J. Mackin and E. Tazaki, Emergence of Group behavior in multiagent systems using self trained agent communication, IEEE Proceedings, pp. 2241-2244, 1999. [11] H.D. Parunak et. al., Pheromone learning for self-organizing agents, IEEE Trans. On SMC, Part A, No. 3, vol. 35, pp. 316-326, 2005 [12] J. Sil, Adaptive Agent Integration in Designing Object-Based Multiagent System, Lecture Notes in Computer Science, Springer Berlin / Heidelberg, ISSN: 0302-9743, No. 3215, pp. 24-30, 2004. [13] J. Pearl, Distributed revision of composite beliefs, AI, No. 33 pp. 173-213, 1987. [14] N.J. Nilson, Artificial Intelligence: A New Synthesis, Elsivier, SF, CA, 2000. [15] A. Bonarini, Evolutionary learning, Reinforcement learning, and fuzzy rules for knowledge acquisition in Agent-based systems, Proceedings of the IEEE, No. 89, vol. 9, pp. 13341345, 2001.