UnBBayes-MEBN: Comments on Implementing a Probabilistic Ontology Tool © IADIS 2008

UNBBAYES-MEBN: COMMENTS ON IMPLEMENTING A PROBABILISTIC ONTOLOGY TOOL Rommel N. Carvalho, Marcelo Ladeira, Laécio L. Santos, Shou Matsumoto Universidade de Brasília – Departamento de Ciência da Computação Brasília, DF - Brazil {rommel.carvalho, laecio, cardialfly}@gmail.com; mladeira@unb.br

Paulo C. G. Costa George Mason University – C4I Center Fairfax, VA - USA pcosta@gmu.edu

ABSTRACT The quest for principled approaches to represent and reason under uncertainty in the Semantic Web (SW) is a very active research subject. Recently, the World Wide Web Consortium (W3C) created the Uncertainty Reasoning for the World Wide Web Incubator Group - URW3-XG [Laskey, K.J. et al., 2007] to better define the challenge of reasoning with and representing uncertain information available through the World Wide Web and related WWW technologies. One of the most promising approaches is the use of a Bayesian framework to handle uncertainty in SW ontologies. Working within this approach, Costa [2005] proposed a probabilistic ontology language, denoted PR-OWL, to represent and to reason with probabilistic ontologies. PR-OWL language is based on MEBN – Multi-Entity Bayesian Network [Laskey & Mahoney, 1997; Laskey & Costa, 2005; Laskey, 2007], a formalism that brings together the expressiveness of first-order logic (FOL) and the inferential power of Bayesian Networks (BN) to support probabilistic reasoning. Since both MEBN and PR-OWL are still under development, there is no tool that implements MEBN/PR-OWL as a knowledge representation formalism and probabilistic reasoner. This paper discusses the technical problems encountered, as well as how they were addressed in such an implementation that is currently under development at the University of Brasilia, with technical support from the C4I Center at George Mason University.

KEYWORDS Multi-Entity Bayesian Network, Bayesian networks, probabilistic ontology Web, probabilistic reasoning, Semantic Web.

1. INTRODUCTION What was first thought to be a great advantage for decision makers is now becoming a bottleneck. The huge amount of information available due to the increasing connectivity across the globe, has been making its timely processing by humans into knowledge almost impossible. This information overload needs to be overcomed by IT techniques, enabling the jump from the “information technology revolution” to the “knowledge revolution”, a natural sequence predicted in Alvin Toffler’s The Third Wave [Toffler, 1980].

The “knowledge revolution” will be seen, in the future, as the phase where the arduous and manual task of identifying, accessing and utilizing the information was assigned successfully to computers, allowing human beings to change their focus from data to knowledge driven activities. However, how will IT accomplish such a daunting task? One widely shared response is that the solution lies with semantics.

Technologies for making semantic information explicit and computationally accessible are key to effective exploitation of data from disparate sources [Laskey et al, 2007]. In this context emerges the Semantic Web (SW) to define common formats for integration and combination of data drawn from diverse sources, and a language for recording how data relates to real world objects. Explicit semantics is essential for appropriate processing of syntactically identical but semantically different terms (e.g., “Washington” the President, the city, or the football team). Ontologies, or shared repositories of precisely defined concepts expressed in standardized languages, are a vital tool for enabling semantic interoperability among web

resources [Costa et al, 2006]. Yet, traditional ontology languages, such as OWL, a W3C Recommendation [Heflin, 2004; Patel-Schneider et al., 2004] can at best list multiple possible senses for a word such as “Washington,” with no ability to grade their relative plausibility within a given context. This is inadequate for an open world environment where incomplete information is the rule and plausible reasoning is required [Costa & Laskey, 2006].

To face that shortcoming, the W3C has recently created the URW3-XG [Laskey, K.J. et al., 2007]. The group’s mission is to better define the challenge of reasoning with and representing uncertain information available through the World Wide Web and related WWW technologies. BN [Pearl, 1988], as previously mentioned, is one of the most promising approaches to deal with uncertainty in the SW. As it provides a graphical, flexible means of parsimoniously expressing joint probability distributions (JPD) over many interrelated hypotheses. However, BN have some limitations in representational power that restricts its use in the SW such as the fact that the number of variables has to be known in advance and the lack of support to recursive definitions. In order to address these limitations, Costa (2005) proposed a First-Order Bayesian framework to probabilistic ontologies that provides a basis for representation and reasoning under uncertainty in the SW. This framework is based on a probabilistic ontology language, denoted PR-OWL [Costa, 2005; Costa et al., 2005; Costa & Laskey, 2006] which is build on MEBN, a formalism that brings together the expressiveness of FOL and the power of BN to support probabilistic reasoning.

As both MEBN and PR-OWL were proposed very recently and are still under development, there is no software that implements MEBN/PR-OWL as a knowledge representation formalism and probabilistic reasoner. This paper discusses the technical problems encountered, as well as how they were addressed in such an implementation that is being currently under development with the technical support from the C4I Center at George Mason University. To the best of our knowledge, that is the first implementation of MEBN logic and PR-OWL language available.

This paper is structured as follows. Section 2 describes the MEBN formalism, the PR-OWL language, and the major problems faced in implementing them. Section 3 discusses an approach to accomplish a PROWL and MEBN implementation that turn around those problems and presents the UML class diagrams resultant of the object modeling done.

2. ANALYSIS OF A FORMALISM TO HANDLE UNCERTAINTY IN SW This Section contain a MEBN and PR-OWL overview, a description of the basic concepts necessary to understand how to create an PR-OWL based model, and a report of some difficulties arising when implementing a MEBN/PR-OWL-based reasoner. Finally, we present an overall view of the algorithm available for generating the situation-specific Bayesian network – SSBN (SSBN is a normal BN generated from an MTheory to handle a specific situation), and its limitation for implementation.

In order to make easier the grasping of those concepts, the Star Trek example proposed by Costa (2005) is used. This example is based on the famous Paramount Star Trek™ series, which convey the missions of the U.S.S. Enterprise starship. The starship is equipped with various sensors that provide some information the commander needs to keep the starship out of danger, and MEBN is used to help him doing his job.

Similar to BN, MEBN uses directed acyclic graphs to specify JPDs for a collection of related random variables. MEBN logic extends ordinary Bayesian networks to provide full first-order expressive power, and also extends FOL to provide a means of specifying probability distributions over interpretations of first-order theories [Laskey, 2007]. With MEBN, the knowledge is represented as a set of MEBN fragments (MFrags, for short) organized as a MEBN Theory (MTheories, for short). Each MFrag represents probability information about a group of related random variables. Just as first-order logic extends propositional logic to provide an inner structure for sentences, MEBN theories extend ordinary Bayesian networks to provide an inner structure for random variables. Random variables in MEBN theories take arguments that refer to entities in the domain of application. A MEBN theory implicitly expresses a JPD over truth-values of sets of FOL sentences. Any sentence that can be expressed in first-order logic can be represented as a random variable in a MTheory. MEBN logic is modular and compositional. That is, probability distributions are specified locally over small groups of hypotheses and composed into globally consistent probability distributions over sets of hypotheses. As an example within the Star Trek domain, the predicate DangerToSelf(st, t) might represent the danger level for the starship designated by the variable st during the

specified time step designated by the variable t. To refer to the danger level for the starship U.S.S. Enterprise at 10pm, we would fill in values for st and t to obtain an instance of DangerToSelf(!USSEnterprise, !10pm) of the DangerToSelf(s, t) random variable. A given situation might involve any number of instances of the DangerToSelf(s, t) random variable, referring to different starships and/or different time steps.

The main elements in a MFrag are context, input and resident nodes (Figure 1). Context nodes are Boolean variables that represent conditions that have to be satisfied so that the probabilistic distribution of the resident nodes applies. Their possible values are: True (the condition is satisfied), False (the condition is not satisfied), and Absurd (a condition expression that does not make sense). Input nodes are variables that influence the probabilistic distribution of its child resident nodes, but their distributions are defined within their own MFrags. In other words, in a complete MTheory, every input node must be a resident node in another MFrag, where its probabilistic distribution is defined. Resident nodes have the local probabilistic distributions defined in that MFrag, including the probabilistic dependence on its parent values (that can be input or resident nodes).

The task of defining FOL sentences and evaluating them, as far as implementation goes, was not really the main concern in the MEBN and PR-OWL articles published so far. Nevertheless, it is a main subject in MEBN's implementation and some restrictions are natural to be made for performance.

Figure 1: Star Trek MFrags for background support

As in BNs, a local distribution maps configurations of values of the parents of a random variable instance to probability distributions on its possible values. When all ordinary variables in the parents set of a resident random variable term also appear in the resident term itself, as for the HarmPotential(st, t), StarshipClass(st), and CloakMode(st) random variables of the Starship MFrag of Figure 1, a local distribution can be specified simply by listing a probability distribution for the child random variable for each combination of values of the parent random variables. This classic situation does not hold when ordinary variables in a parent random variable do not appear in the child. In this case, there may be an arbitrary, possibly infinite number of instances of a parent for any given instance of the child. For example, in the same MFrag of Figure 1, if the zone where a starship is located is uncertain, the number of enemies and friends (ZoneEStarships(z) and ZoneFStarships(z)) in any zone it might be located is relevant to the distribution of the OpSpec(st) random variable. If time step t has previous time steps, then more than one distance (DistanceFromOwn(st, tprev)) must be evaluated, which makes the distance measured in all time steps relevant to the distribution of the DistFromOwn(st, tprev) random variable in time t. Thus, any number of instances of the ZoneEShips(z), ZoneFShips(z), and DistFromOwn(st, tprev) random variables might be relevant to the distributions of the OpSpec(st) and DistFromOwn(st, tprev) random variables in time step t. In this case, the local distribution for a random variable must specify how to combine influences from all relevant instances of its parents.That was the first problem to be addressed, as the original MEBN specification has no proposition on how to define probability distributions when they differ from the classical BN distributions definitions as explained above.

Devised as a wide-purpose logic, MEBN also does not guarantee type-safety by default, although it is easily implemented with the definitions of Type and IsA MFrag as seen in Figure 1. However, when implementing MEBN, it quickly became obvious this is a feature that must be built-in. Another non-defined feature in MEBN is the use of ordered types in a way they can be used in recursion. To implement that, the user would have to define a MFrag such as the TimeStep MFrag (Figure 1) for every single entity that would be used in recursion. This can be a manual, error prone, and tedious process for the user.

Before presenting PR-OWL language, we begin by defining a probabilistic ontology. A probabilistic ontology is defined in Costa [2005] as an explicit, formal knowledge representation that expresses knowledge about a domain of application. This includes: (i) types of entities that exist in the domain; (ii) properties of those entities; (iii) relationships among entities; (iv) processes and events that happen with those entities; (v) statistical regularities that characterize the domain; (vi) inconclusive, ambiguous, incomplete, unreliable, and dissonant knowledge related to entities of the domain; and (vii) uncertainty about all the above forms of knowledge. In this definition, the term entity refers to any concept (real or fictitious, concrete or abstract) that can be described and reasoned about within the domain.

PR-OWL is an extension that enables OWL ontologies to represent complex Bayesian probabilistic models in a way that is flexible enough to be used by diverse Bayesian probabilistic tools. That level of flexibility can only be achieved using the underlying semantics of first-order Bayesian logic, which is not a part of the standard OWL semantics and abstract syntax.

The ability to perform probabilistic reasoning with incomplete or uncertain information conveyed through an ontology is a major advantage of PR-OWL. However, it should be noted that in some cases solving a probabilistic query might be intractable or even undecidable. The W3C defined three different versions of the OWL language, imposing restrictions to make it less expressive but more decidable, while the first proposal of PR-OWL only refers to expressivity restrictions as a future possibility [Costa, 2005]. Therefore, implementing PR-OWL involved performing a trade-off between performance and expressivity that is specific to the tool being developed.

An overview of the general concepts involved in the definition of a MEBN Theory in PR-OWL is depicted in Figure 2. In this diagram, the ovals represent general classes, while arrows represent major relationships between classes. A probabilistic ontology must have at least one individual of class MTheory, which is a label linking a group of MFrags that collectively form a valid MEBN Theory. In actual PR-OWL syntax, this link is expressed via the object property hasMFrag (which is the inverse of object property isMFragIn).

Figure 3 shows an expanded version conveying the main elements of Figure 2, their subclasses, the secondary elements that are needed for representing a MEBN Theory and relationships that were necessary for expressing the complex structure of a Bayesian probabilistic model using OWL syntax.

Figure 2: Overview of a PR-OWL MEBN Theory Concepts

Although the scheme in Figure 3 shows all the elements needed to represent a complete MEBN Theory, in the actual tool some elements were not explicitly implemented, while others had to be modified in the object-oriented (OO) data structure that ensures full compatibility with the PR-OWL specification.

MEBN inference begins, as explained in [Costa & Laskey, 2006], when a query is posed to assess the degree of belief in a target random variable given a set of evidence random variables. We start with a generative MTheory, add a set of finding MFrags representing problem-specific information, and specify the target nodes for our query. The first step in MEBN inference is to construct a SSBN, which is a Bayesian network constructed by creating and combining instances of the MFrags in the generative MTheory. When each MFrag is instantiated, instances of its random variables are created to represent known background information, observed evidence, and queries of interest to the decision maker. If there are any random variables with undefined distributions, then the algorithm instantiates their respective home MFrags. The process of retrieving and instantiating MFrags continues until there are no remaining random variables

having either undefined distributions or unknown values. A SSBN may contain any number of instances of each MFrag, depending on the number of entities and their interrelationships. Next, a standard Bayesian network inference algorithm is applied. Finally, the answer to the query is obtained by inspecting the posterior probabilities of the target nodes.

Figure 3: PR-OWL ontology elements

The underlying logic that supports this algorithm is defined in [Laskey, 2007]. However, trade-offs between performance, decidability, and expressivity had to be made when implementing the original MEBN specification. In other words, the complexity of the logic and the fact that it is still in development imply that any implementation has to include alternative algorithms and optimizations to make a working, feasible tool.

3. AN APPROACH FOR IMPLEMENTING MEBN AND PR-OWL The first task at hand was to design the object-oriented (OO) MEBN data structure. Some elements were eliminated right away as their presence in the implementation was either not relevant or outside the scope of the tool (such as the exemplar variables described in [Laskey, 2007]). Initially, extra elements were included into the design to account for the algorithms’ inner logical tasks such as the finding MFrag, finding nodes and others. However, design choices such as the use of PowerLoom rendered much of these extra elements unnecessary, facilitating the implementation. That was possible because PowerLoom was capable of handling all finding information, as it will be explained later on. The main MEBN elements, as implemented in the tool, can be seen in Figure 4. There, MultiEntityBayesianNetwork (MEBN for short) is naturally a Network, and MFrag is part of a MEBN, even though it is not shown that a MEBN has a list of MFrags. The same is valid for all elements that have an association with name mFrag, meaning that MFrag has a list of MultiEntityNode and OrdinaryVariable, for instance. There are three types of nodes defined: context, input, and resident. The input and resident nodes have more specific definitions: generative and domain respectively. The ResidentNode is the only one that has a table associated, through the interface ITabledVariable. Another useful information is that the InputNode is an input instance of a BuiltInRV or a ResidentNode.

Another challenge was to design the built-in implementation of type and ordered type for recursion as shown in Figure 1. In Figure 5, every entity has a type and the implementation enforces a type-safe policy.

As for recursion, the design solution involves the ObjectEntityInstanceOrdereable, also in Figure 5, which ensures that a certain type, TimeStep for instance, has a total order for every instance created, enabling the assessment of the previous and next instances. Using this approach, implementing recursion was mostly reduced to allowing a node to be set as recursive and setting the stop condition by defining the last instance in the recursion for a given entity. This approach is currently being considered to be included as part of the PROWL specification.

Figure 4: Main MEBN elements design

Figure 5: Main MEBN Entity and Type design

MEBN, as a first-order Bayesian logic, poses the implementation challenge of how to evaluate FOL sentences. A difficult task on its own, the design option involved searching for an open-source application programming interface (API) to deal with it. The choice was PowerLoom, a Knowledge Representation and Reasoning (KR&R) tool (http://www.isi.edu/isd/LOOM/PowerLoom) that is a highly expressive, logic-based KR&R system with multiple built-in deductive reasoning capabilities including a query processor, a description classifier, and a context mechanism. It was developed at the University of Southern California as a successor to the successful Loom KR&R system, with Hans Chalupsy as the project leader. PowerLoom is now distributed in three options of open source licensing terms: GPL, LGPL, and Mozilla. Using PowerLoom in the UnBBayes-MEBN implementation forced the team to design the KnowledgeBase and PowerLoomKB to build, load, and save a knowledge base (KB) from a MTheory. It can load and save finding and generative information through its modules within the environment. Through the environment, the KBFacade and PowerLoomFacade make it easier within the MEBN implementation to verify whether an entity exists, to evaluate formulas, to search for findings, and to obtain other useful information about the KB. This design is shown in Figure 6.

For the same reasons governing the MEBN logic design, the original work on the PR-OWL language [Costa, 2005] does not contain a rigid specification for representing formulas that perform the dynamic definition of probability distributions of instantiated MFrag’s nodes. A feature that is especially important when the number of parents is unknown. However, the original PR-OWL proposal presents the pseudo code example, depicted in Figure 7, which was used as a basis for specifying conditional probabilistic table (CPT).

Figure 6: Knowledge Base and PowerLoom design

The formal definition of the pseudo code of Figure 7 was the grammar defined in Figure 8. Based on the specification of this grammar a complete new compiler was designed, featuring lexical, syntactic, and semantic analyzer. It also generates the intermediate code, which is used in the definition of a node's CPT when constructing a SSBN for a given context.

Figure 7: Example code to produce a CPT Figure 8: Grammar for the CPT pseudo code

Another contribution to PR-OWL made during the development of the tool was to include information on global exclusivity for a node's state. This is necessary in situations when only one finding is allowed for a specific node in a given state. For instance, in Starship Data MFrag of Figure 1, the IsOwnStarship(st) has the state True as possible for just one starship st. That is, the state True has globally exclusive with respect to the random variable IsOwnStarship(st). Global Exclusivity was accepted as a contribution by the PR-OWL team, and was inserted in PR-OWL version 1.05 (www.pr-owl.org).

For the generation of the SSBN for a given context, it was necessary to allow the entry of known entities and findings as shown in Figure 6. The first restriction made was to limit the query to just one resident node with its given parameters, DangerToSelf(!USSEnterprise, !10pm) for instance. The algorithm has the following general steps: (i) For a given entry in the form NODE(ENTITY-LIST), search for evidence in the KB. If there is a finding for this given entry, stop. If not, go to the next step.; (ii) Search for the resident node that has the name NODE and get its MFrag. Once NODE(RV-LIST) is found, verify if the type of ENTITYLIST is the same as RV-LIST.; (iii) Verify in the KB which context node refers to the RVs in RV-LIST, replacing its values by ENTITY-LIST. If any of them is false, mark the MFrag to use the default distribution.

Go to next step; (iv) If the context node in (iii) does not have a solution, create it as father of NODE.; (v) For

each father of NODE, go to step (i), replacing the RVs by the know entities (contained in the query or KB).; (vi) Create the NODE's CPT.; (vii) finish. As explained in section 2, if context nodes such as z=StarshipZone(st) cannot be evaluated, then all zones need to be taken in consideration, which is done in step (iv). However, in complex formulas this lack of information may have strong impacts in the performance of the algorithm, so the designed solution involves asking the user for more information. In the current implementation, if she doesn’t provide such information the algorithm will just halt. Another design option was to restrict memory usage in a way that a possible memory overload triggers a warning to the user and stops the algorithm. In step (iii), a design optimization over the algorithm in Laskey [2007], only the necessary context nodes for a given MFrag are evaluated, in contrast with the original approach of revising all the context nodes for that MFrag. For the sake of conciseness, although other optimization issues were considered in the implementation only the most relevant are listed here.

4. CONCLUSION This paper discusses the implementation of a framework and a Java API intend to support the domain modeling and the reasoning with probabilistic ontologies based on PR-OWL and MEBN formalism.

Actually, this implementation: a) has a graphical interface to facilitate the creation of probabilistic ontologies built on MFrags and MTheories, b) presents a MEBN CPT formula editor that can deal with previously unknown number of nodes, c) generates the correspondent SSBN of the MTheory, and d) has features to create, to load, and to save MEBN models in the PR-OWL file format. Probabilistic ontologies have a potential application use in the probabilistic Semantic Web field. Because of that, this research represents a contribution to the SW community and, more specifically, to the current work of the URW3-XG Incubator Group, created by the W3C to better define the challenge of reasoning with and representing uncertain information available through the World Wide Web and related WWW technologies. The framework and API will be distributed as a free/open software under GNU GPL license.

ACKNOWLEDGEMENT This research was partially supported by CAPES and CNPq, Brazilian research agencies.

REFERENCES Costa, P.C.G., 2005. Bayesian Semantics for the Semantic Web. Department of Systems Engineering and Operational Research, George Mason University (Ph.D. Thesis).

Costa, P.C.G. & Laskey, K.B. 2006. PR-OWL: A Framework for Probabilistic Ontologies. Proceedings of the Fourth International Conference on Formal Ontology in Information Systems. Baltimore, USA. Costa, P.C.G., Laskey, K.B. & Laskey, K.J., 2006. Probabilistic Ontologies for Efficient Resource Sharing in Semantic Web Services. Proceedings of the Second Workshop on Uncertainty Reasoning for the Semantic Web. Athens, USA. Costa, P.C.G., Laskey, K.B. & Laskey, K.J., 2005. PR-OWL: A Bayesian Ontology Language for the Semantic Web. Proceedings of the ISWC Workshop on Uncertainty Reasoning for the Semantic Web. Galway, Ireland.

Heflin, J., 2004. OWL Web Ontology Language - Use Cases And Requirements (W3C Recommendation). Available as www.w3.org/TR/2004/REC-webont-req-20040210.

Laskey, K.B., 2007. MEBN: A Language for First-Order Bayesian Knowledge Bases. Artificial Intelligence (to appear).

Currently available from The Volgenau School of Information Technology and Engineering, George Mason University, as http://ite.gmu.edu/~klaskey/papers/Laskey_MEBN_Logic.pdf Laskey, K.B. & Costa, P.C.G., 2005. Of Klingons and Starships: Bayesian Logic for the 23rd Century. Proceedings of the Twenty-first Conference Uncertainty in Artificial Intelligence (UAI -05). Edinburgh, Scotland, pp. 346-353.

Laskey, K.B., Costa, P.C.G., Wright, E.J. & Laskey, K.J., 2007. Probabilistic Ontology for Net-Centric Fusion.

Proceedings of the Tenth International Conference on Information Fusion. Quebec, Canada.

Laskey, K.B. & Mahoney, S., 1997. Network Fragments: Representing Knowledge for Construction Probabilistic Models. Proceedings of the Thirteenth Conference on Uncertainty in Artificial Intelligence (UAI-97). August 1-3, 1997, Brown University, Providence, Rhode Island, USA. Morgan Kaufmann 1997.

Laskey, K.J., Laskey, K.B. & Costa, P.C.G (eds.), 2007. Uncertainty Reasoning for the World Wide Web Incubator Group Charter (W3C Incubator Activity). Available as www.w3.org/2005/Incubator/urw3/charter.

Patel-Schneider, P.F., Hayes, P. & Horrocks, I., 2004. OWL Web Ontology Language - Semantics and Abstract Syntax (W3C Recommendation). Available as www.w3.org/TR/owl-semantics/ (2004, Feb 10) Pearl, J., 1988. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann Publishers, San Mateo, CA, USA. Toffler, A., 1980. The Third Wave, 1st edition. Morrow, New York, NY, USA.