Integrating Decision Support Systems: Expert, Group, and Collective Intelligence

Integrating Decision Support Systems: Expert, Group, and Collective Intelligence Steve Diasio* & NĂşria Agell ESADE Business School- Barcelona GREC Research Group

IC’ AI 09 Las Vegas, 2009

* This research has been partially supported by the AURA research project (TIN2005-08873-C02), funded by the Spanish Ministry of Science and Information Technology and the Commission for Universities and Research of the Ministry of Innovation, Universities, and Enterprises of the Government of Catalonia.

Road Map • • • •

Introduction and Motivation Framework for Integration Terms and Concepts Leveraging Expertise in Decision Support Technology – Expert Systems (ESs) – Group Decision Support Systems (GDSSs) – Collective Intelligence Tools (CI Tools) • Enhancing Decision-Making and CI Tools • Conclusions and Future Work

Empty trading pit @ CBOT

Introduction and Motivation •

Organizations today face a changing environment; – external conditions change rapidly (Ilinitch et al, 1996).

– organizational structures flat and dispersed (Malone, 2006).

– traditional roles of experts have been “squeezed” or of decreased importance (Mauboussin, 2008).

•

Today’s new environment places a premium on collaboration creating renewed interest in decision support technology to survive and remain competitive (Hamel & Breen, 2008).

• Information technology is playing an increasing role in facilitating a firm’s success and is woven thread in the fabric of the organization (Zammuto et al, 2007).

• The paper aims to understand how integration of expert systems (ESs), group decision support systems (GDSSs), and collective intelligence tools (CI tools) can enhance decision-making.

Framework for Integration • Abundance of decision support tools at their disposal.

• Tools have been independently built (Turban & Watkins, 1986) for individual problems but be flexible to adapt to the changing conditions and needs.

• Individually shown advantages of using such systems, however have not extended or offered in theory or practice an integrated system that supports organizational needs in expertise for decision-making.

Integrated System Expert Systems

GDSSs

Proposed integrated support system

CI Tools

Framework for Integration • Abundance of decision support tools at their disposal.

•

•

Tools have been independently built (Turban & Watkins, 1986) for individual problems but be flexible to adapt to the changing conditions and needs.

Individually shown advantages of using such systems, however have not extended or offered in theory or practice an integrated system that supports organizational needs in expertise for decision-making.

Integrated System

Expert System

GDSSs

Proposed integrated support system

CI Tools

Terms and Concepts What is Expertise?

• • • •

Multi-dimensional (Sternberg, 1997) with expert knowledge as the essential part (Tynjala, 1999) Short supply and difficult to represent Highly specialized or domain specific (Chi, Glaser, & Farr, 1988) Skills honed through practice (Jackson, 1999)

Perform consistently more accurate in relation to others (Hartely, 1985)

Formal Knowledge

•

Expert Knowledge Dimensions

Practical Knowledge

e it v la e u g edg e f-r owl l Se Kn

Expertise in Law Lawyer Expertise Practical Knowledge

Formal Knowledge •Factual knowledge •Learning of explicit information •In school or cases

Self-regulative Knowledge

•Intuition •Experience in legal setting •Tacit and difficult to express

•Reflective skill •Evaluation of action •Monitor argument and presentation to jury

Expertise by Means of Technology • Expertise not limited to humans • Technology built to capture knowledge or represent expertise (Barton, 1987; Liou & Nunamaker, 1990; Smith, 1994)

• Level of expertise can be augmented by increasing the amount of participants in the decision-making process

Expertise in Design Expert Systems GDSSs Collective Intelligence Tools

Number of People Level of Expertise in Systems Design

Leveraging Expertise Expert Systems Objective:

To represent expertise to its users for decision-making when a human expert can not be found or is in short supply.

Attributes:

Playing a critical role for organizations and are a source for competitive advantage (Gill, 1995).

Contributing to decision-making through their representation of knowledge and reasoning of human experts (Weiss & Kulikowski, 1984).

By mimicking and replicating the cognitive process of a human expert, novice users can be supported to perform as well as experts (Cascante et al, 2002).

ES are a technology that facilitates learning through the transfer of tacit and explicit knowledge (Yoon et al., 1995; Gregor & Benasat, 1999).

Leveraging Expertise Group Decision Support Systems Objective:

Attributes:

To capture the knowledge and contribution from the individual users to facilitate solutions to problems.

Occupies the center point for the aggregation of information and expertise from each participant.

Support the changing organizational structure, project based teams, dispersed workforce, and greater emphasis on collaboration.

Aided groups to deal with to the changing dynamics characterized by greater knowledge, complexity, and turbulence (Huber, 1982; 1984).

Shown to reduce time, costs (Gallup, 1985), foster collaboration, communication, deliberation, and negotiations (Kull, 1982).

What is Collective Intelligence?

• The collective judgment of group can predict or forecast better than experts or groups of experts (Surowiecki, 2004) • Diverging from traditional thought- high levels of expertise are the best source for decision-making • Including many people in decision-making by harnessing lower levels of expertise for peak solutions (Page, 2007)

Leveraging Expertise Collective Intelligence Tools Objective:

Attributes:

To facilitate the summative body of knowledge, information, and resources of its users.

Democratize decision-making by including many people in and outside the organization into the information gathering and decision-making process.

Prediction markets, incubates information scattered around the organization or network that allows nonexperts to produce expert like results.

Challenges traditional roles of experts, may change answer givers to inquiry mediators in effort to harness the knowledge of the masses in decision-making.

Offer an additional tool in decision-making.

Enhancing Decision-Making and CI Tools • Past attempts have made steps (Aiken et al. 1991; Turban & Watkins,

WellStructured

1986).

• Opportunities for system integration to solve a wider spectrum of problems.

• AI techniques to CI Tools – Transforming from passive to active agents – Intelligent components to increase participation – Managing interaction and collaboration between users

*Figure adapted from Aiken et al. 1991

ES

Problem Structure

CI Tools GDSS

DSS

IllStructured Few

Many

Group Size Supported

Decision Support Technologies *

Differences Between ES, GDSSs, CI Tools Attributes

ES

GDSS

CI TOOLS

Objective

Replicate or mimic human experts

Facilitate solutions for a group of people

To sum the knowledge and information of many people

Who makes the recommendation (decision)?

The system or heavily weighted if human is involved

The group and/ or systems through ranking

The System/ Tool

Major orientation (characteristic)

Transfer of expertise (humanmachine-human)

Build group consensus

Transfer of hard to find information or qualitative to quantitative data

Nature of support

Individual or group

Group

Individual or group

Problem area characteristic

Narrow domain

Semi/ Unstructured, broad

Limited variability

Type of problem treated

Repetitive

Unique/ not often / important

Forecasting/ dispersed collaborators/ Probabilistic

Reasoning capability

Yes (deduction)

No

Yes (depending on the tool (induction)

Assumptions

Closed-world

Limited to users boundaries

Changing

Expertise Level or In-depth knowledge of problem

Specific/ Expert Level

Dependent on task or problem

All levels including learning capacity with use

Figure 4 Differences between ES, GDSS, CI Tools Adapted from Aiken et al, 1991]

Conclusions Shown an evolutionary perspective of expertise supported by decision support technologies.

Highlighted how organizational expertise in short supply can be augmented

Indicated how organizational use of expertise is changing which reflects the new roles of experts and non-experts in decision-making

Explored issues of design for integration with existing decision support technology

Thank You! Steve Diasio & NĂşria Agell {stephen.diasio; nuria.agell} @esade.edu ESADE Business School- Barcelona GREC Research Group

References [1] M. Aiken, O. Liu Sheng and D. Vogel, “Integrating expert systems with group decision support systems”, ACM Transactions on Information Systems (TOIS) , Jan. 1991. (9)1, 75-95.

[2] L. Bañón and K. Schmidt, “CSCW: Four characters in search of a context”, Proceedings of the First European Conference on Computer-Supported Cooperative Work - EC-CSCW Gatwick, London, Sept., 1989, 358-372.

[3] A. Barton, “Experiences in Expert Systems”, The Journal of the Operational Research Society, The Practice of OR in Central Government Oct., 38(10), 1987, 965-974.

[4] J. Berg and T. Rietz, “Prediction Markets as Decision Support Systems”, Information Systems Frontiers, 5(1), 2003, 79-93.

[5] J. Berg, F. Nelson and T Rietz, “Accuracy and forecast standard error of prediction markets”, Working Paper, The University of Iowa, 2001.

[6] E. Bonabeau, “Decisions 2.0: The Power of Collective Intelligence”, MIT Sloan Management Review. Winter 50(2), 2009.

[7] B.G. Buchan, D. Barstow, R. Bechtel, J. Bennett, W. Clancey, C. Kulikowski, T. Mitchell and D.A. Waterman, “Constructing and Expert System”, Building Expert Systems, Hayes-Roth, F., Waterman, D.A. and Lenat, D.B. (eds), 1983. pp 127-167, Addison-Wesley, Reading, MA. [8] L. Cascante, M. Plaisent, P. Bernard and L. Maguiraga, “The impact of expert decision support systems on the performance of new employee”, Information Resource Management Journal; Oct-Dec 15(4), 2002, 67-78.

[9] M. T. H. Chi, R. Glaser, and M. J. Farr, eds., The Nature of Expertise, 1988. Hillsdale, NJ: Erlbaum.

[10] B. Cowgil, J. Wolfers, and E. Zitzewitz, “Using Prediction Markets to Track Information Flows: Evidence from Google”, 2008, Working Paper.

[11] K. Crowston and T. W. Malone, “Intelligent software agents”, Byte Magazine, Dec., 1988, 267-272.

[12] G. DeSantics and R. Gallup, “A Foundation for the Study of Group Decision Support Systems”. Management Science, 33(5), 1987, 589-609.

[13] S. Diasio, “Collaborative Fusion: Expert, Group, and Collective Intelligence in Decision-making Support”, unpublished paper, 2008.

[14] R. Dye, “The Promise of Prediction Markets: A Roundtable”. McKinsey Quarterly (2) 2008.

References [15] E.A Feigenbaum, “The Art of Artificial Intelligence: Themes and Case Studies of Knowledge Engineering’, International Joint Conference on Artificial Intelligence, 1977, 1014-1029.

[16] B. Gallup, “The impact of task difficulty on the use of a group decision support system”. Ph.D. dissertation, Dept. of Information and Decision Sciences, Univ. of Minnesota, 1985.

[17] M. J. Gannon, Organizational Behavior: A Managerial and Organizational Perspective, Boston: Little, Brown, 1979.

[18] T. Gill, “Early Expert Systems: Where are they now?”, MIS Quarterly. March, 1995.

[19] R. Hanson, “Decision Markets”, IEEE Intelligent Systems, 14, 1999, 16-19.

[20] R. Hartley, “Expert System Methodology: A Conceptual Analysis”, International Journal of Systems Research and Information Science, 1, 1985, 11-22.

[21] T. Ho, and K. Chen, (2007). “New Product Blockbusters: The Magic and Science of Prediction Markets”, California Management Review. Fall, 50(1), Berkeley, California, 2007.

[22] J. Howe, Crowdsourcing: Why the Power of the Crowd Is Driving the Future of Business, New York, Crown Business, 2008.

[23] G. P. Huber, “Group decision support systems as aids in the use of structured group management techniques”, In DDS-82 Conference Proceedings, 1982, 96-108.

[24] G. P. Huber, “Issues in the Design of Group Decision Support Systems”, MIS Quarterly, Sept. 8(3), 1984, 195- 204.

[25] P. Jackson, Introduction to Expert Systems, (3rd) ed, Pearson Addison Wesley, 1999.

[26] D. Kull, “Group decisions: Can a computer help?”, Computer Decisions, vol. 15(5), 1982, 64-70.

[27] Y. Liou, and J. Nunamaker, “Using a Group Decision Support System Environment for Knowledge Acquisition: A Field Study”, Proceedings of the ACM SIGBDP conference on Trends and directions in expert systems . Orlando, Florida, United States, 1990, 212 – 236.

[28] D. S. Murphy, and S. A. Yetmar, “Auditor evidence evaluation: Expert systems as credible sources”, Behavior and Information Technology, 15, 1996, 14-23.

[29] S. Page, The Difference: How the Power of Diversity Creates Better Groups, Firms, Schools, and Societies, Princeton University Press, 2007.

[30] E. H. Shortliffe, and B. Buchanan, “Rule-Based Expert Systems: The MYCIN Experiments of the Stanford Heuristic Programming Project”, 1984.

References [31] E. H. Shortliffe, “Computer-based medical consultation MYCIN”, American Elsevier, 1976.

[32] H. Simon, Administrative Behavior: A Study of Decision- Making Processes in Administrative Organizations , 4th ed.

The Free Press, 1997.

[33] B. Smith, Collective Intelligence in Computer-Based Collaboration , Hillsdale, NJ: Lawrence Erlbaum, 1994.

[34] R. J. Sternberg, “Cognitive conceptions of expertise”, In P.J. Feltovich. K. M. Ford and R. R. Hoffman. Expertise in context. Human and Machine, Menlo Park, CA: AAAI Press/ The MIT Press, 1997. 149-162.

[35] J. Surowiecki, The Wisdom of Crowds: Why the Many Are Smarter Than the Few and How Collective Wisdom Shapes Business, Economies, Societies and Nations , Little, Brown, 2004.

[36] P. Tynjala, “Towards expert knowledge? A comparison between a constructivist and a traditional learning environment in the university”, International Journal of Education Research. 31, 1999, 357-442.

[37] J.P. Shim, M. Warkentin, J.F. Courteny, D.J. Power, R. Sharda, and C. Carlsson, “Past, Present, and Future of Decision Support Technology”, Decision Support Systems, 33, 2002, 111-126.

[38] S. Weiss and C. A. Kulikowski, A Practical Guide to Designing Expert Systems , Chapman and Hall Ltd, 1984.

[39] S. E. White, J. E. Dittrich and J. R. Lang, “The Effects of Group Decision Making Process and Problem Situation Complexity on Implementation Attempts”, Administration Science Quarterly, 25(2), 1980, 428-440.

[40] J. Wolfers and E. Zitzewitz, “Prediction Markets in Theory and Practice”, In S. Durlauf and L. Blume (eds) The Palgrave Dictionary of Economics, McMillan, 2nd ed, 2006 [41] M. Woodridge and N.R. Jennings, “Intelligent Agents: Theory and Practice”, The Knowledge Engineering Review, 10(2), 1995, 115-152.