The Evolution of Expertise in Decision Support Technologies: A Challenge for Organizations

The Evolution of Expertise in Decision Support Technologies: A Challenge for Organizations Steve Diasio* & NĂşria Agell ESADE Business School- Barcelona GREC Research Group

* This research has been partially supported by the AURA research project (TIN2005-08873-C02), funded by the Spanish Ministry of Science and Information Technology and the Commission for Universities and Research of the Ministry of Innovation, Universities, and Enterprises of the Government of Catalonia.

Road Map • Introduction and Motivation • Terms and Concepts • Leveraging Expertise in Decision Support Technology – Expert Systems (ESs) – Group Decision Support Systems (GDSSs) – Collective Intelligence Tools (CI Tools) • Enhancing Decision-Making and CI Tools • Conclusions

Empty trading pit @ CBOT

Introduction and Motivation • Importance of decision-making • Researchers must evolve as technology changes • Computer Supported Cooperative Work (CSCW) provides a center point • An evolutionary view of three decision support technologies that support the use of expertise. Expert Systems

Group Decision Support Systems

Collective Intelligence Tools

Terms and Concepts What is Expertise?

• • • •

Multi-dimensional (Sternberg, 1997) with expert knowledge as the essential part (Tynjala, 1999) Short supply and difficult to represent Highly specialized or domain specific (Chi, Glaser, & Farr, 1988) Skills honed through practice (Jackson, 1999)

Perform consistently more accurate in relation to others (Hartely, 1985)

Formal Knowledge

•

Expert Knowledge Dimensions

Practical Knowledge

e it v la e u g edg e f-r owl l Se Kn

Expertise in Law Lawyer Expertise Practical Knowledge

Formal Knowledge •Factual knowledge •Learning of explicit information •In school or cases

Self-regulative Knowledge

•Intuition •Experience in legal setting •Tacit and difficult to express

•Reflective skill •Evaluation of action •Monitor argument and presentation to jury

Expertise by Means of Technology • Expertise not limited to humans • Technology built to capture knowledge or represent expertise (Barton, 1987; Liou & Nunamaker, 1990; Smith, 1994)

• Level of expertise can be augmented by increasing the amount of participants in the decision-making process

Expertise in Design Expert Systems GDSSs Collective Intelligence Tools

Number of People Level of Expertise in Systems Design

Leveraging Expertise Expert Systems Objective:

To represent expertise to its users for decision-making when a human expert can not be found or is in short supply.

Attributes:

Playing a critical role for organizations and are a source for competitive advantage (Gill, 1995).

Contributing to decision-making through their representation of knowledge and reasoning of human experts (Weiss & Kulikowski, 1984).

By mimicking and replicating the cognitive process of a human expert, novice users can be supported to perform as well as experts (Cascante et al, 2002).

ES are a technology that facilitates learning through the transfer of tacit and explicit knowledge (Yoon et al., 1995; Gregor & Benasat, 1999).

Leveraging Expertise Group Decision Support Systems Objective:

Attributes:

To capture the knowledge and contribution from the individual users to facilitate solutions to problems.

Occupies the center point for the aggregation of information and expertise from each participant.

Support the changing organizational structure, project based teams, dispersed workforce, and greater emphasis on collaboration.

Aided groups to deal with to the changing dynamics characterized by greater knowledge, complexity, and turbulence (Huber, 1982; 1984).

Shown to reduce time, costs (Gallup, 1985), foster collaboration, communication, deliberation, and negotiations (Kull, 1982).

What is Collective Intelligence?

• The collective judgment of group can predict or forecast better than experts or groups of experts (Surowiecki, 2004) • Diverging from traditional thought- high levels of expertise are the best source for decision-making • Including many people in decision-making by harnessing lower levels of expertise for peak solutions (Page, 2007)

Leveraging Expertise Collective Intelligence Tools Objective:

Attributes:

To facilitate the summative body of knowledge, information, and resources of its users.

Democratize decision-making by including many people in and outside the organization into the information gathering and decision-making process.

Prediction markets, incubates information scattered around the organization or network that allows nonexperts to produce expert like results.

Challenges traditional roles of experts, may change answer givers to inquiry mediators in effort to harness the knowledge of the masses in decision-making.

Offer an additional tool in decision-making.

Enhancing Decision-Making and CI Tools • Past attempts have made steps (Aiken et al. 1991; Turban & Watkins,

WellStructured

1986).

• Opportunities for system integration to solve a wider spectrum of problems.

• AI techniques to CI Tools – Transforming from passive to active agents – Intelligent components to increase participation – Managing interaction and collaboration between users

*Figure adapted from Aiken et al. 1991

ES

Problem Structure

CI Tools GDSS

DSS

IllStructured Few

Many

Group Size Supported

Decision Support Technologies *

Conclusions Shown An evolutionary perspective of expertise supported by decision support technologies.

Highlighted how organizational expertise in short supply can be augmented

Indicated how organizational use of expertise is changing which reflects the new roles of experts and non-experts in decision-making

Explored issues of design for integration with existing methodologies

Thank You! Steve Diasio & NĂşria Agell stephen.diasio; nuria.agell {@esade.edu} ESADE Business School- Barcelona GREC Research Group

References [1] M. Aiken, O. Liu Sheng and D. Vogel, “Integrating expert systems with group decision support systems”, ACM Transactions on Information Systems (TOIS) , Jan. 1991. (9)1, 75-95.

[2] L. Bañón and K. Schmidt, “CSCW: Four characters in search of a context”, Proceedings of the First European Conference on Computer-Supported Cooperative Work - EC-CSCW Gatwick, London, Sept., 1989, 358-372.

[3] A. Barton, “Experiences in Expert Systems”, The Journal of the Operational Research Society, The Practice of OR in Central Government Oct., 38(10), 1987, 965-974.

[4] J. Berg and T. Rietz, “Prediction Markets as Decision Support Systems”, Information Systems Frontiers, 5(1), 2003, 79-93.

[5] J. Berg, F. Nelson and T Rietz, “Accuracy and forecast standard error of prediction markets”, Working Paper, The University of Iowa, 2001.

[6] E. Bonabeau, “Decisions 2.0: The Power of Collective Intelligence”, MIT Sloan Management Review. Winter 50(2), 2009.

[7] B.G. Buchan, D. Barstow, R. Bechtel, J. Bennett, W. Clancey, C. Kulikowski, T. Mitchell and D.A. Waterman, “Constructing and Expert System”, Building Expert Systems, Hayes-Roth, F., Waterman, D.A. and Lenat, D.B. (eds), 1983. pp 127-167, Addison-Wesley, Reading, MA. [8] L. Cascante, M. Plaisent, P. Bernard and L. Maguiraga, “The impact of expert decision support systems on the performance of new employee”, Information Resource Management Journal; Oct-Dec 15(4), 2002, 67-78.

[9] M. T. H. Chi, R. Glaser, and M. J. Farr, eds., The Nature of Expertise, 1988. Hillsdale, NJ: Erlbaum.

[10] B. Cowgil, J. Wolfers, and E. Zitzewitz, “Using Prediction Markets to Track Information Flows: Evidence from Google”, 2008, Working Paper.

[11] K. Crowston and T. W. Malone, “Intelligent software agents”, Byte Magazine, Dec., 1988, 267-272.

[12] G. DeSantics and R. Gallup, “A Foundation for the Study of Group Decision Support Systems”. Management Science, 33(5), 1987, 589-609.

[13] S. Diasio, “Collaborative Fusion: Expert, Group, and Collective Intelligence in Decision-making Support”, unpublished paper, 2008.

[14] R. Dye, “The Promise of Prediction Markets: A Roundtable”. McKinsey Quarterly (2) 2008.

References [15] E.A Feigenbaum, “The Art of Artificial Intelligence: Themes and Case Studies of Knowledge Engineering’, International Joint Conference on Artificial Intelligence, 1977, 1014-1029.

[16] B. Gallup, “The impact of task difficulty on the use of a group decision support system”. Ph.D. dissertation, Dept. of Information and Decision Sciences, Univ. of Minnesota, 1985.

[17] M. J. Gannon, Organizational Behavior: A Managerial and Organizational Perspective, Boston: Little, Brown, 1979.

[18] T. Gill, “Early Expert Systems: Where are they now?”, MIS Quarterly. March, 1995.

[19] R. Hanson, “Decision Markets”, IEEE Intelligent Systems, 14, 1999, 16-19.

[20] R. Hartley, “Expert System Methodology: A Conceptual Analysis”, International Journal of Systems Research and Information Science, 1, 1985, 11-22.

[21] T. Ho, and K. Chen, (2007). “New Product Blockbusters: The Magic and Science of Prediction Markets”, California Management Review. Fall, 50(1), Berkeley, California, 2007.

[22] J. Howe, Crowdsourcing: Why the Power of the Crowd Is Driving the Future of Business, New York, Crown Business, 2008.

[23] G. P. Huber, “Group decision support systems as aids in the use of structured group management techniques”, In DDS-82 Conference Proceedings, 1982, 96-108.

[24] G. P. Huber, “Issues in the Design of Group Decision Support Systems”, MIS Quarterly, Sept. 8(3), 1984, 195- 204.

[25] P. Jackson, Introduction to Expert Systems, (3rd) ed, Pearson Addison Wesley, 1999.

[26] D. Kull, “Group decisions: Can a computer help?”, Computer Decisions, vol. 15(5), 1982, 64-70.

[27] Y. Liou, and J. Nunamaker, “Using a Group Decision Support System Environment for Knowledge Acquisition: A Field Study”, Proceedings of the ACM SIGBDP conference on Trends and directions in expert systems . Orlando, Florida, United States, 1990, 212 – 236.

[28] D. S. Murphy, and S. A. Yetmar, “Auditor evidence evaluation: Expert systems as credible sources”, Behavior and Information Technology, 15, 1996, 14-23.

[29] S. Page, The Difference: How the Power of Diversity Creates Better Groups, Firms, Schools, and Societies, Princeton University Press, 2007.

[30] E. H. Shortliffe, and B. Buchanan, “Rule-Based Expert Systems: The MYCIN Experiments of the Stanford Heuristic Programming Project”, 1984.

References [31] E. H. Shortliffe, “Computer-based medical consultation MYCIN”, American Elsevier, 1976.

[32] H. Simon, Administrative Behavior: A Study of Decision- Making Processes in Administrative Organizations , 4th ed.

The Free Press, 1997.

[33] B. Smith, Collective Intelligence in Computer-Based Collaboration , Hillsdale, NJ: Lawrence Erlbaum, 1994.

[34] R. J. Sternberg, “Cognitive conceptions of expertise”, In P.J. Feltovich. K. M. Ford and R. R. Hoffman. Expertise in context. Human and Machine, Menlo Park, CA: AAAI Press/ The MIT Press, 1997. 149-162.

[35] J. Surowiecki, The Wisdom of Crowds: Why the Many Are Smarter Than the Few and How Collective Wisdom Shapes Business, Economies, Societies and Nations , Little, Brown, 2004.

[36] P. Tynjala, “Towards expert knowledge? A comparison between a constructivist and a traditional learning environment in the university”, International Journal of Education Research. 31, 1999, 357-442.

[37] J.P. Shim, M. Warkentin, J.F. Courteny, D.J. Power, R. Sharda, and C. Carlsson, “Past, Present, and Future of Decision Support Technology”, Decision Support Systems, 33, 2002, 111-126.

[38] S. Weiss and C. A. Kulikowski, A Practical Guide to Designing Expert Systems , Chapman and Hall Ltd, 1984.

[39] S. E. White, J. E. Dittrich and J. R. Lang, “The Effects of Group Decision Making Process and Problem Situation Complexity on Implementation Attempts”, Administration Science Quarterly, 25(2), 1980, 428-440.

[40] J. Wolfers and E. Zitzewitz, “Prediction Markets in Theory and Practice”, In S. Durlauf and L. Blume (eds) The Palgrave Dictionary of Economics, McMillan, 2nd ed, 2006 [41] M. Woodridge and N.R. Jennings, “Intelligent Agents: Theory and Practice”, The Knowledge Engineering Review, 10(2), 1995, 115-152.

Terms and Concepts What is Expertise?

•

• •

Multi-dimensional (Sternberg, 1997) Highly specialize or domain specific (Chi, Glaser, & Farr, 1988) Skills honed through practice (Jackson, 1999) Perform consistently more accurate in relation to others (Hartely, 1985)

Formal Knowledge

•

Expertise Dimensions (Tynjala, 1999)

Practical Knowledge

e it v la e u g edg e f-r owl l Se Kn

Leveraging Expertise Expert Systems

GDSSs

Objective

To represent expertise to its users for decision-making when a human expert can not be found or is in short supply.

Playing a critical role for organizations and a source for competitive advantage (Gill, 1995).

Contributing to decision-making through their representation of knowledge and reasoning of human experts for its users (Weiss & Kulikowski, 1984).

CI Tools

By mimicking and replicating the cognitive process of a human expert novice users can be supported to perform as well as experts (Cascante et al, 2002).

Users of ESs learn faster for decision-making that would occur through development over time.

Conclusions • Showed an evolutionary perspective of expertise supported decision support technologies.

• Highlighted how organizational expertise in short supply can be augmented.

• Explored issues of design for integration with existing methodologies.

• Indicated how organizational use of expertise is changing which reflects the new roles of experts and non-experts in decision-making.

Leveraging Expertise Expert Systems

GDSSs

– Replicating the knowledge and reasoning of human expert (Cascante et al, 2002).

– To capture the knowledge and contributions from the individual users for group consensus and collaboration (DeSantis & Gallup, 1987).

CI Tools

– To facilitate the summative body of knowledge, information, and resources of its users and the premise that the collective judgment of many are better then individual experts or groups at predicting or problemsolving (Surowiecki, 2004; Page, 2007).

Box-Office Forecasting: An Analysis of Expert Systems and Collective Intelligence Tools

FILM INDUSTRY APPROACHES TO LEVERAGING EXPERTISE – New movies consisting of long production cycles, complicated logistics, and high production costs reinforce the need for correct and accurate forecasts for box office revenue.

– Box office success prior to movie production and release are ridden with high levels of uncertainty for customer demand (Sawhney & Eliashberg 1996, Eliashberg et al. 2000), which has lead to a high percentage of films failing; further emphasizing that film forecasting is largely nonscientific.

– With high financial stakes involved and significant failure rates it is understandable that movie studios and production houses are willing to invest in new methods to accurately predict box office success (Davenport & Harris, 2009).

Espagogix Ltd. a UK-based company

– Epagogix’s proprietary expert system, based on a neural network (algorithms) analysis, uses movie script attributes to correlate future success or failures on past movies produced.

– The expert system uses variables such as genre, star actors, technical effects, and release time to find statistical patterns to forecast financial success.

– In addition, to being able to forecast, Epagogix’s expert system is also able to make recommendations either to reduce costs based on production or increased opportunities for greater box off revenue success (Davenport & Harris, 2009).

Hollywood Stock Exchange (HSX)

– Hollywood Stock Exchange (HSX) (www.HSX.com) is an internet based stock exchange where participants trade shares of virtual stocks that bet on the future box office revenue of movies.

– Participants buy and sell shares of movie stocks (like a real stock market).

– Web, traders are able to make new predictions as they acquire new information, making trades reflect in real-time stock prices and predictions of the future market demand for the movie.

– The Web for combined with the collective knowledge of many has transformed the ambiguities and uncertainties of movie forecasting into a consumer interactive predictive market.