Towards ethical governance of social machines

2013 IEEE Third International Conference on Cloud and Green Computing

Towards Ethical Governance of Social Machines Mark Hartswood, Barbara Grimpe and Marina Jirotka Computer Science, University of Oxford Oxford, UK. mark.hartswood@cs.ox.ac.uk; barbara.grimpe@cs.ox.ac.uk; marina.jirotka@cs.ox.ac.uk

Abstract— We introduce the concept of Hybrid Diversity – Aware Collective Adaptive Systems (HDA-CAS) and their proposed role in addressing social problems associated with urban living, health, and financial markets. Our concern is for their responsible development and deployment, and to this end, we suggest perspectives on the governance of social machines and a framework from which to design governance regimes for HDACAS.

II.

Keywords—governance; ethical governance; social machines; collective adaptive systems.

I.

INTRODUCTION

1

The SmartSociety EU funded Integrating Project (IP) is exploring innovative ways of supporting urban life through Collective Adaptive Systems (CAS) to solve problems around exploiting shared resources including traffic congestion, access to care, and community safety. In SmartSociety, representations of city life garnered from human and sensor observations allow citizens to adapt their actions to achieve community goals (such as reduced congestion or increased prosperity) and personal goals (enriched life experiences, improved care). The sorts of CAS envisaged are hybrid – they aim to achieve a symbiosis of humans and machine capabilities, and diversity aware – they are able to accommodate and/or exploit the different goals, expertise and values of participating agents. In HDA-CAS data from an instrumented city will feed multiple cycles of learning and adaptation. One goal of this is to better target incentives to subpopulations and thereby align their behaviour with the broader social goals of the CAS.

Hybrid CAS aim to exploit the blend of human and machine capabilities made possible by Social Machines. SmartSociety envisages tapping into human creativity, imagination and judgment to make sense of data from an instrumented city via participation in a serious game. We contend that ethical governance for Social Machines, including serious games, has inward and outward facing aspects. Inwardly, the way that the Social Machine is constituted needs to reflect the ethos of the computations (e.g. meeting standards of authorship in Wikipedia [11]), to offer reciprocity for participation and discourage malicious forms of participation. Outwardly, the role that Social Machines play within a wider societal context is important, e.g. preventing social machines from being put to malicious purposes such as their use by spammers to circumventing Captchas, or in promulgating exploitative labour practices [12].

Apart from urban living, CAS are likely to be increasingly relevant in various domains such as, health and financial markets [1,2]. CAS need to be built based on multidisciplinary approaches, including the social sciences, and ethics in particular [3]. Triggered by a series of recent EU initiatives and research projects [4,5] our role is to bring Responsible Research and Innovation practices [6] to a range of technologies, including HDA-CAS, so as to shape their impact upon privacy and other social values. An important aspect of this is to devise a framework for the ethical governance of HDA-CAS. This paper draws out a series of ethical issues for CAS derived from their Adaptive, Hybrid, Diversity Aware characteristics and considers governance responses.

CAS that are diversity-aware aim to be sensitive to the mix of capabilities and values present within the collective, and able to stratify populations accordingly to effectively target incentives and recruit expertise. However, such an approach is open to undesirable forms of social sorting, identified by Lyon as the ways that surveillance technologies sift populations and thereby regulate entitlement or access to resources [13].

Similarly internet and call centre technologies afford tailoring service quality and priority according to customer profiles, where ‘good’ customers can be apportioned early response or greater bandwidth [14].

1

SmartSociety (FP7/2007-2013) Grant agreement n. 600854. http://www.smart-society-project.eu/

978-0-7695-5114-2/13 $26.00 © 2013 IEEE DOI 10.1109/CGC.2013.73

ETHICS OF HDA-CAS

Adaptivity can have negative as well as positive connotations. Internet search (one of the commonest, most palpable encounters we have with a CAS-like social machine) is increasingly ethically and politically charged due to its powerful role in ordering our experience of the virtual world and its physical world referents. Attention has recently been drawn to how cultural assumptions can be covertly embedded within the mechanisms for ranking search results in ways that are opaque to users [7]. Moreover, searching contributes to everybody's search experience through subtle incremental adjustments to ranking algorithms resulting in ‘imprints’ that can be read as dominant cultural preferences. These can reflect negative sentiments including racial biases or prejudices [8].

Prior work examining the ethical principles important for developing and deploying algorithms embedded in computer models may prove a useful starting point for unpicking these issues, including relationships of responsibility between algorithm developers and their ultimate users [9], and the need for algorithms and embedded values to be made visible [10].

426

III.

ETHICAL GOVERNANCE OF CAS

to support mechanisms that can surface frictions and controversies and provide space and resources for them to emerge into evolving political processes and embedded community norms.

The issues raised above imply that CAS need to be governed effectively. Governance has been embedded in the financial and corporate worlds for some time and more recently explicit governance frameworks are emerging in response to sensitive areas of scientific research and innovation practice including geo-engineering and nano-technology [15]. Yet, while CAS are ways of creating social order where numerous heterogeneous actors meet and interact without any central authority in place, they are also prone to social disruption if values such as privacy and security are not managed well [3].

Given these and other potential social problems of the “risk society� [16] including large-scale “normal accidents� [17], we make a case for reconceptualising governance. Since CAS cross different levels of micro, meso and macro social organization (individual action; group interaction; collective aggregate behaviour), governance must be “re-scaled� [18] taking into account the characteristics of each level. Moreover, some nodes, or subsystems, in the CAS will have equal rights in the overall system (e.g. different major train stations of a big city), so the overall governance must be polycentric [19].

Above all, this re-scaled and polycentric governance must be extremely flexible, or responsive, according to the dynamic nature of CAS and their environments (e.g. cities are to be considered as urbanization processes, not container-like clearcut entities). Different time-scales are relevant, too: CAS must respond adequately to short-term user needs without discrimination (e.g. efficient parking directions for all participants in big city events) as well as to long-term goals of societal stability ensuring overall justice and welfare, but without compromising on desired evolution.

REFERENCES [1]

[2]

[3]

[4] [5]

[6]

[7] [8] [9] [10]

In addition, any new governance mode tailored to CAS must account for existing governance structures, which are socially persistent but change over time nevertheless. This concurrence of old and new is likely to provoke social conflict and to trigger unintended social consequences. For instance, any CAS that aims to have a transformative effect on the behaviour of urban populations might well lead to changes in city life in ways that short-circuit the usual planning function of local authorities. In this case, any aggrieved communities would find it hard to exert influence via the usual democratic channels. We expect that both the new and existing governance modes are “regimes of flows� of people, objects and information [20] that will interact in unpredictable ways, thus requiring some sort of time-sensitive meta-governance – that is extremely difficult to design a-priori. One approach may be to implement governance for CAS as one or a series of Social Machines that support collective social sense-making of emergent CAS behaviour [21], and collective approaches to policy formation and revision [11].

[11]

[12]

[13] [14]

[15]

[16] [17] [18] [19]

Given all these real-life challenges to CAS in practice we strongly advocate a multidisciplinary research programme to surface the socio-technical complexities and temporal dynamics of these new systems, and to conceive of appropriate, i.e. ethical governance measures without succumbing to the belief that there is any final perfect state of socio-technical order within and around CAS. On the contrary we would aim

[20] [21]

427

S. Bullock. Prospects for large-scale financial system simulation. The future of computer trading in financial markets - Foresight driver review DR 14. 2011. http://eprints.soton.ac.uk/272759/ (Accessed 9/8/2013).

D. Isern, D. Sanchez & A. Moreno. Agents applied in health care: A review. International Journal of Medical Informatics 79(3) 145-166, 2010.

Collective adaptive systems, expert consultation workshop report, 3 & 4 November.

European Commission, 2009. ftp://ftp.cordis.europa.eu/pub/fp7/ict/docs/fet-proactive/shapefetipwp2011-12-02_en.pdf (Accessed 9/8/2013) http://www.progressproject.eu/more-rri-resources/ (Accessed 9/8/2013) European Commission. Options for strengthening responsible research and innovations. Report of the expert group on the state of art in Europe on Responsible Research and Innovation. http://ec.europa.eu/research/sciencesociety/document_library/pdf_06/options-for-strengthening_en.pdf (Accessed 9/8/2013) B. C. Stahl, G. Eden, & M. Jirotka. Responsible research and innovation in information and communication technology - Identifying and engaging with the ethical implications of ICTs. In R. Owen, M. Heintz, & J. Bessant (Eds.), Responsible Innovation (pp. 199-218). Wiley. 2013.

L. D. Introna & H. F. Nissenbaum. Shaping the web: Why the politics of search engines matters. Information Society, 16(3) 2000.

L. Sweeney. Discrimination in online ad delivery. acmqueue. 11(3), 2013.

D. G. Johnson and J. M. Mulvey, Accountability and computer decision systems, Communications of the ACM 38(12) 58-64, 1995.

K. Fleischman & W. Wallace. Ensuring transparency in computational modeling. Communications of the ACM, 52(3) 131-134, 2009.

A. Aaltonen & L. G. Francesco, Governing social production in the Internet : The case of Wikipedia ' in The proceedings of 19th European Conference on Information Systems (ECIS), 2011.

M. S. Silberman, L. Irani and J. Ross,. Ethics and tactics of professional crowdwork. XRDS: Crossroads, The ACM Magazine for Students. 17(2) 39-43, 2010.

D. Lyon (ed). Surveillance as social sorting. Privacy, risk and digital discrimination. Routledge 2003. Ǥ ‘‘† ĆŹ Ǥ ”ƒŠƒÂ?Ǥ ‡”Â?‡ƒ„Ž‡ „‘—Â?†ƒ”‹‡• ‹Â? –Š‡ Â•Â‘ÂˆÂ–Â™ÂƒÂ”Â‡ÇŚÂ•Â‘Â”Â–Â‡Â† •‘…‹‡–›ǣ •—”˜‡‹ŽŽƒÂ?…‡ ƒÂ?† –Š‡ †‹ˆˆ‡”‡Â?–‹ƒ–‹‘Â? ‘ˆ Â?‘„‹Ž‹–›ǥ ‹Â?ÇŁ Ǥ Š‡ŽŽ‡” ĆŹ Ǥ ””› Č‹ Â†Â•ČŒÇĄ ‘„‹Ž‡ ‡…ŠÂ?‘Ž‘‰‹‡• ‘ˆ –Š‡ ‹–›Ǥ ‘Â?†‘Â?ÇŁ ‘—–Ž‡†‰‡ǥ ʹͲͲ͸Ǥ D. Barben: Anticipatory governance. In: David H. Guston (Eds): Encyclopedia of Nanoscience and Society, Vol. 1. Thousand Oaks, Cal. Sage, 17-18, 2010.

U. Beck. Risk Society. Sage. 1992.

C. Perrow. Normal sccidents: Living with high risk technologies.

Princeton University Press, Princetown, New Jersey, 1999.

K. Hannam. M. Sheller, & J. Urry. Mobilities, immobilities, and moorings. Mobilities 1(1) 1–22, 2006.

E. Ostrom, Beyond Markets and States: Polycentric Governance of Complex Economic Systems. The American Economic Review, 100(3) 641-672, 2010.

J. Law. Disaster in agriculture: or foot and mouth mobilities.

Environment and Planning A, 38, 277-239, 2006.

M. Hartswood, M. Wolters, J. Ure, S. Anderson and M. Jirotka. Sociomaterial design for computer mediated social sensemaking. In Proc. CHI Workshop on Explorations in Social Interaction Design, April 2013.