SmartSociety: Collaboration Between Humans and Machines, Promises and Perils

SmartSociety: Collaboration Between Humans and Machines, Promises and Perils Mark Hartswood(&) and Marina Jirotka Department of Computer Science, Oxford University, Oxford, UK {mark.hartswood,marina.jirotka}@cs.ox.ac.uk

Abstract. As the European Union (EU) funded SmartSociety project aims to create a toolset for rapidly and systematically engineering collective intelligence systems to support daily living, it simultaneously wants to ameliorate the risks to individuals of participating in these types of hyper-connected digital systems.

This paper reports on a panel session at the close at of the 2015 IFIP summer school that reflected upon a keynote speech covering SmartSociety concepts, technologies and ethical dilemmas. The panel session was conceived as a consultative exercise as part of the ongoing Responsible Research and Innovation (RRI) approach embedded within the SmartSociety project. In this chapter we present an analysis of the panel session discussion, which touched on several key issues, including the relationships between technology and society, what we should expect from a ‘SmartSociety’, barriers and horizons in managing ethical issues, and brokerage as a methodological approach to weaving multiple perspectives into design.

Keywords: SmartSociety Responsible Research and Innovation session Techno-social visions Brokerage

Panel

1 Introduction SmartSociety1 is a 4 year EU funded project that is at the forefront of how digital technologies are transforming our lives. SmartSociety builds on existing trends towards increasingly closely coupled systems of people, devices, data and algorithms designed to guide people in their everyday activities. SmartSociety aims to leverage collectives of people and machines to provide Smart City services in ways that satisfy individual goals while simultaneously tackling societal challenges such as sustainability. It welcomes the increasing interconnectedness of our physical and digital existences to support new ways for people to collectively solve problems by connecting them to remote pools of expertise and resources possessed by participating humans and machines [1].

SmartSociety aims to assist people in their everyday activities, while using the collective intelligence of the system as a whole to protect and preserve our shared resources. A traffic system enabled by SmartSociety would give advice to individuals on how to speedily complete their journey, while at the same time influencing overall flows of traffic to minimize pollution, reduce congestion, and reduce the impact of 1

http://www.smart-society-project.eu/.

© IFIP International Federation for Information Processing 2016 Published by Springer International Publishing Switzerland 2016. All Rights Reserved D. Aspinall et al. (Eds.): Privacy and Identity 2015, IFIP AICT 476, pp. 30–48, 2016.

DOI: 10.1007/978-3-319-41763-9_3

Smart Society: Collaboration Between Humans and Machines, PP

31

traffic incidents. Similarly, a SmartSociety tourism solution would utilize local knowledge to provide a customized experience for the individual user while at the same time smoothing the impact of tourism on the local infrastructure.

A SmartSociety keynote speech and panel session were included as part of the 2015 IFIP Summer School on Privacy and Identity Management2 with the aim of presenting the project’s vision and eliciting feedback from the privacy community concerning privacy and other ethical challenges that the project faces. This approach of engagement forms part of the Responsible Research and Innovation agenda within SmartSociety, where issues pertaining to privacy and other social values are addressed as an integral part of realizing the project’s goals. The panel session was recorded and transcribed so as the consortium could benefit fully from the discussion. An analysis of this transcript forms the main contribution of this chapter.

In this chapter we briefly introduce the emerging paradigm of Responsible Research and Innovation, and how this is being realized within SmartSociety. We recap some of the themes of SmartSociety that were featured in the keynote presentation under the rubric of the ‘Promises and Perils’ of a SmartSociety. Finally, we present an analysis of the panel session discussion, which touched on several key issues, including the relationships between technology and society, what we should expect from a ‘SmartSociety’, barriers and horizons in managing ethical issues, and brokerage as a methodological approach to weaving multiple perspectives into design.

2 Responsible Research and Innovation in SmartSociety Responsible Research and Innovation (RRI) seeks to open up innovation processes so that they incorporate a broader range of perspectives of the techno-social futures that innovation may bring about. RRI aims to achieve this by coordinating a varied range of multi-level activities3 undertaken by multiple actors at multiple points with the research /innovation lifecycles towards ensuring the outcomes of research and innovation are ‘socially acceptable’ and ‘socially beneficial’. These may include traditional ethics processes, risk assessments and foresight procedures, as well as more innovative activities tailored to specific domains or research streams that emphasize multi-stakeholder involvement [2].

RRI has been incorporated into funding bodies’ research agendas. For the EU it is an integral part of the H2020 programme both as standalone actions as well as being integrated within other research themes4. In the UK, the Engineering and Physics Sciences Research Council (EPSRC) expects (but does not mandate) that RRI activities be built into research processes, offering the AREA5 framework as guidance for

2 3

4 5

http://www.ifip-summerschool.org/.

E.g. in the planning, conduct and dissemination of research; during the formulation of research programmes and by policy makers in anticipating the regulatory requirements of innovation emerging from research. http://ec.europa.eu/programmes/horizon2020/en/h2020-section/responsible-research-innovation.

The AREA acronym stands for: Anticipate, Reflect, Engage and Act. https://www.epsrc.ac.uk/ research/framework/area/.

32

M. Hartswood and M. Jirotka

researchers wishing to pursue an RRI approach. While the stances of the EU and UK differ in detail, they both place considerable emphasis on consultation and engagement across a broad range of publics as a central pillar of RRI. An obvious quandary for RRI is how to decide which outcomes are actually socially beneficial - especially since innovation events themselves tend to alter our values and perspectives as a basis for judgment. Two proposals for solving this problem have been proposed. The first, promoted by von Schomberg, is to envisage RRI as encoding existing values enshrined within national or international charters, such as the European Union treaty, as providing “normative anchor points” for shared values such as the right to privacy, enhancing sustainability, promoting equality and so on [3]. The second is less prescriptive of specific values and focuses instead on opening up spaces for reflection and dialogue where processes of value formation (including elements of consensus and conflict) can be played out [4]. Our approach draws on both of these proposals by following Brey [5] in framing issues in relation to values of democracy, fairness, autonomy and privacy. At the same time we treat each of these categories as a ‘discursive space’ where the implications of a SmartSociety for a given social values can be explored. This blended approach works especially well, since there is actually no single version of, for example, democracy and democratic values.

Instead, many versions of democracy, democratic structures, processes and values are possible in different combinations that each has varying implications for participation and governance. Thus we may agree that democracy is important, but if we do so this is only a starting point for a more detailed conversation about what shape of democracy might be desirable or effective within Smart Societies.

To give a schematic overview of RRI, the following shows elements of RRI which have been proposed within the various sources cited above: • [Upstream] Starting early in the research and innovation process.

• Anticipating the transformations and impacts of new products and processes.

• Giving a voice to multiple publics and stakeholders to explore the consequences of research and its desirability.

• [Midstream] Being responsive to the dialogue.

• Adjusting the trajectory of research.

• [Downstream] Creating the right policy and regulatory environment for the technology to emerge into.

Responsible Research and Innovation (RRI) was built in as an integral feature of the SmartSociety. The premise was that existing SmartSociety-like systems are often ad hoc and not as powerful as they could be, or else organized and powerful, yet poorly adaptive and often seemingly socially irresponsible. Examples include the emergent and entrenched asymmetries in power over personal data obtained though use of digital services such as Google and Facebook [6]. SmartSociety aims to improve this situation by creating the tools to build more powerful systems that have considerations of ethics and social values ‘built in’.

We have developed a dedicated RRI procedure for SmartSociety that we are currently writing up for a separate publication. In essence, this process has four steps (1) Case studies and consultations to tease out issues with existing SmartSociety-type applications; (2) Synthesis of issues relevant to individual technical work-packages

Smart Society: Collaboration Between Humans and Machines, PP

33

within the SmartSociety into a series of challenge documents that lays out the case study context and outlines the challenge; (3) Facilitating the technical work-package to respond to the challenge and (4) Structuring the responses into a series of project wide design guidelines and operating procedures. The IFIP Summer School keynote speech and workshop reported here forms part of step 1 in the above process. That is to say it has been a means of consulting the privacy research community about the ethics, privacy and social values challenges posed by SmartSociety. In the following section we spell out our consultation approach in greater detail.

3 Consultation Approach Part of the RRI process within SmartSociety has been to undertake a series of consultative exercises with stakeholders in a variety of domains to create a tapestry of varied perspectives on the SmartSociety vision and possible consequences of SmartSociety technologies. The aim has not been to undertake a systematic and exhaustive consultation – mainly because SmartSociety technologies and ideas are still forming and still quite fluid, and may be applied across a diverse range of application areas – but rather to enrich the project with a series of external perspectives that may not otherwise feature endogenously.

In the closing sessions of the 2015 IFIP Summer School on Privacy and Identity Management, we were given an opportunity to present to the members of the privacy community represented at the workshop one version of the SmartSociety vision, and to elicit feedback from that community.

The following format was used. A keynote speech was given by the RRI researcher which covered the following topics: (a) An overview of SmartSociety; (b) an overview of RRI. This included the screening of a video created as part of an earlier project (FRRIICT) that demonstrates the issues for RRI of innovating in ICT6; (c) the SmartSociety vision as portrayed by a cartoon movie of imagined participants in a SmartSociety using SmartSociety services; and (d) a presentation of some of the presumed ethical and societal issues posed by the SmartSociety vision, which were framed in terms of ‘the perils and promises’ of a SmartSociety.

After a short break, the presentation was followed by a panel session in which panelists gave their response to the presentation, and thereafter addressed questions raised by members of the audience. Prior to the event, the consent of panelists and the audience was obtained to audio-record the panel session discussion. An analysis of the recording forms the basis of this chapter, which will be also used internally within SmartSociety to raise social values and ethics related issues posed by the panel and the audience.

The four members of the panel consisted of (a) the leader of the one of the SmartSociety technical WPs; (b) a Social Scientist and Policy consultant; (c) a computer scientist and privacy advocate and (d) one of the co-authors of the SmartSociety proposal.

6

https://www.youtube.com/channel/UCKITrA6PaVRkTsfdJtVP41w.

34

M. Hartswood and M. Jirotka

4 The Promise and Perils of a SmartSociety7 SmartSociety builds on seven technical work-packages each supplying a socio-technical component that contributes an important capability in order to realize a SmartSociety system as a functioning whole. These are: • Provenance and Trust Provenance is a data trail that supports audit. Trust mechanisms include reputations systems such as those commonly found on internet platforms.

• Sensor fusion Sensors in the environment, or worn by the user, are interpreted by computers to give computers access to a high-level description of what is happening within a SmartSociety system.

• Peer profiles Data stores for information about people who are participating in a SmartSociety system.

• Social Orchestration and algorithms Providing the mechanisms by which SmartSociety tasks are composed and the algorithms that support the activities of the participating collectives.

• Incentives How to deliver incentives within a SmartSociety system to make participating more attractive and to direct the actions of the collective.

• Programming framework To give the application programmer pro-gramming constructs that apply directly to social entities such as col-lectives.

• Platform The infrastructure that ties the technical elements of a SmartSociety system into a cohesive whole.

Many familiar social web platforms, or cyber-social systems, such as Amazon or Uber (a controversial Ride Sharing platform), already include the key SmartSociety elements of reputation, incentives, algorithms and collectives etc. The SmartSociety vision is to provide better engineered components so that ‘application developers’ can rapidly build these classes of systems at will. Two advances unique to SmartSociety assist the engineering approach. One is the abstraction and modularization of key social web functionalities – e.g. reputation (e.g. ratings and reviews), incentives (e.g. badges and other rewards), embedded sensor systems (such as those present in phones and Google Glass) and user profiles. The second comprises three new technologies for combining the above components into new types of application. The first is a ‘social orchestrator’, that allows the specification and enactment of social activities such as negotiation. The second is a ‘programming framework’, which includes programming primitives that can be used to invoke collectives of people and/or machines and to give them the resources they need to undertake some task. The third is a platform that knits all of the above elements together.

7

This section draws on a presentation authored by the following members of the SmartSociety consortium which is used with their permission. Michael Rovatsos (University of Edinburgh), Daniele Moirandi (U-Hopper), Vincenzo Maltese (University of Trento), Ronald Ronald Chenu-Abente (University of Trento), Alethia Hume (University of Trento).

Smart Society: Collaboration Between Humans and Machines, PP

35

In the SmartSociety keynote given at the IFIP workshop, the SmartSociety concept was demonstrated via a video8 that was created within the project to show off a vision for the ‘Ride Share’ system called ‘Smart Share’. Smart Share is an early demonstrator of SmartSociety capabilities, and ties together aspects of peer profiling, algorithms, reputation to enable collective sharing of individually owned resources – namely spare capacity in cars. Algorithms find matches between those wishing to travel. Peer profiles assert preferences to assist making a match. Reputation systems help establish trust.

Incentives and ‘gamification’ mechanisms, such as ‘badges’, are used to encourage involvement.

The keynote was adapted from a talk given at the ICT Days meeting in Trento in 20149. The aim, on that occasion, was also to be provocative and to spur discussion of the ethical issues relating to SmartSociety-like systems. The slides used for ICT Days were actually authored by technologists within the project, who were responding to their own interest in stimulating debate around ethical concerns.

The slides, aiming to be provocative, outlined the ‘promises’ and ‘perils’ of a future envisioned by SmartSociety. They achieved this by framing a series of utopias and dystopias – juxtaposing various promises of the SmartSociety project with various perils.

One example is the promise of collective intelligence to solve previously unsolvable problems, versus the peril of assimilation within a collective where autonomy is erased through totalizing mechanisms of automated influence. The slides employed illustrations and imagery from popular culture that play on such fears and dystopian possibilities, including, for example, a reference to the Star Trek entity ‘The Borg’, which assimilates individuals into its ‘hive mind’ collective with the slogan “Resistance is futile”.

Table 1. Privacy and governance Promise Control over personal data Being able to specify who can access data and for what purposes Anonymity Privacy enhancing technologies may provide various ways of acting anonymously, but accountably within a system Controlling disease Collective pooling of data may be of huge social benefit

8 9

Peril Useless information Incomplete and unrepresentative collective data sets because of opt-outs Unmanageable complexity Leading to increased technical, organizational (e.g. trusted third parties) and social complexity

All-knowing state Amassed data also has a huge surveillance potential and people may experience temptations to use it beyond its original purposes

https://www.youtube.com/watch?v=SAhWCaCsXrA. http://2015.ictdays.it/en.

36

M. Hartswood and M. Jirotka

The polarity of promise and peril was used as an alliterative device to high-light some of the issues, and should not be read as a perspective literally held by the slides authors’ on each of the issues raised. Tables 1, 2, 3 and 4 below showed the various promises and perils juxtaposed within the presentation.

Table 2. Augmented but self-determined users Promise Augmented users Benefiting from bodily monitoring and worn sensors Context-based services Providing advice based upon an interpretation of what a person is doing Proactively Acting on a person’s behalf in helpful ways

Peril Amplified data flows New flows of personal data streaming out of any conceivable situation Invasive or intrusive Misreading the context and interrupting at inopportune moments Wrong data interpretation Computers making mistakes that put people at a disadvantage

Table 3. Embedded algorithms Promise Man-machine collaboration Benefiting from machine intelligence that can check countless options Personalization Knowing a person well enough to give him/her what s/he wants Collective Intelligence Benefiting from the wisdom of the crowd

Peril Manipulation ‘Silently’ embedding agency and interests that shape a person’s actions or capacity to act Surveillance Knowing a person too well and acting inappropriately on that knowledge Humans as cheap labor Exploiting the cloud and undermining traditional labor rights

Table 4. Collective People-Machine Intelligence Promise Better health and care Critical mass for rare diseases or improved coordination of professional and informal care Smarter use of natural resources (energy, water) Better management of the commons via collective management Knowledge economy Greater independence and control over work

Peril Facebook replacing your social life Virtual venues impoverishing the physicality of conventional activities A terawatt of power to win Jeopardy Increasing energy consumption of pervasive ICT infrastructures Job destruction, eSlavery Erosion of careers and deterioration of working conditions

Smart Society: Collaboration Between Humans and Machines, PP

37

5 Interpretative Approach An anonymized transcript was made of the workshop recording as the basis for the analysis. A thematic analysis approach was used whereby emergent themes were identified and iteratively refined [7]. In addition, anonymized excerpts from the panel members’ statements were presented in a data session involving wider members of the research group so as to elicit further interpretations of the material. The workshop approach is close to a focus group format where insights emerge discursively through the interplay of perspectives and experiences of the participants, yet it differs in that there are elements of performance and staging, and an interrogative style, which are typically absent from focus groups. The analysis undertaken resembles a continuation of this process, picking out the emergent themes, but also intertwining further perspectives available to the researchers who have the benefit of a lengthier reflection unavailable to the panelists who responded on the spot. Hopefully it is clear in the discussion below where the researcher’s voice is more prominent. Hopefully, too, this voice is not perceived as being critical of participants or their views, but rather as taking those views as starting points for further deliberation. Inevitably, since the authors have a background in computer science, sociology and anthropology, as well as being deeply embedded within SmartSociety project as RRI researchers, their perspective on the issues raised in this chapter will reflect the perspectives, experiences and agendas of these a priori commitments.

6 Issues Raised: Managing Ethical Issues and Social Hazards of a SmartSociety This section details the issues raised by the panelists and the questions addressed to the panel by the workshop audience.

6.1

Panel Members’ Responses

Our four panelists included two men and two women. Female pronouns are used throughout this chapter to help hide panelist’s identities. Distinct themes can be attributed to each of the panelists’ responses. Panelist Tech (leader of a SmartSociety technical workpackage) raised the difficulty of providing guarantees that the interests of users are respected in SmartSociety-like systems. Panelist SocSci (a social scientist and policy consultant) drew attention to the ever-present dialogue around humans’ ambivalent relationship to technology. Panelist PrivAdv (a computer scientist advocating the importance of privacy) raised fundamental questions over what the vision for a ‘SmartSociety’ might be. Finally, panelist SmartSoc (one of the original authors of the SmartSociety proposal) provided a context for SmartSociety’s capacity to solve increasingly urgent social and economic problem around care. Therefore, two of the panelists were closely associated with the SmartSociety project, whereas the other two were not.

38

M. Hartswood and M. Jirotka

Below we consider each of these contributions in more detail.

Panelist Tech. Tech drew attention to the way that computer systems have a social effect, opening up a thread of discussion in this paper about the different ways that technology may stand in relation to social processes: “What we have seen in the [SmartSociety] project is that constantly there are – there’s a tension – between what you try and do with technology – because basically you want as much data – and you want to manipulate… you want to determine the outcome of social interactions with machine support so that you can introduce more intelligence into the system and help optimize – let’s say help solve the travel problem in a city. And on the other hand, of course, you have the privacy concerns – more than just privacy, I think it’s also accountability, transparency and governance – because what we’ve seen in our project is that all these algorithms essentially [they] all introduce biases.”

In the first part of the above quote - technology is conceived as an instrument to bring about certain social outcomes that are desired by the sponsor of the technology (perhaps tackling what are more broadly agreed concerns). The second part of the quote acknowledges that, at the same time, technology carries social dangers through unwanted side effects, which necessitate regulation and oversight. The dilemma outlined is that of wanting to use technology to do helpful things but, at the same time, introducing all sorts of new complexities, some of which have negative implications.

“And the question is, which of these irresistible services and temptations that the data world offers are you going to turn down – which social processes are you going to stay out of – what is the price you pay for that – socially”

Yet, building the appropriate values and safeguards into the system is hard, especially since people may be complicit in bringing about harmful side-effects, as it can be very difficult to resist data-based services and the social web, and the costs of opting out of these solutions may be high.

A first reading of these quotes suggests that they are describing a technology-led process directed by a technical elite that frames societal problems and enacts a vision of how computers can help. But we have to acknowledge that technologists are society members too (!) and bring their life-perspectives to bear and enact values that have currency and are shared more broadly in society, even if they may only be partially representative of the wider population. That is to say, RRI researchers should not make a knee-jerk assumption that the values held by technologists may not be more broadly shared. On the other hand, an RRI approach would advocate consultative processes to widen the perspectives that are drawn upon in creating new techno-social visions.

A second possibility is to read within these quotes an implicit distinction between technology on the one hand (which is prior to, and acts upon, society), and society on the other (which has to respond, or resist, or cope with technology). But this distinction is also hard to sustain because the technology itself emerges from existing ideas that form part of the cultural zeitgeist. This presumed distinction between technology / society turned up a number of times within the workshop dialogue and represents an important theme in this chapter.

Smart Society: Collaboration Between Humans and Machines, PP

39

Panelist SocSci. SocSci pointed out that becoming reconciled to technology has been an ever-present issue for humankind. She applauded the interdisciplinary approach within SmartSociety, and saw as valuable the internal dialogues that have been initiated in the project.

“We too [referring to the organization which she is affiliated] have worked with, if you want, instruments or approaches that have ranged from use of theatre, media, games, and have particularly placed an emphasis on dialogue – I’m think-ing about the ‘Court of Ethics’ which is a wonderful piece of drama, a play around robotics. This dialogue, about technology and society is as old as we are as human beings and I mean that as many ten thousands of years old … I’m pretty sure there might have been debates about the perils of fire versus the warmth and the cooking benefits that it could bring – it wouldn’t have surprised me if there were not a debate and dialogue as long ago as that.”

It is patently true that such an ongoing dialogue exists, and has existed across various epochs, as well as being enacted at different levels and locations throughout society, and through different cultural forms (including theatre, as indicated above). Dialogue is also a hugely important component of RRI as well as being a vital mechanism by which cultures anticipate and come to terms with new techno-social eras.

We can extend SocSci’s contribution to consider several further facets to dialogue that are important. One is that the presence of dialogue is not by itself a guarantor of beneficial outcomes, with much depending on how any given dialogue is geared into political processes. Another is that what is at stake, and what dialogues are possible and effective in any given historical epoch, may vary considerably. Habermas, for example, identified the coffee house culture of the Enlightenment as leading to the emergence of a ‘public sphere’, which had not existed in such a cohesive and egalitarian fashion prior to that moment in time [8]. A third is the hegemonic aspects of dialogue that determine the legitimacy of content, participants and venues within particular dialogical spheres.

Fourth and finally, there is an issue of the implied separation between “technology and society” which carries over from the previous panelist, Tech. We might want to keep in mind how technology itself underpins and transforms dialogic possibilities, with the social web as an example of this.

Panelist PrivAdv. This panelist questioned more fundamentally the presumptions underpinning the concept of a SmartSociety, and posited an alternative set of values and meanings that may be attached to being ‘Smart’. Do our technologies that promote convenience make us Smarter – or does ‘Smartness’ come instead from a focus on empowerment, education, participation in decisions and the capacity to opt out of the presumed benefits of digital living? Actually, the smartness of society may be reduced through technology, but improved through education and thoughtfulness: “So Smart is a buzzword, coming from SmartCards, SmartCars, Smart-Everything, and usually what used to be intelligent, or networked, or computer-aided or -supported – but what really is needed I think is a SmartSociety – smart people, educated people – and people who don’t have to rely on technology.”

“So is SmartSociety now really something that is [heading?] to educated people who can join the discourse about SmartSociety, for example, who are better off than before, but not only

40

M. Hartswood and M. Jirotka because of convenience but also [because] of participating in decisions but also opting out, or refraining from cooperating with others?”

Interestingly, when these views were further discussed within our research group (as part of this analysis), then opinion was divided between those who were more sympathetic to the panelist’s position, and those who more wholeheartedly embraced the role of technology to help people to collectively share resources and solve problems.

Those viewing the SmartSociety ambition in a positive light also rejected commonly suggested negative trajectories, such as job erosion or loss of autonomy. Discussion around the SmartSociety vision evidently contributes to a broader debate over how, collectively, we identify and wish to tackle contemporary social problems, and what forms of living we aspire to. Thus the panelist provides a helpful challenge to SmartSociety and asserts values that SmartSociety represents less well. This question of what values the SmartSociety project upholds is another recurrent one within this paper.

This panelist, picking up on the project’s focus on SmartCities, also identified presumed values within the SmartSociety project about what is desirable about participating in city life – for example, that everyone should be enthusiastically sociable in every situation: “So I come from (place) and we are said to be very stubborn. I think that (place) should be the same. (It’s not, I don’t know.). Where it is, for many people, fully ok to live on your island and not to interact with so many people. Those who you are choosing to interact with, they are your really good friends really. So it takes a longer time, but then you choose, these are my small group of people I want to interact with, and not I’m choosing from a big list of people who want to talk about Jazz all the time in the car. Although it might be my interest to meet people at some part of the time, but not this is perhaps (the) first thing (that I wish to do). I don’t think that (people from place) and others here are anti-social and don’t want to socialize – it’s more that there should also be the opportunity not be part of a SmartSociety which forces you to play according to those rules.”

Our interpretation of these comments is that PrivAdv is not referring to the privacy aspects of being sociable per se, but rather the presumption (visible from the keynote presentation) within SmartSociety solutions that sociability is always a desirable quality.

One concrete way these comments have been figured into the project is within a ‘tourism scenario’ that is under development as part of demonstration of SmartSociety ideas and technologies. We have pointed out that tourists are a diverse group with some wishing to solitary, as opposed to sociable, experiences. These perspectives have also contributed to internal debates within the project concerning diversity, and in particular how far SmartSociety systems can cater for diverse sensibilities.

This panelist questioned the how Smart Society may interfere with the autonomy of its participants by the sorts mechanisms proposed to shape the behavior of participants towards global objectives: “for example, that I can go through Edinburgh without having a navigation that not only tells me where – what’s the quickest way, but also, which already anticipates I where I am going or that I should go – or where I get the best vouchers for the 100th ride or so – so the incentives – the persuasion – in a direction where many people probably want to take it don’t think that this may be manipulation – so not personalization but manipulation. And if you think about the interests and incentives of the stakeholders – big companies – their incentive has to be to maximize the money they make out of that. This is the reason for their existence. If they don’t

Smart Society: Collaboration Between Humans and Machines, PP

41

think about that, but only about how to empower people – smart people – making people Smart that would be a different story – however, that’s not the task of the company”

Usefully she highlights the importance of ownership of a SmartSociety application, and how ownership plays a strong role in dictating whose interests it ultimately serves, and how the mechanisms of the platform may be the (perhaps silent) bearers of those interests. These comments have assisted us to develop a framework where we interpret hybrid systems (i.e. ones involving people and machines) by analyzing of how interests are represented and balanced within the system.

Panelist SmartSoc. This panelist re-grounded SmartSociety by proposing the serious role of counter-balancing the effect in the West of ever rising demands upon already overstretched care and medical systems. She suggested that SmartSociety-inspired approaches could increase the capacity of care systems by supporting local collectives of lay and professional carers, whilst simultaneously delivering improved outcomes for patients. Moreover, she contends that privacy and ethical issues are more easily solved when contained within local communities, but presumes that data can be contained at this level: “So, for example, in Sweden at the moment there are some hospitals where people with kidney disease – they go in, and they self-dialyze. They go in and they connect themselves up to the machines, and the dialysis work gets done when they want it to rather than when the hospital system wants it to. What we see when that happens is there is a huge amount of sharing of information be-cause there’s a local context – the collective around self-dialysis – where the sharing of information, and the information adapts and changes to the circum-stances. What you also see is a huge reduction in cost to the hospital – 50 % less money needs spent on doing that dialysis – and the number of infections goes down – the number of infections and errors is much smaller in the self-dialysis community than in the professionally dialyzed community.”

“And for me, at any rate, when I start talking about the big story of privacy, for example, and data protection and so on – I think top-down is just foolish. Particularly for these kinds of situation. You have to do it bottom-up. And these things can be built in from the bottom. And we have to understand, actually, a much more radical notion of what privacy by design is because actually the governance model, the local governance model, for how that information is interpreted and understood, has to be built into the development process – has to be built into the developer culture and the user culture, and those two overlap, and are built in from the bottom.”

In the second quote the panelist is alluding to how ‘top-down governance’, such as via the application of privacy laws, still somehow fails to protect people in how their data is used by large institutions such as governments and corporations. The ‘bottom-up’ model that is advocated instead posits community ownership and the co-design of technologies to secure data locally within these ‘data communities’.

However, this is an artful re-framing of the SmartSociety vision that circumvents some of the issues raised or hinted at by prior speakers. In this version of a SmartSociety, factors previously seen as matters of preference and privilege (deciding to be sociable or not; having time and resources to be a tourist) are replaced in this new context by matters characterized by hard choices and necessity. By positioning SmartSociety within would generally be considered a worthy application context, and as solving a serious societal problem, it becomes much harder to formulate a critique. One way of regaining a critical

42

M. Hartswood and M. Jirotka

stance is to point at how technological approaches to dealing with issues of care and caring often focus too narrowly on the mechanics of care, as opposed to the emotional, spiritual or social needs of a dependent person, e.g. [9].

This panelist ‘solved’ some of the ethical and privacy concerns posed within a SmartSociety by including elements of co-production, and elements of community and individual control over personal data. Although this seems like an important strategy, it perhaps has the disadvantage that it repositions SmartSociety further away from the grander vision contained within the project’s original grant proposal – which actually does propose ‘internet-scale’ systems. Perhaps many of the difficulties attached to SmartSociety arise because of the scale of the systems involved. While co-production, and minimizing scale and scope, render the issues more manageable, in doing so does the original SmartSociety vision remain intact?

6.2

Questions to the Panel

Following the panelists’ responses roughly eight questions were put to the panel by audience members. Instead of going through each question sequentially, as we have done for the panelists’ individual contributions, this section pulls out some recurrent themes that were identified in the questions and the panelists’ responses.

In the main, questions focused less on SmartSociety and its facets, but more on general issues around ICT that have a social impact, and the difficulties inherent in tackling these issues. In the following sections we consider the problems the audience and panelists described with identifying and understanding issues and solutions, anticipating the problems that new technologies may bring about, and in embedding known solutions into policy processes. We then explore how one panelist’s suggestion of brokerage provides a possible solution to these issues.

Horizons and Barriers. On several occasions discussion referred to several types of horizons and barriers that were perceived to constrain how the ethical social issues raised by SmartSociety may be addressed. ‘Horizons’ is taken to mean some limit to appreciation or perception form a given perspective – such as how far into the future a person can see, or how far their knowledge extends. One such horizon related to limits to our understanding: “What happens where the functions of a SmartSociety [are] so complex they are beyond the comprehension of most people? People are ‘not smart enough’ to make sense of the complex systems in which they are enmeshed.”

“I think we have a fairly poor understanding of the informal processes that go on in these communities - and turning them into something that we can actively support with technology…” [Responding to an audience asking about horizontal processes that connect disparate communities.]

The first quote above (from an audience member) expresses a concern that people may not understand the technical systems in which they participate, implying that the future may be only comprehensible to a small number of technical elites. The second quote (from a technologist) expresses the reciprocal concern that technologists

Smart Society: Collaboration Between Humans and Machines, PP

43

themselves may lack knowledge of social processes from the perspective of being able to develop technologies to support those processes. Taken together, these statements raise the specter of technology proceeding without a full social understanding to create something that members of society themselves cannot comprehend. These concerns resonate with the earlier theme of a supposed separation between technology and society. Yet we know that technology and society are not separate in the ways that some of the panelists may have been construed as implying. As new socio-technical systems emerge from a given socio-technical milieu, then both technologists and non-technologists will already have some grasp of each other’s perspective, even if initially these shared perspectives may be limited. While horizons may exist at specific moments, over time, and through processes of social learning, the technology and its deeper historical roots or distant locales of production often become even more comprehensible, either through experience, education, through voluntary disclosure, or via processes of investigation and revelation. Similarly, there are very well established design approaches for opening up understandings of social processes and feeding them into system development, which are able to extend the designers’ horizon of how certain social processes function.

Another is horizon is the future and our ability to anticipate outcomes: “Lawmakers are in a much more difficult position … because this is information technology and it applies to all kinds of applications - it’s very difficult for them to anticipate the consequences.”

“Basically the changes we propose with big data and these kinds of analytics, say, in the long run, they will change society tremendously and we currently we cannot really anticipate how.”

These statements express the paradox that in controlling the negative aspects of the techno-social we must act on what is essentially an unknowable future. This resembles Collingridge’s dilemma that indicate at which point we are most in control and able to shape a technology, is precisely the point where we lack information about its consequences (see [10] for a discussion of Collingridge). However, the RRI literature draws a distinction between the possibility of anticipating, as opposed to the intractability of prediction, and how anticipation and prediction may be conflated [11].

Thus, it may be hard (if not impossible) to predict precisely what future data and analytics will actually herald, but relatively easy to imagine (anticipate) alternative techno-social futures that involve them.

Perhaps these epistemological and temporal horizons, which are perceived to limit how ethical and privacy issues may be addressed, have little to do with absence of the necessary skills and knowledge from society as a whole. Locally, hazards and potential remedies may be well understood, but also this knowledge may be compartmentalized and hard to assimilate into new settings. Indeed, another barrier alluded to during the panel session relates precisely to this difficulty of embedding values and safeguards into processes and systems: “So even to get to the point where - the next stage - where you have to frame a procurement so that it is privacy-enhancing - well, how do you do that when you have no expertise about privacy-enhancing in people who are writing the procurement document and there are no products which can meet things that you might want to make mandatory requirements?”

44

M. Hartswood and M. Jirotka “So at the moment we are very far away from actually being able to say we can provide guarantees and safeguard that people are being treated fairly, and equitably, and that the values they care about are reflected in those systems.”

Barriers to assimilation were also identified in relation to political processes that were perceived as failing to ensure the appropriate policy and governance environment for emerging technologies: “– it’s a debate that’s a social debate – and what I’ve seen of this debate so far hasn’t been very well informed when politicians are involved.” … “[politicians at a Smart City event] agreed on the common vision that having real-time data on every citizen and everything that is going on, and everything that every citizen wants, is for them the perfect democracy.” “..but usually – politicians for example, … they have not (inaudible words?) been helpful in the last time [i.e. in recent times] – they also want to stay in the position – they may do something for long effect but very often it’s more – well – that – they decide what is good for people and not that the people are part of that.”

An interesting question concerns how far these barriers and horizons represent hard, global constraints, and how much they are real, but only within a certain framing or context. Any given ‘knowledge horizon’ may only be local, and even seemingly ‘hard’ barriers, such as influencing policy makers, may be more tractable (for some) than the quotes above suggest. For example, the panel session discussion took an interesting turn when an audience member familiar with the lobbying process showed how this particular barrier could be overcome.

Brokerage. One audience member contrasted ‘top down’, ‘bottom up’ approaches with the idea of influencing a project “middle-sideways” through the process of brokerage: “About creating brokerage between different kinds of lateral developments … and disseminating learning about good and bad approaches and so forth. That’s a very creative role of brokerage.”

In a sense this statement provides an answer to some of the issues identified as barriers and horizons above. To overcome barriers, or to extend horizons, one needs new knowledge, expertise, processes and routines. Often the relevant experience already exists, but is hard to access. Brokerage plays the role of connecting otherwise compartmentalized knowledge, and creating supportive circumstances for knowledge to flow and be assimilated into a new locale. In many ways brokerage is a key activity within RRI. It was actually part of what was happening in the panel session itself as the session created a space for reflection and cross-fertilization between people with different kinds of experience and expertise.

Brokerage was demonstrated by this specific audience member as she (in asking her question) directed SmartSociety’s attention to the SWAMI project10 that had previously considered ethical issues in relation to ambient Intelligence – a precursor of SmartSociety technologies. Another panel member recommended the work of Henry 10

http://is.jrc.ec.europa.eu/pages/TFS/SWAMI.html.

Smart Society: Collaboration Between Humans and Machines, PP

45

Mintzberg (e.g. [12]), who’s approach from the field of management science resonates with the brokerage approach.

Brokerage is visible in these simple acts of recommending and sharing references to reports or academic papers. The recommendation is as important as the actual reference, as a key challenge for brokerage includes establishing trust and relevance because of the vast array of perspectives available for people to draw upon.

The audience member recommending the brokerage approach did so in response to a point made in the keynote presentation that SmartSociety aims to address ethical issues by designing components in ways that avoid unethical outcomes. Brokerage was suggested as an alternative to this ‘prior ethical design’ as it was seen as better supporting elements of ‘creativity’ and ‘spontaneity’ that may be lacking in either ‘top down’ or ‘bottom up’ design approaches. Again, this resonates with ideas within RRI of the importance of engagement - with publics and other stakeholders - not only to encourage new lines of influence in technology formation, but also to foster the creativity that comes from the cross-fertilization of perspectives.

Influencing Politicians as an Example of Brokerage. Following several (mainly negative) comments about how politicians are unapproachable and how they fail to understand important techno-social issues, one audience member was moved to declare her own expertise in liaising with and informing politicians: “You need to understand what politicians do when they work - and when can you influence them and with what can you influence them. Basically they don’t respond well to ethical issues because they ask you “who should do what”? Because that’s the kind of things they can demand in their political arena. So you have to really translate also those issues about who should do what, and then what is the role of the governance in making the person do that. Timing is extremely important, so a politician, or at least a member of parliament, usually can only talk about things when it’s on the political agenda.”

This audience member is evidently a skilled mediator on the topic of ‘ethical concerns’ and politicians. She offered a very different perspective to the exasperation expressed towards the political process documented in earlier quotes. Her contribution demystifies how political influence may be achieved, in particular, by revealing the rather mundane series of practical steps and pragmatic considerations involved in it. It is also another example of brokerage in action, where the venue /mechanism of the summer school panel session itself created opportunities for opening up otherwise compartmentalized knowledge.

7 Conclusions SmartSociety carries with it its own vision, which is continuously undergoing evolution and redefinition to take on shapes and directions that are not always aligned across all project partners. This is a healthy state of affairs for a large, complex, cutting-edge research project. Indeed, the Summer School panel session made a significant contribution to further exploring what a SmartSociety might aim to be.

46

M. Hartswood and M. Jirotka

Various versions of the SmartSociety vision were on show during the keynote talk, panel session and in the questions and answers that followed. Summer School attendees saw SmartSociety as part of digitally enhanced everyday living, as a solution to profound social problems, as being smarter through less engagement with technology, and more through learning and thoughtfulness. They saw the grand, all-encompassing, societal level vision shine through the keynote presentation, as well as a more low-key reimagining of a localized community-level SmartSociety in the reply by one of the panelists. Whilst this contestation and exploration of the SmartSociety vision has positive aspects, there are downsides too, especially if the vision is so fluid that it proves always to be slippery and impossible to pin down. Indeed, the SmartSociety system itself has a chameleon-like ability to fit within practically any application context and, as it shifts between domains, the complexion of relevant values changes too. This leads to another problem whereby it becomes easy for advocates of SmartSociety to evade any given critique by giving it a new guise within an alternative setting. Also, different values come into play depending on scale and the application area, and repositioning SmartSociety solutions to operate over a smaller scale diffuses many of the ethical issues posed when it is articulated in terms of its grander ambitions.

These observations suggest that, on the one hand, it could be easy for the SmartSociety project to deflect various ethical concerns by deftly repositioning itself but, on the other, that it is genuinely hard to pin down the complexities of values attached to the various guises of SmartSociety.

One way forward may be to explore actively how the SmartSociety vision fits within the field of existing and already deeply considered visions and critiques of techno-social futures. These may include ones that are more naturally allied to the SmartSociety vision, such as ‘Social Physics’ [13], and others that may be more oppositional, such as Morozov‘s critique of ‘Technological Solutionism’ [14]. As part of this process, it would be useful to strive to articulate the possible combinations of values that may attach to SmartSociety in its different guises, to discover on which occasions they are complementary and able to co-exist, and when they are actually antagonistic or truly mutually exclusive. These measures may help the consortium to articulate SmartSociety values in a clearer, more contextualized way, as well as to stimulate reflection upon and refine which values it ultimately wishes the SmartSociety system to uphold.

There were several moments in the workshop discussion where arguments hinged around a distinction between technology and society, such as where technology may be seen as coming to the aid of societal problems, with society needing to respond to unwanted side-effects. Such issues were often couched in terms of barriers or horizons.

While these are real and limiting in each individual case, their relative severity seems also to depend on the situation or context in which they are experienced. For example, one audience member, steeped in the mechanisms underpinning political processes, hardly felt this to be a barrier at all, whilst others experienced the political world as frustrating and troubling.

A key contribution came from an audience member who highlighted an important role for brokerage. Brokerage involves creating networks of connections and flows of experience and expertise between otherwise discrete and compartmentalized communities to stimulate creativity and gain answers to issues where approaches already

Smart Society: Collaboration Between Humans and Machines, PP

47

exist11. Brokerage resonates strongly with the processes involved in RRI, which also seek to draw multiple perspectives into innovation processes. Brokerage may be a way of positioning RRI as a source of creativity and an enabler of innovation, as opposed to the perception that RRI sometimes attracts of seeking to constrain and regulate. If we shift our perspective from thinking of technology and society as being separate entities (where society has to cope with more and more varied technologies) and instead consider the direction of travel as towards weaving ever more dense techno-socio hybrids, then to manage these trends effectively perhaps we do need processes similar to brokerage that help synthesize trans-disciplinary perspectives.

Finally, it is worth reflecting on the value of the panel session as a venue for stimulating debate. One aspect to highlight is that the discussion typically did not dwell on the implications of the core features of SmartSociety itself – such as algorithms, incentives, collective intelligence and so on, but rather focused on higher-level issues relating to the SmartSociety vision and the complexities of addressing ethical concerns more generally. Whilst these were useful discussions to have, in a future staging of the panel session we would consider adjusting the approach and reiterating key questions from the initial keynote to refocus any discussion around those core elements.

Overall, we believe that holding this workshop session was a valuable experience, both to enrich the summer school and to assist with the reflective process within SmartSociety project itself. We are grateful to the panelists and the summer school attendees for consenting to participate, and for their highly insightful and stimulating contributions.

Acknowledgements. This work was supported by EU FP7 FET SmartSociety project (http:// www.smart-society-project.eu/) under the Grant agreement n.600854.

References 1. Miorandi, D., Maltese, V., Rovatsos, M., Nijholt, A., Stewart, J. (eds.): Social Collective Intelligence: Combining the Powers of Humans and Machines to Build a Smarter Society.

Springer, New York (2015) 2. Stahl, B.C., Eden, G., Jirotka, M., Coeckelbergh, M.: From computer ethics to responsible research and innovation in ICT: the transition of reference discourses informing ethics-related research in information systems. Inf. Manag. 51(6), 810–818 (2014) 3. von Schomberg, R.: A vision of responsible innovation. In: Owen, R., Heintz, M., Bessant, J. (eds.) Responsible Innovation. John Wiley, London (2013) 4. Owen, R., Macnaghten, P., Stilgoe, J.: Responsible research and innovation: from science in society to science for society, with society. Sci. Public Policy 39(6), 751–760 (2012) 5. Brey, P.: Values in technology and disclosive computer ethics. In: Floridi, L. (ed.) The Cambridge Handbook of Information and Computer Ethics. Cambridge University Press, Cambridge (2009) 6. Lanier, J.: Who Owns the Future?. Simon and Schuster, New York (2014)

11

Interestingly enough, brokerage could be couched in SmartSociety parlance as ‘leveraging diversity in a collective social intelligence to create enriched hybrid perspectives’.

48

M. Hartswood and M. Jirotka

7. Guest, G.: Applied Thematic Analysis. SAGE Publications, Inc., California (2012) 8. Habermas, J.: The Structural Transformation of the Public Sphere: An Inquiry into a Category of Bourgeois Society. MIT press, Cambridge (1991) 9. Thieme, A., Vines, J., Wallace, J., Clarke, R.E., Slovák, P., McCarthy, J., Mas-simi, M., Grimes Parker, A.: Enabling empathy in health and care: design methods and challenges. In: CHI ‘14 Extended Abstracts on Human Factors in Computing Systems (CHI EA’14), pp. 139–142. ACM, New York (1985) 10. Croy, M.J.: Collingridge and the control of educational computer technology. Techné Res. Philos. Technol. 1(3/4), 107–115 (1996) 11. Guston, D.H.: Understanding ‘anticipatory governance’. Soc. Stud. Sci. 44(2), 218–242 (2014) 12. Mintzberg, H., Waters, J.A.: Of strategies, deliberate and emergent. Strateg. Manag. J. 6(3), 257–272 (1985) 13. Pentland, A.: Social Physics: How Good Ideas Spread-The Lessons from a New Science.

Penguin, London (2014) 14. Morozov, E.: To Save Everything, Click Here: The Folly Of Technological Solutionism.

PublicAffairs, New York (2014)