Emerging digital/AI tools to enhance democratic participation
Democratic input into AI-powered emerging technologies
Multimodal analysis
Sentiment analysis
Body language
Multimodal analysis
Sentiment analysis
Argumentation
Emotional expression
Body language
• Aristotle (IVth BC)
• New Rhetoric (Perelman and Olbrechts-Tyteca) (1966)
• Toulmin’s (1958) model
• Douglas Walton (1996) argumentation schemes
• Habermas’ discourse ethics and Steenbergen’s discourse quality index (DQI)
Argumentation mining
Different ways to approach argumentation mining
Claim Premise
Lippi, M., & Torroni, P. (2016). Argumentation Mining. ACM Transactions on Internet Technology, 16(2), 1–25.
Carstens, L., & Toni, F. (2015). Towards relation based Argumentation Mining. Proceedings Ofthe 2nd Workshop on Argumentation Mining, 29–34.
Argumentation mining
Different ways to approach argumentation mining
Nuclear disasters like Chernobyl or Fukushima cause a lot of harm.
Nuclear energy has a large capacity of energy production and makes up 20% of US energy generation. We should use
Lippi, M., & Torroni, P. (2016). Argumentation Mining. ACM Transactions on Internet Technology, 16(2), 1–25.
Carstens, L., & Toni, F. (2015). Towards relation based Argumentation Mining. Proceedings Ofthe 2nd Workshop on Argumentation Mining, 29–34.
US presidential debates
Argument/non-argument classification
Meeting of the Association for Computational Linguistics, Proceedings of the Conference, 4684–4690. All the videos!
Haddadan, S., Cabrio, E., & Villata, S. (2020). Yes, we can! Mining arguments in 50 years of US presidential campaign debates. ACL 2019 - 57th Annual
Multimodal models
Argument / Other classification
Mestre, R., Middleton, S. E., Ryan, M., Gheasi, M., Norman, T. J. & Zhu, J. (2023, May). Augmenting pre-trained language models with audio feature embedding for argumentation mining in political debates. In Findings of the 17th conference on European chapter of the Association for Computational Linguistics (EACL). https://github.com/rafamestre/Multimodal-USElecDeb60To16
Feature extraction
Low-level descriptors:
• MFCCs
• Spectral centroids
• Spectral bandwidth
• Spectral roll-off
• Spectral contrast
• Chroma
Adapted from: Cai, L., Hu, Y., Dong, J., & Zhou, S. (2019). Audio-textual emotion recognition based on improved neural networks. MathematicalProblemsin Engineering , 2019.
Feature extraction
Adapted from: Cai, L., Hu, Y., Dong, J., & Zhou, S. (2019). Audio-textual emotion recognition based on improved neural networks. MathematicalProblemsin Engineering , 2019.
Our results
Argument / Other classification
Rafael Mestre, Stuart E. Middleton, Matt Ryan, Masood Gheasi, Timothy Norman, and Jiatong Zhu. 2023. Augmenting pre-trained language models with audio feature embedding for argumentation mining in political debates. In Findings of the Association for Computational Linguistics: EACL 2023, pages 274–288, Dubrovnik, Croatia. Association for Computational Linguistics.
Our results
BERT-based models still perform better than any multimodal audio/text models with the full dataset
However, when the data is scarce, multimodal models show an improved performance, comparable to large dataset cases
Rafael Mestre, Stuart E. Middleton, Matt Ryan, Masood Gheasi, Timothy Norman, and Jiatong Zhu. 2023. Augmenting pre-trained language models with audio feature embedding for argumentation mining in political debates. In Findings of the Association for Computational Linguistics: EACL 2023, pages 274–288, Dubrovnik, Croatia. Association for Computational Linguistics.
Speech characterisation
Speech characterisation
Emotion recognition
• Classification between “angry”, “happy”, “neutral”, “sad” with a finetuned Wav2Vec 2.0 model
Dominant speaking styles or model biases?
Emotion recognition
• Classification in the Valence-Arousal-Dominance framework
We can extend the notion of textual sentiment to a more nuanced understanding
Computational social science and ethics need to go hand
by hand
Technology is not neutral and can reproduce the same forms of discrimination and inequalities that tries to solve
Practical example
Classic research for democracy scholars
• In deliberative democracy, political discussion should arise from equal dialogue, driven by an exchange of reasons and arguments, and a readiness to revise one’s viewpoints.
• Deliberative processes can also reproduce societal hierarchies, and participants from marginalised groups are often less present and heard.
• Who is involved when in collective decisions? And How?
– What is effective participation?
– What role can and will “ordinary” citizens play?
– Who has a voice and vote and what are the consequences for governance?
Argument mapping
Asynchronous argument mapping in a minipublic
• In response to a UK Justice Select Committee on the topic of ‘Sentencing’
• Included 28 people over three half-day sessions online.
• Only England and Wales.
• Aim: to better understand public awareness and perceptions of sentencing and understand their “priorities” for sentencing. Co-delivered with
The Deliberatorium
The effect of argument mapping
Argument mapping helped drawing attention to contributions from participants who were less confident or in minority, creating a more equal deliberation:
“Ithinkalittlebitofjusthavingthatstructureautomaticallymakesotherpeoplerespectyour pointsinthediscussionabitmore.Ithinkitaddslegitimacyifyou'renotaveryconfidentor forcefulperson.Ifyouhaveamousylittlecommentthatpeopledon'treallyhearit'snownoted downinthisargumentmappingandpeoplecan'tjustskirtaroundorsteamrollit.”(ParticipantG)
The effect of argument mapping
Facilitators highlighted that argument mapping created a space for collective reflection, separating contributors from the contributions, depersonalising arguments:
“Therewassomethingaboutbeingabletopointtoanideaorathoughtthatwasn'tpointingata person.Itwasmorelookingatthetextandscrutinising.Sojustgaveusalottoplaywiththatfelt verydemocraticandyouknow,morecooperativeaswell...[…]”.(Facilitator2).
Pros and cons:
– Could make individuals more reflective and self-critical of own arguments, as they stop being “theirs”.
– Deliberation might need to become more personal to break the dominance of privileged groups.
– Might overshadow personal narratives or storytelling, which convey important situated knowledge → Could they be embedded into argument maps as well?
Mid-way conclusions
• Novel multimodal machine learning models allow us to understand nuanced aspects of (political) speech.
• Biases (gender, accents…) and issues of data (mis)representation need to be carefully considered.
• Argument mapping can help people consider different viewpoints from underrepresented groups. It can depersonalise arguments, separating them from the contributor.
• However, there’s no AI-powered solution (not even LLMs!) that can bridge argument mining with argument mapping in live deliberations.
Emerging digital/AI tools to enhance democratic participation
Democratic input into AI-powered emerging technologies
Responsible Research
and
Innovation
(in AI and emerging technologies)
• Consider bigger picture
• Long-term impacts
• Value to society
• Von Schomberg (2023):
ethical acceptability,
sustainability and
societal desirability.
Anticipate, reflect, engage, act (AREA)
• Anticipate
– Describe and analyse the impacts, intended or otherwise, that might arise.
• Reflect
– Reflect on the purposes of, motivations for and potential implications of the research, together with the associated uncertainties, areas of ignorance, assumptions, framings, questions, dilemmas and social transformations these may bring.
• Engage
– Open up such visions, impacts and questioning to broader deliberation, dialogue, engagement and debate in an inclusive way.
• Act
– Use these processes to influence the direction and trajectory of the research and innovation process itself.
Three project snapshots around RRI for emerging technologies
AI for mental health
Biohybrid robotics
Distributed fibre optic sensing
Fairness And Bias Of Multimodal Natural Language Processing For Mental Health
International Partnership grant between the Institute for Experiential AI at Northeastern University (US) and the University of Southampton from the Responsible AI UK (RAI UK) programme
+ Industry stakeholders
http://rai4mh.com/
• Research on bias/fairness of ML models
• Workshops with stakeholders
• Policy engagement
Rafael Mestre
Stuart Middleton
Annika Schoene Agata Lapedriza Cansu Canca
Bio-hybrid robotics
Bio-hybrid robotics
Guix, M., Mestre, R., et al. (2021). Biohybrid soft robots with self-stimulating skeletons. Science Robotics, 6(53), eabe7577.
Douglas Blackiston and Sam Kriegman via The Washington Post. https://www.washingtonpost.com/science/2021/11/30/living-robotsreproduction-study/
• “People will ask: is it a robot, is it a machine, is it an animal?” [1]
• “An uneducated public may see this as Frankenstein-like”. [1]
• “Regulators, scientists and society should carefully weigh up the risks and rewards.” [2]
[1] Sokol, J. (2020, April 3). Meet the xenobots, virtual creatures brought to life. The New York Times. Retrieved June 9, 2022, from https://www.nytimes.com/2020/04/03/science/xenobots-robots-frogs-xenopus.html.
[2] Coghlan, S., & Leins, K. (2022, March 28). Will self-replicating Xenobots' cure diseases, yield new bioweapons, or simply turn the whole world into Grey Goo? The Conversation. Retrieved June 9, 2022, from https://theconversation.com/will-self-replicating-xenobots-cure-diseases-yieldnew-bioweapons-or-simply-turn-the-whole-world-into-grey-goo-173244.
Emerging technologies
• How can citizens give input into the development of emerging technologies that will affect them directly, following principles of RRI?
– Unknown nature of the technology.
– Technological imaginaries still not formed.
• How can policymakers get ahead of the regulations that might be needed to avoid moral panic and path dependencies?
– Very reactionary → thinking short term.
• What if AI-powered solutions revolutionise the development of these early-stage emerging technologies at an unprecedented pace?
Surveys
Sociotechnical scenarios
Policy analysis
https://biohybrid-futures.ac.uk/
Ethical analysis
Deliberative workshops
Interviews
Distributed
Distributed Fibre Optic Sensing in Smart Cities
• Traffic monitoring
• Perimeter monitoring
• Buildings’ structural integrity
• Sound pollution
• Environmental monitoring
• Privacy?
• Data ownership?
• Local spatial inequalities?
• Sustainability?
Source: https://www.internationales-verkehrswesen.de/fibre-opticdistributed-acoustic-sensing-technology-to-improve-air-quality/
Distributed Fibre Optic Sensing in Smart Cities
Citizen panel
Research priorities
Inter- and trans-
disciplinary research
Distributed sensing measurements in cities
Key takeaways
Emerging digital/AI tools to enhance democratic participation
Democratic input into AI-powered emerging technologies
