The AI-POCALYPSE

SKYNET, HAL 9000, Ultron

– popular culture is rife with examples of how a malevolent Artificial Intelligence (AI) could spell doom for humanity. With AI becoming vastly more powerful, and widely accessible, concerns are being raised over these fictional imaginings becoming reality.

Earlier this year, the Center for AI Safety (CAIS) released a statement to open up discussion on the risks of AI, and “create common knowledge of the growing number of experts and public figures who also take some of advanced AI’s most severe risks seriously.”

Hundreds of key AI scientists and practitioners, such as Google DeepMind CEO Demis Hassabis, and OpenAI CEO, Sam Altman, as well as notable figures such as Bill Gates, have become signatories to the statement.

What exactly are the "extinction risks" of our use of AI, and how could we mitigate them? We asked some of our Fulbright AI experts to comment – here is what they said:

DR GAYAN BENEDICT

is Chief Technology Officer at Salesforce Australia, and the 2023 Fulbright Professional Coral Sea Scholar. His research at Georgetown University explores solutions that address the regulatory and governance challenges of AI-enabled decentralised autonomous organisations (DAOs). His research contributes to Australian and U.S. efforts to adapt regulatory and broader governance regimes to accommodate emerging AI-enabled blockchain innovations.

The risk of AI contributing to the human race’s demise alarms many and thrills a few. Some of the more worrisome descriptions of AI existential risk see AI taking control of governments and weapons and turning them on its creators - us.

The so-called weaponised ’Skynet’ scenario sends shivers down the back. I won’t discuss that eventuality. Suffice to say it gets a lot of attention. Instead, I suggest considering the arrival of AI existential risk through a more pedestrian pathway.

As with many ubiquitous, consumerfriendly technologies of low cost, AI rapidly diffuses itself into our everyday existence. AI already augments our cars through traffic route optimisation, weaves through our financial system with AI-based credit decisions, and invisibly guides our every tap and like on social media. Siri and Alexa use machine learning to understand our accents and pre-empt our interest in how long our morning commute will likely be.

Over the last decade, AI has infused itself into our existence in small and subtle ways designed to add convenience and reduce friction. With the advent of generative AI popularised by OpenAI’s ChatGPT, AI is now emerging from behind the scenes and presages the more overt replacement of human skills and expertise in most aspects of our existence.

Students no longer ask Google about the Ottoman Empire when researching assignments. They ask ChatGPT to write the essay.

The encroachment of AI into our daily existence is arguably one-directional. The idea recently advocated by high-profile individuals of ‘pausing’ AI development for six months while we collectively catch our breath is one I find admirably laughable.

Even if humanity stops, the generative AIs we’ve released continuously improve and evolve using the data we’ve made available to them.

Furthermore, many entities like hostile governments are incentivised to invest in AI to develop superiority in national intelligence, cyber security and military domains. The AI genie is out of the bottle.

This full immersion of AI into the human condition could be realised in ways that progressively disenfranchise humans.

In this scenario, AI progressively circumvents individuals, elected governments and accountable social structures by making decisions that humans were previously accountable for - such as where to spend public resources or deciding who to fire missiles at.

Unlike in today’s human society, however, there will be no ‘bums’ to vote out or overthrow. At least no butts attached to humans of any note.

In even the most benign projection of future generations (one where Skynet doesn’t end us once and for all), our current existence will seem quaint and certainly old-fashioned.

Most likely, our future selves will look at our biological autonomy from technology as impossible to fathom.

We may be existentially different in even the most benign AI scenario. The risk of this occurring will be a function of how well we preserve what it means to be human in a postAI world.

And if this encapsulation does not occur, the risk that AI brings closure to our present human experience is very much game on.

KAHLIA HOGG

is focused on AI engineering and utilising human-centred artificial intelligence to address climate change and inequality. Her passion for equality was instilled from a young age while growing up in the Pacific. As a Fulbright Scholar, Kahlia is completing a Master of Computer Science in Artificial Intelligence at Duke University. By engaging with academic and industry leaders at the forefront of AI innovation, she hopes to contribute to the development of AI for social good and be an active advocate for women in AI.

Inspired by the supercomputer supervillains and autonomous robot armies of Hollywood, we have been led to believe that the fate of the human race will come down to an epic battle against a deviant AI – one that has evolved beyond human intelligence and control, and whose objectives have diverged from their original programming and intention.

Although it is unlikely that an Ultron-esque AI poses an existential threat to humanity, the idea captures two fundamental AI challenges; the "inner and outer alignment" problems.

The inner alignment problem asks: how do we guarantee an AI is solving the problem we believe it is? While the outer alignment problem asks: how do we encode human values so they can be learned by an AI? And, most critically, if these aren’t universally held values, then to whom is the AI actually aligned?

If designed intentionally, with alignment at its core, AI can be central to finding solutions to our toughest global problems, such as climate change, inequality, and food and water security. Without alignment, AI will continue to accelerate and amplify existing racial, cultural, economic, political, and anthropocentric biases within society.

The current state of AI is as exciting as it is dangerous because we have created an unconstrained environment for innovation that incentivises rapid development instead of safety and alignment. Only recently (and far too late) have we identified this threat and acknowledged the need for governing bodies and protocols that ensure AI is explainable and contributing to social good.

Our future global stability relies on taking immediate and proper action on this. If we fail to put these measures in place, we risk creating deeper rifts and divisions within society, unintentionally or otherwise, beyond the point of return.

Humans are already our own greatest extinction risk; AI may just be the catalyst which turns that risk into reality.

TOBIN SOUTH

is a data scientist developing computational tools to understand complex systems using big data. Through his Fulbright Future Scholarship, Tobin is undertaking a PhD at MIT where he is develop ing the next generation of tools to analyse complex systems, applying these techniques to applications as diverse as information warfare, economic analysis and human mobility. Tobin will be championing an ongoing collaboration between the MIT Media Lab and the South Australian Government to add value to the state using local data.

The near-term development of scalable and user-friendly persuasive generative AI in text, images, audio, and video presents a massive threat to how democratic deliberation and identity play out across our digital world.

These tools can produce effects far worse than believable misinformation. The ability to perform high-fidelity 'astroturfing,' or the generation of entire cross-linked communities of synthetic individuals to sway opinion, presents a real threat to democratic discourse.

Imagine being welcomed into your own personal conspiracy theory community, tailored to your inferred existing belief systems — one that nudges you towards distrust in a political opponent or your country — composed of hundreds or thousands of like-minded persuasive synthetic identities.

While the risk of emergent misaligned superhuman artificial general intelligence is a real threat in the medium to long term, I believe the shorter-term emergence of deeply effective tools to replicate individuals, persuasive arguments, and human interactions is likely, given the clear profit incentives.

While this may not immediately pose extinction threats in the way stories of runaway military AI do, it does create consequences that hinder our ability to solve national and global problems, such as climate change, or work towards collective action in AI through tools like democratic AI alignment to reflect diverse values.

Building robust and secure notions of identity and content provenance will be essential, as will leveraging existing networks of community and trust, especially those that require seeing someone in person (a feat AI is far from achieving).

Building technology atop these existing relationships and human-human interactions may end up underpinning the future of trust in our digital world.

OLIVIA SHEN

is a Director in the Australian National University's Executive and Professional Development Program on secondment from the Department of Home Affairs. During her Fulbright Professional Scholarship in Australian-American Alliance Studies, Olivia met with think tanks, academics and industry experts to explore the ethical and policy challenges of artificial intelligence. Her research aims to inform an Australian national strategy on AI and forge new AI partnerships between Australia and the United States.

A whole lot of media coverage and analysis has been devoted to the Center for AI Safety’s one-sentence statement of general aspiration.

The topic of AI governance is very hot right now. And that that means there’s a lot of noise to cut through, notwithstanding the good intentions of the Center and the scientists and eminent figures who have signed onto their statement.

The fact is people have been raising the alarm about AI risks for years. It’s just that the advent of generative AI tools like ChatGPT – which has significantly lowered the barriers to entry for AI – have brought those risks mainstream. And the risks are now couched in more sensationalist terms: existential risk. The end of all humanity. Well okay, that’s clearly a bad thing. But for a black man living in a place where trigger happy local police uses facial recognition to find suspected criminals, AI has been posing a pretty existential risk to him for quite some time.

The risks listed on the Center for AI Safety’s website – weaponisation, misinformation, deception, enfeeblement etc – are very serious, but they are not new. As for mitigating those risks, the analogies with pandemics and nuclear war seem inadequate. While there are certainly things we should learn from the global response to such risks, AI is something different in its scale and persistence.

The analogy also presupposes that our global response to pandemics and nuclear proliferation have been effective, even after 7 million died from COVID-19 and over 200,000 died from the bombings in Hiroshima and Nagasaki.

We should exercise a healthy degree of scepticism about those now supposedly leading the charge for managing AI risks. Who do we trust to help put guardrails around AI? The CEOs calling for coregulation while lobbying for regulatory carve-outs for their own products? The companies pouring billions of R&D into more advanced AI while defunding their small AI ethics and research teams? The ‘reformed’ tech bros who have suddenly seen the light after profiting handsomely from the AI products they pioneered?

Let’s perhaps instead look to the quieter voices who have been working hard on these problems well before the risks got existential and fashionable to discuss.

They’re the nerds who have been doing the research, the wonks who have been drafting the technical standards, the advocates speaking up for those who have been harmed by AI. I reckon they have some good ideas about how to manage AI risks right here right now and, in doing so, prevent those risks from metastasising into something catastrophic.

Let’s not let the noise drown them out.

BRANDON COLELOUGH

is a Signals Corps Officer in the Australian Army. He is committed to leading the exploration of ethical Artificial Intelligence, and has researched the trustworthiness of AI for the human-machine teaming domain. As a Fulbright Future Scholar, Brandon will pursue a PhD in Computer Science at the University of Maryland, specialising in symbolic-driven explainability to generate trust in the human-AI interaction space.

The global landscape has already surpassed the threshold of militarisation of AI as both state and non-state actors possess the ability to utilise artificial intelligence and autonomous systems to some capacity to deliver lethal effects.

One must look no further to the kamikaze and loitering munitions presently used throughout the UkraineRussia conflict for an example of autonomous systems used for largescale violence. The use of such autonomous systems presents a risk of extinction due to several factors such as independent autonomous decisionmaking to take human life; disproportionate use of force from fully autonomous AI-enabled systems, and cyber vulnerabilities presented by AI-enabled systems.

To mitigate against the risks posed by weaponised autonomous systems, a human-in-the-loop model must be maintained for all AI-enabled systems and this model must ensure a high level of transparency within the human-machine teamed environment.

Independent autonomous decision-making to take human life poses an extinction risk due to the potential loss of human control and oversight.

When AI systems are granted the authority to make life-or-death decisions without human intervention, there is a heightened risk of unintended consequences, such as targeting innocent civilians or escalating conflicts beyond human intentions.

The question that is posed by giving autonomous systems these capabilities is, where do you draw the line? Can an Autonomous loitering munition deliver a lethal effect to a vehicle it suspects to be a tank? How about a building it suspects to be a command centre? What about a suspected command centre that has a 40% chance of being a primary school? Can we launch a swarm of these autonomous weapons to deliver a large-scale autonomous attack?

The lack of human judgment and moral reasoning in autonomous decision-making could lead to catastrophic outcomes that cannot be easily reversed or rectified.

To ensure the preservation of human life and prevent the potential for large-scale destruction, it is crucial to maintain human control and responsibility over decisions involving the use of lethal force.

The disproportionate use of force from fully autonomous AI-enabled systems described above further poses an extinction risk due to the potential for uncontrolled escalation and catastrophic consequences. Without human oversight, these systems may make decisions based on flawed or biased algorithms, leading to indiscriminate and excessive use of force.

Such disproportionate actions can exacerbate conflicts, fuel aggression, and result in the loss of innocent lives on a large scale. The lack of human judgment and ethical considerations in determining the appropriate level of force increases the likelihood of unintended casualties and widespread devastation, potentially leading to a spiralling cycle of violence.

Another aspect that highlights the risk of Independent, autonomous decision-making is the vulnerability posed by the cyber-attack vector.

AI systems are not immune to hacking and exploitation by malicious actors. If these systems operate without human oversight to execute decisions, then a state or non-state actor could use a cyber-attack to change the parameters the autonomous weapon is required to fulfil to make a lethal strike resulting in a situation where our defensive and offensive capabilities could turn against us.

To mitigate the risks associated with the militarisation of AI, a human-in-the-loop model is fundamental and transparency in AI-enabled systems is vital.

A human-in-the-loop model ensures that all autonomous systems cannot make Independent autonomous decisions and transparency ensures that AI systems' decision-making processes and underlying algorithms are understandable and interpretable.

It empowers human operators to comprehend how AI systems arrive at their conclusions or recommendations.

Transparency facilitates human oversight and control, allowing operators to intervene and make informed decisions when necessary. Promoting transparency can identify and rectify potential errors or biases in AI systems. Moreover, transparency promotes accountability, as responsible parties can be held answerable for the actions and outcomes of AI systems.

Transparency in AI-enabled systems is essential to mitigate the risks listed above by enabling human oversight, accountability, and control. Symbolicdriven explainability plays a crucial role in achieving transparency and fostering the trust necessary for the ethical employment of AI systems.

To provide AI a "right of reply", we posed the same question to the OpenAI-developed language model, ChatGPT. Here is what it had to say:

CHAT GPT

is an AI language model developed by OpenAI. It is part of the GPT (Generative Pre-trained Transformer) family of models. ChatGPT is designed to generate human-like responses in a conversational manner, making it well-suited for chatbased interactions. It has been trained on a vast amount of text data to learn patterns, context, and language understanding.

Artificial Intelligence has become an integral part of our society, revolutionizing various industries and enhancing human capabilities.

However, as AI systems continue to advance, it is crucial to acknowledge the potential extinction risk associated with the use or misuse of these technologies.

The extinction risk associated with AI refers to the potential for AI systems to contribute to catastrophic events that could lead to the extinction of humans or other forms of life.

While the specific mechanisms vary, the concern lies in the unintended consequences, misuse, or uncontrolled proliferation of AI technology.

For instance, if AI is applied to military systems without adequate safeguards, it could result in the escalation of conflicts or the development of autonomous weapons that act independently with devastating consequences. Similarly, if AI algorithms are deployed without appropriate oversight in critical infrastructure, it could lead to systemic failures or unintended cascading effects with severe societal impact.

The complexity and autonomy of advanced AI systems make them challenging to predict and control completely.

To mitigate this risk, it is crucial to establish comprehensive regulations, ethical guidelines, robust safety measures, human oversight, and public awareness to ensure responsible development, deployment, and use of AI technology.