Alarming warnings about what AI can do from industry insiders are adding to pressure to limit artificial superintelligence.
By Robert BOOTH
Is artificial superintelligence really a bigger threat than nuclear weapons? Is there a significant chance of a “Chernobyl-sized disaster”? Could there even be more than a 10% chance that AI will “kill all humans” within the next decade? These warnings have been made over the past 48 hours on both sides of the Atlantic, about the potential impact of a technology that most people still perceive as just “a more chatty search engine”. Some of these threats have been raised in the UK, as MPs and members of the House of Lords began to grapple with the risk of AI surpassing human capabilities. The warnings, which compare the risk to that of nuclear weapons, came from Des Browne, the former Defence Secretary, and Professor Stuart Russell, a renowned computer scientist at Berkeley.
Beatrice Fihn, who won the Nobel Peace Prize in 2017 for leading the International Campaign to Abolish Nuclear Weapons, told parliamentarians: “This is not the first time we are faced with capabilities that could end up killing us all.”
Monday’s hearing was organised by Control AI, a lobby group that calls for international regulation of technology. It supports a bill introduced in parliament this week by Labour MP Alex Sobel, which aims to ban the creation of artificial superintelligence (ASI). On Tuesday evening, a senior Anthropic executive admitted he believed there was more than a 10% chance the technology could “kill all humans” within the next decade and warned that his AI company did not have a plan to ensure ASI was aligned with human interests, meaning it would not cause harm. Predictions for when ASI might be achieved range from a few years to more than a decade.
Evan Hubinger, chief science officer at Anthropic, made the comment after Jacob Coxon, a 28-year-old researcher at the San Francisco-based company and previously at OpenAI, resigned, claiming that “no company was acting responsibly.” He said they were “gambling with our lives.”
Coxon said the risks at Anthropic were well understood, but companies were “locked in a race to be first.” However, in a glimmer of hope, he added that he was optimistic about the possibility of coordination between US labs to control the pace of their progress in this race. Anthropic has been contacted for comment on the matter. OpenAI referred to a statement by its chief scientist, Jakub Pachocki, who last week said: “International coordination on the future development of AI must become a top priority for governments around the world.” The push for broader coordination on controlling ASI was at the heart of developments in Westminster this week. According to the Financial Times, government officials expressed concern that Anthropic had refused to submit its latest model – Mythos 5.1 – to the UK’s AI Safety Institute for testing before release.
Only a few US organisations have had access to Mythos 5.1, the Cabinet Office confirmed. A spokesman added: “The AI Safety Institute continues to work closely with industry partners, including Anthropic.” Labour MP Darren Jones has written to world leaders to call for a multilateral treaty to ensure the safe development of AI.
Jones has written to Prime Minister Andy Burnham and the heads of the UN and OECD, calling for a “multilateral treaty for the regulated and safe development of superintelligence – not a ban on innovation or scientific research, but a safety-first approach to the rapid development of this technology.” Referring to Coxon’s resignation, Jones added: “The debate ranges from the end of humanity to claims that we are dealing with ‘marketing hype’. Either way, governments must now intervene.” On the other side of the Atlantic, Bernie Sanders this week intensified his campaign on AI safety, again calling on Congress to regulate the technology. He cited polls showing that 81% of Americans believe their politicians should take action.
The independent senator from Vermont said: “We cannot allow a handful of greedy people to play God and determine the future of humanity – the economy, the environment, democracy, our privacy and more – without public participation.”
ControlAI is funded by Jaan Tallinn, the multibillionaire founder of Skype, who describes himself as an “anti-extinctionist.” He has dedicated a portion of his fortune to campaigning for AI safety and says that some senior AI industry executives would be happy if humanity went extinct. Although he was an early investor in Anthropic and Google DeepMind, two of the leading AI labs, Tallinn estimates that 10-15% of AI workers believe that the technology would be a worthy successor to humanity. In an interview earlier this summer, he said, “A very well-known AI researcher told me, ‘Jaan, don’t worry about it. Humans are a volatile species.’ From what I understand, he had no problem with us going extinct.”
The Tallinn lobby group organised a hearing for MPs and members of the House of Lords in Westminster on Monday, urging them to support measures to limit the most powerful AI models. A copy of the book If Anyone Builds It, Everyone Dies: The Case Against Superintelligent AI was placed on every chair.
Politicians listened to Russell, who warned about Chernobyl and said: “The other possibility is a much bigger catastrophe, in which humanity loses control irreversibly. But we have no say in whether we will continue to exist.” Browne said that, when he was defence secretary, he thought nuclear war was the most likely threat facing humanity. Now he believed that “a superintelligent AI poses a threat of the same magnitude, perhaps even greater.” However, other experts were more cautious. Dr Andrew Rogoyski, of the Surrey Institute for People-Centred AI, said: “In reality, these systems are nowhere near as versatile as humans, let alone as humans acting collectively. I suspect we are heading for the ‘great disappointment’, where advanced AI will prove too expensive and not useful enough to continue in its current form.”
David Barber, director of Sofair, a state-backed AI research lab that brings together academics from Oxford, Cambridge, Edinburgh and UCL, said he worried that people might be “throwing the baby out with the bathwater” – that is, by fighting the risks, they could also miss out on the benefits of the technology.
“AI is not going to go away,” he said. “It is incredibly useful. Whether or not you allow it to have completely unrestricted access to the internet and to different systems, that could be problematic. We may need to learn how to better control these things. There are vulnerabilities in the software architecture that need to be fixed. But it can be done. “What we need as a country is to understand this dual nature: we have an incredibly important and useful technology and, at the same time, something that needs to be thought through and controlled very carefully.” Sandra Wachter, a professor at the Oxford Internet Institute, said she did not believe in “Terminator scenarios,” but that AI posed real threats, including its impact on the environment, the spread of disinformation and the replacement of jobs. “These problems are real and urgent and need to be addressed now. Terminator scenarios are a huge distraction from the real problems.”
Gary Marcus, an AI industry commentator and academic, said: “There is a difference between a superintelligence aligned with human interests – if such a thing is possible – and a superintelligence that is not. It is at least conceivable that the former could have a generally positive effect. So far we have neither, but an extraordinary excess of hype and promise, combined with a surprising lack of prudence on the part of OpenAI, has brought us to where we are, with deep distrust on all sides.” (The Guardian)