But hedging against low-probability existential risks is notoriously difficult. In the dystopian world that uncontrolled AI could bring, traditional hedging tools like bonds and put options would be of little use, as financial accounts, payment systems, and records could themselves be compromised.
By Dambisa MOYO
Warnings of an AI apocalypse are reaching a crescendo. Last week, Anthropic researcher Jacob Coxon shocked the world when he publicly resigned, revealing that “the people who build AI honestly believe it could kill us all by the end of the decade.” His comments were immediately echoed by Evan Hubinger, the company’s head of compliance science, who put the probability of the “what if” scenario of AI killing all humans at more than 10%. Within days, Anthropic CEO Dario Amodei, OpenAI CEO Sam Altman, and other leading AI figures called for a slowdown in the pace of its development. Fears of an AI apocalyptic scenario have been growing for some time. Geoffrey Hinton, considered the “godfather of artificial intelligence,” has long argued that losing control over superintelligent AI systems could “lead to the extinction of humanity.” More recently, Bill Gates has highlighted the threat of economic disruption caused by AI. And Elon Musk has called for a halt to the AI race by at least 2023.
Public anxiety extends far beyond doomsday scenarios, fueled by concerns about job losses, data centers’ thirst for water and energy, the use of AI by children, and rising cybersecurity and national security risks. A recent Gallup poll found that seven in ten Americans now oppose building data centers near their homes. Only 32% of Americans trust artificial intelligence, compared to 87% in China. How should politicians, business leaders, and investors protect themselves from the risks associated with artificial intelligence? Even the most extreme AI scenarios are too serious to ignore, no matter how unlikely they may be. Politicians have a responsibility to protect the public, while business leaders have a fiduciary duty to safeguard the interests of shareholders and customers. Given the potentially catastrophic consequences, it may be wise to reduce exposure before a disaster strikes.
But hedging against low-probability existential risks is notoriously difficult. In the dystopian world that uncontrolled AI could bring, traditional hedging tools like bonds and put options would be of little use, as financial accounts, payment systems, and records could themselves be compromised. Nor can investors necessarily rely on governments to step in and restore market stability. If the very institutions that normally provide a cushion during crises are damaged, enforcing contracts and property rights would be nearly impossible. Real assets like land and water rights might retain their value in less extreme scenarios, but even these would be irrelevant in an AI apocalypse.
However, there are some practical steps that policymakers and business leaders can take. One is to establish greater security distances, isolating critical organizational networks and systems from insecure networks like the public internet. For now, they can take solace in the fact that today’s AI models remain too unstable and unpredictable to launch the kind of coordinated campaign that would threaten the survival of humanity. But their increasing sophistication and reach require an urgent reassessment of existing cyber defenses. Second, there is considerable opportunity for more aggressive regulation of AI, including rigorous security reviews and licensing requirements. Of course, regulating AI across borders will not be easy at a time of increasing geopolitical competition. No international body currently has the authority to oversee, let alone control, the development of emerging artificial intelligence in the United States, China, or elsewhere, especially when it comes to open-source models.
Nuclear arms control agreements provide a clear, if imperfect, precedent. In July 2026, more than 1,000 employees of advanced AI companies signed a letter calling for an “international effort” to develop “the technical and governance tools necessary to deliberately control the frontier of automated AI.” Some might scoff at the idea that the US and China, locked in a bitter rivalry, would collaborate on AI, but neither country has an interest in an uncontrolled technological race that ends in mutual destruction. The threat of extinction from AI may therefore pose the greatest challenge to collective action, forcing governments and companies to confront risks that no country or organization can address alone. Companies are already working together through trade associations and industry groups to mitigate cyber threats, and artificial intelligence gives them even more reason to do so.
The prospect of an iteratively self-improving AI, which underlies many doomsday scenarios, could make the usual crisis playbook obsolete. The resilience measures that helped institutions survive the COVID-19 pandemic will be of little use against powerful AI systems capable of manipulating records and disrupting critical infrastructure and business operations. Even today’s AI agents can quickly find ways to circumvent the controls designed to contain them, as the Hugging Face security breach demonstrated. Business leaders don’t have to wait for governments to act. They can slow or halt the deployment of AI and install kill switches that would allow them to shut down problematic systems if necessary. They may also support calls for a pause in the development of advanced artificial intelligence or to stop the pursuit of superintelligence.
Then there is the risk that powerful AI tools could fall into the wrong hands. Some of the greatest advances in AI development have come from countries with values and interests at odds with those of the West. Even without autonomous superintelligence, hostile states and terrorist organizations could use AI tools to cause immense harm. The fact that the leading minds in the AI industry see a real possibility that their products could cause the extinction of humanity in the near future should be a wake-up call for politicians and companies. If these apocalyptic predictions are even remotely plausible, conventional risk management strategies will not suffice.