The AI threat keeping scientists up at night

When news breaks, you need to understand what actually matters — and what to do about it. At Vox, our mission to help you make sense of the world has never been more vital. But we can’t do it on our own.

We rely on readers like you to fund our journalism. Will you support our work and become a Vox Member today?

At least, this is what a growing number of technologists and policymakers fear.

Last week, a lead researcher at Anthropic declared that there was a greater than 10 percent chance of AI “killing all humans” within the next decade. This nerve-racking forecast, combined with recent AI-initiated cyberattacks, have fueled calls for slowing the progress of artificial intelligence — a proposal that Anthropic and OpenAI subsequently embraced.

Not all prophets of AI doom agree about precisely how this apocalypse would come about. Many focus on scenarios reminiscent of The Terminator films, in which superintelligent machines conspire to exterminate humanity — perhaps in the same not-quite-intentional way that humans laid waste to the dodo, or passenger pigeon.

Synthetic pandemics are among the biggest existential risks facing humanity.

Governments can make us safer from both natural and artificial pathogens by investing more into pandemic preparedness.

But there is another, arguably more intuitive way that AI could bring about mass death, if not human extinction: by making it easier for people to develop and deploy catastrophic bioweapons.

Importantly, this hypothetical doesn’t require AI models to become conscious or superintelligent; it merely requires them to accelerate preexisting trends in biotechnology.

For those alarmed by this hypothetical, the threat AI poses is essentially twofold. First, when combined with technological progress in gene editing and DNA synthesis, chatbots could vastly increase the number of people capable of assembling dangerous viruses already known to medical science. And the more potential bioengineers there are in the world, the higher the odds that one will prove to be a diligent psychopath.

Second, AI could help highly trained biologists to discover — or engineer — a custom supervirus, which could then fall into the wrong hands.

Earlier this year, biologists warned that, with a few simple prompts, they’d gotten chatbots to dispense detailed instructions for engineering a treatment-resistant strain of an infamous pathogen, step-by-step protocols for recreating a once-pandemic virus, and advice on dispersing biological payloads via weather balloon.