Sam Altman says some AI accidents are 'unavoidable'

Sam Altman says some AI accidents are unavoidable, but it's how you learn from them.

Speaking at Salesforce's Dreamforce conference on Tuesday, the OpenAI CEO said tech companies should embrace a culture of transparent accident reporting, similar to that of the aviation industry.

"I do think that some degree of accidents are unavoidable with new technology," Altman said. "What I really care about is that we have got a great culture of accident reporting and learning."

He cited the Federal Aviation Administration and the National Transportation Safety Board as examples the AI industry should follow, saying the agencies' emphasis on investigating incidents has helped make flying safer.

"You know, accidents are going to happen," he said of the FAA's and NTSB's approach. "We're going to learn as much as we can and react from each one."

Altman's comments come as the AI industry faces renewed questions about whether its biggest labs are moving too quickly toward powerful autonomous systems.

Jacob Coxon, a researcher who quit Anthropic last week and previously worked at OpenAI, kickstarted a debate by saying that the companies were "racing straight to self-improving superintelligence and gambling with our lives."

Other AI safety researchers have since told Business Insider that advanced systems could one day evade human control or make cyber, biological, and other catastrophic threats easier to carry out — although they would face hurdles in accessing the physical world.

Altman acknowledged that AI doomsday fears are legitimate. "The world is right to be afraid of this," he said at Dreamforce, pointing to the possibility of a loss-of-control accident and the risk that a small number of AI developers could concentrate too much power.

Add BI in Google so our reporting is easier to find when you’re searching for what matters.

He said that the answer is not to abandon the technology, and that AI companies must instead ensure that "alignment and safety and monitoring" remain "way ahead of capabilities." He added that they should also be prepared "to slow down or stop" if they cannot do so.

His comments may also sharpen a central criticism from AI-safety advocates: that frontier labs say they recognize the stakes yet continue building more capable models.