How AI Could Make Software Verification More Practical

A recent project used Claude to prove selected security properties in compiled code from open-source libraries, but the approach has important limits.

Tech & Online SafetyOctober 23, 20264 min readHow Sureno writes this

The short version

Formal software verification uses mathematical proofs to check whether code meets defined properties; it gives assurance only about the properties and code examined. ¹
Anthropic reports that Claude helped prove selected security properties in compiled RISC-V code from 123 open-source libraries. ¹
The project also reported finding bugs that had escaped earlier detection, but it does not establish that all bugs were found or that the libraries are bug-free. ¹
AI-assisted static analysis tools are also being developed for agent skills and plugins, but these are a separate application and do not independently validate Anthropic’s results. ²

What is formal software verification?

Formal software verification is a way to use mathematics to check whether software satisfies clearly stated requirements. A requirement might describe a security property, such as a limit on what a particular operation can do. The proof applies to the property that was specified; it does not establish that the software is correct in every possible respect. ¹

This differs from ordinary software testing. Tests run a program on chosen inputs and check what happens. A formal proof aims to establish that a specified property holds under the assumptions and conditions built into the proof. Testing remains useful, but tests alone do not cover every possible input or execution.

Why has formal verification been difficult to use widely?

Writing a formal specification and a proof can require specialist knowledge and substantial time. Anthropic describes formal verification as having historically taken years of expert effort, which has made it impractical for many real-world projects. ¹

The work must also match the software being checked. In the reported project, researchers targeted compiled code at the assembly level, rather than relying only on a description of what the original source code was intended to do. That is a particular verification target, not a guarantee about every part of a software product. ¹

How could a large language model help prove software properties?

A large language model, or LLM, is an AI system trained to generate and work with text and other structured material. In this project, Claude helped produce formal proofs for selected security properties in compiled RISC-V code. The work was carried out with Theorem and covered 123 open-source libraries, according to Anthropic. ¹

If an AI can help create proof material more quickly, that may reduce some of the effort that previously made verification hard to scale. The important part is not that the model sounds confident: a proof must be checked against its formal rules and the property being claimed. The research source describes the project’s results, but does not provide enough information here to assess the full review process or independently compare its cost and speed with other methods. ¹

Does a proof mean the software has no bugs?

No. A proof establishes a stated property within a defined scope and under the assumptions used. It does not automatically show that the specification captures every security concern, that other parts of the system are safe, or that the software has no defects.

Anthropic says the work uncovered bugs that had not been caught despite extensive testing. That is a result reported by the project, not evidence that formal verification will find every hidden bug or that testing is no longer useful. The source does not give enough information to judge how often the method will find defects in other projects. ¹

What does this mean for developers and users?

For developers, AI-assisted proof writing could make it more practical to apply formal methods to some software that would otherwise be too costly to verify. It could fit alongside testing and code review, rather than replace them. The reported work is an example of what may be possible, not proof that the approach is ready or suitable for every codebase. ¹

Other tools apply formal analysis to different problems. For example, the SkillFortify project describes static analysis for AI agent skills and plugins, with security checks based on its formal model. That is a separate tool and scope; it should not be confused with proof of general-purpose software or taken as independent confirmation of Anthropic’s project. ²

For users, the development may eventually contribute to stronger assurances about particular software properties. It does not make a program automatically safe, and the source reports work on open-source libraries rather than a general guarantee for everyday apps or devices. How broadly the method works remains an open question. ¹

What we don't know yet

How much time and specialist effort the method saves compared with conventional proof-writing.
How consistently the approach works across programming languages, processors, and kinds of software.
How the proofs and reported bug findings will fare under independent evaluation.
Whether the method can be maintained as verified software changes over time.

A browser that helps you stay safe

Sureno is a Chromium browser with plain-language privacy settings and an assistant that only reads a page when you ask.

Get Sureno for Mac

Questions people ask

Does AI-assisted verification replace code testing?

The reported project does not establish that testing can be replaced. Formal proofs address defined properties, while testing checks selected program runs; each can reveal different problems. The research presents AI-assisted verification as a possible way to add proven security guarantees to development workflows. ¹

What did Claude verify in the reported project?

Anthropic says Claude helped prove selected security properties in compiled RISC-V assembly code from 123 open-source libraries. This describes a defined set of properties and code, not every part of those libraries or every possible security concern. ¹

What is static analysis for AI agent skills?

Static analysis examines code or configuration without running it. SkillFortify describes a tool that analyzes agent skills and plugins against a formal model, and says its checks cover multiple frameworks. It is a separate project from Anthropic’s formal verification work on compiled libraries. ²

Can a formal proof show that software is completely safe?

No. A proof supports a claim about specified properties under the proof’s assumptions and scope. It does not automatically show that the specification covers every risk or that unrelated parts of the software are free of defects. ¹

Sources

LLMs are making formal software verification feasible at a much larger scale — anthropic.com, 2026-10-22
GitHub - qualixar/skillfortify: First formal security scanner for AI agent skills & plugins. Static analysis, supply chain verification, SBOM generation. 22… — GitHub

Free tools for this

They run in your browser. Nothing is uploaded and there is nothing to sign up for.

QR code generatorMake a QR code for a link, some text or your Wi-Fi.Morse code translatorTurn text into Morse code, hear it, or decode it.Colour converterConvert a colour between HEX, RGB and HSL.