Anthropic and Theorem automate software safety with massive proof generation

The central tension in modern software development lies in the gap between rapid code generation and rigorous safety checks. For decades, proving that software runs without error under all possible conditions was so expensive that it was reserved for niche applications like aircraft avionics. A team of specialists once required twenty person-years to formally verify the small operating system kernel seL4. Now, Anthropic and its partner Theorem have compressed that timeline from years to weeks, suggesting that the era of expensive, manual verification may be ending.

The collaboration focused on verifying existing software rather than writing new applications. Anthropic's Frontier Red Team directed Claude to translate crash tests from 123 open-source libraries in Google's OSS-Fuzz project into mathematical problems. The model analyzed compiled machine code and generated approximately 180 million lines of proof code in a matter of weeks. An independent verification program named Rocq checks these proofs, allowing the process to operate with minimal human review. The researchers describe this as fully autonomous proof generation, though the underlying formal foundations and tools were developed by humans at Theorem.

The practical impact of this automation became evident when Claude identified hundreds of bugs that conventional testing had missed. These defects often remained hidden because they only appeared under specific conditions, such as unusually large input files or unique access sequences. In one notable instance, the team discovered as many security bugs in a widely used PDF reader within a single week as are typically reported for that software in an entire year. In another case, the system detected a logical error that had persisted undetected for over a decade. Even when a complete proof fails, the attempt still surfaces these real-world issues.

Despite these successes, the approach has strict practical limitations. Anthropic clarifies that formally verified does not mean error-free. The proofs hold only under specific constraints, including a particular processor architecture, single-threaded programs, and inputs under 64 MB. Since these libraries often run on varied hardware and configurations in the real world, the verification is conditional rather than absolute. Additionally, the computational cost is significant, with individual proofs requiring hundreds of millions of tokens. Anthropic did not disclose the total project costs, but the effort highlights the heavy resource demand of this method.

The economic and strategic implications are shifting how developers view AI-generated software. If AI can verify its own code, the time and cost spent on manual review decrease, saving resources across the entire coding process. Smaller test runs indicate that these proofs are reusable and that newer models are becoming more efficient. For example, Opus 5.5 solves half the proofs with about 40 percent fewer tokens than its predecessor. This efficiency gain supports the argument that formal verification can become a routine part of the development cycle, similar to standard testing.

Looking ahead, the project suggests a fundamental shift in software automation. The industry question is moving from whether AI can program to whether it can prove its code is safe. Anthropic expects this process to become as practical as standard testing in the near future. Theorem’s roadmap already targets critical infrastructure components such as glibc, OpenSSL, and cURL. If successful, this method could extend beyond coding tools, potentially transforming how safety is established in critical digital systems.

The next steps involve scaling this autonomous verification to more complex and diverse environments. As the technology matures, the focus will remain on reducing the gap between mathematical correctness and real-world reliability. The success of this initial phase sets the stage for broader adoption in sectors where software safety is paramount.