Can AI models spot decades-old flaws in open-source code?

The short version

• Anthropic researchers used Claude to audit open-source codebases.
• The AI found 493 security flaws across 25 distinct projects.
• One vulnerability had existed undetected for 21 years.

The details

Anthropic’s Chris Rohlf reported that the company’s large language model, Claude, successfully identified 493 security issues within 25 open-source software projects. According to Techmeme, the audit covered more than 5 million lines of code written in C, C++, Java, PHP, and Python. The research highlights the capability of frontier AI models to discover and potentially exploit security vulnerabilities in complex codebases. One particular flaw was significant for its longevity, having gone unnoticed for 21 years prior to this automated detection. This finding underscores how AI tools can uncover legacy issues that human reviewers may have missed over decades of development.

The context

Over the past year, Anthropic has published data demonstrating that frontier AI models are more than capable of discovering and exploiting security vulnerabilities. This latest report adds to that body of evidence by showing practical application across multiple programming languages and large code volumes. The ability of an AI to parse millions of lines of code quickly allows for a scale of auditing that is difficult to achieve manually. It suggests that automated AI review could become a standard part of the security lifecycle for open-source maintainers who lack the resources for comprehensive manual audits.

Next steps

Teams should consider how AI-assisted auditing fits into their current security workflows. Organizations relying on open-source components should evaluate their vulnerability management processes. Developers may want to explore integrating AI code analysis tools into their continuous integration pipelines. Security teams should monitor vendor announcements for new AI-based security features. Maintainers of open-source projects should review their historical code for similar legacy vulnerabilities that may have been overlooked by previous human-only review processes.

What defenders should look for

Security operations teams should monitor for unusual access patterns in source code repositories, especially automated scans from new IP addresses. Check alerts for unexpected vulnerability reports from AI-driven tools, which may flag issues differently than traditional scanners. Review commit histories for sudden spikes in security-related patches. Ensure that vulnerability management systems can ingest and prioritise findings from AI-generated reports. Verify that all flagged issues are validated by human analysts before deployment. Monitor for lateral movement attempts that might exploit newly discovered flaws.

What to do and how to stay safe: Anthropic

• Audit your open-source dependencies for known vulnerabilities.
• Implement code review processes that include AI-assisted analysis.
• Monitor security advisories for newly discovered legacy flaws.
• Validate AI-generated vulnerability reports before applying patches.

Step-by-step guide: How Virtual Patching Works: The Mechanics Behind Inline Protection

General security guidance from the Honeypot Daily newsroom. It is not confirmed advice from the organisations named in this story.

Frequently asked questions

How many projects did Claude scan?

Claude scanned 25 open-source projects.

Which programming languages were involved?

The scan covered C, C++, Java, PHP, and Python.

How long was the oldest flaw hidden?

One vulnerability went unnoticed for 21 years.