Going Closed-Source Won't Stop AI Finding Your Bugs. Auditing First Will.
When a fast-growing startup pulled its code private to dodge AI attackers, it misread the threat. The danger it was hiding from has little to do with who can read the source, and everything to do with code nobody has ever audited. That reasoning is the part likely to spread.
A quiet assumption props up most software estates: nothing bad has happened yet, so the code must be sound. Two demonstrations from the last decade should retire it. In 2016, DARPA's Cyber Grand Challenge put seven autonomous systems on a DEF CON stage and had them find, prove and patch vulnerabilities in compiled binaries with no source and no human in the loop. Eight years on, in November 2024, Google's Big Sleep agent found a previously unknown, exploitable memory-safety flaw in SQLite, which its researchers called the first such bug an AI agent had discovered in real-world software. Neither result is a press release about what might come; both already happened.
What follows from them is prediction, and I will flag it as such. The cost of pointing this capability at old, never-audited code is collapsing toward zero. Legacy web apps, unpatchable routers, IoT firmware, industrial controllers, even microcode: the stuff that has run untouched for years because nobody had a reason to look. Now the looking is getting cheap, and a decade of accumulated, unexamined risk starts to look like something a motivated attacker can work through in days rather than years.
The early reflex, in at least one visible case, has been to reframe openness itself as the danger. If attackers can point a model at every public repository and harvest flaws at scale, the argument runs, then visibility is the attack surface, so pull the shutters down. Cal.com did exactly that, closing its commercial codebase with its chief executive arguing that AI attackers exploit source-code transparency. For now it reads as a lone example rather than a movement. The panic is understandable, but the reasoning is where the damage sits, because it is the reasoning, not the one company, that other businesses are about to copy.
Does hiding your source code protect you from AI attackers?
Start with the mechanism. An attacker needs behaviour it can probe, and source is only the fastest route to understanding that behaviour. Closed binaries decompile. Firmware images pull apart and emulate. Fuzzers throw millions of malformed inputs at a running service and record what falls over, without ever seeing a line of source. Vulnerability research predates open source and spent decades working blind against proprietary systems, so what AI changes is the reach of that older craft, and it extends it squarely into binaries and firmware rather than only into public repositories.
Concede the honest version of the closed-source case, because it is real: obscurity does raise attacker cost at the margin. Reading source is faster and cleaner than reconstructing logic from a decompiler or inferring it from a heap of fuzzing crashes, so for an authentication bypass or an injection path, taking the source away does buy a genuine time premium. Two things spoil the trade. The premium keeps shrinking as models get better at binary and firmware analysis, which is exactly the frontier the DARPA Cyber Grand Challenge marked out back in 2016, when its winning system worked directly on compiled binaries with no source at all. And you pay for that premium by evicting the friendly researchers who might have reached the bug before an attacker did, which means the delay you buy is the one commodity an already unpatched estate has in plentiful supply.
So closing the source removes one convenience for the attacker while forfeiting something real for the defender: the many eyes. Open code is auditable code, and once the auditor can be a model running at scale, the defensive value of readable source rises rather than falls. The trade closes a shrinking liability and opens a growing one.
The variable everyone is arguing about, visibility, is not the exposure at all. The exposure is that the code was never audited in the first place. A private repository full of unexamined flaws is nearly as vulnerable as a public one, short of the chance that a friendly researcher reaches the bug before an unfriendly one does.
The capability is symmetric, so speed is the only edge
Here is the uncomfortable part. AI-assisted vulnerability discovery does not care whose side runs it. The same model that surfaces the flaw in your dependency tree surfaces it for whoever points it there first, so the strategic picture collapses to a single lever, which is time. The durable advantage belongs to whoever audits a system first, because the first competent scan turns an unknown risk into either a patch or an exploit, depending on who is holding the keyboard when it lands.
That is why delay is the expensive option. Treating adversarial AI auditing as something to procure next budget cycle quietly assumes your adversary is on the same schedule, and they are not. The workable posture is to run the audit against your own code, your dependencies and your connected devices now, on the assumption that someone else can and eventually will. We help clients build exactly this kind of human-supervised assurance into the way agentic systems are designed and operated, because the defending is done by the audit, and no source-code policy substitutes for it.
What if you cannot audit everything?
Run the audit first assumes you can, and plenty of organisations cannot. If you are sitting on four hundred firmware images you will never rebuild, or a fleet of appliances whose vendors have vanished, a race decided by time is one you lose on some fronts before it even starts, and pretending otherwise is how security theatre gets a budget line.
So triage honestly. Rank by reachability multiplied by blast radius, not by how uncomfortable a given system makes you feel. Anything internet-facing, or one hop from authentication, goes to the front of the queue, because that is what an attacker's scan reaches first as well. For the unpatchable remainder, drop the assumption that patching is the only control on offer: isolate those devices onto segments that cannot reach the internet or your crown jewels, filter their egress, and instrument them so that an exploit you cannot prevent is at least one you can see. You will lose some of these races, so fund the detection and response that decides what a lost race actually costs. Where you can move first, auditing wins outright; where you cannot, containment and monitoring carry the weight, and an honest "we could not audit it" should trigger a compensating control rather than a shrug.
Why you cannot price this from a vendor's press release
None of this means swallowing the marketing whole. Vendors now advertise how many fundamental bugs a security-tuned model has surfaced, and those totals are close to unverifiable. When Anthropic discussed its model's alleged findings, it could disclose only a fraction because most of the flaws remained unpatched, which makes independent confirmation genuinely hard. A headline number cannot tell you how much of the work was the model and how much was the humans steering it, nor whether the bugs were real, novel, or actually reachable. Ask instead for a methodology, a reproducible sample of findings, and evidence that the model rather than the analyst did the discovering. Price the capability from the brochure and you are paying for a story, a caution worth applying to every security-AI pitch that lands this year.
What the estate actually looks like
The flood of critical disclosures will not land evenly. It will concentrate in the systems nobody wants to touch: the firmware on a device that shipped in 2014 and will never get an update, the internal service whose author left three reorganisations ago, the dependency buried six levels deep in a lockfile. Thinner engineering teams, cut in the same cost squeeze that orphaned those systems in the first place, inherit a widening audit gap with fewer hands to close it.
The problem itself is old, and what has changed is only the price. Latent flaws have always sat untouched for years until one researcher made them practical, the way Dan Kaminsky turned known DNS cache-poisoning weaknesses into a demonstrably practical attack across many implementations at once. What AI does to that discovery process is what the assembly line did to manufacturing: it industrialises it.
Strip away the open-versus-closed debate and the instruction is plain. Inventory what you run, including the firmware and the dependencies you have never read. Point the same class of tooling an attacker would use at your own estate, under human control, and fix what it turns up before someone else files it. The open-versus-closed question is a policy argument you can hold at your leisure; the audit is the part that decides whether you are still standing. Build it into your technical strategy now, while finding your own bugs first is still an advantage on offer.
Questions people ask
Is open source less secure than closed source now that AI can scan code?
Not straightforwardly. Attackers can still extract vulnerabilities from closed binaries and firmware through decompilation, fuzzing and emulation, so hiding the source raises their cost and buys time rather than removing the exposure. The exposure is unaudited code, and open source at least keeps that code readable by the defensive tooling and researchers who might reach the flaw first.
How do I start an AI code security audit of my own systems?
Begin with an inventory that includes dependencies and any firmware on connected devices, since those are where untouched flaws concentrate. Then run adversarial vulnerability discovery against your own estate under human supervision, prioritising internet-facing and authentication-adjacent systems, and treat any finding as something to patch before it is disclosed elsewhere. Where a system genuinely cannot be patched, contain and monitor it instead.
Can I trust a vendor's claim about how many bugs its security model found?
Not on the headline number alone. Many such totals are unverifiable and cannot be attributed to the model rather than the researchers guiding it. Ask for a reproducible methodology, a sample of the actual findings, and evidence the model did the discovery before you price the capability.
Related
- The Sovereignty Premium: Why Sovereign AI Solutions for Enterprise Are Winning on Access, Not Speed
- Washington Put Its Own AI Lab on a Risk List. That Changes What AI Vendor Lock-In Means
- Why Enterprise AI Pilots Fail to Scale: It's Trust, Not Capability
- Security & Trust
Written by an AI editorial persona of Abyshire's proprietary editorial system and reviewed by our team.