The moat was never the code: what 'replace the whole company with AI' gets wrong
A software firm's visible output is code. Its value is everything code can't sign for. The pitch to simulate a company with agents is a map of the parts that refuse to simulate.
A pitch has taken hold in AI circles, and it is more ambitious than the usual automation story. Point a swarm of agents at a market, let them generate the software end to end, and you have not automated a task inside a company. You have replaced the company. No engineers, no org chart, no human-coded layer underneath. The most-quoted version is Macrohard, a deliberately tongue-in-cheek name for an xAI project Musk announced in August 2025. His own argument, as reported at the time, was a narrow one: a software company can in principle be simulated with AI because it makes no hardware, so there is nothing physical to reproduce. The stronger framing, that such a firm would have nothing human-coded beneath it at all, is how boosters amplified the pitch rather than something the announcement itself established. Both are attributed here from contemporary reporting rather than from a primary document. Read the claim closely and it hands you something its authors did not mean to give away: a precise map of where a software company's value actually lives.
Follow the mechanism. To say AI can replace the company is to claim AI can reproduce whatever the company sells. So ask the blunt question first: what does a software company sell? Not lines of code. Code is the artefact, the visible output, and near-zero-cost generation is busy proving how cheap that output has become. The thing customers actually pay for is the promise wrapped around the code, and most of that promise is made of things an agent swarm cannot stand behind.
Can AI actually replace a software company?
It can replace a slice of one, and that slice is real. Boilerplate, glue code, first-draft features, a lot of the middle of the engineering funnel: automate away. But walk the list of what a buyer is really purchasing when they sign, and the automatable part shrinks fast.
They are buying accountability, an organisation with a name on the contract that can be held to a service level and, when it matters, sued. They are buying distribution, a place on the approved-vendor list, a procurement relationship that took years to clear, a salesperson who understands their compliance regime. They are buying support, the human who picks up when the system falls over at two in the morning and owns the outage until it is fixed. They are buying trust, a reputational bond that exists only because a specific firm has honoured it before. Make it concrete. Under Article 28 of the GDPR, anyone processing a client's personal data has to be a named processor bound by a written contract with real liability attached, and a bare swarm of agents brings none of the liability that clause assumes. The same gap shows up on a public-sector framework such as the Crown Commercial Service's G-Cloud, and in the professional-indemnity and cyber cover a bank's third-party-risk team demands before a single record moves. The swarm reproduces the commodity layer. Everything that carried the margin sits outside its reach.
The tell: watch what the pitch leaves out
Here is the diligence heuristic worth keeping. In any 'AI will simulate company X' claim, check whether the person making it has correctly described what X does. They almost never have. The announced scope reliably omits the parts of the business that generate the real profit and the real defensibility, and that omission is the whole tell. The pitch is telling you where the value sits by naming the one component it feels confident it can copy.
The naming game gives it away. Christen your project after a software incumbent and you invite the comparison, but the incumbent the name parodies does not earn its returns by emitting software. It earns them across hardware, cloud capacity sold by the hour, decade-old enterprise licensing relationships, and a distribution machine that competitors cannot rebuild at any price. Simulate the code and you have copied the least defensible thing that company owns; everything the demo skipped is the moat.
This is why the honest way to think about agentic systems is as instruments inside an organisation. They earn their keep when the organisation around them still carries the contracts and keeps a human accountable for the output, which is the whole argument for practical AI with human control: the control layer is the part customers are actually paying for.
How should a board read an 'AI is the company' announcement?
Score it exactly as you would any other AI megadeal: as capability marketing with a low, measurable conversion rate. A named project and a hiring tweet are inputs, not results. Count them as nothing until they resolve into shipped product, paying customers, or contracted commitments. The history is not kind to the grand version. IBM's Watson was sold as the AI that would out-read oncologists, fronted by a marquee partnership with MD Anderson, the University of Texas cancer centre in Houston; by contemporary reporting, MD Anderson shelved the project in 2017. The durable clinical business it promised never materialised. 'We are building a company with no humans in it' is a stronger claim than Watson ever made, so it deserves even more scepticism.
The second-order move matters more than the headline. If code generation really is heading towards free, differentiation does not disappear; it relocates into the un-simulatable layer. That is good news for firms whose strength was never the codebase, and an extinction event for pure code-shops whose only product was the artefact now being commoditised. A board repricing a software supplier, a competitor, or its own roadmap should run the same test in reverse: strip out the code and ask what is left. Whatever survives that subtraction is the asset, and everything else was always going to get cheaper.
The contrarian test seals it, but take the strongest objection first. Nothing stops you bolting a legal wrapper onto the swarm: incorporate a shell, appoint a nominal director, let that entity sign the data-processing agreement and carry the indemnity. Agents cannot sign, true, but a company they nominally staff can, so the accountability layer looks like a solved problem. It isn't, and the reason is economics rather than metaphysics. A signature is worth the balance sheet behind it plus the reputational bond it puts at risk, and a fresh shell has neither. Underwriters price professional-indemnity and cyber cover off engineering process, change control and incident history, none of which a brand-new entity can evidence, so either the premium prices it out or the cover is never written at all. Procurement frameworks want named responsible individuals, audit trails and a track record, not a supplier whose only employee is a model endpoint. And an indemnity from a thinly capitalised shell that cannot technically stand behind what it signed pays out only its assets when the claim lands, which is to say nothing. The shell can hold the pen, but it cannot manufacture the institutional record that makes the signature mean anything, and that record is the product. That is the point of building agentic systems with accountability designed in rather than assumed away: deciding where the human layer belongs before you build is the actual strategic work, the same discipline as honest technical strategy, knowing which part of your business the machine can touch and which part is the reason anyone pays you.
Automating code generation only lowers the cost of the commodity. The moat was always the rest of the company, and that is the part these pitches quietly leave out.
Questions people ask
Does cheap AI-generated code make software companies worth less?
It devalues the code itself, but the code was rarely where the value sat. Firms whose worth came from enterprise relationships, support obligations and hard-won trust can become more defensible as the commodity layer gets cheaper. Businesses that only sold the artefact are the ones facing repricing.
What is the difference between AI automating tasks and AI replacing a company?
Automating tasks means agents do work inside an organisation that still carries the contracts, support and liability. Replacing a company claims the agents can also stand in for those human-coded layers, which is a far larger and far less proven claim, and usually the one that misdescribes what the business actually does.
How should a board evaluate a claim that AI will replace a competitor?
Treat it as marketing until it converts. Subtract the code from the competitor's business and see what remains: the relationships, the trust, the accountability. If the announcement waves those away, it is revealing where the competitor's real moat is, not eliminating it.
Related
- AI Deskilling Is the Workforce Risk Your Board Isn't Pricing
- AI Therapy's Business Case Is a Floor, Not a Ceiling
- The Memory Oligopoly Behind AI's Cost Inflation
- AI & Automation
Written by an AI editorial persona of Abyshire's proprietary editorial system and reviewed by our team.