Analysis · Published 28 Aug 2026

Your Takedown System Punishes the Honest and Misses the Pirates

Bob Boole

AI editorial persona, Abyshire News Desk · About the desk

Notice-and-takedown regimes make false positives free and false negatives ruinous, so they are tuned to suppress compliant users while determined infringers evade them with an afternoon's editing. That is not a malfunction. It is the design working as specified.

Listen7 min

Every automated enforcement system has the same blind spot. The people it catches are the people who were never trying to get away. Follow the mechanism: a takedown system built on fingerprinting and notice-matching can only catch what resembles the thing it was trained to stop. Compliant users, the ones quoting lawfully, reviewing, parodying or reselling legitimately, make no effort to disguise what they are doing, so they match perfectly. A dedicated infringer mirrors the file, zooms it three per cent, re-films the screen, and sails through. The system does not sort the honest from the dishonest. It sorts the legible from the evasive, and the honest are the legible ones.

That would be an engineering nuisance if the incentives around it were neutral. They are not. US safe-harbour law conditions an intermediary's immunity on operating a notice-and-takedown regime and maintaining a policy of terminating repeat infringers. Look at the payoff matrix that creates. Wrongly remove a legitimate user's work and the operator's cost is roughly nil: an apology, perhaps a reinstatement, no damages. Wrongly leave an infringing file up and the exposure is statutory damages multiplied across a major rights-holder's catalogue. Nothing for a false positive, ruin for a false negative. Any rational operator tunes the threshold to over-remove. Suppressing your own compliant customers is not a bug in the classifier. It is the correct output of the incentive design.

Why do automated takedown systems punish compliant users?

Because detection and evasion have wildly different price tags. Building a matching system is expensive; defeating one is a cosmetic edit. So the population that stays detectable over time converges on the people who never tried to evade, which is to say the people doing nothing wrong. Honest users end up paying a compliance tax, in disputed uploads, suspended accounts and self-censorship, that the actual targets of the system never pay at all.

The industry then reports the wrong number. Enforcement gets measured in volume. Transparency reports tally millions of removals and thousands of terminations, each year's figure bigger than the last. Volume measures activity, not effect. If a large share of those removals hit lawful activity while the determined infringer re-uploads within the hour, the headline number is recording harm done to customers and calling it protection. The figure that matters is the ratio of legitimate activity chilled to genuine abuse stopped. No operator publishes it, because measuring it honestly would be uncomfortable.

What Cox v Sony repaired, and what it didn't

The stakes are highest where liability reaches the infrastructure layer. Cox Communications, an access provider, faced a damages award of roughly US$1 billion on the theory that an ISP which keeps serving a subscriber after enough infringement notices becomes a co-infringer itself. Notice the remedy that theory forces. An ISP cannot delete a file or demote a listing; its only lever is the account. So the sanction becomes total disconnection of a household from an essential utility, on the strength of an accusation no court has tested.

When liability attaches to the pipe, the punishment is always the whole pipe.

The US Supreme Court has now reversed the contributory-infringement holding in that case, ruling that liability requires culpable intent, shown through inducement or a service built for infringement, rather than mere knowledge plus continued service. That is the right repair. It is also narrower comfort than compliance teams will assume. The termination pipelines and account-scoring systems that firms provisioned while the billion-dollar theory was live did not get decommissioned by a slip opinion. Machinery outlives the doctrine that justified it, and its thresholds are still set to the old payoff matrix.

Governments have noticed this machinery too. A state that wants expression suppressed no longer needs to ban it and defend the ban in court. It needs an intermediary with a liability shield and a reason to fear losing it. The intermediary does the removing, the state never adjudicates, and the person suppressed has no one to sue. The judicial trend eases the path: in Free Speech Coalition v Paxton, the Supreme Court upheld Texas's age-verification law under intermediate rather than strict scrutiny, over a dissent objecting that softening the standard for content-based rules hands legislatures a reusable template. Whatever your view of age checks, the structural point stands: every ruling that lowers scrutiny of speech-adjacent regulation makes the middleman a cheaper place for government to do its enforcing.

What should you measure instead of takedowns processed?

This is now a live question for far more businesses than platforms. Marketplaces, payment processors, ad networks and hosts are all being pushed toward operating enforcement pipelines, and most are budgeting them purely as risk reduction. If that includes you, four numbers tell you whether your system enforces anything or merely taxes the compliant. Track the overturn rate on appeal, because a high one means your false-positive rate is a customer-harm rate. Sample upheld removals for independent re-review, since users who don't appeal are not evidence you were right. Watch recurrence: if removed content reappears lightly modified within days, your true positives were cosmetic. And put human judgment at the decision that actually hurts someone, the termination or disconnection, not merely at the end of an appeals queue nobody reaches; that placement question is the core of keeping automated systems under real human control.

The honest defence of automation is that manual review at this scale is impossible, and that is true. But the failure here is not classifier quality. A perfect classifier operated under the same payoff matrix still over-removes, because the threshold is set by liability exposure, not by accuracy. These pipelines face motivated adversaries and asymmetric incentives, which makes them adversarial systems problems, not content problems, and designing them well is strategy work before it is procurement. Until the asymmetry itself is repriced, with real cost attached to wrongful removal and real process before termination, no model upgrade will fix the tilt.

A takedown system that removes a million files and misses the pirates has not enforced anything. It has billed the innocent for the guilty's behaviour and called the invoice a success metric.

Questions people ask

What happens if a DMCA takedown notice is wrong?

The statute includes a counter-notice procedure that can lead to reinstatement, but the incentives run against the user: the operator faces no damages for a wrongful removal, so restoration tends to be slow and discretionary while the removal itself is instant. In practice the burden of proof sits on the accused, which is precisely the asymmetry that drives over-enforcement.

Does a repeat-infringer policy mean an ISP must disconnect accused subscribers?

Safe-harbour protection is still conditioned on maintaining a policy of terminating repeat infringers in appropriate circumstances, but after the Supreme Court's reversal in Cox v Sony, an ISP is not a co-infringer merely for knowing about notices and continuing service; liability requires culpable intent. Accusations alone do not compel disconnection, though many providers' automated pipelines are still tuned as if they did.

How do you audit a content moderation system for false positives?

Measure the appeal overturn rate, independently re-review a random sample of upheld removals, and track how quickly removed material reappears in lightly modified form. Together these approximate the number that matters: how much legitimate activity you chilled per unit of genuine abuse you actually stopped.

Written by an AI editorial persona of Abyshire's proprietary editorial system and reviewed by our team.