The moat isn't the data, it's the cleaning

The first thing that goes when you actually work in a data domain is the fantasy that access is an advantage. The feeds exist, the sources are shared, and your competitors are pulling from the same well you are. On the raw numbers, you're all identical. Competing on "we have the data" is competing on the thing that makes you interchangeable. The differentiation has to come from something you do to the data after you get it, because the getting is table stakes.

Trust is what you can build that the next person can't just copy. A user who has been burned by wrong numbers elsewhere and finds that yours are consistently right develops a reliance that's genuinely sticky. That reliance isn't in the data — it's in the reputation of your data, earned over time by being right when others weren't. Commodities compete on price and availability; trust competes on nothing, because there's no substitute for it once someone depends on you having gotten it right.

If your advantage is having the data, you have no advantage — everyone has it. The advantage is being the one people trust to have made it correct.

The reason cleaning is a moat is that it's genuinely awful work and most people won't sustain it. Deduplicating records that refer to the same real thing in slightly different ways. Catching the impossible value that slipped in. Reconciling two sources that disagree. Finding the orphaned record, the phantom entry, the stat that's off by an order of magnitude because of a units mistake upstream. None of it is exciting, all of it is necessary, and it never ends because the raw data keeps flowing in just as dirty as before. It's janitorial work, and janitorial work is exactly what ambitious people skip.

That skipping is the opportunity. Because the work is tedious and invisible — nobody sees the errors you caught, only the errors you missed — most operators under-invest in it, cut corners, or automate it badly and move on to something shinier. The one who takes it seriously, who treats validation and reconciliation as the core product rather than a chore around the edges, builds something the corner-cutters can't match without doing the same unglamorous slog. The moat is dug by being willing to do the boring thing thoroughly, forever.

The mistake that makes cleaning fail is treating it as a phase — import the data, clean it once, ship it. Real data sources are living things; they keep producing new errors, new edge cases, new ways to be wrong. So cleaning can't be a step you finish; it has to be a standing property of the system, with validation baked into the pipeline so that bad data gets caught on the way in, every time, not discovered by a user later. The stance is that data is guilty until proven clean, and the proving is continuous.

That means building the checks into the ingestion itself — the impossible-value guards, the deduplication, the reconciliation between sources — so that trustworthiness is enforced by the machinery rather than by a periodic human audit. When the pipeline itself refuses to pass along data it can't validate, quality stops being something you hope for and becomes something the system maintains. It's the same principle as everything I build: don't rely on vigilance for the thing that has to be true; build the truth-keeping into how the thing works.

It's easy to say "validation" and leave it abstract, so here is what it looks like in practice. In my ATLAS data I keep a validator that walks every record before any of it ships and refuses to accept the ones it can't stand behind. It checks referential integrity — a model that points at a company that doesn't exist is an orphan, and orphans get flagged, not served. It hunts duplicate IDs, because two records claiming the same identity is a silent corruption waiting to surprise you downstream. It even checks that a file's name agrees with the ID the file claims to be, since a mismatch there means something drifted. The findings sort into errors and warnings: errors are things that must be fixed before the data is allowed through, warnings are things a human should lay eyes on. The validator, not my attention on a given afternoon, is what decides whether the data is clean enough to pass.

The sports side is the same discipline at a different scale. The warehouse under my sports work — OZ Intelligence — tracks well over a hundred and seventy thousand games, and the number isn't the point; the point is that each one has been cross-checked against another source rather than trusted on arrival. When two feeds disagree about a single score, something has to decide which is right, and that reconciliation is the actual product. The value lives precisely in the gap between what a feed claims and what's true — closing that gap, row by row, is the work that can't be downloaded.

A moat that can be copied in an afternoon isn't a moat. Raw data can be re-downloaded that fast; a reputation for being right cannot. Trust is the rare asset that compounds — every month you're correct when others weren't adds to a reliance a competitor can't purchase, only rebuild slowly by grinding through the same tedium you did. And the asymmetry runs downhill, too: being wrong once can be disqualifying, especially in a domain where people make real decisions on your numbers. The upside accrues in small quiet increments; the downside arrives all at once. That shape is exactly what makes the careful, boring work worth more than it looks.

It also means the leverage points are counterintuitive. The instinct is to invest in the visible surface — more features, prettier presentation — because that's what a demo shows. But the surface is the copyable part, and pouring effort into it is pouring effort into the commodity. The effort that compounds goes underneath, into the cleaning that no screenshot can capture. I'd rather ship a plainer product on data nobody can match than a beautiful one on data everyone already has.

The thing that surprised me most, building on top of a data layer I'd cleaned obsessively, is that the boring foundation turned out to be the most valuable thing I'd made. The features on top — the interfaces, the presentations — were replaceable and copyable, and a whole site on top of the data can be relaunched for the price of a configuration change. The clean, validated, trustworthy data underneath was the part that was genuinely hard to reproduce, because reproducing it meant doing all the tedious work I'd done, and few people will. The moat wasn't the shiny part; it was the foundation nobody could see.

So when I think about where a durable advantage actually comes from in a data business, it's not the acquisition and it's not the presentation — it's the middle, the cleaning, the part that reads as overhead on a pitch deck and is actually the whole thing. I'll admit I've come to genuinely like the work: the small, real satisfaction of a dataset that's correct, the quiet confidence of numbers you can stand behind. That taste for the janitorial part isn't a quirk. In a world where everyone has the same raw data, it's the competitive advantage — because it's the one thing that can't be downloaded.