0.978 AUC, 12% precision

Both numbers describe the same model on the same day. Neither is a mistake, and the gap between them is the most expensive thing I know about benchmarks — because almost nobody prices it in.

A benchmark hands a model a balanced problem: roughly as many positives as negatives, curated and labelled. Area under the ROC curve asks a question suited to exactly that world — draw one of each; how often do you rank them correctly? Nearly always, it turns out. It is a genuinely good model.

Then you take it somewhere the interesting class is rare. The model keeps its skill. The world changes underneath it.

Where the collapse comes from

Precision — the share of your alarms that are real — is not a property of the model alone. It is a property of the model and how often the thing occurs. Bayes' rule, in the only form that matters here:

precision = TPR·π ⁄ (TPR·π + FPR·(1−π))

where π is prevalence. The term that kills you is FPR·(1−π). When positives are rare, 1−π is nearly everyone, so even a small false-positive rate is applied to a very large population — and that flood of false alarms can easily exceed the entire population of true ones.

A real predictor, reprojected

Taking the leading model on a public clinical variant-effect benchmark at its published operating characteristics, and holding the model completely fixed:

Nothing in the leaderboard warns you. The ranking does not change. The model does not degrade. What changes is the base rate — and the base rate never appears in the metric that gets published, quoted and funded.

This is not a genomics problem

It is the shape of every screening claim in machine learning. Fraud detection. Security triage. Content moderation. Insider-threat scoring. Defect finding. Anywhere the interesting class is rare, a benchmark score describes a world with a suspiciously convenient class balance, and deployment does not live there.

I have been on the wrong end of it with my own work: a bug-finding pipeline that looked excellent on a held-out set and was close to useless against a real codebase — because real codebases are mostly not vulnerable, and the pipeline's small false-positive rate met a very large number of clean files.

The fix costs one line

Before you quote the score, state the prevalence you expect in the field and recompute what fraction of your alarms will be real at that prevalence. It belongs in the abstract, beside the headline number — not in a supplementary table where nobody meets it.

And if the answer is uncomfortable, that is the finding. A detector that fires mostly on innocents is not a bad model; it is a good model deployed at a base rate its threshold was never chosen for. That is fixable — by moving the threshold, by triaging in stages, by narrowing who gets screened — but only once someone computes the number.