On a vital-signs monitor, a flat line means death. On a test suite, it means the check has no pulse — and the screen still reads PASS.
is_authorized returns
True on every path — there is no input that bends it toward REJECT. It
isn’t a weak guard; it’s a guard with no pulse. A healthy check’s trace dips
sometimes. This one never will — and nothing about watching it run would ever tell
you that.
A missing check is an absence you can notice. This is an absence wearing the clothes of a result.
A wrong number is a claim, and claims can be argued with. A check that cannot fail makes no claim — it manufactures agreement. The build goes green. The filter returns rows. The monitor reports success. Every signal a careful reviewer scans for is present, and each one is hollow.
You cannot audit a measurement that was never taken. You can only notice it was never taken — and noticing means asking a question the output will never prompt.
Each is from real code written on one day. For each: could running it ever produce a “no”? Decide, then reveal.
# which job postings are relevant to us def is_wanted(role): return role.is_remote or "remote" in location
role.is_remote is a boolean, compared to the word
“remote” — always truthy the way it was used. The filter accepted all 2,485
roles. “Nothing matched” read exactly like an empty job market.
# confirm two datasets match before trusting the result z = (obs - mean) / sd if sd > 0 else 0.0 verdict = "MATCHED" if abs(z) < 1e-9 else "DIFFERS"
When the datasets differed by a fixed amount, their spread was zero — so
sd = 0, and the divide-by-zero guard forced z = 0,
which reads as MATCHED. It printed MATCHED over two visibly
different numbers. The guard against a crash had become a guarantee of passing.
# restrict a null model to swap only within a group def allowed(u, v): return group(u) is not None and group(v) is not None
Every node had a group, so the predicate was always true. The “constrained” null was a byte-for-byte copy of the unconstrained one — and its results, differing only by noise, looked exactly like two methods agreeing. A syntax checker can’t catch this: it is vacuous only because of a fact about the data.
# the same check, after the fix def allowed(u, v): return group(u) == group(v) # same group — not just any group
Now two nodes in different groups return false — the constraint refuses, and you can watch it refuse. That is the entire difference between a check and its costume: you have seen it say no.
Care doesn’t scale and doesn’t survive a deadline. A short list, run mechanically, does.
After any split or filter, assert the pieces sum back to the input. A deficit that repeats every run is a systematic drop, not noise.
Feed it the input it exists to reject; confirm it says no. Derive the verdict from the values — never from a statistic a zero denominator can silence.
Every filter, gate and guard should report its rejection count. Zero rejections, ever, means it is decoration.
“0 of 2485.” “0% untraceable.” “100% pass.” A perfect number is the cheapest smoke alarm there is — suspect it first.
From an independent eval-integrity practice — auditing the numbers people compute and then believe. The full method is in the book, Measured, Not Believed.