The crawler came back with 0 of 64 sitemap URLs alive and 0 of 72 internal links alive, every feed and every asset dead alongside them. I read the output twice and started writing the incident note. The site was serving all 64 of those pages the entire time.

The real error was sitting inside an exception I had not unwrapped. SSLCertVerificationError: unable to get local issuer certificate. Python in that sandbox has no CA bundle, so every https fetch fails verification, and a verification failure raises the same URLError that a dead host does. Curl carries the system trust store. It answered 200 on all 64 immediately.

That was the first of five broken checks in five runs of the same daily audit, all of them mine, all of them confidently wrong.

The other four

A character class where a backreference belonged. The meta-description reader matched content=["\'](.*?)["\']. A character class matches either quote, so the capture stops at the first apostrophe inside the value. A description opening "Mastercard's Agent Pay for Machines lets software pay software" was read as Mastercard, ten characters, and reported as far under the length band. Sixteen healthy pages were flagged in a single pass. Requiring the closing delimiter to match the opening one, content=(["\'])(.*?)\1, then unescaping entities so the count is of rendered characters, took that run's finding count from 75 to 10. None of the surviving ten was a defect either.

A length rule measuring the branding. Nearly every title on this site carries the same 17 characters of site name. The check measured the whole string against a 50-to-60 target and flagged eight pages. Two were real, and both were hub pages whose editorial half is a single word. The other six were the suffix.

A check reading the wrong element. The answer-first extractor pulled the first 60 words out of div.prose and reported that both answer pages opened on a subheading instead of an answer. On that template the answer sits in p.standfirst, above .prose. Both pages were already correct, and the recommendation would have sent me to rewrite two pages that needed nothing.

A flag that does not exist on this platform. sed 's|</\?loc>||g' silently does nothing under BSD sed, which has no \? in basic regular expressions. The sitemap crawl returned 000 on all 65 URLs and looked exactly like a total outage.

They all failed toward more work

Not one of these ever said a broken page was fine. They said fine pages were broken, sixteen at a time, and two of them were one approval away from producing a commit that edited correct pages into worse ones. A check with a bug does not fail evenly in both directions. It fails toward more work, because the code path that raises a finding is the one carrying the assumption, and the code path that stays quiet is usually just an early return.

A person reading that report over coffee glances at an implausible line and moves on. An agent reading the same output opens the file and fixes it, because fixing things is the job I gave it.

What actually catches them

  1. Treat uniform failure as a bug in the checker. A real outage is rarely total and never that tidy. 0 of 64, 0 of 72, every feed, every asset, including the page open in the browser tab beside the terminal. That is one broken assumption applied 136 times. The tell has now caught three of these before they reached a report.
  2. Make the check print the text it read, beside the verdict. The answer-first bug survived because it published a conclusion without the evidence. Show me the 60 words the check measured and I can settle it in a second.
  3. Confirm with a second tool before believing a negative. One curl -o /dev/null -w '%{http_code}' would have ended the certificate incident before the incident note existed.
  4. Write retired rules down where the checker can see them. I retired the 50-to-60 title rule on measurement four days ago. The script running the daily pass still carries it, so this afternoon it produced fourteen findings, of which two were real and both were already open. The other twelve were the branding suffix again, or pages that are supposed to look exactly as they do.

The cost of the good version

Every fix above is cheap. Print the evidence, use curl, distrust a total failure, keep the retired rules where the code can reach them. Together they are perhaps forty lines. The expensive part was the five runs it took to notice that on those checks the audit was not measuring the site at all, because a report full of findings feels like a working audit, and I wanted it to be working.

The 64 URLs from that first run are still there. They answered 200 on the day the crawler called them dead, and they answered 200 again this afternoon. The only thing that has ever changed is the code doing the asking.