Four numbers tried to lie to us in 24 hours. We shipped anyway.

Yesterday we wrote about a lookup table that scored a perfect 100 and nearly talked us out of our own architecture. We ended that post with a rule: write the bar down before you look at the score. What we didn't know was that the next 24 hours would test that rule four separate times, and that the fourth test would flip a conclusion we'd been quoting for two weeks.

This is the story of shipping v4.2.0: a consolidation, a bar that lowered itself, a consultant who was confidently wrong twice, a capacity wall with a number on it, and a benchmark harness that had been starving our own model for four releases.

The trade

We consolidated every proven lever of the parity campaign into one training run: the unit shard, the affix shard, the country gazetteer anchor, the multi-locale balance. The core tags soared. Locality up thirteen points, region up eleven, country from nothing to 87. And the street-affix split, which a solo run had just taught to 78, crashed to 27.

If you've trained multi-task models, you know this trade. Capacity is a blanket that's slightly too small. Pull it up to your chin and your feet get cold.

Lie number one: the bar that lowered itself

While writing up the gate criteria for the fix attempts, we restated the targets from memory. Affix ≥78 became "≥72". Unit 92 became 91. And the US street floor, the one number that would have exposed a regression hiding in every single consolidation run, fell out of the table entirely.

Nobody decided this. That's the unsettling part. Restating numbers from memory is how a contract becomes a vibe, and a vibe always drifts toward whatever you already built. The operator caught it by asking one question about one number. The fix cost ten minutes, and the rule it produced is now non-negotiable: the training config is the canonical gate, and any change to a bar is a stated decision with a reason, in writing, before the eval runs. When a model misses the bar, you confront the miss: re-baseline it with reasons, or iterate. Re-describing it as a pass is how you wake up six weeks later wondering when street tagging got worse.

Lies two and three: the consultant's confidence

We consult an external model on consequential forks, and it earns its keep often enough that we keep asking. This time it was sure, twice, with numbers. The affix crash was a scheduling problem, not a capacity problem. Five-times density would clear 72. And the 75 we'd seen in a quick diagnostic? Not a transient.

Both predictions died by experiment, and we know that precisely because we spent five minutes of GPU on a cheap diagnostic before each expensive run instead of trusting either of us. Five-times density plateaued at 65. The 75 was a transient: hold the density long enough to reach it and the peak decays under your feet, 75 down to 53, while the pressure collapses French region tagging from 25 to 5. The blanket is too small, and yanking it hard enough to cover your feet tears it somewhere you weren't looking.

So that's a capacity wall with a number on it: 29 million parameters cannot hold the affix split and the rest of the distribution at once. It can visit. It can't stay. We wrote months ago that the first model-size increase would have to earn its way in with evidence, and this is what earning it looks like. The escalation now sits in an issue with a pre-registered gate and a cheap probe to try first — a dedicated affix head — before anyone reaches for more parameters.

Shipping the miss, out loud

So we shipped the trade. v4.2.0 is the consolidation's strongest stable variant, and its ledger entry says exactly what it gave up: street down 2.3, unit down 1.7, the affix split present but at 65-level, fluent nowhere near its solo 78. The same entry says what everyone got: locality +13, region +11, country from absent to 89.8, French house numbers at their best ever, German locality beating the system we're chasing.

Name your regressions in the model card and something nice happens downstream: the next regression has a recorded baseline to measure against. The cards that only list wins go stale the moment anything moves.

Lie number four: the harness was starving our model

Hours after shipping, one number kept itching. Our capability arenas, the unbiased head-to-head against the rules parser, showed the new model slightly down on whole-parse accuracy. Plausible, right? We'd just traded street precision away, and we very nearly filed it under the stated costs.

Then we checked how the arena harness loads models. Well. It had no way to feed the gazetteer lexicon or the postcode-anchor channel. Every anchor-trained model we've benchmarked in those arenas — four releases' worth — was graded with those input channels zeroed out. We had been publishing numbers for a parser with its eyes taped shut and treating them as the parser.

Fed its actual shipping configuration, v4.2.0 doesn't trail the rules system on clean, canonical addresses. It beats it: 41% to 29%, with the noisy-input lead stretching to 32 points. "Rules win on clean input" had been our routing principle, our documented truth, the premise of a planned arbitration layer's priors. It was a measurement artifact. The model had been better than its own scoreboard since before we knew to ask.

Calibrate the instrument too

Four numbers lied to us in 24 hours: a gate that softened itself in retelling, a consultant's confident extrapolation, a transient peak dressed as an equilibrium, and a benchmark grading a handicapped model. None of them lied maliciously. Every one of them was plausible, and plausible is the only kind of wrong number that survives long enough to cost you something.

The machinery that caught them is boring on purpose. Bars written down before looking. Cheap diagnostics before expensive commitments. Misses confronted in writing. And a suspicious habit worth stealing: whenever a result confirms what you already believe, or condemns what you just built, go read how the measurement was taken before you react to it.

The lookup table from yesterday's post scored 100 and was wrong. The arena scored our model low and was wrong. The instrument is part of the experiment. Calibrate accordingly.