B4 — the NO fragment probe did NOT clear its pre-registered bar
2026-07-16. The 2k falsifier for the Norwegian house-number-licence lever.
Verdict: the probe did not clear the pre-registered MOVE criterion, and the do-no-harm guard on the French board drifted the wrong way. Per the config header's own pre-registration, the full 8k run is NOT auto-warranted. The decision to run it anyway is the operator's — with the honest caveat below on the threshold.
The 2k probe cost ~$1–2 of A100 and saved a full 8k run from being launched on a hunch. That is exactly what the probe-before-fix discipline is for.
The read
Pre-registered in v3.3.0-no-fragment-probe.yaml, written before the numbers existed; graded on
board 3 (the NO digit board) against the SHIPPED-v310 baselines registered before the run.
Board 3 — the target:
| class | role | v310 | v330 (2k) | Δ |
|---|---|---|---|---|
| bare-streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.-hn | TARGET | 0.693 [0.646, 0.736] | 0.710 [0.664, 0.752] | +1.7pp |
| bare-pc | guard (negative) | 1.000 | 1.000 | ±0 ✓ |
| streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.-led-hn | guard (ceiling) | 0.968 | 0.965 | −0.3 ✓ |
| citylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.-first-hn | guard (ceiling) | 0.953 | 0.953 | ±0 ✓ |
| pc-first-hn | guard (ceiling) | 0.940 | 0.945 | +0.5 ✓ |
| slash-hn | monitored | 0.650 | 0.655 | +0.5 |
The pre-registered bar: "bare-streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.-hn rises materially... clear motion, not the full gain. If it does NOT move at 2k, the licence-transfer hypothesis is wrong and a full run is not warranted."
+1.7pp with the two intervals almost entirely overlapping is not clear motion. The bar is not
met. Every guard held — bare-pc stayed pinned at 1.000, the three ceiling classes did not regress —
so there is no trade, but there is also no target win.
Board 2 — the FR do-no-harm guard drifted:
| class | v310 | v330 (2k) | Δ |
|---|---|---|---|
| bare-streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. | 0.715 | 0.675 | −4.0 |
| streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.-particle | 0.855 | 0.830 | −2.5 |
| admin-streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.-homonym | 0.517 | 0.465 | −5.2 |
| OVERALL | 0.733 | 0.716 | −1.7 |
Every French class drifted down. The CIs overlap, so no single cell is a clean violation, but the consistency of the drift is the concern — 2k of extra fine-tuning is mildly eroding the fr-fragment win the shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. is required to protect. A full 8k run would have more room to amplify that.
Deconfounded (2026-07-16, same session): re-reading board 2 on shipped v310 in the same
session, same harness, same int8 export gives 0.733 overall — identical to the T2 verdict, cell
for cell (bare-streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. 0.715, admin-streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.-homonym 0.517). So the v310 baseline is stable
session-to-session and the v330 drift is real, not cross-session quantization noise. That
strengthens the do-not-escalate call. The specificity is the tell: it is the FR fragment classes
— the ones fr-fragment fixed — that drift, not the board at random, which points at weightparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values.
dilution: synth-no-fragment @ 12.0 crowds fr-fragment's relative share over 2k steps, and the
modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' drifts back toward its pre-fr-fragment reading. So B4b is two knobs, not one — the bare-streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.
ratio and the no-fragment weightparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. relative to fr-fragment — and they must be probed one at a time.
Two reasons not to escalate, and one caveat
- The target did not clear its pre-registered bar.
- The French guard drifted the wrong way across every cell.
The caveat, stated because integrity requires it: "material move at 2k" was a quantitative prediction I could not strongly ground. fr-fragment's +50pp was measured at its 8k final; I do not have its own 2k-vs-8k trajectory on a board, so I cannot say whether +1.7pp at 2k is "flat" or "early but real." That is precisely the class of threshold prediction that should not, by itself, gate a decision — so the call goes to the operator rather than being made by a bar I cannot defend. What I will not do is relax the bar I wrote in order to launch the 8k myself.
The likely reason it barely moved — a named hypothesis, not a knob spun
B0 established the mechanism: the failing Norwegian rows have the streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. mistagged as a localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.,
and the digit follows it (Hallingrudveien 32 → localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. + postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one.). The fix is therefore the same
as the French bare-streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. polarity fix: teach that a bare streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. is not a bare localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. — which
requires the streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. to appear without a number at all, so the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' cannot lean on the number.
This shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row.'s --bare-street-prob defaults to 0.30: only 30% of its signal rows are pure bare
streetsstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.; the other 70% are {street} {number}, a form the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' mostly already handles (board 3's
bare-streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.-hn is already 0.693, not 0.215 like French was). So the shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. spends most of its weightparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values.
on a class that is not broken and little on the one that is.
The testable next move is to raise the bare-streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. ratio so the shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. hits the streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.→localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. confusion directly, the way fr-fragment did. That is one config knob. Per the treadmill guard I am not spinning it solo at 2k — it is a hypothesis for the operator or the next shift, to be pre-registered and probed, not tuned in the dark.
What stands regardless
- The instrument (board 3) and the recipe (
no-fragment, tested, split-disciplined) are correct and reusable. The probe worked — it gave a clean, cheap read that stopped an unjustified 8k run. bare-pcholding at 1.000 confirms the counter-distribution does its job: the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' did not learn to stop emitting postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. to chase the digit. The shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row.'s shape is sound; its ratio is the lever.- The Norway YAML fix (#1145) is the load-bearing result of the night regardless of this probe — Norway now trains at all, and this probe is the first read that could ever have measured it.
Reproduce
python3 scratchpad/build-no-fragment-full.run.ts
python3 scratchpad/assemble-no-fragment-overlay.py
# push overlay + config to the volume, then:
modal run -d corpus-python/modal/train_remote.py --config v3.3.0-no-fragment-probe.yaml --resume none
bash scratchpad/read-v330-probe.sh