The relabel probe — #511 confirms the #492 contradiction theory (2026-06-10 evening)
Same-day sequel to the width gate + affix audit. That note ended with a theory (the affix ceiling is contradictory base-corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus. labelscomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag. at ≥ 1,039:1) and a fix design (#511, a loader-level relabel pass). This note records the probe that tested it.
Design
v1.0.6-relabel-probe: single-variable contrast against probe 0 (v1.0.4) — same clean
consolidation step-040000 checkpointcheckpointA saved snapshot of the model weights and optimizer state during training. Mailwoman saves a checkpoint periodically so training can resume after a GPU hang., same shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. weightsparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. (synth-affix 20.0), same +4k steps,
choreography off, ONE change: data.affix_relabel_lexicon_path set. Probe 0 is the measured
control: prefix 81.0 at +2k decaying to 61.1 at +4k with relabel off.
The relabel pass (corpus-python/src/mailwoman_train/relabel.py) applies the affix shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row.
builder's exact split semantics to every streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. spanspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree. at load time, after augmentation. Builder
parity is the critical property — "W Park Ave" gets NO split because the builder rejects
affix-shaped names, and a looser pass would introduce a third labeling. VocabvocabularyThe fixed set of tokens a tokenizer can produce. Mailwoman's SentencePiece vocabulary is tens of thousands of subword pieces, with byte fallback for anything outside it. is codex-derived
(scripts/build-affix-relabel-lexicon.mjs, 16 directional + 549 suffix variants). Pre-train
audit on 250K real base rows across five shardsshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row.: every sampled split correct, per-shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. split
rates 8–87% consistent with the 69.4% aggregate measurement.
Design was pressure-tested in a 3-turn DeepSeek consult (curl fallback — the pi wrapper timed
out again at 180s): concurrence on all-rows scope (GB splits are schema-correct; a countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. filter
adds brittleness), relabel probability 1.0 (partial relabeling is a weaker dose of the same
contradiction — p=0.9 still leaves ~156:1), and probe decisiveness. Its push-back we adopted: the
32-row affix evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. is too small to gate on (one instance ≈ 4pp); expand to ≥100/≥100 instances
before the full-run gate. One correction ours: the probe keeps shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. weightparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. 20.0 for probe-0
parity — the weightparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values.-reduction discussion belongs to the full run.
Result: HOLD
Scored with score-affix (fp32fp32 / fp1632-bit and 16-bit floating-point formats. Mailwoman trains in bf16 (a 16-bit variant) and exports the ONNX model in int8 for size., ship-config channels), real-affix evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. (n=32):
| +2k (step 42000) | +4k (step 44000) | probe 0 control | |
|---|---|---|---|
| street_prefix recallrecallOf the spans whose gold label is a given tag, the fraction the model found. High recall means few misses. Paired with precision to compute F1. | 88.0 | 84.0 | 81.0 → 61.1 |
| street_prefix F1 | 93.6 | 91.3 | — |
| street_suffix F1 | 96.7 | 100.0 | ~59 |
| streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. F1 | 87.5 | 87.5 | — |
| guardrail (hn/loc/regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality./postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one.) | 100 each | 100 each | — |
Honest reading of the strict criterion: we pre-registered "drop ≤ 2pts +2k→+4k"; prefix recallrecallOf the spans whose gold label is a given tag, the fraction the model found. High recall means few misses. Paired with precision to compute F1. dropped 4.0 (F1 2.3). That is exactly ONE flipped instance (tp 22→21) on a 25-instance evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. — the quantization noise the consult flagged — against the control's 20-point collapse, while suffix ROSE to 100. Verdict: HOLD, with the caveat recorded here rather than hidden.
Residual misses at +4k: four prefixes, all prefix+ordinal/numeric names ("Northwest 23rd
Avenue", "E 63rd StreetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.", "NE Loop 410"). The suffix splits fire correctly even on those rows
(model street="northwest 23rd"). A long tail that 4k corrective steps on a 20.0-weightparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. resume
has not migrated; the full run owns it.
The full run
v1.1.0-relabel-consolidation launched the same evening (from scratch, 40k): the v4.2.0 recipe
on the consistent mix with both anti-contradiction compensations REVERTED as stated decisions —
synth-affix 17.0 → 2.0 and affix tag class-weightsparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. 2.0/4.0 → 1.5/1.5 (suffix now appears on ~65%
of base streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. rows; a 4× boost on a majority tag risks over-fire). Gate: the v4.2.0 ship floors
unchanged, affix measured at 20k AND 40k (stability is the claim under test), plus the consult
watches (folded-streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. invariance, short-streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. recallrecallOf the spans whose gold label is a given tag, the fraction the model found. High recall means few misses. Paired with precision to compute F1., house_number→street_prefix transition).
The expanded NADNAD (National Address Database). A US Department of Transportation dataset of structured address points, added to the training corpus as a major source of real US addresses.-native affix evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. re-baselines before the verdict; the 32-row result is recorded
alongside, not substituted. All hold → v4.3.0 candidate.
Probe economics, for the record
The entire #492 → #511 arc — three falsification probes, the audit, the corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus. measurement, the fix, and its confirmation — cost under one hour of A100 total. The width run alone (the hypothesis the ladder was built to avoid betting on blind) would have been ~1h for a 6/12 FAIL with no explanation. Pre-registered cheap probes against a measured control remain the best purchase in this project.