T2 — the BAN fragment shard: the licence hypothesis holds
Pre-registered gate (config header + baselines.json profile fragment-fr-v264, registered against
the shipped modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' before the run): targets move, guards hold.
Verdict: PASS on both boards. Every target moved, every guard held, and the two contextful guards improved. The house-number-licence diagnosis (T1c) is confirmed by the fix working.
This is not a promote gate — see §5.
| Run | ap-onoBImKJriMFy7Pz3i5t0D, config v3.1.0-fr-fragment |
| Change | ONE variable vs shipped v264: the synth-fr-fragment shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. @ 12.0 |
| ShardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. | 144,865 rows, BANBAN (Base Adresse Nationale). France's authoritative open national address register — the highest-quality training source for French addresses, with full component structure. (Licence Ouverte), 0 evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. surfaces |
| TrainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input. | 8000/8000 steps, 0 errors, no NaNNaN (not a number). A floating-point result for an undefined operation (log of a negative, 0/0). Appearing in the training loss usually halts the run; recovering from it follows the NaN protocol., cross_pollution 0.00% |
| PrecisionprecisionOf the spans the model labeled as a given tag, the fraction it got right. High precision means few false positives. Paired with recall to compute F1. | int8 candidate vs int8 baselines (asserted, see §4) |
1. Board 2 — the fragment board (the read)
| class | v264 | 95% CI | v310 | 95% CI | Δ |
|---|---|---|---|---|---|
| bare-streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. | 0.215 | [0.178, 0.258] | 0.715 | [0.669, 0.757] | +50.0pp |
| streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.-particle | 0.273 | [0.231, 0.318] | 0.855 | [0.817, 0.886] | +58.2pp |
| admin-streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.-homonym | 0.087 | [0.064, 0.119] | 0.517 | [0.469, 0.566] | +43.0pp |
| date-name | 0.055 | [0.037, 0.082] | 0.158 | [0.125, 0.196] | +10.3pp |
| streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.-housenumber (guard) | 0.925 | [0.895, 0.947] | 0.948 | [0.921, 0.965] | +2.3pp |
| alnum-housenumber (guard) | 0.925 | [0.895, 0.947] | 0.960 | [0.936, 0.975] | +3.5pp |
| bare-localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. (guard) | 0.980 | [0.961, 0.990] | 0.980 | [0.961, 0.990] | ±0.0 |
| OVERALL | 0.494 | [0.476, 0.513] | 0.733 | [0.717, 0.749] | +23.9pp |
No target's interval overlaps its baseline. These are not readings anyone has to squint at.
The archetype:
"Rue Montmartre" v264 -> locality="Rue Montmartre" v310 -> street ✓
2. The prediction that mattered: bare-locality held
T1c's standing prediction was that this cell might collapse. It read 0.980 for the wrong reason — the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' called everything without a house numberhouse numberThe numeric or alphanumeric identifier of a building on a street. Mailwoman's house_number component; its position relative to the street name flips between locales. a localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy., and on bare localities that was accidentally right. Teaching bare streetsstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. gave it every incentive to flip that default and trade one broken prior for another.
It held exactly: 0.980 → 0.980. Same interval. The modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' learned the distinction rather than a new default, and the discriminating evidence is the designatordesignatorThe closed-vocabulary leading word of a secondary-address phrase — 'Apt', 'Suite', 'Floor', 'PO Box', 'Level' — paired with an identifier to form a complete subpremise., which is the only thing that actually separates the two classes.
That is what the counter-distribution was for. 20% of the shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. is bare communes carrying no streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. labelcomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag.; without them, the cheapest way to satisfy every other row is to start calling bare toponymstoponymA proper name for a geographic place. streetsstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.. The guard is why we can say the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' learned rather than swapped.
The contextful guards did not merely hold — they improved (+2.3pp, +3.5pp). The shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. did not buy fragments with full addresses.
3. Board 1 — the global parity floor (the guard)
| labelcomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag. | v264 | v310 | Δ |
|---|---|---|---|
| streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. | 154/267 = 0.577 | 162/267 = 0.6067 | +3.0pp |
| house_number | 117/146 = 0.8014 | 117/146 = 0.8014 | identical |
| postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. | 71/72 = 0.9861 | 71/72 = 0.9861 | identical |
Nothing regressed; streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. gained +8 fixtures on the broad corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus.. The FAIL verdicts printed by
eval parity are the v7 campaign floors (streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. 0.90 / hn 0.97) — v264 fails them too. They are a
target, not a regression gate.
4. What the in-training eval said, and why it is not the verdict
| v264 | @2000 | @4000 | @6000 | @8000 | |
|---|---|---|---|---|---|
| streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. | 0.8105 | 0.798 | 0.811 | 0.810 | 0.805 |
| regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. | 0.8402 | 0.833 | 0.830 | 0.830 | 0.831 |
| localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. | 0.7969 | 0.799 | 0.794 | 0.794 | 0.795 |
| val_loss | 1.1819 | 1.1992 | 1.1990 | 1.1995 | 1.2009 |
The val split contains no bare streetsstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.. It is base-distribution, so it could only ever price the cost — never the win. Read alone it says "regression"; the fragment board says +50pp. Both numbers are correct about different questions.
Two things survive from it:
- streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. dipped and re-converged (0.798 → 0.811) while carrying the fragment mass — the same story the boards tell.
- regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. found a new lower equilibrium at ~0.830 (−1.0pp) and plateaued across four evalsevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error.. It looked like the open item — #1102 is titled "fragment/twin trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input. mass erodes US regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.+localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. recallrecallOf the spans whose gold label is a given tag, the fraction the model found. High recall means few misses. Paired with precision to compute F1. (~2.5pp) — the promote blocker", the same failure shape at a larger magnitude, and it blocked a promote before. It did not reproduce on the canonical gate (§4.1).
The precisionprecisionOf the spans the model labeled as a given tag, the fraction it got right. High precision means few false positives. Paired with recall to compute F1. confound, caught in the act. The first read graded an fp32fp32 / fp1632-bit and 16-bit floating-point formats. Mailwoman trains in bf16 (a 16-bit variant) and exports the ONNX model in int8 for size. export against
int8 baselines — the shipped weightsparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. package ships int8 (39.4 MB), a fresh export_onnx is fp32fp32 / fp1632-bit and 16-bit floating-point formats. Mailwoman trains in bf16 (a 16-bit variant) and exports the ONNX model in int8 for size.
(157 MB). The gate specs cap int8-vs-fp32fp32 / fp1632-bit and 16-bit floating-point formats. Mailwoman trains in bf16 (a 16-bit variant) and exports the ONNX model in int8 for size. at 1.5pp, larger than several cells above. The read script's
md5 check passed it happily, because "differs from the baseline modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.'" is trivially true of a precisionprecisionOf the spans the model labeled as a given tag, the fraction it got right. High precision means few false positives. Paired with recall to compute F1.
change: an md5 difference proves the file changed, not that it is comparable. Fixed (5e5f9c2a) —
the pipelinestaged pipelineMailwoman's runtime architecture: a sequence of pure-function stages (normalize → query-shape → locale-gate → kind-classifier → phrase-grouper → classifier → decoder) connected by typed handoffs. Each stage is published as its own npm package. is export → quantize → grade, and the script now asserts the precisionprecisionOf the spans the model labeled as a given tag, the fraction it got right. High precision means few false positives. Paired with recall to compute F1. class matches.
Every number on this page is int8-vs-int8.
4.1 The golden per-tag battery — the 2pp pre-publish gate
The canonical instrument the promote rule names (eval error-analysis, strict createScorer, full
ship config, int8 both sides):
| tag | v264 | v310 | Δ |
|---|---|---|---|
| regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. | 78.8% | 78.6% | −0.2pp |
| localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. | 48.8% | 48.7% | −0.1pp |
| postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. | 97.3% | 97.2% | −0.1pp |
| house_number | 97.1% | 97.0% | −0.1pp |
| venuevenueA named, non-address place — a business, building, park, or stadium. Mailwoman's free-text point-of-interest component, added as a Tier 2 fine label. | 37.1% | 37.1% | ±0 |
| streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. | 15.8% | 16.3% | +0.5pp |
| countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. | 89.0% | 90.2% | +1.2pp |
| exact matchexact matchThe share of eval items whose every component is correct (compared per-span or per-token). Stricter than per-tag F1, which credits partial correctness. | 25.5% | 25.7% | +0.2pp |
Every tag is inside the 2pp bar. Two improve. The pre-publish gate PASSES.
So the val split's −1.0pp regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. was a false positive, not a preview. Worth stating plainly because it inverts the arc's usual failure: the other instruments here missed problemsexpectation-maximizationAn iterative algorithm that estimates model parameters when some variables are unobserved. In Mailwoman's matcher, EM learns the Fellegi-Sunter m and u parameters from unlabeled data — no training labels needed. that were real; this one reported a problem that was not. Same root — a metric on a distribution nobody asked about — and the same rule: read the instrument the gate names.
That does not make region uninteresting. It is the tag the fragment mass pulls on, #1102 is real,
and a heavier weightparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. or a second localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for.'s shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. could push −0.2pp into −2pp. It is a thing to watch,
not a thing to ignore, and it is cheap to watch: this table is one command.
4.2 The pre-registered "ALSO" — which did NOT come out as predicted
The gate carried a third clause: "hallucination_rate@v301-span 0.352 should FALL if T1c's diagnosis
holds." Measured on the 54 streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.-free parity rows — the MESSY population (venuesvenueA named, non-address place — a business, building, park, or stadium. Mailwoman's free-text point-of-interest component, added as a Tier 2 fine label., all-caps junk,
French zone names), as distinct from the fragment board's clean BANBAN (Base Adresse Nationale). France's authoritative open national address register — the highest-quality training source for French addresses, with full component structure. communes:
| rate | 95% Wilson | |
|---|---|---|
| v264 (shipped) | 12/54 = 0.222 | [0.132, 0.349] |
| v310 (fr-fragment) | 17/54 = 0.315 | [0.207, 0.447] |
It rose. 6 introduced, 1 removed. The intervals overlap heavily and McNemar on the paired discordants (6 vs 1) gives p ≈ 0.125, so the rate change is not established — but the direction is against the prediction, and two of the six have a mechanism worth naming:
✗ "ZAC sous la Combe Lavancia-Epercy" -> street="Combe Lavancia-Epercy"
✗ "ZA Entraigues Embrun" -> street="ZA"
ZAC / ZA are French zone designatorsdesignatorThe closed-vocabulary leading word of a secondary-address phrase — 'Apt', 'Suite', 'Floor', 'PO Box', 'Level' — paired with an identifier to form a complete subpremise. (zone d'aménagement concerté), and sous la is exactly
the particle pattern the shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. teaches. This is on-thesis over-generalizationgeneralizationA trained model's performance on data unlike its training set — new regions, new input distributions. The property honest eval is designed to measure.: the grammar the
shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. installs — "French designatordesignatorThe closed-vocabulary leading word of a secondary-address phrase — 'Apt', 'Suite', 'Floor', 'PO Box', 'Level' — paired with an identifier to form a complete subpremise. + particle + name = streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels." — is slightly too wide, and French
zone names sit just inside its edge. The other four (BOOM → Boom, new south wales aus,
philadelphia museum of art) are pre-existing non-French confusions that shifted.
The two populations do not contradict each other; they partition:
- Clean French communes (board, n=400):
bare-localityheld at 0.980. The counter-distribution worked exactly where it was aimed. - French zone names: a new, narrow class the counter-distribution does not cover, because it mints only communes.
- Non-French junk: noise.
The fix is a sibling of what is already there — extend the counter-distribution to French zone
designatorsdesignatorThe closed-vocabulary leading word of a secondary-address phrase — 'Apt', 'Suite', 'Floor', 'PO Box', 'Level' — paired with an identifier to form a complete subpremise. (ZAC, ZA, Lotissement, Résidence as NOT-streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.). Not a new idea, a wider one.
It is a shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. change, so it belongs to the next run, not to this verdict.
This is what pre-registration is for. Two of the three gate clauses passed decisively; the third went the other way and is on the page at the same size as the wins.
5. What this does and does not establish
Establishes: the failure was the trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input. distribution, exactly as T1c diagnosed. A modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' that read
Rue Montmartre as a localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. now reads it as a streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels., and does so without giving up the contextful
case or starting to hallucinate streetsstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. on bare communes. The BANBAN (Base Adresse Nationale). France's authoritative open national address register — the highest-quality training source for French addresses, with full component structure. registry as a soft priorsoft priorOutside knowledge fed to the model or resolver as overridable evidence (a feature, a score term) rather than a hard filter. A focus-country hint becomes an anchor feature; a focus-point becomes a ranking term. +
synthetic coveragecoverageThe fraction of a population or region for which a data source has real, non-placeholder entries — e.g. 47% rooftop coverage on Texas addresses. Distinct from accuracy on the rows that are present. source — the doctrine's whole claim — works on its first real test.
Does not establish:
- Promotion. The 2pp pre-publish gate passes (§4.1), which clears the largest single hurdle — but
that is one leg, not the battery. A promote still wants
mailwoman eval gate --gate <spec>(the per-localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for. floors, the int8↔fp32fp32 / fp1632-bit and 16-bit floating-point formats. Mailwoman trains in bf16 (a 16-bit variant) and exports the ONNX model in int8 for size. delta cap, the cascade smoke, the mask-regression lock) and an operator GO. Nothing ships on this page. - The v7 floors. streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. 0.6067 vs a 0.90 floor. +3.0pp is real and it is not 32pp.
date-name. 0.158 is +10.3pp and still 84% wrong. BANBAN (Base Adresse Nationale). France's authoritative open national address register — the highest-quality training source for French addresses, with full component structure. holds only ~1,418 date-name streetsstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. and the extractor already takes every one; the shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. has 655 rows of it. The lever there is shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. weightparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values., not more data — and, per T1c, the underlying confusion (a digit inside a streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. name vs a house numberhouse numberThe numeric or alphanumeric identifier of a building on a street. Mailwoman's house_number component; its position relative to the street name flips between locales.) is the same one the spanspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree. decode shows from the other side.- Other localeslocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for.. FR only. GeneralizationgeneralizationA trained model's performance on data unlike its training set — new regions, new input distributions. The property honest eval is designed to measure. goes by tiertierInternal versioning of which label classes the model emits. Tier 1 is the coarse components (country, region, locality, postcode); Tier 2 adds venue, street, house_number; Tier 3 (future) would add attention, po_box, and POI venue subtyping. Historically called 'Stage 1/2/3' before the runtime-pipeline naming made that ambiguous. (BANBAN (Base Adresse Nationale). France's authoritative open national address register — the highest-quality training source for French addresses, with full component structure. → BAG → TIGERTIGERThe US Census Topologically Integrated Geographic Encoding and Referencing database. Used as a corpus source for street-segment data.-with-scope), never folded into one step.
- Hallucination. §4.2: it did not fall, and on the messy population it rose (n.s.). The FR-zone class is real, narrow, and uncovered. Anyone reading the +50pp should read that too.
Reproduce: bash scratchpad/read-v310-gate.sh 008000 (export → quantize → both boards, with the
precisionprecisionOf the spans the model labeled as a given tag, the fraction it got right. High precision means few false positives. Paired with recall to compute F1. assertion).