2026-07-14 — v263 country-channel (#1104): grade result + the promote decision
v263 (v2.6.3-country-channel) activates the dedicated countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. soft-feed channel (#1104) — the permanent
fix for the countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. regression that v261 (6.1.0) shipped as a documented cosmetic exception. It is an
init_from fine-tune off v261 (single variable: the countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. channel ON), trained 8k steps on A100
(lossloss functionA number measuring how wrong the model's predictions are on a batch of examples. Training minimizes it. Mailwoman's loss combines per-token negative log-likelihood with the CRF sequence loss. 0.9745, macro_f1 0.748 held, no NaNNaN (not a number). A floating-point result for an undefined operation (log of a negative, 0/0). Appearing in the training loss usually halts the run; recovering from it follows the NaN protocol.). Modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' int8 md5 34289d215e0c1ba1e663337999ca3cbd.
The channel works. It recovers countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. and holds the assembled coordinatecoord metricThe primary evaluation metric: distance from the resolved coordinate to the true address point. Measured at percentiles (p50, p90) and as 'within X meters.' Prevents the label-F1 trap where a model scores higher on token labels but geocodes worse.. It also introduces a coordinate-invisible per-tag regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. trade. The promote-to-default call is the operator's — mirroring the v261 documented-exception decision — because the regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. delta is a per-tag regression not pre-approved.
Grade (package-shaped throughout — --weights-cache, never --model alone, #718)
| Gate | shipped (v261 / 6.1.0) | v263 | verdict |
|---|---|---|---|
| PRIMARY — golden countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. recallrecallOf the spans whose gold label is a given tag, the fraction the model found. High recall means few misses. Paired with precision to compute F1. | 190/224 = 84.8% | 200/224 = 89.3% | ✓ +4.5pp — clears the 88.6% v241 bar; fixes 19, breaks 9 (netneural networkA model made of layers of simple numeric units whose connection strengths (weights) are learned from data. The transformer encoder at Mailwoman's core is a neural network. +10) |
| GUARD — real-postal countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. recallrecallOf the spans whose gold label is a given tag, the fraction the model found. High recall means few misses. Paired with precision to compute F1. (falsifier) | 3/4 | 3/4 | ✓ identical |
| GUARD — hallucination (300 real no-countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. rows) | 1% | 1% | ✓ identical |
| NON-INF — held-out US coordinate (300 FDIC, ≤5km) | 279 | 278 (z −0.16) | ✓ PASS — not significantly worse |
| NON-INF — held-out FR coordinate (300 BANBAN (Base Adresse Nationale). France's authoritative open national address register — the highest-quality training source for French addresses, with full component structure., ≤5km) | 281 | 280 (z −0.17) | ✓ PASS |
| gauntlet regression + metamorphic | PASS | PASS (same 6 tracked xfails) | ✓ |
| per-tag regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. (golden, exact-match) | 556 fails | 580 fails | ⚠ +24 — the regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. trade |
| countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.-homographhomonymyOne surface, many referents: 'Georgia' is a US state in 'Atlanta, Georgia' and a country in 'Tbilisi, Georgia.' Handled by disambiguation, split across two stages — the parser resolves the tag from in-string context; the resolver late-binds the referent with geographic context. F1 (real, n=54) | 89.8 (countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.-OFF) | 82.6 (countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.-ON) | ⚠ −7.2pp — the homographhomonymyOne surface, many referents: 'Georgia' is a US state in 'Atlanta, Georgia' and a country in 'Tbilisi, Georgia.' Handled by disambiguation, split across two stages — the parser resolves the tag from in-string context; the resolver late-binds the referent with geographic context. trade (surfaced by the ledger tooling) |
CountrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head./postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one./hn/streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels./localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. per-tag deltas on golden are small (postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. +6, streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. +5, localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. +4, hn +2); aggregate labelcomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag.-fails 1805 → 1835 (+30), dominated by regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.. Note the two countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. lenses disagree by design — see the two trade sections below.
The region trade — real, but coordinate-invisible
The regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. regression concentrates on trailing-countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. tails: v263 drops the regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. on inputs like
Cider Mill Rd VT 05161, US, STATE RTE 100, VT 05350, USA, and FR Manailly Creuse FRANCE — 11 rows
that shipped labelscomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag. correctly, regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. → "". Mechanism: the dedicated countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. channel (immune to
suppress_gazetteer_near_postcode) fires on the trailing countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. surface and out-competes the adjacent
regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. tag on that boundary. It is a genuine v263 behavior, not a near-miss.
But it does not move the assembled coordinatecoord metricThe primary evaluation metric: distance from the resolved coordinate to the true address point. Measured at percentiles (p50, p90) and as 'within X meters.' Prevents the label-F1 trap where a model scores higher on token labels but geocodes worse.. The #566-correct measure — the held-out coordinate z-test on 300 fresh real US/FDIC and 300 fresh real FR/BANBAN (Base Adresse Nationale). France's authoritative open national address register — the highest-quality training source for French addresses, with full component structure. addresses — is flat in both localeslocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for. (z −0.16 / −0.17, PASS). The golden regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. misses are on redundant-trailing-countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. formats that real-address draws don't emphasize; the resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon. still pins those without the regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.. This is the same class as v261's documented countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. exception, in the opposite tag: a per-tag delta that the coordinate does not see.
The homograph trade — the country channel cuts both ways (surfaced 2026-07-14 by the ledger tooling)
The --weights-cache promotion-gate path added for the ledger backfill graded v263 package-shaped on the
country-homograph-real probe (n=54) and exposed a second, countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.-side trade the golden grade above
did not measure. v263's countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. channel is a 2-dim [country_surface, country_ambiguous] featurefeatureAn input signal a model conditions on. Beyond the raw tokens, Mailwoman feeds soft features — gazetteer-membership channels and the postcode anchor — that inform predictions without overriding them.; the
country_ambiguous bit is a learnable false-positive guard for homographhomonymyOne surface, many referents: 'Georgia' is a US state in 'Atlanta, Georgia' and a country in 'Tbilisi, Georgia.' Handled by disambiguation, split across two stages — the parser resolves the tag from in-string context; the resolver late-binds the referent with geographic context. surfaces (a countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. name that is
also a US stateregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality., a citylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy., or a common word). It works — but on this all-homographshomonymyOne surface, many referents: 'Georgia' is a US state in 'Atlanta, Georgia' and a country in 'Tbilisi, Georgia.' Handled by disambiguation, split across two stages — the parser resolves the tag from in-string context; the resolver late-binds the referent with geographic context.-are-countries test it is
too conservative: v263 (countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.-ON) misses exactly three rows countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.-OFF emits — Georgia (US stateregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.),
Jordan (given name), Jamaica (NYC neighborhood) — and gains none, so countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. F1 drops 89.8 → 82.6.
This is the mirror image of the WOFWOF (Who's On First). An open-source gazetteer of places maintained by Mapzen/whosonfirst. Mailwoman builds a custom SQLite database from WOF GeoJSON repos, extended with postcode data, importance scores, and coincident-role relations.-admin win. On the leading-long-form WOFWOF (Who's On First). An open-source gazetteer of places maintained by Mapzen/whosonfirst. Mailwoman builds a custom SQLite database from WOF GeoJSON repos, extended with postcode data, importance scores, and coincident-role relations.-admin distribution (golden, the
#1104 target) the channel lifts countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. recallrecallOf the spans whose gold label is a given tag, the fraction the model found. High recall means few misses. Paired with precision to compute F1. 84.8→89.3%; on the trailing-homographhomonymyOne surface, many referents: 'Georgia' is a US state in 'Atlanta, Georgia' and a country in 'Tbilisi, Georgia.' Handled by disambiguation, split across two stages — the parser resolves the tag from in-string context; the resolver late-binds the referent with geographic context. distribution it
trades recallrecallOf the spans whose gold label is a given tag, the fraction the model found. High recall means few misses. Paired with precision to compute F1. for precisionprecisionOf the spans the model labeled as a given tag, the fraction it got right. High precision means few false positives. Paired with recall to compute F1.. The netneural networkA model made of layers of simple numeric units whose connection strengths (weights) are learned from data. The transformer encoder at Mailwoman's core is a neural network. is coordinate-invisible — the held-out coordinate z-test passed on
both localeslocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for., and the country_ambiguous guard's whole purpose is fewer false "Georgia → countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head." emissions
on real mixed input, which the homographhomonymyOne surface, many referents: 'Georgia' is a US state in 'Atlanta, Georgia' and a country in 'Tbilisi, Georgia.' Handled by disambiguation, split across two stages — the parser resolves the tag from in-string context; the resolver late-binds the referent with geographic context.-recallrecallOf the spans whose gold label is a given tag, the fraction the model found. High recall means few misses. Paired with precision to compute F1. test cannot credit. It does not change the v263 ship
(coordinate is the ship gate, #566), but it sharpens the countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. story: v263 helps admin-hierarchy countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.
and trades a little homographhomonymyOne surface, many referents: 'Georgia' is a US state in 'Atlanta, Georgia' and a country in 'Tbilisi, Georgia.' Handled by disambiguation, split across two stages — the parser resolves the tag from in-string context; the resolver late-binds the referent with geographic context.-countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. recallrecallOf the spans whose gold label is a given tag, the fraction the model found. High recall means few misses. Paired with precision to compute F1. for homographhomonymyOne surface, many referents: 'Georgia' is a US state in 'Atlanta, Georgia' and a country in 'Tbilisi, Georgia.' Handled by disambiguation, split across two stages — the parser resolves the tag from in-string context; the resolver late-binds the referent with geographic context. precisionprecisionOf the spans the model labeled as a given tag, the fraction it got right. High precision means few false positives. Paired with recall to compute F1.. A future tune could soften the
guard (a lower country_ambiguous weightparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values.) if the homographhomonymyOne surface, many referents: 'Georgia' is a US state in 'Atlanta, Georgia' and a country in 'Tbilisi, Georgia.' Handled by disambiguation, split across two stages — the parser resolves the tag from in-string context; the resolver late-binds the referent with geographic context.-recallrecallOf the spans whose gold label is a given tag, the fraction the model found. High recall means few misses. Paired with precision to compute F1. lens is judged to matter more than the
precisionprecisionOf the spans the model labeled as a given tag, the fraction it got right. High precision means few false positives. Paired with recall to compute F1. it buys. Recorded in evals/scores-by-version.json (the 6.2.0 row's us.country_homograph).
The decision — operator's, per the v261 precedent
The night-shift 2pp pre-publish gate aborts a promotion on any tag regressing >2pp from the default unless the operator pre-approved the trade. The regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. regression was not pre-approved (the greenlit retrain targeted countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. recovery). So — exactly as v261 shipped only because the operator said "promote with the documented exception" — the promote-to-default call for v263 is the operator's, with this trade on the table.
Recommendation: promote. v263 recovers the #1104 countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. target with the real fix (the atlas channel, not a cosmetic exception), holds the real-postal guard, and is coordinate-flat on both localeslocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for.. The regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. cost is coordinate-invisible and on a low-prevalence format. If instead the regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. per-tag delta is judged unacceptable, the channel is validated and the next iteration is a targeted regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.-preservation tune (a lower countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.-channel confidence near a regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.-abbrev, or a regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. class-weightparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. bump), not a rebuild — the channel code is already merged (default-OFF) on main.
Reproduce
- Config:
corpus-python/src/mailwoman_train/configs/v2.6.3-country-channel.yaml(init_from v261 step-008000). - Grade:
scratchpad/grade-v263.sh(export → quantize → package-cache → falsifier + failure-report + gauntlet). - Cross-modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' report:
docs/articles/evals/competitive-parity/2026-07-14-v263-country-channel.mdx. - Channel code: merged in #1116 (
1c40dd7e), default-OFF;data/gazetteer/country-surface-lexicon-v1.json.