2026-07-15 — v264 country-softguard (#1104): softening the homograph guard is a clean win
v264 (v2.6.4-country-softguard) is a single-variable fine-tune off v263: model.country_ambiguous_scale
1.0 → 0.5. It strictly dominates v263 on countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. with no offsetting trade — the first countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.
iteration in the #1104 arc that carries no documented exception.
Why
The eval gate --weights-cache ledger backfill surfaced that v263's countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. channel cut both ways: its
country_ambiguous guard (a hard suppression of homographhomonymyOne surface, many referents: 'Georgia' is a US state in 'Atlanta, Georgia' and a country in 'Tbilisi, Georgia.' Handled by disambiguation, split across two stages — the parser resolves the tag from in-string context; the resolver late-binds the referent with geographic context. countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. surfaces — Georgia/Jordan/Jamaica)
recovered WOFWOF (Who's On First). An open-source gazetteer of places maintained by Mapzen/whosonfirst. Mailwoman builds a custom SQLite database from WOF GeoJSON repos, extended with postcode data, importance scores, and coincident-role relations.-admin countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. recallrecallOf the spans whose gold label is a given tag, the fraction the model found. High recall means few misses. Paired with precision to compute F1. (84.8→89.3%) but dropped the countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.-homographhomonymyOne surface, many referents: 'Georgia' is a US state in 'Atlanta, Georgia' and a country in 'Tbilisi, Georgia.' Handled by disambiguation, split across two stages — the parser resolves the tag from in-string context; the resolver late-binds the referent with geographic context. probe 89.8→82.6. The
guard was too strong. v264 softens it.
The mechanism — country_ambiguous_scale (bakes into the ONNX, no lexicon/inference change)
A modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.'-config scalar scales the country_ambiguous featurefeatureAn input signal a model conditions on. Beyond the raw tokens, Mailwoman feeds soft features — gazetteer-membership channels and the postcode anchor — that inform predictions without overriding them. dim (index 1) of country_features BEFORE
country_projection, via a non-persistent registered buffer [1.0, scale]. Because the scale lives in the
forward pass, it exports as a constant into the ONNXONNX (Open Neural Network Exchange). An open format for machine learning models that enables interoperability between training frameworks and inference runtimes. Mailwoman ships its trained model as an ONNX file so it can run in Node.js and the browser via onnxruntime. graph — inferenceinferenceRunning the trained model on new input to get predictions, as opposed to training, which produces the model. In Mailwoman that means a small transformer encoder reads an address string and classifies every token — house number, street, locality, region, postcode, and the rest. A Who's On First gazetteer can feed soft location hints into the pass, but the model makes the final call on every label. Where a generative model writes text token by token, Mailwoman's output is a retrieval-augmented token classification: one label per input piece. feeds the raw [country_surface, country_ambiguous] clue and the graph does the scaling. No countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.-surface-lexicon change, no
country-inference.ts/country_lexicon.py change, no browser change. 1.0 = v263 (bit-identical);
v264 = 0.5. (Serialize footgun fixed en route: the scale must be written into the checkpointcheckpointA saved snapshot of the model weights and optimizer state during training. Mailwoman saves a checkpoint periodically so training can resume after a GPU hang.
config.json, else export silently rebuilds at 1.0 — the first v264 export was byte-identical to v263 on
the homographhomonymyOne surface, many referents: 'Georgia' is a US state in 'Atlanta, Georgia' and a country in 'Tbilisi, Georgia.' Handled by disambiguation, split across two stages — the parser resolves the tag from in-string context; the resolver late-binds the referent with geographic context. probe until that was caught.)
Grade (package-shaped throughout — eval gate --weights-cache / loadFromWeights, #718)
| Gate | v263 (6.2.0) | v264 | verdict |
|---|---|---|---|
| countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.-homographhomonymyOne surface, many referents: 'Georgia' is a US state in 'Atlanta, Georgia' and a country in 'Tbilisi, Georgia.' Handled by disambiguation, split across two stages — the parser resolves the tag from in-string context; the resolver late-binds the referent with geographic context. F1 (real, n=54) | 82.6 | 85.1 | ✓ +2.5pp — recovers a homographhomonymyOne surface, many referents: 'Georgia' is a US state in 'Atlanta, Georgia' and a country in 'Tbilisi, Georgia.' Handled by disambiguation, split across two stages — the parser resolves the tag from in-string context; the resolver late-binds the referent with geographic context. miss |
| golden WOFWOF (Who's On First). An open-source gazetteer of places maintained by Mapzen/whosonfirst. Mailwoman builds a custom SQLite database from WOF GeoJSON repos, extended with postcode data, importance scores, and coincident-role relations.-admin countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. recallrecallOf the spans whose gold label is a given tag, the fraction the model found. High recall means few misses. Paired with precision to compute F1. | 200/224=89.3% | 204/224=91.1% | ✓ +1.8pp — the win holds AND grows |
| real-postal countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. recallrecallOf the spans whose gold label is a given tag, the fraction the model found. High recall means few misses. Paired with precision to compute F1. (falsifier) | 3/4 | 3/4 | ✓ held |
| hallucination (300 real no-countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.) | 1% | 1% | ✓ held |
| held-out US coordinate (300 FDIC, ≤5km) | 279 | 269 (z −0.14) | ✓ PASS |
| held-out FR coordinate (300 BANBAN (Base Adresse Nationale). France's authoritative open national address register — the highest-quality training source for French addresses, with full component structure., ≤5km) | 281 | 264 (z +0.25) | ✓ PASS |
| aggregate golden labelcomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag.-fails | 1101 | 1098 | ✓ −3 (netneural networkA model made of layers of simple numeric units whose connection strengths (weights) are learned from data. The transformer encoder at Mailwoman's core is a neural network. better) |
The finding beneath the numbers: softening the guard recovered countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. recallrecallOf the spans whose gold label is a given tag, the fraction the model found. High recall means few misses. Paired with precision to compute F1. on both distributions at
once — the homographhomonymyOne surface, many referents: 'Georgia' is a US state in 'Atlanta, Georgia' and a country in 'Tbilisi, Georgia.' Handled by disambiguation, split across two stages — the parser resolves the tag from in-string context; the resolver late-binds the referent with geographic context. test (its target) and the WOFWOF (Who's On First). An open-source gazetteer of places maintained by Mapzen/whosonfirst. Mailwoman builds a custom SQLite database from WOF GeoJSON repos, extended with postcode data, importance scores, and coincident-role relations.-admin hierarchy (a bonus). The v263 hard guard was
over-suppressing countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. broadly, not only on true homographshomonymyOne surface, many referents: 'Georgia' is a US state in 'Atlanta, Georgia' and a country in 'Tbilisi, Georgia.' Handled by disambiguation, split across two stages — the parser resolves the tag from in-string context; the resolver late-binds the referent with geographic context.; the country_ambiguous bit fires on any
flagged surface and v263 trusted it too little everywhere. At scale 0.5 the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' trusts it more, recovers
recallrecallOf the spans whose gold label is a given tag, the fraction the model found. High recall means few misses. Paired with precision to compute F1., and precisionprecisionOf the spans the model labeled as a given tag, the fraction it got right. High precision means few false positives. Paired with recall to compute F1. holds (falsifier hallucination unchanged) because the softer bit is still an
informative false-positive signal, just not a near-veto.
The 0.25 probe (v265) — the operator asked; here's the sweep
v265 (country_ambiguous_scale: 0.25, init_from v263, same A/B base) probed whether a stronger soften
recovers more. It does — without over-softening (precisionprecisionOf the spans the model labeled as a given tag, the fraction it got right. High precision means few false positives. Paired with recall to compute F1. + hallucination held):
| scale | countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.-homographhomonymyOne surface, many referents: 'Georgia' is a US state in 'Atlanta, Georgia' and a country in 'Tbilisi, Georgia.' Handled by disambiguation, split across two stages — the parser resolves the tag from in-string context; the resolver late-binds the referent with geographic context. | golden WOFWOF (Who's On First). An open-source gazetteer of places maintained by Mapzen/whosonfirst. Mailwoman builds a custom SQLite database from WOF GeoJSON repos, extended with postcode data, importance scores, and coincident-role relations.-admin countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. | halluc | aggregate fails | coord US / FR |
|---|---|---|---|---|---|
| v263 (1.0, shipped) | 82.6 | 89.3% | 1% | 1101 | flat |
| v264 (0.5) — PROMOTED | 85.1 | 91.1% | 1% | 1098 | PASS / PASS |
| v265 (0.25) | 87.5 | 90.6% | 1% | 1101 | PASS / PASS |
Both scales beat v263 on every axis. The split: 0.25 maximizes the narrow homographhomonymyOne surface, many referents: 'Georgia' is a US state in 'Atlanta, Georgia' and a country in 'Tbilisi, Georgia.' Handled by disambiguation, split across two stages — the parser resolves the tag from in-string context; the resolver late-binds the referent with geographic context. probe (n=54, synthetic), 0.5 maximizes the broad WOFWOF (Who's On First). An open-source gazetteer of places maintained by Mapzen/whosonfirst. Mailwoman builds a custom SQLite database from WOF GeoJSON repos, extended with postcode data, importance scores, and coincident-role relations.-admin countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. (n=224, the #1104 target) and aggregate (n=4255), by ~1 row each. PrecisionprecisionOf the spans the model labeled as a given tag, the fraction it got right. High precision means few false positives. Paired with recall to compute F1. (0 false-positives) and hallucination (1%) held at every scale — 0.25 did not over-soften. Decision (operator, 2026-07-15): promote v264 (0.5) — the balanced default, strongest on the larger-sample metrics; the homographhomonymyOne surface, many referents: 'Georgia' is a US state in 'Atlanta, Georgia' and a country in 'Tbilisi, Georgia.' Handled by disambiguation, split across two stages — the parser resolves the tag from in-string context; the resolver late-binds the referent with geographic context. edge of 0.25 is on a small synthetic slice. 0.25 stays a graded candidate should the homographhomonymyOne surface, many referents: 'Georgia' is a US state in 'Atlanta, Georgia' and a country in 'Tbilisi, Georgia.' Handled by disambiguation, split across two stages — the parser resolves the tag from in-string context; the resolver late-binds the referent with geographic context. lens ever be weighted higher.
Reproduce
- Config:
corpus-python/src/mailwoman_train/configs/v2.6.4-country-softguard.yaml(init_from v263,country_ambiguous_scale: 0.5). - Mechanism:
model.pycountry_feature_scalebuffer + forward multiply; serialized inconfig_dict. - Grade:
scratchpad/grade-v264.sh(export → quantize → package-cache → homographhomonymyOne surface, many referents: 'Georgia' is a US state in 'Atlanta, Georgia' and a country in 'Tbilisi, Georgia.' Handled by disambiguation, split across two stages — the parser resolves the tag from in-string context; the resolver late-binds the referent with geographic context. probe + falsifier + failure-report + gauntlet, all--weights-cache). - int8 md5
3e534072985d92bbbfa8b88d89ec53dc.