v0.9.4 dual-injection + the PIP pivot — the German "collapse" was two different problems

Verdict: v0.9.4 not promoted, and we stop retraining German. Three retrains (v0.9.2 both-order, v0.9.3 regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.-tail, v0.9.4 dual-injection) chased a single international-order localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.-match number that turns out to conflate two unrelated problemsexpectation-maximizationAn iterative algorithm that estimates model parameters when some variables are unobserved. In Mailwoman's matcher, EM learns the Fellegi-Sunter m and u parameters from unlabeled data — no training labels needed. — one of which is a measurement artifact, the other of which no anchor change can touch. A non-gameable metric (PIP-containment) pulled them apart. Tracking: #327.

v0.9.4 result

The dual-injection retrain (model.inject_first_token=true, same corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus., one variable vs v0.9.3) gave the 2×2 (DE localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.-MATCH by name):

	anchor OFF	anchor ON
US order	47.3%	43.7%
native DE	48.3%	83.5%

US 96.4%, FR 84.9% (no-regression OK). International anchor-ON localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. is 43.7%, below v0.9.3's 44.7% and far under the 70% promote bar. Adding the position-0 countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. cue moved regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.-match (38→58%) and resolved-rate (72→88%) but not localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.-match. Across all three retrains the anchor consistently lowers international localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.-match (off ~47 > on ~44) while coordinate p50 holds ~6 km. That pattern — the geography stays right while the string-match metric drops — is the tell.

PIP-containment splits the problem in half

So we measured the thing that can't be gamed: is the gold OpenAddressesOpenAddresses (OA). A global open aggregation of address points collected from many official sources. A primary source of component-supervised training data outside proprietary registries. point inside the polygon of the localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. we resolved? (scripts/eval/pip-containment.py, point-in-polygon over the WOF GeoJSONWOF GeoJSONThe raw one-feature-per-file GeoJSON distributed by Who's On First repositories — the input that Mailwoman's WOF SQLite build consumes..)

intl DE (v0.9.4, 3000 rows)   name-match   PIP-containment   delta
OVERALL                          43.7%          56.1%        +12.4pp
Berlin   (n=1500)                36.3%          36.3%         -0.1pp
Sachsen  (n=1500)                51.1%          75.9%        +24.8pp

Two completely different stories under one average.

Saxony is a name-match artifact. The modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' and resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon. place Saxon addresses correctly 76% of the time, but the name-match metric only credits 51% — a 24.8-point gap. The reason is visible in every miss: OpenAddressesOpenAddresses (OA). A global open aggregation of address points collected from many official sources. A primary source of component-supervised training data outside proprietary registries. gold carries a regional suffix that WOFWOF (Who's On First). An open-source gazetteer of places maintained by Mapzen/whosonfirst. Mailwoman builds a custom SQLite database from WOF GeoJSON repos, extended with postcode data, importance scores, and coincident-role relations.'s canonical name drops.

gold "Plauen Vogtl"   resolved "Plauen"       (point inside Plauen ✓)
gold "Chemnitz Sachs" resolved "Chemnitz"     (point inside Chemnitz ✓)
gold "Marienberg Erzgeb" resolved "Marienberg" (point inside Marienberg ✓)
gold "Treuen Vogtl"   resolved "Treuen"       (point inside Treuen ✓)

These are not errors. The resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon. found the right place; the metric demands an exact string the gazetteergazetteerA geographical index that maps place names and postcodes to real-world coordinates. Mailwoman uses a custom-built Who's On First (WOF) SQLite database as its gazetteer — the 'atlas' half of the grammar/atlas architecture. doesn't use. Retraining cannot fix a measurement bug.

Berlin is a genuine failure — but not the one we were retraining for. Berlin's PIP equals its name-match (36.3%), so there's no hidden artifact. Looking closer: of 1,500 Berlin rows, 955 resolve to nothing at all, and the 545 that resolve all resolve correctly to Berlin. Berlin's problem is not a wrong pick or a too-small polygon — it's that the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' drops the localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. spanspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree. entirely in the city-statedual-role placeA place that is two placetypes at once — Berlin as both a city and a state, Washington DC as city and district. Resolved using the coincident-roles relation plus hierarchy completion. layout …, Berlin, Berlin 14199, where localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. and regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. are the same word. One "Berlin" gets labeled regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality., the other is lost, and there's nothing left for the resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon. to place. This is a real segmentation bug, but it's specific to city-statesdual-role placeA place that is two placetypes at once — Berlin as both a city and a state, Washington DC as city and district. Resolved using the coincident-roles relation plus hierarchy completion. (Berlin/Hamburg/Bremen) and has nothing to do with the postcode anchoranchor inferenceA technique where structured knowledge (postcode locations, gazetteer place names) is injected into the model as soft input features — not as deterministic overrides. The model still decides the final labels, but the anchor signal biases it toward correct admin tags. or word order in general — which is exactly why three anchor/order retrains never touched it.

Native German is essentially solved

The other half of the story is native order — the layout real feeds and users actually use. By PIP it is 96.2% (anchor on), against a name-match that said 83.5%, a 12.7pp undersell. Berlin native is 99.2%, so the city-statedual-role placeA place that is two placetypes at once — Berlin as both a city and a state, Washington DC as city and district. Resolved using the coincident-roles relation plus hierarchy completion. bug is international-order ONLY. The full 2×2 (scripts/eval/de-pip-eval.sh):

order	anchor	name-match	PIP-containment
native	on	83.5%	96.2%
native	off	48.3%	61.0%
intl	on	43.7%	56.1%
intl	off	47.3%	59.6%

Two things fall out. The anchor does real work on native order (PIP 61 → 96), and it slightly hurts international PIP (59.6 off → 56.1 on) — the anchor was never the international lever. And native German, measured honestly, beats PeliasPeliasAn open-source geocoder, Mailwoman's spiritual predecessor. comfortably and is done. The genuine residual is one thing: Berlin in international order. Everything else is the name-match artifact.

Why three retrains "failed"

They optimized an aggregate that was half artifact, half a bug the lever couldn't reach. The anchor was never the problem (coord p50 ~6 km throughout); the metric was conflating a gazetteergazetteerA geographical index that maps place names and postcodes to real-world coordinates. Mailwoman uses a custom-built Who's On First (WOF) SQLite database as its gazetteer — the 'atlas' half of the grammar/atlas architecture.-name mismatch in Saxony with a city-statedual-role placeA place that is two placetypes at once — Berlin as both a city and a state, Washington DC as city and district. Resolved using the coincident-roles relation plus hierarchy completion. parseaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates. failure in Berlin. With those separated, neither calls for another anchor retrain.

What's next (no A100)

DeepSeek signed off this pivot across four turns. The two real fixes, both off the GPU:

1. ResolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon. name-match: alias the regional suffixes (Vogtl→Vogtland, Sachs→Sachsen, Erzgeb→ Erzgebirge, OL→Oberlausitz, …) so Plauen Vogtl credits Plauen. This recovers the 24.8pp Saxon gap in the evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. metric without any modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' change. Filed as a follow-up.
2. Berlin city-statedual-role placeA place that is two placetypes at once — Berlin as both a city and a state, Washington DC as city and district. Resolved using the coincident-roles relation plus hierarchy completion. segmentation — the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' needs to learn the City, City Postcode layout where localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. == regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.. A focused data-augmentation pass (city-statedual-role placeA place that is two placetypes at once — Berlin as both a city and a state, Washington DC as city and district. Resolved using the coincident-roles relation plus hierarchy completion. addresses in both orders) is the fix, but it's a gated retrain for a future shift, not tonight. Filed as a follow-up.

v0.9.5 (the [COUNTRY_DE] start-tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. that was queued if PIP fell below 90%) is cancelled: the countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. signal helps neither half — Saxony is already correct, and Berlin's countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. is never in doubt. The corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus. and anchor levers are spent; the German story moves to the resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon. and a narrow city-statedual-role placeA place that is two placetypes at once — Berlin as both a city and a state, Washington DC as city and district. Resolved using the coincident-roles relation plus hierarchy completion. data fix.