Dual-role places and hierarchy completion

Some places are two things at once. Berlin is a citylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. and a federal stateregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.. Milano is a comune and the provinceregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. named after it. Bristol is a citylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. and a unitary authority. Utrecht is a citylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. and a provinceregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.. A census of our gazetteergazetteerA geographical index that maps place names and postcodes to real-world coordinates. Mailwoman uses a custom-built Who's On First (WOF) SQLite database as its gazetteer — the 'atlas' half of the grammar/atlas architecture. found about 122 of these across nine countries — Italy alone has 70 (its provincesregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. are nearly all named after their capital), then Spain (14), the UK (12), Japan (10), Korea (7), and a handful each for France, Germany, the Netherlands, and China.

These cause a specific failure. In international order — 5 Hauptstraße, Berlin, Berlin 10115 — the citylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. and the stateregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. are the same word, and the parser often labelscomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag. one Berlin as the regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. and drops the other entirely. The resolved tree then carries a regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. but no localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy., and the resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon. has nothing to place: on a 1,500-row Berlin evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error., 955 rows resolved to nothing.

Completing the hierarchy

When the parser drops the localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. of a dual-role placedual-role placeA place that is two placetypes at once — Berlin as both a city and a state, Washington DC as city and district. Resolved using the coincident-roles relation plus hierarchy completion., the gazetteergazetteerA geographical index that maps place names and postcodes to real-world coordinates. Mailwoman uses a custom-built Who's On First (WOF) SQLite database as its gazetteer — the 'atlas' half of the grammar/atlas architecture. still knows what it was — the regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. and the localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. are two linked records with the same name and a coincident centroid. So the resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon. can finish the job. With hierarchyCompletion enabled, after the walk, if a regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. resolved and the tree has no localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. node, the resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon. looks the regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. up in a precomputed coincident-roles relationcoincident rolesA precomputed Who's On First relation that pairs a region with its same-named locality (Berlin city ↔ Berlin state) so the resolver can complete the hierarchy at resolution time. and synthesizes the missing localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. from it.

The relation is the important part. Rather than a hardcoded list of city-statesdual-role placeA place that is two placetypes at once — Berlin as both a city and a state, Washington DC as city and district. Resolved using the coincident-roles relation plus hierarchy completion. or a magic distance threshold, it's derived from the gazetteergazetteerA geographical index that maps place names and postcodes to real-world coordinates. Mailwoman uses a custom-built Who's On First (WOF) SQLite database as its gazetteer — the 'atlas' half of the grammar/atlas architecture.'s own structure at build time: a regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. and a localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. are recorded as a pair when they share a name, the localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. descends from the regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. in the ancestors table, and their centroids fall within a relative tolerance (a fraction of the regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.'s bounding-box diagonal, so a large Italian provinceregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. admits a citylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. tens of kilometres from its centroid while a tiny city-statedual-role placeA place that is two placetypes at once — Berlin as both a city and a state, Washington DC as city and district. Resolved using the coincident-roles relation plus hierarchy completion. stays tight). At runtime the resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon. does an O(1) membership lookup — no distance maths, no constant to tune. Build the table once with mailwoman-wof-build-coincident-roles; it ships in the admin gazetteergazetteerA geographical index that maps place names and postcodes to real-world coordinates. Mailwoman uses a custom-built Who's On First (WOF) SQLite database as its gazetteer — the 'atlas' half of the grammar/atlas architecture..

Each completion carries a relationship_type (city-state, capital-seat, or consolidated-county) on metadata.relationship_type, and the synthesized node is marked metadata.resolver_synthesized = true — it has no spanspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree. in the raw input, so a consumer can tell it apart from a parsed component.

Picking the right city

A few admins map to more than one same-name localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy., so completion has to choose. The rule is population first: the principal citylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. is the populous one, and it can sit farther from the admin centroid than a tiny same-name hamlet — Niigata prefecture's centroid is dragged out to sea, so the real Niigata citylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. is 41 km away while a 0-population point is closer. Nearest centroid breaks a population tie. When two candidates tie on both, completion abstains rather than guess, because a wrong localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. is worse than a missing one.

Completion is on by default. A Berlin address that used to resolve to nothing comes back with its localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.; pass hierarchyCompletion: false to opt out, and resolution is byte-for-byte what it was. It's safe by construction — completion only fires for a dual-role regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. whose localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. the parser dropped, and no-ops entirely when the loaded gazetteergazetteerA geographical index that maps place names and postcodes to real-world coordinates. Mailwoman uses a custom-built Who's On First (WOF) SQLite database as its gazetteer — the 'atlas' half of the grammar/atlas architecture. carries no coincident_roles table.

Attaching the lineage

A related option, includeAncestors, stamps each resolved node's full containment chain — county, regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality., countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head., nearest-first — onto metadata.ancestors. This is the move PeliasPeliasAn open-source geocoder, Mailwoman's spiritual predecessor. and Nominatim make as a matter of course: a single resolved place comes back carrying its whole admin ladder. It's the same gazetteergazetteerA geographical index that maps place names and postcodes to real-world coordinates. Mailwoman uses a custom-built Who's On First (WOF) SQLite database as its gazetteer — the 'atlas' half of the grammar/atlas architecture. ancestors data the coincident-roles relationcoincident rolesA precomputed Who's On First relation that pairs a region with its same-named locality (Berlin city ↔ Berlin state) so the resolver can complete the hierarchy at resolution time. is built from, and it's also off by default.

Where this sits

Hierarchy completionhierarchy completionThe post-resolution step that synthesizes a dropped locality when a dual-role region resolves but the parser omitted the city (e.g. supplying 'Berlin' the city under 'Berlin' the state). lives in the resolve stagestageOne of the dataflow stages in the runtime pipeline (normalize, locale gate, kind classify, phrase group, token classify, sequence correct, reconcile, resolve). Distinct from tier (model vocabulary) and phase (plan milestone). as a post-walk pass. The cleaner long-term home is the reconcile stage, whose concordanceconcordanceA joint-decode signal that rewards a parse whose spans form a consistent Who's On First parent-child chain and vetoes contradictory ones — a hard veto for conflicts, a log-space bonus for full agreement. scoring already ranks parsesaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates. by how well their resolved places agree in the admin hierarchy — exactly the signal that says "the regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. Berlin and the localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. Berlin are the same place." Until joint decodingjoint decodingA decode strategy where the neural model's proposals and rule-based solver's output are reconciled into a single parse tree. Formerly the default (Route A Phase II), now retired in favor of argmax after it was found to break the street+house_number geocode precondition. is the default decode path, the relation-driven completion bridges the gap: it recovers the dropped localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. from the gazetteergazetteerA geographical index that maps place names and postcodes to real-world coordinates. Mailwoman uses a custom-built Who's On First (WOF) SQLite database as its gazetteer — the 'atlas' half of the grammar/atlas architecture. without needing the classifier to have surfaced it. The coincident-roles relationcoincident rolesA precomputed Who's On First relation that pairs a region with its same-named locality (Berlin city ↔ Berlin state) so the resolver can complete the hierarchy at resolution time. and the lineage attachment are both data a future concordanceconcordanceA joint-decode signal that rewards a parse whose spans form a consistent Who's On First parent-child chain and vetoes contradictory ones — a hard veto for conflicts, a log-space bonus for full agreement. default would score against.