Skip to main content

#822 placer-frontier diagnostic — bare vs country-hint, by country

⚠ Config caveat + correction (added after the fact). These numbers were measured on the default drop-in config — the admin gazetteergazetteerA geographical index that maps place names and postcodes to real-world coordinates. Mailwoman uses a custom-built Who's On First (WOF) SQLite database as its gazetteer — the 'atlas' half of the grammar/atlas architecture. (admin-global-priority.db) with NO candidate-ranking DB (MAILWOMAN_CANDIDATE_DB unset, which is exactly what npx @mailwoman/nominatim serve gives). They are valid for that config, but resolution is highly config-dependent and I over-framed them as the general non-US recallrecallOf the spans whose gold label is a given tag, the fraction the model found. High recall means few misses. Paired with precision to compute F1. truth. With a candidate DB loaded, large distinctive cities that read as "coveragecoverageThe fraction of a population or region for which a data source has real, non-placeholder entries — e.g. 47% rooftop coverage on Texas addresses. Distinct from accuracy on the rows that are present." misses here actually resolve (London 553→2 km, Beijing 1661→9 km), so the "residual coveragecoverageThe fraction of a population or region for which a data source has real, non-placeholder entries — e.g. 47% rooftop coverage on Texas addresses. Distinct from accuracy on the rows that are present." framing below conflated true coveragecoverageThe fraction of a population or region for which a data source has real, non-placeholder entries — e.g. 47% rooftop coverage on Texas addresses. Distinct from accuracy on the rows that are present. with a missing candidate table. The candidate backend is population-first and ignores the countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. hint, and the builds vary (the -intl build even regressed the aggregate to ~12.8%), so the precise bare/+hint percentages and the exonym-vs-coveragecoverageThe fraction of a population or region for which a data source has real, non-placeholder entries — e.g. 47% rooftop coverage on Texas addresses. Distinct from accuracy on the rows that are present. split are NOT settled — they need the operator's canonical candidate config pinned, then a re-run.

What survives, config-independent: the placer-namesake gap is real — Vienna, Austria, Sydney, Warsaw, Toronto resolve to their US namesakes even with a candidate DB loaded, because the population-first ranking can't break the tie without a countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. prior. That's #822. And the default drop-in's non-US recallrecallOf the spans whose gold label is a given tag, the fraction the model found. High recall means few misses. Paired with precision to compute F1. is poor without a candidate DB. A separate exonym-sizing pass (try-the-alternatenames) was withdrawn — it was both cap-buggy and confounded by this same config gap.

geonamesGeoNamesA free global gazetteer combining administrative, postal, and POI data across 200+ countries. Supplements Who's On First for postcode centroids and places where WOF has gaps. cities15000, top 3/countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. by population (≥ 50000). "Resolved" = within 50 km of the citylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.'s true coordinate. Bare = no countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. constraint (what the drop-in sends for a bare query); +hint = with the countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. as a countrycodes constraint. The bare→hint lift is what growing the placer would buy (#822); what stays unresolved with a hint is the exonym/coveragecoverageThe fraction of a population or region for which a data source has real, non-placeholder entries — e.g. 47% rooftop coverage on Texas addresses. Distinct from accuracy on the rows that are present. lever.

  • Cities: 506 across 187 countries
  • Resolve-rate bare: 29.2% → +hint: 46.6% (lift +17.4 pp)
  • Bare US-namesake misroutes: 16.2% (82/506) — undercounts the placer gap
  • Countries: 54 bare-supported · 36 placer-recoverable (#822) · 97 residual
  • Residual splits: 92 name-not-found (English name matches no in-countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. record — exonym fix where the record exists under a local name, else coveragecoverageThe fraction of a population or region for which a data source has real, non-placeholder entries — e.g. 47% rooftop coverage on Texas addresses. Distinct from accuracy on the rows that are present.-absence) · 5 wrong-place (coveragecoverageThe fraction of a population or region for which a data source has real, non-placeholder entries — e.g. 47% rooftop coverage on Texas addresses. Distinct from accuracy on the rows that are present./disambiguation)

How to read this. Bare resolve-rate is the placer ceiling, not the geocoder's capability — a bare query carries no countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. hint. The +hint column is the honest #822 prize: countries that resolve once the countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. is known but not before. The residual set fails even with the hint, so the placer can't fix it — that's alt-name (Warsaw/Warszawa) + gazetteergazetteerA geographical index that maps place names and postcodes to real-world coordinates. Mailwoman uses a custom-built Who's On First (WOF) SQLite database as its gazetteer — the 'atlas' half of the grammar/atlas architecture. coveragecoverageThe fraction of a population or region for which a data source has real, non-placeholder entries — e.g. 47% rooftop coverage on Texas addresses. Distinct from accuracy on the rows that are present., a parallel lever.

Placer-recoverable (#822) — a country hint fixes it; growing the placer captures it​

CountrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.ISO2Bare+hint
AngolaAO0/32/3
ArgentinaAR0/32/3
BoliviaBO0/32/3
ChinaCN0/32/3
EcuadorEC0/32/3
BahrainBH1/32/3
Sri LankaLK1/32/3
MalaysiaMY1/32/3
SerbiaRS1/32/3
SlovakiaSK1/32/3
ThailandTH1/32/3
UkraineUA1/32/3
AustraliaAU0/33/3
BelarusBY0/33/3
CanadaCA0/33/3
Costa RicaCR0/33/3
EgyptEG0/33/3
IrelandIE0/33/3
MoroccoMA0/33/3
Russian FederationRU0/33/3
SingaporeSG0/33/3
South AfricaZA0/33/3
United Arab EmiratesAE1/33/3
BangladeshBD1/33/3
BulgariaBG1/33/3
Cote DIvoireCI1/33/3
ColombiaCO1/33/3
CubaCU1/33/3
Dominican RepublicDO1/33/3
AlgeriaDZ1/33/3
LatviaLV1/33/3
New ZealandNZ1/33/3
Saudi ArabiaSA1/33/3
TurkeyTR1/33/3
UruguayUY1/33/3
VenezuelaVE1/33/3

Residual A — name-not-found (exonym fix, or coverage-absence)​

The hint returns NOTHING: the English query name matches no place in the countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.. Where the record exists under a LOCAL name (Warsaw vs Warszawa — proven end-to-end), indexing alt-name surface forms onto the candidate table fixes it cheaply (#823, no modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' change). Where the countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. has no candidate records at all, it's coveragecoverageThe fraction of a population or region for which a data source has real, non-placeholder entries — e.g. 47% rooftop coverage on Texas addresses. Distinct from accuracy on the rows that are present.. European exonyms dominate; the per-countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. split needs a local-name probe. hint→∅ = of the hint-unresolved cities, how many returned no result (vs a wrong place).

CountrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.ISO2Bare+hinthint→∅
AfghanistanAF0/30/33/3
Antigua and BarbudaAG0/10/11/1
AlbaniaAL0/30/33/3
ArmeniaAM0/30/33/3
AzerbaijanAZ0/30/33/3
Bosnia and HerzegovinaBA0/30/33/3
BarbadosBB0/10/11/1
Burkina FasoBF0/30/33/3
BurundiBI0/30/33/3
BeninBJ0/30/33/3
Brunei DarussalamBN0/10/11/1
BahamasBS0/10/11/1
BhutanBT0/10/11/1
BotswanaBW0/30/33/3
BelizeBZ0/10/11/1
Congo, the Democratic Republic of theCD0/30/33/3
Central African RepublicCF0/30/33/3
CongoCG0/30/33/3
Cape VerdeCV0/20/22/2
CuraçaoCW0/10/11/1
CyprusCY0/30/33/3
DjiboutiDJ0/20/22/2
EritreaER0/20/22/2
FijiFJ0/30/33/3
GabonGA0/30/33/3
GeorgiaGE0/30/33/3
GambiaGM0/30/33/3
GuineaGN0/30/33/3
GuadeloupeGP0/10/11/1
Equatorial GuineaGQ0/20/22/2
Guinea-BissauGW0/20/22/2
GuyanaGY0/10/11/1
Hong KongHK0/30/33/3
HondurasHN0/30/33/3
HaitiHT0/30/33/3
JamaicaJM0/30/33/3
KyrgyzstanKG0/30/33/3
ComorosKM0/10/11/1
Korea, Democratic People's Republic ofKP0/30/33/3
KuwaitKW0/30/33/3
Lao People's Democratic RepublicLA0/30/33/3
LiberiaLR0/30/33/3
LesothoLS0/20/22/2
LibyaLY0/30/33/3
Moldova, Republic ofMD0/30/33/3
MontenegroME0/20/22/2
MadagascarMG0/30/33/3
Macedonia, the Former Yugoslav Republic ofMK0/30/33/3
MaliML0/30/33/3
MongoliaMN0/30/33/3
MacaoMO0/30/33/3
MartiniqueMQ0/10/11/1
MauritaniaMR0/30/33/3
MauritiusMU0/30/33/3
MaldivesMV0/10/11/1
MalawiMW0/30/33/3
MozambiqueMZ0/30/33/3
NamibiaNA0/30/33/3
New CaledoniaNC0/10/11/1
NigerNE0/30/33/3
NicaraguaNI0/30/33/3
Papua New GuineaPG0/20/22/2
Puerto RicoPR0/30/33/3
Palestine, StateregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. ofPS0/30/33/3
ParaguayPY0/30/33/3
RwandaRW0/30/33/3
Solomon IslandsSB0/10/11/1
SudanSD0/30/33/3
Sierra LeoneSL0/30/33/3
SomaliaSO0/30/33/3
SurinameSR0/10/11/1
South SudanSS0/30/33/3
Sao Tome and PrincipeST0/10/11/1
El SalvadorSV0/30/33/3
Syrian Arab RepublicSY0/30/33/3
SwazilandSZ0/20/22/2
TogoTG0/30/33/3
TajikistanTJ0/30/33/3
Timor-LesteTL0/10/11/1
TurkmenistanTM0/30/33/3
Trinidad and TobagoTT0/30/33/3
UzbekistanUZ0/30/33/3
Virgin Islands (U.S.)VI0/10/11/1
YemenYE0/30/33/3
ZambiaZM0/30/33/3
ZimbabweZW0/30/33/3
ReunionRE1/30/33/3
ChadTD1/30/33/3
GreeceGR0/31/32/2
HungaryHU0/31/32/2
BelgiumBE1/31/32/2
DenmarkDK1/31/32/2

Residual B — gazetteer coverage / disambiguation​

The hint returns a WRONG place: the countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. has a same-name match but the target citylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. isn't in the candidate gazetteergazetteerA geographical index that maps place names and postcodes to real-world coordinates. Mailwoman uses a custom-built Who's On First (WOF) SQLite database as its gazetteer — the 'atlas' half of the grammar/atlas architecture., or loses disambiguation. Needs more data, not alt-names (Beijing, Rio).

CountrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.ISO2Bare+hinthint→∅
BrazilBR0/30/30/3
IraqIQ0/31/31/2
PolandPL0/31/31/2
SwedenSE1/31/30/2
United StatesregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.US1/31/30/2

Bare-supported (≥50% resolve with no hint) — US + the #743 safelist + tail​

CountrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.ISO2Bare
Western SaharaEH1/2
French GuianaGF1/1
MayotteYT1/1
AustriaAT2/3
SwitzerlandCH2/3
CameroonCM2/3
Czech RepublicCZ2/3
United KingdomGB2/3
GuatemalaGT2/3
CambodiaKH2/3
LebanonLB2/3
MexicoMX2/3
NigeriaNG2/3
NepalNP2/3
PanamaPA2/3
PhilippinesPH2/3
PortugalPT2/3
RomaniaRO2/3
UgandaUG2/3
ChileCL2/3
EstoniaEE2/3
IsraelIL2/3
IndiaIN2/3
Iran, Islamic Republic ofIR2/3
KazakhstanKZ2/3
MyanmarMM2/3
NorwayNO2/3
OmanOM2/3
PeruPE2/3
PakistanPK2/3
QatarQA2/3
SenegalSN2/3
TunisiaTN2/3
GermanyDE3/3
SpainES3/3
EthiopiaET3/3
FinlandFI3/3
FranceFR3/3
GhanaGH3/3
CroatiaHR3/3
IndonesiaID3/3
IcelandIS1/1
ItalyIT3/3
JordanJO3/3
JapanJP3/3
KenyaKE3/3
Korea, Republic ofKR3/3
LithuaniaLT3/3
LuxembourgLU1/1
NetherlandsNL3/3
SloveniaSI2/2
Taiwan (ProvinceregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. of China)TW3/3
Tanzania, United Republic ofTZ3/3
Viet NamVN3/3