So you ship that parser. It looks great on the addresses you tested it against. And you have no idea what it's dropping in production, because the output format was built to hide exactly that.

We got tired of not knowing. So we made the decoder keep a promise: every byte of the input lands in exactly one typed span, including the bytes we can't classify. The runs the model leaves untagged stop being absence and become a thing you can hold — an unknown span, with a start, an end, and the literal text the model couldn't place. Concatenate all the spans back together and you get the original string, character for character. Nothing gets lost, because there's nowhere left for it to go.

That's a small change to a decoder. It turns into a diagnostic you can point at production addresses.

The shopping list​

Once the gaps are typed, you can count them. We ran the parser over a few thousand real addresses and folded every unknown span into a frequency table: a ranked list of what the model systematically fails to read. We started calling it the shopping list, because that's what it is. The next retrain, in priority order, written by the model's own failures instead of our hunches.

The first thing the list did was catch us lying to ourselves. Ninety-eight percent of inputs had an unknown span. Ninety-eight! That number is worthless, and it took about four minutes to see why: the gaps were almost all whitespace and commas, the spaces between tokens that no component owns. A metric that's mostly punctuation will tell you the sky is falling on every single address. Once we separated the trivial delimiter gaps from gaps holding an actual letter or digit, the honest figure dropped to nine percent. Nine percent of addresses carry a run of real content the model can't place. That one you can act on.

What it can't read is Polish​

Then we aimed the same tool at OpenAddresses data from nine countries, and the shopping list stopped being abstract.

The United States and Australia came back spotless: zero content gaps. France, ten percent. Italy, two. And then Czechia at eighty-four percent and Poland at seventy-seven, towering over everything else on the board.

The top entries said why in the plainest possible terms: ł, á, ó, í, ě, ś, ę, č. The whole Latin-Extended block. Our tokenizer isolates a Slavic diacritic into its own little subword, the model was never shown enough of them to learn what they are, so it files them under background noise. Grudziądz comes apart at the ą: the model labels Grudzi, drops the accent, and picks the city back up at dz, as if the letter in the middle were a coffee stain. Bohaterów splits at the ó and takes the neighboring house number down with it.

We had a comfortable theory that this was a decoder bug, a character-offset mismatch around multi-byte characters, the kind of thing you fix in an afternoon without touching a GPU. We checked before we wrote it down, which is the entire point of the exercise. It wasn't the decoder. The offsets were fine; the model simply doesn't know these letters, and there is no clever preprocessing that fixes not-knowing. It's a retrain. The difference is that now we know precisely which characters to weight and which locales to leave alone, because Australia and the US don't need a thing.

Honesty you can query​

A model that guesses without a word is a liability, because you learn what it can't do from a customer. A model that hands you a typed, countable record of its own blind spots is something you can plan around. We didn't make the parser smarter this week. We made it honest about what it doesn't know, and that honesty turned out to be a feature you can run a GROUP BY over.

The shopping list is checked in, and it runs before and after every retrain now. Next time you build something that classifies text, make it account for the text it couldn't classify. You'll be surprised how loudly it starts telling you where to look.

Update — July 2​

We wrote "it's a retrain" and then we tested it, because writing a thing down is not the same as it being true. The retrain held English and made Czech worse. What the ranked list had actually found was one layer lower: the tokenizer's vocabulary carried every one of those characters as its own lonely subword and not a single piece containing them, and a unigram model cannot emit a subword that isn't in its table. No volume of training data teaches a model to say a word its alphabet can't spell.

So the fix wasn't a retrain. We trained a small SentencePiece model on Czech and Polish addresses, spliced its 10,582 diacritic-bearing pieces into the vocabulary, and initialized each new embedding from the mean of the pieces it used to shatter into. Zero GPU hours. Wrong-city rates fell from 22.4% to 14.8% in Czechia and from 27.9% to 7.9% in Poland, and Slovak and Slovenian came along for free because they share the codepoints. It shipped the next day.

The shopping list paid for itself in under twenty-four hours, which is the part worth repeating: the fix was cheap because the blind spots were countable. Grudziądz is whole now.

It does this with Sydney too (a hamlet in Ohio), and Zurich (Kansas), and Toronto (also Ohio, as it happens). The pattern is always the same: you name a famous foreign city, you name the country it's in, and the geocoder hands you back its smallest American namesake like it's doing you a favor.

The fix that felt like progress​

When you find a bug like this, the first idea is almost always the expensive one. Ours was: the country router doesn't know enough countries. Teach it that "Austria" means Austria — feed it more examples, retrain the model, ship a new set of weights. It's a satisfying plan. It feels like progress, because training a model is the kind of work that looks like work.

We had the GPU budget queued and the issue written up as a model gap. And then, because we've been burned before by fixing the wrong layer, we did the cheap thing first: we asked the model what it actually saw.

Here is Vienna, Austria straight out of the parser:

Vienna   → locality
Austria  → country

The model knew. It tagged Austria as a country, cleanly, every time — Australia, Switzerland, Canada, all of them. There was no model gap. The thing we were about to spend a retrain on was already working perfectly. The bug lived one layer down, in the part of the system that takes a correct parse and turns it into a place on Earth.

Forty-five Viennas and a popularity contest​

The resolver's job is to take the word "Vienna" and decide which Vienna. Our gazetteer knows about forty-five of them in the United States alone, plus the real one in Austria, plus a few others. Faced with that pile, the resolver did the reasonable-sounding thing: it picked the most prominent match by population and name — and then it threw the country token away. It never asked the one question the address had already answered for it.

So Vienna resolved on its own merits, the populous American candidates outranked the Austrian one in the bag, and Austria sat there in the parse, ignored, like a return address nobody read. The model had said which country in plain text, and the resolver wasn't listening.

The honest version of the bug: the parse and the resolve each made a locally sensible decision, and nobody checked that they agreed.

The fix we didn't make​

There's a tempting shortcut here, and it's worth naming because we've watched it sink geocoders before. You could keep a list. "If the address says Austria, force the country to AT." It would work for Vienna, and Sydney, and Zurich. It would also be the first paving stone on the road to a thousand-entry exception table that someone maintains forever and nobody trusts — the same road that turned older rules-based geocoders into archaeology. We don't pin, and we don't keep lists of special cases the model is supposed to defer to.

The fix instead asks the parse and the resolve to be consistent with each other. When the address carries an explicit country and the locality it resolved to lives somewhere else, re-pick the locality to the one that actually sits inside the country the address named. The country code comes from the model's own country emission, normalized through an ISO table we already had on the shelf; the gazetteer's own country column does the rest. No list. No prior. It generalizes to every country at once, because it isn't about any particular country — it's about the address agreeing with itself.

There's a guard rail worth mentioning, because it's where this kind of fix usually goes wrong. It only fires when the country is the nearest thing the address says about geography. Write Springfield, IL, USA and the state is right there doing the disambiguating, so the new pass stays out of the way and Springfield stays in Illinois — not the largest Springfield in America. The fix earns its keep precisely by knowing when not to act.

What it bought, and what it cost​

We measure these things against coordinates, not labels, because a name-match metric would have told us this bug didn't exist. The honest yardstick is a bare City, Country query — the exact thing a user types into the box — scored by how close the answer lands to the real place.

Forty-five countries that used to need a hint and now don't, a twenty-four-point jump on the query shape people actually use, and the cost on the GPU meter was zero. The thing we'd budgeted a retrain for was a hundred-odd lines in the resolver.

The cost it does carry is the honest kind: it can only re-pick to a place the gazetteer actually has. Where we don't carry a city, the country in the address can't summon it from nothing, and the fix correctly does nothing rather than guess. That's a coverage problem, and it's a different lever for a different day.

The part worth keeping​

We almost retrained a model to fix a bug the model didn't have. The diagnostic that saved us was thirty seconds of parsing seven addresses and reading the output — cheaper than the issue we'd written, never mind the GPU hours behind it.

The reflex says train. The discipline says look first. The model knew it was Austria the whole time; we just had to teach the rest of the system to listen.

(One more for the honesty file: the test that proves Sydney now lands in Australia failed the first time I ran it — by 7,532 kilometers. The fix was fine. I'd dropped the minus sign on Sydney's latitude and parked it in the northern hemisphere. The gate caught my typo before it caught a real regression, which is exactly the order you want.)

The geocoder couldn't find Tirana​

Start with the symptom. Ask mailwoman's drop-in API for Tirana, Albania and you got nothing. Same for Yerevan, Kabul, Ulaanbaatar, Sarajevo — 147 countries where the candidate gazetteer, the table the demo and the drop-ins actually query, held exactly zero rows. You didn't get a wrong coordinate, you got silence, as if the country didn't exist.

That gap is easy to miss, because the cities that do resolve — Berlin, Paris, Tokyo — resolve beautifully, and those are the ones you check by hand. The hole was in the long tail of countries nobody on the team types into a demo at 2am.

If you've shipped a data pipeline you know how this happens. A from-scratch rebuild enriched every existing place with alternate-language names, so Berlin now answers to Berlino, Берлин, 柏林, all 134 of its surface forms. Wonderful. But enriching the places you have does nothing for the places you don't, and the gap-fill that was supposed to add the missing countries lived in a side index the candidate table never folded in. Every piece worked. The join between them was empty, and nothing failed loudly enough to notice.

The plan was wrong, and a ten-minute probe said so​

The plan blamed exonyms — the Warsaw-versus-Warszawa problem, where the address says one name and the gazetteer holds another. Reasonable theory, and an expensive fix: a model retrain to learn the aliases.

So before betting the night on it, we ran the cheap version first. For the top cities of every failing country, is the local name already in the table? If it is, exonyms are the problem. If it isn't, it's coverage. The answer came back in ten minutes: exonym reliance was about one percent. The alt-names were already there. The cities themselves were missing.

That killed the expensive theory and pointed at a boring data fix: pull the real GeoNames town-level dumps for all 147 countries and rebuild the table. It worked the way data fixes do — completely. Before, zero of twenty-five sampled capitals resolved. After, twenty-five of twenty-five, every one inside six kilometers, Tirana to the meter. The countries that already worked came through byte-for-byte identical, because every new row sat in a country that previously had none. No regression to argue about.

That's the win, and we staged it to ship. It's the part you'd put in the changelog. I learned more from what came next.

Then the eval lied to me​

With coverage handled, we asked a different question: when mailwoman does parse a foreign address, does it land in the right place? We ran the assembled-coordinate eval on Portugal, Poland, and Australia — real addresses, scored on distance to truth.

The numbers were ugly. Portugal's median error came back at 47 kilometers. Poland 116. Australia 798. The parser was reading the street name as the city and resolving to a same-named town three regions over. Feed it a perfect parse and the error dropped under a kilometer. Clean story: the parser is the bottleneck, this earns a GPU retrain, write it up as a GO.

I almost did. Then I checked which model the eval had loaded.

It was a six-week-old build. The dev symlink that points at "the shipped model" had drifted to an old one — a test re-creates that symlink as a side effect, and it had been pinning the wrong file for weeks. Every eval that loaded the model by its default path was grading something we retired in May.

Point the eval at what we actually ship and Portugal's 47 kilometers becomes 0.8. Poland 2.3. Australia 1.2. The disaster evaporated. The shipped parser was fine the whole time; the only broken thing was the ruler I measured it with. The retrain I was about to recommend would have fixed a problem that didn't exist — and the multilocale parser that does fix this, for the cases where it's real, already sits in a validated checkpoint waiting on a promotion that never came. We didn't need a GPU. We needed to read a symlink.

What went wrong, both times​

Twice in one night the thing being measured wasn't the thing we thought. The plan's diagnostic ran without the candidate database loaded, so it blamed coverage on exonyms — the same missing-database confound that put Warsaw in Indiana and faked a whole scoreboard a few weeks back. My parser eval ran against a stale model, so it blamed the shipped parser for a retired one's mistakes. Same failure wearing two costumes: the harness reported its numbers and stayed quiet about its assumptions.

The only thing that caught either one was the same dull habit — before you trust an aggregate, check what produced it. Dump the rows. Read the config. Print the path the symlink resolves to. It is never the interesting work, and it is most of the job. A model that scores 47 kilometers and a model that scores 0.8 can be the same model graded two ways, and the gap between "ship a fix" and "burn a week on nothing" is whether you looked.

We fixed the symlink, bumped its default to the shipped weights, and left a louder note, since the script's own comments admit this drift cost an eval shift once before. It will drift again the first time someone forgets to bump it. That's the lesson, and the unsatisfying one: the bug is rarely in the model. It's in what you handed the eval, and the eval will not volunteer it.

from geopy.geocoders import Nominatim

geo = Nominatim(domain="localhost:8080", scheme="http")
geo.geocode("1600 Pennsylvania Ave NW, Washington DC", addressdetails=True)

That query comes back at the rooftop, with the house number and the road and a country code, plus an annotations block Nominatim doesn't return at all — timezone, calling code, the coordinate in six formats — the same block OpenCage bills for. We were pleased with ourselves. So we asked it about Europe.

It put Warsaw in Indiana.

Not metaphorically. Warsaw, Poland came back at 41.2, -85.8 — Warsaw, Indiana, a comfortable ocean away from the one in Poland with one-point-eight million people. Berlin went to New Hampshire. Vienna went to Virginia. Sydney went to Nova Scotia, which isn't even the right country to be wrong in. Every city we tried resolved, confidently, to its smaller American twin.

If you've read Which Berlin?, you know we've met this collision before. Bare city names repeat across borders, and something downstream has to decide which dot on the map you meant. We'd met that part before. What we hadn't done before was build a machine that answered "the American one" every time.

The line that hardcoded America​

Here's the line. When we wire the geocoder, there's a country fallback:

const defaultCountry = resolveCandidateDbPath() ? undefined : "US"

It reads as harmless. "If you don't have the big ranking database loaded, assume the United States." And once upon a time it was harmless, because the model couldn't route countries on its own, so anchoring to the US was the least-bad guess for a US-centric demo.

The trouble is what defaultCountry does downstream. It's a hard override: it wins outright, over every other signal, including the model's own read of the address. A word that sounds like a gentle fallback turns out to be the last one the resolver hears. We have a small classifier whose entire job is to look at "Warsaw, Poland" and say Poland, and it does, correctly, with confidence. The override threw that answer in the bin and substituted "US" before the resolver ever saw it. We had taught a model to read the country off the page and then covered its eyes.

The fix was to delete the line. The model already knew the answer; the job was to stop overriding it. Drop the constraint, and the country the address names flows through. Berlin lands in Germany. Madrid lands in Spain. The five European countries our resolver covers well snapped into place, and our drop-in went from resolving four of ten test cities to nine of ten, with zero change to the American ones.

We shipped that. It's the right fix and it's live.

Four cities that won't move​

Nine of ten. The tenth was Vienna, and once we looked past the headline, Vienna had company. Delete the override, ask again with nothing on the scale, and four cities still walk to the wrong country on their own: Vienna to Virginia, Sydney to Nova Scotia, Warsaw to Indiana, Toronto to Ohio.

The override is gone now, so this is the collision winning on its own. The resolver ranks candidates by population, and for most cities that's enough: there is no Tokyo big enough to beat Tokyo. But Vienna, Austria has to outrank a Vienna the resolver reaches faster, and without a thumb on the scale for country, "the biggest Vienna" isn't a question the ranker is answering. The model knows it's Austria. The resolver, on this query, never gets told in a way it can't ignore.

There's a manual escape today, and it's the one Nominatim gives you too. Pass countrycodes=au and Sydney lands in Australia; the restriction is honored. It clears the easy collisions. It does not fix the deeper ones — Warsaw plus a Poland filter returns nothing, because the record in the gazetteer is filed under Warszawa and the English exonym matches neither it nor any other Polish place. That's a different problem with a different fix, and we filed it as one.

So far this is a tidy story. A harmless-looking line, a one-line deletion, a known residual with a workaround. We wanted a number on the residual — how much of the world is the model-can't-route-the-country problem, and how much is something else? So we built a measurement: every country's most-populous cities, geocoded bare, scored by distance from the truth. One hundred and eighty-seven countries.

And the measurement lied to us, the way they do when you forget to check what they ran on.

The database we never switched on​

The first sign was a contradiction we'd written down ourselves. The breakdown classified Poland as coverage-absence — no record at all, the gazetteer doesn't have it. But two hours earlier, chasing the exonym, we'd proven the opposite by hand: Warszawa, Poland resolves to 52.2, 21.0, dead on. The record exists. We had a report saying "no Warsaw" sitting next to a transcript saying "here is Warsaw."

Both can't be true, so one of them was measuring the wrong thing. The report was. The whole sweep had run with the candidate ranking database switched off — the same resolveCandidateDbPath() ? … : "US" branch from the top of this post, except this time the missing database didn't just change a fallback, it changed which resolver answered every non-US query. We'd measured the world on an engine that was never meant to answer for it, and then written the numbers down as if they were the verdict.

Switch the database on and the picture moves under your feet. London, which the sweep had called a coverage gap, resolves two kilometers from the truth. So does Beijing. They were never missing; they were waiting for the index we hadn't loaded.

We pulled the numbers back. The report carries a caveat at the top now, the two issues we'd filed off it got correction notes, and the retrospective says plainly that the percentages were measured on the wrong config. This is the part of the night we're least proud of and most want on the record, because it's the same mistake as Which Berlin? — a metric grading the wrong thing — except a layer lower. There the number was sound and measured the wrong quantity. Here the number measured the right quantity on the wrong machine. The discipline that catches both is the same: before you trust a measurement of the engine, pin and state the data it ran on. We caught it, later than we'd have liked, off a contradiction we were lucky to have written down.

Where this leaves us​

Here's the standing, stated without flinching.

The drop-in is real and shipping: point your Nominatim client at it, keep your code, drop the database stack. The America-by-default bug is fixed — the model routes the country it reads, and countrycodes is there for when you want to insist. That much is solid and config-independent.

The namesake gap is also real, and it outlived the correction. Switch the ranking database on and Vienna, Sydney, Warsaw, and Toronto still resolve to the wrong country, because population-first ranking cannot break a same-name tie without a prior on the country. That's the one finding the bad config didn't manufacture — we re-checked it with the database loaded, specifically because we'd been burned. Closing it is model work: teach the country router to emit the next tranche of countries confidently, so the resolver gets told in a way it can't ignore.

And the precise size of the rest — how much is exonyms like Warsaw-versus-Warszawa, how much is missing coverage — we don't know yet, because the only way to measure it that means anything is on the production config, and pinning that config is the next thing on the list rather than something we'll guess at.

Nominatim carries the whole planet on community data and sets the bar we're trying to clear. We can answer its questions from a file now, and on a good chunk of the map we answer them well. We also spent a night proving that the fastest way to lie to yourself about a geocoder is to grade it on the wrong machine and forget you did. We'll keep grading the coordinate, and we'll write down what the grader was plugged into.

That's the gap we set out to close. Every geocoder chases accuracy, be-right-more-often, and so do we. The piece almost nobody hands you is the one underneath it: a number on each answer that tells you which ones to trust, so you can keep the good ones and send the rest to a human.

The forecaster's contract​

Think about a weather forecaster. A good one who says "70% chance of rain" is right on about seven of every ten days they say it. That's the whole contract: the number means what it says. A neural model makes the same promise implicitly every time it hands you a softmax probability, and for a long while ours quietly broke it, confident in some bands and timid in others, with no single number you could lean on.

We fixed that part first (it's its own story). The short version: a small post-hoc correction maps the model's raw confidence onto an honest one, so when a parsed span reads conf="0.94" it's right about 94% of the time, measured on addresses the fit never saw. When the model says it's sure, now you can believe it.

A calibrated number per span still isn't a number you can act on per answer, though. So we carried it the last step, to the resolved coordinate.

Dialing precision​

Here's the move that makes it pay rent. mailwoman returns a coordinate and a confidence you can put a threshold on. Set a bar (call it τ) and accept only the answers the model is at least that sure about; route everything below it to review.

On a deliberately messy held-out set (lowercased, abbreviated, postcodes stripped, the real-world soup), graded by whether the resolved point lands within 25 km of the truth, the dial does what you'd want:

Raise the bar and the precision of what you keep climbs from 84% to 97%. The price is coverage: you answer fewer rows, and you pay it knowingly, because the number is honest about where it stands. A search index returns one best guess and one confidence, which is to say no dial at all. You take what it gives and hope.

The part that makes this trustworthy rather than a nice-looking curve is that it holds on data we didn't draw the curve on. Split the messy set in half, fit nothing new, and the high-confidence answers still beat the low-confidence ones out-of-sample by fourteen points. The model is most unsure exactly where it ought to be: the locales whose coverage we're still building. It has some sense of what it doesn't know.

What you do with it​

Go back to the hundred thousand records. With a threshold you can finally split them in two: the answers the model stands behind, which you auto-accept and move past, and the ones it's hedging on, which a person looks at. The threshold means what you set it to. Set it at 0.95 and your auto-accepted set is right about that often, a rate you can put in a report and defend rather than a hope you re-check by hand.

That's the difference between a vibe and a number. A pin with a vibe attached is a thing you re-litigate on every row. A pin with a calibrated confidence is a thing you build a pipeline around.

It's a lever, not a wand. Crank the threshold and you hand more rows to humans; that's the cost, stated plainly, and it's yours to set against your own tolerance for a wrong match. What we've done is make the trade legible and put an honest number on both sides of it.

It's live now in mailwoman@4.14.0. Pass a calibrator to the parser, read the conf= off the resolved tree, pick your bar. Then stop guessing which answers to trust, and start knowing.

That sat badly. Not because losing is shameful — Nominatim is the bar, it carries the whole planet on community-contributed data, and clearing it anywhere is the goal. It sat badly because we didn't understand it. We knew our parser wasn't ten points worse in Europe than in America. So what was the gap actually made of?

This is the answer, the two fixes, and — because we'd be kidding you otherwise — the parts the fixes didn't reach. For the European leg we added a third system to grade against: Pelias, by way of geocode.earth, the hosted Elasticsearch stack a lot of people reach for. It turns out to be the real bar, and we'll be honest about where it still beats us.

How we kept score​

We graded the way we always argue you should: not on whether the model's labels matched our labels, but on where the address lands on Earth. OpenAddresses ships a truth coordinate with every row, so for each address we parse it, resolve it to a point, and measure the great-circle distance to where it really is. Inside 25 kilometers, we call it right-placed; a no-result counts as a miss, same as a wrong answer. One honest number per system, on the same real addresses, each given the same country hint so nobody's racing with a handicap.

That bar is stricter than the one we'd been quietly grading ourselves on. Our internal "resolve rate" had been counting any resolution as a win — including ones that landed a region away. Re-graded at 25 km, it had been flattering us by fifteen to twenty points in Europe. So the first casualty of this benchmark was our own scoreboard. Fine. Grade the coordinate.

The US, briefly — because it's why Europe stung​

In the US, mailwoman right-places 99% of these addresses at 25 km; Nominatim, 84%. The difference is almost entirely no-result: ours is near zero, Nominatim's is about one in six. That's not a knock on them — it's where a purpose-built stack shows, with TIGER and a fifty-state rooftop layer covering ground that community OSM data hasn't filled in yet.

The part that matters for the rest of this post: we do that in a 30 MB model that runs in your browser, no Elasticsearch, no server round-trip. Same engine, same deployability, pointed at Europe. Which is exactly why a double-digit loss there was confusing rather than merely disappointing.

Europe: we weren't wrong, we were silent​

Graded by the same ruler across six European locales, mailwoman started at about 66%. Nominatim sat at 78%, and Pelias — leaning on Elasticsearch and a pile of mixed sources — at 89%. We were behind both, and the reflex reading is "the parser is worse in Europe." It isn't. When mailwoman does resolve a European address, the point is tight — a kilometer or two from truth. The gap was almost all the other failure mode: on a messy European address, better than one in four came back empty. The model didn't place them wrong. It placed them nowhere.

That's a more fixable problem than being wrong. So we went looking for what made it go quiet — and found two different things, one clever and one just embarrassing.

Why it went quiet, part one: a word it mispronounced​

The model fragments. Its subword tokenizer takes a town like Grudziądz and, on the accented ą, splits it into Grudzi + dz. Neither half is a place. Neither resolves. The whole address fails — and the town's name was sitting right there in the input, intact, the entire time. The tokenizer mispronounced a word it had been handed correctly.

Once you see it, you see it everywhere in the unresolved pile: Polish, Czech, Portuguese towns the model chopped on a diacritic and then couldn't find.

Why it went quiet, part two: postcodes we never had​

The second cause had no cleverness to it at all. For Poland and Czechia, we simply didn't have the postcodes. WhosOnFirst, our gazetteer's backbone, ships rich postcode coverage for some countries and none for others, and PL and CZ were in the none column — zero rows. A Polish address whose town the model did read correctly could still die, because the only anchor we could have resolved it by wasn't in the database to begin with. Silence by omission.

The fixes: read the word, and fill the gap​

For the first, when a parse resolves nothing, we now go back to the raw text — not the model's subwords, the actual characters, diacritics intact — and try the obvious thing. Enumerate the whitespace tokens, match each against the same-country gazetteer, and take the longest exact match (the real town is usually the longer, specific name, not its ambiguous prefix). We made ourselves prove it recovered real coordinates before we trusted it: the recovered towns land about 1.8 km from truth, not in some same-named place a country away.

For the second, we pulled the missing postcodes from GeoNames — about 73,000 for Poland, 15,000 for Czechia, each with its own centroid, openly licensed — and folded them straight into the gazetteer. No model change, no retrain. Just the rows that should have been there all along.

Here's the part we didn't expect, and it's worth sitting with. The two fixes mostly do the same job by different routes. Where we now have the postcode, the address resolves directly and the word-recovery has nothing left to do; where we don't, the recovery carries it. On the same ruler, the postcode fill alone lifted Europe from 66% to 84%. The word-recovery alone, also 66% to 84%. Together: 88%, with our empty-answer rate dropping from better-than-one-in-four to under one in twelve.

The third fix: stop being confidently wrong​

Eighty-eight percent still left a stubborn pile of misses, and they weren't silence — they were the opposite. "06260 Saint-Pierre, La Placette 7" came back 617 kilometers from truth. Saint-Pierre is one of the most common town names in France, and we'd picked the wrong one. But the postcode, 06260, had resolved correctly the entire time — it pins the Alpes-Maritimes, right where the address is. We had the answer sitting in the same parse and threw it away, because the coordinate-picker trusted the town name and never checked it against the postcode next to it.

So we taught it to check. When a town lands far from its own postcode, re-pick the same-named town nearest the postcode instead; if none is close, fall back to the postcode's own point. It's the consistency idea from the recovery gate, moved out of the rescue path and into the main one — no model change. It took Europe from 88% to 94%, and once we'd extended the GeoNames postcode fill to Portugal, Austria, and Australia, it pulled Australia from 35% to 65% — a resolved postcode finally gave the disambiguation something to stand on.

The honest part​

Here's where we resist the victory lap, because we've embarrassed ourselves before by taking one early. Re-graded on that same European ruler, all three systems side by side: mailwoman 94%, Nominatim 78%, Pelias 89%. We're clear of both now — five points over a hosted Elasticsearch stack, sixteen over Nominatim — from a 30 MB model running in a browser tab. Italy, Portugal, France, Czechia: ahead outright.

Across all seven locales, Australia included, it's 90% for us, 81% Nominatim, 88% Pelias. Still ahead, but the margin is thin and the whole of it is Australia, where we score 65% to Nominatim's 97. And here we owe you the thing we only just worked out: Australia is a reading problem, down in the parser. Australian addresses lead with the postcode — "3053 Carlton, Barry Street 50" — and our model, raised on American and European formats where the number comes first, reads that leading 3053 as a house number and mangles everything after it. Hand it the same address reordered the American way and it parses perfectly; 65% becomes 87%. So the fix lives in the model: we teach it the Australian format, with the national address file. We didn't close Australia. But we finally know what's holding it shut, and it isn't the map.

There's a precision cost worth stating plainly, too. At 25 km we're competitive; at one kilometer we're not. We resolve to admin and postcode centroids, so we right-place the area — within a couple of kilometers — where Nominatim and Pelias drop a rooftop pin. That's the trade behind the headline: right neighborhood, not right doorstep. For a lot of jobs the neighborhood is enough; when it isn't, the incumbents win that round and we'll say so.

And the word-recovery stays off by default, because about one in six of the newly recovered addresses lands more than 25 km off — the recovery matched a wrong same-named place. Where we have postcode coverage, a consistency check throws those out before they ship; the GeoNames fill means that check now fires for Poland and Czechia, where a month ago it couldn't. Australia is the place it still can't. So a batch pipeline leaves recovery off, because a wrong coordinate poisons a join silently; the demo turns it on, because in an interactive tool "something you can see and correct" beats "nothing." Every recovered place is tagged with whether the postcode check actually fired, so a caller decides for itself rather than inheriting our guess.

The thing a scoreboard can't show​

One more, because it's the reason we don't think a single resolve-rate tells the whole story. A search index returns a result. It doesn't tell you how much to trust it. Mailwoman's confidence is calibrated — a span it marks 90% is correct about 90% of the time, measured on held-out data. That's a number you can build a rule on: auto-accept above a threshold, route the rest to a human, and the threshold means what you set it to. It's the one thing the recovered-but-unverified coordinate above is careful not to fake — we flag that separately rather than dress it up as a calibrated probability, because the whole value of the number is that it doesn't lie.

So here's the standing, stated without flinching. Pelias is ahead of us across the full panel, and it places rooftops where we place neighborhoods. On Europe we beat Nominatim handily and matched Pelias — in 30 megabytes, in your browser, with no Elasticsearch and a confidence number that means what it says. We're still the challenger on a chunk of the map, and Australia is a problem we haven't solved. We mostly closed a double-digit gap by learning to read a word we'd been mispronouncing and adding the postcodes we should have shipped in the first place. There's more to learn. We'll keep grading the coordinate.

That metric is measuring the wrong thing. The moment we measured the right one, the panic fell apart.

We've told this story once before, about a resolver that scored 93.7% on a name-match metric while landing 326 kilometers from the truth. We thought we'd learned the lesson — grade the coordinate, not the word — and filed it under "things we now know." This is that same lesson knocking again, in a new accent, and catching us flat-footed in a new way. So it's worth telling, because the second time you fall for a trick is the one that should embarrass you into never falling for it again.

A test that grades the words, not the place​

Here is the number that started it. We took 120 real Spanish addresses — pulled from OpenAddresses, not invented — and ran them through the model, then scored its component labels against the truth: did it tag the street as street, the city as locality, and so on. Macro-F1 came back at 28.5%. Street alone: 24.9%. By that yardstick, the model can barely read Spanish.

You can talk yourself into a panic from a number like that. We almost did.

The trouble is what the number is actually counting. A Spanish street is Calle Mayor, Avenida de la Constitución — a type word, then a name. Our gold tags the whole span as the street, and the strict match wants the model's span to line up with it exactly. It mostly doesn't, and we wanted to know why: is the model cleanly slicing the type word off — calling Calle a prefix and Mayor the street, which would be a harmless tagging-convention difference — or is it genuinely losing the street? So we re-graded against a bare-name gold, Mayor with the type word stripped from both sides. If the model were splitting Calle off, that should reward it. It did the opposite: street-F1 fell from 25% to 2%. The model's span matches the whole Calle Mayor far more often than the bare name — so it's keeping the type word in, not slicing it off. Its boundaries simply don't sit where our gold puts them, and the strict label match scores every such mismatch a flat zero, whether the address was understood or mangled.

A test like that can only tell you whether the model used your exact words. It has nothing to say about whether it found the building.

So we graded the coordinate​

The honest question isn't "did it use my labels." It's "did the address land in the right place on Earth." OpenAddresses ships a truth coordinate with every row, so we can ask it directly: parse the address, resolve it to a point, measure the great-circle distance to where it really is. We built held-out sets of 150 real addresses each across the locales where we have that coordinate truth, and graded the production model on the metric we actually ship.

One subtlety carries the whole result, so it gets its own column. We separate how often the model produces a resolvable answer from how good that answer is when it does. Lump them together and a locale that fails half the time but nails the other half reads as mediocre everywhere, which is exactly wrong. Pull them apart and the model tells you something useful.

Spain isn't in that table — its open-address data is cadastral, with no clean coordinate to grade against, so for Spanish we can only score labels. We can't put a Spanish address on the map in this test and measure it directly, and we won't pretend otherwise. What we can do is take the exact street-boundary quirk that tanked the Spanish label score and watch what it costs next door, in the Romance-language locales we can measure. The answer is: nothing. Where the model resolves an Italian or Portuguese address at all, it lands within a few kilometers — city grain, sometimes the rooftop; Luxembourg comes back at 300 meters. The model gets the locality and the postcode right often enough to find the right town while it draws the street boundary somewhere the label test hates. The 28% was never measuring geocoding. It was measuring vocabulary — and the redemption holds everywhere we have a coordinate to check it against, which is the strongest claim the data lets us make for Spanish without Spanish coordinates.

The gap is how often it answers, not how well​

So that's the good half, and we'll take it. The honest half is the left column, and it's where the real work is.

Look down the resolves rate and a pattern falls out that has nothing to do with accuracy and everything to do with us. France and Italy — the two locales we've actually poured training data into — resolve about 80% of the time. Everywhere we haven't, the rate drops by a third or more: Portugal, Poland, Austria around half; Czechia at 43; Australia, which we've barely touched and which collides its own town names across states, at 28, and misplacing what little it does resolve. The model isn't worse at understanding those addresses. It's worse at producing an answer for them at all, and the cliff lines up exactly with where our corpus stops.

That's the uncomfortable, useful shape of it. The wall isn't precision — when the model commits to an answer abroad, the answer is good. One asterisk, and it's an honest one: those are the answers the model chose to give, and the addresses it drops skew toward the harder ones, so read the resolved distances as a ceiling rather than an average. The wall is recall, and recall is a map of our own training data drawn in someone else's country.

Not the gazetteer's fault​

There's a comfortable version of this story where the model parses everything correctly and the gazetteer is missing half of Poland, and the fix is a data import nobody has to feel bad about. We checked, because that's the version we'd have preferred. It isn't true.

When a Portuguese address fails to resolve, it's not because the town is absent from our index — we folded the whole world's place names into the index months ago, and a spot check confirmed every Polish and Portuguese locality GeoNames knows is already there. It's that the model never cleanly emits the locality in the first place. The label test agrees, for once: Portuguese locality-recall sits at 39%, right alongside its 52% resolve rate. If the gazetteer were the bottleneck, the model would extract the town and the index would shrug; instead the model and the resolver fail together, in lockstep. The places are on the shelf. The model just doesn't reliably ask for them. We ran that check directly for Portuguese; across the rest of the mid-tier the same shape repeats — low resolve, low locality-recall, the towns demonstrably in the index — which is exactly what you'd expect if the cause is the same one.

Which tells us what not to do. There's no clever rule, no Calle-handling special case, that fixes this — bolt on a Spanish street splitter and you've taught the parser one convention by hand and learned nothing about the next forty. The model didn't fail because it lacked a rule. It failed because it hasn't seen enough of the place. The lever is the training distribution, and we now have a coordinate-graded yardstick that will tell us, locale by locale, whether feeding it more actually moves the recall it's supposed to.

What we're keeping​

The lesson is the same one as last time, which is the point. We had it, written down, and the label metric still walked us to the edge of the wrong conclusion before the coordinate pulled us back. Grade the thing you ship. Distance to the truth can't be talked out of, can't be gamed by matching a word, and doesn't care which language the street type is in. Every other metric is a proxy, and proxies flatter you in the languages you trained on and slander you in the ones you didn't.

And then there's the quieter half, the one the two columns made impossible to ignore: a model can be right and unavailable at the same time, and if you only measure one number you'll never know which problem you have. Pull recall apart from precision before you decide what's broken. Ours turned out to be accurate everywhere it answers — and the whole job now is getting it to answer more often, in more places, by showing it more of the world.

I want to be clear about how that happens, because it didn't happen through some dramatic mistake. It happened the quiet way — the way most data gaps happen — and the most useful part of the story is the trick that finally made it visible.

A missing place never files a complaint​

What makes coverage gaps so good at hiding: a missing place doesn't error. When your parser mislabels a token you get a wrong field you can see in a diff. When your resolver picks the wrong Springfield you get a pin in the wrong state and an angry test. But when a place isn't in the gazetteer at all, the system does exactly what it does for a typo or a fictional street — it shrugs and returns nothing, politely, every single time. Afghanistan returning nothing looks identical to "Hgsdfgh Street" returning nothing.

So you can run your eval suite, watch it pass, ship the thing, and announce world coverage, and every number you're looking at is true. Your US accuracy is real. Your German coord error is real. They're just measured against the places that are in the file, and the file quietly decides what gets measured. We were grading ourselves on the 97 countries we had, and the missing half of the planet never showed up to the exam to lower the average.

This is the part I'd flag for anyone building on top of open geodata: your coverage is whatever your sources' coverage is, and you inherit it silently. We build the gazetteer from Who's On First plus Overture's administrative divisions — both excellent, both things we'd choose again. But "excellent" and "complete" aren't the same word, and the admin DB those two produce carries localities for 97 countries. We never decided to drop Libya. We just never had it, and nothing in the pipeline was shaped to notice the absence.

The trick: audit against a yardstick you didn't build​

You cannot find an invisible gap by staring at what you have — what you have looks fine, by definition. You find it by holding it up against something independent and asking where the two disagree.

So we did the cheapest possible version of that. GeoNames publishes cities15000 — every populated place on Earth above 15,000 people, a flat file we already had on disk. For each country, we counted the localities in our gazetteer and counted the cities GeoNames knows about, and printed the ratio. The audit took about a minute to write. Most countries came back sensible. And then a long, ugly column of zeros:

country  ours  GeoNames cities>15k
AF       0     54
HK       0     141
KP       0     97
CD       0     114
AL       0     25
...

Hong Kong: a hundred and forty-one cities in the reference, zero in ours. That's not a tuning problem, that's a hole you could drive a continent through. We checked whether it was the candidate build filtering them out — maybe the data was there and we were dropping it — and no, the source admin DB itself stops at 97 countries. The gap was real and it was upstream.

I'll editorialize for a second: this audit-against-an-external-yardstick move is the single most productive hour of the whole effort, and it's almost embarrassingly simple. We didn't need a model or a clever metric. We needed to stop trusting our own file and compare it to someone else's count. If you take one thing from this post, take that. Go diff your coverage against a reference you didn't build, per whatever dimension matters to you, today. The gap you can't see is the one that's been quietly setting your ceiling.

Fixing it without retraining anything​

The good news about a pure data gap is that the fix is pure data. No GPU, no model, no retrain — GeoNames already has the places, with coordinates and per-language names, for every country we were missing. We taught our gap-fill builder one new trick (register a country code it had never seen) and pointed it at the full allCountries dump, keeping only current populated places and dropping the historical and abandoned ones.

Out the other side: 4.6 million town-level localities across 248 countries, up from 97. And because "it resolves" and "it resolves correctly" are different claims, we checked the second one too, and pulled a handful of major cities in formerly-empty countries to confirm the coordinates land where they should. Kabul resolves to Kabul, within a few hundred metres. Hong Kong, Tirana, Tripoli, Sarajevo, Yerevan — all home. Nine out of nine within thirty kilometres, most of them dead on. A spot-check of the long tail of small towns put them inside the right country's bounding box two hundred times out of two hundred.

The cost, named​

I told you the house rule is to name the tradeoff, so here it is. Full town-level depth turns the gazetteer from 811 MB into 1.26 GB. That's free on a server and not free in the browser, where the demo streams the database a few kilobytes at a time over byte-range requests — and a 50%-bigger file is a 50%-bigger thing to range over. So the fix is two builds, not one: the major cities go to the browser (light, covers the overwhelming majority of what anyone types), and the full town depth goes to the CLI and server, where the bytes are cheap. One builder, two depths, your call which artifact gets which.

And the boundary I'd be lying to skip: a place resolving is not the same as a full address parsing. Putting Kandahar in the gazetteer means that when something hands the resolver the word "Kandahar," it now lands in Afghanistan instead of nowhere. Whether our parser reliably pulls "Kandahar" out of a raw Afghan address string is a different question with a different answer — and a different post. Coverage was the necessary half. We did the necessary half.

So: go find your invisible gap. Hold your data up to a yardstick you didn't make, count the disagreements, and look hard at the zeros. Ours had been hiding in plain sight, passing every test, for weeks. Now get to work — yours is in there too.

So the questions for the day: why does looking up one name cost 243 round trips? What goes wrong when you search a database you can only read a slice at a time? And how do you get a global gazetteer — every country, region, county, and city we resolve against — down to about a dozen reads without putting a server back in the loop?

Reading a database through a straw​

Here's the setup, because the constraint is the whole story. The admin database is 2.6 GB. Nobody is downloading 2.6 GB to type "Chicago" into a search box. What makes this work at all is sql.js-httpvfs: a SQLite build whose file-reading layer, instead of reading from disk, issues HTTP Range requests against the file on the CDN. SQLite asks for "bytes 4,210,688 through 4,276,224," the VFS turns that into a range GET, the CDN hands back that one 64 KB chunk, and the query engine carries on as if the file were local. You're reading a multi-gigabyte database through a straw, and most of the time you never touch 99.99% of it.

For a B-tree lookup on a primary key, this is close to magic. The engine walks root → interior → leaf, and each hop is one range request. Three or four reads and you're holding your row. The straw is wide enough.

Why search scatters​

The trouble started because we weren't doing a primary-key lookup. We were doing a name search, and to make name search fast inside SQLite you reach for FTS5 — full-text search, an inverted index. For each token it stores a posting list: every row that contains it. Search becomes "go read the posting list for this token," which is exactly the kind of thing a real database does well when the disk is right there.

The disk is not right there. It's at the end of a straw. And an FTS posting list for a common name is not one tidy run of bytes — it's a chain of pages scattered through the file, because the index grew as the data loaded and the postings landed wherever there was room. Reading one posting list means following that chain, and every hop in the chain is its own range request. A common place name dragged in a long chain. Then the candidate rows themselves lived somewhere else again. The reads piled up: 243 of them, about 16.5 MB, to answer one query.

The maddening part came when we tried to tune it. The VFS lets you widen the straw — ask for bigger chunks per request, fetch 64 KB or 256 KB at a swing. We widened it, expecting fewer round trips. We got more bytes and barely fewer trips, because the postings were scattered, so a bigger chunk mostly bought us neighbours we didn't want. We'd been getting away with it on a "slim" database — a stripped 53 MB build with three countries in it — and the slim build was cheap for the dullest possible reason: it was small, so even a scattered search couldn't scatter very far. That's not an optimization. That's a database too little to misbehave.

Stop searching. Start addressing.​

We talked it over with DeepSeek, and the reframe that came back was the one worth keeping: a database you read through a range request isn't a database you should search. It's a database you should address. Don't make the browser hunt for the answer across an index — precompute the answer and put it where one short read can reach it.

Concretely: a single table, one row per name a place can be found by. Each row carries everything the resolver needs already on it — the canonical name, the centroid, the bounding box, the country, and the population rank baked in as a sortable column — so once you've found the row, you're done, with no second trip to go fetch the details. The table is WITHOUT ROWID, which in SQLite means the rows are the B-tree, sorted by the key, and the key is the normalized name. We load it pre-sorted and VACUUM it so a name and its neighbours sit on adjacent pages. Now a lookup is the thing the straw was always good at: walk the tree to the name, read the leaf, and the matches for "springfield" are sitting there in a row, ranked, ready.

One discipline holds the whole thing up: the key has to be normalized the same way when we build the table and when we query it, or the walk lands one page off and finds nothing. So both sides call the same function — normalizeLocalityForKey — by construction, not by convention. It folds case, strips diacritics so Saint-Étienne keys as saint-etienne, and leaves scripts that aren't Latin alone so 北京 stays 北京. Build-side and query-side can't drift, because there's only one of it.

The cost we almost shipped​

Now the catch. The candidate table carries US postcodes and not yet the rest of the world's, the same coverage the old slim build had. So when we first wired it up and typed a Berlin address — 10115 — the demo dropped a pin in Manhattan. 10115 is a real Berlin postcode and a real New York ZIP, the table only knew the New York one, and the cascade had reached for the postcode first and taken the only match it found.

The fix wasn't more postcodes. It was order. Resolve the locality first — "Berlin," which ranks to Germany by population — then let that resolved place gate the postcode lookup to its own country. No German 10115 in the table? Fine: fall through to Berlin's own centroid, which is exactly where the address wanted to be. An ambiguous postcode can't drag a German address across the Atlantic anymore, because it never gets to vote before the city does.

What it bought​

A name now resolves in about 12 range requests and under a megabyte, down from 243 and 16.5 MB. The whole table is 490 MB with global coverage — every place we know, not three countries — in a single file the browser reads a dozen pages of. US locality resolution holds at 96.8%; the European cities we'd just added through Overture come back at parity. And the slim build — the per-model-version, three-country database we'd been maintaining — is gone, along with the chore of deciding which countries were worth shipping.

The lesson sticks because the constraint was never going away. When the only way to touch your data is one short read at a time, searching is the expensive verb. So we stopped searching. We built the answer ahead of time and gave it an address, and now the browser just goes and reads it.

So, the questions for the morning. How does a database that passes every check still land the answer 600 kilometres wrong? What does it actually mean to "validate" a thing? And why is Toronto in Ohio?

The build that felt right​

The task was simple enough: add Canada to the map. The obvious lever is postcodes — they're precise, they're public, and Overture ships them. I'd done exactly this for a dozen European countries the night before, same tool, same shape: read the country's address points, group by postal code, take a trimmed centroid, write one row per code. Canada came back with 843,000 of them, alphanumeric A1A 1A1 codes that survive the build intact, each pinned to its own block. I spot-checked Toronto's financial district and it sat exactly where it should. I diffed the new gazetteer against the old — US postcodes untouched, every European set intact, 843,000 new Canadian rows. Every number I could check, I checked, and every one was right.

That's the part worth sitting with, because it's the trap. I had checked everything I'd built.

The check that grades the pin​

There's one test in the stack that doesn't care what I built. It loads the actual demo in a real browser, types an address into the box, and reads the coordinate off the map — the pin the user would see. The whole point of it is to grade the assembled answer, not any single piece of the machinery. Before promoting the new gazetteer I added a Canadian case to it: 100 Queen Street West, Toronto, ON M5H 2N2, and assert the pin lands somewhere in Toronto.

Four cases passed — Chicago, a Berlin postcode, the White House, all green. The fifth came back 40.458, −80.6. I had to look it up. That's Toronto, Ohio: population five thousand, on the bank of the Ohio River, about as far from the address I typed as Madrid is from London.

Why the perfect postcode never got a vote​

The gazetteer has two layers. There's the postcode layer I'd just rebuilt, and underneath it the admin layer — the one that knows what a "Toronto" is, which country it sits in, how many people live there. So I went and counted the admin layer: 1.7 million places, twenty-six countries. The number of Canadian ones is zero. Not Toronto, not Montréal, not Vancouver — Canada simply isn't in it. We'd built the gazetteer out of the European set, the US, and the CJK countries, and nobody had ever folded Canada in.

Now watch what that does. The parser reads "Toronto" off the address and hands it to the resolver. The only Toronto the resolver has ever heard of is the one in Ohio, so that's the city it picks. And then the cleverest piece of the cascade turns around and finishes the job. We added that piece a couple of weeks ago to keep Berlin's 10115 — which is also a perfectly good New York ZIP — from dragging German addresses to Manhattan: resolve the city first, then let that city gate the postcode to its own country. It's a good rule. Here it's a guillotine. The city resolved to the United States, so the correct Canadian postcode is now "from the wrong country," and out it goes. All 843,000 of my perfect coordinates never get a vote, because the city votes first and the city is in Ohio.

I validated the layer I made, not the one that ships​

Here's the lesson, and it's an old one in a new coat. I validated the postcodes. The postcodes were never the problem. I checked the thing I'd built and not the thing the user receives — the coordinate, the pin, the assembled output of every layer working together. Each green check was accurate and beside the point, because not one of them graded the only number that matters.

This keeps happening to us, in different forms. A model's per-tag F1 climbs while the coordinate it produces goes nowhere. A reconcile pass improves a label and breaks the geocode that depended on it. The discipline that survives every version of it is the same: grade the assembled coordinate against the truth, not the component you happened to touch. The e2e gate isn't ceremony. It's the one thing in the stack that reads the pin the user sees, and it's the only thing that caught this.

What it costs, and what it doesn't​

The accounting: the night wasn't wasted. The 843,000 centroids are correct and sitting ready, and the tooling that made them is the same tooling that'll make the next country. But the premise was half a country short. A country, it turns out, is its admin skeleton first — the cities and regions that anchor a name to a place — and a postcode only means something once that skeleton is there to hang it on. Fold Canada's divisions into the gazetteer the way we folded Europe's, and the postcodes I built light up for nothing. Ship them on their own and all you've done is hand the resolver a stack of right answers to a question it can't ask.

So we didn't promote it. The map still doesn't have Canada — but it knows it doesn't now, and so do we. Build the test that grades the pin your user actually sees, and run it before you trust the green checkmarks upstream of it. They'll tell you the data is right. The pin tells you whether you are.

That's the pitch, and it's worth being clear about why it's unusual, because the three things the rest of the market hands you each ask you to give something up.

The three options, and what each one costs you​

Start with a cloud API: Google, Mapbox, HERE, AWS, Azure. You send the address over the wire and get coordinates back, which is fast and globally accurate and, depending on who you signed with, comes with strings. Google's terms let you cache a coordinate for thirty days, then you must delete it, so if you geocode the same ten thousand delivery addresses every morning, you pay for them every morning. AWS won't let you cache autocomplete results at all, and bills storage at a higher tier when you do cache. The address always leaves your infrastructure, which is a non-starter the moment the address is the sensitive part: a shelter, a clinic, a protected residence.

Or self-host one: Nominatim, Pelias, Photon, all real software, free, and yours. The license was never the catch; the machine room is. Nominatim wants a Postgres/PostGIS install north of a terabyte of disk and a planet import measured in days. Pelias is Elasticsearch plus five supporting services and a 450 GB index. Photon is OpenSearch and a 64 GB heap. You own your data, absolutely. You just have to operate a search cluster to use it, and none of them run anywhere near a browser.

Or run a parser by itself. libpostal is the open address parser everyone embeds, and it's good. It's also a 1.8 GB trained model (2.2 GB for the Senzing variant), written in C with no supported path to the browser, and it stops at parsing. It tells you which token is the street; it doesn't tell you where the street is.

So the menu is: rent it and leak the query, host it and run a cluster, or parse it and stop halfway. Mailwoman is the entry that wasn't on the menu.

What if it just ran on the device​

The whole pipeline, parse and then resolve to a coordinate, runs client-side. The parser is a 29.8 MB int8 ONNX model executed by onnxruntime-web. The gazetteer is a single global SQLite file, around 530 MB covering every country we resolve against, and the page never downloads it. It reads it over HTTP range requests, pulling roughly a dozen small slices to answer a query instead of fetching the file. No Elasticsearch, no PostGIS, no native module to compile: the resolver is plain node:sqlite, and the same code runs in Node and in a tab.

The "dozen slices instead of the whole file" part has its own story. It used to take 243 round trips, and getting it down to twelve meant rethinking how the database is indexed rather than how it's fetched. That's a whole separate post. The point here is the consequence: a global geocoder that fits through a straw, with no server at the other end of it.

A neural parser the size of a photo​

Worth dwelling on the 29.8 MB, because the comparison is stark: roughly sixty times smaller than libpostal's default model, seventy-odd times smaller than the Senzing one. And the smaller model is the better parser on the input that actually shows up in a search box, the typos, the half-finished lines, the prefix you've typed so far. A transformer trained on degraded queries reads 123 Penslyvania Av NW the way you meant it; a CRF over hand-built features tends not to. Small enough to ship to a browser, and better where browsers get used. You rarely get both.

Your queries never leave, and the results are yours forever​

This is the part that sells itself to anyone holding addresses they can't send to a third party. There's no storage clause because there's no server: nothing to retain, nothing to delete on a thirty-day clock, nothing to leak. You geocode once and keep the coordinate as long as you like. For healthcare, for anything touching protected addresses, "the data never leaves the device" stops being a feature and becomes the entire procurement requirement.

I'll be fair about the boundary here: geocode.earth's hosted Pelias and Smarty both already give you generous own-your-data terms, so this argument bites hardest against the rent-and-delete crowd, Google, AWS, and HERE. Against them it's decisive.

The bill is zero​

Google's geocoder runs five dollars per thousand requests after the free cap. That's five thousand dollars a month at a million addresses, six figures a year once you're past ten million. On-device, the marginal cost of a query is nothing — it's running on hardware you already paid for, which is the user's. Again, in fairness: a self-hosted geocoder is also zero-marginal-cost, so this one is really aimed at the cloud APIs, where on-device kills both the per-call meter and the server you'd otherwise stand up.

Where it stops​

I'd rather you hear the limits from me than discover them in production.

Rooftop precision, the actual building rather than the neighborhood, currently fires only where we've shipped per-state address-point data: California, New York, Michigan, and DC. Everywhere else, the resolver lands you at an administrative or postcode centroid, which is the right town and the right ZIP but not the right doorstep. The big cloud vendors beat us on global rooftop coverage, decisively, and I'm not going to pretend otherwise. If you need a rooftop in Lagos today, call Google.

Mapbox, to its credit, does ship on-device geocoding, but it's their native mobile SDK, and it's reverse geocoding (coordinate to address) rather than forward. So I'll keep the claim precise: the only geocoder that runs the full forward pipeline, in a browser, with no server. Narrow as that sounds, it's a space nobody else occupies. And there's no batch endpoint and no SLA yet; this is infrastructure you embed, not a hosted service you page.

None of which changes the thing in the network tab. You typed an address, you got a building, and nobody else saw it.

npm install mailwoman @mailwoman/neural-web

Or just open the demo and watch the requests that aren't there.

Green is the National Address Database; orange is OpenAddresses, which for New York means the city's own NYC Open Data. You can explore the interactive version and click any dot to see its publisher.

The split is geography you can see​

The federal release blankets the state. New York City sits in its own color because the city publishes its own address file, and OpenAddresses carries it. Nothing was guessed about which is which — the source rode along on every point, from ingest all the way to this map. That's the whole trick, and it's a boring one: a source column that nobody dropped.

Credit where the data's due​

None of this is our data. The National Address Database is a federal release; OpenAddresses is a decade of county and municipal publishers doing the unglamorous work of putting their address files online; the street network underneath is Census TIGER. They're free, they're open, and they're good. The lookup was never the hard part of geocoding. Assembling these sources, keeping them fresh, and not sanding off the provenance somewhere along the way — that's the work, and the provenance is the first thing most pipelines lose. We made a rule of keeping it, so that when a coordinate is wrong you can see exactly which dataset and release to take it up with.

The layers under the dots​

This map shows the rooftop-point sources. Where there's no rooftop point, the geocoder falls back to TIGER street interpolation, a position estimated along the block, and below that to an administrative centroid from Who's On First. Each tier is a different open dataset, and every coordinate records which rung it landed on, so a rooftop in Manhattan and a block-interpolated guess in a thinly-covered county never get mistaken for each other.

See it, build it​

Explore the interactive map — serve it over localhost, since the basemap tiles are CORS-scoped to the house domains. The provenance is a plain column on the address-point table:

SELECT lat, lon, source FROM address_point WHERE locality_norm = 'brooklyn' LIMIT 1;
-- 40.6782 | -73.9442 | overture:OpenAddresses/NY/NYC Open Data

Point the same map at any state:

node \
  scripts/record-matcher/viz/source-provenance-map.ts --state ny

So you do what everyone does: you start a spreadsheet, you sort by name, and you give up around row 400.

Here's what resolving them anyway looks like, scoped to Texas:

Every dot is an entity that resolved across at least two of those datasets. Their identifiers didn't match, because there are none to match; they resolved because the records geocoded to the same place and their names agreed once you normalized them. The resolved location is the key the three publishers never agreed on.

How you join files that share no key​

The trick is to stop matching strings and start matching places. Each record's address goes through the neural parser, resolves to a coordinate and a place in the hierarchy, and gets blocked by where it is. Only then do the names come in, canonicalized — legal-form noise stripped, abbreviations expanded — and scored with label-free Fellegi-Sunter weights (the geocode-first record-matching story walks through this). "Baylor Coll. of Medicine" and "Baylor College of Medicine" stop being different strings; "123 Main St" and "123 Main Street, Ste 400" stop being different places. The geocode does the blocking, the normalized name does the deciding, and neither file had to carry the other's ID.

The numbers, and exactly what they are​

219 entities resolved across two or more of these sources. Twenty-eight of them span different agencies — CMS's provider registry, the FCC's funding program, Texas's licensing list — and those are the hard ones, because inside a single agency you can sometimes lean on an internal ID, while across agencies you have nothing but the place and the name. (The other 191 links are FCC-internal: Rural Health Care filings matched back to their own commitment records.) You can see the cross-agency slice on its own in the interactive map; the full set is here.

Then the coverage reconciliation. Of the 2,771 entities carrying an eligibility record — an NPPES organization or a TX HHSC facility — 22 also resolve to a funding record. That's 0.8%, and it's a floor, not a coverage rate. Imperfect resolution and a capped sample only ever miss links; they never invent them. The set is the deliverable; the percentage is just how big it happens to be.

Read this part before you read anything into it​

This is a set-membership reconciliation, not a determination. The 2,749 eligibility entities that didn't resolve to a funding record are a candidate set for review, and nothing more. A missing funding link can mean the entity didn't apply, applied under a name the resolver didn't catch, isn't eligible, or simply that its funding record fell outside this capped 2,000-row sample — a sampling artifact, not a finding. At full scale the set tightens, and it's still only a set of candidates.

What we produce is the reconciled join and the candidate set. What a gap means, and whether it means anything at all, is the data consumer's call, not the tool's. Nothing here is an allegation, and the map makes no claim about any provider on it beyond "these records appear to describe the same place."

Run it on your own files​

The map above is the same toMapHTML the record matcher ships, on the house MapLibre and Protomaps stack. The whole thing is one command over a config that tags each file with a role:

mailwoman registry --sources sources.json --reconcile --map-out map.html

Point it at your own pile of registries. The resolved place is the key they forgot to share.

The market hands you two tools for this, and each one solves a different half of the problem. Neither one finishes.

The two tools, and the halves they skip​

The first tool is a string matcher. Splink, Zingg, dedupe, the fuzzy-match block inside Tamr or Tilores: you feed them name and address columns and they score how similar the text is. This is the obvious move, and used as the whole answer it inherits an obvious flaw. Text similarity is not place similarity. Two doors down the same street is one character apart. The same building written two different ways can be string-distant. You spend the next month tuning a threshold that's too loose in the cities and too tight in the suburbs, and it's never right in both at once. (We took this apart line by line in Match where it is, not how it's spelled; the short version is that Jyllandsgade 15 and Jyllandsgade 75 score 0.96 similar and sit 650 metres apart, and no threshold saves you from that.)

That doesn't make strings useless. Names are some of the best evidence you have, and in a minute I'm going to argue you should lean on them hard. The catch is narrower: a string only knows how a record is spelled, never where it sits, so it can't be the thing you block on. Hold that thought.

The second tool is a join key. Placekey, SmartyKey, our own @mailwoman/address-id: they normalize the address, geocode it, and hand you back a deterministic identifier. Records that mint the same key are at the same place. This is better than string matching, because it keys on where instead of on spelling. Placekey will happily tell you those two records are the same building.

And then it stops. Because same building is the easy eighty percent.

A key gets you to the doormat​

Here's the part the key vendors are quiet about. At 1504 Taub Loop in Houston, four separate provider registrations resolve to the same rooftop, and all four are Baylor College of Medicine, one organization filed four ways. A deterministic key buckets them together, correctly. Good. But one address over, a single medical-office building holds a dozen distinct practices: different owners, different specialties, different everything except the street number. The key buckets them together too. Now it's wrong, and it can't tell you which case you're in, because a hexagon doesn't have an opinion.

Placekey's own FAQ says it plainly: it "operates as a universal join key rather than a matching service… it doesn't perform record linkage itself." Read that as the spec, because it is one. A key is where the problem starts. Same building is where the interesting question begins: same building, different company name on the door, are these one entity or two?

That question needs a probability, not a bucket.

The missing half: a number you can read, and the humility to abstain​

This is the half Mailwoman is built around. We resolve every record to a coordinate and a place hierarchy first, blocking on where, and then run a real probabilistic scorer over the candidates that share a place. It's Fellegi-Sunter. Every field agreement or disagreement contributes a weight in bits, the weights add up, and you get a calibrated probability that two records are the same entity, with the evidence laid out so you can see why. A shared rare street counts for more than a shared common one. A shared rooftop counts for more than a shared interpolated guess, because the geocoder tells the matcher how sure it was. The string comparison is right there in the score too: Jaro-Winkler over the names, a rare-token agreement on the organization name pulling hard while a shared LLC barely moves the needle. The names kept their job. We just moved them off deciding where and onto deciding who, which is what they were always good at. And when the number lands in the murky middle, the matcher doesn't flip a coin — it abstains, and routes the pair to a human, where it belongs.

So the pitch, in one breath: Placekey tells you two records are the same building. Mailwoman tells you, with a calibrated probability and a reason you can read, whether they're the same company, and says "I don't know, look at this one" when it shouldn't guess. That's the linker the key vendors stop short of, the geography the string-only matchers skipped, and a place to put the name-matching that mattered all along.

When the string is the answer, normalize it first​

Which ought to settle the strings-versus-coordinates argument, because sometimes the string really is the answer. Reconciling the federal provider registry against the FCC's funding records is exactly that case. The same health system lands in one file as BAYLOR COLLEGE OF MEDICINE and in the other as Baylor Coll. of Medicine, or as a d/b/a, or with a , LLC that the other publisher dropped. Geography gets you into the same neighborhood, so the two records are plausibly the same place. What actually decides same entity is the name, once you stop treating it as a literal string and start treating it intelligently.

So you normalize first. canonicalizeOrganizationName strips the legal-form noise (LLC, Inc, a leading The), splits the d/b/a, normalizes punctuation, and leans on the ISO 20275 entity-form list so Coll and College stop reading as different words. Then the names get compared, and a rare-token agreement on Baylor counts for a great deal more than a shared Medical Center. It's the same mechanism doing two opposite jobs at once. The building with a dozen distinct practices stays a dozen entities, same coordinate but different normalized names, while one facility filed twice under different operational names collapses into one. The coordinate does the blocking. The intelligent string match does the deciding. Neither one alone gets NPPES and the FCC to agree; together they do.

You don't have labels, and that's fine​

The other thing every ML-flavored matcher assumes is that you have training data. Zingg wants you to label thirty to fifty pairs to start; dedupe runs an active-learning loop that's all labeling; Tamr wants ongoing human curation. You don't have any of that. You have a pile of CSVs and a deadline.

Mailwoman's scorer estimates its own parameters from the data, unsupervised, with expectation-maximization. That's the same label-free Fellegi-Sunter math Splink uses, which is the fair comparison here: Splink does this too, and does it at a scale we haven't matched. The difference is that Splink expects you to have already turned the addresses into coordinates, while Mailwoman owns the whole stack from raw string to resolved place to calibrated link. And when you do have labels, an optional gradient-boosted scorer picks up the non-linear interactions the hand-built weights miss, worth about six points of pairwise F1 and five of clustering F1 in our held-out tests, across every random seed we tried.

It travels​

The worry with any learned scorer is that it's just memorizing one dataset's quirks. So we trained the boosted scorer on Texas providers and turned it loose on states it had never seen. It beat the unsupervised baseline by 20.5 points of F1 on California and 21.9 on New York, held out. The signal it learns, what over-merging actually looks like, isn't Texas-shaped. It transfers.

Show your ruler​

One more thing, because it's the part that earns trust. "F1 of 68%" is meaningless until you say against what. Grade the exact same clusters against four different definitions of truth and the number walks from 53.6% to 68.1%. The clusters never changed; the only thing that moved was the ruler, as it got more precise about what counts as one entity. We hand-adjudicated 120 of the hardest co-located, name-similar, distinct-ID pairs in Texas: all 120 were the same real-world organization under different registrations, and zero were unrelated companies wrongly fused. We report the ruler next to the number. Most vendors won't show you theirs.

Own it, and what it doesn't do​

It runs on your machine. Open source, your infrastructure, no per-record meter. That matters when the alternatives are AWS at $0.25 per thousand records billed on the non-matches too, Senzing starting around $58k a year, and Tamr, Quantexa, and Tilores all behind a "request a quote." Your data never leaves your control, which for compliance and healthcare records is frequently the whole requirement.

And the boundaries, because the angles above only hold if I'm upfront about these. Placekey does embody place-identity matching; angle four is the rebuttal, but the overlap is real. Melissa ships patented coordinate-proximity matching, so "nobody touches a coordinate" would be a lie; our narrower, true claim is that we make the calibrated coordinate the primary evidence in a probabilistic model. Splink has the label-free baseline at far more scale. The enterprise incumbents beat us on raw volume, multi-domain mastering, support, and decades of name-matching lore. What none of them put together is open, label-free, geocode-first, with the geocoder and the linker in one auditable stack. That's the difference, and it's a real one.

Whatever the matcher surfaces is a candidate for review, not a verdict. What a correlation means, and whether it means anything, is your call.

npm install @mailwoman/registry

import { resolveEntities, toMapHTML } from "@mailwoman/registry"

const entities = await resolveEntities(records, {/* sensible defaults */})
// → clustered entities, each with a coordinate, a cohesion score, and the records it merged

resolveEntities runs the whole block → score → cluster pipeline; toMapHTML drops the result on a map so you can see, dataset by dataset, what linked to what.

Our evals did, three times this week. One of them nearly talked us out of a model we should ship. One invented a coverage problem we don't have. And one had us writing "3.3 km" into a model card for a geocoder that puts most addresses within a hundred meters. Each time the fix was the same, and embarrassingly cheap: stop reading the summary row and pull the actual records the summary is averaging over.

One bad row out of twenty-seven​

We were grading a candidate model against a country-homograph set — addresses where the country name is also a US town, like "Lima, Peru" sitting next to "Lima, Ohio." The candidate scored 80.9 where the shipped model scored 83.3. A 2.4-point country regression, right there in the table. That's the kind of number that holds up a promote.

So we dumped the rows and diffed them, model against model. The entire 2.4 points was one address: Avenida Arequipa, Lima 15046, Peru. The candidate kept the street, the locality, and the postcode identical — it just dropped the trailing "Peru" to nothing. Every other country in the same sentence shape resolved fine; swap "Peru" for Chile, Bolivia, France, Jordan and the candidate gets them all. The set only has twenty-seven rows carrying a gold country, so a single flip is worth 3.7 points.

A metric that one record can swing by three and a half points isn't measuring a property of the model. It's measuring that record. The "regression" was a rumor, and we'd have let it sway a decision that mattered if we hadn't read the row.

The towns the scorer threw away​

The same week, the same eval told us our rural US locality resolution was a disaster: South Dakota matching 62%, Vermont 31%. That fit a story we already believed — the gazetteer is thin in rural states, the model can't resolve what isn't there — so it was easy to nod and file it under "known coverage gap."

Then we pulled the misses. The resolver was landing the right place on nearly all of them. The problem was the scorer: it only counted a match if the resolved place was tagged locality, and New England-style civil towns are localadmin in our gazetteer, not locality. "Barre Town," "Saint Albans Town," every Vermont township — the resolver found them, and the metric threw them on the floor for having the wrong placetype label. Credit the group the resolver actually treats as a locality and Vermont goes from 31% to 93%, South Dakota from 62% to 98%. The model was fine. The data was fine. The ruler was bent.

Kilometers on paper, meters in production​

This is the one that stings. Our flagship accuracy number — the one in the docs, the one in the model card, the one behind every "the US bottleneck is rural coverage" conversation we'd had for weeks — was coordinate error p50 3.3 km, p90 10 km. We read that as the ceiling. A city centroid is legitimately a few kilometers from an edge address, so we'd long since made our peace with it and gone looking for the next gazetteer to ingest.

The eval was resolving to the admin centroid. That's it. It took the parsed address, found the city, and returned the city's center point. But that is not what the geocoder ships. Production runs a cascade — it reads the parsed street, pulls that state's rooftop and interpolation data, and resolves the actual point, only falling back to the centroid when no point data exists. The eval never wired that layer in, so it had been reporting the blunt fallback as if it were the product.

We pointed the eval at the real cascade and ran the same ten thousand addresses. p50 went from 3.3 km to zero. p90 from 10 km to 1 km. 85.9% of US addresses land within a hundred meters; 90% within a kilometer — 80% on an exact rooftop point, another 8% on a street interpolation, and only the remaining 12% falling back to the centroid we'd been quoting as the headline. We had been underselling our own geocoder by three orders of magnitude, in public, because the eval graded a path the product doesn't take. And the "rural coordinate bottleneck" we'd been planning campaigns around? Same bent ruler. The rural states land on rooftops like everywhere else.

Read the rows​

Three different failures, one fix. A country regression that was a single record. A coverage gap that was a placetype-tag mismatch. A flagship accuracy number that was grading the wrong object entirely. None of them survived contact with the actual rows, and all of them looked authoritative as a number in a table.

There's a tidy version of this lesson — grade the thing your user actually receives — and it's true, and we keep it on the wall. But the operational habit underneath it is blunter than that. Before a number changes a decision, dump the records it's summarizing and read them. Ask whether there's a pattern or whether it's one row on a small denominator. Ask whether the thing you scored is the thing you ship. It costs ten minutes and a --errors-json flag, and it would have caught all three of these before they reached a card or a roadmap.

The averages lie in both directions. Yesterday it flattered us — a macro-F1 climbed while the model got worse at the most common address there is. This week it sandbagged us, three times, and we almost believed it because a number that says you're losing feels like honesty. It isn't honesty or dishonesty. It's an abstraction, and abstractions drop exactly the detail you needed. The rows still have it. Go read the rows.

It wasn't, and it took three probes to prove it. The same model that scored higher on the average had gotten worse at the most basic address there is — a town and a state. This is the story of how the average lied to us, and how we caught it.

What we were trying to fix​

Mailwoman's biggest measured weakness is boundary instability: the model wobbling on where one part of an address ends and the next begins. A street suffix swallowed into the street name. A French house number that trails the street instead of leading it. The kind of thing that's invisible on a clean "123 Main St, Springfield, IL 62701" and falls apart the moment the punctuation goes missing.

So we built a synthetic training shard that puts the gold boundary on exactly those hard cases, mixed it in at a conservative weight, and retrained. It worked — on the boundaries. French street-prefix recognition went from a coin flip (55%) to essentially perfect (99.7%). Every boundary shape moved up. The macro-F1 rose. We had the release notes half-written.

The catch​

We don't ship on the macro. We ship on a battery of per-locale floors ("don't regress US locality below 72.9," that sort of thing), graded against the assembled output: the actual parse a user would get, not the per-token score the trainer optimizes.

One floor came back red. US locality: 66.2, floor 72.9. The retrain that aced the boundaries had dropped six and a half points on locality, on real held-out US addresses, while the aggregate went up. The gains were concentrated where we'd added training data; the loss was spread across everything else, and the average happily averaged it away.

This is the trap we've been burned by before, and it has a name on this project: grade the assembled coordinate, never the label-F1. We wrote it on the wall. We still almost walked past it.

Three probes, one culprit​

A failing floor tells you that something regressed, not what. So we wrote probes.

The first asked a simple question of every boundary the model got wrong: is it confidently wrong, or is it uncertain? If the model spreads its probability mass and shrugs, that's a capacity ceiling — you need a bigger model, and more data is wasted GPU. If it commits hard to the wrong answer, that's a signal problem — more of the right data can flip it. Every failure came back confident. Good: this is fixable with data.

The second probe went after the locality drop directly. It took the model before the retrain and the model after, ran both over 1,616 real US addresses, and isolated every row where locality went from right to wrong. Then it asked where the locality text went.

We expected the street to be eating it. It wasn't. That was 3% of the damage. 84% of the regression was the model dropping the locality entirely on short, bare rows: "LaMoure, ND." "Frannie, Wyoming." "Buckhorn, Wyoming." A town and a state, nothing else, and the model now returned nothing for the town.

The mechanism, once you see it, is almost funny. Every example in our hard-case shard had a street before the city — that's what made the boundary hard. So the model learned, a little too well, that a city is the thing that comes after a street. Take the street away, hand it a bare "LaMoure, ND," and it no longer recognized LaMoure as a place at all. We didn't teach it a new skill so much as we taught it a superstition.

The lesson we keep re-learning​

The boundary shard's labels were sound; the trouble was that it only ever showed them in one shape. Every row was a clean, full, structured address, so the model over-fit that shape and fell off the moment the input stopped matching it. The fix is to surround the hard cases with the ordinary ones they need to be told apart from. Teach the bare "City, STATE" next to the full address. Teach the house number before the street as well as after, so position stops being a shortcut the model can lean on.

But the deeper lesson is the one about the number we started with. The macro-F1 went up. The model got worse at its most common real-world input. Both of those things were true at the same time, and only one of them mattered. If you're tuning a model on an aggregate score, the aggregate is the first thing that will lie to you — and it will do it while pointing at a bigger number and smiling. Grade the thing your user actually receives. We had to be told twice.

It's worth pulling apart what actually went wrong with that number, because two separate things did, and fixing one wouldn't have caught the other. The first is the averaging. A gain on rare shapes and a small loss on the common one net out positive, so the loss disappears into the mean; the cure is to stop reading the mean and start reading each slice on its own, US locality by itself rather than folded into a global score. The second is the one that nearly shipped the model. We were grading the wrong object: the trainer scores a label on every word, but the user receives an assembled parse, and the two can move in opposite directions. You can label more words correctly and still hand back a worse address, because one wrong word in the wrong place reshuffles the structure around it. Slice the word-labels as finely as you like and you're still grading the wrong product. So we grade the finished parse against what a real user would hand us, and we read it slice by slice. Both halves, every time.

Coda: did the fix work?​

Yes, on the labels — and in a way that drove the lesson home twice. We built exactly what the probe pointed at: teach the model bare "City, STATE" rows, so it stops needing a street to recognize a town. Held-out locality F1 climbed from 66 back up to 80, past where it started before any of this. The macro-F1? It barely twitched.

That's the whole point, restated from the other direction. The aggregate that almost fooled us into shipping the broken model was the same aggregate that wouldn't have told us the fix landed, either. Both times, the truth was in the held-out core (the town, the street, the house number a real user hands us), and both times the average just shrugged. So we read the core.

And then the lesson came back for a third helping. Before promoting the new model we graded the one thing we hadn't: the assembled coordinate, the actual point on the map a user receives, measured against the model already in production. It was flat. Locality F1 had climbed, but locality-match on real addresses and the median coordinate error did not move. The label gain never reached the map. We had done the very thing this post opens by warning against, one level down: graded the assembled labels and called it a win without grading the assembled coordinate. So we left the production model where it was.

The flatness pointed somewhere we hadn't been looking. The addresses still wrong on the map are small rural towns the gazetteer itself resolves poorly (a third of Vermont's, in one sample), places no amount of model training reaches. For now the model has caught up to its own database, and the next gain is in the atlas. That's a different post, and we'll grade it on the coordinate.

One footnote, because "add more data" and "buy a bigger model" aren't the only two endings a stuck case has. When a lever refuses to move, there are three reasons, and they ask for different things. Maybe the model picked up a bad habit from the data; better data breaks it, which is the whole story above. Maybe it's a thirty-million-parameter model out of room; only a bigger one helps. Or maybe the answer was never in the text: "Paris," on its own, is Paris, France or Paris, Texas, and no amount of data or parameters settles that, because the string doesn't carry the answer. The cheap way to tell the second reason from the third is to ask whether a person could do it from the very same input. Our four stubborn boundary cases are the second kind. A person segments a comma-less "City STATE" without trouble, so that's capacity, not confusion; we banked the locality win and left the boundaries for a bigger model. The third kind was never the model's job in the first place. It tags "Paris" as a place, and the gazetteer decides which Paris, leaning on the rest of the address and, when there's nothing else to go on, on which one more people mean. Telling the three apart is worth as much as the fix. That's a different post.

The first, which we call v0, is our TypeScript port of the Pelias parser — a rules engine: tokenize, classify each token against dictionaries and patterns, solve for the most plausible arrangement under a pile of hand-written constraints. It is fast, deterministic, and the product of years of accumulated postal wisdom. It is also the thing we set out to beat.

The second is the neural classifier — a sequence labeler trained on a BIO-tagged corpus, the one this blog has spent most of its life arguing with.

So: a year in, did the neural net actually beat the rules parser? The answer is mostly a tie, until the address gets weird — and the weird is where it gets interesting.

The scoreboard​

We don't grade the two parsers against each other's labels — that just measures which one looks more like the other. We grade both through the same resolver, against real address points: take 2,000 OpenAddresses US rows the model never trained on, parse each one, resolve it to a coordinate, and measure how often we land in the right city, the right state, and how far the resolved point is from the truth.

On clean US addresses it is close. Neural is a couple of points ahead on locality, a hair ahead on region, dead even at the median coordinate — and the difference you can actually feel is in the tail: neural's p99 error is 239 km versus 277. Both parsers nail the easy ones; the neural net just throws fewer wild misses.

(Pelias publishes no accuracy numbers of its own — only architecture docs — which is exactly why v0 is our baseline. We can't compare to a number nobody printed, so we ported the thing and graded it ourselves.)

A couple of points on clean addresses is not the headline. The headline is what happens off the beaten path.

Where the neural net pulls ahead​

Here is the same handful of addresses through both parsers. Watch the rules engine on anything that isn't <number> <street>, <city>, <state> <zip>.

A PO box — the classic. PO Box 1207, Anchorage, AK 99510:

v0    : street="PO Box"  house_number=1207  locality=Anchorage  region=AK  postcode=99510
neural: po_box="PO Box 1207"  locality=Anchorage  region=AK  postcode=99510

The rules parser reads "PO Box 1207" as a street named "PO Box" with house number 1207. It's not wrong by its own logic — "Box" looks like a street type, a number follows it — it just has no concept that this is a post office box, a different kind of place entirely. The neural net learned the po_box tag and uses it.

An intersection — 5th & Main, Springfield, IL:

v0    : street="5th"  locality=Springfield  region=IL
neural: intersection_a="5th"  intersection_b="Main"  locality=Springfield  region=IL

v0 keeps the first street and drops "Main" on the floor. The neural net knows an intersection has two sides and labels both.

A unit — 1600 Pennsylvania Ave NW Apt 4B:

v0    : house_number=1600  street="Pennsylvania Ave NW"  unit="4B"  ...
neural: house_number=1600  street="Pennsylvania Ave NW"  unit="Apt 4B"  ...

Both find the unit; the neural net keeps the designator ("Apt") attached, where v0 strips it to the bare "4B".

None of these are flukes. They're the payoff of a discipline we've written about before — negative space: you teach the model the tags the rules engine never had, by training on examples where they appear. The receipts, from our parity evals: unit coverage went from 0 → 92%, street_prefix 0 → 78%, street_suffix 0 → 67% — components the Pelias port simply doesn't emit. And on multi-locale evals the same recipe is why native-order German resolves locality at ~90% where a US-order assumption falls to the 40s (a story we've told before — the right name in the wrong state).

The pattern: on the addresses that fit the template, the parsers tie. On PO boxes, intersections, units, sub-street components, and non-US word order, the neural net is doing something the rules engine architecturally cannot.

Where we still struggle (and what we haven't touched)​

Honesty is the house style, so:

• Clean addresses are a tie, not a win. The two parsers above agree on 350 5th Ave down to the components — neural splits street="5th" + street_suffix="Ave" where v0 keeps "5th Ave" whole, and both are correct. For a model this good on US structure, the rules engine is not a thing to beat there; it's a thing to match.
• Reordered house numbers. French addresses that lead with the postcode still trip the model — fr.house_number plateaus around 87% on the hardest reordered slice, where the model wants to grab the leading number as the house number. We pushed it from 54% to 87% and then hit a wall that a training-weight tweak won't move; the next lever is real reordered data, not more synthetic.
• Whole locales we haven't tackled. Japanese, Korean, and Taiwanese addressing — block-and-lot systems that don't map onto a Western street grid — are scoped but unbuilt. So is the last stubborn 0% class in the postal arena: Commonwealth and military PO-box formats (BFPO, APO/FPO).
• Coordinate precision. Today's US coordinate is the admin-centroid tier — we land you in the right city, and "right city" is a centroid that can sit tens of kilometers from an edge address. The finer tiers (postcode centroid, then street-level interpolation) are wired but not yet the default everywhere.

We even tried to have both​

If the neural net wins on weird addresses and ties on clean ones, the obvious move is to run both parsers and keep the better answer per component. We built exactly that — a per-component arbitration layer — and it taught us something worth the detour.

On our label-match arena, arbitration looked like a triumph: it captured 122 parses the neural net alone "missed." Then we graded the assembled coordinate output instead of the labels, and it was a catastrophe — locality match fell from 83% to 57%, the median coordinate error blew from 3 km to over a thousand, and half the addresses lost their street-and-number entirely.

The diagnosis is the interesting part. The arena's "neural missed it" column conflates two very different things: addresses the neural net gets wrong, and addresses it gets differently right. When v0 writes street="Seminary Dr" and neural writes street="Seminary" + street_suffix="Dr", arbitrating toward v0 "wins" on label-match and loses on the actual parse — and flattening the two parsers' outputs together destroyed the tree structure the resolver needs to tell one "Mill Valley" from another. We rebuilt the layer to preserve that structure; the regression vanished, and so did the apparent win. The safe version is a net wash.

The lesson we're keeping: for a model this strong on the addresses we serve, there is no free lunch in blending it with the rules parser — neural isn't wrong where it differs, it's differently right, and "differently right" is not something to overwrite. We grade the coordinate now, never the label, and we let the arbitration machinery sit behind a default-off flag for the day a weaker model or a new locale makes it pay.

What's on deck​

The parser is in a good place: at or ahead of a mature rules engine on clean US, clearly ahead on the hard structured tags, and we've now mapped exactly where it isn't. The next moves are about feeding it and placing it more precisely:

• Real data over synthetic. Overture Maps address ingestion — to widen the corpus with real reordered, multi-locale, and long-tail formats, the thing the fr-house-number wall told us we need.
• Finer coordinate tiers. Make postcode-centroid and street-level interpolation the default where the data supports it, so "right city" becomes "right block."
• Geocode-first record matching. The other half of the box: once every record resolves to a coordinate, you can match records by where they are, not how they're spelled — the entity-resolution layer we've been building on top of all of this, and the subject of its own recent post.

A tie on Main Street was never the goal. Matching a battle-tested rules engine on its home turf, beating it everywhere the turf gets strange, and knowing precisely where the next kilometer of accuracy comes from — that's the state of affairs.

The weakness with many faces​

Boundary instability is what you get when two address components meet and the model puts the seam in the wrong place. Give it Country Club Rd and the street swallows its own suffix. Give it North Sydney NSW 2060 with the commas stripped out and the city and the state melt into each other. Hand it a French street like Neuve-des-Capucines 5 and the house number gets absorbed into the name.

For a while we filed a lot of this under "within-token punctuation" and blamed the apostrophes and hyphens. Then we actually measured that class, and the punctuation turned out to be innocent almost every time. O'Connell parses fine. Coeur d'Alene parses fine. What breaks is the boundary sitting right next to the punctuation, the same wobble wearing a different hat each time. One weakness, a dozen faces.

The plan​

Build a synthetic shard that drops the gold boundary exactly where the model wobbles, spread across a wide enough range of real-looking addresses that a retrain learns the boundary from context instead of memorizing the strings. Generate it, baseline the current model so we know how big the hole is, write the recipe, wire it into the corpus. Routine, on paper.

Gate one: the shard that graded on a curve​

The first version pulled from a thin vocabulary: sixteen street names and a handful of cities. Twenty thousand rows out of sixteen streets means the model sees Country Club Rd a thousand times over, so it learns the lexeme and never the seam. That part we expected. The sneaky part showed up when we baselined the current model against that same thin data and it scored 48% on the suffix boundary and 70% on the French prefix. Respectable. Encouraging, even.

Then we tripled the vocabulary, a hundred streets and real cities with every row unique, and the same model scored 41 and 48. So the thin data had done two things at once: it taught the lexeme instead of the seam, and it graded its own exam on a curve. The runbook already warns that thin diversity teaches the pattern-matcher and skips the boundary. Now there's a number sitting next to the warning.

Gate two: the postcode that was already a house number​

To teach the Australian slash convention, where 4/2A means unit 4, number 2A, the shard reached for Australian addresses. The base-consistency lint stopped it cold. In our training corpus the token 3000 shows up thousands of times, every one of them a US house number, and the shard wanted to teach it as an Australian postcode. A coverage shard can't outvote the base corpus it sits on top of. It can only pick a fight with it and lose, slowly, over a release or three, and we still have the scars from the last time. The lint caught the collision before we'd burned a single GPU-hour on it. Australia came back out, the shard is base-locales-only now, and the slash convention got filed as its own job, the kind that has to bring its own base coverage to the table.

Gate three: the city that was usually a street​

Even with the shard confined to the locales we actually train on, the lint wasn't done with us. Paris. Springfield. Burlington. In our corpus those tokens are overwhelmingly streets, the Paris Avenues and Springfield Streets of the world, and only rarely cities. Our shard labels them as localities. Both readings are true depending on where the token lands, and the model is context-aware enough that it might well sort it out on its own. The lint's job is to not take that on faith. It flagged the minority sense, because the last time we waved this exact thing through, a theatre on 5th Avenue talked a shard into believing 5th Avenue was a venue. So the retrain will watch this number, and a cleaner shard draws its cities from names the corpus already agrees are cities.

Why we didn't retrain​

Here's where the night actually ended. The shard is built. The baseline is measured. The recipe is written and signed off, the corpus is assembled, and the manifest is tested against the real base. By every box on the checklist it's ready, sitting on the shelf one command short of a training run. And we left it there on purpose.

Those three refusals were the actual work. Naming the weakness took a taxonomy; fixing it correctly took the gates saying not yet, once about the data and twice about the corpus, and each no cost us a few minutes and spared us a retrain that would have looked fine up top and been wrong all the way down. That's a trade we'll take every time. The fastest way we know to fix your worst weakness is to let the gates make you earn it.

123 Main Street, Suite 400, Springfield IL 62704
123 Main St #400, Springfield, Illinois

Easy — yes. Now these two:

Jyllandsgade 15, 9000 Aalborg
Jyllandsgade 75, 9000 Aalborg

Also easy — no. They're 650 metres apart.

Now imagine a string-similarity matcher looking at those same four lines. The first pair, the same place, scores low: different punctuation, "Street" vs "St", a reordered unit. The second pair, different places, scores 0.96 — one character apart. The tool you'd reach for gets both backwards. This isn't a tuning problem you can threshold your way out of. It's the wrong coordinate system.

An address is a location wearing a costume​

An address is not text. It is a point on the Earth that happens to be written down, and the writing is lossy, inconsistent, and adversarially formatted by a hundred different intake forms. "650 metres apart" is a fact about the world. "One character apart" is a fact about the costume.

Here is the same matching decision, scored two ways:

String-first draws its decision boundary vertically: spell it close enough and it is a match, regardless of where on Earth the two records sit. Geocode-first draws it horizontally: near is plausible, far is no, and the spelling only breaks ties between neighbours. The two traps — the same place wearing different costumes, the different places wearing nearly the same one — live in the corners where those boundaries disagree. Geocode-first gets both right. The image above is the same Fellegi-Sunter scorer with real per-field weights, run twice: once seeing only the name, once seeing distance too. Geography carries the heavier evidence by design.

So we stopped matching the costume. Mailwoman is a calibrated street-level geocoder — give it a messy line, it returns a coordinate, a resolution tier, and a calibrated uncertainty. Once you have that, entity resolution changes shape. You match on the resolved place: the coordinate, the canonical key, the admin hierarchy. The string becomes evidence, not identity.

Three things change, and they drive the rest.

Blocking becomes a partition, not a search cluster. The expensive part of matching a million records is not scoring a pair — it is finding the pairs worth scoring without comparing all 500 billion of them. The textbook answer is a search cluster: stand up Elasticsearch, index everything, query for candidates. Geocode-first replaces that infrastructure with a single pass through the data. Two records falling into the same ~5 km cell are candidates; everything else is skipped. The geocoder already did the partition. We have measured the matcher resolving half a million records in about a minute in one Node process on commodity hardware, against a baseline that requires standing up a search index — no server, no Docker, no process to keep alive. The number is what it is; the architecture is what matters.

Distance becomes a feature you can calibrate. Same building: ~30 metres. Same block: ~300 metres. The geocoder knows its own error — rooftop points are tight, interpolated points looser, centroids looser still — so distance evidence can be weighted by placement quality. A string matcher has no equivalent. It can tell you two streets are spelled similarly. It cannot tell you whether they are the same building.

Shared addresses carry different evidentiary weight by rarity. Two records at the same clinic tower agreeing on the address is weak evidence they are the same entity — fifty providers share that building. Two records agreeing on a rural address with one occupant is strong evidence. It is TF-IDF for places: weight an address agreement by how many distinct entities sit on it. On a real provider-registry deduplication, turning that on moved F1 by roughly twenty points in a single change. Most of the gain was recall — a provider's own rare address re-stitched records that name drift had split apart — rather than the precision we had expected. The data surprised us. We adjusted.

The receipts​

We pointed this at real public data: the national provider registry (NPPES), the FCC's Rural Health Care funding filings, and a state nursing-facility registry. Three datasets with no shared key, each published in the fragmented shape compliance data typically arrives in.

Resolving within the provider registry, we used the registry's own entity ID as held-out ground truth. The result was an unflattering number — and then a surprise about that number, which gets its own section below. We also have detail on which levers move real errors and which backfire. Negative results are still results.

Resolving across the three datasets is where the architecture shows its purpose. With no shared ID — purely on geocoded location plus name agreement — the matcher links rural Texas hospitals across the funding program and the provider registry: Hunt Memorial Hospital District, Uvalde County Hospital Authority, Winkler County Memorial Hospital. Each is a record that says "I exist" in one dataset and a record that says "I got funding" in another, stitched because they are the same building, which is a thing you can only know if you geocode first.

Once stitched, you can ask the question the fragmented data was hiding: which entities appear in the eligibility sources but not the funding ones? That anti-join is what someone currently finds by squinting at a map. We produce it as a layer that drops onto the map directly.

A careful word on that, because it matters. We produce the reconciliation, not the conclusion. An entity in the "eligible, not enrolled" set is a candidate for review, not an allegation. A gap can mean the entity did not apply, applied under a name we did not resolve, is not actually eligible, or any number of mundane things. The tool's job is to resolve public records into an analyzable shape and surface candidates. What a correlation means is the analyst's call.

The number was lying too​

To measure deduplication within the registry we needed ground truth. The registry hands you a gift: a stable, government-issued ID on every provider. Grade your clusters against it, count disagreements, done. Clean, objective, unarguable.

It is also wrong, and wrong in the direction that makes you look bad. That ID is a registration number, not an organization number — one hospital system holds a separate registration for every billing subpart. Baylor College of Medicine shows up four times at a single address. So when the matcher does the correct thing and fuses those four records into one entity, the registration-number ruler scores those pairings as over-merges. You get penalized for being right. The better the matcher, the worse the number looks.

Grade the same clusters — identical output, not a different model — against a ruler that collapses same-organization records at the same building, and the F1 climbs without touching a line of code:

We froze the hardest cases — co-located records with near-identical names that the registration ID splits apart — and adjudicated them one at a time: 120 of 120 were the same real-world organization. Zero were two unrelated outfits wrongly fused. The thing we had been logging as our biggest error class was, almost entirely, the matcher being right and the yardstick being too literal.

Believing the first number cost us work. We built corroboration features to cut that over-merge, and they moved precision not at all — because there was no over-merge to cut. We optimized against a phantom.

So the lesson: a matching F1 with no named truth-grain is unreadable. State the ruler next to the number, every single time. When the number is unflattering, check the ruler before you blame the model.

And 68.1% is still a C-grade F1. The yardstick argument contextualizes the number; it does not excuse it. The matcher still misclusters. Rural route addresses and PO boxes defeat the geocoder — a shared box at a post office reads as the same rooftop coordinate for every resident, and the distance feature becomes noise. Rooftop interpolation can place a strip-mall suite on the wrong unit. The performance table includes those rows. We are working on them.

Why pure Node matters​

You could bolt a geocoder onto a record-linkage library and get some of this. What you cannot get is the integration — the same calibrated geocoder feeding the blocking, weighting the distance evidence, and running in one process with no external service.

That last part is architectural, not merely convenient. Geo-blocking is what lets candidate generation run in-process at all; a string matcher needs the search cluster precisely because it cannot partition the problem cheaply. Geocode-first removes the reason the cluster existed. The whole pipeline — parse, geocode, block, score, cluster — installs with npm install and runs as a function call. No Elasticsearch, no Docker, no server to keep alive. For a compliance team with a folder of CSVs and no infrastructure budget, that is the difference between a project and an afternoon.

That is the bet. Match the location, not the spelling, and the coordinate system does the rest.

The questions that opened up: why does a function that nails San choke on New Yor? What's the difference between completing a word and completing the word a person is in the middle of typing? And how does an autocomplete that knows ten thousand cities fail to finish one of them?

The autocomplete we already had​

Our gazetteer — every country, region, county, and city we resolve against — is stored as a finite-state transducer: a trie over place-name tokens, with each accepting state carrying the places that spell out to that point. mailwoman autocomplete "San" walks the trie to the San node and reads off everything beneath it, ranked by importance: San Francisco, San Diego, San Bernardino. It's quick because there's no search — the FST is the index. You walk to a node and the answers are sitting there.

For a command line, this is exactly right. You type a whole word, you hit tab, you get completions. The unit of input is the word, and the trie is keyed by words. The shapes match.

Then we typed half a word​

A search box does not hand you whole words. It hands you New Yor. It hands you Chic. It hands you the front half of a word and asks what you meant.

To walk a token trie you need tokens, and Yor is not a token — it's the beginning of one. The walk steps through New, reaches for an edge labeled Yor, finds nothing, and fails. That part is forgivable; a trie can't match a fragment. What was not forgivable was the fallback: a fuzzy text search over the gazetteer that did its earnest best and returned the nearest fuzzy match it could find. For New Yor, that was Denver. We do not know why Denver — whether the fuzzy distance metric weighted string length unexpectedly, or whether token boundaries threw the scoring off, or something else. We did not investigate, because the right answer was not to fix the fallback. It was to stop needing one.

It was worse than Denver​

So we fed it whole tokens, expecting relief, and got New → New London. Four times.

Two problems, stacked. The first was duplicate collapse: New London the city, New London the county, and a couple of smaller New Londons each took a slot, so the dropdown was four rows of the same name before it offered anything else. The second was subtler and worse. The New node has 311 children. The walk collects completions breadth-first under a budget, and a dense branch — all those New Londons — filled the budget before the search ever reached New York. The autocomplete knew New York perfectly well. It just never got there. The most prominent place a person could possibly mean, starved out by smaller ones that happened to sort first.

The fix​

When the walk fails on a partial last token, don't fall back to fuzzy search. Walk the complete prefix instead — New — then filter that node's outgoing edges by the fragment: keep every edge whose token starts with yor. york survives. london doesn't. You've completed the word using the trie you already have, no fuzzy search, no Denver.

Then cap how many places any single branch can contribute, so one dense New London branch cannot crowd out New York before the search reaches it. Collapse same-name places to the most prominent one. Three changes — character-level completion via edge filtering, a per-branch cap, name dedup — and New Yor becomes New York, Chic becomes Chicago, San Fr becomes San Francisco. The search box finishes your word.

Two problems wearing one name​

"Complete a place word" and "complete the word a person is typing" look like the same feature. They are not. One assumes the token is finished and asks what comes after it; the other assumes the token is unfinished and has to finish it character by character, while ranking results well enough that the obvious choice surfaces. The CLI only ever exercised the easy half, so the hard half shipped untested and we never knew.

The same function was correct on the command line and a liability in the search box, and nothing about its name, its tests, or its track record warned us — because the surface it was correct on never asked it the hard question. "Autocomplete" was two features in a trench coat, and we only met the second one when a stranger typed half a word and it tried to send them to Colorado.