We lost to Nominatim in Europe. Then we found out why.

We had just watched our geocoder beat Nominatim across the United States by fifteen points, and we were feeling good about ourselves. So we pointed the same benchmark at Europe expecting a victory lap. Europe handed us a double-digit loss instead.

That sat badly. Not because losing is shameful — Nominatim is the bar, it carries the whole planet on community-contributed data, and clearing it anywhere is the goal. It sat badly because we didn't understand it. We knew our parser wasn't ten points worse in Europe than in America. So what was the gap actually made of?

This is the answer, the two fixes, and — because we'd be kidding you otherwise — the parts the fixes didn't reach. For the European leg we added a third system to grade against: Pelias, by way of geocode.earth, the hosted Elasticsearch stack a lot of people reach for. It turns out to be the real bar, and we'll be honest about where it still beats us.

How we kept score

We graded the way we always argue you should: not on whether the model's labels matched our labels, but on where the address lands on Earth. OpenAddresses ships a truth coordinate with every row, so for each address we parse it, resolve it to a point, and measure the great-circle distance to where it really is. Inside 25 kilometers, we call it right-placed; a no-result counts as a miss, same as a wrong answer. One honest number per system, on the same real addresses, each given the same country hint so nobody's racing with a handicap.

That bar is stricter than the one we'd been quietly grading ourselves on. Our internal "resolve rate" had been counting any resolution as a win — including ones that landed a region away. Re-graded at 25 km, it had been flattering us by fifteen to twenty points in Europe. So the first casualty of this benchmark was our own scoreboard. Fine. Grade the coordinate.

The US, briefly — because it's why Europe stung

In the US, mailwoman right-places 99% of these addresses at 25 km; Nominatim, 84%. The difference is almost entirely no-result: ours is near zero, Nominatim's is about one in six. That's not a knock on them — it's where a purpose-built stack shows, with TIGER and a fifty-state rooftop layer covering ground that community OSM data hasn't filled in yet.

The part that matters for the rest of this post: we do that in a 30 MB model that runs in your browser, no Elasticsearch, no server round-trip. Same engine, same deployability, pointed at Europe. Which is exactly why a double-digit loss there was confusing rather than merely disappointing.

Europe: we weren't wrong, we were silent

Graded by the same ruler across six European locales, mailwoman started at about 66%. Nominatim sat at 78%, and Pelias — leaning on Elasticsearch and a pile of mixed sources — at 89%. We were behind both, and the reflex reading is "the parser is worse in Europe." It isn't. When mailwoman does resolve a European address, the point is tight — a kilometer or two from truth. The gap was almost all the other failure mode: on a messy European address, better than one in four came back empty. The model didn't place them wrong. It placed them nowhere.

That's a more fixable problem than being wrong. So we went looking for what made it go quiet — and found two different things, one clever and one just embarrassing.

Why it went quiet, part one: a word it mispronounced

The model fragments. Its subword tokenizer takes a town like Grudziądz and, on the accented ą, splits it into Grudzi + dz. Neither half is a place. Neither resolves. The whole address fails — and the town's name was sitting right there in the input, intact, the entire time. The tokenizer mispronounced a word it had been handed correctly.

Once you see it, you see it everywhere in the unresolved pile: Polish, Czech, Portuguese towns the model chopped on a diacritic and then couldn't find.

Why it went quiet, part two: postcodes we never had

The second cause had no cleverness to it at all. For Poland and Czechia, we simply didn't have the postcodes. WhosOnFirst, our gazetteer's backbone, ships rich postcode coverage for some countries and none for others, and PL and CZ were in the none column — zero rows. A Polish address whose town the model did read correctly could still die, because the only anchor we could have resolved it by wasn't in the database to begin with. Silence by omission.

The fixes: read the word, and fill the gap

For the first, when a parse resolves nothing, we now go back to the raw text — not the model's subwords, the actual characters, diacritics intact — and try the obvious thing. Enumerate the whitespace tokens, match each against the same-country gazetteer, and take the longest exact match (the real town is usually the longer, specific name, not its ambiguous prefix). We made ourselves prove it recovered real coordinates before we trusted it: the recovered towns land about 1.8 km from truth, not in some same-named place a country away.

For the second, we pulled the missing postcodes from GeoNames — about 73,000 for Poland, 15,000 for Czechia, each with its own centroid, openly licensed — and folded them straight into the gazetteer. No model change, no retrain. Just the rows that should have been there all along.

Here's the part we didn't expect, and it's worth sitting with. The two fixes mostly do the same job by different routes. Where we now have the postcode, the address resolves directly and the word-recovery has nothing left to do; where we don't, the recovery carries it. On the same ruler, the postcode fill alone lifted Europe from 66% to 84%. The word-recovery alone, also 66% to 84%. Together: 88%, with our empty-answer rate dropping from better-than-one-in-four to under one in twelve.

The third fix: stop being confidently wrong

Eighty-eight percent still left a stubborn pile of misses, and they weren't silence — they were the opposite. "06260 Saint-Pierre, La Placette 7" came back 617 kilometers from truth. Saint-Pierre is one of the most common town names in France, and we'd picked the wrong one. But the postcode, 06260, had resolved correctly the entire time — it pins the Alpes-Maritimes, right where the address is. We had the answer sitting in the same parse and threw it away, because the coordinate-picker trusted the town name and never checked it against the postcode next to it.

So we taught it to check. When a town lands far from its own postcode, re-pick the same-named town nearest the postcode instead; if none is close, fall back to the postcode's own point. It's the consistency idea from the recovery gate, moved out of the rescue path and into the main one — no model change. It took Europe from 88% to 94%, and once we'd extended the GeoNames postcode fill to Portugal, Austria, and Australia, it pulled Australia from 35% to 65% — a resolved postcode finally gave the disambiguation something to stand on.

The honest part

Here's where we resist the victory lap, because we've embarrassed ourselves before by taking one early. Re-graded on that same European ruler, all three systems side by side: mailwoman 94%, Nominatim 78%, Pelias 89%. We're clear of both now — five points over a hosted Elasticsearch stack, sixteen over Nominatim — from a 30 MB model running in a browser tab. Italy, Portugal, France, Czechia: ahead outright.

Across all seven locales, Australia included, it's 90% for us, 81% Nominatim, 88% Pelias. Still ahead, but the margin is thin and the whole of it is Australia, where we score 65% to Nominatim's 97. And here we owe you the thing we only just worked out: Australia is a reading problem, down in the parser. Australian addresses lead with the postcode — "3053 Carlton, Barry Street 50" — and our model, raised on American and European formats where the number comes first, reads that leading 3053 as a house number and mangles everything after it. Hand it the same address reordered the American way and it parses perfectly; 65% becomes 87%. So the fix lives in the model: we teach it the Australian format, with the national address file. We didn't close Australia. But we finally know what's holding it shut, and it isn't the map.

There's a precision cost worth stating plainly, too. At 25 km we're competitive; at one kilometer we're not. We resolve to admin and postcode centroids, so we right-place the area — within a couple of kilometers — where Nominatim and Pelias drop a rooftop pin. That's the trade behind the headline: right neighborhood, not right doorstep. For a lot of jobs the neighborhood is enough; when it isn't, the incumbents win that round and we'll say so.

And the word-recovery stays off by default, because about one in six of the newly recovered addresses lands more than 25 km off — the recovery matched a wrong same-named place. Where we have postcode coverage, a consistency check throws those out before they ship; the GeoNames fill means that check now fires for Poland and Czechia, where a month ago it couldn't. Australia is the place it still can't. So a batch pipeline leaves recovery off, because a wrong coordinate poisons a join silently; the demo turns it on, because in an interactive tool "something you can see and correct" beats "nothing." Every recovered place is tagged with whether the postcode check actually fired, so a caller decides for itself rather than inheriting our guess.

The thing a scoreboard can't show

One more, because it's the reason we don't think a single resolve-rate tells the whole story. A search index returns a result. It doesn't tell you how much to trust it. Mailwoman's confidence is calibrated — a span it marks 90% is correct about 90% of the time, measured on held-out data. That's a number you can build a rule on: auto-accept above a threshold, route the rest to a human, and the threshold means what you set it to. It's the one thing the recovered-but-unverified coordinate above is careful not to fake — we flag that separately rather than dress it up as a calibrated probability, because the whole value of the number is that it doesn't lie.

So here's the standing, stated without flinching. Pelias is ahead of us across the full panel, and it places rooftops where we place neighborhoods. On Europe we beat Nominatim handily and matched Pelias — in 30 megabytes, in your browser, with no Elasticsearch and a confidence number that means what it says. We're still the challenger on a chunk of the map, and Australia is a problem we haven't solved. We mostly closed a double-digit gap by learning to read a word we'd been mispronouncing and adding the postcodes we should have shipped in the first place. There's more to learn. We'll keep grading the coordinate.