The model knew it was Austria. The resolver sent it to West Virginia.
Type Vienna, Austria into our geocoder and, until this week, it answered with a confident set of coordinates: 39.32, −81.54. That's Vienna, West Virginia — population about ten thousand, a few miles up the Ohio River from Parkersburg. The capital of Austria is 7,500 kilometers and one ocean away, and the geocoder had no doubt whatsoever.
It does this with Sydney too (a hamlet in Ohio), and Zurich (Kansas), and Toronto (also Ohio, as it happens). The pattern is always the same: you name a famous foreign city, you name the country it's in, and the geocoder hands you back its smallest American namesake like it's doing you a favor.
The fix that felt like progress
When you find a bug like this, the first idea is almost always the expensive one. Ours was: the country router doesn't know enough countries. Teach it that "Austria" means Austria — feed it more examples, retrain the model, ship a new set of weights. It's a satisfying plan. It feels like progress, because training a model is the kind of work that looks like work.
We had the GPU budget queued and the issue written up as a model gap. And then, because we've been burned before by fixing the wrong layer, we did the cheap thing first: we asked the model what it actually saw.
Here is Vienna, Austria straight out of the parser:
Vienna → locality
Austria → country
The model knew. It tagged Austria as a country, cleanly, every time — Australia, Switzerland, Canada, all of them. There was no model gap. The thing we were about to spend a retrain on was already working perfectly. The bug lived one layer down, in the part of the system that takes a correct parse and turns it into a place on Earth.
Forty-five Viennas and a popularity contest
The resolver's job is to take the word "Vienna" and decide which Vienna. Our gazetteer knows about forty-five of them in the United States alone, plus the real one in Austria, plus a few others. Faced with that pile, the resolver did the reasonable-sounding thing: it picked the most prominent match by population and name — and then it threw the country token away. It never asked the one question the address had already answered for it.
So Vienna resolved on its own merits, the populous American candidates outranked the Austrian one in the bag, and Austria sat there in the parse, ignored, like a return address nobody read. The model had said which country in plain text, and the resolver wasn't listening.
The honest version of the bug: the parse and the resolve each made a locally sensible decision, and nobody checked that they agreed.
The fix we didn't make
There's a tempting shortcut here, and it's worth naming because we've watched it sink geocoders before. You could keep a list. "If the address says Austria, force the country to AT." It would work for Vienna, and Sydney, and Zurich. It would also be the first paving stone on the road to a thousand-entry exception table that someone maintains forever and nobody trusts — the same road that turned older rules-based geocoders into archaeology. We don't pin, and we don't keep lists of special cases the model is supposed to defer to.
The fix instead asks the parse and the resolve to be consistent with each other. When the address carries an explicit country and the locality it resolved to lives somewhere else, re-pick the locality to the one that actually sits inside the country the address named. The country code comes from the model's own country emission, normalized through an ISO table we already had on the shelf; the gazetteer's own country column does the rest. No list. No prior. It generalizes to every country at once, because it isn't about any particular country — it's about the address agreeing with itself.
There's a guard rail worth mentioning, because it's where this kind of fix usually goes wrong. It only fires when the country is the nearest thing the address says about geography. Write Springfield, IL, USA and the state is right there doing the disambiguating, so the new pass stays out of the way and Springfield stays in Illinois — not the largest Springfield in America. The fix earns its keep precisely by knowing when not to act.
What it bought, and what it cost
We measure these things against coordinates, not labels, because a name-match metric would have told us this bug didn't exist. The honest yardstick is a bare City, Country query — the exact thing a user types into the box — scored by how close the answer lands to the real place.
| before | after | |
|---|---|---|
City, Country resolves correctly | 54.2% | 77.9% |
| countries the country-name now rescues | — | 45 of 57 |
Forty-five countries that used to need a hint and now don't, a twenty-four-point jump on the query shape people actually use, and the cost on the GPU meter was zero. The thing we'd budgeted a retrain for was a hundred-odd lines in the resolver.
The cost it does carry is the honest kind: it can only re-pick to a place the gazetteer actually has. Where we don't carry a city, the country in the address can't summon it from nothing, and the fix correctly does nothing rather than guess. That's a coverage problem, and it's a different lever for a different day.
The part worth keeping
We almost retrained a model to fix a bug the model didn't have. The diagnostic that saved us was thirty seconds of parsing seven addresses and reading the output — cheaper than the issue we'd written, never mind the GPU hours behind it.
The reflex says train. The discipline says look first. The model knew it was Austria the whole time; we just had to teach the rest of the system to listen.
(One more for the honesty file: the test that proves Sydney now lands in Australia failed the first time I ran it — by 7,532 kilometers. The fix was fine. I'd dropped the minus sign on Sydney's latitude and parked it in the northern hemisphere. The gate caught my typo before it caught a real regression, which is exactly the order you want.)