Skip to main content

Why addresses are hard

The question​

Addresses look like the easiest structured data in the world. They have parts, the parts come in an order, and every one of us can read one at a glance. Any engineer given the problem writes a plausible parser in an afternoon, and it works. Then it meets production data and the afternoon turns into a year.

The interesting question is not which addresses break it — Falsehoods about addresses is twenty-five of those, each with a real counterexample, and this page will not repeat them. The question is why the list never ends. What is it about addresses that makes the failures structural rather than a backlog?

Three things, and none of them is fixed by trying harder.

The analog​

A postal address is not a data format. It is a note written to a person, and it succeeds when a human courier can act on it. That is a much weaker constraint than a schema, and it is why addresses have the properties they do: anything a knowledgeable local can act on is a valid address, including things no format would permit. "The blue house past the church" is a working address in places where houses have no numbers. It is also unparseable, and it is not malformed — it is doing its job.

Everything below follows from that one fact. We are parsingaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates. something that was never designed to be parsed.

One: there is no universal grammar​

The first reflex is to write a grammar. Number, then streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels., then town, then regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality., then postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one.. It describes a large fraction of the addresses most engineers have seen, which is exactly why it feels like a description of addresses rather than a description of one countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.'s convention.

Germany writes the house numberhouse numberThe numeric or alphanumeric identifier of a building on a street. Mailwoman's house_number component; its position relative to the street name flips between locales. after the streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.. France writes the postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. before the town. Japanese addresses start at the prefecture and work inward, and number blocks rather than streetsstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. — so there is no streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. name to parseaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates. at all. Mannheim's citylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighborhood in the hierarchy. center is a lettered grid where R 5 is a block, not a road. Addressing around the world walks those conventions properly.

The structural point is not that there are exceptions. It is that there is no privileged ordering for the exceptions to be exceptions to. Each convention is complete and internally consistent in its own countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head., so a grammar cannot be extended into a universal one — it can only be replaced by a per-countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. set, and the number of countries is not the problem. The problem is that a single input string does not come labeled with which set to apply, so choosing the grammar requires having already partly understood the string.

Two: the ambiguity is intrinsic, not incidental​

Here is the part that resists every amount of engineering: you cannot group the words correctly without knowing what they are, and you cannot tell what they are without knowing how they group.

Take Saint Petersburg, FL. Split it into words and classify each one independently, which is what a rule-based approach does, and every individual verdict is defensible. Saint really is a common street prefixstreet affixA modifier on a street name indicating type or direction — Street, Avenue, rue, Calle, N, East. Mailwoman tags these as street_prefix / street_suffix, recognized via a morphology FST.. Petersburg really is the name of populated places. FL really is a state abbreviation. Each classifier is right, and the combination is wrong, because Saint Petersburg is one town and the boundary was drawn before anyone asked what the piecesECE (Expected Calibration Error). A metric that measures how well a model's confidence scores align with its actual accuracy. Lower is better. Mailwoman's held-out ECE drops from 0.067 (raw) to 0.0035 (calibrated). were.

Worse, the confidence looks fine. Every component scored high, because every component's evidence was real. A consumer reading the scores sees a clean parseaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates.. This is the failure mode that makes the problem structural rather than merely difficult: the error is in the combination, and per-piece confidence has nowhere to record it.

The same shape appears wherever a name is doing a job other than naming its own place. A venuevenueA named, non-address place — a business, building, park, or stadium. Mailwoman's free-text point-of-interest component, added as a Tier 2 fine label. called after a citylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighborhood in the hierarchy.. A streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. named after a state. A town whose name is also a direction word. Any string that appears in both the place vocabularyvocabularyThe fixed set of tokens a tokenizer can produce. Mailwoman's SentencePiece vocabulary is tens of thousands of subword pieces, with byte fallback for anything outside it. and the address vocabularyvocabularyThe fixed set of tokens a tokenizer can produce. Mailwoman's SentencePiece vocabulary is tens of thousands of subword pieces, with byte fallback for anything outside it. will be classified wrongly, with high confidence, whenever the surrounding structure is what decides it — and the surrounding structure is what the classifier has not looked at yet.

There are two ways out and neither removes the ambiguity. One is to reverse the order: propose the boundaries first, from structural cues rather than dictionaries, and only then ask what each proposed spanspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree. is. The other is to decide both at once, over the whole string, so the labelcomponent tagOne of the 25 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag. for one spanspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree. can depend on its neighbours. Mailwoman does both — a grouping stagestageOne of the dataflow stages in the runtime pipeline (normalize, locale gate, kind classify, phrase group, token classify, sequence correct, reconcile, resolve). Distinct from tier (model vocabulary) and phase (plan milestone). before the labeling stagestageOne of the dataflow stages in the runtime pipeline (normalize, locale gate, kind classify, phrase group, token classify, sequence correct, reconcile, resolve). Distinct from tier (model vocabulary) and phase (plan milestone)., and a labeling stagestageOne of the dataflow stages in the runtime pipeline (normalize, locale gate, kind classify, phrase group, token classify, sequence correct, reconcile, resolve). Distinct from tier (model vocabulary) and phase (plan milestone). that reads the entire input rather than one tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. at a time — which moves the decision to where the most evidence is. It does not make the input unambiguous. Springfield on its own is undecidable no matter what reads it, because the information required is not in the string.

That last point is the one worth carrying: some inputs are irreducibly ambiguous, and the correct output for them is not a better guess. It is a result that says so.

Three: the register users type is not the register data is stored in​

GazetteersgazetteerA geographical index that maps place names and postcodes to real-world coordinates. Mailwoman uses a custom-built Who's On First (WOF) SQLite database as its gazetteer — the 'atlas' half of the grammar/atlas architecture. store Montréal, İstanbul, São Paulo, in title case, with the accents, under the official name. Users type montreal, on a phone, in one lowercase run, having learned from a decade of map search boxes that capitalization is not required and diacriticsdiacriticAn accent mark that modifies a letter (é, ñ, ç). Address normalization must fold diacritics for matching without discarding the information a user typed. are not reachable.

Lowercase is not degraded input. It is the registerinput modeThe Decision-A register switch: 'fragmented' (human-typed fragments — feeds the evidence channels) vs 'formatted' (complete records — runs the trained absence identity). Explicit on CLI/API; per-endpoint defaults (batch→formatted, autocomplete→fragmented); kind-derived otherwise. real users type, and a system that treats it as an error case has misidentified where its users live. The same is true of the colloquial name (Bombay for Mumbai, Saigon for Ho Chi Minh City) — both names in each pair are in current use by millions of people, and the choice between them can carry political weightparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values.. Rejecting one is rejecting real usage; silently rewriting it is making a decision on the user's behalf that they did not ask for.

And the same is true of the wrong-but-workable name: the neighborhood given as the citylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighborhood in the hierarchy., the independent citylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighborhood in the hierarchy. given as its larger neighbour, the postal town that is not the municipalitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighborhood in the hierarchy.. In every one of those cases the person supplied enough for a knowledgeable local to find the place. The parser failed because it was built for addresses rather than for people.

The practical consequence is that case, accents and colloquial forms are signal to be handled rather than noise to be stripped — and that a normalization which flattens them can destroy the evidence the labeling stagestageOne of the dataflow stages in the runtime pipeline (normalize, locale gate, kind classify, phrase group, token classify, sequence correct, reconcile, resolve). Distinct from tier (model vocabulary) and phase (plan milestone). needed. Handle messy input shows what the pipelinestaged pipelineMailwoman's runtime architecture: a sequence of pure-function stages (normalize → query-shape → locale-gate → kind-classifier → phrase-grouper → classifier → decoder) connected by typed handoffs. Each stage is published as its own npm package. absorbs unaided, what the normalizenormalizeStage 1 of the runtime pipeline: deterministic input preprocessing (Unicode NFC, punctuation normalization, whitespace collapse). Returns a NormalizedInput with an offsetMap that maps normalized positions back to the raw input. stagestageOne of the dataflow stages in the runtime pipeline (normalize, locale gate, kind classify, phrase group, token classify, sequence correct, reconcile, resolve). Distinct from tier (model vocabulary) and phase (plan milestone). repairs, and the one case where stripping does harm.

What follows from all three​

If the grammar is per-countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head., the ambiguity is intrinsic, and the input registerinput modeThe Decision-A register switch: 'fragmented' (human-typed fragments — feeds the evidence channels) vs 'formatted' (complete records — runs the trained absence identity). Explicit on CLI/API; per-endpoint defaults (batch→formatted, autocomplete→fragmented); kind-derived otherwise. is not the storage registerinput modeThe Decision-A register switch: 'fragmented' (human-typed fragments — feeds the evidence channels) vs 'formatted' (complete records — runs the trained absence identity). Explicit on CLI/API; per-endpoint defaults (batch→formatted, autocomplete→fragmented); kind-derived otherwise., then a parser cannot be finished. There is no state in which the rule set covers the cases, because the cases combine: a Cyrillic streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. name, in a non-English order, with an apostrophe in it, in a town that was renamed last year.

So the design goal moves. It stops being "return the right answer" and becomes "return an answer whose own reliability is legible" — surface the alternatives when several readings are live, report the grain of the coordinate rather than only the coordinate, and let the consumer decide how much ambiguity its task can absorb. That is the same discipline the postal system itself runs on: the sorting machine passes what it cannot read to a person, and the person passes what they cannot resolve to the carrier who knows the route. Each link hands the next one enough to decide.

There is a limit on the learning side too, and it belongs in the same breath. A modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' that has seen no fractional house numbershouse numberThe numeric or alphanumeric identifier of a building on a street. Mailwoman's house_number component; its position relative to the street name flips between locales. will not invent the category; it will assign the piecesECE (Expected Calibration Error). A metric that measures how well a model's confidence scores align with its actual accuracy. Lower is better. Mailwoman's held-out ECE drops from 0.067 (raw) to 0.0035 (calibrated). to whatever its trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input. distribution makes likely. Learning the distribution beats enumerating the rules, and it is not a way of escaping the need for data about the shapes you care about.

  • Falsehoods about addresses — the specimens, twenty-five of them.
  • Addressing around the world — the conventions the first section refers to.
  • Handle messy input — what the pipelinestaged pipelineMailwoman's runtime architecture: a sequence of pure-function stages (normalize → query-shape → locale-gate → kind-classifier → phrase-grouper → classifier → decoder) connected by typed handoffs. Each stage is published as its own npm package. repairs, and what it leaves alone.
  • Understand a parse — reading spansspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree., offsets and confidence on a real result.
  • How close is close enough? — what to do with an answer that reports its own uncertainty.