Skip to main content

How it used to work — Mailwoman v1

Mailwoman v1 (the pre-2026 version, still living on as the rule classifiers inside v2) parsesaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates. an address in four steps. This article walks through each one with a concrete example.

The input we will use:

West 26th Street, New York, NYC, 10010

Step 1 — Tokenization

The input string is split into tokenstokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. — single words or punctuation marks. Tokenization is more than string.split(" ") because Mailwoman keeps track of where each tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. came from in the original string.

Each tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. carries metadata: its character position in the original string, what kind of separator came before it (comma, space, tab, newline), and what other tokenstokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. it is grouped with. The grouping idea — Mailwoman calls these sections — is important: tokenstokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. that are separated by commas usually belong to different address components, so the solver treats them differently.

Tokenization is also the place where pre-processing happens: lowercase normalization, abbreviation expansion (St.Street), and accent handling for non-English input.

Deep dive: concepts/tokenization.md.

Step 2 — Rule classifiers vote

Each tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. is shown to every rule classifier in parallel. A rule classifier is a small piece of code that tries to labelcomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag. a tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. as one address component type.

Examples of rule classifiers in Mailwoman v1:

  • house_number — does this tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. look like a number (with optional letter suffix like 123A)?
  • postcode — does this tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. match a countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.-specific postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. pattern? (10010 matches the US 5-digit pattern; 75008 matches the FR 5-digit pattern.)
  • street_prefix — is this tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. a known direction (North, West, SE) or a known streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.-type prefix (Avenue, Rue, Boulevard)?
  • whos_on_first — is this tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. (or this short phrase) found in the Who's On FirstWOF (Who's On First). An open-source gazetteer of places maintained by Mapzen/whosonfirst. Mailwoman builds a custom SQLite database from WOF GeoJSON repos, extended with postcode data, importance scores, and coincident-role relations. gazetteergazetteerA geographical index that maps place names and postcodes to real-world coordinates. Mailwoman uses a custom-built Who's On First (WOF) SQLite database as its gazetteer — the 'atlas' half of the grammar/atlas architecture. as the name of a countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head., regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality., localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy., or neighbourhood?

Each classifier produces zero or more classifications for each tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words.. A classification is a triple:

{ component: "postcode", confidence: 0.95, source: "rule:postcode" }

A single tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. can collect multiple classifications. For our example, New might collect:

  • { component: "locality", confidence: 0.4, source: "rule:whos_on_first" } (because "New" alone matches a few WOFWOF (Who's On First). An open-source gazetteer of places maintained by Mapzen/whosonfirst. Mailwoman builds a custom SQLite database from WOF GeoJSON repos, extended with postcode data, importance scores, and coincident-role relations. entries)
  • { component: "street_prefix", confidence: 0.1, source: "rule:street_prefix" } (because "New" sometimes precedes a streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. name)

This is intentional: rule classifiers are allowed to be uncertain and contradictory. The next step resolves the contradiction.

Deep dive: concepts/rule-based-classifiers.md.

Step 3 — The solver picks a winning combination

The solver looks at all the classifications produced for all the tokenstokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. and tries to find a self-consistent interpretation. "Self-consistent" means:

  • Each address component appears at most once (you can have one locality, not two).
  • Components do not overlap on the same tokenstokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words..
  • The combination obeys soft preferences (a US-style postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. after a regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. is more likely than a postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. at the start, for example).

Mailwoman v1's solver is an ExclusiveCartesianSolver with filters and augmenters. In plain terms: it generates every plausible combination, filters out the ones that violate the hard rules, scores the rest, and returns them ranked.

For our example, a top-ranked output looks like:

[
{ "component": "street_prefix", "value": "West", "confidence": 0.85 },
{ "component": "house_number", "value": "26th", "confidence": 0.6 },
{ "component": "street", "value": "Street", "confidence": 0.4 },
{ "component": "locality", "value": "New York", "confidence": 0.9 },
{ "component": "locality", "value": "NYC", "confidence": 0.7 },
{ "component": "postcode", "value": "10010", "confidence": 1.0 }
]

Notice the two locality candidates. The solver returns multiple ranked solutions; the consumer (the CLI, the API) picks the top one or shows all of them.

Step 4 — Resolve

Parsingaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates. answered "what kind of thing is each part of this string?". Resolving answers "where is the resulting place?". The parsed components are looked up in a gazetteergazetteerA geographical index that maps place names and postcodes to real-world coordinates. Mailwoman uses a custom-built Who's On First (WOF) SQLite database as its gazetteer — the 'atlas' half of the grammar/atlas architecture.Who's On FirstWOF (Who's On First). An open-source gazetteer of places maintained by Mapzen/whosonfirst. Mailwoman builds a custom SQLite database from WOF GeoJSON repos, extended with postcode data, importance scores, and coincident-role relations., in Mailwoman's case — and the gazetteergazetteerA geographical index that maps place names and postcodes to real-world coordinates. Mailwoman uses a custom-built Who's On First (WOF) SQLite database as its gazetteer — the 'atlas' half of the grammar/atlas architecture. returns coordinates, a stable place ID, and optionally a bounding box.

The resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon. is a separate concern from the parser. Read about it in concepts/resolver-and-wof.md.

What this approach is good at

  • Determinism. A rule classifier produces the same answer on the same input every time. No retraining, no randomness.
  • Explainability. When the parser is wrong, you can read the rule and see why.
  • Fast iteration on a single bug. "The postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. pattern misses Canadian H1A-X9X" is a one-line code change.

What this approach is bad at

  • The long tail. Every new address shape needs a new rule or a tweak to an existing one. The list of rules grows forever.
  • Words that look like multiple things. "Buffalo" is a US localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy., a venuevenueA named, non-address place — a business, building, park, or stadium. Mailwoman's free-text point-of-interest component, added as a Tier 2 fine label. name (Buffalo Wild Wings), and an animal. Rules can declare all three; only data can rank them.
  • Multi-word components. "Saint Petersburg" is one localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy., not two. A rule that recognises common multi-word names is brittle.
  • Languages other than English. Every localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for. needs its own rule set, hand-written, by someone who reads that language well.

These weaknesses are why Mailwoman v2 brought in the neural classifierneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.'. Continue with how-it-works-now.md.