Skip to main content

Concept deep dives

Fifty articles, one per concept. Each is independent — read them in any order. If you want the problem framed before the solution, read understanding/ first.

New to Mailwoman? Start with How Mailwoman parses an address for the walkthrough — one address, followed through the whole parseaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates. — then What Mailwoman is for the product boundary: what the system claims to do, and what it deliberately doesn't. Everything else here is a deep dive behind one of those two pages. The parseaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates. handoff continues in How Mailwoman resolves a place; Data, locales, and coverage and Quality and evaluation cover what's supported where and how to trust the numbers.

The clusters​

  • Parsingaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates. internals — tokenization, BIO labelsBIO tagging (Begin-Inside-Outside). A token-level labeling scheme where each token is tagged as B-X (beginning of an entity of type X), I-X (inside an entity of type X), or O (outside any entity). Mailwoman uses BIO over SentencePiece tokens to annotate address components., the ViterbiViterbi decodingA dynamic programming algorithm that finds the most likely sequence of hidden states (labels) given a sequence of observations (token emissions). Mailwoman uses Viterbi over a linear-chain CRF to produce globally coherent BIO label sequences from per-token model scores. decoderdecoderIn a transformer encoder-decoder model, the part that produces output sequences. Mailwoman's classifier is encoder-only (no decoder); the 'CRF decoder' is a different thing — a structured-prediction layer that picks the best label sequence from the encoder's outputs., the FSTFST (finite-state transducer). A compact automaton that reads an input sequence and emits an output sequence. Mailwoman encodes gazetteer names and street affixes as FSTs for fast prefix matching and prior injection without search overhead. priors, attentionattentionThe core mechanism inside a transformer encoder. Each token's representation is updated by looking at every other token, with learned weights deciding how much each one matters.. What happens between a raw string and a labeled tree. Start at How the model reasons.
  • TrainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input. and corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus. — how the trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input. data gets built, validated, and turned into a modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.'. Start at Training pipeline.
  • ResolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon. and gazetteergazetteerA geographical index that maps place names and postcodes to real-world coordinates. Mailwoman uses a custom-built Who's On First (WOF) SQLite database as its gazetteer — the 'atlas' half of the grammar/atlas architecture. — turning labeled spansspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree. into coordinates against Who's On FirstWOF (Who's On First). An open-source gazetteer of places maintained by Mapzen/whosonfirst. Mailwoman builds a custom SQLite database from WOF GeoJSON repos, extended with postcode data, importance scores, and coincident-role relations.. Start at Resolver and Who's On First.
  • Record matchingrecord matchingThe process of determining whether two database records refer to the same real-world entity. Mailwoman's matcher uses a geocode-first approach (match the resolved place, not the address string) with Fellegi-Sunter probabilistic scoring. — geocode-firstgeocode-firstThe matcher's core design principle: resolve addresses to geographic coordinates first, then compare the resolved places — not the raw address strings. Two records at the same coordinates match even if one says '123 Main St' and the other says '123 MAIN STREET.' entity resolutionrecord matchingThe process of determining whether two database records refer to the same real-world entity. Mailwoman's matcher uses a geocode-first approach (match the resolved place, not the address string) with Fellegi-Sunter probabilistic scoring. for messy records with no shared ID. Start at Geocode-first record matching.
  • Switching guides — coming from PeliasPeliasAn open-source geocoder, Mailwoman's spiritual predecessor., libpostallibpostalAn open-source C address parser used by Pelias. Mailwoman's rule-based v0 and neural classifier supersede it., Nominatim, Photon, or OpenCage? These map the differences directly, starting with How Mailwoman compares.

Article shape​

Each article is about 5–10 minutes to read, with a short motivation, an explanation that defines its terms on first use, a diagram where structure helps, a short code-or-data example, pointers into the source for readers who want to go further, and a "See also" list.