Containment and tree projection

After the encoderencoderThe part of a transformer that turns input tokens into contextualized vector representations. Mailwoman's classifier is a small encoder-only transformer (~30M parameters). produces emissions, the FSTFST (finite-state transducer). A compact automaton that reads an input sequence and emits an output sequence. Mailwoman encodes gazetteer names and street affixes as FSTs for fast prefix matching and prior injection without search overhead. priors add their biases, and ViterbiViterbi decodingA dynamic programming algorithm that finds the most likely sequence of hidden states (labels) given a sequence of observations (token emissions). Mailwoman uses Viterbi over a linear-chain CRF to produce globally coherent BIO label sequences from per-token model scores. finds the best valid BIO sequence (see those sub-articles), the last step turns that flat labelcomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag. sequence into a hierarchical AddressTree. This is the piece that lets downstream consumers ask "what's the localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.?" or "how does this streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. nest inside its admin parent?" without doing the structural walk themselves.

The piece is small but critical. It's where the schema's parent/child rules turn an array of labeled tokenstokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. into a queryable tree.

What the tree looks like

For 123 Main St, Boston, MA 02101:

Five AddressNodes with these relationships:

• region parents nothing (root in this address)
• locality parents to region
• street parents to locality
• house_number parents to street
• postcode parents to locality (siblings of streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.)

The decoderdecoderIn a transformer encoder-decoder model, the part that produces output sequences. Mailwoman's classifier is encoder-only (no decoder); the 'CRF decoder' is a different thing — a structured-prediction layer that picks the best label sequence from the encoder's outputs. didn't "discover" this structure from the address text. It walked a rule table that says "given these tags, here's how they nest."

The rule table: PARENT_OF

core/decoder/containment.ts exports PARENT_OF: Partial<Record<ComponentTag, ComponentTag[]>>. Each tag lists its preferred parents in priority order. Excerpt:

export const PARENT_OF: Partial<Record<ComponentTag, ComponentTag[]>> = {
	// Universal coarse — geographic granularity
	region: ["country"],
	subregion: ["region", "country"],
	locality: ["subregion", "region", "country"],
	dependent_locality: ["locality"],
	postcode: ["locality", "subregion", "region", "country"],
	cedex: ["postcode", "locality"],

	// Street-level
	street: ["dependent_locality", "locality", "subregion", "region"],
	street_prefix: ["street"],
	street_suffix: ["street"],
	house_number: ["street"],
	unit: ["street", "house_number"],

	// Venue / mailing
	venue: ["street", "locality"],
	po_box: ["locality", "subregion", "region"],

	// JP-specific
	prefecture: ["country"],
	block: ["district"],
	sub_block: ["block"],
	building_number: ["sub_block", "block"],
}

Reading the rule for street: a streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. tries to nest under dependent_locality first, then locality, then subregion, then region. The tree builder walks this list and parents the streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. to the first listed tag that ACTUALLY has a labeled spanspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree. in this address. Missing tags fall through.

The tree-building walk

core/decoder/build-tree.ts does the projection in three passes:

1. SpanspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree. aggregation. Walk the BIO sequence, group consecutive same-tag tokenstokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. into spansspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree.. [B-street, I-street, I-street] becomes one streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. spanspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree. covering all three tokenstokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words..
2. Parent resolution. For each spanspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree., walk its PARENT_OF list. Find the first parent-tag that has a labeled spanspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree. in this address. If multiple spansspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree. of the parent tag exist (multiple localities, say), pick the one nearest by character distance.
3. Tree assembly. SpansspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree. whose parent tag has no labeled spanspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree. become tree roots. All others attach as children of their resolved parent.

The output is AddressTree { raw, roots: AddressNode[] }. Each AddressNode carries tag, start, end, value, confidence, children.

What the projection buys you

It's deterministic, given the labels

The encoderencoderThe part of a transformer that turns input tokens into contextualized vector representations. Mailwoman's classifier is a small encoder-only transformer (~30M parameters). + FSTFST (finite-state transducer). A compact automaton that reads an input sequence and emits an output sequence. Mailwoman encodes gazetteer names and street affixes as FSTs for fast prefix matching and prior injection without search overhead. priors + ViterbiViterbi decodingA dynamic programming algorithm that finds the most likely sequence of hidden states (labels) given a sequence of observations (token emissions). Mailwoman uses Viterbi over a linear-chain CRF to produce globally coherent BIO label sequences from per-token model scores. produce the labelscomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag.. Everything after that is rule-table application. Two different runs that produce the same BIO sequence will produce bit-identical trees. This makes the decoderdecoderIn a transformer encoder-decoder model, the part that produces output sequences. Mailwoman's classifier is encoder-only (no decoder); the 'CRF decoder' is a different thing — a structured-prediction layer that picks the best label sequence from the encoder's outputs. behavior easy to reason about and easy to test.

It surfaces structural correctness as a separate axis

A modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' can produce per-tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. labelscomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag. that are correct in isolation but structurally weird:

• A localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. spanspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree. with no parent regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. (just floats at the root)
• Two localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. spansspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree. in one address (Brooklyn AND New York)
• A house_number without a streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. parent (the resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon. downstream doesn't know how to handle this)

These are "labelscomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag. correct, structure wrong" failures. They show up in the tree, not in per-tag recallrecallOf the spans whose gold label is a given tag, the fraction the model found. High recall means few misses. Paired with precision to compute F1.. A tree validator (planned, not yet shipped — see corpus-poisoning-vulnerability.md's deferred itemsexpectation-maximizationAn iterative algorithm that estimates model parameters when some variables are unobserved. In Mailwoman's matcher, EM learns the Fellegi-Sunter m and u parameters from unlabeled data — no training labels needed.) would check the AddressTreeparse treeThe hierarchical address structure produced by decoding BIO labels into spans and nesting them per the schema's containment rules (house_number → street → locality → region → country).'s structural validity as a final quality gate.

It's the schema's source of truth

The PARENT_OF map is the canonical statement of "what nests in what." The street-supplement architecture treats this as the schema's source of truth — extending it (e.g. adding dependent_street → street) is a schema-level decision that flows into trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input. corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus. design, evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. golden setsgolden setA hand-labeled evaluation dataset of US, French, and adversarial addresses used as ground truth for evals., and downstream consumers.

The WOF hierarchy gap doc explores how WOFWOF (Who's On First). An open-source gazetteer of places maintained by Mapzen/whosonfirst. Mailwoman builds a custom SQLite database from WOF GeoJSON repos, extended with postcode data, importance scores, and coincident-role relations.'s placetypeplacetypeThe Who's On First hierarchical classification of places: planet → continent → country → region → county → locality → neighbourhood. The resolver uses placetype to rank candidates — an exact locality match outranks a county-level match. DAG and mailwoman's PARENT_OF map are related but distinct ontologies — the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.''s tags answer "what role does this spanspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree. play in an address," while WOFWOF (Who's On First). An open-source gazetteer of places maintained by Mapzen/whosonfirst. Mailwoman builds a custom SQLite database from WOF GeoJSON repos, extended with postcode data, importance scores, and coincident-role relations.'s placetypesplacetypeThe Who's On First hierarchical classification of places: planet → continent → country → region → county → locality → neighbourhood. The resolver uses placetype to rank candidates — an exact locality match outranks a county-level match. answer "what kind of geographical featurefeatureAn input signal a model conditions on. Beyond the raw tokens, Mailwoman feeds soft features — gazetteer-membership channels and the postcode anchor — that inform predictions without overriding them. is this." The containment rules let us project modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' output into either ontology cleanly.

What this CAN'T do

• Multi-parent. Each spanspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree. has exactly one parent. The DAG-like fact that a borough can nest under both locality and localadmin in WOFWOF (Who's On First). An open-source gazetteer of places maintained by Mapzen/whosonfirst. Mailwoman builds a custom SQLite database from WOF GeoJSON repos, extended with postcode data, importance scores, and coincident-role relations. doesn't carry over — the decoderdecoderIn a transformer encoder-decoder model, the part that produces output sequences. Mailwoman's classifier is encoder-only (no decoder); the 'CRF decoder' is a different thing — a structured-prediction layer that picks the best label sequence from the encoder's outputs. picks one.
• Cross-address linkage. The tree is per-address. If you parseaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates. a list of addresses and want to detect that they all share the same localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy., that's a resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon. concern, not a decoderdecoderIn a transformer encoder-decoder model, the part that produces output sequences. Mailwoman's classifier is encoder-only (no decoder); the 'CRF decoder' is a different thing — a structured-prediction layer that picks the best label sequence from the encoder's outputs. one.
• Disagreement resolution. If the BIO sequence has two localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. spansspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree. that disagree (say one is "Brooklyn" and one is "New York" in the same input), the tree builder produces both as separate spansspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree. parenting to whatever they parent to. The downstream resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon. decides what to do with that.

See also

• How the model reasons — the central pipelinestaged pipelineMailwoman's runtime architecture: a sequence of pure-function stages (normalize → query-shape → locale-gate → kind-classifier → phrase-grouper → classifier → decoder) connected by typed handoffs. Each stage is published as its own npm package.; this article expands stagestageOne of the dataflow stages in the runtime pipeline (normalize, locale gate, kind classify, phrase group, token classify, sequence correct, reconcile, resolve). Distinct from tier (model vocabulary) and phase (plan milestone). 6
• Viterbi and BIO validity — what produces the BIO sequence this article consumes
• WOF hierarchy gap — the relationship between mailwoman's PARENT_OF and WOFWOF (Who's On First). An open-source gazetteer of places maintained by Mapzen/whosonfirst. Mailwoman builds a custom SQLite database from WOF GeoJSON repos, extended with postcode data, importance scores, and coincident-role relations.'s placetypeplacetypeThe Who's On First hierarchical classification of places: planet → continent → country → region → county → locality → neighbourhood. The resolver uses placetype to rank candidates — an exact locality match outranks a county-level match. DAG
• Street-supplement architecture — the layered design that extends PARENT_OF for streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.-side components
• core/decoder/containment.ts — the actual rule table
• core/decoder/build-tree.ts — the projection algorithm