I'm trying to make sense of a blocker but can't quite figure out how to proceed. Can you review this write up and let me know what you think? I think what embarrasses me is that this is an encoderencoderThe part of a transformer that turns input tokens into contextualized vector representations. Mailwoman's classifier is a small encoder-only transformer (~30M parameters)., the smallest sort of ML task. I think I'm not understanding what's going on because I don't know enough about machine learningmachine learning (ML). Building systems that learn patterns from examples instead of following hand-written rules. Mailwoman's neural classifier is trained on millions of labeled addresses rather than programmed with parsing rules. 101
The segmentation thesis: a review
For a reader who knows postal addresses cold and ML in general terms. Everything here is measured on the shipped modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' (v264 / v6.3.0) or the probe checkpointcheckpointA saved snapshot of the model weights and optimizer state during training. Mailwoman saves a checkpoint periodically so training can resume after a GPU hang. (v301), on committed fixtures, with the commands in the repo. Where we were wrong, the correction is in place rather than in a footnote.
1. The problem, stated precisely
Mailwoman is a ~29M-param encoderencoderThe part of a transformer that turns input tokens into contextualized vector representations. Mailwoman's classifier is a small encoder-only transformer (~30M parameters). that emits per-subword BIO tags over 33 address tags, decoded by ViterbiViterbi decodingA dynamic programming algorithm that finds the most likely sequence of hidden states (labels) given a sequence of observations (token emissions). Mailwoman uses Viterbi over a linear-chain CRF to produce globally coherent BIO label sequences from per-token model scores. under a structural BIO mask. Downstream, a WOFWOF (Who's On First). An open-source gazetteer of places maintained by Mapzen/whosonfirst. Mailwoman builds a custom SQLite database from WOF GeoJSON repos, extended with postcode data, importance scores, and coincident-role relations.-backed resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon. geocodes the parse treeparse treeThe hierarchical address structure produced by decoding BIO labels into spans and nesting them per the schema's containment rules (house_number → street → locality → region → country)..
Classification is solved. Segmentation is not. On the rescued v1 parity corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus. (321 hand-written fixtures, 20 countries, deliberately fragment-heavy):
| tag | shipped v264 | floor |
|---|---|---|
| postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. | 0.986 | 0.97 |
| house_number | 0.808 | 0.97 |
| streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. | 0.573 | 0.90 |
Coarse geography is ~99%. The modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' knows what things are. What it cannot reliably do is say where one thing ends and the next begins.
The archetype, on the shipped modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.':
Korunní 810, Praha → street="Korunní 8" house_number="10"
It knows there's a streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. and a number. It puts the boundary inside the number.
2. The thesis
Under flat BIO, the output factorizes into T per-tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. decisions. The encoderencoderThe part of a transformer that turns input tokens into contextualized vector representations. Mailwoman's classifier is a small encoder-only transformer (~30M parameters). is shared, so the tokenstokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. are not independent in any representational sense — but the lossloss functionA number measuring how wrong the model's predictions are on a batch of examples. Training minimizes it. Mailwoman's loss combines per-token negative log-likelihood with the CRF sequence loss. and the decode treat them as separable. "These five subwords are ONE streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels." is therefore not a decision the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' is ever scored on; it's an emergent property of five separate votes that happen to agree. There is no object anywhere in the objective or the decode that represents the solution.
This has two consequences we can point at in the data:
(a) Nothing rewards governance. Rue is deterministic in French — it can only be a street prefixstreet affixA modifier on a street name indicating type or direction — Street, Avenue, rue, Calle, N, East. Mailwoman tags these as street_prefix / street_suffix, recognized via a morphology FST.,
governing what follows. A transformerneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.''s attentionattentionThe core mechanism inside a transformer encoder. Each token's representation is updated by looking at every other token, with learned weights deciding how much each one matters. is perfectly capable of carrying that implication
forward; the representation is not the constraint, and we should not claim it is. What's missing is
narrower and more fixable: nothing in the objective or the decode rewards segmentsegmentA punctuation-bounded chunk of the normalized input — the comma-separated parts of 'Portland, OR' — used to give downstream stages structural context.-level coherencecoherenceThe property of a parse whose resolved places form a consistent geographic hierarchy — the resolved locality really does sit inside the resolved region.,
and the prior on bare toponymstoponymA proper name for a geographic place. is wrong. Cross-entropycross-entropyThe standard classification loss: it penalizes a model for putting low probability on the correct label. Per-token negative log-likelihood is the cross-entropy of each token's label. over T positions pays for each tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. being
individually right. A reading that is locally reasonable everywhere and incoherent as a whole costs
the lossloss functionA number measuring how wrong the model's predictions are on a batch of examples. Training minimizes it. Mailwoman's loss combines per-token negative log-likelihood with the CRF sequence loss. nothing. So:
Avenue Victor Hugo → street ✓ ("Victor Hugo" is a person)
Rue Montmartre → locality ✗ (Montmartre IS a Paris district)
Rue de Rome → locality ✗ (Rome IS a city)
12 Rue Montmartre → hn / prefix / street ✓
The prefix isn't the variable — toponymtoponymA proper name for a geographic place. identity is, and a house numberhouse numberThe numeric or alphanumeric identifier of a building on a street. Mailwoman's house_number component; its position relative to the street name flips between locales. is what breaks the tie. "Montmartre" votes localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy., which is what the trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input. distribution taught it to do: bare streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. fragments are rare in the corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus. and bare localities are not, so the prior is doing exactly its job and the job is wrong. Nothing in the objective makes propagating the prefix's implication cheaper than ignoring it.
(b) No structural coherencecoherenceThe property of a parse whose resolved places form a consistent geographic hierarchy — the resolved locality really does sit inside the resolved region.. Nothing enforces that the output looks like any address. Compare the rules-based ancestor (PeliasPeliasAn open-source geocoder, Mailwoman's spiritual predecessor.-style scheme matching), which always emits a structurally coherent solution — because a scheme is a claim about the whole input at once. The two systemsexpectation-maximizationAn iterative algorithm that estimates model parameters when some variables are unobserved. In Mailwoman's matcher, EM learns the Fellegi-Sunter m and u parameters from unlabeled data — no training labels needed. fail from opposite defects: rules fail on messy input because the hypothesis space is closed (a finite scheme list; anything off-template falls through); flat BIO fails on clean input because the space is too open (T independent votes; nothing constrains the shape).
The proposal: score whole segmentations — a semi-Markov CRFCRF (Conditional Random Field). A statistical modeling method that predicts structured outputs by modeling dependencies between adjacent labels. Mailwoman uses a linear-chain CRF as the Viterbi decoder at inference time to enforce BIO label consistency — a B-street must be followed by I-street or O, never I-locality. over spansspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree., with a segmentsegmentA punctuation-bounded chunk of the normalized input — the comma-separated parts of 'Portland, OR' — used to give downstream stages structural context.-level transition grammar. Same idea as scheme matching, except the scores are learned and the hypothesis space is every segmentation rather than a hand-written list. And critically: k-best output, because the useful artifact is a ranked list of readings, not one answer.
Worth noting for anyone who's been here before: this project abandoned a tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words.-level CRFCRF (Conditional Random Field). A statistical modeling method that predicts structured outputs by modeling dependencies between adjacent labels. Mailwoman uses a linear-chain CRF as the Viterbi decoder at inference time to enforce BIO label consistency — a B-street must be followed by I-street or O, never I-locality. at v0.5.0
(bf16bf16 (bfloat16). A 16-bit floating-point format used during training. Half the memory of fp32 with similar numeric range, which lets the training batch fit on the lab's GPU. NaNNaN (not a number). A floating-point result for an undefined operation (log of a negative, 0/0). Appearing in the training loss usually halts the run; recovering from it follows the NaN protocol., crf_loss_weight=0.0 ever since). That scar was treated as "CRFCRF (Conditional Random Field). A statistical modeling method that predicts structured outputs by modeling dependencies between adjacent labels. Mailwoman uses a linear-chain CRF as the Viterbi decoder at inference time to enforce BIO label consistency — a B-street must be followed by I-street or O, never I-locality. diverged" for ~200 modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.'
versions. It doesn't transfer: at subword granularity, transitions are noise — "must 1 follow ▁8"
is not grammar. At segmentsegmentA punctuation-bounded chunk of the normalized input — the comma-separated parts of 'Portland, OR' — used to give downstream stages structural context. granularity (5–8 segmentssegmentA punctuation-bounded chunk of the normalized input — the comma-separated parts of 'Portland, OR' — used to give downstream stages structural context., not 40 subwords), "house_number is adjacent to
streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.", "one postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. per reading" is well-posed. Same table, right altitude.
3. The address taxonomy that predicts failure
This is the part a rules person will recognise, and the part that surprised us. Measured on a purpose-built 63-fixture Paris corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus. (famous streetsstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels., homonyms, esoterica, elisions, date-names, plus contextful controls), shipped v264:
| class | example | v264 streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. exact |
|---|---|---|
| contextful/multi-class | 12 Rue du Chat-qui-Pêche, Paris | 9/10 |
| contextful/homonym | 8 Rue de Rome, Paris | 6/6 |
| bare/esoteric | Rue de la Grande-Truanderie | 7/10 |
| bare/elision | Rue de l'Hôtel-de-Ville | 3/6 |
| bare/homonym | Rue de Rome | 4/12 |
| bare/famous | Avenue des Champs-Élysées | 3/15 |
| date-name | Avenue du 11-Novembre-1918 | 1/4 |
The exotic morphology is not the problem. Apostrophe elisions, hyphenated compounds, date-based
names, accented capitals — 12 Rue du Chat-qui-Pêche, Paris parsesaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates. correctly. Meanwhile
Avenue des Champs-Élysées returns the empty string.
The variable is the house numberhouse numberThe numeric or alphanumeric identifier of a building on a street. Mailwoman's house_number component; its position relative to the street name flips between locales.. On the general corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus., the same split:
| fail rate | |
|---|---|
| no house numberhouse numberThe numeric or alphanumeric identifier of a building on a street. Mailwoman's house_number component; its position relative to the street name flips between locales. present | 66% (80/122 streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. failures) |
| leading US-style number | 22% |
And a result that killed a planned tokenizertokenizerThe component that converts a raw address string into a sequence of numeric token IDs the model can process. Mailwoman's tokenizer is a SentencePiece unigram model trained specifically on postal addresses. change: multi-digit numbers are the best-performing
digit form (17% fail) vs short-digit 29% and alphanumeric (16a) 73%. We had a digit-atomicity
splice queued on the theory that per-digit tokenization (810 → ▁8 1 0) caused the boundary bleed.
The partition counter-evidenced it before it cost a trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input. run.
The two failure mechanisms
Of 30 unfixed Paris cases, 26 have no digit at all, and they split cleanly:
17 — the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' calls the whole string a locality. Not silence; confidence.
Avenue des Champs-Élysées → locality. This is a recallrecallOf the spans whose gold label is a given tag, the fraction the model found. High recall means few misses. Paired with precision to compute F1./polarity failure.
13 — the spanspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree. truncates, always at a French function word or a digit:
Rue des Rosiers → "Rue des"
Rue de l'Hôtel-de-Ville → "Rue de"
Avenue du 11-Novembre-1918 → "Avenue du 11"
Rue d'Amsterdam → "d'Amsterdam" (dropped the affix!)
The through-line is French streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. morphology: <affix> <particle> <name>. SpansspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree. die at the
particle. Which is telling, because street_prefix_particle is already a tag in our schema — the
schema anticipated the structure; flat BIO can't execute it.
4. What we built and what it measured
Phase 1 — the span head
Additive-biaffine spanspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree. scorer over start/end projections → per-type scores for every spanspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree. ≤8 tokenstokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words.; segmentsegmentA punctuation-bounded chunk of the normalized input — the comma-separated parts of 'Portland, OR' — used to give downstream stages structural context. transition table; fp32fp32 / fp1632-bit and 16-bit floating-point formats. Mailwoman trains in bf16 (a 16-bit variant) and exports the ONNX model in int8 for size. semi-Markov CRFCRF (Conditional Random Field). A statistical modeling method that predicts structured outputs by modeling dependencies between adjacent labels. Mailwoman uses a linear-chain CRF as the Viterbi decoder at inference time to enforce BIO label consistency — a B-street must be followed by I-street or O, never I-locality. lossloss functionA number measuring how wrong the model's predictions are on a batch of examples. Training minimizes it. Mailwoman's loss combines per-token negative log-likelihood with the CRF sequence loss. co-trained alongside the untouched BIO headattention headOne of several parallel attention computations in a layer, each free to focus on a different kind of relationship between tokens. Their outputs are concatenated — 'multi-head attention'. Mailwoman uses 4 heads.. Both DP routines (log-partition, ViterbiViterbi decodingA dynamic programming algorithm that finds the most likely sequence of hidden states (labels) given a sequence of observations (token emissions). Mailwoman uses Viterbi over a linear-chain CRF to produce globally coherent BIO label sequences from per-token model scores.) verified against brute-force enumeration of every valid segmentation — a DP that's subtly wrong still trains, it just trains toward the wrong thing.
The pre-registered gate: seg@1 > token@1, plus a secondary read (oracle@10 must rise).
First probe failed and we nearly called it falsified. Lossloss functionA number measuring how wrong the model's predictions are on a batch of examples. Training minimizes it. Mailwoman's loss combines per-token negative log-likelihood with the CRF sequence loss. 26.4 → 17.77, still falling; decode emitted a random type per tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words.; seg@1 = 0.004. The diagnostic that saved it: "is the lossloss functionA number measuring how wrong the model's predictions are on a batch of examples. Training minimizes it. Mailwoman's loss combines per-token negative log-likelihood with the CRF sequence loss. even decreasing?" It was, and nowhere near converged (raw spanspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree. NLLNLL (Negative Log Likelihood). The standard form of a loss function in probability-based models: loss = −log P(correct label | input). The CRF uses the NLL of the entire label sequence. ~35 where a converged semi-CRFCRF (Conditional Random Field). A statistical modeling method that predicts structured outputs by modeling dependencies between adjacent labels. Mailwoman uses a linear-chain CRF as the Viterbi decoder at inference time to enforce BIO label consistency — a B-street must be followed by I-street or O, never I-locality. is O(1)).
The cause was ours. We inherited lr: 1e-5 from a recipe that fine-tunes existing weightsparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values.. The
spanspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree. headattention headOne of several parallel attention computations in a layer, each free to focus on a different kind of relationship between tokens. Their outputs are concatenated — 'multi-head attention'. Mailwoman uses 4 heads. is randomly initialized. A fresh headattention headOne of several parallel attention computations in a layer, each free to focus on a different kind of relationship between tokens. Their outputs are concatenated — 'multi-head attention'. Mailwoman uses 4 heads. cannot train at a pretrained encoderencoderThe part of a transformer that turns input tokens into contextualized vector representations. Mailwoman's classifier is a small encoder-only transformer (~30M parameters).'s fine-tuning LR.
One variable — a param-group LR of 1e-3 for the headattention headOne of several parallel attention computations in a layer, each free to focus on a different kind of relationship between tokens. Their outputs are concatenated — 'multi-head attention'. Mailwoman uses 4 heads. — and lossloss functionA number measuring how wrong the model's predictions are on a batch of examples. Training minimizes it. Mailwoman's loss combines per-token negative log-likelihood with the CRF sequence loss. converged to 1.37 (raw NLLNLL (Negative Log Likelihood). The standard form of a loss function in probability-based models: loss = −log P(correct label | input). The CRF uses the NLL of the entire label sequence. ~2.7).
The headline, and its correction
We first reported +7.9pp. That was measured in a Python harness that feeds none of the production soft channels (postcode anchoranchor inferenceA technique where structured knowledge (postcode locations, gazetteer place names) is injected into the model as soft input features — not as deterministic overrides. The model still decides the final labels, but the anchor signal biases it toward correct admin tags. / gazetteergazetteerA geographical index that maps place names and postcodes to real-world coordinates. Mailwoman uses a custom-built Who's On First (WOF) SQLite database as its gazetteer — the 'atlas' half of the grammar/atlas architecture. / countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. lexicon). With channels fed:
| triaged parity (n=267) | v264, summed-BIO spansspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree. | v301, learned spansspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree. |
|---|---|---|
| tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words.@1 | 0.573 | 0.5693 |
| seg@1 | 0.453 | 0.5768 |
| oracle@5 | 0.663 | 0.7228 |
| oracle@10 | 0.749 | 0.7753 |
The margin over the tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. decode is +0.75pp — two fixtures, inside noise. Much of the original +7.9pp was the BIO headattention headOne of several parallel attention computations in a layer, each free to focus on a different kind of relationship between tokens. Their outputs are concatenated — 'multi-head attention'. Mailwoman uses 4 heads.'s starvation, not the spanspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree. headattention headOne of several parallel attention computations in a layer, each free to focus on a different kind of relationship between tokens. Their outputs are concatenated — 'multi-head attention'. Mailwoman uses 4 heads.'s strength. Feed the channels and BIO recovers most of it.
What survives, and matters:
- The trained scorer beats the summed-BIO stand-in by +12.4pp (0.453 → 0.5768), same instrument. That was the actual falsifier and it holds decisively.
- The secondary read passed: oracle@10 rose 0.749 → 0.775, oracle@5 0.663 → 0.723. The list improved — the config had pre-registered "if seg@1 crosses but oracle@10 is flat, the scorer reshuffled without learning."
- On the target class it is not close. Paris fixture:
token@1byte-identical to v264 (33/63),seg@148/63 — +23.8pp, oracle@5 0.905.
And the archetype is fixed:
Korunní 810, Praha → Korunni:street 810:house_number Praha:locality
Phases 2–3 — export and decode
span_scores as a named ONNXONNX (Open Neural Network Exchange). An open format for machine learning models that enables interoperability between training frameworks and inference runtimes. Mailwoman ships its trained model as an ONNX file so it can run in Node.js and the browser via onnxruntime. output (fetched by name, so a runtime that ignores it prunes the
branch); the transition table as a JSON sidecar with the segmentsegmentA punctuation-bounded chunk of the normalized input — the comma-separated parts of 'Portland, OR' — used to give downstream stages structural context.-type axis in the file (never
hardcoded); k-best decode in JS, brute-force verified, shared by node and browser.
Costs, measured on the runtime that ships (onnxruntime-web WASM EP, not the node bench):
| ONNXONNX (Open Neural Network Exchange). An open format for machine learning models that enables interoperability between training frameworks and inference runtimes. Mailwoman ships its trained model as an ONNX file so it can run in Node.js and the browser via onnxruntime.↔torch spanspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree. parity | worst 7.3e-06 |
| int8 size | +0.22 MB (+0.57%) |
| browser latency, spansspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree. read every inferenceinferenceRunning the trained model on new input to get predictions, as opposed to training, which produces the model. In Mailwoman that means a small transformer encoder reads an address string and classifies every token — house number, street, locality, region, postcode, and the rest. A Who's On First gazetteer can feed soft location hints into the pass, but the model makes the final call on every label. Where a generative model writes text token by token, Mailwoman's output is a retrieval-augmented token classification: one label per input piece. | +0.08 ms (+0.5%) |
Phase 4a — the rerank, and where the thesis broke
The plan: the resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon. arbitrates the k-best list. A rank-2 parseaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates. that resolves to a real place beats a rank-1 that resolves to a countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. centroid. Evidence-based, no hand-weightsparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. — explicitly avoiding the PeliasPeliasAn open-source geocoder, Mailwoman's spiritual predecessor. failure mode (see §6).
Result: null, with a mechanism.
| parity (267) | Paris (63) | |
|---|---|---|
| seg@1 | 0.5768 | 0.7619 |
| rerank@1 | 0.5768 (+0) | 0.7460 (−1) |
The circularity: the arc targets bare fragments. A bare fragment has no house numberhouse numberThe numeric or alphanumeric identifier of a building on a street. Mailwoman's house_number component; its position relative to the street name flips between locales. (the address-point tiertierInternal versioning of which label classes the model emits. Tier 1 is the coarse components (country, region, locality, postcode); Tier 2 adds venue, street, house_number; Tier 3 (future) would add attention, po_box, and POI venue subtyping. Historically called 'Stage 1/2/3' before the runtime-pipeline naming made that ambiguous. is keyed by number) and no localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. (the streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.-centroid tiertierInternal versioning of which label classes the model emits. Tier 1 is the coarse components (country, region, locality, postcode); Tier 2 adds venue, street, house_number; Tier 3 (future) would add attention, po_box, and POI venue subtyping. Historically called 'Stage 1/2/3' before the runtime-pipeline naming made that ambiguous. has nothing to scope by). So every hypothesis resolves to the same admin centroid. Evidence rate — fixtures where any hypothesis reaches streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. tiertierInternal versioning of which label classes the model emits. Tier 1 is the coarse components (country, region, locality, postcode); Tier 2 adds venue, street, house_number; Tier 3 (future) would add attention, po_box, and POI venue subtyping. Historically called 'Stage 1/2/3' before the runtime-pipeline naming made that ambiguous.:
| US | FR | NL | DE | ZZ/NZ/AU/NO/PT/PL/RO |
|---|---|---|---|---|
| 0.01 | 0.03 | 0.22 | 0.23 | 0.00 |
On the Paris fixture with the FR streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.-centroid DB loaded: 314 hypothesis geocodes → admin 314, streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. 1.
It is not a noisy signal. There is no signal. Resolution evidence can only adjudicate addresses that are already resolvable — which are the ones the parseaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates. mostly gets right anyway.
5. Where it stands
The floors are unmoved. streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. 0.90 vs seg@1 0.577. oracle@5 is 0.723 — so even a perfect reranker lands short of the gate. The decode was never going to clear it alone. This is the fact that phasephaseA milestone in the implementation plan (Foundation, Corpus, Training, Integration, and forward-looking phases). Distinct from stage (runtime pipeline) and tier (model vocabulary). momentum most wants to obscure.
What's proven: a trained spanspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree. scorer beats the BIO stand-in decisively (+12.4pp), fixes the boundary class including the archetype, adds +24pp on bare fragments, and costs the browser +0.5% latency / +0.6% download.
What's disproven: "the arbiter is the resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon.." For the class that needs arbitrationarbitrationA pipeline stage that compares rule-based (v0) and neural classifier output, resolving disagreements via a policy registry. Built and merged but not promoted — the coordinate gate showed label-F1 gains came at the cost of worse geocoding., the resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon. is blind.
What's unresolved: ~15pp of measured headroom (oracle@5 0.723 vs shipped 0.573) sitting in the list with no arbiter to collect it.
The direction we think is right, and haven't tested
The resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon. answers "where is this?" The question a bare fragment needs is "is this a streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. name
at all?" — an existence check against the BANBAN (Base Adresse Nationale). France's authoritative open national address register — the highest-quality training source for French addresses, with full component structure./gazetteergazetteerA geographical index that maps place names and postcodes to real-world coordinates. Mailwoman uses a custom-built Who's On First (WOF) SQLite database as its gazetteer — the 'atlas' half of the grammar/atlas architecture. name index. No localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy., no house numberhouse numberThe numeric or alphanumeric identifier of a building on a street. Mailwoman's house_number component; its position relative to the street name flips between locales.
required, so it sidesteps the circularity entirely. ban/street-centroids-fr.db contains every French
streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. name; asking whether Rue de Rome is in it is a lexicon lookup, not a geocode.
Two signals have already failed (plausibility veto: inert; resolution specificity: −16, because
preferring "finer" among country / region / locality rewards the localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.-reading failure mode). A
third goes to review before it goes to a branch.
The other half nobody's built yet
The spanspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree. headattention headOne of several parallel attention computations in a layer, each free to focus on a different kind of relationship between tokens. Their outputs are concatenated — 'multi-head attention'. Mailwoman uses 4 heads. provably does not fix the 17 localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.-refusals — that was never its job. That's
option C: feed the kind-classifier posterior as a soft channel (established plumbing — the postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one.
anchor and countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. lexicon work this way) plus a recallrecallOf the spans whose gold label is a given tag, the fraction the model found. High recall means few misses. Paired with precision to compute F1.-weighted streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. lossloss functionA number measuring how wrong the model's predictions are on a batch of examples. Training minimizes it. Mailwoman's loss combines per-token negative log-likelihood with the CRF sequence loss.. The kind posterior is a
synthetic anchor standing in for the missing house numberhouse numberThe numeric or alphanumeric identifier of a building on a street. Mailwoman's house_number component; its position relative to the street name flips between locales..
Explicitly rejected, on review: a hard "must emit streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels." decode mask (kind-classifier errors become hallucinations — and the uncertain cases are the failing ones), and injecting a forced hypothesis into the k-best list (its score comes from a different normalization; incomparable scores in one list is the PeliasPeliasAn open-source geocoder, Mailwoman's spiritual predecessor.-blend antipattern in miniature).
6. Context: what the dictionary approach costs
We read PeliasPeliasAn open-source geocoder, Mailwoman's spiritual predecessor.'s vendored dictionary layerlayerOne transformer block — attention plus a feed-forward network, with normalization and residual connections — applied to every position. Stacking layers lets the model build up richer representations; Mailwoman's encoder has 6. (parser/resources/pelias/dictionaries). 94 of 276
non-comment lines are deletions — !token meaning "libpostallibpostalAn open-source C address parser used by Pelias. Mailwoman's rule-based v0 and neural classifier supersede it. says this is a streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. type; we say
no." The comments are the stories:
# this Italian contracted form of Androna causes issues in English → !and (Trieste loses)
# conflicts with US state abbreviation → !ca (Spain loses to California)
# Causes a bunch of issues with Spanish addresses ("Calle" ...) → !alle (Norway loses to Spain)
# 18 person county in texas → !art (Art, TX pop. 18, loses)
# remove any localities which share a name with a US state → !alabama
#!new york ← commented out. They tried.
#!washington ← same.
Nearly every override is a rare-but-real meaning colliding with a common one — ca, ch, ga, ma,
in, a, art — resolved by globally deleting the rare side. This is correct EV given a
dictionary: "art" appears in millions of queries; Art, Texas has eighteen residents. Nobody was
careless. It becomes scar tissue only in one narrow sense: it's irreversible and context-free, and it
can never be otherwise, because a lookup has no context to consult.
The commented-out #!new york / #!washington is the tell: a blunt global rule, then a hand-kept
exception list for the entries too famous to lose.
Our receipts, including the one that hurts:
Art, TX → {region:TX, locality:Art} ✓
Alabama, NY → {region:NY, locality:Alabama} ✓
Calle Mayor 5, Madrid → street:"Calle Mayor" ✓
Italy, TX → {country:Italy, region:TX} ✗ ← we fail this too
4/5, and the miss is the instructive part. Italy, TX is the exact row PeliasPeliasAn open-source geocoder, Mailwoman's spiritual predecessor. hand-deleted; we get it
wrong differently. So the claim isn't "we win" — it's narrower: holding an ambiguity isn't resolving
it. We get to be wrong recoverably — Italy, TX is the countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.-channel homographhomonymyOne surface, many referents: 'Georgia' is a US state in 'Atlanta, Georgia' and a country in 'Tbilisi, Georgia.' Handled by disambiguation, split across two stages — the parser resolves the tag from in-string context; the resolver late-binds the referent with geographic context. class, whose
knob is country_ambiguous_scale, which v263 set hard, over-suppressed, and v264 relaxed to 0.5 and
measured (homographhomonymyOne surface, many referents: 'Georgia' is a US state in 'Atlanta, Georgia' and a country in 'Tbilisi, Georgia.' Handled by disambiguation, split across two stages — the parser resolves the tag from in-string context; the resolver late-binds the referent with geographic context. F1 82.6 → 85.1, no trade). PeliasPeliasAn open-source geocoder, Mailwoman's spiritual predecessor.'s equivalent knob is a line you delete.
7. Methodology, since two results nearly went out wrong
Both near-misses had the same signature: a number that couldn't reproduce something already known.
- PhasephaseA milestone in the implementation plan (Foundation, Corpus, Training, Integration, and forward-looking phases). Distinct from stage (runtime pipeline) and tier (model vocabulary). 1's gate reported
token@10.348 against a known 0.573. Cause: a Python harness feeding no soft channels, plus a bug welding words together (▁5|th|▁|Ave→"5thAve"— it dropped theO-labelled separator). Nearly published as an architecture verdict. - PhasephaseA milestone in the implementation plan (Foundation, Corpus, Training, Integration, and forward-looking phases). Distinct from stage (runtime pipeline) and tier (model vocabulary). 4a's first rerank showed one signal inert and another −16. Cause: the harness resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon. had no streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. shardsshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. and reached streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. tiertierInternal versioning of which label classes the model emits. Tier 1 is the coarse components (country, region, locality, postcode); Tier 2 adds venue, street, house_number; Tier 3 (future) would add attention, po_box, and POI venue subtyping. Historically called 'Stage 1/2/3' before the runtime-pipeline naming made that ambiguous. 0 times in 267 fixtures. Nearly published as "resolution evidence can't work."
Both times the tell was there before the conclusion. Check the instrument before the hypothesis.
Also worth flagging for a reviewer: we found from_pretrained() never passed map_location, so a
GPU-trained checkpointcheckpointA saved snapshot of the model weights and optimizer state during training. Mailwoman saves a checkpoint periodically so training can resume after a GPU hang. couldn't load on a CPU box at all — that had been silently blockingblockingThe first stage of entity resolution: generate candidate record pairs using cheap, high-recall keys (geo cell, canonical address, phone) instead of comparing every record to every other (O(n²)). The matcher only scores pairs that survive blocking. every
local grading run.
8. What we'd want a reviewer to push on
Answered. All four went to review and came back adjudicated — see
2026-07-16-span-head-arc-review-follow-up.mdfor the verdicts and the converged plan. Kept here as asked, not as open.
- Is +0.75pp at rank-1 enough to justify the decode? We say no, and that the case is the list. Is that motivated reasoning about a phasephaseA milestone in the implementation plan (Foundation, Corpus, Training, Integration, and forward-looking phases). Distinct from stage (runtime pipeline) and tier (model vocabulary). we already built?
- Is the name-existence check the right third signal, or is the rerank a dead end and option C the whole answer?
- The floor is 0.90 and oracle@5 is 0.723. Is the parseaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates.-tag floor even the right gate, or is a coordinate-acceptability gate the right one? (A prior study found the neural parseaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates. is 98.6% within 1 km of the rules parseaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates. where the streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. is correct, with a hard tail on exactly the bare-fragment class.)
- Is a 63-fixture Paris corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus. enough to make per-class claims, or are we reading noise?