Training recipe levers
Every trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input. run is defined by a YAML config with ~30 knobs. Most interact. This document catalogues each lever, what it does, how we arrived at the current value, what happens when you move it, and where we've found ceilings. The goal is to make the reasoning behind the recipe reproducible — not just the recipe itself.
Written after v0.5.0 (CE-only, h256, 50K steps) and v0.5.1 ("unchained," h384, 100K steps). Both trained on corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus.-v0.4.0 with the A1 tokenizertokenizerThe component that converts a raw address string into a sequence of numeric token IDs the model can process. Mailwoman's tokenizer is a SentencePiece unigram model trained specifically on postal addresses..
Architecture levers
hidden_size
What it does. The width of every hidden representation in the transformerneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' encoderencoderThe part of a transformer that turns input tokens into contextualized vector representations. Mailwoman's classifier is a small encoder-only transformer (~30M parameters).. Every tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. is a vector of this many numbers. Wider = more capacity to represent distinctions between address components.
| Value | Where we used it | What happened |
|---|---|---|
| 256 | v0.3.0, v0.4.0, v0.5.0 | Stable. val_macro_f1=0.605 at 50K. The baseline. |
| 384 | v0.5.0 bisect (h384 + dual-lossloss functionA number measuring how wrong the model's predictions are on a batch of examples. Training minimizes it. Mailwoman's loss combines per-token negative log-likelihood with the CRF sequence loss.) | Diverged at step 700-1050 in all attempts. |
| 384 | v0.5.1 (h384 + CE-only) | Stable. val_macro_f1=0.633 at 55K. Peak before overfittingoverfittingWhen a model memorizes quirks of its training set instead of learning general patterns, so it scores well in training but poorly on new data. Guarded against with held-out evals and regularization.. |
How we got to 384. The operator chose it as the v0.5.0 target. The plan doc described 256→384 as "paid for by rented GPU." The h384 bisect during the divergencedivergenceA training failure where loss descends through warmup, plateaus low for a while, then climbs catastrophically back to its starting magnitude — the model unlearns everything despite no obvious component failure. campaign appeared to show h384 was a destabilizer — but it was actually the dual-lossloss functionA number measuring how wrong the model's predictions are on a batch of examples. Training minimizes it. Mailwoman's loss combines per-token negative log-likelihood with the CRF sequence loss. interaction that was unstable at every size. Once CE-only fixed the trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input., h384 worked on the first try.
Ceiling. Not found. 512 is the next natural step (doubles the FFNfeed-forward networkThe per-token transformation inside each transformer layer that follows attention: a small two-layer network applied independently at every position. intermediate from 1536 to 2048). Expected cost: ~2× wall time per step, ~4× parameterparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. count (~68M vs 29M at h384). Whether the additional capacity helps depends on whether the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' is capacity-limited or data-limited. The v0.5.1 overfittingoverfittingWhen a model memorizes quirks of its training set instead of learning general patterns, so it scores well in training but poorly on new data. Guarded against with held-out evals and regularization. past step 65K suggests data-limited at h384 — going larger without more data would overfit faster.
What 256 buys you. ~17M params, trains at 6.9 sps on A100 (h256 batchbatch sizeHow many examples the model processes before each parameter update. Larger batches give smoother gradients but cost more memory; gradient accumulation simulates a big batch on a small GPU.=16 ga=8) or ~120 sps (h256 batchbatch sizeHow many examples the model processes before each parameter update. Larger batches give smoother gradients but cost more memory; gradient accumulation simulates a big batch on a small GPU.=128 direct — untested but estimated). Fits on any GPU including consumer iGPUs.
num_attention_heads
What it does. How many independent "perspectives" each transformerneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' layerlayerOne transformer block — attention plus a feed-forward network, with normalization and residual connections — applied to every position. Stacking layers lets the model build up richer representations; Mailwoman's encoder has 6. uses when deciding which tokenstokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. attend to which. Convention: head_dim = hidden_size / num_headsattention headOne of several parallel attention computations in a layer, each free to focus on a different kind of relationship between tokens. Their outputs are concatenated — 'multi-head attention'. Mailwoman uses 4 heads. = 64.
| hidden_size | num_headsattention headOne of several parallel attention computations in a layer, each free to focus on a different kind of relationship between tokens. Their outputs are concatenated — 'multi-head attention'. Mailwoman uses 4 heads. | head_dim |
|---|---|---|
| 256 | 4 | 64 |
| 384 | 6 | 64 |
How we got here. Following the standard 64-dim-per-headattention headOne of several parallel attention computations in a layer, each free to focus on a different kind of relationship between tokens. Their outputs are concatenated — 'multi-head attention'. Mailwoman uses 4 heads. convention from BERT. No experimentation on headattention headOne of several parallel attention computations in a layer, each free to focus on a different kind of relationship between tokens. Their outputs are concatenated — 'multi-head attention'. Mailwoman uses 4 heads. count — it's always derived from hidden_size.
num_hidden_layers
What it does. How many transformer blockslayerOne transformer block — attention plus a feed-forward network, with normalization and residual connections — applied to every position. Stacking layers lets the model build up richer representations; Mailwoman's encoder has 6. the input passes through. More layerslayerOne transformer block — attention plus a feed-forward network, with normalization and residual connections — applied to every position. Stacking layers lets the model build up richer representations; Mailwoman's encoder has 6. = more sequential processing but slower.
Current value: 6. Unchanged since v0.1.0. The modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' is intentionally shallow — address parsingaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates. doesn't require the deep reasoning chains that language generation does. Most of the useful signal is local (neighboring tokenstokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. in a comma-separated address).
Ceiling. Not explored. 6 layerslayerOne transformer block — attention plus a feed-forward network, with normalization and residual connections — applied to every position. Stacking layers lets the model build up richer representations; Mailwoman's encoder has 6. at h384 is ~29M params. 12 layerslayerOne transformer block — attention plus a feed-forward network, with normalization and residual connections — applied to every position. Stacking layers lets the model build up richer representations; Mailwoman's encoder has 6. would be ~50M. The overfittingoverfittingWhen a model memorizes quirks of its training set instead of learning general patterns, so it scores well in training but poorly on new data. Guarded against with held-out evals and regularization. at step 65K suggests the current depth is sufficient for the data; more layerslayerOne transformer block — attention plus a feed-forward network, with normalization and residual connections — applied to every position. Stacking layers lets the model build up richer representations; Mailwoman's encoder has 6. without more data would overfit faster.
use_phrase_priors
What it does. When enabled, the encoderencoderThe part of a transformer that turns input tokens into contextualized vector representations. Mailwoman's classifier is a small encoder-only transformer (~30M parameters). concatenates a per-tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. featurefeatureAn input signal a model conditions on. Beyond the raw tokens, Mailwoman feeds soft features — gazetteer-membership channels and the postcode anchor — that inform predictions without overriding them. vector from the phrase grouperphrase grouperStage 2.7 of the runtime pipeline: proposes coherent input units (street phrase, locality phrase, postcode, etc.) with structural kind hypotheses. Decouples boundary discovery from type classification so the classifier answers 'what type?' not 'where?' (StagestageOne of the dataflow stages in the runtime pipeline (normalize, locale gate, kind classify, phrase group, token classify, sequence correct, reconcile, resolve). Distinct from tier (model vocabulary) and phase (plan milestone). 2.7) onto the tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. embeddingsembeddingA vector of numbers representing a token (or other item) so that similar items sit near each other in vector space. The first thing the model does is turn each token into an embedding. before the first transformerneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' layerlayerOne transformer block — attention plus a feed-forward network, with normalization and residual connections — applied to every position. Stacking layers lets the model build up richer representations; Mailwoman's encoder has 6.. The featurefeatureAn input signal a model conditions on. Beyond the raw tokens, Mailwoman feeds soft features — gazetteer-membership channels and the postcode anchor — that inform predictions without overriding them. encodes "is this tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. at the start/middle/end of a proposed phrase?" and "what kind of phrase does the grouper think this is?" (numeric, streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels., localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy., etc.).
| Value | Where | What happened |
|---|---|---|
| false | v0.5.0, v0.5.0 bisect (phrase-off) | Diverged (dual-lossloss functionA number measuring how wrong the model's predictions are on a batch of examples. Training minimizes it. Mailwoman's loss combines per-token negative log-likelihood with the CRF sequence loss. issue, not phrase-priors). |
| true | v0.5.1 (h384 + CE-only) | Stable. val_macro_f1=0.633 at 55K. |
How we got here. Phrase priors were the headline architectural contribution of v0.5.0 Thread E (phrase grouperphrase grouperStage 2.7 of the runtime pipeline: proposes coherent input units (street phrase, locality phrase, postcode, etc.) with structural kind hypotheses. Decouples boundary discovery from type classification so the classifier answers 'what type?' not 'where?') + Thread C (classifier conditioning). They were turned off during the bisect campaign because we couldn't afford to test each variable on slow hardware. The v0.5.1 "unchained" run turned them back on. They're stable under CE-only.
What they're supposed to do. Give the classifier a headattention headOne of several parallel attention computations in a layer, each free to focus on a different kind of relationship between tokens. Their outputs are concatenated — 'multi-head attention'. Mailwoman uses 4 heads. start on boundary discovery. Instead of the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' having to learn "these tokenstokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. belong together" purely from BIO labelcomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag. statistics, the phrase grouperphrase grouperStage 2.7 of the runtime pipeline: proposes coherent input units (street phrase, locality phrase, postcode, etc.) with structural kind hypotheses. Decouples boundary discovery from type classification so the classifier answers 'what type?' not 'where?''s structural cues (punctuation, capitalization, hyphenation) provide a prior. The modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' can then focus on "what type is this spanspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree.?" rather than jointly discovering boundaries and types.
Whether they helped. Unclear from the current data — we haven't run h384 + CE-only WITHOUT phrase priors on A100 to isolate the contribution. The 0.633 vs 0.621 delta between v0.5.1 and v0.5.0 conflates h384 + phrase priors + class weightsparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. + direct batchbatch sizeHow many examples the model processes before each parameter update. Larger batches give smoother gradients but cost more memory; gradient accumulation simulates a big batch on a small GPU. + more steps. An ablation (h384 + CE-only + phrase priors OFF) would isolate it.
Loss levers
crf_loss_weight
What it does. Multiplier on the CRFCRF (Conditional Random Field). A statistical modeling method that predicts structured outputs by modeling dependencies between adjacent labels. Mailwoman uses a linear-chain CRF as the Viterbi decoder at inference time to enforce BIO label consistency — a B-street must be followed by I-street or O, never I-locality. NLLNLL (Negative Log Likelihood). The standard form of a loss function in probability-based models: loss = −log P(correct label | input). The CRF uses the NLL of the entire label sequence. term in the dual-lossloss functionA number measuring how wrong the model's predictions are on a batch of examples. Training minimizes it. Mailwoman's loss combines per-token negative log-likelihood with the CRF sequence loss.: loss = CE + crf_loss_weight × CRF_NLL. At 0.0, the CRFCRF (Conditional Random Field). A statistical modeling method that predicts structured outputs by modeling dependencies between adjacent labels. Mailwoman uses a linear-chain CRF as the Viterbi decoder at inference time to enforce BIO label consistency — a B-street must be followed by I-street or O, never I-locality. NLLNLL (Negative Log Likelihood). The standard form of a loss function in probability-based models: loss = −log P(correct label | input). The CRF uses the NLL of the entire label sequence. is not computed during trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input. (CE-only). The CRFCRF (Conditional Random Field). A statistical modeling method that predicts structured outputs by modeling dependencies between adjacent labels. Mailwoman uses a linear-chain CRF as the Viterbi decoder at inference time to enforce BIO label consistency — a B-street must be followed by I-street or O, never I-locality. at inferenceinferenceRunning the trained model on new input to get predictions, as opposed to training, which produces the model. In Mailwoman that means a small transformer encoder reads an address string and classifies every token — house number, street, locality, region, postcode, and the rest. A Who's On First gazetteer can feed soft location hints into the pass, but the model makes the final call on every label. Where a generative model writes text token by token, Mailwoman's output is a retrieval-augmented token classification: one label per input piece. (structural BIO mask + ViterbiViterbi decodingA dynamic programming algorithm that finds the most likely sequence of hidden states (labels) given a sequence of observations (token emissions). Mailwoman uses Viterbi over a linear-chain CRF to produce globally coherent BIO label sequences from per-token model scores. decode) is always active regardless.
| Value | Where | What happened |
|---|---|---|
| 1.0 | v0.5.0 threadsthreadA parallel workstream within a release. Threads compose; they are not sequential milestones like phases. v1-v2 (§1 ON) | Diverged step 700-1000. |
| 0.05 | v0.3.0, v0.4.0, v0.5.0 threadsthreadA parallel workstream within a release. Threads compose; they are not sequential milestones like phases. v3 + bisects | v0.3.0/v0.4.0: stable. v0.5.0: diverged. |
| 0.0 | v0.5.0 CE-only, v0.5.1 | Stable. Best results. |
How we got to 0.0. The gradientgradientThe direction and rate at which the loss would change if each parameter were nudged. Training follows the gradient downhill to reduce error. Huge gradients are tamed by gradient clipping.-norm ratio probe (2026-05-24) revealed CRFCRF (Conditional Random Field). A statistical modeling method that predicts structured outputs by modeling dependencies between adjacent labels. Mailwoman uses a linear-chain CRF as the Viterbi decoder at inference time to enforce BIO label consistency — a B-street must be followed by I-street or O, never I-locality. gradientgradientThe direction and rate at which the loss would change if each parameter were nudged. Training follows the gradient downhill to reduce error. Huge gradients are tamed by gradient clipping. dominates CE by 8-20× at the divergencedivergenceA training failure where loss descends through warmup, plateaus low for a while, then climbs catastrophically back to its starting magnitude — the model unlearns everything despite no obvious component failure. inflection point. Even at weightparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values.=0.05, the effective optimization mix was ~1:0.8 CE:CRF. Below lossloss functionA number measuring how wrong the model's predictions are on a batch of examples. Training minimizes it. Mailwoman's loss combines per-token negative log-likelihood with the CRF sequence loss. 0.41, CE and CRFCRF (Conditional Random Field). A statistical modeling method that predicts structured outputs by modeling dependencies between adjacent labels. Mailwoman uses a linear-chain CRF as the Viterbi decoder at inference time to enforce BIO label consistency — a B-street must be followed by I-street or O, never I-locality. develop opposing curvature — the CRFCRF (Conditional Random Field). A statistical modeling method that predicts structured outputs by modeling dependencies between adjacent labels. Mailwoman uses a linear-chain CRF as the Viterbi decoder at inference time to enforce BIO label consistency — a B-street must be followed by I-street or O, never I-locality. pulls the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' off its CE-preferred basin. See dual-loss curvature conflict.
What 0.05 did. Shipped v0.3.0 and v0.4.0 successfully at h256 with those corpora. Failed at h384 and with the v0.4.0 corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus.. The destabilization is recipe-dependent, not a universal property of 0.05 — it interacted with the larger modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' and/or the new corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus..
Ceiling. 0.0 is the floor. There is no known ceiling because any value > 0 reintroduces the curvature conflict on this data. The structural CRFCRF (Conditional Random Field). A statistical modeling method that predicts structured outputs by modeling dependencies between adjacent labels. Mailwoman uses a linear-chain CRF as the Viterbi decoder at inference time to enforce BIO label consistency — a B-street must be followed by I-street or O, never I-locality. benefits (no orphan I-tags) are preserved at inferenceinferenceRunning the trained model on new input to get predictions, as opposed to training, which produces the model. In Mailwoman that means a small transformer encoder reads an address string and classifies every token — house number, street, locality, region, postcode, and the rest. A Who's On First gazetteer can feed soft location hints into the pass, but the model makes the final call on every label. Where a generative model writes text token by token, Mailwoman's output is a retrieval-augmented token classification: one label per input piece. via the frozen mask. The trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input.-side CRFCRF (Conditional Random Field). A statistical modeling method that predicts structured outputs by modeling dependencies between adjacent labels. Mailwoman uses a linear-chain CRF as the Viterbi decoder at inference time to enforce BIO label consistency — a B-street must be followed by I-street or O, never I-locality. NLLNLL (Negative Log Likelihood). The standard form of a loss function in probability-based models: loss = −log P(correct label | input). The CRF uses the NLL of the entire label sequence. is unnecessary.
class_weights
What it does. Per-tag multiplier on the cross-entropy losscross-entropyThe standard classification loss: it penalizes a model for putting low probability on the correct label. Per-token negative log-likelihood is the cross-entropy of each token's label.. Tags with weightparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. > 1.0 cost more when mislabeled; tags with weightparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. < 1.0 cost less. Used to steer the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.''s attentionattentionThe core mechanism inside a transformer encoder. Each token's representation is updated by looking at every other token, with learned weights deciding how much each one matters. toward underperforming or hallucination-prone tags.
| Tag | v0.5.0 weightparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. | v0.5.1 weightparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. | Rationale |
|---|---|---|---|
| O | 1.0 | 0.5 | O is the majority class (~60% of tokenstokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words.). Downweight to give real tags more gradientgradientThe direction and rate at which the loss would change if each parameter were nudged. Training follows the gradient downhill to reduce error. Huge gradients are tamed by gradient clipping. signal. |
| B/I-streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. | 0.5 | 2.0 | v0.5.0 had streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. F1=0.1% on golden. Upweight to force the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' to learn streetsstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.. |
| B/I-localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. | 1.5 | 2.0 | Coarse labelfine labelThe Tier 2 labels added to the model vocabulary — venue, street, house_number — as opposed to the coarse labels (country, region, locality, postcode). regression from v0.3.0. Upweight. |
| B/I-countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. | 2.0 | 2.0 | Carried from v0.4.0. |
| B/I-dependent_locality | 1.5 | 0.3 | 956 FPs in v0.5.0 evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error.. Modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' hallucinates this tag. Downweight to make hallucination cheap → modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' stops emitting it. |
| B/I-subregion | 1.5 | 0.3 | 272 FPs with 0 TPs. Same hallucination pattern. |
| B/I-venuevenueA named, non-address place — a business, building, park, or stadium. Mailwoman's free-text point-of-interest component, added as a Tier 2 fine label. | 0.5 | 1.5 | Underperforming. Upweight. |
| B/I-house_number | 0.5 | 1.5 | Underperforming on golden. Upweight. |
How we got here. The v0.5.0 evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. matrix showed the 956 dependent_locality FPs and 272 subregion FPs as the most concrete failure. Setting their weightparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. to 0.3 makes mislabeling a tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. as dependent_locality cheap — the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' has no incentive to emit it. Meanwhile upweighting streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels., localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy., venuevenueA named, non-address place — a business, building, park, or stadium. Mailwoman's free-text point-of-interest component, added as a Tier 2 fine label., and house_number pulls the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.''s attentionattentionThe core mechanism inside a transformer encoder. Each token's representation is updated by looking at every other token, with learned weights deciding how much each one matters. toward the tags that matter for exact-match accuracy.
Whether it worked. Pending — v0.5.1 trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input. is still running. The evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. at step 55K showed val_macro_f1=0.633 but we need the per-component breakdown to know if the hallucination was suppressed.
Risks. Setting dependent_locality too low (e.g. 0.1) could make the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' unable to learn it at all — if the data ever includes legitimate dependent_locality examples, the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' would ignore them. 0.3 is a compromise: low enough to suppress hallucination, high enough that a clear signal still trains.
label_smoothing
What it does. Instead of trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input. toward a hard 0/1 target per tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words., smooths the target to (1 - ε) for the correct class and ε/(K-1) for the others. Prevents overconfident predictions.
Current value: 0.0. Disabled in all v0.5.x runs. The 54.5% overconfident-wrong rate suggests we should turn it on — label smoothinglabel smoothingA training regularization technique: instead of putting 1.0 probability on the correct label, train to put 1 − ε on it and ε/(N−1) on each wrong label. Improves calibration; disabled in some Mailwoman runs for stability. directly addresses confidence calibrationconfidence calibrationThe process of adjusting model confidence scores so that '0.6' actually means the model is right about 60% of the time. Mailwoman uses isotonic regression (PAVA) to calibrate per-span confidences against held-out data. Applied opt-in via createCalibrator.. However, the reconciler already addresses the overconfidence problem at the pipelinestaged pipelineMailwoman's runtime architecture: a sequence of pure-function stages (normalize → query-shape → locale-gate → kind-classifier → phrase-grouper → classifier → decoder) connected by typed handoffs. Each stage is published as its own npm package. level (0.1% overconfident-wrong in hybrid-joint mode), so per-modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' calibration is lower priority.
Next experiment. label_smoothing=0.1 is the standard starting point. Expected effect: slightly lower peak macro_f1 (the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' hedges its predictions) but better-calibrated confidence scores.
Optimizer levers
learning_rate
What it does. How large a step the optimizeroptimizerThe component that decides how to update parameters from the gradient — Adam/AdamW being the common choice, adding momentum and per-parameter scaling on top of plain gradient descent. takes per gradientgradientThe direction and rate at which the loss would change if each parameter were nudged. Training follows the gradient downhill to reduce error. Huge gradients are tamed by gradient clipping. update. Too high = overshoots optima (divergencedivergenceA training failure where loss descends through warmup, plateaus low for a while, then climbs catastrophically back to its starting magnitude — the model unlearns everything despite no obvious component failure.). Too low = converges too slowly or gets stuck.
| Value | Where | What happened |
|---|---|---|
| 5e-4 | v0.4.0 runs 1-2 | Diverged step 750-1000. |
| 3e-4 | v0.4.0 run 3 | Diverged step 1000. |
| 1.5e-4 | v0.3.0, v0.4.0 §4-only, all v0.5.x | Stable everywhere under CE-only. |
| 1e-4 | v0.5.0 LR-drop attempt | Diverged (dual-lossloss functionA number measuring how wrong the model's predictions are on a batch of examples. Training minimizes it. Mailwoman's loss combines per-token negative log-likelihood with the CRF sequence loss., not LR). |
How we got to 1.5e-4. v0.3.0 empirically found it as the stable LR for this architecture + corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus.. v0.4.0 tried higher (5e-4, 3e-4) but the bisect showed divergencedivergenceA training failure where loss descends through warmup, plateaus low for a while, then climbs catastrophically back to its starting magnitude — the model unlearns everything despite no obvious component failure. was recipe-driven, not LR-driven. 1.5e-4 has been the constant since.
Ceiling. Unknown under CE-only. The dual-lossloss functionA number measuring how wrong the model's predictions are on a batch of examples. Training minimizes it. Mailwoman's loss combines per-token negative log-likelihood with the CRF sequence loss. divergencedivergenceA training failure where loss descends through warmup, plateaus low for a while, then climbs catastrophically back to its starting magnitude — the model unlearns everything despite no obvious component failure. at higher LRs was caused by the CRFCRF (Conditional Random Field). A statistical modeling method that predicts structured outputs by modeling dependencies between adjacent labels. Mailwoman uses a linear-chain CRF as the Viterbi decoder at inference time to enforce BIO label consistency — a B-street must be followed by I-street or O, never I-locality. curvature conflict, which is gone. Higher LR might now be stable. But 1.5e-4 is working well enough that there's no pressure to explore.
batch_size × grad_accum_steps (effective batch)
What it does. How many examples the optimizeroptimizerThe component that decides how to update parameters from the gradient — Adam/AdamW being the common choice, adding momentum and per-parameter scaling on top of plain gradient descent. sees before updating weightsparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values.. Larger = smoother gradientsgradientThe direction and rate at which the loss would change if each parameter were nudged. Training follows the gradient downhill to reduce error. Huge gradients are tamed by gradient clipping. (less noise), but each step covers more wall-clock time.
| Effective batchbatch sizeHow many examples the model processes before each parameter update. Larger batches give smoother gradients but cost more memory; gradient accumulation simulates a big batch on a small GPU. | How | Where | What happened |
|---|---|---|---|
| 128 | batchbatch sizeHow many examples the model processes before each parameter update. Larger batches give smoother gradients but cost more memory; gradient accumulation simulates a big batch on a small GPU.=32 × ga=4 | v0.3.0, v0.4.0 (local iGPU, h256) | Stable. |
| 128 | batchbatch sizeHow many examples the model processes before each parameter update. Larger batches give smoother gradients but cost more memory; gradient accumulation simulates a big batch on a small GPU.=16 × ga=8 | v0.5.0 (local iGPU, h256) | Stable. |
| 8 | batchbatch sizeHow many examples the model processes before each parameter update. Larger batches give smoother gradients but cost more memory; gradient accumulation simulates a big batch on a small GPU.=8 × ga=1 | v0.5.0 smoke testsverdict-smokeA short diagnostic training run (≈1,500–3,000 steps) that must match the full run's gradient-noise profile, used to catch recipe instability before committing to a full 50k-step launch. | Falsely passed — eff_batch=8 hides the curvature conflict. |
| 128 | batchbatch sizeHow many examples the model processes before each parameter update. Larger batches give smoother gradients but cost more memory; gradient accumulation simulates a big batch on a small GPU.=128 × ga=1 | v0.5.1 (A100, h384) | Stable. 15-30× faster per step. |
How we got to batchbatch sizeHow many examples the model processes before each parameter update. Larger batches give smoother gradients but cost more memory; gradient accumulation simulates a big batch on a small GPU.=128 direct. The A100 has 40 GB VRAM; the h384 modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' uses ~4 GB. No need for gradient accumulationgradient accumulationSumming gradients over several minibatches before taking an optimizer step, to simulate a larger effective batch size under limited GPU memory.. Direct batchbatch sizeHow many examples the model processes before each parameter update. Larger batches give smoother gradients but cost more memory; gradient accumulation simulates a big batch on a small GPU.=128 gives cleaner gradientsgradientThe direction and rate at which the loss would change if each parameter were nudged. Training follows the gradient downhill to reduce error. Huge gradients are tamed by gradient clipping. (no micro-batchbatch sizeHow many examples the model processes before each parameter update. Larger batches give smoother gradients but cost more memory; gradient accumulation simulates a big batch on a small GPU. noise) and much higher throughput (120 sps vs 6.9 sps).
The smoke-batchbatch sizeHow many examples the model processes before each parameter update. Larger batches give smoother gradients but cost more memory; gradient accumulation simulates a big batch on a small GPU. lesson. Smoke testsverdict-smokeA short diagnostic training run (≈1,500–3,000 steps) that must match the full run's gradient-noise profile, used to catch recipe instability before committing to a full 50k-step launch. at eff_batch=8 passed cleanly while full runs at eff_batch=128 diverged. At smaller batch sizesbatch sizeHow many examples the model processes before each parameter update. Larger batches give smoother gradients but cost more memory; gradient accumulation simulates a big batch on a small GPU., the higher per-step gradientgradientThe direction and rate at which the loss would change if each parameter were nudged. Training follows the gradient downhill to reduce error. Huge gradients are tamed by gradient clipping. noise paradoxically stabilizes trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input. against curvature conflicts (the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' can't settle deep enough into the basin where the conflict manifests). This is now documented in VERDICT_SMOKES.md.
warmup_steps
What it does. Number of steps over which the learning ratelearning rate (LR). How big a step training takes along the gradient each update. Too high and training diverges; too low and it crawls. Mailwoman warms it up, then decays it on a cosine schedule. ramps linearly from 0 to its target. Prevents large gradientgradientThe direction and rate at which the loss would change if each parameter were nudged. Training follows the gradient downhill to reduce error. Huge gradients are tamed by gradient clipping. updates on a randomly initialized modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.'.
| Value | Where | Notes |
|---|---|---|
| 500 | v0.5.0 (50K total) | 1% of trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input.. |
| 1000 | v0.5.1 (100K total) | 1% of trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input.. Proportional scaling. |
How we got here. Convention: 1% of total steps. No experimentation. Shorter warmupwarmupThe early phase of training where the learning rate ramps up from 0 to its peak value before cosine decay. Mailwoman uses a linear warmup. risks early instability; longer warmupwarmupThe early phase of training where the learning rate ramps up from 0 to its peak value before cosine decay. Mailwoman uses a linear warmup. wastes steps at low LR that could be learning.
lr_schedule
What it does. How the learning ratelearning rate (LR). How big a step training takes along the gradient each update. Too high and training diverges; too low and it crawls. Mailwoman warms it up, then decays it on a cosine schedule. changes after warmupwarmupThe early phase of training where the learning rate ramps up from 0 to its peak value before cosine decay. Mailwoman uses a linear warmup.. Cosine decayslearning-rate scheduleA plan for changing the learning rate over training — Mailwoman's is linear warmup to a peak, then cosine decay toward zero. Smooths early instability and lets the model settle at the end. to near-zero; constant stays at peak.
| Value | Where | What happened |
|---|---|---|
| cosine | v0.3.0, v0.4.0 | Standard. But masked divergencedivergenceA training failure where loss descends through warmup, plateaus low for a while, then climbs catastrophically back to its starting magnitude — the model unlearns everything despite no obvious component failure. in v0.4.0 smokes. |
| constant | v0.5.0, v0.5.1 | Used per VERDICT_SMOKES.md mode A — new recipe = constant LR. |
How we got to constant. v0.4.0's cosine-LR smoke passed a recipe that diverged in the full run. Constant LR keeps the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' at sustained peak LR for the entire trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input. window — divergencedivergenceA training failure where loss descends through warmup, plateaus low for a while, then climbs catastrophically back to its starting magnitude — the model unlearns everything despite no obvious component failure. surfaces immediately if the recipe is unstable. See VERDICT_SMOKES.md.
Downside. Constant LR can overfit faster (no LR decay to slow learning in late trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input.). The v0.5.1 overfittingoverfittingWhen a model memorizes quirks of its training set instead of learning general patterns, so it scores well in training but poorly on new data. Guarded against with held-out evals and regularization. past step 65K may partly be caused by constant LR — cosine decaylearning-rate scheduleA plan for changing the learning rate over training — Mailwoman's is linear warmup to a peak, then cosine decay toward zero. Smooths early instability and lets the model settle at the end. would reduce the learning ratelearning rate (LR). How big a step training takes along the gradient each update. Too high and training diverges; too low and it crawls. Mailwoman warms it up, then decays it on a cosine schedule. by that point, slowing the overfittingoverfittingWhen a model memorizes quirks of its training set instead of learning general patterns, so it scores well in training but poorly on new data. Guarded against with held-out evals and regularization.. A future experiment: constant LR for the first 60K steps (validation), then cosine decaylearning-rate scheduleA plan for changing the learning rate over training — Mailwoman's is linear warmup to a peak, then cosine decay toward zero. Smooths early instability and lets the model settle at the end. for the remaining 40K (refinement).
Data levers
source_weights
What it does. Per-source sampling weightsparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. in the trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input. data loader. Higher weightparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. = more rows from that source in each epoch.
Current values are carried from v0.4.0's §4 source rebalance (the only recipe lever that shipped clean in v0.4.0):
| Source | WeightparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. | Why |
|---|---|---|
| tigerTIGERThe US Census Topologically Integrated Geographic Encoding and Referencing database. Used as a corpus source for street-segment data. | 4.0 | TIGERTIGERThe US Census Topologically Integrated Geographic Encoding and Referencing database. Used as a corpus source for street-segment data. carries structured US address patterns. Highest weightparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. to compensate for its small raw shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. count. |
| banBAN (Base Adresse Nationale). France's authoritative open national address register — the highest-quality training source for French addresses, with full component structure. | 3.0 | French address patterns. Second-highest weightparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values.. |
| wofWOF (Who's On First). An open-source gazetteer of places maintained by Mapzen/whosonfirst. Mailwoman builds a custom SQLite database from WOF GeoJSON repos, extended with postcode data, importance scores, and coincident-role relations.-admin | 2.0 | WOFWOF (Who's On First). An open-source gazetteer of places maintained by Mapzen/whosonfirst. Mailwoman builds a custom SQLite database from WOF GeoJSON repos, extended with postcode data, importance scores, and coincident-role relations. admin names (localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy., regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality., countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.). |
| wofWOF (Who's On First). An open-source gazetteer of places maintained by Mapzen/whosonfirst. Mailwoman builds a custom SQLite database from WOF GeoJSON repos, extended with postcode data, importance scores, and coincident-role relations.-postalcode | 2.0 | PostcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. patterns. |
| usgov-nadNAD (National Address Database). A US Department of Transportation dataset of structured address points, added to the training corpus as a major source of real US addresses. | 1.0 | NADNAD (National Address Database). A US Department of Transportation dataset of structured address points, added to the training corpus as a major source of real US addresses. was previously at 2.0 but dominated the sample (52% of shardsshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row.). v0.4.0 dropped it to 1.0 to recover postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. positional exposure from other sources. |
| stateregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.-* | 1.5-2.0 | StateregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.-level government sources (Iowa contractors, Texas/NY notaries). |
How we got here. v0.4.0's postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. regression (F1 0.76 → 0.69) was traced to NADNAD (National Address Database). A US Department of Transportation dataset of structured address points, added to the training corpus as a major source of real US addresses.'s downweight removing "postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. comes first" positional patterns. The current weightsparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. are a compromise. Not re-validated since v0.4.0 — the corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus.-v0.4.0 additions (kryptonite + transliterationtransliterationConverting a name from one writing system to another while preserving pronunciation (Cyrillic → Latin, for instance). Needed for multilingual address handling and corpus synthesis.) may shift the optimal mix.
v0.5.2 planned change: Drop wof-admin from 2.0 → 0.3. Root cause of the localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy./regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. confusion in demo presets: WOFWOF (Who's On First). An open-source gazetteer of places maintained by Mapzen/whosonfirst. Mailwoman builds a custom SQLite database from WOF GeoJSON repos, extended with postcode data, importance scores, and coincident-role relations. bare-name entries (which carry no positional context) outnumber OSMOpenStreetMap (OSM). A community-curated global map database (ODbL-licensed) with addr:* tagged features and place hierarchies. A secondary corpus source and a source of street names. full-address entries for ambiguous place namestoponymA proper name for a geographic place. like "Washington." The modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' learns the frequency-dominant pattern (regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.) rather than the positionally-correct pattern (localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. when preceding a regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. abbreviation). See DEMO_PRESET_DIAGNOSIS.md.
Training augmentation (v0.5.2 planned)
Directional expansion. Train on both raw and expanded forms of directional abbreviations: "NW" and "Northwest" with identical BIO labelsBIO tagging (Begin-Inside-Outside). A token-level labeling scheme where each token is tagged as B-X (beginning of an entity of type X), I-X (inside an entity of type X), or O (outside any entity). Mailwoman uses BIO over SentencePiece tokens to annotate address components.. Teaches the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' both forms without requiring inferenceinferenceRunning the trained model on new input to get predictions, as opposed to training, which produces the model. In Mailwoman that means a small transformer encoder reads an address string and classifies every token — house number, street, locality, region, postcode, and the rest. A Who's On First gazetteer can feed soft location hints into the pass, but the model makes the final call on every label. Where a generative model writes text token by token, Mailwoman's output is a retrieval-augmented token classification: one label per input piece.-time normalization. ~30 lines in corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus. pipelinestaged pipelineMailwoman's runtime architecture: a sequence of pure-function stages (normalize → query-shape → locale-gate → kind-classifier → phrase-grouper → classifier → decoder) connected by typed handoffs. Each stage is published as its own npm package..
RegionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.-abbreviation expansion. Train on both "NY" → B-region and "New York" → B-region I-region. Only expand where unambiguous. Teaches tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. equivalence. ~30 lines.
Not done: InferenceinferenceRunning the trained model on new input to get predictions, as opposed to training, which produces the model. In Mailwoman that means a small transformer encoder reads an address string and classifies every token — house number, street, locality, region, postcode, and the rest. A Who's On First gazetteer can feed soft location hints into the pass, but the model makes the final call on every label. Where a generative model writes text token by token, Mailwoman's output is a retrieval-augmented token classification: one label per input piece.-time normalization. SentencePieceSentencePieceA language-independent subword tokenizer that splits text into pieces using a unigram language model. Mailwoman uses a SentencePiece tokenizer with a 48,000-token vocabulary and byte-fallback, trained on address data rather than general text. vocabularyvocabularyThe fixed set of tokens a tokenizer can produce. Mailwoman's SentencePiece vocabulary is tens of thousands of subword pieces, with byte fallback for anything outside it. learned from raw corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus.; feeding normalized forms at inferenceinferenceRunning the trained model on new input to get predictions, as opposed to training, which produces the model. In Mailwoman that means a small transformer encoder reads an address string and classifies every token — house number, street, locality, region, postcode, and the rest. A Who's On First gazetteer can feed soft location hints into the pass, but the model makes the final call on every label. Where a generative model writes text token by token, Mailwoman's output is a retrieval-augmented token classification: one label per input piece. creates train/test distribution mismatch. All augmentation is trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input.-side only.
country_weights
What it does. Per-countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. acceptance probability in the data loader. Only US and FR are weighted (1.0 each) — these are the two localeslocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for. with trained weightsparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values..
Ceiling. Adding more countries (RU, JP, AM, KR, CN) is the v0.6.0 roadmap. The A1 tokenizertokenizerThe component that converts a raw address string into a sequence of numeric token IDs the model can process. Mailwoman's tokenizer is a SentencePiece unigram model trained specifically on postal addresses. already covers these scripts; the corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus. has transliterationtransliterationConverting a name from one writing system to another while preserving pronunciation (Cyrillic → Latin, for instance). Needed for multilingual address handling and corpus synthesis. rows; only the classifier trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input. is missing.
Known ceilings and open questions
-
Data ceiling at h384. OverfittingoverfittingWhen a model memorizes quirks of its training set instead of learning general patterns, so it scores well in training but poorly on new data. Guarded against with held-out evals and regularization. past step 65K suggests the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' has extracted what it can from corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus.-v0.4.0 at this capacity. More data (v0.5.0 corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus. expansion, v0.6.0 multi-localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for.) would raise this ceiling.
-
Composition ceiling. Per-tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. BIO accuracy (0.605-0.633 macro_f1) doesn't chain into per-component exact-match accuracy (0.1-6% on golden). This is a structural limitation of per-tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. training objectivesloss functionA number measuring how wrong the model's predictions are on a batch of examples. Training minimizes it. Mailwoman's loss combines per-token negative log-likelihood with the CRF sequence loss.. A future lossloss functionA number measuring how wrong the model's predictions are on a batch of examples. Training minimizes it. Mailwoman's loss combines per-token negative log-likelihood with the CRF sequence loss. term that penalizes globally invalid parsesaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates. (e.g. sequence-level negative log-likelihood over full addresses) would address this, but hasn't been explored.
-
Reconciler ceiling. Hybrid-joint improved exact-match from 0.1% to 6.0% with v0.5.0 weightsparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values.. Whether this improves further with v0.5.1 weightsparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. depends on the classifier putting better alternatives in its top-K. The reconciler can only re-rank what exists.
-
Calibration ceiling. The 54.5% overconfident-wrong rate with v0.5.0 weightsparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. is a first-generation modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' problem. label_smoothing and temperature scaling are unexplored levers. The reconciler masks this at the pipelinestaged pipelineMailwoman's runtime architecture: a sequence of pure-function stages (normalize → query-shape → locale-gate → kind-classifier → phrase-grouper → classifier → decoder) connected by typed handoffs. Each stage is published as its own npm package. level (0.1% overconfident-wrong) but per-modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' calibration would help all modes.
See also
- VERDICT_SMOKES.md — the smoke discipline that catches recipe failures
- Dual-loss curvature conflict — why crf_loss_weight=0 works
- v0.5.0 — as shipped — the threadthreadA parallel workstream within a release. Threads compose; they are not sequential milestones like phases. table
- What the eval numbers mean — plain-English interpretation