Skip to main content

Phase 2 — Model Training

Goal: train a tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words.-classification modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' on corpus-v0.1.0+, export to ONNXONNX (Open Neural Network Exchange). An open format for machine learning models that enables interoperability between training frameworks and inference runtimes. Mailwoman ships its trained model as an ONNX file so it can run in Node.js and the browser via onnxruntime., quantize to int8, evaluate against the golden setgolden setA hand-labeled evaluation dataset of US, French, and adversarial addresses used as ground truth for evals.. End stateregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. is shippable neural-weights-en-us + neural-weights-fr-fr weightparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. packages ready for publishing.

Cadence (revised 2026-05-18): ~3-7 days per iteration, multiple iterations rather than a single 2-week run. See reference/ARCHITECTURE.md § "Training cadence vs. plan" for the iteration history + roadmap. The original "2 weeks single run to >95% F1" framing has been superseded.

Branch: ad-hoc per iteration (e.g. feat/v0.2.0-shipping, feat/v0.3.0-stage2), merged to main between iterations.

Depends on: PhasephaseA milestone in the implementation plan (Foundation, Corpus, Training, Integration, and forward-looking phases). Distinct from stage (runtime pipeline) and tier (model vocabulary). 1 complete (corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus. build pipelinestaged pipelineMailwoman's runtime architecture: a sequence of pure-function stages (normalize → query-shape → locale-gate → kind-classifier → phrase-grouper → classifier → decoder) connected by typed handoffs. Each stage is published as its own npm package. running, golden setgolden setA hand-labeled evaluation dataset of US, French, and adversarial addresses used as ground truth for evals. v0.1.0 in place).

Language: Python in packages/corpus-python/. No TypeScript in this phasephaseA milestone in the implementation plan (Foundation, Corpus, Training, Integration, and forward-looking phases). Distinct from stage (runtime pipeline) and tier (model vocabulary)..

Terminology note (2026-05-22): This document uses "TiertierInternal versioning of which label classes the model emits. Tier 1 is the coarse components (country, region, locality, postcode); Tier 2 adds venue, street, house_number; Tier 3 (future) would add attention, po_box, and POI venue subtyping. Historically called 'Stage 1/2/3' before the runtime-pipeline naming made that ambiguous." for labelcomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag.-vocabularyvocabularyThe fixed set of tokens a tokenizer can produce. Mailwoman's SentencePiece vocabulary is tens of thousands of subword pieces, with byte fallback for anything outside it. expansion (TiertierInternal versioning of which label classes the model emits. Tier 1 is the coarse components (country, region, locality, postcode); Tier 2 adds venue, street, house_number; Tier 3 (future) would add attention, po_box, and POI venue subtyping. Historically called 'Stage 1/2/3' before the runtime-pipeline naming made that ambiguous. 1 = coarse, TiertierInternal versioning of which label classes the model emits. Tier 1 is the coarse components (country, region, locality, postcode); Tier 2 adds venue, street, house_number; Tier 3 (future) would add attention, po_box, and POI venue subtyping. Historically called 'Stage 1/2/3' before the runtime-pipeline naming made that ambiguous. 2 = +fine, TiertierInternal versioning of which label classes the model emits. Tier 1 is the coarse components (country, region, locality, postcode); Tier 2 adds venue, street, house_number; Tier 3 (future) would add attention, po_box, and POI venue subtyping. Historically called 'Stage 1/2/3' before the runtime-pipeline naming made that ambiguous. 3 = +POIpoint of interest (POI). A named place that is not strictly an address — landmark, transit stop, venue, amenity, or franchise. Mailwoman tags these as venue and resolves them through the gazetteer.). Historically called "StagestageOne of the dataflow stages in the runtime pipeline (normalize, locale gate, kind classify, phrase group, token classify, sequence correct, reconcile, resolve). Distinct from tier (model vocabulary) and phase (plan milestone). 1/2/3" — renamed to free the word "StagestageOne of the dataflow stages in the runtime pipeline (normalize, locale gate, kind classify, phrase group, token classify, sequence correct, reconcile, resolve). Distinct from tier (model vocabulary) and phase (plan milestone)." for runtime-pipelinestaged pipelineMailwoman's runtime architecture: a sequence of pure-function stages (normalize → query-shape → locale-gate → kind-classifier → phrase-grouper → classifier → decoder) connected by typed handoffs. Each stage is published as its own npm package. stagesstageOne of the dataflow stages in the runtime pipeline (normalize, locale gate, kind classify, phrase group, token classify, sequence correct, reconcile, resolve). Distinct from tier (model vocabulary) and phase (plan milestone). (see concepts/the-staged-pipeline). Shipped artifacts (model cardsmodel cardA JSON metadata file (model-card.json) shipped with each weights bundle. It declares the model version, lineage, label set, required inference channels (anchor, gazetteer), calibration data, and training provenance., evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. filenames like stage2-step-001800-eval.md, npm v3.0.0 releases) preserve the old "StagestageOne of the dataflow stages in the runtime pipeline (normalize, locale gate, kind classify, phrase group, token classify, sequence correct, reconcile, resolve). Distinct from tier (model vocabulary) and phase (plan milestone)." naming as historical record.

Iteration log (live)​

  • v0.1.0 (shipped 2026-05-18, PR #42) — first TiertierInternal versioning of which label classes the model emits. Tier 1 is the coarse components (country, region, locality, postcode); Tier 2 adds venue, street, house_number; Tier 3 (future) would add attention, po_box, and POI venue subtyping. Historically called 'Stage 1/2/3' before the runtime-pipeline naming made that ambiguous. 1 ship. F1 below targets (~0.03 macro) due to positional-heuristic overfit; calibration 0.337 in conf>0.9 bucket. Honest below-target ship with v0.2.0 retrain recipe in session-notes.

  • v0.2.0 (shipped 2026-05-18, PR #53) — source_weights mechanism + relaxed coarse gate. 9× macro-F1 (0.037 → 0.335); calibration tightened 2.6× (0.337 → 0.882 in conf>0.9 bucket). Still below 95% target on countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head./regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality./localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. but real, measurable, ship-worthy improvement.

  • v0.3.0 → v3.0.0 (shipped 2026-05-22, PR #115; published as @mailwoman/neural-weights-{en-us,fr-fr}@3.0.0 — major-bumped from npm's 2.0.6 line to signal the 15→21 BIO breaking change). TiertierInternal versioning of which label classes the model emits. Tier 1 is the coarse components (country, region, locality, postcode); Tier 2 adds venue, street, house_number; Tier 3 (future) would add attention, po_box, and POI venue subtyping. Historically called 'Stage 1/2/3' before the runtime-pipeline naming made that ambiguous. 2 labelcomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag. expansion (venue / street / house_number BIO classes), linear-chain CRFCRF (Conditional Random Field). A statistical modeling method that predicts structured outputs by modeling dependencies between adjacent labels. Mailwoman uses a linear-chain CRF as the Viterbi decoder at inference time to enforce BIO label consistency — a B-street must be followed by I-street or O, never I-locality. decoderdecoderIn a transformer encoder-decoder model, the part that produces output sequences. Mailwoman's classifier is encoder-only (no decoder); the 'CRF decoder' is a different thing — a structured-prediction layer that picks the best label sequence from the encoder's outputs. over a frozen BIO transition mask, dual lossloss functionA number measuring how wrong the model's predictions are on a batch of examples. Training minimizes it. Mailwoman's loss combines per-token negative log-likelihood with the CRF sequence loss. (CE + 0.05·CRFCRF (Conditional Random Field). A statistical modeling method that predicts structured outputs by modeling dependencies between adjacent labels. Mailwoman uses a linear-chain CRF as the Viterbi decoder at inference time to enforce BIO label consistency — a B-street must be followed by I-street or O, never I-locality. NLLNLL (Negative Log Likelihood). The standard form of a loss function in probability-based models: loss = −log P(correct label | input). The CRF uses the NLL of the entire label sequence.), corpus-v0.3.0 rebuild (677M aligned rows, adds usgov-nad at 57.9M with full venuevenueA named, non-address place — a business, building, park, or stadium. Mailwoman's free-text point-of-interest component, added as a Tier 2 fine label.+streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.+house_number coveragecoverageThe fraction of a population or region for which a data source has real, non-placeholder entries — e.g. 47% rooftop coverage on Texas addresses. Distinct from accuracy on the rows that are present.). Four hparam iterations converged on lr=1.5e-4 + grad_clip=1.0 + crf_loss_weight=0.05 + label_smoothing=0 (down from v0.2.0's lr=5e-4 CE-only — dual lossloss functionA number measuring how wrong the model's predictions are on a batch of examples. Training minimizes it. Mailwoman's loss combines per-token negative log-likelihood with the CRF sequence loss. is far more LR-sensitive; see DECISIONS.md "v0.3.0 StagestageOne of the dataflow stages in the runtime pipeline (normalize, locale gate, kind classify, phrase group, token classify, sequence correct, reconcile, resolve). Distinct from tier (model vocabulary) and phase (plan milestone). 2 dual lossloss functionA number measuring how wrong the model's predictions are on a batch of examples. Training minimizes it. Mailwoman's loss combines per-token negative log-likelihood with the CRF sequence loss."). Early-stopped at step 1800 of 50K on val plateau. EvalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. against golden v0.1.2 (4,535 entries): macro F1macro F1The unweighted average of per-class F1 scores — treats every class equally. Mailwoman's primary label-level eval metric. 0.32; capability-surface win on the 3 new labelcomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag. classes (house_number F1 0.78 near #57's 0.8 target, venue 0.39, street 0.27 — both below floor; v0.4.0 targets); coarse F1 regressed substantially vs v0.2.0 (regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. 0.83 → 0.18, localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. 0.65 → 0.27, postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. 0.86 → 0.76) — under-trained at step 1800 + expanded labelcomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag. space pulled prior mass off coarse predictions. CRFCRF (Conditional Random Field). A statistical modeling method that predicts structured outputs by modeling dependencies between adjacent labels. Mailwoman uses a linear-chain CRF as the Viterbi decoder at inference time to enforce BIO label consistency — a B-street must be followed by I-street or O, never I-locality. makes orphan-I-* decode structurally impossible (verified on the demo's "Saint Petersburg" case).

  • JS ViterbiViterbi decodingA dynamic programming algorithm that finds the most likely sequence of hidden states (labels) given a sequence of observations (token emissions). Mailwoman uses Viterbi over a linear-chain CRF to produce globally coherent BIO label sequences from per-token model scores. (shipped 2026-05-22) — neural/viterbi.ts lands the ViterbiViterbi decodingA dynamic programming algorithm that finds the most likely sequence of hidden states (labels) given a sequence of observations (token emissions). Mailwoman uses Viterbi over a linear-chain CRF to produce globally coherent BIO label sequences from per-token model scores. decoderdecoderIn a transformer encoder-decoder model, the part that produces output sequences. Mailwoman's classifier is encoder-only (no decoder); the 'CRF decoder' is a different thing — a structured-prediction layer that picks the best label sequence from the encoder's outputs. + BIO structural transition mask. NeuralAddressClassifier defaults to ViterbiViterbi decodingA dynamic programming algorithm that finds the most likely sequence of hidden states (labels) given a sequence of observations (token emissions). Mailwoman uses Viterbi over a linear-chain CRF to produce globally coherent BIO label sequences from per-token model scores. decode; orphan-I-* sequences are now structurally impossible at JS runtime (matching the evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error.-time path). Works on the v3.0.0 weightsparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. as-is — no modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' retraining required because the structural mask is built from the labelscomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag. list. Learned CRFCRF (Conditional Random Field). A statistical modeling method that predicts structured outputs by modeling dependencies between adjacent labels. Mailwoman uses a linear-chain CRF as the Viterbi decoder at inference time to enforce BIO label consistency — a B-street must be followed by I-street or O, never I-locality. transitions will compose on top once a future weightsparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. release ships them.

  • v0.4.0 (shipped to packaged artifacts 2026-05-23, NOT npm-published — issue #116) — six planned work areas: (1) per-tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. CRFCRF (Conditional Random Field). A statistical modeling method that predicts structured outputs by modeling dependencies between adjacent labels. Mailwoman uses a linear-chain CRF as the Viterbi decoder at inference time to enforce BIO label consistency — a B-street must be followed by I-street or O, never I-locality. NLLNLL (Negative Log Likelihood). The standard form of a loss function in probability-based models: loss = −log P(correct label | input). The CRF uses the NLL of the entire label sequence. normalization; (2) longer trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input. (reach step-5000+ before judging); (3) class-weighted CE biased toward coarse classes; (4) source-weightparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. rebalance away from NADNAD (National Address Database). A US Department of Transportation dataset of structured address points, added to the training corpus as a major source of real US addresses.-heavy fine-labelcomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag. mix; (5) JS-side ViterbiViterbi decodingA dynamic programming algorithm that finds the most likely sequence of hidden states (labels) given a sequence of observations (token emissions). Mailwoman uses Viterbi over a linear-chain CRF to produce globally coherent BIO label sequences from per-token model scores. decode + modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.'-card labelcomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag. loading (landed 2026-05-22, see entry above); (6) reuse corpus-v0.3.0. Shipped recipe is §4 + §5 only; §1 and §3 deferred to v0.4.1 after destabilizing the dual-lossloss functionA number measuring how wrong the model's predictions are on a batch of examples. Training minimizes it. Mailwoman's loss combines per-token negative log-likelihood with the CRF sequence loss. trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input..

    Ablation campaign (2026-05-23, 5 divergencedivergenceA training failure where loss descends through warmup, plateaus low for a while, then climbs catastrophically back to its starting magnitude — the model unlearns everything despite no obvious component failure. runs + 3 verdict smokes + 1 full 50K + 1 successful full run):

    1. The full §1+§3+§4 recipe diverged at all three tested LRs (lr=5e-4 step 750, lr=3e-4 step 1000, lr=1.5e-4 step 2000). LR delays divergencedivergenceA training failure where loss descends through warmup, plateaus low for a while, then climbs catastrophically back to its starting magnitude — the model unlearns everything despite no obvious component failure. proportionally — destabilizer is in the recipe, not LR.
    2. At lr=5e-4, both single-knob ablations failed identically (ablate-§1: 0.35 → 0.11; ablate-§3: 0.35 → 0.14). lr=5e-4 is structurally unreachable for this codebase's dual-lossloss functionA number measuring how wrong the model's predictions are on a batch of examples. Training minimizes it. Mailwoman's loss combines per-token negative log-likelihood with the CRF sequence loss. landscape regardless of which §1/§3 knob is active.
    3. Switched to lr=1.5e-4 (v0.3.0-stable) verdict-smokeverdict-smokeA short diagnostic training run (≈1,500–3,000 steps) that must match the full run's gradient-noise profile, used to catch recipe instability before committing to a full 50k-step launch. matrix at max_steps=3000:
      • source-only (§4): PASS, peak 0.4190 step 2250
      • cw-only (§3+§4): PASS, peak 0.4279 step 2250
      • crfCRF (Conditional Random Field). A statistical modeling method that predicts structured outputs by modeling dependencies between adjacent labels. Mailwoman uses a linear-chain CRF as the Viterbi decoder at inference time to enforce BIO label consistency — a B-street must be followed by I-street or O, never I-locality.-only (§1+§4): FAIL, train_loss=1.24 at step 3000
    4. Verdict-smokeverdict-smokeA short diagnostic training run (≈1,500–3,000 steps) that must match the full run's gradient-noise profile, used to catch recipe instability before committing to a full 50k-step launch. framework had a false-positive on cw-only: the smoke's cosine LR decayed to ~0 by step 2750, masking sustained-peak-LR divergencedivergenceA training failure where loss descends through warmup, plateaus low for a while, then climbs catastrophically back to its starting magnitude — the model unlearns everything despite no obvious component failure.. When promoted to the full 50K run, cw-only diverged at step 2250 (macro_f1 collapse 0.41 → 0.29). Process improvement for v0.4.1: verdict smokes should use a constant LR or much longer max_steps. (Landed in v0.5.0 Thread F as --smoke-mode constant|long-tail; framework + decision matrix in VERDICT_SMOKES.md.)
    5. Math sanity-check of model.py + crf.py found no implementation bug — per_token reduction is nll.sum() / total_tokens.clamp(min=1); class_weightsparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. enter via cross_entropy(weight=...). Destabilization is a real recipe interaction, not a coding artifact.
    6. Shipped checkpointcheckpointA saved snapshot of the model weights and optimizer state during training. Mailwoman saves a checkpoint periodically so training can resume after a GPU hang.: v0_4_0-stableLR-source-only/step-002200 — §4 source rebalance + v0.3.0 dual-lossloss functionA number measuring how wrong the model's predictions are on a batch of examples. Training minimizes it. Mailwoman's loss combines per-token negative log-likelihood with the CRF sequence loss. base + lr=1.5e-4. Only recipe that stayed clean AND outperformed cw-only on golden v0.1.2 evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error..

    Golden v0.1.2 evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. (4535 entries) — mixed result, issue #116 success metric NOT cleanly met:

    tagv0.4.0 shippedv0.3.0Δ
    countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.0.210.28-0.07 regression
    regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.0.190.18+0.01
    localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.0.270.27flat
    postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one.0.690.76-0.07 regression
    venuevenueA named, non-address place — a business, building, park, or stadium. Mailwoman's free-text point-of-interest component, added as a Tier 2 fine label.0.390.39flat
    streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.0.300.27+0.03 improvement
    house_number0.790.78+0.01 (issue #57 floor held)

    Macro F1macro F1The unweighted average of per-class F1 scores — treats every class equally. Mailwoman's primary label-level eval metric. raw average: 0.357 vs 0.293. Mean tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. confidence: 0.806 vs 0.857. Full-parseaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates. exact matchexact matchThe share of eval items whose every component is correct (compared per-span or per-token). Stricter than per-tag F1, which credits partial correctness.: 0.082 vs 0.107 (regression — better per-component agreement, worse full-address agreement).

    Issue #116 asked for "clear progress on at least two of (coarse F1, fine F1, calibration, trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input. stability)":

    • coarse F1: NEGATIVE (countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head./postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. each -0.07)
    • fine F1: SMALL POSITIVE (streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. +0.03, house_number +0.01)
    • calibration: FLAT
    • trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input. stability: NEGATIVE (recipe destabilization is the campaign's central finding)

    Only one clean improvement axis. §1 (per_token CRFCRF (Conditional Random Field). A statistical modeling method that predicts structured outputs by modeling dependencies between adjacent labels. Mailwoman uses a linear-chain CRF as the Viterbi decoder at inference time to enforce BIO label consistency — a B-street must be followed by I-street or O, never I-locality.) and §3 (class_weightsparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values.) deferred to v0.4.1 after a corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus.-side investigation of why the full recipe destabilizes past step 2000.

    Post-hoc regression diagnostic (categorized, 4535 entries, 1217 postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. FNs + 194 countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. FNs):

    error classpostcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. FNcountrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. FN
    empty_pred (modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' emits nothing)789 (65%)9 (5%)
    non-Latin transliterationtransliterationConverting a name from one writing system to another while preserving pronunciation (Cyrillic → Latin, for instance). Needed for multilingual address handling and corpus synthesis. (v0.3.0 pre-existing)213 (18%)178 (92%)
    num_confused (house# picked instead of postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one.)136 (11%)n/a
    bio_slip (boundary off ±1 sentencepieceSentencePieceA language-independent subword tokenizer that splits text into pieces using a unigram language model. Mailwoman uses a SentencePiece tokenizer with a 48,000-token vocabulary and byte-fallback, trained on address data rather than general text.)73 (6%)small
    other6 (0.5%)~5%

    Headline takeaways:

    • The dominant postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. FN is empty_pred (65%) on mid-position / short-form postcodespostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. (Paris 75008, 47110 Sainte-Livrade-sur-Lot). v0.4.0's source rebalance traded structured-address postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. exposure for coarse-only 10118-style forms.
    • Country FN is 92% adversarial transliterationtransliterationConverting a name from one writing system to another while preserving pronunciation (Cyrillic → Latin, for instance). Needed for multilingual address handling and corpus synthesis. (CJK/Cyrillic raw input → English countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. gold). This is a v0.3.0 pre-existing failure mode; after excluding adversarials, countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. FN drops 194 → ~16. The countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. -0.07 F1 regression is mostly a golden-set adversarial-weighting artifact, not a real recipe regression.
    • DecoderdecoderIn a transformer encoder-decoder model, the part that produces output sequences. Mailwoman's classifier is encoder-only (no decoder); the 'CRF decoder' is a different thing — a structured-prediction layer that picks the best label sequence from the encoder's outputs. spanspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree.-trim sidecar (commit c72ab4c, no retrain required) addresses the 6% bio_slip slice + an unmeasured share of FP rate. Material impact on the headline numbers is smaller than the initial diagnostic suggested but the trim is still a correct decoderdecoderIn a transformer encoder-decoder model, the part that produces output sequences. Mailwoman's classifier is encoder-only (no decoder); the 'CRF decoder' is a different thing — a structured-prediction layer that picks the best label sequence from the encoder's outputs. bug fix.

    v0.4.1 implications: source-weightparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. tweak alone partially addresses the 11% num_confused slice. The 65% empty_pred slice requires a different intervention (likely source-weightparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. + synthesis pass over component-order permutations, or an aggressive wof-postalcode bump). Full v0.4.1 scope draft staged at .playpen/control/drafts/v0_4_1-scope.md on the i116 container.

  • v0.5.0 threadthreadA parallel workstream within a release. Threads compose; they are not sequential milestones like phases. C-s — scaffold only (shipped 2026-05-23, see Phase 8 fresh-slate plan § Thread C) — v0.4.1 was explicitly skipped in favor of a fresh-slate iteration. Thread C-s lands the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' code path for the v0.5.0 architecture changes — no trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input. run as part of this ship; the trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input. kicks off after Threads A (tokenizertokenizerThe component that converts a raw address string into a sequence of numeric token IDs the model can process. Mailwoman's tokenizer is a SentencePiece unigram model trained specifically on postal addresses. retrain) and B (corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus.-v0.4.0 with adversarial expansion) complete.

    Architectural changes shipped in the scaffold:

    • Top-k inferenceinferenceRunning the trained model on new input to get predictions, as opposed to training, which produces the model. In Mailwoman that means a small transformer encoder reads an address string and classifies every token — house number, street, locality, region, postcode, and the rest. A Who's On First gazetteer can feed soft location hints into the pass, but the model makes the final call on every label. Where a generative model writes text token by token, Mailwoman's output is a retrieval-augmented token classification: one label per input piece. (predict_top_k): the encoderencoderThe part of a transformer that turns input tokens into contextualized vector representations. Mailwoman's classifier is a small encoder-only transformer (~30M parameters). now emits the K most-probable tag sequences with calibrated log-probability scores (crf.top_k_decode, list-ViterbiViterbi decodingA dynamic programming algorithm that finds the most likely sequence of hidden states (labels) given a sequence of observations (token emissions). Mailwoman uses Viterbi over a linear-chain CRF to produce globally coherent BIO label sequences from per-token model scores. over the structural mask). Default k=5. This is what StagestageOne of the dataflow stages in the runtime pipeline (normalize, locale gate, kind classify, phrase group, token classify, sequence correct, reconcile, resolve). Distinct from tier (model vocabulary) and phase (plan milestone). 5 reconcile (Thread D) consumes — the classifier becomes a candidate generator rather than a single-answer predictor. Argmax path (predict) unchanged for back-compat.
    • Phrase-prior input-layerlayerOne transformer block — attention plus a feed-forward network, with normalization and residual connections — applied to every position. Stacking layers lets the model build up richer representations; Mailwoman's encoder has 6. conditioning: when model.use_phrase_priors=true, the encoderencoderThe part of a transformer that turns input tokens into contextualized vector representations. Mailwoman's classifier is a small encoder-only transformer (~30M parameters). takes a per-tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. featurefeatureAn input signal a model conditions on. Beyond the raw tokens, Mailwoman feeds soft features — gazetteer-membership channels and the postcode anchor — that inform predictions without overriding them. tensor (B, S, PHRASE_FEATURE_DIM=10) carrying BIE markers (phrase_start/phrase_mid/phrase_end) + a 7-way one-hot over the PhraseKind taxonomy (NUMERIC, STREET_PHRASE, LOCALITY_PHRASE, REGION_ABBREVIATION, POSTCODEpostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one., VENUE_PHRASE, HYPHENATED_COMPOUND — mirrored from Thread E's TS contract). FeaturesfeatureAn input signal a model conditions on. Beyond the raw tokens, Mailwoman feeds soft features — gazetteer-membership channels and the postcode anchor — that inform predictions without overriding them. are concatenated onto the tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words.+position embeddingembeddingA vector of numbers representing a token (or other item) so that similar items sit near each other in vector space. The first thing the model does is turn each token into an embedding. and projected back to hidden_size with a learned linear. Default false preserves v0.3.0/v0.4.0 behavior bit-identically — enables clean ablation of the phrase-prior contribution.
    • Hidden-size knob kept at v0.3.0/v0.4.0 baseline (256). The plan doc mentions a 256 → 384 or 512 bump "likely paid for by rented GPU." For the scaffold, hold the bump — validate the new architecture trains cleanly at the current size first; bump becomes an orthogonal v0_5_0-classifier-large.yaml follow-up once the baseline lands clean.
    • Labelcomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag. vocabvocabularyThe fixed set of tokens a tokenizer can produce. Mailwoman's SentencePiece vocabulary is tens of thousands of subword pieces, with byte fallback for anything outside it. unchanged from v0.3.0 (21 BIO classes; see mailwoman_train.labels.ACTIVE_BIO_LABELS). POIpoint of interest (POI). A named place that is not strictly an address — landmark, transit stop, venue, amenity, or franchise. Mailwoman tags these as venue and resolves them through the gazetteer. taxonomy expansion is out of scope per the plan doc.
    • Recipe parked: configs/v0_5_0-classifier-smoke.yaml defines the short verdict-smokeverdict-smokeA short diagnostic training run (≈1,500–3,000 steps) that must match the full run's gradient-noise profile, used to catch recipe instability before committing to a full 50k-step launch. (constant-LR per Thread F's VERDICT_SMOKES.md, --smoke-mode constant). Invocable once Threads A + B + F have all landed. NOT executed as part of this PR.

    Forward-pass smoke (tests/mailwoman_train/test_v0_5_0_forward_pass.py, 14 cases) verifies wiring on stub batchesbatch sizeHow many examples the model processes before each parameter update. Larger batches give smoother gradients but cost more memory; gradient accumulation simulates a big batch on a small GPU. in seconds — no lossloss functionA number measuring how wrong the model's predictions are on a batch of examples. Training minimizes it. Mailwoman's loss combines per-token negative log-likelihood with the CRF sequence loss..backward, no optimizeroptimizerThe component that decides how to update parameters from the gradient — Adam/AdamW being the common choice, adding momentum and per-parameter scaling on top of plain gradient descent. step. It is the integration check that decides whether the new architecture is ready for verdict-smokeverdict-smokeA short diagnostic training run (≈1,500–3,000 steps) that must match the full run's gradient-noise profile, used to catch recipe instability before committing to a full 50k-step launch. trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input., not whether it converges.

Pre-flight​

  • corpus-v0.1.0 exists at /data/corpus/versioned/corpus-v0.1.0/
  • Golden setgolden setA hand-labeled evaluation dataset of US, French, and adversarial addresses used as ground truth for evals. v0.1.0 exists at /data/eval/golden/v0.1.0/
  • Python environment set up with PyTorch, Transformersneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.', ONNXONNX (Open Neural Network Exchange). An open format for machine learning models that enables interoperability between training frameworks and inference runtimes. Mailwoman ships its trained model as an ONNX file so it can run in Node.js and the browser via onnxruntime., ONNX RuntimeONNX (Open Neural Network Exchange). An open format for machine learning models that enables interoperability between training frameworks and inference runtimes. Mailwoman ships its trained model as an ONNX file so it can run in Node.js and the browser via onnxruntime., datasets, sentencepieceSentencePieceA language-independent subword tokenizer that splits text into pieces using a unigram language model. Mailwoman uses a SentencePiece tokenizer with a 48,000-token vocabulary and byte-fallback, trained on address data rather than general text.
  • GPU available (PhasephaseA milestone in the implementation plan (Foundation, Corpus, Training, Integration, and forward-looking phases). Distinct from stage (runtime pipeline) and tier (model vocabulary). 2 is GPU-bound). If lab has no GPU, document trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input. time and proceed on CPU — the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' is small.

Tasks​

1. Python project setup​

  • packages/corpus-python/pyproject.toml with deps locked
  • packages/corpus-python/src/mailwoman_train/ package
  • Subcommand CLI: python -m mailwoman_train <command>

2. Data loading​

  • data_loader.py — load ParquetParquetThe open columnar file format the corpus is written and streamed in. The training pipeline reads shards row-by-row from Parquet. shardsshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. via datasets.load_dataset('parquet', ...). Lazy, streaming, memory-stable.
  • Stratified sampling: per-countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. weightsparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. to prevent imbalance. Configurable in YAML.
  • Length filtering: drop rows where tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. count > 128. Address text is short by nature; long rows are usually adapter bugs.
  • Verify tokenizertokenizerThe component that converts a raw address string into a sequence of numeric token IDs the model can process. Mailwoman's tokenizer is a SentencePiece unigram model trained specifically on postal addresses. alignmentalignmentThe step in the corpus pipeline that takes a (raw, components) pair from an adapter and produces a (raw, tokens, BIO labels) row by finding each component's text inside the raw string and labeling the matching tokens.: load tokenizer-v0.1.0, re-tokenize a sample of raw strings, assert the tokenization matches the stored tokens field. If it doesn't, the corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus. is corrupt — stop and investigate.

3. Model architecture​

  • model.py — small encoderencoderThe part of a transformer that turns input tokens into contextualized vector representations. Mailwoman's classifier is a small encoder-only transformer (~30M parameters).-only transformerneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.'
    • LayerslayerOne transformer block — attention plus a feed-forward network, with normalization and residual connections — applied to every position. Stacking layers lets the model build up richer representations; Mailwoman's encoder has 6.: 6
    • Hidden: 256
    • Attention headsattention headOne of several parallel attention computations in a layer, each free to focus on a different kind of relationship between tokens. Their outputs are concatenated — 'multi-head attention'. Mailwoman uses 4 heads.: 4
    • FF intermediate: 1024
    • Max position: 128
    • VocabvocabularyThe fixed set of tokens a tokenizer can produce. Mailwoman's SentencePiece vocabulary is tens of thousands of subword pieces, with byte fallback for anything outside it.: from tokenizertokenizerThe component that converts a raw address string into a sequence of numeric token IDs the model can process. Mailwoman's tokenizer is a SentencePiece unigram model trained specifically on postal addresses. v0.1.0
    • Token classificationsequence labelingA machine learning task where a model assigns a label to each element in a sequence. In Mailwoman, the sequence is the tokenized address string and the labels are address component types (street, locality, postcode, etc.). headattention headOne of several parallel attention computations in a layer, each free to focus on a different kind of relationship between tokens. Their outputs are concatenated — 'multi-head attention'. Mailwoman uses 4 heads.: linear → |BIO_LABELS| logitslogitA raw, unnormalized per-label score the model outputs before softmax. Priors and biases are added in logit space, then softmax turns logits into probabilities.
  • Use HuggingFace transformers library: BertConfig + BertForTokenClassification with custom small config. Or RoBERTa. Pick one and stick with it.
  • From-scratch initialization (not pretrained). Address vocabularyvocabularyThe fixed set of tokens a tokenizer can produce. Mailwoman's SentencePiece vocabulary is tens of thousands of subword pieces, with byte fallback for anything outside it. is too small to benefit from English-internet pretraining.

4. Training loop​

  • train.py
  • OptimizeroptimizerThe component that decides how to update parameters from the gradient — Adam/AdamW being the common choice, adding momentum and per-parameter scaling on top of plain gradient descent.: AdamWoptimizerThe component that decides how to update parameters from the gradient — Adam/AdamW being the common choice, adding momentum and per-parameter scaling on top of plain gradient descent., lr 5e-4, weightparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. decay 0.01
  • LR schedulelearning-rate scheduleA plan for changing the learning rate over training — Mailwoman's is linear warmup to a peak, then cosine decay toward zero. Smooths early instability and lets the model settle at the end.: linear warmupwarmupThe early phase of training where the learning rate ramps up from 0 to its peak value before cosine decay. Mailwoman uses a linear warmup. 1000 steps, then cosine decaylearning-rate scheduleA plan for changing the learning rate over training — Mailwoman's is linear warmup to a peak, then cosine decay toward zero. Smooths early instability and lets the model settle at the end.
  • Batch sizebatch sizeHow many examples the model processes before each parameter update. Larger batches give smoother gradients but cost more memory; gradient accumulation simulates a big batch on a small GPU.: 256 (lower if OOM)
  • Steps: ~50k for TiertierInternal versioning of which label classes the model emits. Tier 1 is the coarse components (country, region, locality, postcode); Tier 2 adds venue, street, house_number; Tier 3 (future) would add attention, po_box, and POI venue subtyping. Historically called 'Stage 1/2/3' before the runtime-pipeline naming made that ambiguous. 1 coarse-only. EvalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. every 2k steps on val set.
  • Mixed precisionprecisionOf the spans the model labeled as a given tag, the fraction it got right. High precision means few false positives. Paired with recall to compute F1. (fp16fp32 / fp1632-bit and 16-bit floating-point formats. Mailwoman trains in bf16 (a 16-bit variant) and exports the ONNX model in int8 for size./bf16bf16 (bfloat16). A 16-bit floating-point format used during training. Half the memory of fp32 with similar numeric range, which lets the training batch fit on the lab's GPU.) on GPU
  • Save checkpointcheckpointA saved snapshot of the model weights and optimizer state during training. Mailwoman saves a checkpoint periodically so training can resume after a GPU hang. every 5k steps to /data/models/checkpoints/
  • Track: train lossloss functionA number measuring how wrong the model's predictions are on a batch of examples. Training minimizes it. Mailwoman's loss combines per-token negative log-likelihood with the CRF sequence loss., val lossloss functionA number measuring how wrong the model's predictions are on a batch of examples. Training minimizes it. Mailwoman's loss combines per-token negative log-likelihood with the CRF sequence loss., val F1 per component, val full-parseaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates. exact matchexact matchThe share of eval items whose every component is correct (compared per-span or per-token). Stricter than per-tag F1, which credits partial correctness.
  • Use Weights & Biases or TensorBoard or plain CSV — pick one and stick with it. Don't ship a logging refactor in the middle of trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input..

5. Tiered training plan​

Tier 1: Coarse-only (this phase)​

  • Train only on rows where country and at least one of (region, locality, postcode) is present
  • Labelscomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag. restricted to: country, region, locality, dependent_locality, postcode, subregion, cedex, O
  • Target: > 95% F1 per component on golden setgolden setA hand-labeled evaluation dataset of US, French, and adversarial addresses used as ground truth for evals.
  • This is the v0.1.0 modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.'.

TierstierInternal versioning of which label classes the model emits. Tier 1 is the coarse components (country, region, locality, postcode); Tier 2 adds venue, street, house_number; Tier 3 (future) would add attention, po_box, and POI venue subtyping. Historically called 'Stage 1/2/3' before the runtime-pipeline naming made that ambiguous. 2 (streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.) and 3 (venuevenueA named, non-address place — a business, building, park, or stadium. Mailwoman's free-text point-of-interest component, added as a Tier 2 fine label.) are explicitly future iterations. Do not attempt to train all tierstierInternal versioning of which label classes the model emits. Tier 1 is the coarse components (country, region, locality, postcode); Tier 2 adds venue, street, house_number; Tier 3 (future) would add attention, po_box, and POI venue subtyping. Historically called 'Stage 1/2/3' before the runtime-pipeline naming made that ambiguous. in PhasephaseA milestone in the implementation plan (Foundation, Corpus, Training, Integration, and forward-looking phases). Distinct from stage (runtime pipeline) and tier (model vocabulary). 2.

6. Evaluation​

  • eval.py — load checkpointcheckpointA saved snapshot of the model weights and optimizer state during training. Mailwoman saves a checkpoint periodically so training can resume after a GPU hang., run inferenceinferenceRunning the trained model on new input to get predictions, as opposed to training, which produces the model. In Mailwoman that means a small transformer encoder reads an address string and classifies every token — house number, street, locality, region, postcode, and the rest. A Who's On First gazetteer can feed soft location hints into the pass, but the model makes the final call on every label. Where a generative model writes text token by token, Mailwoman's output is a retrieval-augmented token classification: one label per input piece. on golden setgolden setA hand-labeled evaluation dataset of US, French, and adversarial addresses used as ground truth for evals., compute metrics
  • Metrics:
    • Per-component F1, precisionprecisionOf the spans the model labeled as a given tag, the fraction it got right. High precision means few false positives. Paired with recall to compute F1., recallrecallOf the spans whose gold label is a given tag, the fraction the model found. High recall means few misses. Paired with precision to compute F1.
    • Full-parseaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates. exact matchexact matchThe share of eval items whose every component is correct (compared per-span or per-token). Stricter than per-tag F1, which credits partial correctness. (all components correct)
    • Mean tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. confidence
    • Calibration: bucket predictions by confidence, check accuracy per bucket
  • Output: a markdown report saved alongside the checkpointcheckpointA saved snapshot of the model weights and optimizer state during training. Mailwoman saves a checkpoint periodically so training can resume after a GPU hang.
  • Compare against rule-based Mailwoman on the same golden setgolden setA hand-labeled evaluation dataset of US, French, and adversarial addresses used as ground truth for evals.. Rule baseline numbers should be cached so this doesn't require running Mailwoman during trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input..

7. ONNX export​

  • export_onnx.py
  • Export with dynamic axes for batchbatch sizeHow many examples the model processes before each parameter update. Larger batches give smoother gradients but cost more memory; gradient accumulation simulates a big batch on a small GPU. and sequence length
  • Opset 17
  • Verify ONNXONNX (Open Neural Network Exchange). An open format for machine learning models that enables interoperability between training frameworks and inference runtimes. Mailwoman ships its trained model as an ONNX file so it can run in Node.js and the browser via onnxruntime. inferenceinferenceRunning the trained model on new input to get predictions, as opposed to training, which produces the model. In Mailwoman that means a small transformer encoder reads an address string and classifies every token — house number, street, locality, region, postcode, and the rest. A Who's On First gazetteer can feed soft location hints into the pass, but the model makes the final call on every label. Where a generative model writes text token by token, Mailwoman's output is a retrieval-augmented token classification: one label per input piece. matches PyTorch inferenceinferenceRunning the trained model on new input to get predictions, as opposed to training, which produces the model. In Mailwoman that means a small transformer encoder reads an address string and classifies every token — house number, street, locality, region, postcode, and the rest. A Who's On First gazetteer can feed soft location hints into the pass, but the model makes the final call on every label. Where a generative model writes text token by token, Mailwoman's output is a retrieval-augmented token classification: one label per input piece. within 1e-4 on a sample of 1000 inputs
  • Output: /data/models/onnx/model-v0.1.0-en-us.onnx, same for fr-fr

⚠ If you trained a single multilingual modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' (recommended for TiertierInternal versioning of which label classes the model emits. Tier 1 is the coarse components (country, region, locality, postcode); Tier 2 adds venue, street, house_number; Tier 3 (future) would add attention, po_box, and POI venue subtyping. Historically called 'Stage 1/2/3' before the runtime-pipeline naming made that ambiguous. 1 — coarse is cheap to share), export it twice with the same weightsparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values., named per localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for.. Splitting into per-localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for. modelsneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' is a PhasephaseA milestone in the implementation plan (Foundation, Corpus, Training, Integration, and forward-looking phases). Distinct from stage (runtime pipeline) and tier (model vocabulary). 3 decision based on size and load behavior.

8. Quantization​

  • quantize.py — int8 dynamic quantization via onnxruntime.quantization
  • Calibrate on 1000 val-set examples
  • Verify quantized modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' F1 on golden setgolden setA hand-labeled evaluation dataset of US, French, and adversarial addresses used as ground truth for evals. drops by less than 0.5% from fp32fp32 / fp1632-bit and 16-bit floating-point formats. Mailwoman trains in bf16 (a 16-bit variant) and exports the ONNX model in int8 for size.. If it drops more, investigate (likely a quantization config issue, not a fundamental limit).
  • Output: /data/models/quantized/model-v0.1.0-en-us-int8.onnx

9. Weights package preparation​

  • packages/neural-weights-en-us/ and packages/neural-weights-fr-fr/
  • Each contains:
    • model.onnx (int8 quantized)
    • tokenizer.model (SentencePieceSentencePieceA language-independent subword tokenizer that splits text into pieces using a unigram language model. Mailwoman uses a SentencePiece tokenizer with a 48,000-token vocabulary and byte-fallback, trained on address data rather than general text.)
    • model-card.json (ModelCard per reference/INTERFACES.md)
    • package.json with name, version, license
    • README.md describing the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.', trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input. corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus., evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. scores
  • These packages are data-only. No JS code. They are loaded by @mailwoman/neural at runtime.
  • Verify package size: aim for < 30MB int8 modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.'. TokenizertokenizerThe component that converts a raw address string into a sequence of numeric token IDs the model can process. Mailwoman's tokenizer is a SentencePiece unigram model trained specifically on postal addresses. is ~1MB. Total package < 40MB.

10. Model card​

  • ModelCard filled in honestly
  • Include: trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input. corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus. version, trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input. duration, hardware, evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. scores per component on golden setgolden setA hand-labeled evaluation dataset of US, French, and adversarial addresses used as ground truth for evals. + holdout splits, known failure modes (e.g., "underperforms on Hawaiian addresses", "confused by historical Paris arrondissement notation pre-1860")
  • This is a public document. Users will read it before adopting.

Success criteria checklist​

  • TiertierInternal versioning of which label classes the model emits. Tier 1 is the coarse components (country, region, locality, postcode); Tier 2 adds venue, street, house_number; Tier 3 (future) would add attention, po_box, and POI venue subtyping. Historically called 'Stage 1/2/3' before the runtime-pipeline naming made that ambiguous. 1 modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' trained, checkpointcheckpointA saved snapshot of the model weights and optimizer state during training. Mailwoman saves a checkpoint periodically so training can resume after a GPU hang. saved
  • ONNXONNX (Open Neural Network Exchange). An open format for machine learning models that enables interoperability between training frameworks and inference runtimes. Mailwoman ships its trained model as an ONNX file so it can run in Node.js and the browser via onnxruntime. export verified parity with PyTorch
  • Int8 quantized modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' meets evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. threshold
  • neural-weights-en-us@0.1.0 and neural-weights-fr-fr@0.1.0 package directories complete
  • Model cardsmodel cardA JSON metadata file (model-card.json) shipped with each weights bundle. It declares the model version, lineage, label set, required inference channels (anchor, gazetteer), calibration data, and training provenance. filled in
  • EvalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. reports committed to git (numbers, not modelsneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.')
  • Beats rule-based Mailwoman on golden setgolden setA hand-labeled evaluation dataset of US, French, and adversarial addresses used as ground truth for evals. for country and region components by at least 2 F1 points. If not, investigate before proceeding — the architecture is fine, the corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus. is probably the issue.

When to ship vs train more (original framing — superseded by iteration cadence)​

If after the first trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input. run, golden F1 is:

  • 95% per coarse component → ship.

  • 90–95% → analyze failure modes. Likely fixable with corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus. tweaks (more synthesis, deduplicationrecord matchingThe process of determining whether two database records refer to the same real-world entity. Mailwoman's matcher uses a geocode-first approach (match the resolved place, not the address string) with Fellegi-Sunter probabilistic scoring., a missing source). One additional iteration is fine.
  • < 90% → stop and re-examine. Could be: tokenizertokenizerThe component that converts a raw address string into a sequence of numeric token IDs the model can process. Mailwoman's tokenizer is a SentencePiece unigram model trained specifically on postal addresses. mismatch, labelcomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag. misalignment, schema bug, severely imbalanced trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input. data. Do not "train longer" without diagnosing first.

⚠ Resist the urge to add streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.-level components in this phasephaseA milestone in the implementation plan (Foundation, Corpus, Training, Integration, and forward-looking phases). Distinct from stage (runtime pipeline) and tier (model vocabulary). to "get more value." TiertierInternal versioning of which label classes the model emits. Tier 1 is the coarse components (country, region, locality, postcode); Tier 2 adds venue, street, house_number; Tier 3 (future) would add attention, po_box, and POI venue subtyping. Historically called 'Stage 1/2/3' before the runtime-pipeline naming made that ambiguous. 1 ships coarse. TiertierInternal versioning of which label classes the model emits. Tier 1 is the coarse components (country, region, locality, postcode); Tier 2 adds venue, street, house_number; Tier 3 (future) would add attention, po_box, and POI venue subtyping. Historically called 'Stage 1/2/3' before the runtime-pipeline naming made that ambiguous. 2 is its own iteration. Mixing them blurs the metrics and slows iteration.

Revised (2026-05-18)​

The above original framing assumed a single trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input. run. After v0.1.0 + v0.2.0, the actual cadence is:

  • Each iteration ships an artifact, target or no target. Below-target ships are OK if they're honest about it (model cardmodel cardA JSON metadata file (model-card.json) shipped with each weights bundle. It declares the model version, lineage, label set, required inference channels (anchor, gazetteer), calibration data, and training provenance. + evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. ledger entry) AND the Ship-of-Theseus coexistence modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' means the rule classifiers still run alongside.
  • Per-iteration F1 floor (not target): each iteration must improve over the prior one in at least one of: per-component F1, calibration tightness, or capability surface (new labelscomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag. supported).
  • VocabularyvocabularyThe fixed set of tokens a tokenizer can produce. Mailwoman's SentencePiece vocabulary is tens of thousands of subword pieces, with byte fallback for anything outside it. tiertierInternal versioning of which label classes the model emits. Tier 1 is the coarse components (country, region, locality, postcode); Tier 2 adds venue, street, house_number; Tier 3 (future) would add attention, po_box, and POI venue subtyping. Historically called 'Stage 1/2/3' before the runtime-pipeline naming made that ambiguous. expansion (TiertierInternal versioning of which label classes the model emits. Tier 1 is the coarse components (country, region, locality, postcode); Tier 2 adds venue, street, house_number; Tier 3 (future) would add attention, po_box, and POI venue subtyping. Historically called 'Stage 1/2/3' before the runtime-pipeline naming made that ambiguous. 2 → venuevenueA named, non-address place — a business, building, park, or stadium. Mailwoman's free-text point-of-interest component, added as a Tier 2 fine label.+streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.+house_number; TiertierInternal versioning of which label classes the model emits. Tier 1 is the coarse components (country, region, locality, postcode); Tier 2 adds venue, street, house_number; Tier 3 (future) would add attention, po_box, and POI venue subtyping. Historically called 'Stage 1/2/3' before the runtime-pipeline naming made that ambiguous. 3 → organization/POIpoint of interest (POI). A named place that is not strictly an address — landmark, transit stop, venue, amenity, or franchise. Mailwoman tags these as venue and resolves them through the gazetteer. venuevenueA named, non-address place — a business, building, park, or stadium. Mailwoman's free-text point-of-interest component, added as a Tier 2 fine label.) happens as iteration deltas within PhasephaseA milestone in the implementation plan (Foundation, Corpus, Training, Integration, and forward-looking phases). Distinct from stage (runtime pipeline) and tier (model vocabulary). 2, not as separate phasesphaseA milestone in the implementation plan (Foundation, Corpus, Training, Integration, and forward-looking phases). Distinct from stage (runtime pipeline) and tier (model vocabulary).. Same encoderencoderThe part of a transformer that turns input tokens into contextualized vector representations. Mailwoman's classifier is a small encoder-only transformer (~30M parameters)., more headattention headOne of several parallel attention computations in a layer, each free to focus on a different kind of relationship between tokens. Their outputs are concatenated — 'multi-head attention'. Mailwoman uses 4 heads. classes. Each lands in its own retrain.
  • The evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. ledger is the success record — evals/scores-by-version.json with corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus. + golden-set sha-pinning makes every iteration's delta empirically defensible.

When to call this phase done​

When the weightsparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. packages exist on disk, model cardsmodel cardA JSON metadata file (model-card.json) shipped with each weights bundle. It declares the model version, lineage, label set, required inference channels (anchor, gazetteer), calibration data, and training provenance. are accurate, evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. scores beat the rule baseline on coarse components, and the only remaining work for shipping is TypeScript integration (PhasephaseA milestone in the implementation plan (Foundation, Corpus, Training, Integration, and forward-looking phases). Distinct from stage (runtime pipeline) and tier (model vocabulary). 3).