Contributing model work — the runbook

You want to improve a tag, add a localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for., or change anything that touches the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.'. This page is the procedure. The concepts behind it live elsewhere (What Mailwoman is, Eval discipline, Negative space); this is the part you follow with your hands.

The iron rules

1. Real-OOD evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. before trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input.. Build (or find) the held-out real-world evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. for your tag first, and pre-register the gate — target numbers, no-regression bounds — in the trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input. config as a comment before step 1 runs. If you can't define the gate, you aren't ready to train.
2. Never compare F1 across tokenizertokenizerThe component that converts a raw address string into a sequence of numeric token IDs the model can process. Mailwoman's tokenizer is a SentencePiece unigram model trained specifically on postal addresses. versions. Same tokenizertokenizerThe component that converts a raw address string into a sequence of numeric token IDs the model can process. Mailwoman's tokenizer is a SentencePiece unigram model trained specifically on postal addresses. or the comparison is void (per-locale-f1.ts --tokenizer enforces this; the model cardmodel cardA JSON metadata file (model-card.json) shipped with each weights bundle. It declares the model version, lineage, label set, required inference channels (anchor, gazetteer), calibration data, and training provenance. records the version).
3. Compare fp32fp32 / fp1632-bit and 16-bit floating-point formats. Mailwoman trains in bf16 (a 16-bit variant) and exports the ONNX model in int8 for size.-to-fp32fp32 / fp1632-bit and 16-bit floating-point formats. Mailwoman trains in bf16 (a 16-bit variant) and exports the ONNX model in int8 for size.. Int8 is a release artifact, not an experiment lens.
4. One variable per run. Prove a lever solo, then consolidate. Stacking shardsshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. in one 20k run dilutes every tag (measured, twice — see the cumulative-dilution note in the parity scorecard).
5. Recompile before evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error.. EvalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. harnesses load compiled core/out — a stale build makes your core change a silent no-op.

Which tool for which tag (the lever-shape taxonomy)

• Open-vocabvocabularyThe fixed set of tokens a tokenizer can produce. Mailwoman's SentencePiece vocabulary is tens of thousands of subword pieces, with byte fallback for anything outside it. / distributional tags (streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels., affixes, unitunitA subdivision of a building — apartment, suite, floor — that refines a street address. Mailwoman's unit component; a designator plus identifier forms a subpremise., localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.): the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' must learn the boundary from context → format-diverse synth shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. + retrain is the tool.
• Closed-vocabvocabularyThe fixed set of tokens a tokenizer can produce. Mailwoman's SentencePiece vocabulary is tens of thousands of subword pieces, with byte fallback for anything outside it. tags (countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head., po_boxPO boxA numbered mailbox at a post office used as a delivery address instead of a physical street location. Mailwoman tags it as the po_box component; structurally the same family as a subpremise., cedexCEDEX (Courrier d'Entreprise à Distribution Exceptionnelle). A French postal routing for high-volume business mail: a CEDEX code delivers directly from a sorting centre, bypassing the local post office. A common negative-space format Mailwoman must parse.): a finite surface set in a predictable slot → gazetteergazetteerA geographical index that maps place names and postcodes to real-world coordinates. Mailwoman uses a custom-built Who's On First (WOF) SQLite database as its gazetteer — the 'atlas' half of the grammar/atlas architecture./codex lexicon as a soft anchoranchor inferenceA technique where structured knowledge (postcode locations, gazetteer place names) is injected into the model as soft input features — not as deterministic overrides. The model still decides the final labels, but the anchor signal biases it toward correct admin tags. featurefeatureAn input signal a model conditions on. Beyond the raw tokens, Mailwoman feeds soft features — gazetteer-membership channels and the postcode anchor — that inform predictions without overriding them. — the matcher informs the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.', it never overrides it. We measured the override version and reversed it; read Closed-vocab fields: model-first before proposing a deterministic tagger.
• A dead/starvedstarvedA tag with too little training representation to learn — near-zero F1 — because the corpus adapter never emits examples of it (intersection tags sat at 0% until an intersection synthesizer existed). closed-vocabvocabularyThe fixed set of tokens a tokenizer can produce. Mailwoman's SentencePiece vocabulary is tens of thousands of subword pieces, with byte fallback for anything outside it. place-hierarchy tag (dependent_locality — a child place under a parent, that the shipped modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' won't emit): resurrect it at DECODE time with a retrieval-augmented pair-index overlay on the untouched base modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' — no retrain. This is a repeatable per-countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. recipe now: follow Onboarding a country's evidence layer (the GB + NZ arcs distilled). Note the closed-vs-open-vocabvocabularyThe fixed set of tokens a tokenizer can produce. Mailwoman's SentencePiece vocabulary is tens of thousands of subword pieces, with byte fallback for anything outside it. wall stated there — it works for place namestoponymA proper name for a geographic place. and FAILS for venuevenueA named, non-address place — a business, building, park, or stadium. Mailwoman's free-text point-of-interest component, added as a Tier 2 fine label./streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. spansspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree. (#1287/#1288 falsified with receipts); don't re-run those.
• Format quirks both systemsexpectation-maximizationAn iterative algorithm that estimates model parameters when some variables are unobserved. In Mailwoman's matcher, EM learns the Fellegi-Sunter m and u parameters from unlabeled data — no training labels needed. miss (rare edge shapes): consider a query-shape route before a shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. — some formats are structurally different enough that routing beats tagging.
• Cross-localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for. grammar leakageleakageTrain/test contamination that inflates reported accuracy when eval data has effectively been seen in training. Mailwoman guards it with held-out-geography evals and locality-aware splits. (a convention from one address system firing in another — USPS suffix logic on French streetsstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.): a conventions row + the decode-time mask, not a shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row.. codex/address-system-conventions.ts keyed by the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.''s own localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for. headattention headOne of several parallel attention computations in a layer, each free to focus on a different kind of relationship between tokens. Their outputs are concatenated — 'multi-head attention'. Mailwoman uses 4 heads.; zero GPU, applies without retrain. Shipped v4.3.0 (#478 slice 1).

The base-consistency check (the #511 lesson — run it BEFORE weighting any shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row.): a coveragecoverageThe fraction of a population or region for which a data source has real, non-placeholder entries — e.g. 47% rooftop coverage on Texas addresses. Distinct from accuracy on the rows that are present. shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. cannot outvote a contradicting base corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus.. The affix shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. spent four releases fighting ~467M base rows that labeled the same surfaces the other way (≥1,000:1 effective gradientgradientThe direction and rate at which the loss would change if each parameter were nudged. Training follows the gradient downhill to reduce error. Huge gradients are tamed by gradient clipping. mass); every weightparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values./architecture lever failed because the equilibrium was the mixing ratio. Audit what the base corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus. says about your tag's surfaces first (scripts/eval/audit-affix-misses.ts is the template — per-row miss classification + absorption check). If the base contradicts the shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row., the lever is a relabel pass (make the mix agree), not a bigger weightparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values.. Corollary, measured on v4.3.0: relabeling one boundary loosens its neighbors — pre-register watches on the adjacent classes (#513).

The standard gate set

Grade against v6.0.0-shipped-baseline, not v5.3.0-family

The current spec is v6.0.0-shipped-baseline.json — mailwoman eval gate --weights-cache <dir> --gate v6.0.0-shipped-baseline.

v5.3.0-family is retired for v6.x candidates. It was calibrated against the v2.2.0-era modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' and never re-cut when the lineage moved, so by 2026-07-16 it had drifted into uselessness in both directions: 16 of 17 floors were slack by +1.8 to +50pp (a candidate could lose 27.9pp of us.street_suffix and PASS), while its one tight floor — fr.postcode 99.2 — failed the shipped modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.'. Confirmed by running the full battery on v264 itself: verdict FAIL, on that floor, alone.

The v6 spec is cut from the shipped modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.''s own readings: floor = v264's reading − max(1.0pp, 2 × the metric's 95% binomial half-width). It tightens 15 of 18 floors. Both v264 and v310 PASS it. Rationale, the per-metric margin, and the two loosened floors (fr.postcode — a real, unexplained drift; us.country_homograph_f1 — noise) are argued in the spec's own $ keys.

When the lineage moves again, cut a new spec the same way — and note what it costs you: a spec cut from current readings has no memory and can only ratchet down. Every floor that moves gets a $revision note saying why, or the re-cut is just silent drift with better manners.

Never re-cut a campaign TARGET from a current reading. This spec re-cuts regression bars ("don't get worse than shipped"). The v7 streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. 0.90 goal lives in the parity floors; cutting it from today's 0.6067 would delete the objective. fr.bare_street_intact (75.0, #949) is likewise kept verbatim — it already carries a designed margin.

Run for every candidate, regardless of what you changed (this is what promotion-gate.ts, #479 automates):

Check	Command	Bar
Per-localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for. floors	scripts/eval/per-locale-f1.ts (US/FR, --tokenizer) + per-locale-f1-floors.py	within 1pp, fp32fp32 / fp1632-bit and 16-bit floating-point formats. Mailwoman trains in bf16 (a 16-bit variant) and exports the ONNX model in int8 for size.-to-fp32fp32 / fp1632-bit and 16-bit floating-point formats. Mailwoman trains in bf16 (a 16-bit variant) and exports the ONNX model in int8 for size.
German native order	scripts/eval/de-order-eval.ts	NO-REGRESSION
UnitunitA subdivision of a building — apartment, suite, floor — that refines a street address. Mailwoman's unit component; a designator plus identifier forms a subpremise. retention	unitunitA subdivision of a building — apartment, suite, floor — that refines a street address. Mailwoman's unit component; a designator plus identifier forms a subpremise.-real designatorsdesignatorThe closed-vocabulary leading word of a secondary-address phrase — 'Apt', 'Suite', 'Floor', 'PO Box', 'Level' — paired with an identifier to form a complete subpremise. evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error.	≥ 88
Affix split	scripts/eval/score-affix.ts (NOT per-locale-f1 — its fold can't see the split)	hold gated numbers
Demo smoke	eval-model skill / mailwoman eval preset-compare	presets stable
Honest geography	scripts/eval/honest-eval.ts	regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.-match + coord p50/p90 + PIP hold
Input robustness	mailwoman eval gauntlet --layer metamorphic (INV/DIR)	no new drift; xfails tracked

The input-robustness gate is the metamorphic layerlayerOne transformer block — attention plus a feed-forward network, with normalization and residual connections — applied to every position. Stacking layers lets the model build up richer representations; Mailwoman's encoder has 6. of the Gauntlet — labelcomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag.-preserving perturbations that must not move the assembled coordinatecoord metricThe primary evaluation metric: distance from the resolved coordinate to the true address point. Measured at percentiles (p50, p90) and as 'within X meters.' Prevents the label-F1 trap where a model scores higher on token labels but geocodes worse.. See Input robustness for the absorption layerslayerOne transformer block — attention plus a feed-forward network, with normalization and residual connections — applied to every position. Stacking layers lets the model build up richer representations; Mailwoman's encoder has 6. behind it and the coveragecoverageThe fraction of a population or region for which a data source has real, non-placeholder entries — e.g. 47% rooftop coverage on Texas addresses. Distinct from accuracy on the rows that are present. matrix.

The ship-config channels are part of the contract: a gaz-trained modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' is graded with --gazetteer-lexicon + --suppress-gaz-near-postcode, and (v4.3.0+) --conventions auto — the gate spec declares them (requires_gazetteer_lexicon, requires_conventions) so the runner threadsthreadA parallel workstream within a release. Threads compose; they are not sequential milestones like phases. them everywhere. A scorer invoked without the declared channels grades a crippled modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' and produces numbers that look real (the zero-fill trap; it has bitten the countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. leg and the first affix audit).

Plus your tag's pre-registered real-OOD gate. Pass → append the run to evals/scores-by-version.json via mailwoman eval ledger-append (the promotion gategated promotionA release discipline where a model ships to production only if it passes pre-registered metrics (e.g. 'DE locality ≥ 70%, no tag regresses > 2pp'). Failing models can be uploaded as experiments but not promoted to default. prints the pre-filled command on every PASS — fill in the npm semver at promote time) and write the dated report in docs/articles/evals/.

The full re-score cadence (standing rule, operator-approved 2026-07-02)

A promote's gate only checks the floors the spec names; labelcomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag. metrics can erode indefinitely in small, individually-justified "coordinate-invisible" steps with nothing forcing a periodic look. So, as a gate, not a virtue:

• Every 5 promotes, or any promote that lowers a gate floor, triggers a full per-tag re-score of the shipped line — the whole promotion-gate.ts battery against the current gate spec, published as a dated docs/articles/evals/parity-scorecard-YYYY-MM-DD.md.
• The scorecard quotes the config-canonical bars above its tables, keeps failing rows, and marks any relaxed threshold (the no-silent-gate-drift rules).
• Grade the artifact by explicit md5-pinned path — for the shipped line, verify against the published npm tarball's model.onnx, never the dev symlink (#259).

Calibration for why: v4.4.0 → v4.15.0 was 11 promotes without one; the 2026-07-02 re-anchor (#885) paid that debt and found the drift bounded — but also found two unsigned drifts (fr.cedex_real −6.7, libpostallibpostalAn open-source C address parser used by Pelias. Mailwoman's rule-based v0 and neural classifier supersede it. clean arenaarenaA standardized test set probing one capability: libpostal (clean canonical), perturb (noisy and degraded), postal (edge formats). Each arena answers a different question about where rule vs neural wins. −6) that per-promote gates structurally cannot catch.

Floor policy — floors ratchet at re-scores, never drift on promotes (#899)

The complement to the cadence rule, decided before a promote needed it rather than under promotion pressure:

1. Between re-scores, floors are immutable contracts. Measured == floor is a PASS — a knife-edge is legal, not a defect. A promote that dips below a floor FAILS; shipping it anyway is a gate revision under the existing no-silent-drift rules (written reason, attribution, operator sign-off) plus a note in that version's ledger row, so margin erosion is visible in the same record as the score.
2. At each cadence re-score, every floor is re-ratified, ratchet-up only. A tag whose measured value has cleared its floor by ≥ 2pp on the last two consecutive scorecards gets its floor raised to (lower of the two readings − 1pp) — gains get locked in instead of becoming slack for future erosion. Floors are never lowered at a re-score; lowering is only ever a per-promote revision under rule 1. Any floor sitting at zero margin gets an explicit disposition in the scorecard: knife-edge accepted (with the reason) or revision proposed — never silence.
3. Why not a standing tolerance: pre-widening floors by a fixed tolerance licenses exactly the slow-erosion pattern the #885 re-anchor exists to prevent. The buffer against measurement noise is the revision protocol's human in the loop, not a constant.

Standing disposition (2026-07-02): the two zero-margin postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. floors — us.postcode 95.0 and fr.postcode 99.3 — are knife-edge accepted. Both were set to measured values at the operator-approved v4.15.0 revision (the #723 anchor-pollution trade), and both held byte-exact across the v5.1.0 tokenizertokenizerThe component that converts a raw address string into a sequence of numeric token IDs the model can process. Mailwoman's tokenizer is a SentencePiece unigram model trained specifically on postal addresses. swap; the anchor-fed postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. channel is structurally stable, so the zero margin is by construction, not fragility. Any dip triggers rule 1, and the #901 campaign gate must name both postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. slices explicitly.

Splice safety gate — codepoint overlap is pre-registered, not post-measured (#900)

Any tokenizertokenizerThe component that converts a raw address string into a sequence of numeric token IDs the model can process. Mailwoman's tokenizer is a SentencePiece unigram model trained specifically on postal addresses. vocabvocabularyThe fixed set of tokens a tokenizer can produce. Mailwoman's SentencePiece vocabulary is tens of thousands of subword pieces, with byte fallback for anything outside it. splice must run the overlap gate before grading begins (tokenizer_splice.py build-tokenizer --trained-samples <locale=sample,…>): it computes the non-ASCII codepoints shared between the NEW piecesECE (Expected Calibration Error). A metric that measures how well a model's confidence scores align with its actual accuracy. Lower is better. Mailwoman's held-out ECE drops from 0.067 (raw) to 0.0035 (calibrated). and every trained localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for.'s character inventory, writes the per-localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for. report artifact next to the spliced tokenizertokenizerThe component that converts a raw address string into a sequence of numeric token IDs the model can process. Mailwoman's tokenizer is a SentencePiece unigram model trained specifically on postal addresses., and fails loud on any overlapping localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for. that was not explicitly accepted. Accepting a localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for. (--accept-overlap fr,de) is a commitment, not a waiver: a per-localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for. non-inferiority leg for each accepted localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for. must be pre-registered in the gate spec before the first measurement — the FR n=3000 coordinate leg from v5.1.0 is the template.

Why this is a gate: the v5.1.0 splice shipped on "byte-identical by construction," which turned out ASCII-only — FR/DE/ES shared é/ü/ö with the spliced piecesECE (Expected Calibration Error). A metric that measures how well a model's confidence scores align with its actual accuracy. Lower is better. Mailwoman's held-out ECE drops from 0.067 (raw) to 0.0035 (calibrated). and 52/15,000 EU rows re-tokenized, measured only after the fact. Netneural networkA model made of layers of simple numeric units whose connection strengths (weights) are learned from data. The transformer encoder at Mailwoman's core is a neural network.-positive that time; the gate removes the luck.

Adding a shard

1. Generator lives in corpus/ (synthesize-*.ts) or scripts/build-*-shard.mjs; every row carries synth: {method, base_source_id} ancestry.
2. Audit format diversity against the real evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error.'s observed forms before generating — thin diversity teaches lexical pattern-matching, not the boundary.
3. Overlay manifests declare lineage against a base corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus. version (scripts/assemble-*-overlay-manifest.py is the pattern); never hand-edit a manifest.
4. Train via modal run -d scripts/modal/train_remote.py --config <yaml> --resume auto (~1h A100). Config lives in corpus-python/src/mailwoman_train/configs/ — copy the latest, change one variable, write the gate comment.

Reading the scorecard

Two lenses that disagree on purpose: arenaarenaA standardized test set probing one capability: libpostal (clean canonical), perturb (noisy and degraded), postal (edge formats). Each arena answers a different question about where rule vs neural wins. headattention headOne of several parallel attention computations in a layer, each free to focus on a different kind of relationship between tokens. Their outputs are concatenated — 'multi-head attention'. Mailwoman uses 4 heads.-to-headattention headOne of several parallel attention computations in a layer, each free to focus on a different kind of relationship between tokens. Their outputs are concatenated — 'multi-head attention'. Mailwoman uses 4 heads. is whole-parseaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates.-strict (the strict "usable parseaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates." lens; understatesregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. per-tag wins) and per-tag F1 is what the campaign moves. Golden-dev per-tag numbers lie for tags the golden setgolden setA hand-labeled evaluation dataset of US, French, and adversarial addresses used as ground truth for evals. barely contains (unitunitA subdivision of a building — apartment, suite, floor — that refines a street address. Mailwoman's unit component; a designator plus identifier forms a subpremise. reads 6.3 there and 92.3 on real-OOD) — for campaign tags, the real-OOD column is the truth.

When your lever doesn't gate

Tune-and-retry once, then stop and write the postmortem instead of a third run — the v0.6.x cycle is the cautionary retrospective. A negative result recorded in the ledger is a contribution; a recipe-tweak treadmill is not.