Night Shift 2 — 2026-05-28

Second night shiftnight shiftAn autonomous overnight agent session — training launches, evals, publishing, issue triage — that ends with a structured postmortem (what shipped, regressions, open questions) committed for handoff. focused on v0.6.0: StagestageOne of the dataflow stages in the runtime pipeline (normalize, locale gate, kind classify, phrase group, token classify, sequence correct, reconcile, resolve). Distinct from tier (model vocabulary) and phase (plan milestone). 3 labelcomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag. activation + PO boxPO boxA numbered mailbox at a post office used as a delivery address instead of a physical street location. Mailwoman tags it as the po_box component; structurally the same family as a subpremise. corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus. synthesis + multi-localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for. variant alias foundation.

Shipped

PO box synthesis pipeline

LocalelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for.-aware PO boxPO boxA numbered mailbox at a post office used as a delivery address instead of a physical street location. Mailwoman tags it as the po_box component; structurally the same family as a subpremise. generator covering en-US/CA/GB/AU, fr-FR/CA, es-ES/MX/AR.

• corpus/src/synthesize-po-box.ts — per-localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for. leader templates, number-format noise (10%), PMB variant logic
• corpus/src/adapters/synth-po-box/adapter.ts — JSONL tuples → CanonicalRows
• scripts/extract-tuples.py — pulls (localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy., regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality., postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one., countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.) from WOFWOF (Who's On First). An open-source gazetteer of places maintained by Mapzen/whosonfirst. Mailwoman builds a custom SQLite database from WOF GeoJSON repos, extended with postcode data, importance scores, and coincident-role relations. SQLite via localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.→county→regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. join + stateregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.-prefix-to-ZIP synthesis
• scripts/build-po-box-shard.mjs — runs synthesizer + alignRow, writes LabeledRow JSONL
• scripts/jsonl-to-parquet.py — converts to v0.4.0 parquetParquetThe open columnar file format the corpus is written and streamed in. The training pipeline reads shards row-by-row from Parquet. schema

Produced: 50K labeled rows / 3 MB parquetParquetThe open columnar file format the corpus is written and streamed in. The training pipeline reads shards row-by-row from Parquet. shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row.. Sample:

P.O. Box 9, Bancroft, ID 83603
  tokens: ['P', 'O', 'Box', '9', 'Bancroft', 'ID', '83603']
  labels: ['B-po_box', 'I-po_box', 'I-po_box', 'I-po_box', 'B-locality', 'B-region', 'B-postcode']

Design decisions

Consulted DeepSeek (3 turns) + USPS Pub 28 §28C2.040 + DMM 508 §4.1.4/§4.5.4:

1. PMB shares po_box tag. USPS treats PMB as a PO BoxPO boxA numbered mailbox at a post office used as a delivery address instead of a physical street location. Mailwoman tags it as the po_box component; structurally the same family as a subpremise. alias in CASS. Downstream code can distinguish via presence of a streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. line.
2. Whole-phrase spanspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree. ("PO BoxPO boxA numbered mailbox at a post office used as a delivery address instead of a physical street location. Mailwoman tags it as the po_box component; structurally the same family as a subpremise. 123" not "123"). Matches existing golden evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. convention.
3. Strategy A (replace streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels., not augment) — PO boxPO boxA numbered mailbox at a post office used as a delivery address instead of a physical street location. Mailwoman tags it as the po_box component; structurally the same family as a subpremise. and streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. are mutually exclusive per USPS. Synthesizing fake (streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. + PMB) tuples would teach the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' an invalid pattern.
4. 10% number-format noise — commas, dashes, embedded spaces. Real OCR/transcription input is lousy; ship with that as native.

Stage 3 activation

# Before
ACTIVE_TAGS = STAGE2_TAGS  # 10 tags, 21 BIO labels

# After
ACTIVE_TAGS = STAGE3_TAGS  # 16 tags, 33 BIO labels

STAGE3 appends to STAGE2 without reordering — IDs are preserved. Existing v0.4.0 shardsshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. work unchanged. Modelsneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' trained on STAGE2 IDs decode correctly against STAGE3; the new logitlogitA raw, unnormalized per-label score the model outputs before softmax. Priors and biases are added in logit space, then softmax turns logits into probabilities. slots just never get argmax'd.

Golden eval expansion

PO boxPO boxA numbered mailbox at a post office used as a delivery address instead of a physical street location. Mailwoman tags it as the po_box component; structurally the same family as a subpremise. coveragecoverageThe fraction of a population or region for which a data source has real, non-placeholder entries — e.g. 47% rooftop coverage on Texas addresses. Distinct from accuracy on the rows that are present. in data/eval/golden/v0.1.2/:

• Before: 1 entry (just "PO Box 123, Burlington, VT 05401")
• After: 27 entries
  • 20 US variants: PO BoxPO boxA numbered mailbox at a post office used as a delivery address instead of a physical street location. Mailwoman tags it as the po_box component; structurally the same family as a subpremise., P.O. BoxPO boxA numbered mailbox at a post office used as a delivery address instead of a physical street location. Mailwoman tags it as the po_box component; structurally the same family as a subpremise., P. O. Box, POB, POBOX, Post Office BoxPO boxA numbered mailbox at a post office used as a delivery address instead of a physical street location. Mailwoman tags it as the po_box component; structurally the same family as a subpremise., Box, P.O.Box; PMB at CMRA; single-digit through 7-digit
  • 6 FR/CA variants: BP, B.P., Boîte Postale, Case Postale, CP

Per-tag error analysis

scripts/eval-error-analysis.ts now emits a per-tag recallrecallOf the spans whose gold label is a given tag, the fraction the model found. High recall means few misses. Paired with precision to compute F1. table. v0.5.4 baseline:

• StagestageOne of the dataflow stages in the runtime pipeline (normalize, locale gate, kind classify, phrase group, token classify, sequence correct, reconcile, resolve). Distinct from tier (model vocabulary) and phase (plan milestone). 2 tags (localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy., regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality., postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one., streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels., house_number, ...) — high recallrecallOf the spans whose gold label is a given tag, the fraction the model found. High recall means few misses. Paired with precision to compute F1. as expected
• StagestageOne of the dataflow stages in the runtime pipeline (normalize, locale gate, kind classify, phrase group, token classify, sequence correct, reconcile, resolve). Distinct from tier (model vocabulary) and phase (plan milestone). 3 tags (street_prefix, street_suffix, unitunitA subdivision of a building — apartment, suite, floor — that refines a street address. Mailwoman's unit component; a designator plus identifier forms a subpremise., po_boxPO boxA numbered mailbox at a post office used as a delivery address instead of a physical street location. Mailwoman tags it as the po_box component; structurally the same family as a subpremise., intersection_a/b, attentionattentionThe core mechanism inside a transformer encoder. Each token's representation is updated by looking at every other token, with learned weights deciding how much each one matters., cedexCEDEX (Courrier d'Entreprise à Distribution Exceptionnelle). A French postal routing for high-volume business mail: a CEDEX code delivers directly from a sorting centre, bypassing the local post office. A common negative-space format Mailwoman must parse.) — 0% recallrecallOf the spans whose gold label is a given tag, the fraction the model found. High recall means few misses. Paired with precision to compute F1. (modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' doesn't emit them yet). Baseline for v0.6.0 to compare against.

NaN recovery

v0.6.0 trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input. diverged with NaNNaN (not a number). A floating-point result for an undefined operation (log of a negative, 0/0). Appearing in the training loss usually halts the run; recovering from it follows the NaN protocol. lossloss functionA number measuring how wrong the model's predictions are on a batch of examples. Training minimizes it. Mailwoman's loss combines per-token negative log-likelihood with the CRF sequence loss. at step 950 (right at end of warmupwarmupThe early phase of training where the learning rate ramps up from 0 to its peak value before cosine decay. Mailwoman uses a linear warmup., peak LR). Root cause per DeepSeek: CRFCRF (Conditional Random Field). A statistical modeling method that predicts structured outputs by modeling dependencies between adjacent labels. Mailwoman uses a linear-chain CRF as the Viterbi decoder at inference time to enforce BIO label consistency — a B-street must be followed by I-street or O, never I-locality. transition gradientgradientThe direction and rate at which the loss would change if each parameter were nudged. Training follows the gradient downhill to reduce error. Huge gradients are tamed by gradient clipping. at peak LR amplified by the new 33-labelcomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag. transition table.

Two-knob fix:

• learning_rate: 1.5e-4 → 1.0e-4
• crf_loss_weight: 0.5 → 0.1

Skipped: increased warmupwarmupThe early phase of training where the learning rate ramps up from 0 to its peak value before cosine decay. Mailwoman uses a linear warmup. (delays blowup), tighter grad clip (masks rather than fixes). After ship, raise CRFCRF (Conditional Random Field). A statistical modeling method that predicts structured outputs by modeling dependencies between adjacent labels. Mailwoman uses a linear-chain CRF as the Viterbi decoder at inference time to enforce BIO label consistency — a B-street must be followed by I-street or O, never I-locality. weightparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. incrementally.

Variant alias table (#166 follow-up)

Foundation shipped earlier in the day (@mailwoman/variant-aliases). 37 entries covering amenityamenityA point of interest referenced by category ('gas station', 'pharmacy', 'ATM') rather than by name. Resolved by mapping the query onto a category taxonomy. + brand variantssub-brandA branded variant within a larger franchise — Walmart Supercenter vs Walmart Neighborhood Market vs Sam's Club, all under Walmart. in en-AU/GB/CA, fr-FR/CA, ja-JP. LocalelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for.-gated lookup with confidence scoring. The runtime integration into the kind classifierkind classifierStage 2.5 of the runtime pipeline: categorizes the input into one of seven query kinds (structured_address, postcode_only, locality_only, intersection, po_box, landmark, vague) so the coordinator can route to the right parsing strategy. remains v0.6.0+ work.

v0.6.0 ship pipeline

scripts/ship-v0.6.0.sh — staged end-to-end: export ONNXONNX (Open Neural Network Exchange). An open format for machine learning models that enables interoperability between training frameworks and inference runtimes. Mailwoman ships its trained model as an ONNX file so it can run in Node.js and the browser via onnxruntime. from ModalModalA cloud GPU platform (modal.com) where Mailwoman trains its neural models on NVIDIA A100 GPUs. Training runs are launched via scripts/modal/train_remote.py and typically complete in ~1 hour. → quantize int8 → link as dev → 9 demo presets (6 canonical + 3 PO boxPO boxA numbered mailbox at a post office used as a delivery address instead of a physical street location. Mailwoman tags it as the po_box component; structurally the same family as a subpremise.) → error analysis → upload to HF.

Open issues

• #189 — alt_names FTS5FTS5SQLite's built-in full-text-search module (with BM25 ranking). Mailwoman uses it for prefix and name matching against the gazetteer. split. SQLite BM25's doc-length normalization can't be fixed with column weightsparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. alone. Real fix requires schema migration to separate FTS5FTS5SQLite's built-in full-text-search module (with BM25 ranking). Mailwoman uses it for prefix and name matching against the gazetteer. tables. Documented in docs/articles/concepts/importance-vs-population.md.
• #166 — variant alias runtime integration. Requires new QueryKindquery kindThe coarse category the kind classifier assigns to the whole input — postcode_only, locality_only, structured_address, intersection, po_box, landmark, or vague — used to route processing. values + POIpoint of interest (POI). A named place that is not strictly an address — landmark, transit stop, venue, amenity, or franchise. Mailwoman tags these as venue and resolves them through the gazetteer. index.

v0.6.0 results

TrainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input. completed at step 100K. CE-only (CRFCRF (Conditional Random Field). A statistical modeling method that predicts structured outputs by modeling dependencies between adjacent labels. Mailwoman uses a linear-chain CRF as the Viterbi decoder at inference time to enforce BIO label consistency — a B-street must be followed by I-street or O, never I-locality. trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input. disabled after two NaNNaN (not a number). A floating-point result for an undefined operation (log of a negative, 0/0). Appearing in the training loss usually halts the run; recovering from it follows the NaN protocol. attempts).

Demo presets: 11/11 pass (6 canonical + 5 StagestageOne of the dataflow stages in the runtime pipeline (normalize, locale gate, kind classify, phrase group, token classify, sequence correct, reconcile, resolve). Distinct from tier (model vocabulary) and phase (plan milestone). 3 variants).

Golden evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. per-tag (v0.5.4 → v0.6.0):

Tag	v0.5.4	v0.6.0	Δ
postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one.	75.7%	76.0%	+0.3
house_number	78.7%	79.0%	+0.3
regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.	65.0%	65.0%	flat
localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.	39.4%	39.7%	+0.3
streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.	28.0%	27.9%	flat
venuevenueA named, non-address place — a business, building, park, or stadium. Mailwoman's free-text point-of-interest component, added as a Tier 2 fine label.	29.4%	29.2%	flat
po_boxPO boxA numbered mailbox at a post office used as a delivery address instead of a physical street location. Mailwoman tags it as the po_box component; structurally the same family as a subpremise.	0.0%	51.9%	+51.9
street_prefix	0.0%	0.0%	flat (corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus. rebuild pending)
street_suffix	0.0%	0.0%	flat (corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus. rebuild pending)
unitunitA subdivision of a building — apartment, suite, floor — that refines a street address. Mailwoman's unit component; a designator plus identifier forms a subpremise.	0.0%	0.0%	flat (corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus. rebuild pending)
intersection_a/b	0.0%	0.0%	flat (corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus. rebuild pending)

v0.6.0 ships:

• HF modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' repo sister-software/mailwoman-en-us updated
• HF bucket en-us/v0.6.0/* populated (modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' + tokenizertokenizerThe component that converts a raw address string into a sequence of numeric token IDs the model can process. Mailwoman's tokenizer is a SentencePiece unigram model trained specifically on postal addresses. + FSTFST (finite-state transducer). A compact automaton that reads an input sequence and emits an output sequence. Mailwoman encodes gazetteer names and street affixes as FSTs for fast prefix matching and prior injection without search overhead. + wofWOF (Who's On First). An open-source gazetteer of places maintained by Mapzen/whosonfirst. Mailwoman builds a custom SQLite database from WOF GeoJSON repos, extended with postcode data, importance scores, and coincident-role relations.-hot.db + modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.'-card)
• releases.json updated, defaultVersion: v0.6.0
• neural-weights-en-us package bumped to 0.6.0

Deferred to v0.6.1:

• CRFCRF (Conditional Random Field). A statistical modeling method that predicts structured outputs by modeling dependencies between adjacent labels. Mailwoman uses a linear-chain CRF as the Viterbi decoder at inference time to enforce BIO label consistency — a B-street must be followed by I-street or O, never I-locality. learned transitions (NaNNaN (not a number). A floating-point result for an undefined operation (log of a negative, 0/0). Appearing in the training loss usually halts the run; recovering from it follows the NaN protocol. root cause investigation — bf16bf16 (bfloat16). A 16-bit floating-point format used during training. Half the memory of fp32 with similar numeric range, which lets the training batch fit on the lab's GPU. + 33×33 transition table)
• StreetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. decomposition recallrecallOf the spans whose gold label is a given tag, the fraction the model found. High recall means few misses. Paired with precision to compute F1. (needs corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus. rebuild with updated TIGERTIGERThe US Census Topologically Integrated Geographic Encoding and Referencing database. Used as a corpus source for street-segment data./NADNAD (National Address Database). A US Department of Transportation dataset of structured address points, added to the training corpus as a major source of real US addresses./BANBAN (Base Adresse Nationale). France's authoritative open national address register — the highest-quality training source for French addresses, with full component structure. adapters)
• StagestageOne of the dataflow stages in the runtime pipeline (normalize, locale gate, kind classify, phrase group, token classify, sequence correct, reconcile, resolve). Distinct from tier (model vocabulary) and phase (plan milestone). 3 intersectionintersectionAn address that names a location by two crossing streets ('5th & Main') rather than a number and street. Mailwoman tags the two streets as intersection_a and intersection_b — a negative-space format that starved the early model. / unitunitA subdivision of a building — apartment, suite, floor — that refines a street address. Mailwoman's unit component; a designator plus identifier forms a subpremise. tags (same corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus. rebuild requirement)