2026-07-23 — placetype-pair-prior arc: ship-prep scorecard (Task 8)
Branch feat/placetype-pair-prior (unswitched throughout the arc). This is the ship-PREP
scorecard for Task 8 — no release was dispatched, nothing was promoted, no npm version was
bumped. It documents every number gating the decision, both checkpointscheckpointA saved snapshot of the model weights and optimizer state during training. Mailwoman saves a checkpoint periodically so training can resume after a GPU hang., and the one new
ship-blockingblockingThe first stage of entity resolution: generate candidate record pairs using cheap, high-recall keys (geo cell, canonical address, phone) instead of comparing every record to every other (O(n²)). The matcher only scores pairs that survive blocking. finding this task surfaced (the Gauntlet). Full battery/ablation source:
.superpowers/sdd/task-7-report.md; this task's own additions (Gauntlet, model cardsmodel cardA JSON metadata file (model-card.json) shipped with each weights bundle. It declares the model version, lineage, label set, required inference channels (anchor, gazetteer), calibration data, and training provenance., release
wiring): .superpowers/sdd/task-8-prep-report.md.
Config-canonical bars (pre-registered, v3.11.0-deploc-feed.yaml header)
PRIMARY : decode-layerlayerOne transformer block — attention plus a feed-forward network, with normalization and residual connections — applied to every position. Stacking layers lets the model build up richer representations; Mailwoman's encoder has 6. dependent_locality emission MATERIALLY above probe-2 floors (NZ > 3/246, GB > 1/69) with tag-correct majority; raw-BIO emission + gap trajectory reported.
GUARDS (each REGISTERED, none composite-maskable):
- golden us micro within ±0.7pp of v385 (probe-2 missed by 0.1pp — must recover)
- golden fr micro within ±0.7pp of v385
- FR-fragment BARE-LOCALITYlocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. class ≥ 0.90 (v385 baseline 0.978; probe-2 collapsed to 0.603 — the composite board number is NOT the bar; this class is)
- digit board bare-streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.-hn flat vs 0.902 (probe-1/2 level; v385 fresh baseline 0.755)
- 6 demo presets byte-identical to v385
- val macro_f1 within 1.0pp of v385's 0.7047
SHIP GATE: full error-analysis vs v385 — no tag regresses >2pp (config-canonical); gauntlet PASS. Ship path (HF upload → promote → npm CI → demo) is OPERATOR-AUTHORIZED for the 2026-07-22 night shiftnight shiftAn autonomous overnight agent session — training launches, evals, publishing, issue triage — that ends with a structured postmortem (what shipped, regressions, open questions) committed for handoff. CONDITIONAL on every bar above passing. Any bar fails → stagestageOne of the dataflow stages in the runtime pipeline (normalize, locale gate, kind classify, phrase group, token classify, sequence correct, reconcile, resolve). Distinct from tier (model vocabulary) and phase (plan milestone). only, report, no promote (no silent gate drift; treadmill guard: no knob iteration).
Digit gate revision (operator-ratified 2026-07-23, Teffen Ellis)
The pre-registered digit bar above (flat vs 0.902) was written against the sibling en-gb-localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for.
probe lineage's own 8k checkpointcheckpointA saved snapshot of the model weights and optimizer state during training. Mailwoman saves a checkpoint periodically so training can resume after a GPU hang. (v3.10.1-gb-probe2) — a same-lineage probe number, not a real
ship baseline for a checkpointcheckpointA saved snapshot of the model weights and optimizer state during training. Mailwoman saves a checkpoint periodically so training can resume after a GPU hang. forked from v385. That comparison is not apples-to-apples for
v3.11.0-deploc-feed, which forks from v385 (6.6.0) directly. The bar is restated against
v385's own fresh-measured baseline, taken in the same session on the same board:
| value | |
|---|---|
| v385 fresh baseline (bare-streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.-hn) | 0.755 |
| feed-8k (candidate) | 0.868 [0.831, 0.897] |
| Δ | +11.3pp |
| old bar (0.902, probe-anchor) | historical context only — CI excludes it, not the active bar |
| result | PASS (bar restated against 0.755) |
The 0.902 probe-anchor number is retained in this record as historical context; it is not the bar this ship decision was graded against.
Provenance block
| Artifact | Path / identity | md5 | Notes |
|---|---|---|---|
| feed-2k checkpointcheckpointA saved snapshot of the model weights and optimizer state during training. Mailwoman saves a checkpoint periodically so training can resume after a GPU hang. | v3.11.0-deploc-feed step-002000 | fc7ad3cb1b8982eb047e8db40ab3f7f7 | 39,411,976 bytes |
| feed-8k checkpointcheckpointA saved snapshot of the model weights and optimizer state during training. Mailwoman saves a checkpoint periodically so training can resume after a GPU hang. (SHIP CHECKPOINT, operator-ratified) | v3.11.0-deploc-feed step-008000 | 7f75fc449d8abcaa06fb6d4a5502ced7 | 39,411,976 bytes |
| v385 (shipped baseline, 6.6.0) | model-v385-latam-step-008000-int8.onnx | 700f3cf4c08f31536f0bc053c281b19e | dereferenced from the real shipped neural-weights-en-us/model.onnx |
| TokenizertokenizerThe component that converts a raw address string into a sequence of numeric token IDs the model can process. Mailwoman's tokenizer is a SentencePiece unigram model trained specifically on postal addresses. | v0.9.0-multisplice/tokenizer.model | 5c01cdcd4ae25849c5cb26b69fd3dde9 | byte-identical across all three, no tokenizertokenizerThe component that converts a raw address string into a sequence of numeric token IDs the model can process. Mailwoman's tokenizer is a SentencePiece unigram model trained specifically on postal addresses. change |
| PPD source CSV | $MAILWOMAN_DATA_ROOT/ppd/2026-07-22/gb-tuples.csv | dbd675bdc970ea60f96f0a470429a128 | 25,674,048 rows |
pair-index-gb.bin (shipped artifact) | neural-weights-en-gb/pair-index-gb.bin | header delta: 5.0 | 19,209 pairs, 457,842 bytes — matches feed-8k's calibrated δ |
| Model cardmodel cardA JSON metadata file (model-card.json) shipped with each weights bundle. It declares the model version, lineage, label set, required inference channels (anchor, gazetteer), calibration data, and training provenance. | neural-weights-en-us/model-card.json v6.7.0 | num_labels: 33 (STAGE3) | labelscomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag. unchanged from 6.6.0 |
Probe mode: "segment" throughout (the library default since d2a1242f) — every board below
uses the shared, un-overridden default.
Full battery — both checkpoints
No dropped failing rows: feed-2k's FR-fragment bare-locality FAIL stays in this table exactly
as measured.
Golden us/fr micro-F1 (v385 fresh baseline, same session)
| LocalelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for. | v385 (fresh) | feed-2k | Δ | feed-8k | Δ |
|---|---|---|---|---|---|
| us micro-F1 | 86.7% | 86.1% | −0.6pp (within ±0.7, PASS) | 86.2% | −0.5pp (within ±0.7, PASS) |
| fr micro-F1 | 89.9% | 90.8% | +0.9pp (outside magnitude, favorable) | 91.1% | +1.2pp (outside magnitude, favorable) |
FR-fragment bare-locality ≥ 0.90
| v385 (fresh) | feed-2k | feed-8k | |
|---|---|---|---|
| bare-localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. | 0.983 [0.964,0.991] | 0.665 [0.617,0.709] — FAIL | 0.988 [0.971,0.995] — PASS |
| OVERALL (composite) | 0.676 [0.659,0.693] | 0.780 [0.764,0.795] | 0.745 [0.729,0.761] |
feed-2k fails this bar outright — a 400-row board, CI entirely below 0.90. Not a marginal miss: a hallucinated-streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. regression, the same shape as the sibling en-gb-localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for.-arc's own probe-2 checkpointcheckpointA saved snapshot of the model weights and optimizer state during training. Mailwoman saves a checkpoint periodically so training can resume after a GPU hang. at its equivalent step (bare-localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. collapsed 0.978→0.603 there too). feed-8k avoids it and beats v385's own 0.983.
Digit board bare-street-hn — see the gate-revision block above
| feed-2k | feed-8k | |
|---|---|---|
| bare-streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.-hn | 0.890 [0.856, 0.917] | 0.868 [0.831, 0.897] |
| flat vs the RETIRED 0.902 probe-anchor? | YES (CI brackets it) | NO (CI excludes it) |
| vs the REVISED bar (v385 fresh 0.755) | +13.5pp | +11.3pp — PASS |
| OVERALL | 0.915 [0.903,0.926] | 0.907 [0.895,0.918] |
feed-8k's 0.868 independently reproduced across two separate grading sessions (Task 7 and Task 8), cross-validating the harness.
val macro_f1 (±1.0pp of v385's 0.7047)
- feed-8k: 0.7176 (coordinator-reported, cross-confirmed live). Δ = +1.29pp — outside the literal magnitude, favorable direction.
- feed-2k: unrecoverable. ModalModalA cloud GPU platform (modal.com) where Mailwoman trains its neural models on NVIDIA A100 GPUs. Training runs are launched via scripts/modal/train_remote.py and typically complete in ~1 hour.'s log retention had already truncated the step-2000 evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. by the time Task 7 queried it; no other doc in this repo records this run's own step-2000 number. Open item for the operator if this matters to the promotion decision.
Four dependent_locality boards, full pipeline
gb-golden via en-gb weightsparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. + prior ON @ δ*; nz-suburb-golden/es-pedania-golden/fr-lieudit-golden
via en-us control weightsparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. (no prior possible for these three — they measure the checkpointcheckpointA saved snapshot of the model weights and optimizer state during training. Mailwoman saves a checkpoint periodically so training can resume after a GPU hang.'s own
multi-localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for. resurrection, the actual reason these shardsshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. were fed into this trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input. run).
| Board | n | feed-2k emit/tag-correct | feed-8k emit/tag-correct |
|---|---|---|---|
| gb-golden (prior ON @ δ*) | 69 | 69/69, 67/69 (97.1%) | 69/69, 67/69 (97.1%) |
| gb-golden, comma-stripped (prior ON @ δ*) | 69 | 0/69, 0/69 (inert, documented v1 trade) | 0/69, 0/69 (inert, documented v1 trade) |
| nz-suburb-golden (checkpointcheckpointA saved snapshot of the model weights and optimizer state during training. Mailwoman saves a checkpoint periodically so training can resume after a GPU hang. only) | 246 | 0/246, 0/246 | 0/246, 0/246 |
| es-pedania-golden (checkpointcheckpointA saved snapshot of the model weights and optimizer state during training. Mailwoman saves a checkpoint periodically so training can resume after a GPU hang. only) | 65 | 11/65, 9/65 (13.8%) | 9/65, 7/65 (10.8%) |
| fr-lieudit-golden (checkpointcheckpointA saved snapshot of the model weights and optimizer state during training. Mailwoman saves a checkpoint periodically so training can resume after a GPU hang. only) | 80 | 0/80, 0/80 | 2/80, 1/80 (1.25%) |
GB recallrecallOf the spans whose gold label is a given tag, the fraction the model found. High recall means few misses. Paired with precision to compute F1. is tied between checkpointscheckpointA saved snapshot of the model weights and optimizer state during training. Mailwoman saves a checkpoint periodically so training can resume after a GPU hang. (67/69 both) — the same two rows miss under both
("Goulbourne Road, St Georges, Telford, TF2 9LE" — word-boundary, wrong-value; and "101 Coniston Avenue, Knott End on Sea, Poulton-le-Fylde" — pre-existing modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.'-level miss). This is
the arc's central structural finding: the pair-index prior, not the checkpointcheckpointA saved snapshot of the model weights and optimizer state during training. Mailwoman saves a checkpoint periodically so training can resume after a GPU hang.'s own
resurrected classifier row, carries nearly all of GB's dependent_locality recallrecallOf the spans whose gold label is a given tag, the fraction the model found. High recall means few misses. Paired with precision to compute F1. — see the
ablation table below. NZ shows zero decode-layerlayerOne transformer block — attention plus a feed-forward network, with normalization and residual connections — applied to every position. Stacking layers lets the model build up richer representations; Mailwoman's encoder has 6. recovery at either checkpointcheckpointA saved snapshot of the model weights and optimizer state during training. Mailwoman saves a checkpoint periodically so training can resume after a GPU hang., despite being
the arc's original target localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for. (the sibling en-gb-localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for.-arc's independent checkpointcheckpointA saved snapshot of the model weights and optimizer state during training. Mailwoman saves a checkpoint periodically so training can resume after a GPU hang. sweep
found the same shape: a hot-classifier-LR resurrection window that peaks near step 2000 and
re-buries the tag by step 8000 — docs/articles/evals/2026-07-22-night-en-gb-postmortem.md).
Venue-confound FP @ δ*
| δ* | FP | FP% | |
|---|---|---|---|
| feed-2k | 4.5 | 37/6500 | 0.569% |
| feed-8k | 5.0 | 48/6500 | 0.738% |
Both well under the retired δ=6.0 working number (122/6,500, 1.877%). Not the pre-registered FP=0 bar — the residual is a single, honestly-characterized class (a venuevenueA named, non-address place — a business, building, park, or stadium. Mailwoman's free-text point-of-interest component, added as a Tier 2 fine label. that IS, verbatim, its own bare census child) — but substantially smaller than what was previously shipped as the working number.
δ-sweep (both checkpoints, holdout tag-correct recall n=653 / venue-confound FP n=6500)
feed-2k:
| δ | holdout tag-correct | confound FP% |
|---|---|---|
| 3 | 75.7% | 0.215% |
| 4 | 93.3% | 0.369% |
| 4.5 (selected) | 96.2% | 0.569% |
| 5 | 97.7% | 0.954% |
| 6 | 98.9% | 1.877% |
| 7 | 99.2% | 2.092% |
feed-8k:
| δ | holdout tag-correct | confound FP% |
|---|---|---|
| 3 | 56.0% | 0.215% |
| 4 | 87.9% | 0.323% |
| 4.5 | 93.4% | 0.523% |
| 5.0 (selected, shipped artifact) | 96.3% | 0.738% |
| 6 | 98.2% | 1.646% |
| 7 | 98.9% | 2.062% |
FP grows monotonically with δ for both checkpointscheckpointA saved snapshot of the model weights and optimizer state during training. Mailwoman saves a checkpoint periodically so training can resume after a GPU hang. (roughly 6–10× from δ=3 to δ=7) — δ is a genuine recallrecallOf the spans whose gold label is a given tag, the fraction the model found. High recall means few misses. Paired with precision to compute F1./FP dial, not a free lunch; the smallest δ clearing the recallrecallOf the spans whose gold label is a given tag, the fraction the model found. High recall means few misses. Paired with precision to compute F1. bar is load-bearing.
6 demo presets
Both checkpointscheckpointA saved snapshot of the model weights and optimizer state during training. Mailwoman saves a checkpoint periodically so training can resume after a GPU hang., both weightsparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. worlds (en-us control path AND en-gb weightsparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. with the prior ON @ δ*), all 6 presets, vs the fresh v385-en-us baseline: ALL byte-identical, every combination.
Full error-analysis vs v385, per tag (2pp rule)
| Tag | v385 | feed-2k | Δ | feed-8k | Δ |
|---|---|---|---|---|---|
| localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. | 48.6% | 49.7% | +1.1 | 50.1% | +1.5 |
| regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. | 78.1% | 79.1% | +1.0 | 80.1% | +2.0 |
| postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. | 97.4% | 97.9% | +0.5 | 97.8% | +0.4 |
| streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. | 15.4% | 14.6% | −0.8 | 13.9% | −1.5 |
| house_number | 97.0% | 97.4% | +0.4 | 96.7% | −0.3 |
| venuevenueA named, non-address place — a business, building, park, or stadium. Mailwoman's free-text point-of-interest component, added as a Tier 2 fine label. | 37.1% | 35.4% | −1.7 | 36.0% | −1.1 |
| countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. | 89.8% | 94.3% | +4.5 | 94.3% | +4.5 |
| dependent_locality | 0.0% | 0.0% | 0.0 | 0.0% | 0.0 |
| po_boxPO boxA numbered mailbox at a post office used as a delivery address instead of a physical street location. Mailwoman tags it as the po_box component; structurally the same family as a subpremise. | 88.9% | 88.9% | 0.0 | 88.9% | 0.0 |
| Exact-match | 1150/4561 (25.2%) | 1152/4561 (25.3%) | +0.02pp | 1147/4561 (25.1%) | −0.07pp |
Worst mover: feed-2k venuevenueA named, non-address place — a business, building, park, or stadium. Mailwoman's free-text point-of-interest component, added as a Tier 2 fine label. −1.7pp; feed-8k streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. −1.5pp. No tag regresses >2pp for either checkpointcheckpointA saved snapshot of the model weights and optimizer state during training. Mailwoman saves a checkpoint periodically so training can resume after a GPU hang. — both PASS this leg cleanly.
Three-way ablation (GB dependent_locality, full pipeline)
| Condition | GB emit/tag-correct | Notes |
|---|---|---|
| v385 (pre-feed) + prior @ δ=6 (rung-3 control) | 3/69 | weightsparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. matter — even the un-fine-tuned base gets a little lift from the prior alone |
| feed (either checkpointcheckpointA saved snapshot of the model weights and optimizer state during training. Mailwoman saves a checkpoint periodically so training can resume after a GPU hang.) + prior OFF | 0/69 | prior matters — the fine-tune's own resurrection is invisible at decode without it |
| feed (either checkpointcheckpointA saved snapshot of the model weights and optimizer state during training. Mailwoman saves a checkpoint periodically so training can resume after a GPU hang.) + prior ON @ δ* | 69/69 emit, 67/69 (97.1%) tag-correct | together: +67pp over either ingredient alone |
Each ingredient is provably necessary; the fine-tune's resurrection puts the tag within δ of winning, the prior supplies the calibrated final push. Neither alone reaches production-usable recallrecallOf the spans whose gold label is a given tag, the fraction the model found. High recall means few misses. Paired with precision to compute F1..
CORRECTION + SUPERSESSION (2026-07-23, Option-A δ-sweep — later the same day). Two fixes to the table above. (1) The 3/69 "v385 + prior" row was a rung-3 hand-rolled probe at δ=6, not a production-decode measurement — the production number is 0/69 at δ=5–6's low end (δ=5: 0/69; δ=6: 6/69). (2) The conclusion "neither alone reaches production-usable recallrecallOf the spans whose gold label is a given tag, the fraction the model found. High recall means few misses. Paired with precision to compute F1." was δ-conditional and is now falsified: sweeping δ on the untouched v385 shows both curves saturate at δ=10 — 69/69 emit, 69/69 tag-correct, golden-FP 0/51 — exceeding every trained candidate in the arc, at a venuevenueA named, non-address place — a business, building, park, or stadium. Mailwoman's free-text point-of-interest component, added as a Tier 2 fine label.-confound floor of 3.338% (vs 0.738% for the feed candidate at δ=5). v385's dead-classifier deficit is large but uniform (~7.0 logitslogitA raw, unnormalized per-label score the model outputs before softmax. Priors and biases are added in logit space, then softmax turns logits into probabilities. mean), so a δ that clears it clears the whole board at once. The ablation's necessity claim survives only in this form: at δ≤6, weightsparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values.-resurrection is necessary; at δ=10, the prior alone suffices for this board, and the trade moves entirely into the venuevenueA named, non-address place — a business, building, park, or stadium. Mailwoman's free-text point-of-interest component, added as a Tier 2 fine label.-confound FP floor. Full sweep:
.superpowers/sdd/task-8-report.md§ "Option A: v385 δ-sweep".
Gauntlet — NEW finding, Task 8, 2026-07-23
Not run by Task 6 or Task 7; this task closed that gap and found a real ship blocker.
| LayerlayerOne transformer block — attention plus a feed-forward network, with normalization and residual connections — applied to every position. Stacking layers lets the model build up richer representations; Mailwoman's encoder has 6. | feed-8k verdict | Detail |
|---|---|---|
| regression | PASS | 33/33 gated cases; 1 tracked known_fail now passes (should be promoted to status=pass) |
| metamorphic | FAIL | NEW violation: INV[comma-drop] — "1600 Pennsylvania Ave NW, Washington DC 20500" → comma-dropped form loses the rooftoprooftopGeocoding precision at the building or parcel level — coordinates within a few metres — the highest tier of the geocode cascade. Sourced from address-point and situs data. resolution entirely (38.8977,-77.0365 → 0,0). v385 HOLDS this exact case (confirmed same session, same board) |
| held-out | PASS | z=0.00 (candidate not significantly worse), n=300 fresh FR/BANBAN (Base Adresse Nationale). France's authoritative open national address register — the highest-quality training source for French addresses, with full component structure. draw |
| combined | FAIL — do not ship | per the recipe's own pre-registered ship gate |
This is not checkpointcheckpointA saved snapshot of the model weights and optimizer state during training. Mailwoman saves a checkpoint periodically so training can resume after a GPU hang.-specific. feed-2k independently FAILS the same layerlayerOne transformer block — attention plus a feed-forward network, with normalization and residual connections — applied to every position. Stacking layers lets the model build up richer representations; Mailwoman's encoder has 6. with DIFFERENT
violations: BAND[num-ordinal] and INV[comma-drop]/INV[abbrev] all mis-resolve "350 Fifth Avenue, New York, NY" 283.5km away (a different stateregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. entirely). Two different checkpointscheckpointA saved snapshot of the model weights and optimizer state during training. Mailwoman saves a checkpoint periodically so training can resume after a GPU hang. from
the same trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input. lineage, two different NY/DC-class admin-resolution robustness regressions — the
pattern reads as a lineage-wide v3.11.0-deploc-feed cost, not a single-step artifact. Full
transcripts: .superpowers/sdd/task-8-prep-report.md.
Ship-checkpointcheckpointA saved snapshot of the model weights and optimizer state during training. Mailwoman saves a checkpoint periodically so training can resume after a GPU hang. choice is unaffected by this finding — the operator-ratified reasoning (feed-2k's FR-fragment failure is the more dangerous, unbounded class; feed-8k's digit miss is narrower and now covered by the gate revision) stands regardless of which checkpointcheckpointA saved snapshot of the model weights and optimizer state during training. Mailwoman saves a checkpoint periodically so training can resume after a GPU hang. also happens to fail the Gauntlet. Both do. The Gauntlet failure blocks promotion of either checkpointcheckpointA saved snapshot of the model weights and optimizer state during training. Mailwoman saves a checkpoint periodically so training can resume after a GPU hang. until triaged — it is a new, independent finding layered on top of the existing checkpointcheckpointA saved snapshot of the model weights and optimizer state during training. Mailwoman saves a checkpoint periodically so training can resume after a GPU hang. decision, not a tiebreaker between the two.
Coverage numbers
| LocalelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for./board | Rows | Source |
|---|---|---|
| GB held-out (δ-calibration) | 2,000 | PPD tail, disjoint from synth-gb-v1 (800k) + gb-golden/gb-venue-confound boards |
| GB venuevenueA named, non-address place — a business, building, park, or stadium. Mailwoman's free-text point-of-interest component, added as a Tier 2 fine label.-confound | 6,500 | FSA-sampled |
| gb-golden | 120 total, 69 dependent_locality | mailwoman/eval-harness/fixtures/gb-golden.jsonl |
| nz-suburb-golden | 246 dependent_locality | promoted NZ suburb board |
| es-pedania-golden | 65 dependent_locality | |
| fr-lieudit-golden | 80 dependent_locality | |
| synth-gb-v1 trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input. shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. | 800,000 rows, 32.8% B-dependent_locality | HM Land Registry PPD (OGL v3.0) |
| synth-nz-v2 trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input. shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. | 800,000 rows, 79.2% density | LINZ-derived OpenAddressesOpenAddresses (OA). A global open aggregation of address points collected from many official sources. A primary source of component-supervised training data outside proprietary registries. NZ (CC-BY 4.0) |
| synth-es-pedania-v1 trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input. shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. | 800,000 rows, 30.3% density | OpenAddressesOpenAddresses (OA). A global open aggregation of address points collected from many official sources. A primary source of component-supervised training data outside proprietary registries. ES (CC-BY per OA source) |
| synth-fr-lieudit-v1 trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input. shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. | 100% density (every row is a lieu-dit) | BANBAN (Base Adresse Nationale). France's authoritative open national address register — the highest-quality training source for French addresses, with full component structure. nom_ld (Licence Ouverte 2.0) |
Decision matrix (carried from Task 7, Gauntlet row added)
| Bar | feed-2k | feed-8k |
|---|---|---|
| Golden us micro ±0.7pp of v385 | PASS (−0.6pp) | PASS (−0.5pp) |
| Golden fr micro ±0.7pp of v385 | outside magnitude, favorable (+0.9pp) | outside magnitude, favorable (+1.2pp) |
FR-fragment bare-locality ≥ 0.90 | FAIL (0.665) | PASS (0.988) |
Digit bare-street-hn, revised bar (v385 fresh 0.755) | PASS (+13.5pp) | PASS (+11.3pp) |
| val macro_f1 ±1.0pp of 0.7047 | unmeasurable (log truncated) | ambiguous magnitude, favorable direction (+1.29pp) |
| GB dep-loc, full pipelinestaged pipelineMailwoman's runtime architecture: a sequence of pure-function stages (normalize → query-shape → locale-gate → kind-classifier → phrase-grouper → classifier → decoder) connected by typed handoffs. Each stage is published as its own npm package., prior ON @ δ* | PASS (67/69, 97.1%) — tie | PASS (67/69, 97.1%) — tie |
| GB dep-loc, comma-stripped | inert (0/69, documented v1 trade) — tie | inert (0/69, documented v1 trade) — tie |
| NZ dep-loc (checkpointcheckpointA saved snapshot of the model weights and optimizer state during training. Mailwoman saves a checkpoint periodically so training can resume after a GPU hang. only) | 0/246 — tie | 0/246 — tie |
| ES dep-loc (checkpointcheckpointA saved snapshot of the model weights and optimizer state during training. Mailwoman saves a checkpoint periodically so training can resume after a GPU hang. only) | 9/65 (13.8%) | 7/65 (10.8%) — feed-2k ahead |
| FR-lieudit dep-loc (checkpointcheckpointA saved snapshot of the model weights and optimizer state during training. Mailwoman saves a checkpoint periodically so training can resume after a GPU hang. only) | 0/80 | 1/80 (1.25%) — feed-8k ahead |
| VenuevenueA named, non-address place — a business, building, park, or stadium. Mailwoman's free-text point-of-interest component, added as a Tier 2 fine label.-confound FP @ own δ* | 0.569% (37/6500) | 0.738% (48/6500) — feed-2k lower |
| 6 demo presets byte-identical (both weightsparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. worlds) | PASS | PASS |
| Error-analysis, no tag >2pp | PASS (worst venuevenueA named, non-address place — a business, building, park, or stadium. Mailwoman's free-text point-of-interest component, added as a Tier 2 fine label. −1.7pp) | PASS (worst streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. −1.5pp) |
| Gauntlet (NEW, Task 8) | FAIL (BAND[num-ordinal]/INV[comma-drop]/INV[abbrev], "350 Fifth Avenue, New York, NY" 283.5km off) | FAIL (INV[comma-drop], "1600 Pennsylvania Ave NW, Washington DC 20500" loses rooftoprooftopGeocoding precision at the building or parcel level — coordinates within a few metres — the highest tier of the geocode cascade. Sourced from address-point and situs data. → 0,0) |
Ship checkpointcheckpointA saved snapshot of the model weights and optimizer state during training. Mailwoman saves a checkpoint periodically so training can resume after a GPU hang. (operator-ratified 2026-07-23): feed-8k. Reasoning carried from Task 7 — the FR-fragment miss is the more dangerous failure class (fires on any bare-localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. input, any localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for.), while feed-8k's digit miss is narrower and now clears the revised bar. GB dep-loc recallrecallOf the spans whose gold label is a given tag, the fraction the model found. High recall means few misses. Paired with precision to compute F1. doesn't differentiate the checkpointscheckpointA saved snapshot of the model weights and optimizer state during training. Mailwoman saves a checkpoint periodically so training can resume after a GPU hang. (the prior carries it, not either checkpointcheckpointA saved snapshot of the model weights and optimizer state during training. Mailwoman saves a checkpoint periodically so training can resume after a GPU hang.). Promotion itself remains blocked — not by the checkpointcheckpointA saved snapshot of the model weights and optimizer state during training. Mailwoman saves a checkpoint periodically so training can resume after a GPU hang. choice, but by the Gauntlet finding above, which applies to both candidates and was not part of the original six pre-registered guards.
Open items for the operator
- The Gauntlet metamorphic-layerlayerOne transformer block — attention plus a feed-forward network, with normalization and residual connections — applied to every position. Stacking layers lets the model build up richer representations; Mailwoman's encoder has 6. regression is unresolved. Both checkpointscheckpointA saved snapshot of the model weights and optimizer state during training. Mailwoman saves a checkpoint periodically so training can resume after a GPU hang. fail it, with different specific violations. This blocks promotion of either one until triaged — root-causing it is out of this task's scope (ship-prep, not modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' debugging).
- val macro_f1 for feed-2k remains unrecoverable from any log or doc found.
neural-weights-en-gbhas not shipped to npm. This entire arc, prior included, is pre-promotion; theruntime-flags.mdxplacetypePairrow stays Default-OFF.- If the operator ultimately promotes feed-2k instead of feed-8k, the shipped
pair-index-gb.binmust be rebuilt at--delta 4.5andneural-weights-en-gb/scripts/link-dev-weights.ts'sPAIR_INDEX_DELTAflipped back down (both one-line changes, documented inline in that file).
Postscript — consolidation verdict (2026-07-23, same day, after this scorecard)
The Gauntlet finding above was not left open indefinitely: the operator authorized ONE
pre-registered repair attempt per the arc's treadmill guard, v3.11.1-deploc-consolidate — resume
feed-8k +2k steps at a damped classifier learning ratelearning rate (LR). How big a step training takes along the gradient each update. Too high and training diverges; too low and it crawls. Mailwoman warms it up, then decays it on a cosine schedule., targeting the comma-drop regression
mechanistically (the resurrection lever that likely over-fit the comma-structured dep-loc shardsshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row.).
CONSOLIDATION VERDICT: NOT CLEAN. The comma-drop break is byte-identical between the 8k and 10k checkpointscheckpointA saved snapshot of the model weights and optimizer state during training. Mailwoman saves a checkpoint periodically so training can resume after a GPU hang. — a stable learned behavior, not trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input.-run churn that a couple more steps would wash out. Every OTHER guard still PASSES at 10k, including golden-us (at a 0.06pp margin), but the one regression this run existed to fix did not move at all.
STOP RULE EXECUTED. Per the recipe's own pre-registration ("no knob iteration inside this
pre-registration"), a second attempt was not made. The v3.11.x lineage is CLOSED for shipping —
neither feed-2k nor feed-8k nor this consolidation checkpointcheckpointA saved snapshot of the model weights and optimizer state during training. Mailwoman saves a checkpoint periodically so training can resume after a GPU hang. promotes. The currently-shipped
modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' remains v385 (6.6.0); neural-weights-en-us/model-card.json's version field was corrected
back to 6.6.0 accordingly (it had briefly carried an unshipped 6.7.0), with the intended candidate
number preserved as staged_candidate_version.
Root-cause hypothesis for the redesign: the four dependent_locality feed shardsshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. (GB/NZ/ES/
FR-lieudit) are ~100% comma-structured — every trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input. row delimits its fields with commas. This
plausibly pushed the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' toward treating a comma as load-bearing boundary evidence, at the cost
of robustness on comma-free US input (the White House / Pennsylvania Ave case) — the same coin as
this arc's own documented GB comma-stripped-input inertness trade (segmentsegmentA punctuation-bounded chunk of the normalized input — the comma-separated parts of 'Portland, OR' — used to give downstream stages structural context.-mode's v1 boundary,
gb_golden_comma_stripped above).
ARC PATH FORWARD: the placetypeplacetypeThe Who's On First hierarchical classification of places: planet → continent → country → region → county → locality → neighbourhood. The resolver uses placetype to rank candidates — an exact locality match outranks a county-level match.-pair-prior CODE ships regardless of the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' decision — the
prior, the PIX1 index, and the release-path packaging are all correct and composed correctly with
the currently-shipped v385 base (the prior is inert-but-ready: neural-weights-en-gb ships no
modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' of its own by design, so shipping the code introduces no behavior change until an operator
explicitly wires a candidate). Modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' promotion itself waits for a clean checkpointcheckpointA saved snapshot of the model weights and optimizer state during training. Mailwoman saves a checkpoint periodically so training can resume after a GPU hang. under a NEW,
separately pre-registered recipe: v3.12, sketched at
docs/superpowers/plans/2026-07-23-v312-comma-robust-recipe.md — a comma-drop augmentation share
extended to the new localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for. shardsshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. (the augmentation machinery already exists from v381), gated on
its own why-3 verification step before any GPU spend. See that document for the full pre-
registration. neural-weights-en-us/model-card.json's phase field carries this same verdict for
lockstep — read it, not just this scorecard, for the authoritative current status.
Postscript 2 (2026-07-24): the arc's actual ending — v1.1 anchored chain, shipped as 6.6.1 / 7.7.0
The correction above turned out to be the hinge. Follow-through, all zero-GPU, all pre-registered
(bars in the ledger before each run): the ship battery caught segmentsegmentA punctuation-bounded chunk of the normalized input — the comma-separated parts of 'Portland, OR' — used to give downstream stages structural context. mode fully inert on
comma-free input (0/69 — structural, not statistical); the window-mode re-sweep confirmed the
historical FP verdict and located the real flaw in the any-to-any pair GEOMETRY (79% venuevenueA named, non-address place — a business, building, park, or stadium. Mailwoman's free-text point-of-interest component, added as a Tier 2 fine label. FP at
δ=10); the brainstorm-response design (anchored adjacent-pair candidates, post-town-anchored,
probe chain segmentsegmentA punctuation-bounded chunk of the normalized input — the comma-separated parts of 'Portland, OR' — used to give downstream stages structural context.→anchored) was built and passed every bar first try: comma-stripped 50/69 emit
/ 94% tag-correct, anchored venuevenueA named, non-address place — a business, building, park, or stadium. Mailwoman's free-text point-of-interest component, added as a Tier 2 fine label. floor 4/6500 (0.062%), segmentsegmentA punctuation-bounded chunk of the normalized input — the comma-separated parts of 'Portland, OR' — used to give downstream stages structural context. path reproduced exactly through
the chain, presets + golden byte-identical, invariance profile improved (138 invariant vs 134).
Shipped: bundle 6.6.1 (δ=10 pair-index, modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' bytes unchanged) + @mailwoman/neural 7.7.0 (auto
probe chain). The trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input.-side resurrection lineage closed at five falsified mechanisms and
three stop-rule executions; the capability ships on the untouched v385. Receipts:
.superpowers/sdd/task-8-report.md §§ "Option A" through "v1.1 anchored-mode battery"; design
record 2026-07-24-pair-prior-comma-scope-{KIMI_HANDOFF,BRAINSTORM_RESPONSE}.md.
Postscript 3 (2026-07-24, night-3): country two shipped + the doors beyond dependent_locality
en-nz shipped (bundle 6.6.2 / npm 7.8.0): LINZ-derived pair index (3,134 pairs, δ=10 — the uniform-deficit shape replicates cross-countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.: NZ gap mean 7.8 logitslogitA raw, unnormalized per-label score the model outputs before softmax. Priors and biases are added in logit space, then softmax turns logits into probabilities. vs GB 7.0, same δ works), plus the identical-adjacent-segmentsegmentA punctuation-bounded chunk of the normalized input — the comma-separated parts of 'Portland, OR' — used to give downstream stages structural context. (x,x) rule for NZ's repeated-name convention ("Mangawhai, Mangawhai" — 21.6% of NZ register names carry identity pairs; GB has zero, verified). NZ board: 246/246 emit + tag-correct as-written (100%), 244/246 comma-free. E2E verified off the public registry, both probe paths. VenuevenueA named, non-address place — a business, building, park, or stadium. Mailwoman's free-text point-of-interest component, added as a Tier 2 fine label. precisionprecisionOf the spans the model labeled as a given tag, the fraction it got right. High precision means few false positives. Paired with recall to compute F1. on REAL data (6,500 Overture NZ rows, synthetic interim retired): segmentsegmentA punctuation-bounded chunk of the normalized input — the comma-separated parts of 'Portland, OR' — used to give downstream stages structural context. 3.569%, anchored 1.354% in the shipped no-anchor-bin configuration (corrected same-night from a borrowed-cache 0.862% — measurement context matters and gets documented, not buried).
The generalizationgeneralizationA trained model's performance on data unlike its training set — new regions, new input distributions. The property honest eval is designed to measure. map (the operator's "full use of the gazetteergazetteerA geographical index that maps place names and postcodes to real-world coordinates. Mailwoman uses a custom-built Who's On First (WOF) SQLite database as its gazetteer — the 'atlas' half of the grammar/atlas architecture." question): healthy-tag reassignment (namesakes) FALSIFIED — the parseaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates. was never wrong there (189/189 regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. tagging); transition-level pair evidence MEASURED OPEN — β=5 on the boundary transition recovers 13/17 of the comma-free path-fusion misses (β=8: 15/17) with zero collateral on 47 correct rows + 200 venuevenueA named, non-address place — a business, building, park, or stadium. Mailwoman's free-text point-of-interest component, added as a Tier 2 fine label. rows; graduates to a build proposal. VenuevenueA named, non-address place — a business, building, park, or stadium. Mailwoman's free-text point-of-interest component, added as a Tier 2 fine label. pairs (#1287) and streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. pairs (#1288) filed with probe designs. The GB postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one.-clip fix (#1290) rode the same night: a 41.5% silent clip rate traced to an unreachable repair pass — one codex conventions row + a card pin, exact 26→83, clips 0, all pair-prior surfaces byte-identical.
Baseline re-anchor (2026-07-24, dated — no silent drift): the comma-free GB number registered
as 50/69 was measured through an en-us-shaped harness cache (no postcode-gb.bin → anchor channel
silent). The production-faithful baseline is 55/69 emit, 52/55 (94.5%) tag-correct — the GB
postcode anchorsanchor inferenceA technique where structured knowledge (postcode locations, gazetteer place names) is injected into the model as soft input features — not as deterministic overrides. The model still decides the final labels, but the anchor signal biases it toward correct admin tags. help the decoderdecoderIn a transformer encoder-decoder model, the part that produces output sequences. Mailwoman's classifier is encoder-only (no decoder); the 'CRF decoder' is a different thing — a structured-prediction layer that picks the best label sequence from the encoder's outputs. split five additional localities. Verified not code drift
(identical-artifact reruns of both code bases reproduce 50/69 exactly under the old cache). Twice
tonight the same lesson: measure in the shipped configuration.