v0.6.x cycle retrospective + v0.7 plan

Iteration window: 2026-05-25 → 2026-05-29, ~4 days wall-clock, four trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input. runs (v0.6.1, v0.6.2, v0.6.2b parallel, v0.6.3). Outcome: no shippable v0.6.x release. The cycle was held after v0.6.3 empirically confirmed a corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus.-dilution failure mode. The pattern revealed by three iterations of recipe-tuning has produced a structured v0.7 plan that addresses the binding constraint (modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' calibration) and defers system-multiplicity architecture (per-localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for. routing) to v0.8.

This is the technical postmortem. The forward plan in section 5 is the synthesis of two DeepSeek consults (turns 11 and 12) plus the postcode-only diagnostic run on v0.6.0.

1. What was attempted

The cycle started after v0.6.0 shipped and v0.6.1 was found to have catastrophically regressed localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. recallrecallOf the spans whose gold label is a given tag, the fraction the model found. High recall means few misses. Paired with precision to compute F1. (-8.6pp) due to a dependent_locality over-emission spike (+1066 hallucinations on the golden setgolden setA hand-labeled evaluation dataset of US, French, and adversarial addresses used as ground truth for evals.). The cycle's goal: restore v0.6.0's quality without losing the v0.6.1 streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.-decomposition signal that motivated the synth-streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. weightparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. bump in the first place.

Run	Recipe change	dep_loc halls	localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. recallrecallOf the spans whose gold label is a given tag, the fraction the model found. High recall means few misses. Paired with precision to compute F1.	house_number recallrecallOf the spans whose gold label is a given tag, the fraction the model found. High recall means few misses. Paired with precision to compute F1.	Harness %	Verdict
v0.6.0	(baseline)	0	40.0%	79.0%	14.4%	shipped
v0.6.1	synth-streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. weightparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. 2.0	1066 ❌	31.1%	75.9%	(similar)	regressed, held
v0.6.2	+ synth-no-streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. 1.0	0 ✓	41.0%	74.0% ❌	14.0%	1 violation, hold
v0.6.2b	synth-no-streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. 0.5 (parallel)	0 ✓	41.1% (20K)	77.3% (20K)	14.7% (20K)	2 violations at 20K
v0.6.3	+ house-venuevenueA named, non-address place — a business, building, park, or stadium. Mailwoman's free-text point-of-interest component, added as a Tier 2 fine label. 1.0, no-streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. 0.5	844 ❌	34.7% ❌	77.0% ✓	12.5% ❌	3 violations, hold
v0.6.4 (not launched)	no-streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. 0.75 to fix dilution	TBD	TBD	TBD	TBD	held

The pattern: every change addresses one regression at the cost of another. v0.6.2 fixed dep_loc but cost house_number. v0.6.3 recovered house_number but reopened dep_loc + cost localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.. The synth-side machinery is fragile in a way that recipe tuning cannot stabilize.

2. Infrastructure that came out of the cycle

The cycle didn't ship a modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.', but it did ship a substantial infrastructure layerlayerOne transformer block — attention plus a feed-forward network, with normalization and residual connections — applied to every position. Stacking layers lets the model build up richer representations; Mailwoman's encoder has 6. that v0.7 will sit on top of:

• 2D pre-publish evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. gate (scripts/eval-gate.ts) — (recall drop > 2pp AND baseline > 10%) OR (hallucination spike > 100 AND rate > 20%). Retro-validated against v0.6.0 → v0.6.1 — catches the regression with 3 violations including dep_loc rate at 2665%.
• v0-vs-neural harness (scripts/harness-v0-neural.ts) — TS AST extraction of every assert(input, ...expected) from mailwoman/test/*.test.ts. 376 assertions; v0 100% vs neural 14.4% at cycle start.
• Per-tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. confidence probe — revealed the binding constraint that recipe tuning was masking: 81% of wrong predictions land at ≥ 0.9 confidence. The modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' is decisively wrong, not uncertain.
• StagestageOne of the dataflow stages in the runtime pipeline (normalize, locale gate, kind classify, phrase group, token classify, sequence correct, reconcile, resolve). Distinct from tier (model vocabulary) and phase (plan milestone). 3 fold for evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. — collapses street_prefix + street + street_suffix into a single street spanspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree. for StagestageOne of the dataflow stages in the runtime pipeline (normalize, locale gate, kind classify, phrase group, token classify, sequence correct, reconcile, resolve). Distinct from tier (model vocabulary) and phase (plan milestone). 2 golden-set comparison. Without this, v0.6.2's correct decomposition of "Main St" → streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. + suffix is double-penalized as a streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. boundary error PLUS a street_suffix hallucination.
• StreetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. morphology FSTmorphology FSTA finite-state transducer encoding street-typing affixes (Avenue, rue, Calle) from libpostal dictionaries, for morphological street recognition. — a deterministic two-pass bias that maps ~1707 libpostallibpostalAn open-source C address parser used by Pelias. Mailwoman's rule-based v0 and neural classifier supersede it. streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.-type canonicals into a street_affix placetypeplacetypeThe Who's On First hierarchical classification of places: planet → continent → country → region → county → locality → neighbourhood. The resolver uses placetype to rank candidates — an exact locality match outranks a county-level match. and biases adjacent tokenstokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. toward street and away from dependent_locality. Insufficient as a decoderdecoderIn a transformer encoder-decoder model, the part that produces output sequences. Mailwoman's classifier is encoder-only (no decoder); the 'CRF decoder' is a different thing — a structured-prediction layer that picks the best label sequence from the encoder's outputs.-only fix on v0.6.1 (modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' overconfidence > -6.0 penalty); now archived as v0.7+ shallow-fusion infrastructure.
• TrainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input.-log charts (scripts/parse-training-log.ts + training-chart.ts) — SVG line-chart generator, no dependencies. Made "same noise pattern across runs" visually rigorous instead of impressionistic.
• CorpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus. linter v1 (scripts/lint-corpus-shard.ts) — distribution outliers, labelcomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag.-vacuum, bigram collisions, anti-pattern rules. Designed to prevent the "5th Avenue Theatre" class of poisoning, where a synthetic-shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. rule emits venuevenueA named, non-address place — a business, building, park, or stadium. Mailwoman's free-text point-of-interest component, added as a Tier 2 fine label. templates that accidentally teach the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' to labelcomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag. ordinal-prefixed streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. names as venuesvenueA named, non-address place — a business, building, park, or stadium. Mailwoman's free-text point-of-interest component, added as a Tier 2 fine label..
• PostcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one.-only harness (scripts/harness-postcode.ts) — per-countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. accuracy gate. Created as the v0.7 regression fence for what should be a slam-dunk metric.

3. What v0.6.x actually revealed

3.1 The corpus-recipe knob is too coarse to stabilize the model

Three iterations of weightparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values.-tuning across synth-street, synth-no-street, and synth-house-venue produced predictable but oscillating trade-offs. Every weightparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. change addressed one regression at the cost of another. The pattern is reproducible:

• v0.6.2 dropped synth-street weightparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. and added a synth-no-street counter-example shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. at 1.0 weightparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values.. dep_loc returned to 0; house_number dropped to 74%.
• v0.6.3 added synth-house-venue at 1.0 while keeping synth-no-street at 0.5. The anti-decompose:companion ratio dropped from 1.0:0 (v0.6.2) to 0.5:1.0. dep_loc came back gradually (1 at 20K → 844 at 100K).

The recipe-knob story is wrong because it treats synthetic data as a different MIXTURE of the same distribution. It's a different DISTRIBUTION entirely. Every synthetic template imports a distributional assumption that doesn't exist in real addresses; weightparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. tuning addresses the magnitude of the mismatch but not its structure.

3.2 Per-tag recall has been the wrong release metric

The 2D gate operates on per-tag recallrecallOf the spans whose gold label is a given tag, the fraction the model found. High recall means few misses. Paired with precision to compute F1.. v0.6.2b passed the gate in isolation (dep_loc 0, localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. 41.1%, house_number 77.3% at 20K) but its harness pass rateharness pass rateWhether a full address parses end-to-end with every component correct and well-segmented and no orphan spans. Stricter than label F1 — a model can win on recall yet fail the harness if boundaries slip. was 14.7% — a 0.3pp improvement over v0.6.0's 14.4%. Three iterations of corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus. engineering produced essentially zero address-level improvement.

Per-tag recallrecallOf the spans whose gold label is a given tag, the fraction the model found. High recall means few misses. Paired with precision to compute F1. collapses 4,561 addresses into one number per tag. The failure modes it masks: tree validity (a B-localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. nested inside a B-streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. is structurally invalid but counted as correct), swap errors ("Springfield, 123 Main St" with localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. and streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. swapped scores 67% per-tag, 0% address-level), missing components ("3 out of 4 tags present" still produces a parseaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates. the resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon. can't use).

DeepSeek turn 11 changed mind from turn 7 on this point: the v0.6.2b ship recommendation was wrong because it was gated against per-tag recallrecallOf the spans whose gold label is a given tag, the fraction the model found. High recall means few misses. Paired with precision to compute F1., not the harness.

3.3 The model is overconfident, not undertrained

The per-tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. confidence probe found 81% of WRONG predictions land at ≥ 0.9 confidence on BOTH v0.6.2 and v0.6.2b. This is calibration crisis-level — the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' is decisively wrong, not uncertain. Two implications:

• It explains v0.6.x fragility. An overconfident modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' amplifies small distributional shifts (a new synth shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row.) into large prediction swings, because the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' commits hard to whatever it learned most recently. The recipe-tuning whack-a-mole pattern is the direct consequence.
• It pre-empts the CRFCRF (Conditional Random Field). A statistical modeling method that predicts structured outputs by modeling dependencies between adjacent labels. Mailwoman uses a linear-chain CRF as the Viterbi decoder at inference time to enforce BIO label consistency — a B-street must be followed by I-street or O, never I-locality. re-enablement. The fp32fp32 / fp1632-bit and 16-bit floating-point formats. Mailwoman trains in bf16 (a 16-bit variant) and exports the ONNX model in int8 for size.-CRFCRF (Conditional Random Field). A statistical modeling method that predicts structured outputs by modeling dependencies between adjacent labels. Mailwoman uses a linear-chain CRF as the Viterbi decoder at inference time to enforce BIO label consistency — a B-street must be followed by I-street or O, never I-locality. fix is verified. But enabling CRFCRF (Conditional Random Field). A statistical modeling method that predicts structured outputs by modeling dependencies between adjacent labels. Mailwoman uses a linear-chain CRF as the Viterbi decoder at inference time to enforce BIO label consistency — a B-street must be followed by I-street or O, never I-locality. on top of an overconfident modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' would make overconfidence WORSE — the CRFCRF (Conditional Random Field). A statistical modeling method that predicts structured outputs by modeling dependencies between adjacent labels. Mailwoman uses a linear-chain CRF as the Viterbi decoder at inference time to enforce BIO label consistency — a B-street must be followed by I-street or O, never I-locality. tightly constrains the transition path, so a confident wrong emission locks the whole sequence into a wrong path. Calibrate first, then add CRFCRF (Conditional Random Field). A statistical modeling method that predicts structured outputs by modeling dependencies between adjacent labels. Mailwoman uses a linear-chain CRF as the Viterbi decoder at inference time to enforce BIO label consistency — a B-street must be followed by I-street or O, never I-locality..

3.4 Postcode failures are endemic, not exotic

The postcode-only diagnostic on v0.6.0 found postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. exact-match at 75.9% overall — 80.5% US, 70.1% FR, 57% DE, 0% GB/CA/NL. The failure mechanism is visible in the data: SentencePieceSentencePieceA language-independent subword tokenizer that splits text into pieces using a unigram language model. Mailwoman uses a SentencePiece tokenizer with a 48,000-token vocabulary and byte-fallback, trained on address data rather than general text. fragments multi-tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. alphanumeric postcodespostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. into piecesECE (Expected Calibration Error). A metric that measures how well a model's confidence scores align with its actual accuracy. Lower is better. Mailwoman's held-out ECE drops from 0.067 (raw) to 0.0035 (calibrated). the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' can't spanspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree., and the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' labelscomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag. SOME of the piecesECE (Expected Calibration Error). A metric that measures how well a model's confidence scores align with its actual accuracy. Lower is better. Mailwoman's held-out ECE drops from 0.067 (raw) to 0.0035 (calibrated). as postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. but not all.

This is a tokenizertokenizerThe component that converts a raw address string into a sequence of numeric token IDs the model can process. Mailwoman's tokenizer is a SentencePiece unigram model trained specifically on postal addresses. concern, not a schema concern. Adding more JP / KR / BR coveragecoverageThe fraction of a population or region for which a data source has real, non-placeholder entries — e.g. 47% rooftop coverage on Texas addresses. Distinct from accuracy on the rows that are present. (system-multiplicity) does not fix it. v0.7 needs to address postcodespostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. at the character or FSTFST (finite-state transducer). A compact automaton that reads an input sequence and emits an output sequence. Mailwoman encodes gazetteer names and street affixes as FSTs for fast prefix matching and prior injection without search overhead. level.

3.5 Test-set leakage at the DECISION level

v0.1.2 has been the gate baseline for every v0.6.x decision. Three iterations consumed the same dataset as decision data. The threshold values in the 2D gate (2pp recallrecallOf the spans whose gold label is a given tag, the fraction the model found. High recall means few misses. Paired with precision to compute F1. drop, 100 hallucination count) were calibrated against the same set we've been reading. Some gate pass/fail decisions may have been false positives.

The modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' has never seen v0.1.2 in trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input.. But the recipe HAS been tuned against it three times. That's decision-side leakageleakageTrain/test contamination that inflates reported accuracy when eval data has effectively been seen in training. Mailwoman guards it with held-out-geography evals and locality-aware splits., even if it isn't trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input.-side.

4. What did NOT cause v0.6.x's pathology

For the record, so future-us doesn't relitigate these:

• System-multiplicity (JP block addressing, KR dual systemsexpectation-maximizationAn iterative algorithm that estimates model parameters when some variables are unobserved. In Mailwoman's matcher, EM learns the Fellegi-Sunter m and u parameters from unlabeled data — no training labels needed., etc.) did not cause v0.6.x failures. All measured regressions were on US-mostly data with US-mostly evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error.. JP/KR/Colombia are real architectural concerns for global coveragecoverageThe fraction of a population or region for which a data source has real, non-placeholder entries — e.g. 47% rooftop coverage on Texas addresses. Distinct from accuracy on the rows that are present. but were not the binding constraint here. See DeepSeek turn 12's tiertierInternal versioning of which label classes the model emits. Tier 1 is the coarse components (country, region, locality, postcode); Tier 2 adds venue, street, house_number; Tier 3 (future) would add attention, po_box, and POI venue subtyping. Historically called 'Stage 1/2/3' before the runtime-pipeline naming made that ambiguous. list.
• CRFCRF (Conditional Random Field). A statistical modeling method that predicts structured outputs by modeling dependencies between adjacent labels. Mailwoman uses a linear-chain CRF as the Viterbi decoder at inference time to enforce BIO label consistency — a B-street must be followed by I-street or O, never I-locality. disablement did not cause v0.6.x failures. v0.6.x was CE-only because of the bf16bf16 (bfloat16). A 16-bit floating-point format used during training. Half the memory of fp32 with similar numeric range, which lets the training batch fit on the lab's GPU. NaNNaN (not a number). A floating-point result for an undefined operation (log of a negative, 0/0). Appearing in the training loss usually halts the run; recovering from it follows the NaN protocol. issue in CRFCRF (Conditional Random Field). A statistical modeling method that predicts structured outputs by modeling dependencies between adjacent labels. Mailwoman uses a linear-chain CRF as the Viterbi decoder at inference time to enforce BIO label consistency — a B-street must be followed by I-street or O, never I-locality. transitions. The fp32-CRF fix is verified — but enabling CRFCRF (Conditional Random Field). A statistical modeling method that predicts structured outputs by modeling dependencies between adjacent labels. Mailwoman uses a linear-chain CRF as the Viterbi decoder at inference time to enforce BIO label consistency — a B-street must be followed by I-street or O, never I-locality. on top of v0.6.x's overconfident modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' would have amplified the failure, not fixed it.
• Modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' capacity was not the binding constraint. 29.3M params, 6 layerslayerOne transformer block — attention plus a feed-forward network, with normalization and residual connections — applied to every position. Stacking layers lets the model build up richer representations; Mailwoman's encoder has 6. × 384 hidden, trained from scratch. Whether this is undersized is unknown — but the failures we measured were calibration failures, not capacity ceilings.

5. v0.7 plan

Synthesis of DeepSeek turns 11 + 12 + postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. diagnostic. The plan addresses the binding constraint (calibration) plus the diagnostic-confirmed structural issue (postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. tokenizertokenizerThe component that converts a raw address string into a sequence of numeric token IDs the model can process. Mailwoman's tokenizer is a SentencePiece unigram model trained specifically on postal addresses.), and defers the speculative-architecture work (per-localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for. routing) to v0.8.

5.1 Binding-constraint tier list

Constraint	Evidence	v0.7 priority
Overconfidence (81% wrong @ ≥ 0.9 conf)	Measured	P0
PostcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. tokenizertokenizerThe component that converts a raw address string into a sequence of numeric token IDs the model can process. Mailwoman's tokenizer is a SentencePiece unigram model trained specifically on postal addresses. fragmentation	Diagnostic-confirmed	P1
Data imbalance (US-dominant corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus.)	Measured via harness zeros	P1
System-multiplicity / schema mismatch	Real but not cause of measured failures	P2 (v0.8)
Modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' capacity (29.3M params)	Untested	P2 (v0.8)

5.2 Three parallel workstreams

P0 — Calibration experiment (~30 min ModalModalA cloud GPU platform (modal.com) where Mailwoman trains its neural models on NVIDIA A100 GPUs. Training runs are launched via scripts/modal/train_remote.py and typically complete in ~1 hour., blocks v0.7 release)

• label_smoothing=0.1 on v0.6.0's UNCHANGED corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus..
• Single-variable change (no synth changes, no corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus. changes).
• Held-out test set: split v0.1.2 randomly 90/10 dev/test.
• Early-evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. gates against DEV; release-gate against TEST (touched once).
• Measure: harness pass rateharness pass rateWhether a full address parses end-to-end with every component correct and well-segmented and no orphan spans. Stricter than label F1 — a model can win on recall yet fail the harness if boundaries slip. (PRIMARY — replaces per-tag recallrecallOf the spans whose gold label is a given tag, the fraction the model found. High recall means few misses. Paired with precision to compute F1. as the release metric), overconfidence %, per-tag recallrecallOf the spans whose gold label is a given tag, the fraction the model found. High recall means few misses. Paired with precision to compute F1. on held-out test.
• Decision: ship if harness improves AND overconfidence drops; pivot if not.

P1 — PostcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. fix (depends on diagnostic + calibration result)

The diagnostic confirms postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. <90% on most countries (US 80.5%, FR 70.1%, GB/CA/NL 0%). Three approaches in priority order:

1. Character-level featurefeatureAn input signal a model conditions on. Beyond the raw tokens, Mailwoman feeds soft features — gazetteer-membership channels and the postcode anchor — that inform predictions without overriding them. extractor — parallel char-CNN or char-bigram embeddingembeddingA vector of numbers representing a token (or other item) so that similar items sit near each other in vector space. The first thing the model does is turn each token into an embedding. alongside subword tokenstokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words.. The postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. shape becomes visible at character level even when fragmented at subword level. Architecture change (~50KB extra params), but contained.
2. PostcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one.-aware tokenizertokenizerThe component that converts a raw address string into a sequence of numeric token IDs the model can process. Mailwoman's tokenizer is a SentencePiece unigram model trained specifically on postal addresses. pre-pass — regex detects and protects postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one.-shaped substrings (/[A-Z]\d[A-Z]?\s*\d[A-Z]\d/ etc.) before SentencePieceSentencePieceA language-independent subword tokenizer that splits text into pieces using a unigram language model. Mailwoman uses a SentencePiece tokenizer with a 48,000-token vocabulary and byte-fallback, trained on address data rather than general text. runs. Single tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words., single labelcomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag., no fragmentation. Simpler to ship than (1) but requires per-countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. patterns.
3. Per-countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. FSTFST (finite-state transducer). A compact automaton that reads an input sequence and emits an output sequence. Mailwoman encodes gazetteer names and street affixes as FSTs for fast prefix matching and prior injection without search overhead. — treat postcodespostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. like admin names; FSTFST (finite-state transducer). A compact automaton that reads an input sequence and emits an output sequence. Mailwoman encodes gazetteer names and street affixes as FSTs for fast prefix matching and prior injection without search overhead. recognizes countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.-specific shapes and biases the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' toward labeling the matched spanspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree. as postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one.. LayerlayerOne transformer block — attention plus a feed-forward network, with normalization and residual connections — applied to every position. Stacking layers lets the model build up richer representations; Mailwoman's encoder has 6. 1.5 of the shallow-fusion architecture.

DeepSeek turn 12 favors (1) + (3). Final choice depends on calibration result — if calibration alone moves postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. accuracy meaningfully upward, the tokenizertokenizerThe component that converts a raw address string into a sequence of numeric token IDs the model can process. Mailwoman's tokenizer is a SentencePiece unigram model trained specifically on postal addresses. fix becomes smaller-scope.

P2 — Pre-classifier design (gated on postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. result, may slip to v0.8)

If postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. accuracy is still < 90% after calibration + tokenizertokenizerThe component that converts a raw address string into a sequence of numeric token IDs the model can process. Mailwoman's tokenizer is a SentencePiece unigram model trained specifically on postal addresses. fix, the localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for.-router approach becomes the next lever. Recommended architecture: lexical-featurefeatureAn input signal a model conditions on. Beyond the raw tokens, Mailwoman feeds soft features — gazetteer-membership channels and the postcode anchor — that inform predictions without overriding them. classifier (regex + char n-gramsn-gramA contiguous run of n tokens (a bigram is 2, a trigram is 3). Counting n-grams is a simple, pre-neural way to model which token sequences are common. → tiny MLPfeed-forward networkThe per-token transformation inside each transformer layer that follows attention: a small two-layer network applied independently at every position.), <0.1ms latency, <100KB serialized. 99% accuracy on postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one.-bearing inputs is achievable via postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one.-pattern featuresfeatureAn input signal a model conditions on. Beyond the raw tokens, Mailwoman feeds soft features — gazetteer-membership channels and the postcode anchor — that inform predictions without overriding them. alone.

This is the entry point for v0.8's system-multiplicity work (per-localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for. containment rules, hybrid encoderencoderThe part of a transformer that turns input tokens into contextualized vector representations. Mailwoman's classifier is a small encoder-only transformer (~30M parameters). + per-system decodersdecoderIn a transformer encoder-decoder model, the part that produces output sequences. Mailwoman's classifier is encoder-only (no decoder); the 'CRF decoder' is a different thing — a structured-prediction layer that picks the best label sequence from the encoder's outputs.).

5.3 Methodology fixes (carry forward from turn 11)

• Harness pass rateharness pass rateWhether a full address parses end-to-end with every component correct and well-segmented and no orphan spans. Stricter than label F1 — a model can win on recall yet fail the harness if boundaries slip. is the primary release metric. Per-tag recallrecallOf the spans whose gold label is a given tag, the fraction the model found. High recall means few misses. Paired with precision to compute F1. becomes a diagnostic-only metric. The 2D gate stays as a secondary safety netneural networkA model made of layers of simple numeric units whose connection strengths (weights) are learned from data. The transformer encoder at Mailwoman's core is a neural network. but no longer controls the ship decision.
• Held-out test set methodology. v0.1.2 split 90/10 dev/test. Test read exactly once per release. Closes decision-side test-set leakageleakageTrain/test contamination that inflates reported accuracy when eval data has effectively been seen in training. Mailwoman guards it with held-out-geography evals and locality-aware splits..
• Synthesis-as-supplement, not synthesis-as-primary. Small targeted shardsshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row., weightparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. < 0.25, one-and-done. Real data is the primary fix for distributional gaps. v0.6.x's 100K+ row synthesis was the wrong philosophy.

5.4 Decision-tree on calibration results

Calibration	PostcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one.	Action
Improved + overconfidence dropped	< 90% (current stateregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.)	v0.7 = calibration + postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. tokenizertokenizerThe component that converts a raw address string into a sequence of numeric token IDs the model can process. Mailwoman's tokenizer is a SentencePiece unigram model trained specifically on postal addresses. fix
Improved + overconfidence dropped	≥ 90% everywhere	v0.7 = calibration only; pre-classifier → v0.8
Flat + overconfidence unchanged	< 90%	v0.7 pivots to structural; hybrid arch becomes focus
Flat + overconfidence unchanged	≥ 90% everywhere	Investigate further; calibration symptom not cause

The diagnostic settles the postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. column as < 90%, so the v0.7 action set is anchored on the top row unless calibration unexpectedly fails.

6. Long-term: where v0.8+ goes

The system-multiplicity question is real (JP block addressing, KR dual systemsexpectation-maximizationAn iterative algorithm that estimates model parameters when some variables are unobserved. In Mailwoman's matcher, EM learns the Fellegi-Sunter m and u parameters from unlabeled data — no training labels needed., Colombian Calle/Carrera grids, Costa Rican landmark- relative addresses, Brasília superquadras, Irish townlands, Icelandic farms). The current PARENT_OF containment table encodes a US/EU streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.-based grammar that doesn't fit these systemsexpectation-maximizationAn iterative algorithm that estimates model parameters when some variables are unobserved. In Mailwoman's matcher, EM learns the Fellegi-Sunter m and u parameters from unlabeled data — no training labels needed..

The v0.8+ direction (informed by DeepSeek turn 12):

• Hybrid modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' architecture. Shared encoderencoderThe part of a transformer that turns input tokens into contextualized vector representations. Mailwoman's classifier is a small encoder-only transformer (~30M parameters). (language-agnostic structural featuresfeatureAn input signal a model conditions on. Beyond the raw tokens, Mailwoman feeds soft features — gazetteer-membership channels and the postcode anchor — that inform predictions without overriding them.) + per-system decodersdecoderIn a transformer encoder-decoder model, the part that produces output sequences. Mailwoman's classifier is encoder-only (no decoder); the 'CRF decoder' is a different thing — a structured-prediction layer that picks the best label sequence from the encoder's outputs. (US, JP, KR specialist headsattention headOne of several parallel attention computations in a layer, each free to focus on a different kind of relationship between tokens. Their outputs are concatenated — 'multi-head attention'. Mailwoman uses 4 heads.). EncoderencoderThe part of a transformer that turns input tokens into contextualized vector representations. Mailwoman's classifier is a small encoder-only transformer (~30M parameters). ships once; new localeslocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for. = new decoderdecoderIn a transformer encoder-decoder model, the part that produces output sequences. Mailwoman's classifier is encoder-only (no decoder); the 'CRF decoder' is a different thing — a structured-prediction layer that picks the best label sequence from the encoder's outputs. headsattention headOne of several parallel attention computations in a layer, each free to focus on a different kind of relationship between tokens. Their outputs are concatenated — 'multi-head attention'. Mailwoman uses 4 heads..
• Per-system containment rules. PARENT_OF becomes per-system. Today's table is the US/EU default; JP table has no street parent for building_number.
• LocalelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for. pre-classifier as the router. Lexical-featurefeatureAn input signal a model conditions on. Beyond the raw tokens, Mailwoman feeds soft features — gazetteer-membership channels and the postcode anchor — that inform predictions without overriding them. classifier picks one system from a probability distribution; the matching specialist decoderdecoderIn a transformer encoder-decoder model, the part that produces output sequences. Mailwoman's classifier is encoder-only (no decoder); the 'CRF decoder' is a different thing — a structured-prediction layer that picks the best label sequence from the encoder's outputs. runs.
• AddressTreeparse treeThe hierarchical address structure produced by decoding BIO labels into spans and nesting them per the schema's containment rules (house_number → street → locality → region → country). versioning. Add system field to parsed output. JP trees keep block → sub_block → building_number; US trees keep street → house_number. Consumers opt in. Avoid lossy universal projection.

This work is meaningful but is NOT what v0.6.x failures were pointing at. v0.7 fixes the measured problemsexpectation-maximizationAn iterative algorithm that estimates model parameters when some variables are unobserved. In Mailwoman's matcher, EM learns the Fellegi-Sunter m and u parameters from unlabeled data — no training labels needed.; v0.8 starts the structural direction.

7. The meta-lesson

The v0.6.x cycle ran three full retrains chasing recipe tweaks because the surface failure mode (dep_loc hallucinations) felt like a recipe problem. The deeper failure mode (calibration crisis, decision-side test-set leakageleakageTrain/test contamination that inflates reported accuracy when eval data has effectively been seen in training. Mailwoman guards it with held-out-geography evals and locality-aware splits., wrong release metric) became visible only after the recipe-tuning iterations had exhausted plausible knobs.

Recipe tuning is fast and feels productive. It produces concrete deliverables (new trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input. runs, new charts, new error reports). But it masks deeper questions about whether the right thing is being optimized at all. The DeepSeek consult modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' — broad architectural question at a fixed cadence, designed to challenge framing rather than refine within it — was the single highest-leverage intervention of the cycle.

For the next cycle: cap the recipe-tuning budget at two iterations before forcing a methodology rethink.

See also

• Postcode-only diagnostic (v0.6.0)
• v0.6.2 step 100K eval
• v0.6.3 step 100K eval
• v0.4.0 ablation campaign — the previous retrospective; follows the same shape.
• How the model reasons — the architecture overview the v0.7 plan operates within.