v0.7.2 eval — intersection-bare retrain (2026-05-30)

Verdict: EXPERIMENTAL. Best neural modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' to date and strictly dominates v0.6.0, but does not clear the ≥25% harness PROMOTE bar. Do not promote to default on the strict gate; ship as experimental. The path to ≥25% is coveragecoverageThe fraction of a population or region for which a data source has real, non-placeholder entries — e.g. 47% rooftop coverage on Texas addresses. Distinct from accuracy on the rows that are present. and the resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon., not another recipe tweak.

What v0.7.2 was

Single variable vs v0.7.1: the synth-intersectionintersectionAn address that names a location by two crossing streets ('5th & Main') rather than a number and street. Mailwoman tags the two streets as intersection_a and intersection_b — a negative-space format that starved the early model. shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. regenerated ~60% bare (no , City, ST tail) + an @ connector, weightparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. held at 0.2. v0.7.1 had proven the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' learns intersectionsintersectionAn address that names a location by two crossing streets ('5th & Main') rather than a number and street. Mailwoman tags the two streets as intersection_a and intersection_b — a negative-space format that starved the early model. (assigned-prob 0.0001 → 0.99 after the shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row.) but only in the format it trained on, so it fumbled the harness's bare X & Y cases (4/65). v0.7.2 tests whether format diversity in the shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. fixes that. Base recipe, tokenizertokenizerThe component that converts a raw address string into a sequence of numeric token IDs the model can process. Mailwoman's tokenizer is a SentencePiece unigram model trained specifically on postal addresses. (v0.6.0-a0), and everything else unchanged → F1 valid.

Primary metric — harness pass rate

Modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.'	Harness (mailwoman/test, +postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one.-repair)
v0.6.0	14.4%
v0.7.0 calibration (ls=0.1)	13.8%
v0.7.2	19.5% (81/415)
— gate bands	≤14% sidegrade · 15–24% experimental · ≥25% promote

+5.1pp over v0.6.0 — the largest single-release harness gain of the v0.6/0.7 cycle. v0 (rules) is at 93.7% on this suite, as expected (it's a PeliasPeliasAn open-source geocoder, Mailwoman's spiritual predecessor./addressit port; see the capability map — neural is the robustness layerlayerOne transformer block — attention plus a feed-forward network, with normalization and residual connections — applied to every position. Stacking layers lets the model build up richer representations; Mailwoman's encoder has 6., not a better v0).

Intersection cluster — the retrain's target

intersection.test.ts: 4/65 → 11/65 (17%). The bare-format + @ fix worked directionally (+7 assertions) but did not reach "most-of-65." Format diversity helped; it wasn't sufficient. Bare-intersectionintersectionAn address that names a location by two crossing streets ('5th & Main') rather than a number and street. Mailwoman tags the two streets as intersection_a and intersection_b — a negative-space format that starved the early model. coveragecoverageThe fraction of a population or region for which a data source has real, non-placeholder entries — e.g. 47% rooftop coverage on Texas addresses. Distinct from accuracy on the rows that are present. is still thin relative to the 65 hand-curated adversarial cases.

Per-tag gate vs v0.6.0 — PASS

Golden exact-match 21.1% → 22.0%. No tag regressed >2pp. Highlights:

Tag	RecallrecallOf the spans whose gold label is a given tag, the fraction the model found. High recall means few misses. Paired with precision to compute F1. Δ	Hallucination Δ
streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.	+2.9pp (27.7→30.5)	+29
postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one.	+0.9pp	−25
localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.	+0.5pp	−121 (much cleaner)
venuevenueA named, non-address place — a business, building, park, or stadium. Mailwoman's free-text point-of-interest component, added as a Tier 2 fine label.	+0.9pp	+13
countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.	−0.8pp (within tolerance)	−29

The streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. gain + localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.-hallucination drop are the bare-format shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. generalizing beyond intersectionsintersectionAn address that names a location by two crossing streets ('5th & Main') rather than a number and street. Mailwoman tags the two streets as intersection_a and intersection_b — a negative-space format that starved the early model. (streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. segmentation improved a little).

Capability map on v0.7.2 (all three arenas)

ArenaarenaA standardized test set probing one capability: libpostal (clean canonical), perturb (noisy and degraded), postal (edge formats). Each arena answers a different question about where rule vs neural wins.	n	v0	neural	crossover
libpostallibpostalAn open-source C address parser used by Pelias. Mailwoman's rule-based v0 and neural classifier supersede it. (clean/canonical)	69	29%	16%	rules win
corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus.-perturbation (noisy/degraded)	398	39%	61%	neural wins
postal-standards (edge formats)	38	26%	8%	rules ahead; both weak

The crossover holds on v0.7.2: clean → rules, noisy → neural. The third arenaarenaA standardized test set probing one capability: libpostal (clean canonical), perturb (noisy and degraded), postal (edge formats). Each arena answers a different question about where rule vs neural wins. (new this cycle) adds the edge dimension — by edge_class: canonical 36/7, intl-format 43/29, secondary-unitunitA subdivision of a building — apartment, suite, floor — that refines a street address. Mailwoman's unit component; a designator plus identifier forms a subpremise. 29/0, and military APO/FPOAPO/FPO (Army Post Office / Fleet Post Office). US military postal addresses for personnel stationed overseas or aboard ship. They route through military postal hubs rather than geographic locations, so they break geographic parsing assumptions., PO-box, rural-route, directional all 0/0 (coveragecoverageThe fraction of a population or region for which a data source has real, non-placeholder entries — e.g. 47% rooftop coverage on Texas addresses. Distinct from accuracy on the rows that are present. gaps shared by both parsers). A neural-only win surfaced: Dun Laoghaire, Co. Dublin (en-IE).

unit-repair (parser backlog B1) measured here

Opt-in --unit-repair post-decode pass. v0.7.2 showed unitunitA subdivision of a building — apartment, suite, floor — that refines a street address. Mailwoman's unit component; a designator plus identifier forms a subpremise.-drop has two modes:

1. Mislabel-as-localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. (Flat 2 14 Smith St → locality) — fixed by allowing ADD over localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.. Postal unitunitA subdivision of a building — apartment, suite, floor — that refines a street address. Mailwoman's unit component; a designator plus identifier forms a subpremise.-tag recallrecallOf the spans whose gold label is a given tag, the fraction the model found. High recall means few misses. Paired with precision to compute F1. 0/8 → 2/8, 0 localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. regressions.
2. Absorbed-into-streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. (STE 12 inside MAIN ST NW STE 12) — not regex-repairable; a coveragecoverageThe fraction of a population or region for which a data source has real, non-placeholder entries — e.g. 47% rooftop coverage on Texas addresses. Distinct from accuracy on the rows that are present. gap (unitsunitA subdivision of a building — apartment, suite, floor — that refines a street address. Mailwoman's unit component; a designator plus identifier forms a subpremise. underrepresented), same as intersectionsintersectionAn address that names a location by two crossing streets ('5th & Main') rather than a number and street. Mailwoman tags the two streets as intersection_a and intersection_b — a negative-space format that starved the early model. were.

Netneural networkA model made of layers of simple numeric units whose connection strengths (weights) are learned from data. The transformer encoder at Mailwoman's core is a neural network.: the repair patches the localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. half safely; the streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.-absorption half needs a coveragecoverageThe fraction of a population or region for which a data source has real, non-placeholder entries — e.g. 47% rooftop coverage on Texas addresses. Distinct from accuracy on the rows that are present. shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. (backlog B3'). Stays opt-in.

Recommendation

• Don't promote v0.7.2 to default on the strict gate (19.5% < 25%). But it strictly dominates the current default (v0.6.0): +5.1pp harness, +7 intersectionsintersectionAn address that names a location by two crossing streets ('5th & Main') rather than a number and street. Mailwoman tags the two streets as intersection_a and intersection_b — a negative-space format that starved the early model., per-tag gate clean. Shipping it as experimental (model cardmodel cardA JSON metadata file (model-card.json) shipped with each weights bundle. It declares the model version, lineage, label set, required inference channels (anchor, gazetteer), calibration data, and training provenance., not defaultVersion) is the right call — it's the best artifact we have and is A/B-able. The promote-to-default decision is the operator's.
• The 25% bar is not a recipe-tuning problem. Two cycles (v0.6.x, v0.7.x) of corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus. tweaks have moved harness 14→19.5%. The remaining gap is (a) coveragecoverageThe fraction of a population or region for which a data source has real, non-placeholder entries — e.g. 47% rooftop coverage on Texas addresses. Distinct from accuracy on the rows that are present. of the long tail — bare intersectionsintersectionAn address that names a location by two crossing streets ('5th & Main') rather than a number and street. Mailwoman tags the two streets as intersection_a and intersection_b — a negative-space format that starved the early model., streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.-less last-lines, mid-streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. unitsunitA subdivision of a building — apartment, suite, floor — that refines a street address. Mailwoman's unit component; a designator plus identifier forms a subpremise., non-US localeslocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for. (the third arenaarenaA standardized test set probing one capability: libpostal (clean canonical), perturb (noisy and degraded), postal (edge formats). Each arena answers a different question about where rule vs neural wins.'s 0% edge classes) — and (b) the resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon. (Direction B), which routes per-input on quality (rules for clean, neural for noisy) instead of grading neural on rules' home turf. Both are bigger than a shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. weightparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values..
• Next parser-backlog item: B2 countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.-repair (v0.7.2 mislabelscomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag. trailing AUSTRALIA → localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.; countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. −0.8pp) — a closed-lexicon post-decode pass, same low-risk mould as postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one./unitunitA subdivision of a building — apartment, suite, floor — that refines a street address. Mailwoman's unit component; a designator plus identifier forms a subpremise. repair.

Artifacts (volatile, /tmp/v072-eval/ + /tmp/external-arenas-v072/): harness.md, gate-report.md, evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error.-morphology.json, the three arenaarenaA standardized test set probing one capability: libpostal (clean canonical), perturb (noisy and degraded), postal (edge formats). Each arena answers a different question about where rule vs neural wins. results, postal-unitrepair-v2.json. Modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' checkpointcheckpointA saved snapshot of the model weights and optimizer state during training. Mailwoman saves a checkpoint periodically so training can resume after a GPU hang. is durable on the ModalModalA cloud GPU platform (modal.com) where Mailwoman trains its neural models on NVIDIA A100 GPUs. Training runs are launched via scripts/modal/train_remote.py and typically complete in ~1 hour. volume: output-v072-intersection/checkpoints/step-100000.