Parity Scorecard — 2026-06-11 (baseline: v4.3.0, the relabel + conventions release)
Supersedes 2026-06-10. Same two lenses, same rules:
arenaarenaA standardized test set probing one capability: libpostal (clean canonical), perturb (noisy and degraded), postal (edge formats). Each arena answers a different question about where rule vs neural wins. headattention headOne of several parallel attention computations in a layer, each free to focus on a different kind of relationship between tokens. Their outputs are concatenated — 'multi-head attention'. Mailwoman uses 4 heads.-to-headattention headOne of several parallel attention computations in a layer, each free to focus on a different kind of relationship between tokens. Their outputs are concatenated — 'multi-head attention'. Mailwoman uses 4 heads. is whole-parseaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates.-strict (honest, understatesregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. per-tag wins); per-tag F1 is
what the campaign moves; real-OOD columns are the truth for campaign tags. Self-emitted
from external-arenas.sh + per-locale-f1.ts + the real-OOD scorers — do not hand-edit.
What changed since 06-10: the #492 ladder closed (cause = a 1,039:1 labelcomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag. contradiction,
NOT capacity), the #511 relabel run shipped as v4.3.0 after a FAIL→corrective→PASS gate
(2026-06-11-v4.3.0-ship-gate.md), and the conventions layerlayerOne transformer block — attention plus a feed-forward network, with normalization and residual connections — applied to every position. Stacking layers lets the model build up richer representations; Mailwoman's encoder has 6. shipped its first slice (#478:
localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for. headattention headOne of several parallel attention computations in a layer, each free to focus on a different kind of relationship between tokens. Their outputs are concatenated — 'multi-head attention'. Mailwoman uses 4 heads. exported + fr mask). Ship config now includes addressSystemConventions: "auto".
Lens 1 — capability arenas (v4.3.0 int8, TRUE ship config: anchor + gaz + conventions fed)
| arenaarenaA standardized test set probing one capability: libpostal (clean canonical), perturb (noisy and degraded), postal (edge formats). Each arena answers a different question about where rule vs neural wins. | n | v0 | v4.2.0 | v4.3.0 | both | neural-only | v0-only | both-fail |
|---|---|---|---|---|---|---|---|---|
| libpostallibpostalAn open-source C address parser used by Pelias. Mailwoman's rule-based v0 and neural classifier supersede it. (clean/canonical) | 69 | 29% | 41% | 36% | 16% | 20% | 13% | 51% |
| perturb (noisy/degraded) | 398 | 39% | 71% | 64% | 34% | 30% | 5% | 31% |
| postal (edge formats) | 38 | 26% | 18% | 13% | 5% | 8% | 21% | 66% |
Recorded dip, characterized: the perturb drop vs v4.2.0 is dominated by the "glue" perturbation class (regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.+postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. fused:
NY14201— v1.1.0 swallows the tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. as postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one., ~24–31 of 40 flips), plus streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.-boundary wobbles (post-directionalNW,Main St Apt). Same boundary-instability family as the FR digit-split the conventions maskconventions maskA decode-time constraint layer keyed by the model's own address-system detection (the exported locale head): tags that are ungrammatical in the detected system are removed from the Viterbi vocabulary, and the system's postcode shape arms a snap-only repair pass. The first slice forbids USPS street-affix decomposition for French. Same knowledge-outside-the-weights property as the gazetteer anchor — add a codex conventions row, no retrain. fixed; the US glue class needs a trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input.-side augmentation (follow-up filed). Flagged-not-gated, same precedent as night-10. v0 still leads only on rare edge formats.
Lens 2 — per-tag truth (int8, gaz+anchor+conventions fed)
| tag | evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. | v4.1.0 | v4.2.0 | v4.3.0 |
|---|---|---|---|---|
| street_prefix | real-affix (32-row) | 0 | 64.9 | 93.6 |
| street_suffix | real-affix (32-row) | 0 | 48.8 | 96.6 |
| street_prefix | NADNAD (National Address Database). A US Department of Transportation dataset of structured address points, added to the training corpus as a major source of real US addresses.-native v2 (193-row, NEW) | — | 18.2 | 92.2 |
| street_suffix | NADNAD (National Address Database). A US Department of Transportation dataset of structured address points, added to the training corpus as a major source of real US addresses.-native v2 (193-row, NEW) | — | 8.9 | 90.3 |
| unitunitA subdivision of a building — apartment, suite, floor — that refines a street address. Mailwoman's unit component; a designator plus identifier forms a subpremise. | real-designatorsdesignatorThe closed-vocabulary leading word of a secondary-address phrase — 'Apt', 'Suite', 'Floor', 'PO Box', 'Level' — paired with an identifier to form a complete subpremise. | 92.3 | 90.6 | 92.1 |
| countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. | homographhomonymyOne surface, many referents: 'Georgia' is a US state in 'Atlanta, Georgia' and a country in 'Tbilisi, Georgia.' Handled by disambiguation, split across two stages — the parser resolves the tag from in-string context; the resolver late-binds the referent with geographic context.-real | 27 | 89.8¹ | 85.1 |
| us.streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. (folded) | golden dev | 78.5 | 76.2 | 75.5 |
| us.localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. | golden dev | 60.1 | 72.9 | 74.4 |
| us.regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. | golden dev | 78.4 | 89.1 | 89.1 |
| us.postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. | golden dev | 98.3 | 97.3 | 97.8 |
| us.micro | golden dev | 81.6 | 84.8 | 85.1 |
| fr.postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. | golden dev | 99.5 | 99.6 | 99.7 |
| fr.house_number | golden dev | 91.0 | 94.6 | 97.7 |
| fr.regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. | golden dev | 30.2 | 27.6 | 16.2 |
| de.native_locality | de-order (anchor on) | 90.6 | 90.9 | 90.1 |
¹ v4.2.0's countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. figure was measured under the historically gaz-starvedstarvedA tag with too little training representation to learn — near-zero F1 — because the corpus adapter never emits examples of it (intersection tags sat at 0% until an intersection synthesizer existed). countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. leg (fixed this gate) — not directly comparable to v4.3.0's 85.1; both clear the 83.3 floor.
The authoritative gate record, including the honest-evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. VT leg (regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. 99.6, coord p50/p90 3.4/7.4 km) and the FAIL→corrective→PASS story: v4.3.0 ship gate.
Open per-tag gaps after this release: po_boxPO boxA numbered mailbox at a post office used as a delivery address instead of a physical street location. Mailwoman tags it as the po_box component; structurally the same family as a subpremise./cedexCEDEX (Courrier d'Entreprise à Distribution Exceptionnelle). A French postal routing for high-volume business mail: a CEDEX code delivers directly from a sorting centre, bypassing the local post office. A common negative-space format Mailwoman must parse. (deferred coveragecoverageThe fraction of a population or region for which a data source has real, non-placeholder entries — e.g. 47% rooftop coverage on Texas addresses. Distinct from accuracy on the rows that are present. lever — both Montréal gate rows), FR regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. tail (27.6 → 16.2, unfloored), the US glue/post-directional boundary classes (arenaarenaA standardized test set probing one capability: libpostal (clean canonical), perturb (noisy and degraded), postal (edge formats). Each arena answers a different question about where rule vs neural wins. dip above), intersectionsintersectionAn address that names a location by two crossing streets ('5th & Main') rather than a number and street. Mailwoman tags the two streets as intersection_a and intersection_b — a negative-space format that starved the early model. (#487, needs corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus. rows).