Parity Scorecard — 2026-07-02 (the #885 measurement re-anchor: full re-score of the shipped 5.0.0 line)

Supersedes 2026-06-11 — the first complete per-tag scorecard since it. Same two lenses, same rules: arenaarenaA standardized test set probing one capability: libpostal (clean canonical), perturb (noisy and degraded), postal (edge formats). Each arena answers a different question about where rule vs neural wins. headattention headOne of several parallel attention computations in a layer, each free to focus on a different kind of relationship between tokens. Their outputs are concatenated — 'multi-head attention'. Mailwoman uses 4 heads.-to-headattention headOne of several parallel attention computations in a layer, each free to focus on a different kind of relationship between tokens. Their outputs are concatenated — 'multi-head attention'. Mailwoman uses 4 heads. is whole-parseaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates.-strict (honest, understatesregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. per-tag wins); per-tag F1 is what the campaigns move; real-OOD columns are the truth for campaign tags. Self-emitted from scripts/eval/promotion-gate.ts (the #479 battery: per-locale-f1.ts + score-affix.ts + score-country-homograph.ts + de-order-eval.ts + external-arenas.ts), gate spec scripts/eval/gates/v4.15.0-boundary.json — do not hand-edit.

What this is: #885's answer to R1 of the 2026-07-01 trajectory review. Since the north-star moved to the assembled coordinatecoord metricThe primary evaluation metric: distance from the resolved coordinate to the true address point. Measured at percentiles (p50, p90) and as 'within X meters.' Prevents the label-F1 trap where a model scores higher on token labels but geocodes worse., five labelcomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag.-F1 regressions shipped as "coordinate-invisible" with per-case justification but no periodic backstop. This re-score is the backstop: it re-measures every v4.4.0-gate slice against the currently shipped bytes and asks whether the deferred labelcomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag. debt stayed bounded. Verdict: it did — 17/17 floors PASS (verdict.json), and most of the ledger moved the other way. Two unsigned drifts surface and go on the record: fr.cedex_real 96.1 → 89.4 (still 19pp above floor) and the unfloored libpostallibpostalAn open-source C address parser used by Pelias. Mailwoman's rule-based v0 and neural classifier supersede it. clean-canonical arenaarenaA standardized test set probing one capability: libpostal (clean canonical), perturb (noisy and degraded), postal (edge formats). Each arena answers a different question about where rule vs neural wins. 36 → 30%.

Graded artifacts (explicit paths, md5-pinned — never the dev symlink, #259):

• int8 (the shipped bytes): model-v193a3-step-80000-int8.onnx, md5 4dec4f460a934949580d8e7b43adae7e — verified byte-identical to model.onnx inside the published @mailwoman/neural-weights-en-us@5.0.0 npm tarball (5.0.0 was the acronym-casing code major; the weightsparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. are the v4.15.0 line unchanged).
• fp32fp32 / fp1632-bit and 16-bit floating-point formats. Mailwoman trains in bf16 (a 16-bit variant) and exports the ONNX model in int8 for size. (delta reference): model-v193a3-step-80000-fp32.onnx, md5 0ae5ad20313607f31d4dfd3c649cf923 (ModalModalA cloud GPU platform (modal.com) where Mailwoman trains its neural models on NVIDIA A100 GPUs. Training runs are launched via scripts/modal/train_remote.py and typically complete in ~1 hour. output-v193a3-anchor-absorption-s42).
• tokenizertokenizerThe component that converts a raw address string into a sequence of numeric token IDs the model can process. Mailwoman's tokenizer is a SentencePiece unigram model trained specifically on postal addresses. v0.6.0-a0, ship config = anchor + gazetteergazetteerA geographical index that maps place names and postcodes to real-world coordinates. Mailwoman uses a custom-built Who's On First (WOF) SQLite database as its gazetteer — the 'atlas' half of the grammar/atlas architecture. lexicon + conventions:"auto" + spanspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree. bridge (requires_bridge), per the gate spec.

gate (config v4.15.0-boundary.json, which clones the v4.4.0 floors verbatim EXCEPT two operator-approved revisions marked *): us.postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. ≥ 95*, fr.postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. ≥ 99.3*, us.micro ≥ 81.6, us.localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. ≥ 62.2, us.regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. ≥ 80.1, us.streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. ≥ 74, street_prefix/suffix ≥ 78/67, unit_real ≥ 88, country_homograph ≥ 83.3, fr.house_number ≥ 91, fr.regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. ≥ 16.2, de.native_locality ≥ 83.8, arenaarenaA standardized test set probing one capability: libpostal (clean canonical), perturb (noisy and degraded), postal (edge formats). Each arena answers a different question about where rule vs neural wins..perturb ≥ 71, po_box_real/cedex_real ≥ 70, intersection_real ≥ 50.

* v4.15.0 gate revision (2026-06-25, operator-approved, stated coordinate justification — see the spec's $revision_2026_06_25): us.postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. 97 → 95, fr.postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. 99.5 → 99.3.

Lens 2 — per-tag truth (int8 = the shipped bytes, gaz+anchor+conventions+bridge fed)

Columns v4.3.0/v4.4.0 from their ship gates; v5.0.0 is this re-score. Δ is v5.0.0 − v4.4.0 (the last full baseline).

tag	evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error.	floor	v4.3.0	v4.4.0	v5.0.0	Δ	vs floor
street_prefix	real-affix (32-row)	78	93.6	93.6	98.0	+4.4	PASS
street_suffix	real-affix (32-row)	67	96.6	96.6	94.9	−1.7	PASS
street_prefix	NADNAD (National Address Database). A US Department of Transportation dataset of structured address points, added to the training corpus as a major source of real US addresses.-native v2 (193-row)	—	92.2	—	95.7	—	(watch)
street_suffix	NADNAD (National Address Database). A US Department of Transportation dataset of structured address points, added to the training corpus as a major source of real US addresses.-native v2 (193-row)	—	90.3	—	91.6	—	(watch)
unitunitA subdivision of a building — apartment, suite, floor — that refines a street address. Mailwoman's unit component; a designator plus identifier forms a subpremise.	real-designatorsdesignatorThe closed-vocabulary leading word of a secondary-address phrase — 'Apt', 'Suite', 'Floor', 'PO Box', 'Level' — paired with an identifier to form a complete subpremise.	88	92.1	92.1	97.0	+4.9	PASS
countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.	homographhomonymyOne surface, many referents: 'Georgia' is a US state in 'Atlanta, Georgia' and a country in 'Tbilisi, Georgia.' Handled by disambiguation, split across two stages — the parser resolves the tag from in-string context; the resolver late-binds the referent with geographic context.-real	83.3	85.1	89.8	87.5	−2.3	PASS
us.po_boxPO boxA numbered mailbox at a post office used as a delivery address instead of a physical street location. Mailwoman tags it as the po_box component; structurally the same family as a subpremise.	po-box-cedexCEDEX (Courrier d'Entreprise à Distribution Exceptionnelle). A French postal routing for high-volume business mail: a CEDEX code delivers directly from a sorting centre, bypassing the local post office. A common negative-space format Mailwoman must parse.-val	70	—	89.1	91.4	+2.3	PASS
fr.cedexCEDEX (Courrier d'Entreprise à Distribution Exceptionnelle). A French postal routing for high-volume business mail: a CEDEX code delivers directly from a sorting centre, bypassing the local post office. A common negative-space format Mailwoman must parse.	po-box-cedexCEDEX (Courrier d'Entreprise à Distribution Exceptionnelle). A French postal routing for high-volume business mail: a CEDEX code delivers directly from a sorting centre, bypassing the local post office. A common negative-space format Mailwoman must parse.-val	70	—	96.1	89.4	−6.7 ⚠	PASS
us.intersectionintersectionAn address that names a location by two crossing streets ('5th & Main') rather than a number and street. Mailwoman tags the two streets as intersection_a and intersection_b — a negative-space format that starved the early model.	intersectionintersectionAn address that names a location by two crossing streets ('5th & Main') rather than a number and street. Mailwoman tags the two streets as intersection_a and intersection_b — a negative-space format that starved the early model.-real	50	0	100	100	0	PASS
us.streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. (folded)	golden dev	74	75.5	77.9	82.3	+4.4	PASS
us.localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.	golden dev	62.2	74.4	75.7	76.7	+1.0	PASS
us.regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.	golden dev	80.1	89.1	90.3	88.6	−1.7	PASS
us.postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one.	golden dev	95*	97.8	98.3	95.0	−3.3 †	PASS (0.005 margin)
us.micro	golden dev	81.6	85.1	86.1	85.7	−0.4	PASS
fr.postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one.	golden dev	99.3*	99.7	99.6	99.3	−0.3 †	PASS (0.04 margin)
fr.house_number	golden dev	91	97.7	97.2	98.1	+0.9	PASS
fr.regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.	golden dev	16.2	16.2	25.6	48.4	+22.8	PASS
de.native_locality	de-order (anchor on)	83.8	90.1	91.0	91.1	+0.1	PASS
arenaarenaA standardized test set probing one capability: libpostal (clean canonical), perturb (noisy and degraded), postal (edge formats). Each arena answers a different question about where rule vs neural wins..perturb	perturb arenaarenaA standardized test set probing one capability: libpostal (clean canonical), perturb (noisy and degraded), postal (edge formats). Each arena answers a different question about where rule vs neural wins.	71	64	72	78	+6	PASS

† The two documented v4.15.0 gate revisions — the drop is the priced-in #723 trade, not new drift. Note both now sit exactly at their revised floors (us.postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. 95.005/95.0, fr.postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. 99.341/99.3): the floors have zero slack left, which is the correct design (the floor IS the shipped level) but means any future hair of postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. lossloss functionA number measuring how wrong the model's predictions are on a batch of examples. Training minimizes it. Mailwoman's loss combines per-token negative log-likelihood with the CRF sequence loss. fails the gate loudly.

⚠ fr.cedex_real 96.1 → 89.4 is the one >2pp real-OOD move with no written justification in any promotion doc between v4.4.0 and v4.15.0 — the exact pattern #885 exists to catch. It remains 19.4pp above its floor, so nothing gates on it today; it is now on the record as accumulated drift, not silently absorbed. (Plausible source: the multi-localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for./AU/anchor retrains between 06-11 and 06-25 rebalancing FR postal-format mass; diagnosing is a follow-up, not this doc's job.)

fp32fp32 / fp1632-bit and 16-bit floating-point formats. Mailwoman trains in bf16 (a 16-bit variant) and exports the ONNX model in int8 for size. ↔ int8: max per-tag delta 0.8pp, on fr.regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. (cap 1.5) — quantization is not distorting any floor.

Lens 1 — capability arenas (int8, TRUE ship config, whole-parse-strict)

arenaarenaA standardized test set probing one capability: libpostal (clean canonical), perturb (noisy and degraded), postal (edge formats). Each arena answers a different question about where rule vs neural wins.	n	v0	v4.3.0	v5.0.0
libpostallibpostalAn open-source C address parser used by Pelias. Mailwoman's rule-based v0 and neural classifier supersede it. (clean/canonical)	69	29%	36%	30%
perturb (noisy/degraded)	398	39%	64%	78%
postal (edge formats)	38	26%	13%	24%

The perturb arenaarenaA standardized test set probing one capability: libpostal (clean canonical), perturb (noisy and degraded), postal (edge formats). Each arena answers a different question about where rule vs neural wins. — floored at 71 since v4.4.0 because the v4.3.0 dip was a real regression — clears its floor by 7pp (64 → 78 since v4.3.0). The postal (edge-format) arenaarenaA standardized test set probing one capability: libpostal (clean canonical), perturb (noisy and degraded), postal (edge formats). Each arena answers a different question about where rule vs neural wins. nearly doubled (13 → 24%): the po_boxPO boxA numbered mailbox at a post office used as a delivery address instead of a physical street location. Mailwoman tags it as the po_box component; structurally the same family as a subpremise./cedexCEDEX (Courrier d'Entreprise à Distribution Exceptionnelle). A French postal routing for high-volume business mail: a CEDEX code delivers directly from a sorting centre, bypassing the local post office. A common negative-space format Mailwoman must parse./intersectionintersectionAn address that names a location by two crossing streets ('5th & Main') rather than a number and street. Mailwoman tags the two streets as intersection_a and intersection_b — a negative-space format that starved the early model. shardsshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. reach whole-parseaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates. strictness too. The libpostallibpostalAn open-source C address parser used by Pelias. Mailwoman's rule-based v0 and neural classifier supersede it. (clean/canonical) arenaarenaA standardized test set probing one capability: libpostal (clean canonical), perturb (noisy and degraded), postal (edge formats). Each arena answers a different question about where rule vs neural wins. dipped 36 → 30% — the neural parser now edges v0 there (30 vs 29) but the whole-parseaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates.-strict rate on canonical inputs drifted down across the multi-localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for. releases. Unfloored, flagged-not-gated (the same hygiene as the 06-11 perturb note): if a future release wants to gate it, this is the baseline to floor.

The assembled coordinate (the north star, for anchor)

Labelcomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag.-F1 above is the drift backstop, not the verdict. The shipped line's coordinate record, measured this week on the same artifact (see #884 and the day eval): US-2k coord p50 3.31 km / resolve 1.000 / regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. 0.999; CZ-1k resolved-p50 3.29 km / resolve 0.968; PL-1k 2.07 km / 0.985 (the CZ/PL wrong-citylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. defect is #884's fix, promote-pending).

What this re-score settles (and what it leaves for the operator)

1. The "coordinate-invisible debt" is bounded and mostly positive. Of the five deferred labelcomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag. regressions, the two postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. floors are priced-in revisions sitting exactly at floor; us.regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. −1.7 and street_suffix −1.7 are inside noise-and-floor margins; and the same period bought streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. +4.4, unitunitA subdivision of a building — apartment, suite, floor — that refines a street address. Mailwoman's unit component; a designator plus identifier forms a subpremise. +4.9, prefix +4.4, po_boxPO boxA numbered mailbox at a post office used as a delivery address instead of a physical street location. Mailwoman tags it as the po_box component; structurally the same family as a subpremise. +2.3, fr.regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. +22.8, and +6 on the gated arenaarenaA standardized test set probing one capability: libpostal (clean canonical), perturb (noisy and degraded), postal (edge formats). Each arena answers a different question about where rule vs neural wins.. The pattern (grade the coordinate, document labelcomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag. trades) is vindicated — with two uncaptured drifts now recorded: fr.cedex_real −6.7 and the libpostallibpostalAn open-source C address parser used by Pelias. Mailwoman's rule-based v0 and neural classifier supersede it. clean-arenaarenaA standardized test set probing one capability: libpostal (clean canonical), perturb (noisy and degraded), postal (edge formats). Each arena answers a different question about where rule vs neural wins. −6.
2. Ledger fate — DECIDED (operator, 2026-07-02): revived, automated. This re-score is the v5.0.0 row in evals/scores-by-version.json (appended via the new scripts/eval/ledger-append.ts; promotion-gate.ts prints the pre-filled append command on every PASS, so the update no longer depends on discipline). The 4.5–4.16 gap stays unpopulated and is documented in the row's notes.
3. Standing re-score rule — APPROVED (operator, 2026-07-02) and written into CONTRIBUTING_MODEL_WORK.mdx: every 5 promotes, or any promote that lowers a gate floor, triggers a full re-score published as a dated scorecard. v4.4.0 → v4.15.0 was 11 promotes with none; this doc is the debt paid once.

Provenance

• Battery out-dir: /mnt/playpen/mailwoman-data/scratch-825/rescore-885/ (verdict.json, per-leg JSON + md, provenance.txt with artifact md5s).
• Runner: scripts/eval/promotion-gate.ts @ branch feat/885-measurement-reanchor; NADNAD (National Address Database). A US Department of Transportation dataset of structured address points, added to the training corpus as a major source of real US addresses.-native v2 row from a standalone score-affix.ts --file street-affix-real-v2.jsonl run (not a battery leg).
• Known battery defect found during this run: #887 — de-order-eval's anchor-OFF ablation column is broken by the #718 fail-closed scorer gate (empty-anchor idiom refused). The anchor-ON leg that grades the de.native_locality floor is unaffected.