Skip to main content

Parity-corpus GOLD triage proposal

Audit of mailwoman/eval-harness/fixtures/parity-corpus.jsonl (354 live fixtures — the 22 dropped no-solution rows are excluded already and out of scope here). No fixtures were edited; this is a proposal only, per the no-silent-gate-drift rule. Every removal below carries a written reason and a rubric letter (a/b/c per the audit brief). Conservative default: keep unless a concrete defect is identified.

Scope covered: all 85 live country=ZZ fixtures (sources compound_street, functional, intersection, transit, libpostal), every fixture with a digit-leading gold street value (found only in address.deu.test.ts, the rubric's own worked example), and a full read of the remaining 269 fixtures for stray (a)/(c) violations — which surfaced one more cluster in place.fra.test.ts.

Gate mechanics checked against mailwoman/eval-harness/parity-corpus.ts: only house_number, postcode, and the street family are pre-registered floors; region/locality/venue/ country only feed an informational "full-agree" stat, never a gate. Each fixture contributes at most one slot per labelcomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag. regardless of how many values are in that labelcomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag.'s gold array (e.g. an intersectionintersectionAn address that names a location by two crossing streets ('5th & Main') rather than a number and street. Mailwoman tags the two streets as intersection_a and intersection_b — a negative-space format that starved the early model.'s two-streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. array is one street comparison, not two). The arithmetic below uses that same accounting.

Proposed tombstones (33 fixtures)

idinputgold summaryrubricreason
v1-compound_street-2Foostraße RdstreetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.: Foostraße (trailing "Rd" discarded entirely)(b)compound_street.test.ts's own comment says the point is "should not attach a second suffix" — a solver double-suffix-rejection drill. Mixing a terminal German compound suffix ("-straße") with an English type abbreviation ("Rd") on the same name isn't a real-world address form in any convention; it's a synthetic probe.
v1-compound_street-3foo st andstreetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.: "foo st" (dangling "and" discarded)(b)Tests that a trailing, unpaired intersectionintersectionAn address that names a location by two crossing streets ('5th & Main') rather than a number and street. Mailwoman tags the two streets as intersection_a and intersection_b — a negative-space format that starved the early model.-connector tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. doesn't get absorbed into the streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. name or trigger a spurious intersectionintersectionAn address that names a location by two crossing streets ('5th & Main') rather than a number and street. Mailwoman tags the two streets as intersection_a and intersection_b — a negative-space format that starved the early model. parseaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates.. "foo st and" is not a string anyone would submit as an address — it's a solver edge-case probe, same family as intersection.test.ts.
v1-functional-14..171 Foo St N / S / E / Whouse_number: 1, streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.: "Foo St" (trailing single-letter directional silently dropped)(a)+(c)The file's own comment says this exists because "trailing directional causes issue with autocomplete" — an incremental-typing (keystroke) concern from the old rules enginerule-based classifierMailwoman's legacy v0 parser — a library of deterministic token classifiers (house number, street suffix, postcode, place name, etc.) composed by priority. Now primarily used for corpus labeling, fallback classification, and arbitration diagnostics., not a linguistic judgment about a complete string. Given the whole final string (not a partial autocomplete stateregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.), a competent annotator keeps the directional as part of the streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. name — "123 Elm St N" is an ordinary US post-directional. Inconsistent with the SAME file: two-letter directionals (v1-functional-18..21, "Foo St NW/NE/SW/SE") are correctly kept as part of the streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels., and spelled-out directionals (v1-functional-9, "1 north main blvd"; v1-functional-10, "1 main blvd north") are also kept — only the single-letter abbreviated case vanishes, an orthographic-length distinction with no linguistic basis.
v1-intersectionintersectionAn address that names a location by two crossing streets ('5th & Main') rather than a number and street. Mailwoman tags the two streets as intersection_a and intersection_b — a negative-space format that starved the early model.-19..22at street / corner street / carrer en / carrer constreetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.: the 2-word input verbatim(b)Comment: "should correctly parseaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates. streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. names containing an intersectionintersectionAn address that names a location by two crossing streets ('5th & Main') rather than a number and street. Mailwoman tags the two streets as intersection_a and intersection_b — a negative-space format that starved the early model. tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words.." This is an enumerated false-positive guard over the connector-tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. dictionary (at/corner/en/con) across languages, not organic address content — none of these 2-word fragments is a complete real streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. name/address on its own.
v1-intersectionintersectionAn address that names a location by two crossing streets ('5th & Main') rather than a number and street. Mailwoman tags the two streets as intersection_a and intersection_b — a negative-space format that starved the early model.-23..38foo & bar / foo and bar / foo at bar / foo @ bar / main st & side ave / main st & side / main & side ave / main & side / 1st st & 2nd ave / 1st st & 2nd / 1st & 2nd ave / 1st & 2nd / 1 st & 2 ave / 1 st & 2 / 1 & 2 ave / 1 & 2streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.: two-item array per fixture(b)Explicit comment headers ("no street suffixstreet affixA modifier on a street name indicating type or direction — Street, Avenue, rue, Calle, N, East. Mailwoman tags these as street_prefix / street_suffix, recognized via a morphology FST.", "missing street suffixstreet affixA modifier on a street name indicating type or direction — Street, Avenue, rue, Calle, N, East. Mailwoman tags these as street_prefix / street_suffix, recognized via a morphology FST. - alpha/ordinal/cardinal") mark this as a systematic 4-way combinatorial matrix (connector type × suffix-presence on each side), using placeholder words ("foo"/"bar"/"main"/"side") chosen purely for coveragecoverageThe fraction of a population or region for which a data source has real, non-placeholder entries — e.g. 47% rooftop coverage on Texas addresses. Distinct from accuracy on the rows that are present. — textbook solver-permutation testing, not harvested real intersectionsintersectionAn address that names a location by two crossing streets ('5th & Main') rather than a number and street. Mailwoman tags the two streets as intersection_a and intersection_b — a negative-space format that starved the early model.. Contrast with the genuinely real intersectionsintersectionAn address that names a location by two crossing streets ('5th & Main') rather than a number and street. Mailwoman tags the two streets as intersection_a and intersection_b — a negative-space format that starved the early model. a few lines later in the SAME file that are kept (v1-intersection-39 "SW 6th & Pine", v1-intersection-40 "9th and Lambert", v1-intersection-41 "filbert & 32nd" — specific, attested streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. names, no combinatorial comment header).
v1-address.deu-14..1725 Straße 50 / 25 Straße, 50 / 25 Strada 50 / 25 Strada, 50streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.: "25 Straße"/"25 Strada", house_number: "50"(a)This is the rubric's own worked example. "Straße"/"Strada" are the bare generic nouns for "streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels." in German/Italian-Romanian — not proper streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. names in any convention. No competent annotator reading "25 Straße 50" as a complete string would call "25 Straße" a streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. NAME; the generic word is standing in as a placeholder proper noun to drill an ambiguous-numbered-streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. permutation (with/without comma, two placeholder languages). Comment: "addresses on numbered streetsstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. in europe" — the intent doesn't require this particular (arguably backwards) segmentation.
v1-place.fra-11CC des Fours à Chaux MontluçonstreetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.: "CC des Fours à Chaux", localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.: Montluçon(a)+(c)"CC" = Centre Commercial (shopping center), a venuevenueA named, non-address place — a business, building, park, or stadium. Mailwoman's free-text point-of-interest component, added as a Tier 2 fine label./POIpoint of interest (POI). A named place that is not strictly an address — landmark, transit stop, venue, amenity, or franchise. Mailwoman tags these as venue and resolves them through the gazetteer. abbreviation, not a French voie-type. Inconsistent with the SAME file's identical "abbreviation + descriptive name + localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy." pattern tagged venuevenueA named, non-address place — a business, building, park, or stadium. Mailwoman's free-text point-of-interest component, added as a Tier 2 fine label. elsewhere: v1-place.fra-4 "ZAC de la Tuilerie", -6 "ZI les grasses", -9 "ZA Entraigues". The in-file comment ("This should be streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. in French, but it's ok") itself signals author uncertainty.
v1-place.fra-12Parc Des Clots UpiestreetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.: "Parc Des Clots", localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.: Upie(a)+(c)"Parc" = Park — a venuevenueA named, non-address place — a business, building, park, or stadium. Mailwoman's free-text point-of-interest component, added as a Tier 2 fine label., not a thoroughfare, by ordinary reading. Inconsistent with the same file's École/Bibliothèque/Université entries, all tagged venuevenueA named, non-address place — a business, building, park, or stadium. Mailwoman's free-text point-of-interest component, added as a Tier 2 fine label.. Same boilerplate-doubt comment as the CC fixture above.
v1-place.fra-13ZAC du centre Bourg Saint-Sébastien-De-MorsentstreetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.: "ZAC du centre" (drops "Bourg"), localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.: Saint-Sébastien-De-Morsent(a)Self-admitted defect: the in-file comment reads verbatim "TODO: the place should be ZAC du centre Bourg" — the author flags that the pinned gold is known-wrong (wrong spanspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree. AND wrong tag family: should be venuevenueA named, non-address place — a business, building, park, or stadium. Mailwoman's free-text point-of-interest component, added as a Tier 2 fine label., per every other ZAC/ZI/ZA fixture in this file, not streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.).

Arithmetic (triaged-gold denominator)

  • Fixtures proposed for tombstone: 33 (of 354 live)
  • street gold slots removed: 33 (every proposed fixture has a non-empty street value; intersectionintersectionAn address that names a location by two crossing streets ('5th & Main') rather than a number and street. Mailwoman tags the two streets as intersection_a and intersection_b — a negative-space format that starved the early model. two-value arrays still count once each, matching the gate's own per-fixture accounting)
  • house_number gold slots removed: 8 (v1-functional-14..17 × 4, v1-address.deu-14..17 × 4)
  • postcode gold slots removed: 0 (no proposed fixture carries a postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. value)

Triaged floor denominators, if adopted: street 33 fewer than today's live total; house_number 8 fewer; postcode unchanged.

Borderline — considered, kept (21 fixtures across 7 items)

id(s)inputwhy it was consideredwhy it was kept
v1-functional-5, v1-functional-810 ave, 100 10 aveBare cardinal number as streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. name, no ordinal suffix — could read as a rules-only permutation slot ("streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. cardinal" vs. "streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. ordinal" pairing with 10th ave).Numbered streetsstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. written without the ordinal suffix are a real, if informal, real-world form (truncated/OCR'd data, casual usage). Hard-but-fair, not manufactured.
v1-functional-12West 26th Street, New York, NYC, 10010Input redundantly repeats the citylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. as both "New York" and "NYC"; gold silently drops "NYC" with no comment explaining the choice.Plausible as a genuine (if redundant) user-typed query; dropping a duplicate same-referent tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. is defensible dedup behavior, not obviously arbitrary. No comment flags it as a known defect.
v1-intersectionintersectionAn address that names a location by two crossing streets ('5th & Main') rather than a number and street. Mailwoman tags the two streets as intersection_a and intersection_b — a negative-space format that starved the early model.-7Gleimstraße an der ecke von Schönhauser AlleePhrasing is unusually verbose for German ("at the corner of" calqued rather than the idiomatic "Ecke X/Y"); reads as constructed to exercise the connector dictionary's German-language entry.Grammatically valid German with real streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. names (not placeholders); plausible as spoken-register phrasing. Insufficiently confident to propose removal.
v1-address.deu-13Eberswalder Straße 100 104Comment says "autocomplete-style query includes partial postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one." — an explicit rules/autocomplete-era artifact.The graded slots (street, house_number) are both correct and unaffected; postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. isn't asserted at all, so the dangling "104" is simply unchecked, not misgraded. Harmless to the floors as currently defined.
v1-place.fra-10Place Sohier VervinsShares the identical boilerplate doubt-comment ("This should be streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. in French, but it's ok") with the two CC/Parc fixtures proposed above.Linguistically distinct from those two: "Place" (square) is a legitimate French voie-type used in real addressing (BANBAN (Base Adresse Nationale). France's authoritative open national address register — the highest-quality training source for French addresses, with full component structure./La Poste treat it as a thoroughfare designatordesignatorThe closed-vocabulary leading word of a secondary-address phrase — 'Apt', 'Suite', 'Floor', 'PO Box', 'Level' — paired with an identifier to form a complete subpremise.), unlike "CC" (shopping center) or "Parc" (park). Tagging it streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. is defensible on its own facts, independent of the shared comment.
v1-address.usa-16..30N FISKE AVE PN DWIGHT AVE Portland Oregon (15 fixtures)Source comment self-identifies this block as "autocomplete streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. name jitter" — a keystroke-by-keystroke simulation of a partially-typed query, with regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality./localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. flipping unpredictably as characters are appended (e.g. "Por"→regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality., "Port"→localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy., "Portl"→neither). Classic rules-engine mechanics, arguably (b).The only PARITY_FLOORS-graded slot present is street, and it is stable and correct ("N FISKE AVE" / "N DWIGHT AVE") in every variant — which is in fact the comment's stated intent ("we are only testing the streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. name stays the same throughout"). The jittery regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality./localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. assignments are real but land in fields the gate doesn't grade (informational "full-agree" stat only). Kept because removing them would not change any floor-slot count and the one graded field is sound throughout; flagged here for visibility rather than proposed.

Confidence note

High confidence on the digit-leading-streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. DEU block (rubric's own worked example, and the generic-noun-as-proper-noun pattern is unambiguous) and on the place.fra TODO fixture (self-admitted by the author). Moderate-high confidence on the intersectionintersectionAn address that names a location by two crossing streets ('5th & Main') rather than a number and street. Mailwoman tags the two streets as intersection_a and intersection_b — a negative-space format that starved the early model. combinatorial block and the compound_street double-suffix/dangling-conjunction pair — these read very clearly as solver mechanics given the explicit comment headers, but a reviewer could reasonably keep a few of the more name-like ones (e.g. "main st & side ave") if they weightparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. the coincidental plausibility of the placeholder words over the file's own stated test intent. Lower confidence on the single-letter trailing-directional cluster (functional-14..17) — defensible either way; I lean tombstone because the stated rationale is explicitly an autocomplete-era concern rather than a segmentation truth, but this is the softest call in the proposed set. The N-FISKE/DWIGHT-AVE jitter cluster was the closest call for tombstone-vs-borderline; I kept it on the strict criterion that no graded slot is wrong, but a stricter reading of rubric (b) (input itself, not gate impact) would move it to proposed and add 15 more fixtures / 15 more streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. slots to the totals above.