Parser failure report
CorpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus. golden:dev · 4255 fixtures · modelsneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' shipped, v263. Every gold labelground truthThe correct answer for an example, used as the standard a prediction is graded against. Mailwoman's ground truth is the hand-labeled golden set; its quality caps achievable accuracy. is graded (floor labelscomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag. compare the streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels./prefix/suffix family); a fixture "fails" a labelcomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag. when the parseaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates. disagrees with the hand gold.
Note: golden gold uses the flat pre-split
streetschema, so thestreetrow reads confounded here (the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' splits prefix/streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels./suffix).country/region/locality/postcode/house_numberare single-tag and valid — that is where a class trade shows.
Per-model failure count
| modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' | fixtures with ≥1 labelcomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag. failure | rate |
|---|---|---|
shipped | 1095 | 26% |
v263 | 1101 | 26% |
Failures by label — the class each candidate trades
A count that rises across candidates (bold, with the delta vs the first modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.') is a class the candidate silently traded. This is the row that flags a regression the floor gates miss — e.g. country degrading across the fragment lineage.
| labelcomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag. | shipped | v263 |
|---|---|---|
region | 556 | 580 (+24) |
locality | 477 | 481 (+4) |
street | 432 | 437 (+5) |
postcode | 92 | 98 (+6) |
venue | 87 | 86 |
house_number | 74 | 76 |
country | 34 | 24 |
dependent_locality | 34 | 34 |
street_prefix | 6 | 6 |
street_prefix_particle | 6 | 6 |
po_box | 4 | 4 |
attention | 1 | 1 |
cedex | 1 | 1 |
unit | 1 | 1 |
Correlation — failure rate by structural class
Read across a row: a class that fails more on one modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' is the shape of what that candidate traded (delimiter-free, non-ASCII, short inputs, per countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.).
| structural class | shipped | v263 |
|---|---|---|
| whitespace-only | 248/919 (27%) | 255/919 (28%) |
| has comma | 845/3312 (26%) | 844/3312 (25%) |
| non-ASCII | 396/827 (48%) | 390/827 (47%) |
| has countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. gold | 127/224 (57%) | 117/224 (52%) |
| ≤3 tokenstokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. | 349/1251 (28%) | 356/1251 (28%) |
| US | 750/2527 (30%) | 761/2527 (30%) |
| FR | 276/1546 (18%) | 283/1546 (18%) |
| ZZ (synthetic) | 0/0 (—) | 0/0 (—) |
Beyond reach — fails on every model (1073)
The persistent core: addresses no current candidate parsesaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates.. Per-countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. breakdown (the correlation), then a sample.
| countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. | beyond-reach fixtures |
|---|---|
US | 747 |
FR | 273 |
UN | 40 |
AD | 12 |
U. | 1 |
Sample (first 40 of 1073)
| countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. | input | source |
|---|---|---|
AD | New York, New York Steakhouse, 3790 Las Vegas Blvd S, Las Vegas, NV 89109 | adversarial |
AD | Athens, Georgia Café, 200 College Ave, Athens, GA 30601 | adversarial |
AD | Boston, Massachusetts Pizza Co., 50 Broadway, Brooklyn, NY 11201 | adversarial |
AD | Miami, Florida Surf Shop, 100 Ocean Dr, San Diego, CA 92101 | adversarial |
AD | Nashville, Tennessee BBQ Pit, 75 Elm Ave, Atlanta, GA 30308 | adversarial |
AD | P'tit St. Denis Café, 500 Boul. Saint-Laurent, Montreal, QC H2X 2T6 | adversarial |
AD | Sainte-Marie Bistro, 25 Rue Sainte-Catherine, Montreal, QC H3B 1A3 | adversarial |
AD | 123Main St, Portland, OR 97214 | adversarial |
AD | 50.Tremont.St,Boston,MA,02108 | adversarial |
AD | 123 Main St c/o John Smith, Portland, OR 97214 | adversarial |
AD | 123 Main St #4B, Portland, OR 97214 | adversarial |
AD | Center | adversarial |
FR | 10 Rue de la République, 75008 Paris | fr |
FR | 45 Cours Lafayette, 69003 Lyon | fr |
FR | 1 bis Avenue des Champs-Élysées, 75008 Paris, France | fr |
FR | 100 Rue de la Paix, 75008 CEDEX 08 Paris | fr |
FR | Paris 8e | fr |
FR | Hôtel de Ville, 4 Place de l'Hôtel-de-Ville, 75004 Paris | fr |
FR | 7 Promenade des Anglais, 06000 Nice | fr |
FR | Île-de-France, France | fr |
FR | 15 rue de la république, 75008 paris | fr |
FR | Шом, Creuse, France | fr |
FR | Шом, Creuse | fr |
FR | Creuse, Шом, France | fr |
FR | France, Шом, Creuse | fr |
FR | 富尔内勒, Lozère, France | fr |
FR | France, Lozère, 富尔内勒 | fr |
FR | 富尔内勒 Lozère | fr |
FR | LOZÈRE 富尔内勒 FRANCE | fr |
FR | France, Lozere, Runik yozuv | fr |
FR | Lozere, Runik yozuv | fr |
FR | France, Creuse, Сен-Дизье-Леирен | fr |
FR | Фрессіне-де-Лозер, Lozère, France | fr |
FR | FRANCE, LOZÈRE, Фрессіне-де-Лозер | fr |
FR | Фрессіне-де-Лозер Lozère | fr |
FR | Creuse, Thauron | fr |
FR | Creuse, Le Chauchet, France | fr |
FR | France, Creuse, Le Chauchet | fr |
FR | 卡萨尼亚, Lozère | fr |
FR | 卡萨尼亚 Lozère | fr |
Model-specific — where candidates disagree (50)
A ✓ under one modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' and a failure under another = a fix or a regression between candidates — the diff you track release-over-release.
| countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. | input | shipped | v263 |
|---|---|---|---|
AD | Phone: 555-1234 — 200 Elm St, Springfield, IL 62701 | ✓ | house_number→55-1234 200 |
FR | Manailly Creuse FRANCE | ✓ | regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.→Creuse FRANCE · countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.→∅ |
FR | Lizières Creuse France | ✓ | regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.→Creuse France · countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.→∅ |
FR | Montredon Lozère France | ✓ | localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.→Montredon Lozère · regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.→∅ |
FR | Ла-Нуай Creuse France | ✓ | localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.→Ла-Нуай Creuse · regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.→∅ |
FR | 47110 2 place de verdun sainte-livrade-sur-lot | streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.→Place DE · localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.→Verdun Sainte-Livrade-Sur-Lot | ✓ |
FR | Woustviller 57915 France | ✓ | postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one.→57915 France |
FR | 13940 Mollégès, France | ✓ | localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.→∅ |
FR | IFS 14123 France | ✓ | postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one.→14123 France |
FR | 62119 Dourges France | ✓ | postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one.→62119 France |
FR | Le Puy-Sainte-Réparade 13610 | localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.→Le | ✓ |
FR | 59112 Ann ullin, France | ✓ | localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.→ullin |
FR | France, 59112 Ann ullin | ✓ | localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.→France Ann ullin |
FR | Wimereux, 62930 | localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.→∅ | ✓ |
UN | United States of America, Медоу Ејкерс, WY | countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.→∅ | ✓ |
UN | Патни, Vermont, United States of America | ✓ | localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.→Патни, Vermont · regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.→∅ |
UN | United States of America, ND, Лорейн | countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.→∅ | ✓ |
UN | United States of America, ND, هاغو | countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.→∅ | ✓ |
UN | United States of America, Vermont, Содом | countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.→∅ | ✓ |
UN | Vermont, United States of America | ✓ | regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.→∅ |
UN | United States of America, ND, Monroe | countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.→∅ | ✓ |
UN | Wyoming, ہلزڈیل وائیومنگ, United States of America | localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.→Wyoming ہلزڈیل وائیومنگ · regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.→∅ | ✓ |
UN | United States of America, VT, Уэтерсфилд | countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.→∅ | ✓ |
UN | United States of America, Wyoming, هوباک وایومینگ | countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.→∅ | ✓ |
UN | علاالدین وایومینگ, Wyoming, United States of America | ✓ | localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.→علاالدین وایومینگ, Wyoming · regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.→∅ |
UN | United States of America, North Dakota, Bloomfield | countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.→∅ | ✓ |
UN | United States of America, اوپهام قوزئی داکوتا, ND | localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.→اوپهام وزئی داکوتا · countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.→∅ | ✓ |
UN | United States of America, VT, Вотерфорд | countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.→∅ | ✓ |
UN | United States of America, Vermont, Монтпилиер | countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.→∅ | ✓ |
UN | United States of America, لئونارد داکوتای شمالی, ND | countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.→∅ | ✓ |
UN | United States of America, Wyoming, بیر ریور | countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.→∅ | ✓ |
UN | United States of America, Wyoming, Antilop | countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.→∅ | ✓ |
UN | United States of America, North Dakota, واتفورد سیتی قوزئی داکوتا | localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.→واتفورد سیتی وزئی داکوتا · countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.→∅ | ✓ |
US | Cider Mill Rd VT 05161, US | ✓ | streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.→Cider VT Mill Rd · regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.→∅ |
US | STATE RTE 100, VT 05350, USA | ✓ | streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.→State Rte 100 VT · regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.→∅ |
US | 122R Ave Southwest ND 58655 | ✓ | streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.→Ave Southwest |
US | VT 05051 Deerfield Ln | ✓ | streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.→VT Deerfield Ln · regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.→∅ |
US | Farson Branch Library, 30 Hwy 28 Farson-Eden School Bldg, Farson, WY 82932 | ✓ | streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.→Hwy 28 Farson-Eden School |
US | Manchester Community Library, 138 CEMETERY AVE RTE 7A, Manchester VT | streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.→CEMETERY AVE RTE | ✓ |
US | US اوبران North Dakota | localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.→US اوبران | ✓ |
US | بروکهورست وایومینگ, Wyoming, US | ✓ | localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.→بروکهورست وایومینگ, Wyoming · regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.→∅ |
US | North Dakota, Antilopo | ✓ | localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.→North Dakota Antilopo · regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.→∅ |
US | هئیل, North Dakota, US | ✓ | localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.→هئیل, North · regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.→Dakota |
US | US - ND - نومي | ✓ | localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.→ND - نومي · regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.→∅ |
US | US, ND, Antilopalar | ✓ | localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.→Antilo · regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.→ND palar |
US | US, Хак, ND | localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.→US Хак | ✓ |
US | SARDGEEN ARBON PLLC, 110 W FAYETTE ST STE 1110, SYRACUSE, NY 13202 | ✓ | streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.→W Fayette ST |
US | Seyfarth Shaw Fairweather, 1211 Ave of the America, New York, New York | ✓ | streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.→Ave of the |
US | Jo Ann Brinkley, PO BOX 7298, Glen Rose, TX 76043 | ✓ | streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.→BOX 7298 |
US | Virginia Stricklin, 9122 Heathcliff, San Antonio, TX 78254, US | ✓ | regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.→TX US |