Parser failure report
CorpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus. golden:dev · 4255 fixtures · modelsneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' v264, v265. Every gold labelground truthThe correct answer for an example, used as the standard a prediction is graded against. Mailwoman's ground truth is the hand-labeled golden set; its quality caps achievable accuracy. is graded (floor labelscomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag. compare the streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels./prefix/suffix family); a fixture "fails" a labelcomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag. when the parseaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates. disagrees with the hand gold.
Note: golden gold uses the flat pre-split
streetschema, so thestreetrow reads confounded here (the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' splits prefix/streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels./suffix).country/region/locality/postcode/house_numberare single-tag and valid — that is where a class trade shows.
Per-model failure count
| modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' | fixtures with ≥1 labelcomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag. failure | rate |
|---|---|---|
v264 | 1098 | 26% |
v265 | 1101 | 26% |
Failures by label — the class each candidate trades
A count that rises across candidates (bold, with the delta vs the first modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.') is a class the candidate silently traded. This is the row that flags a regression the floor gates miss — e.g. country degrading across the fragment lineage.
| labelcomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag. | v264 | v265 |
|---|---|---|
region | 582 | 585 (+3) |
locality | 481 | 481 |
street | 436 | 437 |
postcode | 97 | 97 |
venue | 87 | 86 |
house_number | 73 | 75 |
dependent_locality | 34 | 34 |
country | 20 | 21 |
street_prefix | 6 | 6 |
street_prefix_particle | 6 | 6 |
po_box | 4 | 4 |
attention | 1 | 1 |
cedex | 1 | 1 |
unit | 1 | 1 |
Correlation — failure rate by structural class
Read across a row: a class that fails more on one modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' is the shape of what that candidate traded (delimiter-free, non-ASCII, short inputs, per countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.).
| structural class | v264 | v265 |
|---|---|---|
| whitespace-only | 255/919 (28%) | 256/919 (28%) |
| has comma | 841/3312 (25%) | 843/3312 (25%) |
| non-ASCII | 390/827 (47%) | 390/827 (47%) |
| has countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. gold | 117/224 (52%) | 117/224 (52%) |
| ≤3 tokenstokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. | 354/1251 (28%) | 353/1251 (28%) |
| US | 760/2527 (30%) | 763/2527 (30%) |
| FR | 281/1546 (18%) | 281/1546 (18%) |
| ZZ (synthetic) | 0/0 (—) | 0/0 (—) |
Beyond reach — fails on every model (1095)
The persistent core: addresses no current candidate parsesaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates.. Per-countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. breakdown (the correlation), then a sample.
| countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. | beyond-reach fixtures |
|---|---|
US | 759 |
FR | 279 |
UN | 43 |
AD | 13 |
U. | 1 |
Sample (first 40 of 1095)
| countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. | input | source |
|---|---|---|
AD | New York, New York Steakhouse, 3790 Las Vegas Blvd S, Las Vegas, NV 89109 | adversarial |
AD | Athens, Georgia Café, 200 College Ave, Athens, GA 30601 | adversarial |
AD | Boston, Massachusetts Pizza Co., 50 Broadway, Brooklyn, NY 11201 | adversarial |
AD | Miami, Florida Surf Shop, 100 Ocean Dr, San Diego, CA 92101 | adversarial |
AD | Nashville, Tennessee BBQ Pit, 75 Elm Ave, Atlanta, GA 30308 | adversarial |
AD | P'tit St. Denis Café, 500 Boul. Saint-Laurent, Montreal, QC H2X 2T6 | adversarial |
AD | Sainte-Marie Bistro, 25 Rue Sainte-Catherine, Montreal, QC H3B 1A3 | adversarial |
AD | 123Main St, Portland, OR 97214 | adversarial |
AD | 50.Tremont.St,Boston,MA,02108 | adversarial |
AD | Phone: 555-1234 — 200 Elm St, Springfield, IL 62701 | adversarial |
AD | 123 Main St c/o John Smith, Portland, OR 97214 | adversarial |
AD | 123 Main St #4B, Portland, OR 97214 | adversarial |
AD | Center | adversarial |
FR | 10 Rue de la République, 75008 Paris | fr |
FR | 45 Cours Lafayette, 69003 Lyon | fr |
FR | 1 bis Avenue des Champs-Élysées, 75008 Paris, France | fr |
FR | 100 Rue de la Paix, 75008 CEDEX 08 Paris | fr |
FR | Paris 8e | fr |
FR | Hôtel de Ville, 4 Place de l'Hôtel-de-Ville, 75004 Paris | fr |
FR | 7 Promenade des Anglais, 06000 Nice | fr |
FR | Île-de-France, France | fr |
FR | 15 rue de la république, 75008 paris | fr |
FR | Шом, Creuse, France | fr |
FR | Шом, Creuse | fr |
FR | Creuse, Шом, France | fr |
FR | France, Шом, Creuse | fr |
FR | 富尔内勒, Lozère, France | fr |
FR | France, Lozère, 富尔内勒 | fr |
FR | 富尔内勒 Lozère | fr |
FR | LOZÈRE 富尔内勒 FRANCE | fr |
FR | France, Lozere, Runik yozuv | fr |
FR | Lozere, Runik yozuv | fr |
FR | France, Creuse, Сен-Дизье-Леирен | fr |
FR | Фрессіне-де-Лозер, Lozère, France | fr |
FR | FRANCE, LOZÈRE, Фрессіне-де-Лозер | fr |
FR | Фрессіне-де-Лозер Lozère | fr |
FR | Creuse, Thauron | fr |
FR | Creuse, Le Chauchet, France | fr |
FR | France, Creuse, Le Chauchet | fr |
FR | 卡萨尼亚, Lozère | fr |
Model-specific — where candidates disagree (9)
A ✓ under one modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' and a failure under another = a fix or a regression between candidates — the diff you track release-over-release.
| countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. | input | v264 | v265 |
|---|---|---|---|
FR | 47110 2 place de verdun sainte-livrade-sur-lot | ✓ | streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.→Place DE · localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.→Verdun Sainte-Livrade-Sur-Lot |
FR | 47110, 1 rue des hirondelles, sainte-livrade-sur-lot | ✓ | house_number→47110 1 · postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one.→∅ |
FR | IFS 14123 France | postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one.→14123 France | ✓ |
FR | 54000 Nancy, 13 Boulevard Albert 1er | house_number→13 1er · streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.→Boulevard Albert | ✓ |
US | STATE RTE 100, VT 05350, USA | ✓ | streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.→State Rte 100 VT · regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.→∅ |
US | N Seventh St, Wyoming 82637 | ✓ | regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.→∅ |
US | SARAH SOWERWINE, 1201 E 7TH ST, POWELL, WYOMING | ✓ | house_number→∅ |
US | Фортјуна, North Dakota, US | localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.→Фортјуна, North · regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.→Dakota | ✓ |
US | روولینگ هیل وایومینگ Wyoming | ✓ | localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.→روولینگ هیل وایومینگ Wyoming · regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.→∅ |