Parser failure report
CorpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus. parity-corpus.triaged.jsonl · 321 fixtures · modelsneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' shipped, v257, v261, v280. Every gold labelground truthThe correct answer for an example, used as the standard a prediction is graded against. Mailwoman's ground truth is the hand-labeled golden set; its quality caps achievable accuracy. is graded (floor labelscomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag. compare the streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels./prefix/suffix family); a fixture "fails" a labelcomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag. when the parseaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates. disagrees with the hand gold.
Per-model failure count
| modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' | fixtures with ≥1 labelcomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag. failure | rate |
|---|---|---|
shipped | 204 | 64% |
v257 | 189 | 59% |
v261 | 191 | 60% |
v280 | 193 | 60% |
Failures by label — the class each candidate trades
A count that rises across candidates (bold, with the delta vs the first modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.') is a class the candidate silently traded. This is the row that flags a regression the floor gates miss — e.g. country degrading across the fragment lineage.
| labelcomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag. | shipped | v257 | v261 | v280 |
|---|---|---|---|---|
street | 150 | 124 | 126 | 127 |
locality | 56 | 59 (+3) | 60 (+4) | 57 |
house_number | 43 | 34 | 34 | 36 |
venue | 22 | 22 | 22 | 22 |
region | 15 | 16 | 16 | 16 |
unit | 15 | 15 | 15 | 15 |
country | 2 | 2 | 1 | 1 |
postcode | 1 | 1 | 1 | 1 |
Correlation — failure rate by structural class
Read across a row: a class that fails more on one modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' is the shape of what that candidate traded (delimiter-free, non-ASCII, short inputs, per countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.).
| structural class | shipped | v257 | v261 | v280 |
|---|---|---|---|---|
| whitespace-only | 133/193 (69%) | 122/193 (63%) | 123/193 (64%) | 126/193 (65%) |
| has comma | 57/113 (50%) | 54/113 (48%) | 55/113 (49%) | 54/113 (48%) |
| non-ASCII | 36/44 (82%) | 33/44 (75%) | 33/44 (75%) | 34/44 (77%) |
| has countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. gold | 34/42 (81%) | 34/42 (81%) | 34/42 (81%) | 34/42 (81%) |
| ≤3 tokenstokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. | 71/119 (60%) | 61/119 (51%) | 62/119 (52%) | 62/119 (52%) |
| US | 58/99 (59%) | 56/99 (57%) | 57/99 (58%) | 56/99 (57%) |
| FR | 34/43 (79%) | 28/43 (65%) | 28/43 (65%) | 31/43 (72%) |
| ZZ (synthetic) | 47/59 (80%) | 46/59 (78%) | 46/59 (78%) | 46/59 (78%) |
Beyond reach — fails on every model (180)
The persistent core: addresses no current candidate parsesaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates.. Per-countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. breakdown (the correlation), then a sample.
| countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. | beyond-reach fixtures |
|---|---|
US | 52 |
ZZ | 46 |
FR | 28 |
NL | 10 |
AU | 9 |
NO | 7 |
PT | 6 |
DE | 5 |
PL | 3 |
SE | 3 |
CZ | 2 |
GB | 2 |
NZ | 2 |
RO | 2 |
BR | 1 |
ES | 1 |
IN | 1 |
Sample (first 40 of 180)
| countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. | input | source |
|---|---|---|
AU | 6000, NSW, Australia | v1-parity:mailwoman/test/address.aus.test.ts |
AU | Unit 12/345 Main St | v1-parity:mailwoman/test/address.aus.test.ts |
AU | U 12 345 Main St | v1-parity:mailwoman/test/address.aus.test.ts |
AU | Apartment 12/345 Main St | v1-parity:mailwoman/test/address.aus.test.ts |
AU | Apt 12/345 Main St | v1-parity:mailwoman/test/address.aus.test.ts |
AU | Lot 12/345 Main St | v1-parity:mailwoman/test/address.aus.test.ts |
AU | U12/345 Main St | v1-parity:mailwoman/test/address.aus.test.ts |
AU | Lot 12/345 Illawarra Road Marrickville NSW 2204 | v1-parity:mailwoman/test/address.aus.test.ts |
AU | Lot 2, Burrows Avenue, EDMONDSON PARK, NSW, Australia | v1-parity:mailwoman/test/address.aus.test.ts |
BR | Rua Raul Leite Magalhães, 65, Tapiraí - SP, 18180-000, Brazil | v1-parity:mailwoman/test/address.bra.test.ts |
CZ | Korunní 810, Praha | v1-parity:mailwoman/test/address.cze.test.ts |
CZ | Kájovská 68, Český Krumlov | v1-parity:mailwoman/test/address.cze.test.ts |
DE | Am Falkpl. 5, 10437 Berlin | v1-parity:mailwoman/test/address.deu.test.ts |
DE | Kaschk Bar, Linienstraße 40 10119 Berlin | v1-parity:mailwoman/test/address.deu.test.ts |
DE | Genter Straße 16a, Munich, Germany | v1-parity:mailwoman/test/address.deu.test.ts |
DE | Königsallee Düsseldorf | v1-parity:mailwoman/test/address.deu.test.ts |
DE | Eberswalder Straße 100 104 | v1-parity:mailwoman/test/address.deu.test.ts |
ES | Carrer d'Aragó 155 08011 Barcelona | v1-parity:mailwoman/test/address.esp.test.ts |
FR | Rue Montmartre | v1-parity:mailwoman/test/address.fra.test.ts |
FR | Rue de Paris | v1-parity:mailwoman/test/address.fra.test.ts |
FR | Rue Saint Anne | v1-parity:mailwoman/test/address.fra.test.ts |
FR | Allée Victor Hugo | v1-parity:mailwoman/test/address.fra.test.ts |
FR | Avenue Aristide Briand | v1-parity:mailwoman/test/address.fra.test.ts |
FR | Rue Henri Barbusse Paris France | v1-parity:mailwoman/test/address.fra.test.ts |
FR | Rue du Général Leclerc Dunkerque France | v1-parity:mailwoman/test/address.fra.test.ts |
FR | Avenue de Sainte Rose de Lima | v1-parity:mailwoman/test/address.fra.test.ts |
FR | Rue du Capitaine Galinat Marseille France | v1-parity:mailwoman/test/address.fra.test.ts |
FR | Mery Sur Oise | v1-parity:mailwoman/test/address.fra.test.ts |
FR | Méry Sur Oise | v1-parity:mailwoman/test/address.fra.test.ts |
FR | 4 Cité Du Cardinal Lemoine 75005 Paris | v1-parity:mailwoman/test/address.fra.test.ts |
FR | Rue de l'Inspecteur Alles Paris | v1-parity:mailwoman/test/address.fra.test.ts |
FR | Rue de l'Empereur Julien Paris | v1-parity:mailwoman/test/address.fra.test.ts |
FR | Rue de l'Adjudant Réau Paris | v1-parity:mailwoman/test/address.fra.test.ts |
FR | 10 Boulevard Saint-Germains Paris | v1-parity:mailwoman/test/address.fra.test.ts |
FR | Esplanade de la Liberté | v1-parity:mailwoman/test/address.fra.test.ts |
FR | Esplanade Méditerranée | v1-parity:mailwoman/test/address.fra.test.ts |
FR | École Paul Valéry Montpellier | v1-parity:mailwoman/test/place.fra.test.ts |
FR | Université de Montpellier | v1-parity:mailwoman/test/place.fra.test.ts |
FR | École Jules Vernes Villetaneuse | v1-parity:mailwoman/test/place.fra.test.ts |
FR | ZAC de la Tuilerie, Villars-les-Dombes, France | v1-parity:mailwoman/test/place.fra.test.ts |
Model-specific — where candidates disagree (34)
A ✓ under one modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' and a failure under another = a fix or a regression between candidates — the diff you track release-over-release.
| countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. | input | shipped | v257 | v261 | v280 |
|---|---|---|---|---|---|
AU | Eight Mile Plains 4113 | ✓ | localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.→∅ | localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.→∅ | ✓ |
AU | Eight Mile Plains 9999 | ✓ | localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.→∅ | localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.→∅ | localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.→∅ |
DE | Am Bürgerpark 15-18, 13156, Berlin | streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.→Am Bürgerpark 1 · house_number→5-18 | ✓ | ✓ | ✓ |
DE | Rathausplatz | streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.→∅ | ✓ | ✓ | ✓ |
DE | Plutoweg | streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.→∅ | ✓ | ✓ | ✓ |
DE | Dorfstrasse | streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.→∅ | ✓ | ✓ | ✓ |
FR | Rue de la Paix | streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.→Rue de | ✓ | ✓ | ✓ |
FR | Boulevard du Général Charles De Gaulle | streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.→Boulevard | ✓ | ✓ | ✓ |
FR | Boulevard Charles De Gaulle | streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.→∅ | ✓ | ✓ | streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.→∅ |
FR | Rue Jean Baptiste Clément | streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.→Rue Jean | ✓ | ✓ | ✓ |
FR | 32 Rue Du 4 Septembre | ✓ | ✓ | ✓ | streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.→Rue Du 4 |
FR | Rue de l'Amiral Galache, Toulouse | streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.→∅ · localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.→Rue de l'Amiral Galache | ✓ | ✓ | ✓ |
FR | Esplanade du Géneral de Gaulle | streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.→∅ | ✓ | ✓ | streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.→du Géneral de Gaulle |
GB | 24-28 Broadway Market, London | ✓ | ✓ | streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.→Broadway · localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.→Market London | streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.→Broadway · localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.→Market London |
NL | Bosserdijk, Hoogland | streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.→∅ · localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.→Bosserdijk | ✓ | ✓ | ✓ |
NL | Bloemendaal | ✓ | localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.→∅ | localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.→∅ | ✓ |
NO | Skredderplassen 20 | streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.→∅ | ✓ | ✓ | ✓ |
NO | Grense Jakobselv veien 311 | streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.→veien · house_number→∅ | ✓ | ✓ | house_number→∅ |
NZ | 183 Vista Paku | house_number→∅ | ✓ | ✓ | house_number→∅ |
PL | Ulica Strzelecka 12, Nowy Sącz | ✓ | localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.→wy Sącz | localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.→wy Sącz | localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.→wy Sącz |
PL | Żorska 11, 47-400 | ✓ | streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.→Żorska 1 · house_number→1 | streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.→Żorska 1 · house_number→1 | streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.→Żorska 1 · house_number→1 |
PT | Tv. da Horta 27A | streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.→da Horta · house_number→∅ | ✓ | ✓ | ✓ |
RO | Bulevardul Iuliu Maniu, Bucharest | streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.→∅ · localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.→Bulevardul Iuliu Maniu | ✓ | ✓ | ✓ |
SK | Divadelná 41/3, Trnava | streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.→Divadeln 4 · house_number→1/3 | ✓ | ✓ | ✓ |
US | wrigley field | streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.→Field | ✓ | ✓ | ✓ |
US | Fm 3009 TX | streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.→Fm · regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.→∅ | ✓ | streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.→Fm · regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.→∅ | streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.→Fm · regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.→∅ |
US | N DWIGHT AVE Portland Oregon | ✓ | streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.→N DWIGHT AVE Portland · localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.→∅ | streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.→N DWIGHT AVE Portland · localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.→∅ | streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.→N DWIGHT AVE Portland · localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.→∅ |
US | Highway 72 | streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.→∅ | ✓ | ✓ | ✓ |
US | Broadway, Manhattan | streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.→∅ · localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.→Broadway Manhattan | ✓ | ✓ | ✓ |
US | East Broadway, Manhattan | streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.→∅ · localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.→East Broadway Manhattan | ✓ | ✓ | ✓ |
US | 123 Main St New York, NY 10010 | streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.→Main New York St · localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.→∅ | streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.→Main New York St · localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.→∅ | streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.→Main New York St · localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.→∅ | ✓ |
US | New York, NY | ✓ | localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.→∅ | localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.→∅ | localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.→∅ |
US | New York, New York | ✓ | localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.→∅ · regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.→New York New York | localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.→∅ · regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.→New York New York | localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.→∅ · regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.→New York New York |
ZZ | main pl | streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.→PL | ✓ | ✓ | ✓ |