Loaded 4561 golden entries Loading modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.'... Running evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error.... 500/4561 (2.5s) 1000/4561 (4.9s) 1500/4561 (7.5s) 2000/4561 (9.8s) 2500/4561 (12.2s) 3000/4561 (14.6s) 3500/4561 (17.0s) 4000/4561 (19.6s) 4500/4561 (21.9s)
Error Analysis Report
Golden setgolden setA hand-labeled evaluation dataset of US, French, and adversarial addresses used as ground truth for evals.: 4561 entries Modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.': NeuralAddressClassifier Time: 22.2s
Summary
| Metric | Count | Rate |
|---|---|---|
| Exact matchexact matchThe share of eval items whose every component is correct (compared per-span or per-token). Stricter than per-tag F1, which credits partial correctness. | 964 | 21.1% |
| Missed entities | 1889 | — |
| Boundary errors | 3921 | — |
| Confused tags | 1607 | — |
| Hallucinated tags | 1383 | — |
Per-tag breakdown
| Tag | Expected | Correct | Missed | Boundary | Confused | Hallucinated | RecallrecallOf the spans whose gold label is a given tag, the fraction the model found. High recall means few misses. Paired with precision to compute F1. |
|---|---|---|---|---|---|---|---|
| localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. | 3383 | 1343 | 48 | 1400 | 592 | 503 | 39.7% |
| regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. | 3227 | 2096 | 705 | 156 | 270 | 117 | 65.0% |
| postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. | 3006 | 2284 | 355 | 100 | 267 | 145 | 76.0% |
| streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. | 2931 | 818 | 351 | 1462 | 300 | 46 | 27.9% |
| house_number | 1745 | 1379 | 160 | 46 | 160 | 381 | 79.0% |
| venuevenueA named, non-address place — a business, building, park, or stadium. Mailwoman's free-text point-of-interest component, added as a Tier 2 fine label. | 1101 | 321 | 90 | 680 | 10 | 86 | 29.2% |
| countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. | 245 | 60 | 104 | 73 | 8 | 101 | 24.5% |
| dependent_locality | 40 | 0 | 40 | 0 | 0 | 0 | 0.0% |
| po_boxPO boxA numbered mailbox at a post office used as a delivery address instead of a physical street location. Mailwoman tags it as the po_box component; structurally the same family as a subpremise. | 27 | 14 | 9 | 4 | 0 | 4 | 51.9% |
| street_prefix | 10 | 0 | 10 | 0 | 0 | 0 | 0.0% |
| street_prefix_particle | 6 | 0 | 6 | 0 | 0 | 0 | 0.0% |
| unitunitA subdivision of a building — apartment, suite, floor — that refines a street address. Mailwoman's unit component; a designator plus identifier forms a subpremise. | 5 | 0 | 5 | 0 | 0 | 0 | 0.0% |
| street_suffix | 2 | 0 | 2 | 0 | 0 | 0 | 0.0% |
| intersection_a | 1 | 0 | 1 | 0 | 0 | 0 | 0.0% |
| intersection_b | 1 | 0 | 1 | 0 | 0 | 0 | 0.0% |
| cedexCEDEX (Courrier d'Entreprise à Distribution Exceptionnelle). A French postal routing for high-volume business mail: a CEDEX code delivers directly from a sorting centre, bypassing the local post office. A common negative-space format Mailwoman must parse. | 1 | 0 | 1 | 0 | 0 | 0 | 0.0% |
| attentionattentionThe core mechanism inside a transformer encoder. Each token's representation is updated by looking at every other token, with learned weights deciding how much each one matters. | 1 | 0 | 1 | 0 | 0 | 0 | 0.0% |
Missed entities (1889)
6220 SE Salmon St, Portland, OR 97215, USA— missing street_prefix="SE"6220 SE Salmon St, Portland, OR 97215, USA— missing street_suffix="St"123 Main St Apt 4B, Springfield, IL 62701— missing unitunitA subdivision of a building — apartment, suite, floor — that refines a street address. Mailwoman's unit component; a designator plus identifier forms a subpremise.="Apt 4B"Main St & 5th Ave, Portland, OR— missing intersection_a="Main St"Main St & 5th Ave, Portland, OR— missing intersection_b="5th Ave"
Boundary errors (3921)
6220 SE Salmon St, Portland, OR 97215, USA— streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.: expected "Salmon" got "SE Salmon St"500 W 7th St Ste 1100, Cincinnati, OH 45203-1234— streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.: expected "7th" got "W 7th St Ste 1100"123 main st portland or 97214— streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.: expected "main st" got "main"CULLEN INSULATION INC, POBOX 3211 FARGO ND 58108, FARGO, ND 58108— venuevenueA named, non-address place — a business, building, park, or stadium. Mailwoman's free-text point-of-interest component, added as a Tier 2 fine label.: expected "CULLEN INSULATION INC" got "CULLEN"CULLEN INSULATION INC, POBOX 3211 FARGO ND 58108, FARGO, ND 58108— streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.: expected "POBOX 3211 FARGO ND 58108" got "POBOX 3211"
Confused tags (1607)
Main St & 5th Ave, Portland, OR— localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.: expected "Portland" got "Main St"05770 VT DELONG LN— streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.: expected "DELONG LN" got "VT"ND 58368 6th St SW— streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.: expected "6th St SW" got "ND 58368 6th St"WY 82601 Mountain Way— streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.: expected "Mountain Way" got "WY"Woodland Ave NE, ND— regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.: expected "ND" got "NE"
Hallucinated tags (1383)
Main St & 5th Ave, Portland, OR— hallucinated streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.="Ave"12345 Mt Tabor Park, Portland, OR 97215— hallucinated streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.="Mt Tabor Park"Vermont, USA— hallucinated localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.="Vermont"05764 Finel Hollow Road, VT— hallucinated house_number="05764"05764 Finel Hollow Road, VT— hallucinated localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.="Finel Hollow Road"