Loaded 4561 golden entries Loading modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.'... Running evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error.... 500/4561 (2.8s) 1000/4561 (5.1s) 1500/4561 (7.5s) 2000/4561 (9.9s) 2500/4561 (12.3s) 3000/4561 (14.9s) 3500/4561 (17.3s) 4000/4561 (19.6s) 4500/4561 (22.0s)
Error Analysis Report
Golden setgolden setA hand-labeled evaluation dataset of US, French, and adversarial addresses used as ground truth for evals.: 4561 entries Modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.': NeuralAddressClassifier Time: 22.3s
Summary
| Metric | Count | Rate |
|---|---|---|
| Exact matchexact matchThe share of eval items whose every component is correct (compared per-span or per-token). Stricter than per-tag F1, which credits partial correctness. | 856 | 18.8% |
| Missed entities | 2470 | — |
| Boundary errors | 3804 | — |
| Confused tags | 1235 | — |
| Hallucinated tags | 2299 | — |
Per-tag breakdown
| Tag | Expected | Correct | Missed | Boundary | Confused | Hallucinated | RecallrecallOf the spans whose gold label is a given tag, the fraction the model found. High recall means few misses. Paired with precision to compute F1. |
|---|---|---|---|---|---|---|---|
| localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. | 3383 | 1053 | 970 | 1144 | 216 | 289 | 31.1% |
| regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. | 3227 | 2135 | 664 | 179 | 249 | 125 | 66.2% |
| postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. | 3006 | 2478 | 191 | 69 | 268 | 217 | 82.4% |
| streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. | 2931 | 806 | 263 | 1557 | 305 | 54 | 27.5% |
| house_number | 1745 | 1325 | 209 | 46 | 165 | 207 | 75.9% |
| venuevenueA named, non-address place — a business, building, park, or stadium. Mailwoman's free-text point-of-interest component, added as a Tier 2 fine label. | 1101 | 331 | 53 | 706 | 11 | 56 | 30.1% |
| countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. | 245 | 69 | 84 | 89 | 3 | 55 | 28.2% |
| dependent_locality | 40 | 12 | 0 | 10 | 18 | 1066 | 30.0% |
| po_boxPO boxA numbered mailbox at a post office used as a delivery address instead of a physical street location. Mailwoman tags it as the po_box component; structurally the same family as a subpremise. | 27 | 14 | 9 | 4 | 0 | 1 | 51.9% |
| street_prefix | 10 | 0 | 10 | 0 | 0 | 31 | 0.0% |
| street_prefix_particle | 6 | 0 | 6 | 0 | 0 | 0 | 0.0% |
| unitunitA subdivision of a building — apartment, suite, floor — that refines a street address. Mailwoman's unit component; a designator plus identifier forms a subpremise. | 5 | 0 | 5 | 0 | 0 | 0 | 0.0% |
| street_suffix | 2 | 0 | 2 | 0 | 0 | 198 | 0.0% |
| intersection_a | 1 | 0 | 1 | 0 | 0 | 0 | 0.0% |
| intersection_b | 1 | 0 | 1 | 0 | 0 | 0 | 0.0% |
| cedexCEDEX (Courrier d'Entreprise à Distribution Exceptionnelle). A French postal routing for high-volume business mail: a CEDEX code delivers directly from a sorting centre, bypassing the local post office. A common negative-space format Mailwoman must parse. | 1 | 0 | 1 | 0 | 0 | 0 | 0.0% |
| attentionattentionThe core mechanism inside a transformer encoder. Each token's representation is updated by looking at every other token, with learned weights deciding how much each one matters. | 1 | 0 | 1 | 0 | 0 | 0 | 0.0% |
Missed entities (2470)
6220 SE Salmon St, Portland, OR 97215, USA— missing street_prefix="SE"6220 SE Salmon St, Portland, OR 97215, USA— missing street_suffix="St"123 Main St Apt 4B, Springfield, IL 62701— missing unitunitA subdivision of a building — apartment, suite, floor — that refines a street address. Mailwoman's unit component; a designator plus identifier forms a subpremise.="Apt 4B"Main St & 5th Ave, Portland, OR— missing intersection_a="Main St"Main St & 5th Ave, Portland, OR— missing intersection_b="5th Ave"
Boundary errors (3804)
6220 SE Salmon St, Portland, OR 97215, USA— streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.: expected "Salmon" got "SE Salmon St"500 W 7th St Ste 1100, Cincinnati, OH 45203-1234— streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.: expected "7th" got "W 7th St Ste 1100"123 main st portland or 97214— streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.: expected "main st" got "main"CULLEN INSULATION INC, POBOX 3211 FARGO ND 58108, FARGO, ND 58108— venuevenueA named, non-address place — a business, building, park, or stadium. Mailwoman's free-text point-of-interest component, added as a Tier 2 fine label.: expected "CULLEN INSULATION INC" got "CULLEN"CULLEN INSULATION INC, POBOX 3211 FARGO ND 58108, FARGO, ND 58108— streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.: expected "POBOX 3211 FARGO ND 58108" got "POBOX 3211"
Confused tags (1235)
05770 VT DELONG LN— streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.: expected "DELONG LN" got "VT"ND 58368 6th St SW— streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.: expected "6th St SW" got "ND 58368 6th St"WY 82601 Mountain Way— streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.: expected "Mountain Way" got "WY"Woodland Ave NE, ND— regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.: expected "ND" got "NE"ND 58203, Woodland Ave NE— streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.: expected "Woodland Ave NE" got "ND 58203"
Hallucinated tags (2299)
Main St & 5th Ave, Portland, OR— hallucinated dependent_locality="Main St"12345 Mt Tabor Park, Portland, OR 97215— hallucinated streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.="Mt Tabor Park"Vermont, USA— hallucinated dependent_locality="Vermont"05764 Finel Hollow Road, VT— hallucinated house_number="05764"Finel Hollow Road, VT— hallucinated dependent_locality="Finel Hollow Road"