Eval Matrix Report
Generated: 2026-05-25T12:09:29.286Z Golden rows: 4535
Summary
| Mode | Exact Matchexact matchThe share of eval items whose every component is correct (compared per-span or per-token). Stricter than per-tag F1, which credits partial correctness. | Macro F1macro F1The unweighted average of per-class F1 scores — treats every class equally. Mailwoman's primary label-level eval metric. | Empty Parseaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates. | Overconf Wrong |
|---|---|---|---|---|
| rule-only | 30.8% | 22.0% | 6.3% | 2.4% |
| neural | 0.1% | 7.3% | 0.3% | 54.5% |
| hybrid | 0.1% | 7.3% | 0.3% | 54.5% |
| hybrid-joint | 6.0% | 16.6% | 0.0% | 0.1% |
rule-only
Per-component F1
| Tag | P | R | F1 | TP | FP | FN |
|---|---|---|---|---|---|---|
| regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. | 96.9% | 84.1% | 90.1% | 2697 | 86 | 508 |
| house_number | 75.5% | 92.0% | 82.9% | 1602 | 520 | 140 |
| postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. | 98.8% | 65.4% | 78.7% | 1948 | 24 | 1032 |
| streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. | 72.6% | 71.2% | 71.9% | 2085 | 786 | 843 |
| localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. | 83.4% | 57.0% | 67.8% | 1915 | 381 | 1442 |
| venuevenueA named, non-address place — a business, building, park, or stadium. Mailwoman's free-text point-of-interest component, added as a Tier 2 fine label. | 38.6% | 17.7% | 24.3% | 195 | 310 | 906 |
| countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. | 21.0% | 25.7% | 23.1% | 63 | 237 | 182 |
| unitunitA subdivision of a building — apartment, suite, floor — that refines a street address. Mailwoman's unit component; a designator plus identifier forms a subpremise. | 0.9% | 20.0% | 1.7% | 1 | 115 | 4 |
| street_prefix | 0.0% | 0.0% | 0.0% | 0 | 0 | 10 |
| street_suffix | 0.0% | 0.0% | 0.0% | 0 | 0 | 2 |
| po_boxPO boxA numbered mailbox at a post office used as a delivery address instead of a physical street location. Mailwoman tags it as the po_box component; structurally the same family as a subpremise. | 0.0% | 0.0% | 0.0% | 0 | 0 | 1 |
| unit_designator | 0.0% | 0.0% | 0.0% | 0 | 115 | 0 |
| intersection_a | 0.0% | 0.0% | 0.0% | 0 | 0 | 1 |
| intersection_b | 0.0% | 0.0% | 0.0% | 0 | 0 | 1 |
| dependent_locality | 0.0% | 0.0% | 0.0% | 0 | 0 | 40 |
| level_designator | 0.0% | 0.0% | 0.0% | 0 | 1 | 0 |
| level | 0.0% | 0.0% | 0.0% | 0 | 1 | 0 |
| street_prefix_particle | 0.0% | 0.0% | 0.0% | 0 | 0 | 6 |
| cedexCEDEX (Courrier d'Entreprise à Distribution Exceptionnelle). A French postal routing for high-volume business mail: a CEDEX code delivers directly from a sorting centre, bypassing the local post office. A common negative-space format Mailwoman must parse. | 0.0% | 0.0% | 0.0% | 0 | 0 | 1 |
| attentionattentionThe core mechanism inside a transformer encoder. Each token's representation is updated by looking at every other token, with learned weights deciding how much each one matters. | 0.0% | 0.0% | 0.0% | 0 | 0 | 1 |
Calibration
| Bucket | Total | Correct | Accuracy |
|---|---|---|---|
| conf > 0.9 | 946 | 835 | 88.3% |
| conf: 0.7–0.9 | 1330 | 564 | 42.4% |
| conf: 0.5–0.7 | 1199 | 0 | 0.0% |
| conf < 0.5 | 1060 | 0 | 0.0% |
Per-failure-class
| Class | Total | Exact Matchexact matchThe share of eval items whose every component is correct (compared per-span or per-token). Stricter than per-tag F1, which credits partial correctness. | Rate |
|---|---|---|---|
| failure/tokenization-trap | 3 | 0 | 0.0% |
| kryptonite/place-shaped-venuevenueA named, non-address place — a business, building, park, or stadium. Mailwoman's free-text point-of-interest component, added as a Tier 2 fine label. | 6 | 0 | 0.0% |
| kryptonite/particle-honorific | 9 | 0 | 0.0% |
| normal | 4503 | 1389 | 30.8% |
| failure/streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.-localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.-collision | 5 | 2 | 40.0% |
| kryptonite/place-name-venuevenueA named, non-address place — a business, building, park, or stadium. Mailwoman's free-text point-of-interest component, added as a Tier 2 fine label. | 10 | 5 | 50.0% |
| failure/ambiguous-localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. | 4 | 3 | 75.0% |
| failure/numeric-chaos | 1 | 1 | 100.0% |
| kryptonite/disambiguation | 4 | 4 | 100.0% |
neural
Per-component F1
| Tag | P | R | F1 | TP | FP | FN |
|---|---|---|---|---|---|---|
| regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. | 84.0% | 59.9% | 69.9% | 1919 | 366 | 1286 |
| localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. | 55.4% | 20.6% | 30.1% | 693 | 558 | 2664 |
| countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. | 26.2% | 26.1% | 26.2% | 64 | 180 | 181 |
| dependent_locality | 1.2% | 30.0% | 2.4% | 12 | 956 | 28 |
| postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. | 24.4% | 0.7% | 1.4% | 22 | 68 | 2958 |
| house_number | 9.0% | 0.5% | 1.0% | 9 | 91 | 1733 |
| streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. | 5.0% | 0.1% | 0.1% | 2 | 38 | 2926 |
| street_prefix | 0.0% | 0.0% | 0.0% | 0 | 0 | 10 |
| street_suffix | 0.0% | 0.0% | 0.0% | 0 | 0 | 2 |
| po_boxPO boxA numbered mailbox at a post office used as a delivery address instead of a physical street location. Mailwoman tags it as the po_box component; structurally the same family as a subpremise. | 0.0% | 0.0% | 0.0% | 0 | 0 | 1 |
| unitunitA subdivision of a building — apartment, suite, floor — that refines a street address. Mailwoman's unit component; a designator plus identifier forms a subpremise. | 0.0% | 0.0% | 0.0% | 0 | 0 | 5 |
| intersection_a | 0.0% | 0.0% | 0.0% | 0 | 0 | 1 |
| intersection_b | 0.0% | 0.0% | 0.0% | 0 | 0 | 1 |
| venuevenueA named, non-address place — a business, building, park, or stadium. Mailwoman's free-text point-of-interest component, added as a Tier 2 fine label. | 0.0% | 0.0% | 0.0% | 0 | 29 | 1101 |
| subregion | 0.0% | 0.0% | 0.0% | 0 | 272 | 0 |
| street_prefix_particle | 0.0% | 0.0% | 0.0% | 0 | 0 | 6 |
| cedexCEDEX (Courrier d'Entreprise à Distribution Exceptionnelle). A French postal routing for high-volume business mail: a CEDEX code delivers directly from a sorting centre, bypassing the local post office. A common negative-space format Mailwoman must parse. | 0.0% | 0.0% | 0.0% | 0 | 0 | 1 |
| attentionattentionThe core mechanism inside a transformer encoder. Each token's representation is updated by looking at every other token, with learned weights deciding how much each one matters. | 0.0% | 0.0% | 0.0% | 0 | 0 | 1 |
Calibration
| Bucket | Total | Correct | Accuracy |
|---|---|---|---|
| conf > 0.9 | 2475 | 4 | 0.2% |
| conf: 0.7–0.9 | 970 | 0 | 0.0% |
| conf: 0.5–0.7 | 832 | 0 | 0.0% |
| conf < 0.5 | 258 | 0 | 0.0% |
Per-failure-class
| Class | Total | Exact Matchexact matchThe share of eval items whose every component is correct (compared per-span or per-token). Stricter than per-tag F1, which credits partial correctness. | Rate |
|---|---|---|---|
| kryptonite/place-name-venuevenueA named, non-address place — a business, building, park, or stadium. Mailwoman's free-text point-of-interest component, added as a Tier 2 fine label. | 10 | 0 | 0.0% |
| failure/ambiguous-localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. | 4 | 0 | 0.0% |
| failure/streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.-localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.-collision | 5 | 0 | 0.0% |
| failure/tokenization-trap | 3 | 0 | 0.0% |
| kryptonite/place-shaped-venuevenueA named, non-address place — a business, building, park, or stadium. Mailwoman's free-text point-of-interest component, added as a Tier 2 fine label. | 6 | 0 | 0.0% |
| kryptonite/particle-honorific | 9 | 0 | 0.0% |
| failure/numeric-chaos | 1 | 0 | 0.0% |
| kryptonite/disambiguation | 4 | 0 | 0.0% |
| normal | 4503 | 4 | 0.1% |
hybrid
Per-component F1
| Tag | P | R | F1 | TP | FP | FN |
|---|---|---|---|---|---|---|
| regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. | 84.0% | 59.9% | 69.9% | 1920 | 366 | 1285 |
| localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. | 55.4% | 20.6% | 30.1% | 693 | 557 | 2664 |
| countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. | 26.2% | 26.1% | 26.2% | 64 | 180 | 181 |
| dependent_locality | 1.2% | 30.0% | 2.4% | 12 | 956 | 28 |
| postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. | 26.1% | 0.8% | 1.6% | 24 | 68 | 2956 |
| house_number | 9.4% | 0.5% | 1.0% | 9 | 87 | 1733 |
| streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. | 5.3% | 0.1% | 0.1% | 2 | 36 | 2926 |
| street_prefix | 0.0% | 0.0% | 0.0% | 0 | 0 | 10 |
| street_suffix | 0.0% | 0.0% | 0.0% | 0 | 0 | 2 |
| po_boxPO boxA numbered mailbox at a post office used as a delivery address instead of a physical street location. Mailwoman tags it as the po_box component; structurally the same family as a subpremise. | 0.0% | 0.0% | 0.0% | 0 | 0 | 1 |
| unitunitA subdivision of a building — apartment, suite, floor — that refines a street address. Mailwoman's unit component; a designator plus identifier forms a subpremise. | 0.0% | 0.0% | 0.0% | 0 | 0 | 5 |
| intersection_a | 0.0% | 0.0% | 0.0% | 0 | 0 | 1 |
| intersection_b | 0.0% | 0.0% | 0.0% | 0 | 0 | 1 |
| venuevenueA named, non-address place — a business, building, park, or stadium. Mailwoman's free-text point-of-interest component, added as a Tier 2 fine label. | 0.0% | 0.0% | 0.0% | 0 | 29 | 1101 |
| subregion | 0.0% | 0.0% | 0.0% | 0 | 271 | 0 |
| street_prefix_particle | 0.0% | 0.0% | 0.0% | 0 | 0 | 6 |
| cedexCEDEX (Courrier d'Entreprise à Distribution Exceptionnelle). A French postal routing for high-volume business mail: a CEDEX code delivers directly from a sorting centre, bypassing the local post office. A common negative-space format Mailwoman must parse. | 0.0% | 0.0% | 0.0% | 0 | 0 | 1 |
| attentionattentionThe core mechanism inside a transformer encoder. Each token's representation is updated by looking at every other token, with learned weights deciding how much each one matters. | 0.0% | 0.0% | 0.0% | 0 | 0 | 1 |
Calibration
| Bucket | Total | Correct | Accuracy |
|---|---|---|---|
| conf > 0.9 | 2477 | 4 | 0.2% |
| conf: 0.7–0.9 | 972 | 0 | 0.0% |
| conf: 0.5–0.7 | 829 | 0 | 0.0% |
| conf < 0.5 | 257 | 0 | 0.0% |
Per-failure-class
| Class | Total | Exact Matchexact matchThe share of eval items whose every component is correct (compared per-span or per-token). Stricter than per-tag F1, which credits partial correctness. | Rate |
|---|---|---|---|
| kryptonite/place-name-venuevenueA named, non-address place — a business, building, park, or stadium. Mailwoman's free-text point-of-interest component, added as a Tier 2 fine label. | 10 | 0 | 0.0% |
| failure/ambiguous-localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. | 4 | 0 | 0.0% |
| failure/streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.-localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.-collision | 5 | 0 | 0.0% |
| failure/tokenization-trap | 3 | 0 | 0.0% |
| kryptonite/place-shaped-venuevenueA named, non-address place — a business, building, park, or stadium. Mailwoman's free-text point-of-interest component, added as a Tier 2 fine label. | 6 | 0 | 0.0% |
| kryptonite/particle-honorific | 9 | 0 | 0.0% |
| failure/numeric-chaos | 1 | 0 | 0.0% |
| kryptonite/disambiguation | 4 | 0 | 0.0% |
| normal | 4503 | 4 | 0.1% |
hybrid-joint
Per-component F1
| Tag | P | R | F1 | TP | FP | FN |
|---|---|---|---|---|---|---|
| house_number | 77.8% | 87.3% | 82.3% | 1520 | 434 | 222 |
| postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. | 85.6% | 67.7% | 75.6% | 2017 | 339 | 963 |
| regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. | 80.8% | 57.0% | 66.9% | 1828 | 433 | 1377 |
| localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. | 68.7% | 44.3% | 53.9% | 1488 | 679 | 1869 |
| countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. | 12.9% | 12.2% | 12.6% | 30 | 202 | 215 |
| streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. | 3.6% | 3.4% | 3.5% | 99 | 2642 | 2829 |
| venuevenueA named, non-address place — a business, building, park, or stadium. Mailwoman's free-text point-of-interest component, added as a Tier 2 fine label. | 3.1% | 3.2% | 3.1% | 35 | 1091 | 1066 |
| dependent_locality | 0.6% | 15.0% | 1.2% | 6 | 926 | 34 |
| street_prefix | 0.0% | 0.0% | 0.0% | 0 | 0 | 10 |
| street_suffix | 0.0% | 0.0% | 0.0% | 0 | 0 | 2 |
| po_boxPO boxA numbered mailbox at a post office used as a delivery address instead of a physical street location. Mailwoman tags it as the po_box component; structurally the same family as a subpremise. | 0.0% | 0.0% | 0.0% | 0 | 0 | 1 |
| unitunitA subdivision of a building — apartment, suite, floor — that refines a street address. Mailwoman's unit component; a designator plus identifier forms a subpremise. | 0.0% | 0.0% | 0.0% | 0 | 0 | 5 |
| intersection_a | 0.0% | 0.0% | 0.0% | 0 | 0 | 1 |
| intersection_b | 0.0% | 0.0% | 0.0% | 0 | 0 | 1 |
| subregion | 0.0% | 0.0% | 0.0% | 0 | 201 | 0 |
| street_prefix_particle | 0.0% | 0.0% | 0.0% | 0 | 0 | 6 |
| cedexCEDEX (Courrier d'Entreprise à Distribution Exceptionnelle). A French postal routing for high-volume business mail: a CEDEX code delivers directly from a sorting centre, bypassing the local post office. A common negative-space format Mailwoman must parse. | 0.0% | 0.0% | 0.0% | 0 | 0 | 1 |
| attentionattentionThe core mechanism inside a transformer encoder. Each token's representation is updated by looking at every other token, with learned weights deciding how much each one matters. | 0.0% | 0.0% | 0.0% | 0 | 0 | 1 |
Calibration
| Bucket | Total | Correct | Accuracy |
|---|---|---|---|
| conf > 0.9 | 5 | 0 | 0.0% |
| conf: 0.7–0.9 | 730 | 78 | 10.7% |
| conf: 0.5–0.7 | 3421 | 188 | 5.5% |
| conf < 0.5 | 379 | 8 | 2.1% |
Per-failure-class
| Class | Total | Exact Matchexact matchThe share of eval items whose every component is correct (compared per-span or per-token). Stricter than per-tag F1, which credits partial correctness. | Rate |
|---|---|---|---|
| kryptonite/place-name-venuevenueA named, non-address place — a business, building, park, or stadium. Mailwoman's free-text point-of-interest component, added as a Tier 2 fine label. | 10 | 0 | 0.0% |
| failure/ambiguous-localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. | 4 | 0 | 0.0% |
| failure/streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.-localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.-collision | 5 | 0 | 0.0% |
| failure/tokenization-trap | 3 | 0 | 0.0% |
| kryptonite/place-shaped-venuevenueA named, non-address place — a business, building, park, or stadium. Mailwoman's free-text point-of-interest component, added as a Tier 2 fine label. | 6 | 0 | 0.0% |
| kryptonite/particle-honorific | 9 | 0 | 0.0% |
| failure/numeric-chaos | 1 | 0 | 0.0% |
| kryptonite/disambiguation | 4 | 0 | 0.0% |
| normal | 4503 | 274 | 6.1% |