Per-address-type head-to-head — neural vs v0 (the Pelias port)
Self-emitted by scripts/eval/per-type-report.ts. Both parsers graded through the same resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon. (Part A) or on parseaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates. structure (Part B). Turns the stateregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.-of-affairs blog's anecdotes into per-type rates.
Part A — coordinate accuracy by bucket (real OpenAddresses US, 2000 rows)
Both parsers through the same resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon., against real address pointssitus dataA dataset of exact address-point coordinates (rooftop-level). Mailwoman's geocoder uses a national situs layer (124.9M US points built from state address-point sources) as the highest-precision tier of the geocode cascade.. Slices overlap (a row can be both directional and multi-word-localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.); plain is the complement.
| bucket | n | neural loc-match | v0 loc-match | neural coord p50 km | v0 coord p50 km |
|---|---|---|---|---|---|
| all rows | 2000 | 84.0% | 82.1% | 3.3 | 3.3 |
| directional streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. | 506 | 87.2% | 85.2% | 3.1 | 3.1 |
| multi-word localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. | 509 | 90.0% | 84.7% | 3.1 | 3.0 |
| plain (neither) | 1152 | 81.5% | 80.9% | 3.4 | 3.4 |
Part B — parse-structure win-rate on the headline types (generated, 150 each)
OpenAddressesOpenAddresses (OA). A global open aggregation of address points collected from many official sources. A primary source of component-supervised training data outside proprietary registries. has ~no PO boxesPO boxA numbered mailbox at a post office used as a delivery address instead of a physical street location. Mailwoman tags it as the po_box component; structurally the same family as a subpremise., intersectionsintersectionAn address that names a location by two crossing streets ('5th & Main') rather than a number and street. Mailwoman tags the two streets as intersection_a and intersection_b — a negative-space format that starved the early model., or unitsunitA subdivision of a building — apartment, suite, floor — that refines a street address. Mailwoman's unit component; a designator plus identifier forms a subpremise., so these are templated from real OA cities; the truth is the known TYPE. We score whether each parser emits the correct STRUCTURE.
| type | n | neural correct | v0 correct |
|---|---|---|---|
| po_boxPO boxA numbered mailbox at a post office used as a delivery address instead of a physical street location. Mailwoman tags it as the po_box component; structurally the same family as a subpremise. | 150 | 100.0% | 0.0% |
| intersectionintersectionAn address that names a location by two crossing streets ('5th & Main') rather than a number and street. Mailwoman tags the two streets as intersection_a and intersection_b — a negative-space format that starved the early model. | 150 | 82.0% | 0.0% |
| unitunitA subdivision of a building — apartment, suite, floor — that refines a street address. Mailwoman's unit component; a designator plus identifier forms a subpremise. (keeps designatordesignatorThe closed-vocabulary leading word of a secondary-address phrase — 'Apt', 'Suite', 'Floor', 'PO Box', 'Level' — paired with an identifier to form a complete subpremise.) | 150 | 100.0% | 0.0% |
Reading
- The US edge is not uniform. Neural's overall localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.-match lead (+1.9pp) concentrates on addresses with structure the rules enginerule-based classifierMailwoman's legacy v0 parser — a library of deterministic token classifiers (house number, street suffix, postcode, place name, etc.) composed by priority. Now primarily used for corpus labeling, fallback classification, and arbitration diagnostics. fumbles: multi-word localities +5.3pp, directional streetsstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. +2.0pp. On plain single-word-citylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. addresses the two are a near-tie (+0.6pp). Coordinate p50 is identical across buckets — the difference is which CITYlocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. resolves, not the point's precisionprecisionOf the spans the model labeled as a given tag, the fraction it got right. High precision means few false positives. Paired with recall to compute F1..
- Structured types are a rout, by construction. The PeliasPeliasAn open-source geocoder, Mailwoman's spiritual predecessor. port emits 0.0% correct structure on PO boxesPO boxA numbered mailbox at a post office used as a delivery address instead of a physical street location. Mailwoman tags it as the po_box component; structurally the same family as a subpremise., intersectionsintersectionAn address that names a location by two crossing streets ('5th & Main') rather than a number and street. Mailwoman tags the two streets as intersection_a and intersection_b — a negative-space format that starved the early model., and unitsunitA subdivision of a building — apartment, suite, floor — that refines a street address. Mailwoman's unit component; a designator plus identifier forms a subpremise. — no
po_boxtag, an intersectionintersectionAn address that names a location by two crossing streets ('5th & Main') rather than a number and street. Mailwoman tags the two streets as intersection_a and intersection_b — a negative-space format that starved the early model. side dropped, the unitunitA subdivision of a building — apartment, suite, floor — that refines a street address. Mailwoman's unit component; a designator plus identifier forms a subpremise. designatordesignatorThe closed-vocabulary leading word of a secondary-address phrase — 'Apt', 'Suite', 'Floor', 'PO Box', 'Level' — paired with an identifier to form a complete subpremise. stripped. Neural emits them because it was trained on the negative spacenegative spaceAddress formats the neural model was never trained on because they're absent from the base training corpus — PO boxes, CEDEX, intersections, units. The parity campaign targets negative space through synthetic shards and corpus augmentation.. The one honest gap: intersectionsintersectionAn address that names a location by two crossing streets ('5th & Main') rather than a number and street. Mailwoman tags the two streets as intersection_a and intersection_b — a negative-space format that starved the early model., where neural is 82.0% — the templatedA & Bform trips it ~1 in 6. - Where we do NOT win: nowhere does v0 beat neural per-bucket here, but the plain-address tie shows neural isn't meaningfully better on the simplest addresses, and the intersectionintersectionAn address that names a location by two crossing streets ('5th & Main') rather than a number and street. Mailwoman tags the two streets as intersection_a and intersection_b — a negative-space format that starved the early model. miss is our internal frontier, not a v0 advantage.
- Caveat: Part B is templated (real OA cities, synthetic forms) — it measures parseaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates.-structure capability, not real-world frequency.