Phase 4a — the reranker exists, its first measurement was confounded, and I nearly shipped the false negative
Status: code landed + unitunitA subdivision of a building — apartment, suite, floor — that refines a street address. Mailwoman's unit component; a designator plus identifier forms a subpremise.-tested. The evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. is VOID. No conclusion about resolution reranking is supported by anything in this document.
What I nearly published
Two resolution-evidence signals, measured on the triaged parity corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus. against the spanspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree. decode's top-5:
| signal | fired | fixed | broke | seg@1 → rerank@1 |
|---|---|---|---|---|
| plausibility guard (veto countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.-centroids) | 3/267 | 0 | 0 | 0.5768 → 0.5768 (+0) |
| resolution specificity (prefer finer tiertierInternal versioning of which label classes the model emits. Tier 1 is the coarse components (country, region, locality, postcode); Tier 2 adds venue, street, house_number; Tier 3 (future) would add attention, po_box, and POI venue subtyping. Historically called 'Stage 1/2/3' before the runtime-pipeline naming made that ambiguous.) | 44/267 | 2 | 18 | 0.5768 → 0.5169 (−16) |
The obvious write-up: "resolution evidence cannot collect the oracle headroom; one signal is inert, the other is actively harmful." That would have been a tidy, plausible, wrong negative — and it would have undercut the entire PhasephaseA milestone in the implementation plan (Foundation, Corpus, Training, Integration, and forward-looking phases). Distinct from stage (runtime pipeline) and tier (model vocabulary).-4 thesis on a bad instrument.
What the check found
Before writing it, the census — can this resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon. even see a streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.?
finest-resolved tier across the 267 parity fixtures (admin + postcode DBs only):
UNRESOLVED 128
locality 118
region 16
country 5
street 0 ← never. not once. no postcode tier, no house_number tier either.
The harness resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon. had no streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.-level data. I constructed it as
WOFSqlitePlaceLookup({ databasePath: [admin-global-priority.db, postcode-locality-intl.db] }) — no
situssitusThe physical site address of a property, as opposed to the owner's mailing address. Parcel records often carry both; the divergence is a real-world data-quality challenge., no interpolationinterpolationA geocoding technique that estimates a coordinate along a street segment based on the house number range. Used as the middle tier of Mailwoman's geocode cascade when exact address-point data is unavailable., no BANBAN (Base Adresse Nationale). France's authoritative open national address register — the highest-quality training source for French addresses, with full component structure./OSMOpenStreetMap (OSM). A community-curated global map database (ODbL-licensed) with addr:* tagged features and place hierarchies. A secondary corpus source and a source of street names. rooftoprooftopGeocoding precision at the building or parcel level — coordinates within a few metres — the highest tier of the geocode cascade. Sourced from address-point and situs data.. It cannot distinguish a correct streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. parseaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates. from a
wrong one because it never resolves streetsstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. at all.
Both results are explained by the confound, not by the thesis:
- The guard was inert because countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.-centroid garbage is rare on this corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus. — the fixtures land on a localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. centroid whether the streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. parseaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates. is right or wrong. Identical evidence for both.
- Specificity actively hurt because "finer" among
{country, region, locality}rewards the hypothesis that produces a localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. reading. For a bare streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. fragment, that is precisely the wrong parseaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates. (Rue Montmartre→ localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.). It fired 44 times and broke 18 by rewarding the failure mode the arc exists to fix.
The shardsshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. exist locally and I simply did not wire them: /mnt/playpen/mailwoman-data/interpolation/
(per-stateregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. US), /ban/ (FR rooftoprooftopGeocoding precision at the building or parcel level — coordinates within a few metres — the highest tier of the geocode cascade. Sourced from address-point and situs data.), /osm/. A valid test runs the geocode cascadegeocode cascadeMailwoman's multi-tier coordinate resolution strategy: first try exact address-point match, then street interpolation, then locality centroid. Each tier is more widely available but progressively coarser.
(geocodeAddress / parseForGeocode + a ShardResolver), not the bare WOFWOF (Who's On First). An open-source gazetteer of places maintained by Mapzen/whosonfirst. Mailwoman builds a custom SQLite database from WOF GeoJSON repos, extended with postcode data, importance scores, and coincident-role relations. admin resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon..
What IS supported
resolver/rerank.ts—rerankByResolution, deliberately minimal: veto implausible resolutions, otherwise keep the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.''s ranking. No score blend, no per-class weightsparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values.. The reason for that austerity is in the file: the moment it grows a hand-tuned ladder we have rebuilt PeliasPeliasAn open-source geocoder, Mailwoman's spiritual predecessor.'s dictionary overrides with extra steps (94 of 276 lines, each resolving a collision by hand).- 7 unitunitA subdivision of a building — apartment, suite, floor — that refines a street address. Mailwoman's unit component; a designator plus identifier forms a subpremise. tests, including the three that encode the judgement calls: rank-2 promotes over a countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.-centroid rank-1 (the arc's whole claim); all-implausible falls back to rank-1 (evidence that everything is bad is not grounds to invent a different answer); a resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon. exception does not veto (an outage is not evidence).
maxResolvedefaults to 5, not 10 — oracle@5 (0.723) captures nearly all of oracle@10's (0.775) ceiling for half the resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon. round-trips.
What Phase 4a still owes
- Re-run on the geocode cascadegeocode cascadeMailwoman's multi-tier coordinate resolution strategy: first try exact address-point match, then street interpolation, then locality centroid. Each tier is more widely available but progressively coarser. with streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.-level shardsshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row.. Until then there is no evidence either way about resolution reranking. The headroom (seg@1 0.577 → oracle@5 0.723) is measured and real; whether this signal collects it is untested.
- Expect the answer to be localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for.-split: the US has situssitusThe physical site address of a property, as opposed to the owner's mailing address. Parcel records often carry both; the divergence is a real-world data-quality challenge./TIGERTIGERThe US Census Topologically Integrated Geographic Encoding and Referencing database. Used as a corpus source for street-segment data. streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. coveragecoverageThe fraction of a population or region for which a data source has real, non-placeholder entries — e.g. 47% rooftop coverage on Texas addresses. Distinct from accuracy on the rows that are present., so the resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon. can adjudicate there; CZ/FR/PT bare fragments resolve to a locality centroidlocality centroidThe representative centre point of a city or locality, used as a coarse coordinate when no exact address point is available — the coarsest tier of the geocode cascade. regardless, so the evidence may genuinely not exist for them. A rerank that helps US and is inert intl would be a perfectly good result — but it has to be measured, per-localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for., not assumed.
- Only then: the isotonic ambiguity gate (4b) and option C (4c).
The lesson, which is the same one twice in two days
The v3.0.0 probe looked like a falsified architecture and was a mis-specified LR. This looked like a
falsified rerank and was an unwired resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon.. Both times the tell was a number that could not
reproduce something already known — there, token@1 0.348 against a known 0.573; here, a resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon.
that never once produced the tiertierInternal versioning of which label classes the model emits. Tier 1 is the coarse components (country, region, locality, postcode); Tier 2 adds venue, street, house_number; Tier 3 (future) would add attention, po_box, and POI venue subtyping. Historically called 'Stage 1/2/3' before the runtime-pipeline naming made that ambiguous. the whole experiment depends on.
Check the instrument before the hypothesis. The aggregate is not the verdict; the evidence is.