Phase 4a verdict — the rerank thesis has a circularity
Status: VALID measurement (the instrument was checked first this time). Result: NULL, with a mechanism. The reranker is built, tested, and correct. Resolution evidence cannot adjudicate the class the arc targets — not because the signal is weak, but because it does not exist for those inputs.
The measurement
Full geocode cascadegeocode cascadeMailwoman's multi-tier coordinate resolution strategy: first try exact address-point match, then street interpolation, then locality centroid. Each tier is more widely available but progressively coarser. — WOFWOF (Who's On First). An open-source gazetteer of places maintained by Mapzen/whosonfirst. Mailwoman builds a custom SQLite database from WOF GeoJSON repos, extended with postcode data, importance scores, and coincident-role relations. admin + per-stateregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. situssitusThe physical site address of a property, as opposed to the owner's mailing address. Parcel records often carry both; the divergence is a real-world data-quality challenge./interpolationinterpolationA geocoding technique that estimates a coordinate along a street segment based on the house number range. Used as the middle tier of Mailwoman's geocode cascade when exact address-point data is unavailable. + BANBAN (Base Adresse Nationale). France's authoritative open national address register — the highest-quality training source for French addresses, with full component structure. national register +
BANBAN (Base Adresse Nationale). France's authoritative open national address register — the highest-quality training source for French addresses, with full component structure. streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.-centroids (#1042) + OSMOpenStreetMap (OSM). A community-curated global map database (ODbL-licensed) with addr:* tagged features and place hierarchies. A secondary corpus source and a source of street names. rooftoprooftopGeocoding precision at the building or parcel level — coordinates within a few metres — the highest tier of the geocode cascade. Sourced from address-point and situs data.. Wiring copied verbatim from
mailwoman/eval-harness/gauntlet/harness.ts (the cascade production actually runs).
| triaged parity (n=267) | Paris fixture (n=63) | ||
|---|---|---|---|
| seg@1 | 0.5768 | seg@1 | 0.7619 |
| rerank@1 | 0.5768 (+0) | rerank@1 | 0.7460 (−1) |
| oracle@5 | 0.7228 | oracle@5 | 0.9048 |
| fired | 1 fixture (0 fixed / 0 broke) | fired | 1 fixture (0 fixed / 1 broke) |
Why — the instrument check that Phase 4a's first attempt skipped
Evidence rate: the share of fixtures where ANY hypothesis reached streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.-level resolution.
| localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for. | n | evidence rate |
|---|---|---|
| US | 75 | 0.01 |
| ZZ | 53 | 0.00 |
| FR | 29 | 0.03 |
| NL | 18 | 0.22 |
| DE | 13 | 0.23 |
| NZ / AU / NO / PT / PL / RO | 5–22 | 0.00 |
Paris fixture, FR, with ban/street-centroids-fr.db present and loaded: 314 hypothesis geocodes →
admin 314, street 1. An evidence rate of 1.6%.
The circularity
The arc targets bare fragments (66% of streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. failures, night-3 partition). A bare fragment:
- has no house numberhouse numberThe numeric or alphanumeric identifier of a building on a street. Mailwoman's house_number component; its position relative to the street name flips between locales. → the address-point tiertierInternal versioning of which label classes the model emits. Tier 1 is the coarse components (country, region, locality, postcode); Tier 2 adds venue, street, house_number; Tier 3 (future) would add attention, po_box, and POI venue subtyping. Historically called 'Stage 1/2/3' before the runtime-pipeline naming made that ambiguous. cannot fire (it is keyed by number),
- has no localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. → the streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.-centroid tiertierInternal versioning of which label classes the model emits. Tier 1 is the coarse components (country, region, locality, postcode); Tier 2 adds venue, street, house_number; Tier 3 (future) would add attention, po_box, and POI venue subtyping. Historically called 'Stage 1/2/3' before the runtime-pipeline naming made that ambiguous. has nothing to scope a streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. lookup by,
- therefore resolves to an admin centroid — identically, for every hypothesis in the k-best list.
So the resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon. returns the same evidence for the right parseaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates. and the wrong one. It is not that the signal is noisy; there is no signal. Resolution evidence can only adjudicate addresses that are already resolvable — which are largely the ones the parseaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates. gets right anyway.
The −1 on Paris is the whole story in miniature: the single fixture where evidence existed, the reranker fired, and it was wrong.
What this does and does not overturn
Does NOT overturn: the k-best headroom is real and measured — oracle@5 0.723 (parity) and 0.905 (Paris) against a shipped 0.573. The spanspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree. decode genuinely puts the right answer in the list. That stands.
Does overturn: the assumption — mine and the arc's, stated on 2026-07-15 — that "the arbiter for the k-best list is the resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon.." For contextful addresses the resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon. could arbitrate, but those barely need it. For the bare fragments that need it, the resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon. is blind. The arbiter has to be something else.
The direction this points (NOT chased here — that would be signal #3 on a losing streak)
The resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon. answers "where is this?" The useful question for reranking a bare fragment is
"is this a streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. name at all?" — an existence check against the gazetteergazetteerA geographical index that maps place names and postcodes to real-world coordinates. Mailwoman uses a custom-built Who's On First (WOF) SQLite database as its gazetteer — the 'atlas' half of the grammar/atlas architecture./BANBAN (Base Adresse Nationale). France's authoritative open national address register — the highest-quality training source for French addresses, with full component structure. name index, not
a geocode. ban/street-centroids-fr.db contains every FR streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. name; asking whether
Rue de Rome appears in it needs no localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. and no house numberhouse numberThe numeric or alphanumeric identifier of a building on a street. Mailwoman's house_number component; its position relative to the street name flips between locales..
That is a lexicon lookup, not resolution — a different signal with a different failure surface, and
it interacts directly with #1142 (the matched vs importance split: "is this a known name" is
exactly the matched bit that today's multiply destroys). It earns a probe only with the same bar
the others got: a pre-registered gate and a measured win.
Two signals have now failed (plausibility veto: inert; resolution specificity: −16, rewards the localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.-reading failure mode). Per the treadmill guard, a third goes to the operator before it goes to a branch.
Phase 4 re-plan
- 4a rerank — DONE, negative. Code stays (7 tests, sound); it is inert by default because
maxResolveonly vetoes countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.-centroids, which is a real if rare guard. It ships nothing on its own. - 4b (isotonic ambiguity gate) — still valuable, and now more so: if evidence cannot pick the winner, calibrated confidence is what lets a caller know the answer is uncertain. Unblocked.
- 4c (option C: kind-posterior + recallrecallOf the spans whose gold label is a given tag, the fraction the model found. High recall means few misses. Paired with precision to compute F1.-weighted lossloss functionA number measuring how wrong the model's predictions are on a batch of examples. Training minimizes it. Mailwoman's loss combines per-token negative log-likelihood with the CRF sequence loss.) — now the primary lever for the bare-fragment class, not a follow-up. It attacks the parseaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates. where evidence cannot.
And the standing fact this does not change: the v7 streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. floor is 0.90. seg@1 is 0.577, oracle@5 is 0.723. Even a perfect reranker would have landed short. The floors need 4c and/or more trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input. — the decode was never going to get there alone.