2026-07-17 — Cascade-viability probe: shape-routed delegation is falsified; "easy-looking" is the hardest class
The delegation vision under test: a query-shape front stagestageOne of the dataflow stages in the runtime pipeline (normalize, locale gate, kind classify, phrase group, token classify, sequence correct, reconcile, resolve). Distinct from tier (model vocabulary) and phase (plan milestone). that handles easy shapes deterministically
(no encoderencoderThe part of a transformer that turns input tokens into contextualized vector representations. Mailwoman's classifier is a small encoder-only transformer (~30M parameters). pass) and escalates the rest to the full modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' — the cascade form of the mixture idea, made
principled by calibrated confidence. This probe measures it on the four standing arenasarenaA standardized test set probing one capability: libpostal (clean canonical), perturb (noisy and degraded), postal (edge formats). Each arena answers a different question about where rule vs neural wins.: what fraction a
confidence-gated front tiertierInternal versioning of which label classes the model emits. Tier 1 is the coarse components (country, region, locality, postcode); Tier 2 adds venue, street, house_number; Tier 3 (future) would add attention, po_box, and POI venue subtyping. Historically called 'Stage 1/2/3' before the runtime-pipeline naming made that ambiguous. would absorb, how often its deterministic parseaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates. matches gold, and how often
the full modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' matches gold on those same rows. Runner: scratchpad/cascade-probe.run.ts (kinds via
classifyKindSync at confidence ≥0.7; handlers: postcode_only → the format-hit spanspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree., locality_only →
bare localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. or the doubleton split at a trailing regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. abbreviation).
The numbers
| arenaarenaA standardized test set probing one capability: libpostal (clean canonical), perturb (noisy and degraded), postal (edge formats). Each arena answers a different question about where rule vs neural wins. | absorbed | front-tiertierInternal versioning of which label classes the model emits. Tier 1 is the coarse components (country, region, locality, postcode); Tier 2 adds venue, street, house_number; Tier 3 (future) would add attention, po_box, and POI venue subtyping. Historically called 'Stage 1/2/3' before the runtime-pipeline naming made that ambiguous. exact | modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' exact (same rows) |
|---|---|---|---|
| golden-us (2660) | 209 (7.9%) | 0.124 | 0.689 |
| golden-fr (1546) | 34 (2.2%) | 0.000 | 0.559 |
| golden-adv (49) | 4 (8.2%) | 0.500 | 0.750 |
| parity (376) | 105 (27.9%) | 0.095 | 0.390 |
| overall | 352/4631 (7.6%) | 0.108 | 0.588 |
The front tiertierInternal versioning of which label classes the model emits. Tier 1 is the coarse components (country, region, locality, postcode); Tier 2 adds venue, street, house_number; Tier 3 (future) would add attention, po_box, and POI venue subtyping. Historically called 'Stage 1/2/3' before the runtime-pipeline naming made that ambiguous. is 5× worse than the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' on the very rows the shape detector calls easy (0.108 vs 0.588 exact). Latency upside, even if it were free: 7.6% of encoderencoderThe part of a transformer that turns input tokens into contextualized vector representations. Mailwoman's classifier is a small encoder-only transformer (~30M parameters). passes ≈ 0.5 ms saved per average query. Falsified on both axes at once.
The mechanism — shape simplicity IS semantic ambiguity
The failure samples say why, and it is not a tunable defect:
"Hummingbird Ln VT" detector: locality_only → front tier: locality "Hummingbird Ln"
gold: street "Hummingbird Ln", region VT
"Finel Hollow Road, VT" doubleton split → locality "Finel Hollow Road"
gold: STREET "Finel Hollow Road"
"Vermont, USA" detector: locality_only → gold: region + country
The inputs that LOOK easy — short, undecorated toponymstoponymA proper name for a geographic place. — are exactly the class the entire Track B / fr-fragment arc has been about: a bare name gives no structural evidence of whether it is a streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels., a localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy., or a regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.. Deciding that requires knowing which names are which — the atlas or the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' — which is precisely what a shape detector, by construction, does not have. On this domain, ease-of-shape and ease-of-parseaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates. are anti-correlated: long inputs are self-disambiguating (a house numberhouse numberThe numeric or alphanumeric identifier of a building on a street. Mailwoman's house_number component; its position relative to the street name flips between locales. licenses the streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels., a postcode anchorsanchor inferenceA technique where structured knowledge (postcode locations, gazetteer place names) is injected into the model as soft input features — not as deterministic overrides. The model still decides the final labels, but the anchor signal biases it toward correct admin tags. the localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.); short inputs are pure ambiguity. The modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' earns its keep most on the smallest queries — the opposite of the cascade's premise. Note the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' itself scores only 0.588 on the absorbed rows: they are hard for everyone; the front tiertierInternal versioning of which label classes the model emits. Tier 1 is the coarse components (country, region, locality, postcode); Tier 2 adds venue, street, house_number; Tier 3 (future) would add attention, po_box, and POI venue subtyping. Historically called 'Stage 1/2/3' before the runtime-pipeline naming made that ambiguous. just makes hard rows 5× worse.
This is the third instrument to convict shape-conditioned routing this week: the queryShape localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. bias
(M1: −7.8 localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy., venuevenueA named, non-address place — a business, building, park, or stadium. Mailwoman's free-text point-of-interest component, added as a Tier 2 fine label. absorption), the deletion counter-case (New York, NY needing the bias), and
now the cascade probe. The consistent shape: conditioning behavior on a shape detector fails wherever
the shape is ambiguous, and the ambiguous cases are the ones that matter.
What survives
postcode_onlyfronting is fine but nearly empty here (2 rows; more in autocomplete traffic) and the postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. binary already serves it downstream — no architecture change needed.- Atlas-verified fronting — absorb a bare name only after a candidate-table hit confirms it is a known localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. — is the one honest form left. But that front tiertierInternal versioning of which label classes the model emits. Tier 1 is the coarse components (country, region, locality, postcode); Tier 2 adds venue, street, house_number; Tier 3 (future) would add attention, po_box, and POI venue subtyping. Historically called 'Stage 1/2/3' before the runtime-pipeline naming made that ambiguous. is a gazetteergazetteerA geographical index that maps place names and postcodes to real-world coordinates. Mailwoman uses a custom-built Who's On First (WOF) SQLite database as its gazetteer — the 'atlas' half of the grammar/atlas architecture. lookup, i.e. the resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon.; the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' + atlas channel already perform that arbitrationarbitrationA pipeline stage that compares rule-based (v0) and neural classifier output, resolving disagreements via a policy registry. Built and merged but not promoted — the coordinate gate showed label-F1 gains came at the cost of worse geocoding. with learned weighting (M1 priced the channel at +10.4). Building it as a bypass buys ~0.5 ms and a second code path to keep honest.
- The scoped doubleton bias (PR #1148, four-line guard on a soft priorsoft priorOutside knowledge fed to the model or resolver as overridable evidence (a feature, a score term) rather than a hard filter. A focus-country hint becomes an anchor feature; a focus-point becomes a ranking term.) remains the template for where shape knowledge helps: small, priced, scoped — inside the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.''s decode, not in front of it.
Verdict
Do not build the shape-routed cascade. Keep the single-modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' + soft-channels architecture; spend the delegation instinct on the resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon.-as-arbiter direction (#727 stagestageOne of the dataflow stages in the runtime pipeline (normalize, locale gate, kind classify, phrase group, token classify, sequence correct, reconcile, resolve). Distinct from tier (model vocabulary) and phase (plan milestone).-2), where the atlas judges parsesaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates. rather than routing inputs. The probe cost one evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. sweep and closes the question with receipts.