Resolver routing + end-to-end eval — execution plan (2026-05-30)

Direction C. Operationalize the capability map by routing each input to the parser that wins on it, feeding the (already-shipped) WOFWOF (Who's On First). An open-source gazetteer of places maintained by Mapzen/whosonfirst. Mailwoman builds a custom SQLite database from WOF GeoJSON repos, extended with postcode data, importance scores, and coincident-role relations. resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon. — and build the first end-to-end "address → correct place" benchmark to prove it. US-first. DeepSeek-signed (consult: .agents/skills/deepseek-consult/session-notes-2026-05-30-resolver.md).

Why

The resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon. (PhasephaseA milestone in the implementation plan (Foundation, Corpus, Training, Integration, and forward-looking phases). Distinct from stage (runtime pipeline) and tier (model vocabulary). 4) is shipped and works end-to-end: parse → resolveTree → WOF place + coords, with parent-constraint inheritance and FTS5FTS5SQLite's built-in full-text-search module (with BM25 ranking). Mailwoman uses it for prefix and name matching against the gazetteer./population/ proximity ranking over US admin + postcodespostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one.. But it consumes neural-only output. The capability map (three unbiased arenasarenaA standardized test set probing one capability: libpostal (clean canonical), perturb (noisy and degraded), postal (edge formats). Each arena answers a different question about where rule vs neural wins.) says neither parser dominates — input quality decides: rules win clean/canonical (libpostallibpostalAn open-source C address parser used by Pelias. Mailwoman's rule-based v0 and neural classifier supersede it. v0 29% > neural 16%), neural wins noisy/degraded (perturbation neural 61% > v0 39%). So neural-only leaves v0's clean-input win on the table. And we have no end-to-end accuracyharness pass rateWhether a full address parses end-to-end with every component correct and well-segmented and no orphan spans. Stricter than label F1 — a model can win on recall yet fail the harness if boundaries slip. number — the resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon. has unitunitA subdivision of a building — apartment, suite, floor — that refines a street address. Mailwoman's unit component; a designator plus identifier forms a subpremise. tests but nothing measures whole-stack correctness.

Architecture (target)

A per-input routing layerlayerOne transformer block — attention plus a feed-forward network, with normalization and residual connections — applied to every position. Stacking layers lets the model build up richer representations; Mailwoman's encoder has 6. in front of the resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon.:

• Cheap lexical "canonical-ness" scorer (pre-parseaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates., O(n) on the raw string): comma/delimiter count, capital-word ratio, gazetteergazetteerA geographical index that maps place names and postcodes to real-world coordinates. Mailwoman uses a custom-built Who's On First (WOF) SQLite database as its gazetteer — the 'atlas' half of the grammar/atlas architecture.-tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. hits (WOFWOF (Who's On First). An open-source gazetteer of places maintained by Mapzen/whosonfirst. Mailwoman builds a custom SQLite database from WOF GeoJSON repos, extended with postcode data, importance scores, and coincident-role relations. FSTFST (finite-state transducer). A compact automaton that reads an input sequence and emits an output sequence. Mailwoman encodes gazetteer names and street affixes as FSTs for fast prefix matching and prior injection without search overhead./bloom), ZIP-shape digits, word-length distribution → a tiny logistic regression → p(v0-wins). Interpretable + debuggable; no extra parseaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates. cost.
• Confidence bands: p > 0.8 → v0; p < 0.2 → neural; the narrow ambiguous band → resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon.-as-arbiter (run both, resolve both, pick the higher resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon.-confidence result). Caps the 2× parseaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates. cost to a small traffic slice.
• resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon.-as-arbiter is the powerful core mechanism — it makes resolvability (gazetteergazetteerA geographical index that maps place names and postcodes to real-world coordinates. Mailwoman uses a custom-built Who's On First (WOF) SQLite database as its gazetteer — the 'atlas' half of the grammar/atlas architecture. support) the routing signal, directly optimizing the end goal. Used three ways: online fallback (ambiguous band), offline auto-labeler for the scorer, and evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. oracle.
• No fusion in v1 (merging v0's flat record + neural's tree is brittle — a bad PARENT_OF nesting poisons the resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon.'s parent-constraint inheritance, its main strength). Revisit only if data demands it.

The output-contract constraint

v0 → flat ClassificationRecord[]; neural → AddressTree; resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon. consumes a tree. So routing v0 into the resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon. requires a flat→tree adapter (PARENT_OF containment). This adapter is the linchpin — on the critical path for every v0-involving baseline.

Build order (each step yields an evaluable artifact)

PhasephaseA milestone in the implementation plan (Foundation, Corpus, Training, Integration, and forward-looking phases). Distinct from stage (runtime pipeline) and tier (model vocabulary). 1 — prove the thesis (no routing code yet):

1. EvalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. harness + ground truthground truthThe correct answer for an example, used as the standard a prediction is graded against. Mailwoman's ground truth is the hand-labeled golden set; its quality caps achievable accuracy.. WOFWOF (Who's On First). An open-source gazetteer of places maintained by Mapzen/whosonfirst. Mailwoman builds a custom SQLite database from WOF GeoJSON repos, extended with postcode data, importance scores, and coincident-role relations.-bootstrap: sample stratified US WOFWOF (Who's On First). An open-source gazetteer of places maintained by Mapzen/whosonfirst. Mailwoman builds a custom SQLite database from WOF GeoJSON repos, extended with postcode data, importance scores, and coincident-role relations. places (localities/regionsregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality./postcodespostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one.; urban/rural/territories) → render to address strings via templates (full / no-streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. / stateregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.+ZIP) → canonical + perturbed variants (lowercase, no-comma, glued NY14201, mis-split ZIP, OCR). Labelcomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag. = WOFWOF (Who's On First). An open-source gazetteer of places maintained by Mapzen/whosonfirst. Mailwoman builds a custom SQLite database from WOF GeoJSON repos, extended with postcode data, importance scores, and coincident-role relations. id at the rendered specificity (hierarchy-tolerant). Plus the golden 4561-row set as a regression detector only (it's PeliasPeliasAn open-source geocoder, Mailwoman's spiritual predecessor.-lineage — overstatesregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. v0). Metrics: hierarchy-tolerant Place-Match Acc@1Acc@1 (accuracy at 1). The share of eval rows whose top-ranked resolver candidate is the correct place, regardless of coordinate distance — 'did we pick the right place', independent of geocode precision. (primary), coordinate error p50/p90, component-F1 (isolates parser vs resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon. error), resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon. success rate.
2. v0→tree adapter (PARENT_OF + tree builder). Preliminary gate: v0-via-adapter must reach ≥85% of v0's standalone component accuracy on canonical golden — else the adapter is destroying info; fix before proceeding.
3. Single-parser baselines: neural-only, v0-via-adapter, on the evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. suite.
4. ResolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon.-arbiter (offline script): dual-parseaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates. + dual-resolve + pick best score → the arbiter and oracle baselines.

KILL/CONTINUE GATE (the point of PhasephaseA milestone in the implementation plan (Foundation, Corpus, Training, Integration, and forward-looking phases). Distinct from stage (runtime pipeline) and tier (model vocabulary). 1):

On WOFWOF (Who's On First). An open-source gazetteer of places maintained by Mapzen/whosonfirst. Mailwoman builds a custom SQLite database from WOF GeoJSON repos, extended with postcode data, importance scores, and coincident-role relations.-bootstrap, the tuned arbiter must beat the better single-parser baseline by ≥5pp Acc@1Acc@1 (accuracy at 1). The share of eval rows whose top-ranked resolver candidate is the correct place, regardless of coordinate distance — 'did we pick the right place', independent of geocode precision. on the clean subset, not regress >1–2pp on the perturbed subset, and ≥3pp overall, with coordinate error not >10% worse. Oracle sits above arbiter (= headroom for the router).

If met → routing is worth building (PhasephaseA milestone in the implementation plan (Foundation, Corpus, Training, Integration, and forward-looking phases). Distinct from stage (runtime pipeline) and tier (model vocabulary). 2). If arbiter ≈ neural-only → routing is a dead end; pivot to coveragecoverageThe fraction of a population or region for which a data source has real, non-placeholder entries — e.g. 47% rooftop coverage on Texas addresses. Distinct from accuracy on the rows that are present. (the backlog's B3'/B5) instead.

PhasephaseA milestone in the implementation plan (Foundation, Corpus, Training, Integration, and forward-looking phases). Distinct from stage (runtime pipeline) and tier (model vocabulary). 2 — build routing (only if the gate passes): 5. Auto-labelcomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag. a real unlabeled corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus. (OpenAddressesOpenAddresses (OA). A global open aggregation of address points collected from many official sources. A primary source of component-supervised training data outside proprietary registries. US strings): run both, resolve both, labelcomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag. by resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon.-score delta — drop both-garbage rows (both below calibrated min-score) and drop marginal rows (delta < win margin; these belong in the online ambiguous band). Calibrate min-score + win-margin on the WOFWOF (Who's On First). An open-source gazetteer of places maintained by Mapzen/whosonfirst. Mailwoman builds a custom SQLite database from WOF GeoJSON repos, extended with postcode data, importance scores, and coincident-role relations.-bootstrap set first. 6. Lexical quality scorer (LR on the auto-labelscomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag.) — the cheap approximation of the arbiter, so we pay 2× parseaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates. only on the ambiguous band. 7. Online router with confidence bands; thresholds set from the evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. suite; ambiguous band → dual-parseaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates. + arbiter (reuse the shipped resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon.). 8. Tune + monitor: log 1% dual-parseaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates. samples, compare router vs arbiter, retrain on decay.

Follow-on (parallel, droppable): OpenAddressesOpenAddresses (OA). A global open aggregation of address points collected from many official sources. A primary source of component-supervised training data outside proprietary registries. evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. track (~10k real US {address, lat/lon} points) → independent great-circle coordinate-error number for external credibility. Not on the gate's critical path.

First PR scope (Phase 1, steps 1–2)

The evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. harness + WOFWOF (Who's On First). An open-source gazetteer of places maintained by Mapzen/whosonfirst. Mailwoman builds a custom SQLite database from WOF GeoJSON repos, extended with postcode data, importance scores, and coincident-role relations.-bootstrap generator + the v0→tree adapter (with its preliminary 85% gate). That unblocks the baselines + the kill/continue gate. No production-pipelinestaged pipelineMailwoman's runtime architecture: a sequence of pure-function stages (normalize → query-shape → locale-gate → kind-classifier → phrase-grouper → classifier → decoder) connected by typed handoffs. Each stage is published as its own npm package. changes — all of PhasephaseA milestone in the implementation plan (Foundation, Corpus, Training, Integration, and forward-looking phases). Distinct from stage (runtime pipeline) and tier (model vocabulary). 1 is offline scripts + one adapter.

Risks / honesty guards

• EvalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. circularity: WOFWOF (Who's On First). An open-source gazetteer of places maintained by Mapzen/whosonfirst. Mailwoman builds a custom SQLite database from WOF GeoJSON repos, extended with postcode data, importance scores, and coincident-role relations.-bootstrap resolves WOFWOF (Who's On First). An open-source gazetteer of places maintained by Mapzen/whosonfirst. Mailwoman builds a custom SQLite database from WOF GeoJSON repos, extended with postcode data, importance scores, and coincident-role relations.-rendered strings back to WOFWOF (Who's On First). An open-source gazetteer of places maintained by Mapzen/whosonfirst. Mailwoman builds a custom SQLite database from WOF GeoJSON repos, extended with postcode data, importance scores, and coincident-role relations.. Mitigated by 142k-candidate ambiguity (real Springfield problem) + perturbation stress + the golden regression set; the OA follow-on track is the independent check. Don't oversell WOFWOF (Who's On First). An open-source gazetteer of places maintained by Mapzen/whosonfirst. Mailwoman builds a custom SQLite database from WOF GeoJSON repos, extended with postcode data, importance scores, and coincident-role relations.-bootstrap as "real-world" until OA lands.
• Admin-level ceiling: the resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon. resolves localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy./regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality./postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one., not streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels./house. "Correct place" = right citylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy./ZIP, not right building. StreetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.-level (OSMOpenStreetMap (OSM). A community-curated global map database (ODbL-licensed) with addr:* tagged features and place hierarchies. A secondary corpus source and a source of street names./OpenAddressesOpenAddresses (OA). A global open aggregation of address points collected from many official sources. A primary source of component-supervised training data outside proprietary registries.) is a later phasephaseA milestone in the implementation plan (Foundation, Corpus, Training, Integration, and forward-looking phases). Distinct from stage (runtime pipeline) and tier (model vocabulary)..
• ResolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon. score isn't a probability: calibrate before using it as arbiter threshold or auto-labelcomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag. signal; reject below a min-score (both-garbage).