POI query board v1 — baseline report (spec §3.6)

Date: 2026-07-19. Command: mailwoman eval poi-board. Fixtures: mailwoman/eval-harness/fixtures/poi-board.jsonl (45 cases, committed). DB: /mnt/playpen/mailwoman-data/poi/poi-full.db (the full 4-countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. build — 13,681,698 rows, release 2026-05-20.0; not a --limit sample, per the poipoint of interest (POI). A named place that is not strictly an address — landmark, transit stop, venue, amenity, or franchise. Mailwoman tags these as venue and resolves them through the gazetteer.-layerlayerOne transformer block — attention plus a feed-forward network, with normalization and residual connections — applied to every position. Stacking layers lets the model build up richer representations; Mailwoman's encoder has 6.-runbook's warning against grading coveragecoverageThe fraction of a population or region for which a data source has real, non-placeholder entries — e.g. 47% rooftop coverage on Texas addresses. Distinct from accuracy on the rows that are present. off a limited build). ResolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon.: default FTS admin shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. (admin-global-priority.db).

Status: BASELINE ONLY. No floors yet — this is the pre-registered first run (spec §3.6: "written floors set at PhasephaseA milestone in the implementation plan (Foundation, Corpus, Training, Integration, and forward-looking phases). Distinct from stage (runtime pipeline) and tier (model vocabulary).-1 baseline"). A case failing here is not a regression; there is nothing to regress from. Floors land in a follow-up PR once these numbers exist to set them against.

Composition (45 cases)

slice	n	notes
category + anchor	22	≥4 per countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.: US 6, CA 5, MX 5, FR 6 — well-known citylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.-center golds, 25 km tolerance
localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for.-gated synonym	5	3 exact-localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for.-gated (er/en-US, petrol station/en-GB, mailbox/en-CA), 2 ungated
abstain	7	3 build-local infra (anchored — see note below), 3 bare no-anchor shipped categories, 1 gated-synonym→build-local
address-guard	6	4 full addresses + 2 venuevenueA named, non-address place — a business, building, park, or stadium. Mailwoman's free-text point-of-interest component, added as a Tier 2 fine label.-led (category, address) — must NOT take the poipoint of interest (POI). A named place that is not strictly an address — landmark, transit stop, venue, amenity, or franchise. Mailwoman tags these as venue and resolves them through the gazetteer. path
near-miss / robustness	6	comma anchors, multiword synonyms, multi-segmentsegmentA punctuation-bounded chunk of the normalized input — the comma-separated parts of 'Portland, OR' — used to give downstream stages structural context. anchors, 2 genuine coveragecoverageThe fraction of a population or region for which a data source has real, non-placeholder entries — e.g. 47% rooftop coverage on Texas addresses. Distinct from accuracy on the rows that are present.-gap probes

A deliberate deviation from the task's literal phrasing: the composition brief lists "bare build-local categories (fire hydrant, drinking fountain, datacenter) → requires_build_local_layer." Traced against poi-executor.ts's actual precedence, that reason is only reachable when a poi.db lookup IS configured and the anchor resolves to a center — a bare (anchor-less) category query hits the anchor_required early-return first whenever a lookup is wired, regardless of whether the category is build-local. Since this board always grades against a real poi.db, the three build-local abstain fixtures carry an anchor ("fire hydrant near Springfield IL"); the three anchor_required fixtures are the ones left bare ("coffee", "hospital", "bank"). Verified against both possible orderings live before fixing the fixtures — see eval-harness/poi-board.ts's header comment.

v1 numbers

POI query board (spec §3.6) — v1, REPORT-ONLY (no floors yet) — db: /mnt/playpen/mailwoman-data/poi/poi-full.db
45 cases, 93.3% overall pass rate

  expect kind     n     pass    rate
  abstain           7      7    100.0%
  address           6      6    100.0%
  results          32     29    90.6%

result rows returned: 496
  gersID non-null rate: 100.0%
  ancestry present rate: 100.0%

nearest-distance distribution (km, results-cases with ≥1 result, n=29): min 0.03  p50 0.58  p95 8.42  max 9.42

• abstain 7/7 (100%) — both abstain paths (requires_build_local_layer, anchor_required) fire exactly where expected, including the gated-synonym→build-local case (mailbox, en-CA).
• address 6/6 (100%) — no address-guard false-positive; the venuevenueA named, non-address place — a business, building, park, or stadium. Mailwoman's free-text point-of-interest component, added as a Tier 2 fine label.-led shape ("hospital, 350 5th Ave, New York, NY 10118") correctly stays on the address path in both the US and FR forms tested.
• results 29/32 (90.6%) — 3 failures, all genuine product behavior (below).
• gersID non-null 100%, ancestry present 100% — every one of the 496 returned result rows carries a GERS id and a read-time WOFWOF (Who's On First). An open-source gazetteer of places maintained by Mapzen/whosonfirst. Mailwoman builds a custom SQLite database from WOF GeoJSON repos, extended with postcode data, importance scores, and coincident-role relations. ancestry chain (the poiQueryKind register row's second debt payment, landed 2026-07-19). Both report-only per the schema's meaning-of-zero convention.
• nearest-distance distribution — p50 0.58 km, p95 8.42 km, all well inside the 25 km tolerance on passing cases. The board is not measuring at the edge of its own tolerance band; when a case fails it fails on zero results, not a near-miss on distance.

Notable failures (3/45, all left failing — not tuned away)

id	query	why
syn-05	grocery near Chicago IL	supermarket category returns 0 rows near Chicago AND near Austin (spot-checked separately) — a real poipoint of interest (POI). A named place that is not strictly an address — landmark, transit stop, venue, amenity, or franchise. Mailwoman tags these as venue and resolves them through the gazetteer..db coveragecoverageThe fraction of a population or region for which a data source has real, non-placeholder entries — e.g. 47% rooftop coverage on Texas addresses. Distinct from accuracy on the rows that are present. gap for that category, not an anchor-resolution miss
nm-06	supermarket near Guadalajara, Mexico	same supermarket coveragecoverageThe fraction of a population or region for which a data source has real, non-placeholder entries — e.g. 47% rooftop coverage on Texas addresses. Distinct from accuracy on the rows that are present. gap, MX side
nm-04	hiking trail near Marseille	trail category (Overture route=hiking) returns 0 rows near Marseille, Toulouse, and Denver (spot-checked) — Overture Places carries few if any point-shaped trail rows; a structural fit issue for a line-geometry featurefeatureAn input signal a model conditions on. Beyond the raw tokens, Mailwoman feeds soft features — gazetteer-membership channels and the postcode anchor — that inform predictions without overriding them., not a query-parsingaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates. bug

Both gaps are subject-match successes, execution misses — matchPOISubject correctly resolved grocery→supermarket and hiking trail→trail in every case; the poipoint of interest (POI). A named place that is not strictly an address — landmark, transit stop, venue, amenity, or franchise. Mailwoman tags these as venue and resolves them through the gazetteer..db search came back empty. That is exactly the signal this board exists to surface: the runner has no bug here, the underlying category coveragecoverageThe fraction of a population or region for which a data source has real, non-placeholder entries — e.g. 47% rooftop coverage on Texas addresses. Distinct from accuracy on the rows that are present. does.

One additional near-anomaly worth recording even though it didn't fail a fixture: anchor resolution is localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for.-sensitive in ways that aren't obvious from the query text alone. "cafe near Montreal QC" resolves the FTS backend to a same-named town in Wisconsin, not Montréal, Québec — dropping the QC/stateregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.-style suffix in favor of "cafe near Montreal, Canada" resolves correctly. "pharmacy near Calgary AB" abstains anchor_required outright (the parseaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates. doesn't attach a resolvable center to the regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.-suffixed form), while bare "bank near Calgary" resolves cleanly. The 22 category+anchor fixtures were phrased around this — not to inflate the pass rate, but because a fixture that fails on an already-known resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon. quirk adds no new signal over what's noted here. The quirk itself is real and worth a resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon.-side look (FTS bm25 tiering picking a same-name homonym over a QC/AB suffix hint), just not this board's job to fix.

Runner notes

• gradeCase (the grading core) is pure and unitunitA subdivision of a building — apartment, suite, floor — that refines a street address. Mailwoman's unit component; a designator plus identifier forms a subpremise.-tested against synthetic outcomes — no db, no classifier, no resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon. (eval-harness/poi-board.test.ts, 20 tests, all green). The live run above exercises the real createRuntimePipeline({ classifier, resolver, poiQueryKind: { poiDatabasePath } }) construction, mirroring commands/poi.tsx.
• Exit code is 0 regardless of case failures — non-zero is reserved for harness errors (missing fixtures file, pipelinestaged pipelineMailwoman's runtime architecture: a sequence of pure-function stages (normalize → query-shape → locale-gate → kind-classifier → phrase-grouper → classifier → decoder) connected by typed handoffs. Each stage is published as its own npm package. construction failure). Confirmed live: the 3-failure run above exits 0.
• --json prints the full machine-readable report (all 45 per-case grades + the aggregate metrics) instead of the human table; verified it round-trips through JSON.parse cleanly at 411 lines — the CLI's own poi --json mode has a known Ink line-wrapping trap on long string values piped through <Text>, which this command avoids by building the report in-process rather than shelling out to the compiled CLI and parsingaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates. its text output.

Floors set — 2026-07-20 (follow-up PR)

The pre-registered floors are now live in the harness (POI_BOARD_FLOORS / evaluateFloors in eval-harness/poi-board.ts), graded and printed on every run:

floor	threshold	rationale
overall	≥ 90%	soft — coveragecoverageThe fraction of a population or region for which a data source has real, non-placeholder entries — e.g. 47% rooftop coverage on Texas addresses. Distinct from accuracy on the rows that are present. gaps (trail/supermarket) may cost a few points without failing the board
abstain	= 100%	hard — an abstain miss is the poipoint of interest (POI). A named place that is not strictly an address — landmark, transit stop, venue, amenity, or franchise. Mailwoman tags these as venue and resolves them through the gazetteer. branch claiming a query poipoint of interest (POI). A named place that is not strictly an address — landmark, transit stop, venue, amenity, or franchise. Mailwoman tags these as venue and resolves them through the gazetteer..db structurally cannot answer
address-guard	= 100%	hard — the poipoint of interest (POI). A named place that is not strictly an address — landmark, transit stop, venue, amenity, or franchise. Mailwoman tags these as venue and resolves them through the gazetteer. branch must never hijack a full address

Floors print on every run; mailwoman eval poi-board --enforce turns a breach into a non-zero exit (the CI-gate mode). Without --enforce the command stays report-only.

Current standing (v1.1, 2026-07-20 promotion battery — 2026-07-20-poi-promotion-battery.md): the fixture set has grown from v1's 45 to 51 (v1.1 added 6 brand cases — see poi-board.test.ts). The board sits at 47/51 = 92.2% overall, with 33/37 results, abstain 8/8 (100%), and address-guard 6/6 (100%) — both hard floors met, and overall clears the 90% soft floor with margin. The results misses are the known coveragecoverageThe fraction of a population or region for which a data source has real, non-placeholder entries — e.g. 47% rooftop coverage on Texas addresses. Distinct from accuracy on the rows that are present.-gap holdouts already logged under "Notable failures" above (the supermarket and trail category gaps — syn-05, nm-06, nm-04), plus a brand case; all are results-kind, so they cost the overall rate but touch neither hard floor. All three floors pass at the current standing.

Breach detection is unitunitA subdivision of a building — apartment, suite, floor — that refines a street address. Mailwoman's unit component; a designator plus identifier forms a subpremise.-tested against synthetic result sets (poi-board.test.ts, evaluateFloors — breach detection) — no live board run needed to prove a floor fails: an overall dip below 90%, a single abstain miss, a single address-guard miss, and an absent category slice each register as a breach.

Next steps (not this PR)

• Set floors off these numbers once the operator reviews them (spec §3.6's own sequencing).
• The supermarket and trail category coveragecoverageThe fraction of a population or region for which a data source has real, non-placeholder entries — e.g. 47% rooftop coverage on Texas addresses. Distinct from accuracy on the rows that are present. gaps are candidates for a poipoint of interest (POI). A named place that is not strictly an address — landmark, transit stop, venue, amenity, or franchise. Mailwoman tags these as venue and resolves them through the gazetteer..db builder investigation — worth checking whether Overture's taxonomy.primary values for those two categories actually match what osmTag/category-id mapping in poi-taxonomy/data/taxonomy.json expects.
• The Montreal/Calgary anchor-resolution quirks are FTS-backend homonym-ranking behavior, independent of this board; worth a look under resolver-wof-sqlite's bm25 tiering, not logged as a poipoint of interest (POI). A named place that is not strictly an address — landmark, transit stop, venue, amenity, or franchise. Mailwoman tags these as venue and resolves them through the gazetteer.-board failure because no fixture encodes an expectation against it.

Standing update — 2026-07-20 (after #1206 / #1208 / #1209)

Three changes since the v1.1 promotion battery moved the board: #1206 fanned category queries out from the curated seed id to its Overture leaves, #1208 switched brand searches to an indexed brand-wide fetch, and #1209 replaced the seed taxonomy with the full ~2,110-category Overture snapshot under a curated overlay. Re-run against the rebuilt poi.db (poi/2026-07-20a/poi.db), the board now sits at 50/51 = 98.0% overall, with 36/37 results, abstain 8/8 (100%), and address-guard 6/6 (100%).

nm-04 (hiking trail near Marseille) is the sole holdout. It is a ring-budget case, not a coveragecoverageThe fraction of a population or region for which a data source has real, non-placeholder entries — e.g. 47% rooftop coverage on Texas addresses. Distinct from accuracy on the rows that are present. gap: the nearest hiking_trail row sits ~3.9 km from the Marseille anchor while the executor's k-ring reach lands at ~4 km, so the row falls just outside the searched cells. The supermarket gaps (syn-05, nm-06) and the trail gap on the other anchors that failed in v1.1 now return rows — the leaf fan-out and the fuller taxonomy reach the categories the seed mapping missed. Both hard floors hold, and overall clears the 90% soft floor with margin.