Coarse-placer (#244) — milestone 1: a calibrated closed-set placer + the OOD wall

2026-06-14. The #244 coarse-placercoarse-placerA lightweight int8 country classifier (~0.79 MB) that predicts which of a set of target countries an address belongs to, feeding a soft prior into resolver disambiguation. is the tiny always-resident modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' that runs first, places an address coarsely, and abstains ("probably off my loaded map") rather than emit a confident mis-parseaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates. — the foundation of the selective-geography / tiered-loading story. Milestone 1 builds the data pipelinestaged pipelineMailwoman's runtime architecture: a sequence of pure-function stages (normalize → query-shape → locale-gate → kind-classifier → phrase-grouper → classifier → decoder) connected by typed handoffs. Each stage is published as its own npm package. and a pure-TS char-n-gramn-gramA contiguous run of n tokens (a bigram is 2, a trigram is 3). Counting n-grams is a simple, pre-neural way to model which token sequences are common. linear classifier, trained on CPU in minutes. It nails the closed-set task and makes the case for milestone 2 (outlier exposure) concrete.

What it is

A fastText-style linear classifier over hashed char 3/4/5-grams + Unicode-script presence tokenstokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. (core/coarse-placer/featurize.ts), mapping an address string → one of 11 well-represented countries with a temperature-calibrated confidence (core/coarse-placer/coarse-placer.ts). Pure, dependency-free, browser-safe; the artifact is 2.9 MB fp32fp32 / fp1632-bit and 16-bit floating-point formats. Mailwoman trains in bf16 (a 16-bit variant) and exports the ONNX model in int8 for size. today (quantizes to ~720 KB int8 — a milestone-3 concern).

• Data (scripts/coarse-placer/build-dataset.mjs): stratified sample from the v0.5.0 corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus. — 40k/5k/5k train/val/test per countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head., balanced (a flat sample is 94% US+FR). Classes: US, FR, GB, CN, NL, IT, DE, JP, ES, KR, TW (the corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus.'s well-represented set; the long tail is held for outlier exposure). Two corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus. gotchas handled: DuckDB USING SAMPLE samples the table-then-filters (so we filter-then-sample), and the val/test shardsshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. only carry US/FR/DE (so all splits are drawn from train with our own per-countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. 80/10/10).
• Train (scripts/coarse-placer/train.mjs): multinomial logistic regression, plain SGDgradient descentThe optimization method behind training: repeatedly compute the gradient on a batch and step the parameters a small amount in the downhill direction. 'Stochastic' gradient descent uses one minibatch at a time., ~minutes on CPU. Temperature fit on val by NLLNLL (Negative Log Likelihood). The standard form of a loss function in probability-based models: loss = −log P(correct label | input). The CRF uses the NLL of the entire label sequence. minimization.

Milestone 1 results

In-distribution (test, n=55k):

metric	value
closed-set accuracy (argmax)	96.63%
accuracy @ abstain-below-0.5	95.61%
ECEECE (Expected Calibration Error). A metric that measures how well a model's confidence scores align with its actual accuracy. Lower is better. Mailwoman's held-out ECE drops from 0.067 (raw) to 0.0035 (calibrated). (10-bucket)	0.0548
temperature	1.0 (already well-calibrated)

Per-class recallrecallOf the spans whose gold label is a given tag, the fraction the model found. High recall means few misses. Paired with precision to compute F1.: US 99.4, FR 98.8, GB 98.8, NL 98.3, KR 95.7, JP 95.1, DE 94.3, CN 94.1, TW 93.5, IT 92.8, ES 90.9. The errors are the expected ones — ES↔IT↔DE (European Latin overlap) and TW↔CN (shared Han). Trains in ~2 epochs (val plateaus at 96.7%).

The wall (and why milestone 2 is the point)

The threshold-only abstention does not solve out-of-distribution. Off-map scripts — Cyrillic, Armenian, Greek, none of them among the 11 trained countries — are confidently mis-classified, not abstained:

cyrillic/RU → DE @0.71     «Новосибирск, ул. Ленина 10»
armenian/AM → ES @0.84     «Երևան, Աբովյան փողոց 5»
greek/GR    → GB @0.88     «Αθήνα, οδός Ερμού 12»

Only 36% of off-map-script rows abstain. This is the classic softmaxsoftmaxThe function that converts a vector of logits into a probability distribution summing to 1, applied after priors and biases are added to the emission logits.-overconfidence-on-OOD pathology: a closed-set modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' has no "none of the above," so it spreads an off-map input's weak signal over the 11 classes and one wins with moderate-to-high confidence. The design anticipated exactly this — the fix is outlier exposure: an explicit "other" class trained on off-map examples so the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' learns the edge of its own competence. That's milestone 2.

Milestone 2 — outlier exposure → an explicit OTHER class (the wall, cleared)

The data unlock: the WOFWOF (Who's On First). An open-source gazetteer of places maintained by Mapzen/whosonfirst. Mailwoman builds a custom SQLite database from WOF GeoJSON repos, extended with postcode data, importance scores, and coincident-role relations. names table carries native-script alternate names in dozens of languages (rus/ukr/ara/ell/heb/hin/tha/kat/hye/…) — exactly the off-map scripts the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' needs to learn to abstain on. scripts/coarse-placer/build-outlier-exposure.mjs extracts ~44k of them, balanced per-language and filtered to a off-map dominant script, PLUS an address-shaped sibling for each (name + a house numberhouse numberThe numeric or alphanumeric identifier of a building on a street. Mailwoman's house_number component; its position relative to the street name flips between locales.) — because real off-map input mixes the script with Latin digits/abbreviations ("ул. Тверская, д. 1"), and the bare place nametoponymA proper name for a geographic place. alone left a gap. Added as a 12th class, OTHER.

metric	M1 (closed-set)	M2 (+ OTHER)
off-map-script handling (route to OTHER / abstain)	36%	86%
OTHER recallrecallOf the spans whose gold label is a given tag, the fraction the model found. High recall means few misses. Paired with precision to compute F1. (test)	—	92.2%
in-distribution accuracy	96.6%	95.0%
ECEECE (Expected Calibration Error). A metric that measures how well a model's confidence scores align with its actual accuracy. Lower is better. Mailwoman's held-out ECE drops from 0.067 (raw) to 0.0035 (calibrated).	0.055	0.050

The explicit OTHER class lifts off-map handling 36% → 86% at a ~1.6pp in-distribution cost — the design's prediction, confirmed. The address-shaped augmentation was decisive (place namestoponymA proper name for a geographic place. alone got only 59%): the numeric/punctuation n-gramsn-gramA contiguous run of n tokens (a bigram is 2, a trigram is 3). Counting n-grams is a simple, pre-neural way to model which token sequences are common. in a real address otherwise pull an off-map input toward a countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head..

Next (milestone 3+)

1. The Latin-off-map residual. Off-map COUNTRIES in Latin script (Poland, Turkey, Brazil…) still mis-place — they share the script with the in-map European 11 and the OTHER trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input. is non-Latin. The fix is full off-map addresses (OpenAddressesOpenAddresses (OA). A global open aggregation of address points collected from many official sources. A primary source of component-supervised training data outside proprietary registries. for more countries), not place namestoponymA proper name for a geographic place..
2. Script/continent headsattention headOne of several parallel attention computations in a layer, each free to focus on a different kind of relationship between tokens. Their outputs are concatenated — 'multi-head attention'. Mailwoman uses 4 heads. — the design wants (script, continent, coarse-regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.); script is deterministic, continent is a grouping, coarse-regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. routes shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. loading.
3. Shrink + ship — int8-quantize (≈720 KB), wire as the first pipelinestaged pipelineMailwoman's runtime architecture: a sequence of pure-function stages (normalize → query-shape → locale-gate → kind-classifier → phrase-grouper → classifier → decoder) connected by typed handoffs. Each stage is published as its own npm package. stagestageOne of the dataflow stages in the runtime pipeline (normalize, locale gate, kind classify, phrase group, token classify, sequence correct, reconcile, resolve). Distinct from tier (model vocabulary) and phase (plan milestone). + the #243 query-type router (bare-landmark → skip parser; address → parseaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates.).

Reproduce: node scripts/coarse-placer/build-dataset.mjs && yarn compile && node scripts/coarse-placer/train.mjs && node scripts/coarse-placer/eval.mjs.