Skip to main content

Coarse-placer M3: the Latin off-map residual is a data wall, not a method one

2026-06-14. The #244 coarse-placercoarse-placerA lightweight int8 country classifier (~0.79 MB) that predicts which of a set of target countries an address belongs to, feeding a soft prior into resolver disambiguation.'s M2 OTHER class was trained on non-Latin scripts (Cyrillic, Arabic, …), so it abstains well on those but still confidently mis-places off-map COUNTRIES written in Latin script — Poland, Brazil, Mexico — onto a trained Latin countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.. This milestone tested the obvious fix: feed it REAL off-map addresses (not synthetic name variants — see #564) as OTHER. The mechanism works cleanly and at zero in-map cost, but it does not generalize past the countries you train on, and Overture's addresses theme doesn't currently have the breadth to train on enough of them. So this is a directional win with an honest ceiling, recorded — not a promotion.

The residual, measured​

A Latin off-map address is handled when the placer routes it to OTHER or abstains; anything else is a confident mis-placement onto a wrong trained countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.. On a fresh held-out set of real Overture addresses from off-map Latin countries (n=17815), the shipped (M2) modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' handles only 23.3% — i.e. it confidently mis-places three out of four. Polish addresses go to JP/NL/US at 0.58–0.96 confidence.

The experiment​

Assemble real address strings from the Overture per-countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. address parquetParquetThe open columnar file format the corpus is written and streamed in. The training pipeline reads shards row-by-row from Parquet. and append them as country: "OTHER". Split the countries deliberately:

  • Train (their rows feed train/val OTHER): PL, BR, MX, PT — the off-map Latin countries Overture actually has rows for.
  • Held out (test only — the generalizationgeneralizationA trained model's performance on data unlike its training set — new regions, new input distributions. The property honest eval is designed to measure. probe): CZ (a distinct Slavic countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' never sees), plus CA and LI (the hard near-twins — Canadian addresses read like US, Liechtenstein like DE).

Retrain (34s CPU, same SGDgradient descentThe optimization method behind training: repeatedly compute the gradient on a batch and step the parameters a small amount in the downhill direction. 'Stochastic' gradient descent uses one minibatch at a time. recipe), evaluate against the shipped modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.'.

Results​

groupshipped (M2)M3 retrain
in-distribution (held-out PL/BR/MX/PT rows)23.1%100.0%
held-out (CZ / CA / LI, never trained)23.3%25.0%
overall23.3%31.4%

Per held-out countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.: CZ 39.5 → 42.8%, CA 7.0 → 7.4%, LI 19.5 → 20.6%.

In-map accuracy held: golden test 94.99 → 95.10% (+0.1pp), every in-map class flat or up (ES 90.4 → 91.1%), non-Latin multi-script handling maintained. No regression anywhere — M3 is a strict Pareto improvement.

What this says​

The mechanism is real: train a countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. on OTHER and it goes to 100% handled, at zero in-map cost. But the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' learns those countries' n-gramsn-gramA contiguous run of n tokens (a bigram is 2, a trigram is 3). Counting n-grams is a simple, pre-neural way to model which token sequences are common. → OTHER, not a general "off my map" concept — the held-out countries barely move (+1.7pp overall), and the near-twins (CA looks like US, and for a coarse placer that's arguably not even wrong) stay where they are. General Latin off-map handling needs broad countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. coveragecoverageThe fraction of a population or region for which a data source has real, non-placeholder entries — e.g. 47% rooftop coverage on Texas addresses. Distinct from accuracy on the rows that are present. — dozens of off-map countries in the OTHER class, not four.

That breadth is the wall. Of the 12 off-map countries requested from Overture's addresses theme (2026-05-20.0, ALPHA), only 5 returned rows (PL/BR/MX/PT/CZ); RO/TR/ID/SE/VN/HU/PH/AR were empty. So this is a data-availability ceiling, not a method failure — the same shape as #564's fr.house_number plateau (real-data realism is the lever; we just don't have enough of it yet).

Decision​

  • Do not promote. model-m3 is a strict improvement but does not meet the ≥90% general target; the coarse-placercoarse-placerA lightweight int8 country classifier (~0.79 MB) that predicts which of a set of target countries an address belongs to, feeding a soft prior into resolver disambiguation. isn't bundled anywhere yet, so there's nothing to gate — this is a recorded finding, and the canonical fp32fp32 / fp1632-bit and 16-bit floating-point formats. Mailwoman trains in bf16 (a 16-bit variant) and exports the ONNX model in int8 for size. modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' stands.
  • The next lever is breadth, not weightparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. or recipe. Broad off-map address coveragecoverageThe fraction of a population or region for which a data source has real, non-placeholder entries — e.g. 47% rooftop coverage on Texas addresses. Distinct from accuracy on the rows that are present. — a full OpenAddressesOpenAddresses (OA). A global open aggregation of address points collected from many official sources. A primary source of component-supervised training data outside proprietary registries. off-map pull, or a later Overture release once the addresses theme fills in — folded into OTHER, with the held-out-countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. probe as the gate. Tracked as a follow-up to #244.

Reproduce​

node scripts/ingest-overture-addresses.ts --release 2026-05-20.0 \
  --countries PL,BR,MX,PT,CZ,CA,LI --limit 6000
node scripts/coarse-placer/build-outlier-latin.mjs            # appends OTHER + writes the test file
node scripts/coarse-placer/eval-latin-offmap.mjs --model .../coarse-placer/model        # baseline
node scripts/coarse-placer/train.mjs --out .../coarse-placer/model-m3
node scripts/coarse-placer/eval-latin-offmap.mjs --model .../coarse-placer/model-m3     # after