The better posterior was confidently wrong
Date: 2026-06-05 Scope: the postcode anchoranchor inferenceA technique where structured knowledge (postcode locations, gazetteer place names) is injected into the model as soft input features — not as deterministic overrides. The model still decides the final labels, but the anchor signal biases it toward correct admin tags.'s country posteriorcountry posteriorA country → probability map (derived from postcodes or the coarse-placer) that re-ranks resolver candidates as a soft prior, never a hard filter. (#240) — uniform vs a de-biased Bayesian weighting, decided on held-out real collisions before changing a line of shipped code.
The postcode anchoranchor inferenceA technique where structured knowledge (postcode locations, gazetteer place names) is injected into the model as soft input features — not as deterministic overrides. The model still decides the final labels, but the anchor signal biases it toward correct admin tags. turns a code like 75001 into a distribution over the countries it could belong to. The shipped version uses a uniform posterior: 1/k over the countries a code exists in. 75001 is in both France and the US, so it comes back {FR: 0.5, US: 0.5} — an honest shrug that leaves the disambiguation to the parser and the resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon..
That shrug throws away real information, and we knew it. The true posterior is P(country | postcode) ∝ N_c(x) — the actual address-count ratio. 75001 is the 1st arrondissement of Paris, dense and central; it is also Addison, Texas, a quiet suburb. Those don't have equal address counts, so the honest answer isn't 50/50. A de-biased Bayesian posterior — within-countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. frequency times a real-world address-volume prior — targets that ratio directly, and on paper it's the more correct shape. So we went to adopt it. Then we did the thing our earlier postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one.-countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. investigation was entirely about: we measured the make-or-break assumption before building on it.
The measurement
We A/B'd three posteriors on the canonical collision — US ↔ FR, the literal 75001 case, and the one pair where we have rich frequency data on both sides:
f̂(train): corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus. v0.1.0 — 4.4M real addresses — counting(country, postcode).- candidate set: the 13,186 five-digit postcodespostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. that exist in both the US and FR postal gazetteersgazetteerA geographical index that maps place names and postcodes to real-world coordinates. Mailwoman uses a custom-built Who's On First (WOF) SQLite database as its gazetteer — the 'atlas' half of the grammar/atlas architecture..
- test (held-out):
openaddresses-{us,fr}-sample.jsonl— a different extraction, sof̂is never graded on the data it was fit to.
Three posteriors, scored per true countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. and balanced (so a US-leaning prior can't win by exploiting that US collisions outnumber FR in the test):
| posterior | balanced logloss | balanced top-1 | high-conf errors (true-FR) |
|---|---|---|---|
| uniform (shipped) | 0.6931 | 50.0% | 0.0% |
| naive-count | 0.7555 | 39.4% | 0.0% |
| de-biased (the "smart" one) | 2.2292 | 51.6% | 96.9% |
De-biased is three times worse-calibrated than the shrug, and confidently wrong about the minority countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. 97% of the time. Its one bragging point — +1.6pp top-1 — is pure test-imbalance arbitrage: it nails the abundant US cases and immolates the FR ones. naive-count, the control, comes in worse than uniform too, exactly as the raw-count skeptics warned.
Why — and it isn't the math
The de-biasing math is sound. The data underneath it is half-broken. f̂ needs per-countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. address frequency on both sides of a collision, and we only have it on one. France arrives as 1.13M real streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. addresses (ban, the national address base), so a busy Paris postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. shows up thousands of times. The US arrives as 58k rows from wof-postalcode — essentially a membership list, one row per code, no frequency signal at all (97.5% of v0.1.0's US rows are wof-admin places that carry no postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one.). So f̂_US is estimated off a tiny, flat denominator and comes out systematically inflated; multiply by a US-favoring volume prior and the posterior screams "US" at nearly every collision, right or wrong.
Feed sound math a lopsided f̂ and you don't get a slightly-off posterior — you get a confidently wrong one. And a confidently-wrong soft priorsoft priorOutside knowledge fed to the model or resolver as overridable evidence (a feature, a score term) rather than a hard filter. A focus-country hint becomes an anchor feature; a focus-point becomes a ranking term. is the single thing that corrupts the resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon. re-rank we're building toward: uniform's honest 0.5 gets gated out when the resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon. is already sure, but a 0.97-to-the-wrong-countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. prior sails through the confidence gate and flips a correct answer. The shrug is worse on paper and far safer in practice.
Verdict
Uniform stays. Not because de-biasing is the wrong idea, but because we can't feed it. This is a data-availability finding wearing a posterior-design costume: the day we have balanced per-countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. address frequency — bulk OpenAddressesOpenAddresses (OA). A global open aggregation of address points collected from many official sources. A primary source of component-supervised training data outside proprietary registries. for the US, or census ZIP-population weightsparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values., neither on disk today — it's worth re-measuring, because the shape it targets really is the correct one. Until then, weighting the guess makes it worse.
The measurement is scripts/eval/postcode-posterior-ab.py; re-run it when the data changes. It cost an afternoon and saved us from shipping a posterior that is wrong with conviction — which is the only kind of wrong a soft anchoranchor inferenceA technique where structured knowledge (postcode locations, gazetteer place names) is injected into the model as soft input features — not as deterministic overrides. The model still decides the final labels, but the anchor signal biases it toward correct admin tags. can't afford to be.