Choosing the dedup yardstick — why org-name, not NPI

"How good is the matcher at deduplicationrecord matchingThe process of determining whether two database records refer to the same real-world entity. Mailwoman's matcher uses a geocode-first approach (match the resolved place, not the address string) with Fellegi-Sunter probabilistic scoring.?" sounds like a question with one number for an answer. It isn't. The answer is "against which ruler?" — and on the NPPES provider registry, the ruler you reach for first is the one that lies to you.

The setup is the cleanest kind of held-out truth: the NPI (National Provider Identifier) is a stable, government-issued key, assigned once per registration. Run the matcher blind to the NPI, then grade its clusters against it. Two records that share an NPI should land together; a cluster fusing two NPIs is an over-merge; an NPI split across clusters is an under-merge. No labelscomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag., no adjudication — just count. Tempting, and on this registry, wrong by construction.

One organization, many NPIs

The NPI is a registration key, not an organization key. A single hospital system holds a separate NPI for every billing subpart — each clinic, each department, sometimes each location — plus legal-form variants, stateregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. qualifiers, and the occasional typo'd re-registration. Baylor College of Medicine appears four times at one address (1504 Taub Loop); WAL-MART STORES TEXAS LLC and … LP are the same Walmart pharmacy under two corporate skins. These are not distinct entities. They are one organization wearing many registration numbers.

So when the matcher does exactly the right thing — recognizing that four co-located Baylor records are one place — NPI-as-truth scores all six of those within-org pairs as over-merges. The matcher is penalized for being correct. The lower its measured precisionprecisionOf the spans the model labeled as a given tag, the fraction it got right. High precision means few false positives. Paired with recall to compute F1., the better it may actually be working.

Four rulers, one set of clusters

The fix is not a better modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.'; it's a better ruler. We grade the identical matcher output against four entity-truths, ordered from over-segmenting to tightest:

• NPI — one entity per registration. Over-segmentssegmentA punctuation-bounded chunk of the normalized input — the comma-separated parts of 'Portland, OR' — used to give downstream stages structural context. every multi-subpart org.
• site — collapse NPPES-flagged subparts back into their parent, then split an NPI's distinct addresses (mailing vs practice). Closer, but the subpart flag is conservative: it misses the 37% of same-org pairs that aren't flagged.
• org-name — the gold-set-validated truth: same NPI implies same entity (recallrecallOf the spans whose gold label is a given tag, the fraction the model found. High recall means few misses. Paired with precision to compute F1. preserved), plus collapse co-located NPIs whose primary organization names agree (tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. Jaccard ≥ 0.7), with no reliance on the subpart flag. Co-location here is the address string key — so it still misses same-building records whose text drifts.
• org-name (coord) — the same rule, but co-location is keyed on the geocoded building (within 50 m), not the address string. 1504 Taub Loop and 1504 Taub Lp Ste 100 key apart as strings yet geocode to one point, so this collapses the same-building same-org pairs the string key leaves split. It is the geocode-firstgeocode-firstThe matcher's core design principle: resolve addresses to geographic coordinates first, then compare the resolved places — not the raw address strings. Two records at the same coordinates match even if one says '123 Main St' and the other says '123 MAIN STREET.' key, and the tightest ruler of the four.

The same 2757-record sample carries 1000 NPI classes → 1456 site → 956 org-name → 928 org-name (coord). The class count first expands (site splits multi-address NPIs) then contracts below the NPI count (org-name fuses the co-located subparts, the coordinate fusing a few more) — the shape of over-segmentation being corrected.

Grade the shipped GBTGBT (Gradient Boosted Trees). A non-linear machine learning model that combines many weak decision trees into a strong predictor. Mailwoman uses a GBT as an optional learned scorer for single-dataset dedup, improving F1 by 5–7 percentage points over the Fellegi-Sunter baseline. scorer's clusters against each ruler and the F1 climbs 53.6% (NPI) → 55.3% (site) → 60.7% (org-name) → 68.1% (coordinate) — a +14.5pp swing with the clusters held identical. The over-merge count falls (109 → 76) and precisionprecisionOf the spans the model labeled as a given tag, the fraction it got right. High precision means few false positives. Paired with recall to compute F1. rises (43.7% → 64.6%) for the same reason: the ruler stops counting correct same-org merges as errors, and the geocode-firstgeocode-firstThe matcher's core design principle: resolve addresses to geographic coordinates first, then compare the resolved places — not the raw address strings. Two records at the same coordinates match even if one says '123 Main St' and the other says '123 MAIN STREET.' building key catches the most of them. The climb is the yardstick getting more precise, not the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' changing.

Why we trust the org-name ruler

A ruler you invent to make your number go up is worthless unless it's right. So the org-name collapse was validated against a frozen gold set: the hardest slice — co-located, name-similar, distinct NPIs that are not flagged subparts — adjudicated one pair at a time. The verdict was unanimous: 120 of 120 were the same real-world organization; zero were two unrelated companies wrongly fused. Every hard pair the matcher merges at a shared address is a correct merge that NPI-truth was scoring as an error.

That result also settled the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' question. An earlier experiment added "corroboration" featuresfeatureAn input signal a model conditions on. Beyond the raw tokens, Mailwoman feeds soft features — gazetteer-membership channels and the postcode anchor — that inform predictions without overriding them. aimed at cutting over-merges — and moved precisionprecisionOf the spans the model labeled as a given tag, the fraction it got right. High precision means few false positives. Paired with recall to compute F1. not at all. The gold set explains why: there was essentially no over-merge to cut. Optimizing the scorer for precisionprecisionOf the spans the model labeled as a given tag, the fraction it got right. High precision means few false positives. Paired with recall to compute F1. here optimizes a phantom; the lever was the ruler all along.

The discipline

Report the number that matches the entity you're resolving. For NPPES organization dedup that is the org-name F1 — ~60.7% by the address-string key, ~68.1% by the geocode-firstgeocode-firstThe matcher's core design principle: resolve addresses to geographic coordinates first, then compare the resolved places — not the raw address strings. Two records at the same coordinates match even if one says '123 Main St' and the other says '123 MAIN STREET.' building key — not the NPI-level 53.6%, which mostly measures NPI over-segmentation, a property of the registry, not of the matcher. StateregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. the ruler next to the number, every time; a dedup F1 with no named truth-grain is unreadable.

This is the same move the rest of the system makes: grade the pipeline against the right target, not a convenient proxy. The remaining org-name over-merge (76 clusters at the coordinate grain) is the frontier — small, and approaching the irreducible ceiling of co-located distinct organizations that no address-based signal can separate.

A closing note on framing: NPI-as-truth is conservative in the direction that matters. A cluster fusing two NPIs is a candidate "same entity, two registrations" surfaced for review, not an adjudicated error; an NPI split across distant addresses is geocode-firstgeocode-firstThe matcher's core design principle: resolve addresses to geographic coordinates first, then compare the resolved places — not the raw address strings. Two records at the same coordinates match even if one says '123 Main St' and the other says '123 MAIN STREET.' behaving correctly. We resolve and report the join; what it means is the consumer's call.

Measurements: 2026-06-16-dedup-dual-level-benchmark (the four-grain table), 2026-06-16-dedup-gold-set-tx120 (the adjudicated slice), 2026-06-16-dedup-ceiling (the irreducible floor). Figure generator: scripts/record-matcher/viz/yardstick-figure.ts.