Learned-scorer CROSS-STATE generalization (#603 Tier 2) — train TX, evaluate CA
Generated by scripts/record-matcher/learned-scorer-crossstate-eval.ts. The GBTGBT (Gradient Boosted Trees). A non-linear machine learning model that combines many weak decision trees into a strong predictor. Mailwoman uses a GBT as an optional learned scorer for single-dataset dedup, improving F1 by 5–7 percentage points over the Fellegi-Sunter baseline. + LR are trained on 2000 TX NPIs (5641 records) and used to cluster 2000 held-out CA NPIs (5574 records) — a stateregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' never saw — through the same resolveEntities pipelinestaged pipelineMailwoman's runtime architecture: a sequence of pure-function stages (normalize → query-shape → locale-gate → kind-classifier → phrase-grouper → classifier → decoder) connected by typed handoffs. Each stage is published as its own npm package. (FS baseline / GBTGBT (Gradient Boosted Trees). A non-linear machine learning model that combines many weak decision trees into a strong predictor. Mailwoman uses a GBT as an optional learned scorer for single-dataset dedup, improving F1 by 5–7 percentage points over the Fellegi-Sunter baseline. scorer / LR scorer), best F1 over a fine per-scorer threshold sweep. This is the generalizationgeneralizationA trained model's performance on data unlike its training set — new regions, new input distributions. The property honest eval is designed to measure. axis the within-stateregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. held-out-NPI A/B couldn't cover.
Result — CA clustering F1 (GBT/LR trained on TX)
| scorer | precisionprecisionOf the spans the model labeled as a given tag, the fraction it got right. High precision means few false positives. Paired with recall to compute F1. | recallrecallOf the spans whose gold label is a given tag, the fraction the model found. High recall means few misses. Paired with precision to compute F1. | F1 | ΔF1 vs FS | over-merged |
|---|---|---|---|---|---|
| FS baseline (EMexpectation-maximizationAn iterative algorithm that estimates model parameters when some variables are unobserved. In Mailwoman's matcher, EM learns the Fellegi-Sunter m and u parameters from unlabeled data — no training labels needed.-fit) | 10.4% | 26.4% | 14.9% | — | 240 |
| logistic regression | 7.8% | 71.1% | 14.1% | -0.8pp | 276 |
| GBTGBT (Gradient Boosted Trees). A non-linear machine learning model that combines many weak decision trees into a strong predictor. Mailwoman uses a GBT as an optional learned scorer for single-dataset dedup, improving F1 by 5–7 percentage points over the Fellegi-Sunter baseline. (fresh TX fit) | 58.8% | 25.4% | 35.5% | +20.5pp | 47 |
| SHIPPED bundled modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' (default-on candidate) | 71.9% | 25.8% | 38.0% | +23.1pp | 46 |
The bundled row is the actual shipped DEDUP_GBT_MODEL (the default-on candidate), evaluated on CA — a stateregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. it never trained on. The "fresh TX fit" row retrains per run for comparison.
The GBTGBT (Gradient Boosted Trees). A non-linear machine learning model that combines many weak decision trees into a strong predictor. Mailwoman uses a GBT as an optional learned scorer for single-dataset dedup, improving F1 by 5–7 percentage points over the Fellegi-Sunter baseline. win GENERALIZES across statesregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. — trained on TX, it still beats the FS baseline on CA clusteringclusteringThe final stage of entity resolution: resolve non-transitive pairwise match decisions (A↔B, B↔C, but not A↔C) into canonical entities via union-find with path compression. Each cluster of records becomes one resolved entity. F1 (35.5% vs 14.9%, +20.5pp). The learned scorer isn't fitting TX-specific structure; the over-merge signal it learns transfers. This is the strongest evidence yet for the #603 production GBMGBT (Gradient Boosted Trees). A non-linear machine learning model that combines many weak decision trees into a strong predictor. Mailwoman uses a GBT as an optional learned scorer for single-dataset dedup, improving F1 by 5–7 percentage points over the Fellegi-Sunter baseline. — one modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.', trained once, helps a stateregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. it never saw.
Caveats
A single train/evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. stateregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. pair (TX→CA), one geocoded sample each, a compact pure-Node GBTGBT (Gradient Boosted Trees). A non-linear machine learning model that combines many weak decision trees into a strong predictor. Mailwoman uses a GBT as an optional learned scorer for single-dataset dedup, improving F1 by 5–7 percentage points over the Fellegi-Sunter baseline. (120 rounds, depth 3). The FS arm is the benchmark baseline (same modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.'), so the comparison is fair. Absolute F1 differs from the within-stateregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. A/B because the evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. population + over-merge density differ by stateregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.. NPI-as-truth is conservative. The within-stateregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. held-out-NPI A/B (learned-scorer-clustering-eval.ts) is the companion; together they bound the generalizationgeneralizationA trained model's performance on data unlike its training set — new regions, new input distributions. The property honest eval is designed to measure. question a production GBMGBT (Gradient Boosted Trees). A non-linear machine learning model that combines many weak decision trees into a strong predictor. Mailwoman uses a GBT as an optional learned scorer for single-dataset dedup, improving F1 by 5–7 percentage points over the Fellegi-Sunter baseline. must answer._