Skip to main content

Per-locale confidence calibration — en-us v4.0.0 (#368 L2)

A separate isotonic table per localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for., vs the single global table. ECEECE (Expected Calibration Error). A metric that measures how well a model's confidence scores align with its actual accuracy. Lower is better. Mailwoman's held-out ECE drops from 0.067 (raw) to 0.0035 (calibrated). is measured on each localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for.'s held-out split under three regimes: raw softmaxsoftmaxThe function that converts a vector of logits into a probability distribution summing to 1, applied after priors and biases are added to the emission logits., the global table, the localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for. table.

localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for.naccuracyECEECE (Expected Calibration Error). A metric that measures how well a model's confidence scores align with its actual accuracy. Lower is better. Mailwoman's held-out ECE drops from 0.067 (raw) to 0.0035 (calibrated). rawECEECE (Expected Calibration Error). A metric that measures how well a model's confidence scores align with its actual accuracy. Lower is better. Mailwoman's held-out ECE drops from 0.067 (raw) to 0.0035 (calibrated). global-tableECEECE (Expected Calibration Error). A metric that measures how well a model's confidence scores align with its actual accuracy. Lower is better. Mailwoman's held-out ECE drops from 0.067 (raw) to 0.0035 (calibrated). localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for.-table
NL2010.9850.17330.04660.0096
DE1970.8930.09320.08890.0411
US53150.9730.06740.00440.0037
FR7960.9660.06410.01380.0117

Where the localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for.-table column beats the global-table column, a single global table is leaving calibration error on the table for that localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for. (the OOD localeslocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for. especially). A multi-localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for. modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' should ship one calibration table per localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for., selected by the locale gatelocale gateStage 2 of the runtime pipeline: rule-based locale detection from the query shape's script and known-format signals. Returns a LocaleHint with the top candidate and alternatives, surfacing disagreement with an explicit --locale flag..