Failure taxonomy — where Mailwoman is wrong, and by how much (#375)
A standing map of the parser + geocoder's failure modes, each grounded in a measured number from a named evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. — not anecdotes, not vibes. The point is to make "where do we lose?" a table you can sort by impact, so the roadmap is driven by measured gaps instead of the last thing someone noticed. Every row cites the evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. it came from; re-run that evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. to refresh the number.
This is v1 (2026-06-17), assembled from the evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus. as it stands. It is meant to be living — when an evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. lands, update the row. Numbers dated 2026-06-17 are from this week's runs; older numbers carry their evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error.'s date so staleness is visible.
How to read a row
status is the verdict of the source evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error., not a wish:
- fixed — a shipped change moved the number and an evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. confirms it.
- open — measured, unfixed, no committed fix.
- deferred — measured, a fix exists or was tried, and we consciously did not ship it (with a reason).
- rejected — a fix was built and the evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. said it didn't earn its place.
The most important column is lever / root cause: it's what turns a number into a next step.
1. Input-shape failures
| class | measured | engine | status | lever / root cause | source |
|---|---|---|---|---|---|
| all-caps input | localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. 90.1% → 99.7% fixed (1172 TX facilities) | neural | fixed | OOD case (PALESTINE→ALESTINE); title-case detected all-caps before the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' (#690, default-OFF) | 2026-06-17-geocoder-vs-provided-coords |
intersectionintersectionAn address that names a location by two crossing streets ('5th & Main') rather than a number and street. Mailwoman tags the two streets as intersection_a and intersection_b — a negative-space format that starved the early model. (A & B) | 82% correct structure (templated) | neural | open | ~1-in-6 trip; thin corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus. coveragecoverageThe fraction of a population or region for which a data source has real, non-placeholder entries — e.g. 47% rooftop coverage on Texas addresses. Distinct from accuracy on the rows that are present. of the bare form | 2026-06-17-per-type-headtohead |
| quoted venuevenueA named, non-address place — a business, building, park, or stadium. Mailwoman's free-text point-of-interest component, added as a Tier 2 fine label. names | neural 74% vs v0 75% | both | open | neural over-extends the spanspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree.; v0's fieldsFuncBoundary treats a quote as a wall | 2026-06-14-punctuation-stress |
dotted abbreviations (St., 123 1/2) | v0 82% vs neural 74% | neural | open | neural absorbs the trailing tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words.; v0 rules cut precisely | 2026-06-14-punctuation-stress |
The all-caps row is the week's clean win, but note its scope: it's fixed on the resolveTree path (default-OFF opt). Wiring it into the record-matcher's geocoder is deferred pending an aggregate artifact (#694) — see §4.
2. Locale / script failures
| class | measured | engine | status | lever / root cause | source |
|---|---|---|---|---|---|
| fr.house_number (postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one.-first reorder) | 87.4% vs a 91% pre-registered floor | neural | deferred | positional shortcut learned on canonical data; the floor was mis-calibrated for the reordered case (SOTA ~90–91); weightparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. falsified — needs real data, not another weightparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. | 2026-06-13-fr-house-number-threshold-research |
| fr.regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. (golden dev) | 16.2% → 25.6% (v4.4.0) | neural | open | unfloored; FR has no real reordered data to train on | parity-scorecard-2026-06-11 |
| non-Latin / thin-coveragecoverageThe fraction of a population or region for which a data source has real, non-placeholder entries — e.g. 47% rooftop coverage on Texas addresses. Distinct from accuracy on the rows that are present. localeslocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for. | in-map right-countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. ~26–35% (NL, KR) | neural | open | en-US-centric trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input.; no localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for.-native evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. set for non-Latin script | 2026-06-14-coarse-placercoarse-placerA lightweight int8 country classifier (~0.79 MB) that predicts which of a set of target countries an address belongs to, feeding a soft prior into resolver disambiguation.-arc-postmortem |
LocalelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for. is the deepest open frontier and the one least amenable to a code lever: the recurring finding (fr.house_number) is that weightparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. tuning is exhausted — the next move is real reordered/native data, not another lossloss functionA number measuring how wrong the model's predictions are on a batch of examples. Training minimizes it. Mailwoman's loss combines per-token negative log-likelihood with the CRF sequence loss.-mask or weightparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. bump.
3. Format failures (po_box, intersection, unit, delimiters)
| class | measured | engine | status | lever / root cause | source |
|---|---|---|---|---|---|
| po_boxPO boxA numbered mailbox at a post office used as a delivery address instead of a physical street location. Mailwoman tags it as the po_box component; structurally the same family as a subpremise. (dotted leader) | 60% fail → 87% (spanspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree. bridge) → 89% (separator exclusion) | neural | fixed | tokenizertokenizerThe component that converts a raw address string into a sequence of numeric token IDs the model can process. Mailwoman's tokenizer is a SentencePiece unigram model trained specifically on postal addresses. dropped standalone punctuation; corrected at decode | 2026-06-11-v4.4.0-ship-gate |
| intersectionintersectionAn address that names a location by two crossing streets ('5th & Main') rather than a number and street. Mailwoman tags the two streets as intersection_a and intersection_b — a negative-space format that starved the early model. (real TIGERTIGERThe US Census Topologically Integrated Geographic Encoding and Referencing database. Used as a corpus source for street-segment data. shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row.) | 100% (real-OOD) vs 82% templated | neural | fixed | a real shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. beats synthetic templates | 2026-06-11-v4.4.0-ship-gate |
| po_boxPO boxA numbered mailbox at a post office used as a delivery address instead of a physical street location. Mailwoman tags it as the po_box component; structurally the same family as a subpremise. / unitunitA subdivision of a building — apartment, suite, floor — that refines a street address. Mailwoman's unit component; a designator plus identifier forms a subpremise. vs v0 | neural 100% / 100%; v0 0% / 0% | v0 | fixed-neural | v0 has no po_box/unit tag — the negative-space win | 2026-06-17-per-type-headtohead |
| cedexCEDEX (Courrier d'Entreprise à Distribution Exceptionnelle). A French postal routing for high-volume business mail: a CEDEX code delivers directly from a sorting centre, bypassing the local post office. A common negative-space format Mailwoman must parse. (FR real) | 96% (v4.4.0) | neural | fixed | deterministic regex path moved into the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' | 2026-06-11-v4.4.0-ship-gate |
| paired-delimiter spanspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree. proposer | −3.9pp vs 77% baseline | neural | rejected | the StagestageOne of the dataflow stages in the runtime pipeline (normalize, locale gate, kind classify, phrase group, token classify, sequence correct, reconcile, resolve). Distinct from tier (model vocabulary) and phase (plan milestone). 2.7 proposer's annotation bias has the wrong sign (merges where it should strip) — did not earn revival | 2026-06-14-punctuation-stress |
This is Mailwoman's strongest quadrant: the structured types (po_boxPO boxA numbered mailbox at a post office used as a delivery address instead of a physical street location. Mailwoman tags it as the po_box component; structurally the same family as a subpremise., unitunitA subdivision of a building — apartment, suite, floor — that refines a street address. Mailwoman's unit component; a designator plus identifier forms a subpremise., intersectionintersectionAn address that names a location by two crossing streets ('5th & Main') rather than a number and street. Mailwoman tags the two streets as intersection_a and intersection_b — a negative-space format that starved the early model., cedexCEDEX (Courrier d'Entreprise à Distribution Exceptionnelle). A French postal routing for high-volume business mail: a CEDEX code delivers directly from a sorting centre, bypassing the local post office. A common negative-space format Mailwoman must parse.) are either fixed or a rout against the rules enginerule-based classifierMailwoman's legacy v0 parser — a library of deterministic token classifiers (house number, street suffix, postcode, place name, etc.) composed by priority. Now primarily used for corpus labeling, fallback classification, and arbitration diagnostics.. The one rejected fix (paired-delimiter proposer) is a useful negative result — the evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. gate stopped a plausible-but-wrong revival.
4. Geocoder coverage / accuracy
| class | measured | engine | status | lever / root cause | source |
|---|---|---|---|---|---|
| admin-centroid fallback | ~40% of TX facilities fall back (p50 3.4 km, p99 catastrophic) | resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon. | open | coveragecoverageThe fraction of a population or region for which a data source has real, non-placeholder entries — e.g. 47% rooftop coverage on Texas addresses. Distinct from accuracy on the rows that are present., not precisionprecisionOf the spans the model labeled as a given tag, the fraction it got right. High precision means few false positives. Paired with recall to compute F1. — no rooftoprooftopGeocoding precision at the building or parcel level — coordinates within a few metres — the highest tier of the geocode cascade. Sourced from address-point and situs data./interp shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. hit on the parsed streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. | 2026-06-17-geocoder-vs-provided-coords |
| rooftoprooftopGeocoding precision at the building or parcel level — coordinates within a few metres — the highest tier of the geocode cascade. Sourced from address-point and situs data. (address_point) tiertierInternal versioning of which label classes the model emits. Tier 1 is the coarse components (country, region, locality, postcode); Tier 2 adds venue, street, house_number; Tier 3 (future) would add attention, po_box, and POI venue subtyping. Historically called 'Stage 1/2/3' before the runtime-pipeline naming made that ambiguous. | fires 47%; 0.7 km p50 where it fires | resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon. | open | coveragecoverageThe fraction of a population or region for which a data source has real, non-placeholder entries — e.g. 47% rooftop coverage on Texas addresses. Distinct from accuracy on the rows that are present. is the frontier — accuracy is solved when a shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. has the point | 2026-06-17-geocoder-vs-provided-coords |
| interpolationinterpolationA geocoding technique that estimates a coordinate along a street segment based on the house number range. Used as the middle tier of Mailwoman's geocode cascade when exact address-point data is unavailable. (streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.) tiertierInternal versioning of which label classes the model emits. Tier 1 is the coarse components (country, region, locality, postcode); Tier 2 adds venue, street, house_number; Tier 3 (future) would add attention, po_box, and POI venue subtyping. Historically called 'Stage 1/2/3' before the runtime-pipeline naming made that ambiguous. | fires 12.5%; 0.1 km p50; raw radius covered only 72% → ×1.70 for a true 90% bound | resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon. | fixed (radius) | a radius is decoration unless calibrated (#374) | 2026-06-14-interp-radius-calibration |
| off-map countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. routing | 88→ clears 90/90 (decision rule, no retrain) | resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon. | fixed | 1 − P(OTHER) in-map mass beats softmaxsoftmaxThe function that converts a vector of logits into a probability distribution summing to 1, applied after priors and biases are added to the emission logits. argmax | 2026-06-14-coarse-placercoarse-placerA lightweight int8 country classifier (~0.79 MB) that predicts which of a set of target countries an address belongs to, feeding a soft prior into resolver disambiguation.-arc-postmortem |
| in-map wrong-regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. misroute | 0 / 2000 across 10 countries | resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon. | fixed | the soft priorsoft priorOutside knowledge fed to the model or resolver as overridable evidence (a feature, a score term) rather than a hard filter. A focus-country hint becomes an anchor feature; a focus-point becomes a ranking term. re-rank never misroutes (tiertierInternal versioning of which label classes the model emits. Tier 1 is the coarse components (country, region, locality, postcode); Tier 2 adds venue, street, house_number; Tier 3 (future) would add attention, po_box, and POI venue subtyping. Historically called 'Stage 1/2/3' before the runtime-pipeline naming made that ambiguous.-safe) | 2026-06-14-coarse-placercoarse-placerA lightweight int8 country classifier (~0.79 MB) that predicts which of a set of target countries an address belongs to, feeding a soft prior into resolver disambiguation.-arc-postmortem |
The headline: where the finer tierstierInternal versioning of which label classes the model emits. Tier 1 is the coarse components (country, region, locality, postcode); Tier 2 adds venue, street, house_number; Tier 3 (future) would add attention, po_box, and POI venue subtyping. Historically called 'Stage 1/2/3' before the runtime-pipeline naming made that ambiguous. fire, the geocoder is rooftoprooftopGeocoding precision at the building or parcel level — coordinates within a few metres — the highest tier of the geocode cascade. Sourced from address-point and situs data.-accurate (0.1–0.7 km, calibrated). The open problem is coveragecoverageThe fraction of a population or region for which a data source has real, non-placeholder entries — e.g. 47% rooftop coverage on Texas addresses. Distinct from accuracy on the rows that are present. — ~40% fall back to a city centroidlocality centroidThe representative centre point of a city or locality, used as a coarse coordinate when no exact address point is available — the coarsest tier of the geocode cascade. for lack of a shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row.. That's a data lever (more situssitusThe physical site address of a property, as opposed to the owner's mailing address. Parcel records often carry both; the divergence is a real-world data-quality challenge./interpolationinterpolationA geocoding technique that estimates a coordinate along a street segment based on the house number range. Used as the middle tier of Mailwoman's geocode cascade when exact address-point data is unavailable. shardsshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row.), not a modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' one. See the companion concept note on coordinate sufficiency ("How close is close enough?") for what these tierstierInternal versioning of which label classes the model emits. Tier 1 is the coarse components (country, region, locality, postcode); Tier 2 adds venue, street, house_number; Tier 3 (future) would add attention, po_box, and POI venue subtyping. Historically called 'Stage 1/2/3' before the runtime-pipeline naming made that ambiguous. are worth per use-case.
5. Parity gaps & boundary instability
| class | measured | engine | status | lever / root cause | source |
|---|---|---|---|---|---|
| us.streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. (golden dev) | 75.5% → 77.9% (below the 80% bar) | neural | open | streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.-eats-affix boundary wobble; #511 helped affixes but didn't fully recover streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. precisionprecisionOf the spans the model labeled as a given tag, the fraction it got right. High precision means few false positives. Paired with recall to compute F1. | parity-scorecard-2026-06-11 |
| us.localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. (golden dev) | 60.1% → 75.7% (below 80%) | neural | open | improving, but still under the canonical bar — note it beats v0 on OA-clean (§6) | parity-scorecard-2026-06-11 |
| perturb arenaarenaA standardized test set probing one capability: libpostal (clean canonical), perturb (noisy and degraded), postal (edge formats). Each arena answers a different question about where rule vs neural wins. (noisy/glued) | 64% vs 71% (v4.2.0); floor restored to 72 in v4.4.0 | neural | partially-fixed | glue perturbation (NY14201) + post-directional wobble — the same boundary-instability family | 2026-06-11-v4.4.0-ship-gate |
The connective tissue across §1, §5: tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words.-boundary instabilityboundary instabilityA class of parser errors where the model wobbles on where one address component ends and the next begins — a street suffix swallowed into the street name, a French house number that trails instead of leads. Mailwoman's #1 weakness as of v1.5.0. (streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. eats affix, regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.+postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. glue, directional wobble, dotted-abbreviation absorption) is one failure family surfacing under many names. It's the most leveraged single area — a boundary-aware decode lever would touch several rows.
6. The inverse — capability asymmetries (wins worth defending)
| class | measured | winner | note |
|---|---|---|---|
| multi-word localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. | neural 90.0% vs v0 84.7% | neural | +5.3pp — this week's per-type finding |
| directional streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. | neural 87.2% vs v0 85.2% | neural | +2.0pp |
| noisy / degraded input | neural 60% vs v0 39% | neural | +21pp; v0 shatters on quotes |
| graceful degradation (unbalanced delimiters) | neural 82.5% vs v0 68.3%, 0 parseaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates. deaths | neural | v0 throws; neural degrades |
within-tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. punctuation (O'Brien, St., 123 1/2) | v0 89/87/72% vs neural 81/81/62% | v0 | the standing neural weakness — v0's rules cut precisely where neural absorbs |
The capability map is not "rules vs ML, ML wins." It's: neural owns structure, robustness, and the negative-space tags (po_boxPO boxA numbered mailbox at a post office used as a delivery address instead of a physical street location. Mailwoman tags it as the po_box component; structurally the same family as a subpremise./unitunitA subdivision of a building — apartment, suite, floor — that refines a street address. Mailwoman's unit component; a designator plus identifier forms a subpremise.); v0 still owns precise within-tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. punctuation. The last row is the honest open edge — the clearest place a rule-assist or a punctuation-aware decode could close a measured gap.
Unmeasured (so we don't fabricate a number)
- Apostrophe-in-tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. root cause — v0 wins 89 vs 81, but no tokenizertokenizerThe component that converts a raw address string into a sequence of numeric token IDs the model can process. Mailwoman's tokenizer is a SentencePiece unigram model trained specifically on postal addresses. trace explains why.
- Natural intersectionintersectionAn address that names a location by two crossing streets ('5th & Main') rather than a number and street. Mailwoman tags the two streets as intersection_a and intersection_b — a negative-space format that starved the early model. frequency — templated 82%; real-world deployment rate unknown.
- Cross-localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for. org-name variants (DBA, nicknames, expanded designations) — flagged as record-matcher follow-up; no parser failure rate measured.
- fr venuevenueA named, non-address place — a business, building, park, or stadium. Mailwoman's free-text point-of-interest component, added as a Tier 2 fine label. / regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. (#330) — marked open on the scorecard, but no current benchmark number.
What the table says about the roadmap
- Boundary instabilityboundary instabilityA class of parser errors where the model wobbles on where one address component ends and the next begins — a street suffix swallowed into the street name, a French house number that trails instead of leads. Mailwoman's #1 weakness as of v1.5.0. is the highest-leverage parser lever — it's one family (§1 dotted, §5 streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels./glue, §6 within-tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words.) under many names; a boundary-aware decode would move several rows at once.
- Geocoder accuracy is solved; coveragecoverageThe fraction of a population or region for which a data source has real, non-placeholder entries — e.g. 47% rooftop coverage on Texas addresses. Distinct from accuracy on the rows that are present. is the frontier — the ~40% admin fallback is a shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row.-data problem, not a modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' one (§4).
- LocalelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for. is a data problem, not a weightparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. problem — fr.house_number falsified weightparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. tuning; real reordered/native data is the only remaining lever (§2).
- The evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. gate earns its keep — the rejected paired-delimiter proposer (§3) and the deferred geocoder wiring (§4, #694) are both cases where a plausible change was stopped by a measured regression. Keep grading the assembled output, not labelcomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag.-F1 (the #566 discipline).
Sources: all rows cite a dated evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. under docs/articles/evals/. Re-run the named evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. to refresh.