Boundary-instability: the current model's gap, quantified (#375)

The failure taxonomy named boundary instabilityboundary instabilityA class of parser errors where the model wobbles on where one address component ends and the next begins — a street suffix swallowed into the street name, a French house number that trails instead of leads. Mailwoman's #1 weakness as of v1.5.0. the #1 parser lever and the within-tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. decomposition (#702) showed it surfacing under many names. The boundary-stress shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. (#703) is the trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input.-data fix. This is the "before" baseline — how badly today's modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' places these boundaries, on the exact synthetic shapes the shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. teaches (scripts/eval/boundary-stress-baseline.ts, 300 rows/shape through the current neural modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.', exact-match per tag). The shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. is base-localeslocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for.-only (US/FR/DE) — see the base-consistency section for why.

stress shape	stress tag	accuracy	streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. accuracy
streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.-eats-affix	street_suffix	41.7%	39%
comma-less CitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. ST	streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.	47%	(localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. 91, regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. 90)
fr-prefix	street_prefix	55.0%	39%
house-number-after-streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.	house_number	51.3%	53%

(Measured on the base-consistent localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. vocabularyvocabularyThe fixed set of tokens a tokenizer can produce. Mailwoman's SentencePiece vocabulary is tens of thousands of subword pieces, with byte fallback for anything outside it. — see the #511 section. Earlier drafts read street_suffix 40.7 / fr-prefix 47.7 on a vocabvocabularyThe fixed set of tokens a tokenizer can produce. Mailwoman's SentencePiece vocabulary is tens of thousands of subword pieces, with byte fallback for anything outside it. that contradicted the base; the numbers above are on the corrected vocabvocabularyThe fixed set of tokens a tokenizer can produce. Mailwoman's SentencePiece vocabulary is tens of thousands of subword pieces, with byte fallback for anything outside it.. house-number-after-streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. is now FR-only.)

(On the delimited shapes the uncontested tags read ~100% — the failures concentrate on the contested boundary, exactly as designed. US comma-less localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy./regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. are easy at 91/100%; the streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. spanspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree. is the casualty there too.)

Diversity correction (the runbook's point, measured): an initial thin-pool shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. (~16 streetsstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels., ~7 tuples) gave an inflated baseline — street_suffix read 48%, fr-prefix 70% — because the few lexemesexpectation-maximizationAn iterative algorithm that estimates model parameters when some variables are unobserved. In Mailwoman's matcher, EM learns the Fellegi-Sunter m and u parameters from unlabeled data — no training labels needed. were memorizable. Expanding the pools ~3× (≈100 distinct streetsstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels., 28 US / 12 FR / 10 DE tuples, ~100% unique rows) drops those to 40.7% / 47.7% — the true gap on the real distribution. Exactly why CONTRIBUTING_MODEL_WORK gates on diversity: a thin shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. teaches lexemesexpectation-maximizationAn iterative algorithm that estimates model parameters when some variables are unobserved. In Mailwoman's matcher, EM learns the Fellegi-Sunter m and u parameters from unlabeled data — no training labels needed. and a thin baseline hides the gap.

Base-consistency (#511 lint) — the gate caught a real contradiction

Running the #511 base-consistency lint (lint-corpus-shard.ts against sampled v0.5.0 base-stats) caught two things, only one of them a true problem:

• AU content was a real contradiction → FIXED. An earlier draft used AU/NZ/UK tuples (for an au-uk-slash-unit shape + AU comma-less). AU isn't a base localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for. (the base is US/FR/DE), and AU 4-digit postcodespostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. collide with US house numbershouse numberThe numeric or alphanumeric identifier of a building on a street. Mailwoman's house_number component; its position relative to the street name flips between locales. (3000 is a common US street numberhouse numberThe numeric or alphanumeric identifier of a building on a street. Mailwoman's house_number component; its position relative to the street name flips between locales.). The lint flagged it; the shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. is now base-localeslocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for.-only, and the AU/UK slash convention (the worst within-tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. class, #702) is deferred to a separately-scoped AU/NZ/UK shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. that also adds AU base coveragecoverageThe fraction of a population or region for which a data source has real, non-placeholder entries — e.g. 47% rooftop coverage on Texas addresses. Distinct from accuracy on the rows that are present. — it cannot ride a US/FR/DE shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. without contradicting the base.
• The affix-tag flags are train-time-relabel artifacts (Ave/Place/NW → suffix/prefix): the lint compares pre-relabel parquetsParquetThe open columnar file format the corpus is written and streamed in. The training pipeline reads shards row-by-row from Parquet., and the affix-relabel lexicon (verified) maps every suffix + directional. Safe.
• The localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy./streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. overlap was RESOLVED by deriving the vocabularyvocabularyThe fixed set of tokens a tokenizer can produce. Mailwoman's SentencePiece vocabulary is tens of thousands of subword pieces, with byte fallback for anything outside it. from the base itself. A targeted scan (what labelcomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag. does the base give each shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.?) found the original US vocabvocabularyThe fixed set of tokens a tokenizer can produce. Mailwoman's SentencePiece vocabulary is tens of thousands of subword pieces, with byte fallback for anything outside it. was a genuine contradiction — Madison 96% streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels., Portland 95%, Springfield IL 84% (the "5th Avenue Theatre" class, well-sampled across ~23 US shardsshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row.). Fixed: the US vocabvocabularyThe fixed set of tokens a tokenizer can produce. Mailwoman's SentencePiece vocabulary is tens of thousands of subword pieces, with byte fallback for anything outside it. is now derived + verified localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.-dominant (Albuquerque 258584:8, Indianapolis 219700:29, Sacramento, Jacksonville…). The FR flag, by contrast, was a sampling artifact — the all-shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. scan barely touched the FR banBAN (Base Adresse Nationale). France's authoritative open national address register — the highest-quality training source for French addresses, with full component structure. block (parts 180–209) and mixed in US streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.-contexts; the FR-block scan shows Paris (515605:24789), Marseille, Lyon are 95–99% localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. in the FR data, so familiar dept-diverse FR cities are kept. DE yielded no localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.-dominant towns (German cities are streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.-dominated too, "Berliner Straße"), so house-number-after-streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. is FR-only (DE's native order is covered by synth-german). Netneural networkA model made of layers of simple numeric units whose connection strengths (weights) are learned from data. The transformer encoder at Mailwoman's core is a neural network.: every shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. now agrees with the base — the contradiction is gone, not deferred.

Reading

• The modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' is 38–51% on the boundary-stress cases vs ~95%+ on clean canonical — a large, real gap.
• The streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. boundary is the common casualty (38% / 46% / 43% / 49% across all four shapes): when an adjacent component is ambiguous, the streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. spanspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree. absorbs or surrenders tokenstokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words.. One failure, many faces.
• house-number-after-streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. 51% is the fr.house_number plateau in miniature (the FR/DE number-follows-streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. order) — this shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row.'s house-number-after-street shape targets that lever too.

So what

The shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. puts the gold boundary on diverse realizations of these shapes. The localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. vocabularyvocabularyThe fixed set of tokens a tokenizer can produce. Mailwoman's SentencePiece vocabulary is tens of thousands of subword pieces, with byte fallback for anything outside it. is now base-derived (the #511 contradiction is resolved, not deferred), so the shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. is retrain-ready. The retrain's success criterion (the v1.6.0-boundary-stress recipe gate): move these four numbers up (street_suffix 41.7 → ≥55, comma-less streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. 47 → ≥65, fr-prefix 55 → ≥70, hn-after 51.3 → ≥65) without regressing the clean canonical per-locale F1per-locale F1Per-tag accuracy computed separately for each locale (US, FR, DE…) rather than aggregated, surfacing locale-specific regressions that macro F1 would mask under the dominant locale. (the US/FR/DE tripwire) and the affix floors — one variable, gated, per CONTRIBUTING_MODEL_WORK.

Source: scripts/eval/boundary-stress-baseline.ts over corpus/src/synthesize-boundary-stress.ts.