Tokenizer A0 — baseline

The v0.5.0 plan (PHASE_8 §A) splits the tokenizertokenizerThe component that converts a raw address string into a sequence of numeric token IDs the model can process. Mailwoman's tokenizer is a SentencePiece unigram model trained specifically on postal addresses. retrain into two steps:

• A0 trains on corpus-v0.3.0 (the existing en-US + fr-FR corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus., no transliterationstransliterationConverting a name from one writing system to another while preserving pronunciation (Cyrillic → Latin, for instance). Needed for multilingual address handling and corpus synthesis.) using the new SentencePieceSentencePieceA language-independent subword tokenizer that splits text into pieces using a unigram language model. Mailwoman uses a SentencePiece tokenizer with a 48,000-token vocabulary and byte-fallback, trained on address data rather than general text. harness. Its purpose is to validate the harness and measure a byte-fallback baseline before the new synthetic data lands.
• A1 retrains on corpus-v0.4.0 (= v0.3.0 + Thread B kryptonite + Thread B2 transliterationtransliterationConverting a name from one writing system to another while preserving pronunciation (Cyrillic → Latin, for instance). Needed for multilingual address handling and corpus synthesis.). Its purpose is to actually hit the plan's < 5% byte-fallback target on non-Latin scripts.

This article records the A0 numbers so the A1 delta is interpretable.

The headline numbers

A0 was trained 2026-05-23 on a single host (no rented compute), 1.04 hours wall-clock. SentencePieceSentencePieceA language-independent subword tokenizer that splits text into pieces using a unigram language model. Mailwoman uses a SentencePiece tokenizer with a 48,000-token vocabulary and byte-fallback, trained on address data rather than general text. configuration:

Knob	A0	v0.1.0 (predecessor)
model_type	unigram	unigram
vocab_size	48,000	16,000
character_coverage	0.9999	0.9995
byte_fallback	true	true
user_defined_symbols	2,110	0
TrainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input. corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus.	corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus.-v0.3.0, US + FR @ 500K each + 2K mined postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. literals	(historical)

EvalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. fixture: 43 hand-curated lines covering CJK / Cyrillic / Armenian / Greek / Arabic / Hebrew / Devanagari / Thai / Latin-with-diacriticsdiacriticAn accent mark that modifies a letter (é, ñ, ç). Address normalization must fold diacritics for matching without discarding the information a user typed.. The fixture lives at data/eval/multi-script/v0.5.0-a0.jsonl.

Script	A0 byte-fallback	v0.1.0 baseline	Δ
overall	36.7%	18.0%	+18.7 pt
cjk	80.0%	51.6%	+28.4 pt
cyrillic	2.2%	0.0%	+2.2 pt
armenian	21.3%	0.0%	+21.3 pt
greek	16.3%	0.0%	+16.3 pt
arabic	5.0%	0.0%	+5.0 pt
hebrew	10.0%	0.0%	+10.0 pt
devanagari	12.8%	0.0%	+12.8 pt
thai	46.7%	10.0%	+36.7 pt
latin	5.8%	0.0%	+5.8 pt
other	34.3%	0.0%	+34.3 pt

A0 is worse than v0.1.0 on every script. This is not a bug. It is the expected outcome and it is the point of running A0.

Why A0 looks worse — the WOF admin-name leakage hypothesis

The v0.1.0 tokenizertokenizerThe component that converts a raw address string into a sequence of numeric token IDs the model can process. Mailwoman's tokenizer is a SentencePiece unigram model trained specifically on postal addresses. was trained on a corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus. that included Who's On FirstWOF (Who's On First). An open-source gazetteer of places maintained by Mapzen/whosonfirst. Mailwoman builds a custom SQLite database from WOF GeoJSON repos, extended with postcode data, importance scores, and coincident-role relations. admin-name variants. WOFWOF (Who's On First). An open-source gazetteer of places maintained by Mapzen/whosonfirst. Mailwoman builds a custom SQLite database from WOF GeoJSON repos, extended with postcode data, importance scores, and coincident-role relations. ships place namestoponymA proper name for a geographic place. in many scripts — Tokyo appears as Tokyo and 東京 and Tōkyō and in Cyrillic transliterationtransliterationConverting a name from one writing system to another while preserving pronunciation (Cyrillic → Latin, for instance). Needed for multilingual address handling and corpus synthesis., all in the same admin record. Mailwoman v0.1.0's trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input. pipelinestaged pipelineMailwoman's runtime architecture: a sequence of pure-function stages (normalize → query-shape → locale-gate → kind-classifier → phrase-grouper → classifier → decoder) connected by typed handoffs. Each stage is published as its own npm package. pulled these admin records to populate StagestageOne of the dataflow stages in the runtime pipeline (normalize, locale gate, kind classify, phrase group, token classify, sequence correct, reconcile, resolve). Distinct from tier (model vocabulary) and phase (plan milestone). 6 lookups, and the tokenizertokenizerThe component that converts a raw address string into a sequence of numeric token IDs the model can process. Mailwoman's tokenizer is a SentencePiece unigram model trained specifically on postal addresses. trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input. scanned the same data.

The result: v0.1.0's byte-fallback rate on Cyrillic / Armenian / Greek / Arabic / Hebrew / Devanagari was effectively zero, not because the corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus. had real coveragecoverageThe fraction of a population or region for which a data source has real, non-placeholder entries — e.g. 47% rooftop coverage on Texas addresses. Distinct from accuracy on the rows that are present. of those scripts but because the place-name admin variants accidentally taught the tokenizertokenizerThe component that converts a raw address string into a sequence of numeric token IDs the model can process. Mailwoman's tokenizer is a SentencePiece unigram model trained specifically on postal addresses. enough sub-piecesECE (Expected Calibration Error). A metric that measures how well a model's confidence scores align with its actual accuracy. Lower is better. Mailwoman's held-out ECE drops from 0.067 (raw) to 0.0035 (calibrated). to avoid byte-fallback on a small evaluationevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. slice.

When we moved to corpus-v0.3.0 (US DOT NADNAD (National Address Database). A US Department of Transportation dataset of structured address points, added to the training corpus as a major source of real US addresses. + WOFWOF (Who's On First). An open-source gazetteer of places maintained by Mapzen/whosonfirst. Mailwoman builds a custom SQLite database from WOF GeoJSON repos, extended with postcode data, importance scores, and coincident-role relations. postalcode + cleaner US/FR address rows), those incidental admin-name variants stopped being in the tokenizertokenizerThe component that converts a raw address string into a sequence of numeric token IDs the model can process. Mailwoman's tokenizer is a SentencePiece unigram model trained specifically on postal addresses. trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input. set. A0 — trained on the same en-US + fr-FR mass without the WOFWOF (Who's On First). An open-source gazetteer of places maintained by Mapzen/whosonfirst. Mailwoman builds a custom SQLite database from WOF GeoJSON repos, extended with postcode data, importance scores, and coincident-role relations. admin leakageleakageTrain/test contamination that inflates reported accuracy when eval data has effectively been seen in training. Mailwoman guards it with held-out-geography evals and locality-aware splits. — has effectively no non-Latin coveragecoverageThe fraction of a population or region for which a data source has real, non-placeholder entries — e.g. 47% rooftop coverage on Texas addresses. Distinct from accuracy on the rows that are present.. Hence the regressions you see in the table.

The v0.1.0 numbers were never an honest measurement of multi-script capability; they were a leakageleakageTrain/test contamination that inflates reported accuracy when eval data has effectively been seen in training. Mailwoman guards it with held-out-geography evals and locality-aware splits. artefact. A0 is the first honest measurement of "what does our tokenizertokenizerThe component that converts a raw address string into a sequence of numeric token IDs the model can process. Mailwoman's tokenizer is a SentencePiece unigram model trained specifically on postal addresses. do on adversarial transliterationtransliterationConverting a name from one writing system to another while preserving pronunciation (Cyrillic → Latin, for instance). Needed for multilingual address handling and corpus synthesis. input when we have not trained it on any". The answer is: badly. As expected.

What A1 will measure

A1 retrains on corpus-v0.4.0 = corpus-v0.3.0 + Thread B (4,771 kryptonite rows) + Thread B2 (~75K transliterationtransliterationConverting a name from one writing system to another while preserving pronunciation (Cyrillic → Latin, for instance). Needed for multilingual address handling and corpus synthesis. pairs into Cyrillic / Japanese / Hangul / Han / Armenian). The transliterationtransliterationConverting a name from one writing system to another while preserving pronunciation (Cyrillic → Latin, for instance). Needed for multilingual address handling and corpus synthesis. pairs are the part that matters for byte-fallback — they put real non-Latin character sequences into the tokenizertokenizerThe component that converts a raw address string into a sequence of numeric token IDs the model can process. Mailwoman's tokenizer is a SentencePiece unigram model trained specifically on postal addresses.'s trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input. set.

The headline metric is the A1 vs A0 delta on each script:

• Target: A1 byte-fallback < 5% overall on the same fixture, with per-script rates approaching the v0.1.0 numbers (the leakageleakageTrain/test contamination that inflates reported accuracy when eval data has effectively been seen in training. Mailwoman guards it with held-out-geography evals and locality-aware splits. numbers) as a floor.
• Stretch: per-script rates well below v0.1.0 on the scripts B2 explicitly covers (cjk, cyrillic, hangul, han, armenian).
• Anti-goal: regression on Latin. A1 should not lose ground on the en-US + fr-FR mass.

A1 retrain is a single re-invocation of the A0 harness against corpus-v0.4.0 — deterministic with --seed 42, takes the same ~1 hour. The output writes to /data/models/tokenizer/v0.5.0-a1/ and the model cardmodel cardA JSON metadata file (model-card.json) shipped with each weights bundle. It declares the model version, lineage, label set, required inference channels (anchor, gazetteer), calibration data, and training provenance. carries the same shape (a JSON file with sentencepieceSentencePieceA language-independent subword tokenizer that splits text into pieces using a unigram language model. Mailwoman uses a SentencePiece tokenizer with a 48,000-token vocabulary and byte-fallback, trained on address data rather than general text. flags, trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input.-line count, trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input. duration, byte-fallback per-script, and a SHA-256 of the .model file).

What A0 is good for, even though it lost

Three things:

1. The harness is validated. A0 trained end-to-end on real data. Reservoir sampling, the user-defined-symbols normalisation step (see SP UDS whitespace gotcha — or, if you are reading on the published docs site, the version in the mailwoman repo at corpus-python/src/mailwoman_train/tokenizer_train.py), the model cardmodel cardA JSON metadata file (model-card.json) shipped with each weights bundle. It declares the model version, lineage, label set, required inference channels (anchor, gazetteer), calibration data, and training provenance. writer, the byte-fallback evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. — every piece worked. A1 has zero new code to write.
2. The evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. fixture is validated. 43 hand-curated multi-script lines is small but the per-script breakdown is informative enough to tell us where a retrain helped vs hurt. Future tokenizertokenizerThe component that converts a raw address string into a sequence of numeric token IDs the model can process. Mailwoman's tokenizer is a SentencePiece unigram model trained specifically on postal addresses. iterations will use the same fixture and the same harness, so A0 / A1 / A2 numbers will be directly comparable.
3. The user-defined-symbols list is validated. A0 has 2,110 UDS — US stateregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. codes, countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. abbreviations, postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. literal patterns (5-digit ZIP, 5-4 ZIP+4, UK alphanumeric blocks, JP 100-0005). The A0 harness substitutes ASCII space → U+2581 internally so SP's normalisation does not silently disable the UDS (the SP UDS whitespace gotcha). That substitution is now battle-tested.

Model card schema

A0's model_card.json is the contract for every future tokenizertokenizerThe component that converts a raw address string into a sequence of numeric token IDs the model can process. Mailwoman's tokenizer is a SentencePiece unigram model trained specifically on postal addresses.. It carries:

{
  "tokenizer_version": "v0.5.0-a0",
  "corpus_version": "v0.3.0",
  "vocab_size": 48000,
  "training_lines": 1000000,
  "training_duration_seconds": 6294.237,
  "generated_at": "2026-05-23T19:53:06Z",
  "git_commit": "378a55d…",
  "model_sha256": "a864fde…",
  "sentencepiece_flags": { … },
  "sampling": {
    "countries": ["US", "FR"],
    "per_country": 500000,
    "seed": 42
  },
  "byte_fallback_eval": {
    "overall": { "lines": 43, "pieces": 1075, "byte_fallback_pieces": 395, "rate": 0.367 },
    "per_script": { "cjk": …, "cyrillic": …, … }
  }
}

META.json is a thin shim with the same content, kept for compatibility with @mailwoman/neural-weights-* package loaders that already read it.

See also

• PHASE_8 §A — the plan's tokenizertokenizerThe component that converts a raw address string into a sequence of numeric token IDs the model can process. Mailwoman's tokenizer is a SentencePiece unigram model trained specifically on postal addresses. scope
• Tokenization — what SentencePieceSentencePieceA language-independent subword tokenizer that splits text into pieces using a unigram language model. Mailwoman uses a SentencePiece tokenizer with a 48,000-token vocabulary and byte-fallback, trained on address data rather than general text. does and why byte-fallback matters
• v0.5.0 — as shipped — where A0 fits in the six-threadthreadA parallel workstream within a release. Threads compose; they are not sequential milestones like phases. bundle
• CORPUS_V0_4_0_GENERATION.md — how the corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus. A1 will train on is being assembled