v8 Phase-0 — JP field→char-span alignment de-risk

Date: 2026-07-18 (night). Verdict: PASS. The named biggest v8 risk is retired. The JP char-path Leg-1 probe is unblocked; the only remaining plumbing is the char_ids data-loader path.

The risk (Fable v8-CJK plan, §"biggest unnamed risk")

CJK addresses have no whitespace, and Overture gives field values (street, number, address_levels) not character offsets. The plan flagged: field→character-spanspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree. alignmentalignmentThe step in the corpus pipeline that takes a (raw, components) pair from an adapter and produces a (raw, tokens, BIO labels) row by finding each component's text inside the raw string and labeling the matching tokens. over unsegmented kanji could be an ambiguous substring-match problem — potentially forcing a MeCab/Sudachi (morphological segmenter) or block-regex dependency before any v8 compute is worth spending. PhasephaseA milestone in the implementation plan (Foundation, Corpus, Training, Integration, and forward-looking phases). Distinct from stage (runtime pipeline) and tier (model vocabulary).-0 gate: ≥95% clean alignmentalignmentThe step in the corpus pipeline that takes a (raw, components) pair from an adapter and produces a (raw, tokens, BIO labels) row by finding each component's text inside the raw string and labeling the matching tokens. on a hand-checkable sample, or the segmenter dependency is named.

The test

scratchpad/jp-phase0/alignment-report.txt. Overture-JP schema: street (kanji aza / chōmechōmeIn Japanese addressing, a district-level subdivision in the block-based chōme / banchi / gō numbering scheme, which uses area-and-block numbers instead of street names. name, e.g. 字崎枝, 神南1丁目), number (Latin digits + hyphens, banchibanchiIn Japanese addressing, a block within a chōme. The middle level of the chōme / banchi / gō hierarchy.-go, e.g. 556-16, 2-3-16), address_levels[] = [prefecture 都道府県, municipalitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. 市区町村]. Concatenate large-to-small (JP reading order): 沖縄県石垣市字崎枝556-16. Greedy left-to-right spanspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree. alignmentalignmentThe step in the corpus pipeline that takes a (raw, components) pair from an adapter and produces a (raw, tokens, BIO labels) row by finding each component's text inside the raw string and labeling the matching tokens. (find each field value from the cursor); a row is CLEAN iff every field is contiguous and the fields cover the whole string with no gap/overlap. Ambiguity metric: does any field value occur >1× in the concatenation (a naive matcher couldn't place it uniquely)?

• Run 1 — 500 rows (regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.-clustered): 500/500 = 100% clean, 0 repeat-substring ambiguity.
• Run 2 — 1,060 rows strided across 159 row-groups spanning all 46 prefectures, incl. 171 rows with urban 丁目 (chōmechōmeIn Japanese addressing, a district-level subdivision in the block-based chōme / banchi / gō numbering scheme, which uses area-and-block numbers instead of street names.) structure: 1,060/1,060 = 100% clean.

Why it's clean (the structural reason)

Japanese address field values are mutually non-colliding: prefecture/municipalitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy./streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. are distinct kanji sequences, and the number is Latin digits — so even a naive substring matcher places each uniquely (0 collisions across 1,560 rows). The urban chōmechōmeIn Japanese addressing, a district-level subdivision in the block-based chōme / banchi / gō numbering scheme, which uses area-and-block numbers instead of street names. case (…神南1丁目3-16) aligns identically — the chōmechōmeIn Japanese addressing, a district-level subdivision in the block-based chōme / banchi / gō numbering scheme, which uses area-and-block numbers instead of street names. lives inside the street value, the trailing 3-16 is the number. No morphological segmenter is needed for the trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input.-corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus. alignmentalignmentThe step in the corpus pipeline that takes a (raw, components) pair from an adapter and produces a (raw, tokens, BIO labels) row by finding each component's text inside the raw string and labeling the matching tokens.. (This is corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus.-build alignmentalignmentThe step in the corpus pipeline that takes a (raw, components) pair from an adapter and produces a (raw, tokens, BIO labels) row by finding each component's text inside the raw string and labeling the matching tokens., where we construct the string from the fields; a segmenter would only be needed if we had to align fields to a pre-existing free-text JP string, which the corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus. never does.)

What this unblocks + the remaining plumbing

• Leg-1 JP char probe is UNBLOCKED — the char-encoderencoderThe part of a transformer that turns input tokens into contextualized vector representations. Mailwoman's classifier is a small encoder-only transformer (~30M parameters). scaffolding exists (corpus-python/src/mailwoman_train/model.py:CharCNNEmbedding + char_tokenizer.py, both gated off, #825), the JP schema tags are declared (SCHEMA.mdx: prefecture/municipalitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy./sub_block, PhasephaseA milestone in the implementation plan (Foundation, Corpus, Training, Integration, and forward-looking phases). Distinct from stage (runtime pipeline) and tier (model vocabulary).-6 forward-compat), and JP Overture data is on disk (19.6M rows, CDLA-Permissive-2.0).
• Remaining work (the real gate, NOT alignmentalignmentThe step in the corpus pipeline that takes a (raw, components) pair from an adapter and produces a (raw, tokens, BIO labels) row by finding each component's text inside the raw string and labeling the matching tokens., and NOT purely mechanical) — a char-encoding DESIGN decision: the existing char encoderencoderThe part of a transformer that turns input tokens into contextualized vector representations. Mailwoman's classifier is a small encoder-only transformer (~30M parameters). char_tokenizer.encode_row_charword is char-WORD (chars grouped by whitespace tokens, one BIO labelcomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag. per word) — built for Latin (#825 Slavic diacriticsdiacriticAn accent mark that modifies a letter (é, ñ, ç). Address normalization must fold diacritics for matching without discarding the information a user typed.). JP has no whitespace and needs per-CHARACTER labelscomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag. (prefecture-chars and streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.-chars sit in the same unsegmented run and carry different tags). So the CharCNN scaffolding is half-built for the Latin char-word case; the JP char-path needs a char-level encoding (per-position ids + per-char BIO), plus the data_loader.py char_ids route. This is the true Leg-1 prerequisite and it's a design choice (char-level vs char-word grouping for a whitespace-less script), not a mechanical wire-up — it wants operator/Fable design sign-off before it's built, since it defines the v8 char modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.''s input contract. Recommend: this is a day-shift collaborative arc, not a pre-handoff rush.

Corpus-build de-risk — also PASS (bonus, #555 does not bite kanji)

Beyond the alignmentalignmentThe step in the corpus pipeline that takes a (raw, components) pair from an adapter and produces a (raw, tokens, BIO labels) row by finding each component's text inside the raw string and labeling the matching tokens. concept, the actual corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus. aligner corpus/src/align.ts::alignRow was tested on JP kanji canonical rows and produces correct char-offset spanspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree. labelscomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag.:

• 沖縄県石垣市字崎枝556-16 → region[0:3] locality[3:6] street[6:9] house_number[9:15]
• 東京都渋谷区神南1丁目3-16 (urban Tokyo, chōmechōmeIn Japanese addressing, a district-level subdivision in the block-based chōme / banchi / gō numbering scheme, which uses area-and-block numbers instead of street names.) → region[0:3] locality[3:6] street[6:11](=神南1丁目) house_number[11:15](=3-16)

#555 (locateSpan over-runs raw on non-Latin combining-mark strings) does not apply to kanji — kanji are single code points with no combining marks, so the spanspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree. offsets are exact. The JP corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus. is buildable now as (raw, char-offset span_tags) rows — encoderencoderThe part of a transformer that turns input tokens into contextualized vector representations. Mailwoman's classifier is a small encoder-only transformer (~30M parameters).-agnostic; the char-level-vs-char-word decision is a trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input.-time read of these spansspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree., not a corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus.-build blocker. (Combining-diacriticdiacriticAn accent mark that modifies a letter (é, ñ, ç). Address normalization must fold diacritics for matching without discarding the information a user typed. scripts — some Indic, heavily-accented Latin — would still need the #555 fix; kanji/hangul/hanzi core CJK do not.)

Netneural networkA model made of layers of simple numeric units whose connection strengths (weights) are learned from data. The transformer encoder at Mailwoman's core is a neural network. v8 data-pipelinestaged pipelineMailwoman's runtime architecture: a sequence of pure-function stages (normalize → query-shape → locale-gate → kind-classifier → phrase-grouper → classifier → decoder) connected by typed handoffs. Each stage is published as its own npm package. status: fully green. Both the alignmentalignmentThe step in the corpus pipeline that takes a (raw, components) pair from an adapter and produces a (raw, tokens, BIO labels) row by finding each component's text inside the raw string and labeling the matching tokens. concept and the corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus.-build infra handle JP cleanly. The only remaining v8 work is trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input.-side: the char-level encoderencoderThe part of a transformer that turns input tokens into contextualized vector representations. Mailwoman's classifier is a small encoder-only transformer (~30M parameters)./loader/modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' rework (above) — a scoped day-shift arc, not a data problem.

One schema decision to make (not a blocker)

The JP number is frequently multi-part (2-3-16 = 2丁目3番16号 = chōmechōmeIn Japanese addressing, a district-level subdivision in the block-based chōme / banchi / gō numbering scheme, which uses area-and-block numbers instead of street names.-banchibanchiIn Japanese addressing, a block within a chōme. The middle level of the chōme / banchi / gō hierarchy.-go). Decide: map the whole 2-3-16 to house_number, or split into the declared sub_block (banchibanchiIn Japanese addressing, a block within a chōme. The middle level of the chōme / banchi / gō hierarchy.) + a go component. This is a schema/labelcomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag. choice for the JP modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.''s vocabvocabularyThe fixed set of tokens a tokenizer can produce. Mailwoman's SentencePiece vocabulary is tens of thousands of subword pieces, with byte fallback for anything outside it. — decide it when building the JP shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row., not before.

Recommendation

KR-first is not executable (no Overture-KR data — the pull returns empty; needs a juso.go.kr / OSMOpenStreetMap (OSM). A community-curated global map database (ODbL-licensed) with addr:* tagged features and place hierarchies. A secondary corpus source and a source of street names.-KR acquisition arc). So JP-first is both the headline choice and the only executable non-Latin path — which resolves the v8 plan's §4 KR-vs-JP reframe with a receipt. Next v8 step: wire the char_ids loader path → run Leg-1 (bare JP char modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' on ~200k rows vs a held-out JP coord board, gate ≥ bare-Latin floor ~0.70). Leg-2 (Latin char bake-off, no JP dependency) exercises the same loader and answers the v9-unification question.