Skip to main content

Per-system transition masks — design note (#478 slice 3, not yet built)

Status: DESIGN ONLY (night-11). Slice 1 (emission mask, v4.3.0) and the spanspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree. bridge (v4.4.0 corrective) shipped; the train-time lossloss functionA number measuring how wrong the model's predictions are on a batch of examples. Training minimizes it. Mailwoman's loss combines per-token negative log-likelihood with the CRF sequence loss.-mask pairing is implemented and banked (use_conventions_loss_mask, probe deferred by consult — rides the next full run). Transition masks are the next decode-side slice, recorded here with their failure mode BEFORE anyone implements them in a hurry.

The idea​

The conventions table grows a forbiddenTransitions field: ordered labelcomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag. pairs that are ungrammatical in a detected system, applied as additive -1e9 entries to the ViterbiViterbi decodingA dynamic programming algorithm that finds the most likely sequence of hidden states (labels) given a sequence of observations (token emissions). Mailwoman uses Viterbi over a linear-chain CRF to produce globally coherent BIO label sequences from per-token model scores. transition matrix (which is structural-BIO-only today). Candidate first rows, each with measured motivation:

  • fr/de: I-postcode → B-street and I-postcode → B-house_number — the digit-split family (a 5-digit-system postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. never hands off mid-number; both systemsexpectation-maximizationAn iterative algorithm that estimates model parameters when some variables are unobserved. In Mailwoman's matcher, EM learns the Fellegi-Sunter m and u parameters from unlabeled data — no training labels needed.' shapes are ^\d{5}$).
  • Universal candidate (needs care): B-postcode → B-postcode within three chars — the double-postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. emission the comma-bridge incident exposed (the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' double-labelscomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag. a following house numberhouse numberThe numeric or alphanumeric identifier of a building on a street. Mailwoman's house_number component; its position relative to the street name flips between locales. as a second postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. fragment).

The pre-registered failure mode (DeepSeek, 2026-06-10 — the one keeper from that consult)​

Banning I-postcode → B-street does not make the orphan digit correct — it forces the probability somewhere, and the most likely refuge is I-postcode: the digit gets ABSORBED INTO the postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one., corrupting it worse than the split did ("4711" + streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. "0 …" at least kept four correct digits; "47110…" absorbing a house numberhouse numberThe numeric or alphanumeric identifier of a building on a street. Mailwoman's house_number component; its position relative to the street name flips between locales. corrupts the field outright). Any transition- mask implementation must ship with this as a regression TEST CASE: the FR digit-split rows + the glue rows, asserted not-worse under the mask.

Why it is NOT being built tonight​

  1. The motivating classes are now covered upstream: the digit-split died with the #511 relabel (in-weightsparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values., measured 6→2 FR misses with no mask), and the glue class died with the #513 augmentation (arenaarenaA standardized test set probing one capability: libpostal (clean canonical), perturb (noisy and degraded), postal (edge formats). Each arena answers a different question about where rule vs neural wins. floor 71 restored). A mask with no live failing class has no read.
  2. The double-postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. emission is real (the comma-bridge incident) but is better attacked at its source first — it is a trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input.-distribution question (why does the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' double-labelcomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag.?) before it is a decode-constraint question.
  3. Slice discipline: each conventions slice shipped against a measured failure. This one currently has none. The design waits for its evidence, like the de emission row waits for the leakageleakageTrain/test contamination that inflates reported accuracy when eval data has effectively been seen in training. Mailwoman guards it with held-out-geography evals and locality-aware splits. audit's class to grow past ~1%.

When to revisit​

  • A gate FAIL whose row characterization shows an ORDERING error (not a spanspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree./boundary error) in a conventioned system.
  • The double-postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. class surviving the next corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus. pass.
  • The de conventions row shipping (its postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. shape + the transition pair come as a unitunitA subdivision of a building — apartment, suite, floor — that refines a street address. Mailwoman's unit component; a designator plus identifier forms a subpremise.).