Skip to main content

Width gate + affix miss audit — the #492 ladder closes (2026-06-10)

The probe ladder for issue #492 ran three falsification experiments in one day, each individually GO'd by the operator, each with a pre-registered gate. All three hypotheses fell. The audit that followed found the real constraint, and it was in the data the whole time.

The ladder

ProbeHypothesisResultVerdict
Probe 0 (v1.0.4, choreography off, +4k)choreography suppression erodes affix recallrecallOf the spans whose gold label is a given tag, the fraction the model found. High recall means few misses. Paired with precision to compute F1.decay reproduced 81 → 61.1 with choreography OFFfalsified
Frozen-encoderencoderThe part of a transformer that turns input tokens into contextualized vector representations. Mailwoman's classifier is a small encoder-only transformer (~30M parameters). headattention headOne of several parallel attention computations in a layer, each free to focus on a different kind of relationship between tokens. Their outputs are concatenated — 'multi-head attention'. Mailwoman uses 4 heads. (v1.0.5, init 40k, 2k)shared-BIO output competition; a dedicated headattention headOne of several parallel attention computations in a layer, each free to focus on a different kind of relationship between tokens. Their outputs are concatenated — 'multi-head attention'. Mailwoman uses 4 heads. bypasses itprefix 38.7 / suffix 46.2, P=100 / R≈25 — the headattention headOne of several parallel attention computations in a layer, each free to focus on a different kind of relationship between tokens. Their outputs are concatenated — 'multi-head attention'. Mailwoman uses 4 heads. only finds what the frozen representationhidden stateThe model's internal vector for a token after the encoder has mixed in surrounding context. The contextualized representation the classifier head reads to assign a label. already separatesfalsified
Width 48M (v2.0.0, 512H/8 headsattention headOne of several parallel attention computations in a layer, each free to focus on a different kind of relationship between tokens. Their outputs are concatenated — 'multi-head attention'. Mailwoman uses 4 heads./FF2048, 40k)29M is capacity-boundprefix 64.9 — identical to 29M Run A's 64.9 at matched 5× densityfalsified

Width gate (fp32-to-fp32, pre-registered floors)

FAIL, 6 of 12 checks:

CheckFloorGot
street_prefix≥ 7864.9
street_suffix≥ 6759.1
US streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.≥ 80.479.8
US localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.≥ 72.971.6
unitunitA subdivision of a building — apartment, suite, floor — that refines a street address. Mailwoman's unit component; a designator plus identifier forms a subpremise.≥ 90.687.1
postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one.no-reg98.1
countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.83.3 band87.5
US microno-reg85.4
regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.no-reg89.5
FR (both)no-regpass
DE deorder int8≥ 83.8NOT FOUND (harness wart on the int8 leg; immaterial)

NOT promoted. Artifacts banked: int8 md5 33527afae87526f667c7e83453a723e6 (43 MB — the size alone disqualifies it as a ship default). The intersectionintersectionAn address that names a location by two crossing streets ('5th & Main') rather than a number and street. Mailwoman tags the two streets as intersection_a and intersection_b — a negative-space format that starved the early model. rider (synth-intersection: 2.0) was a data no-op: the v0.4.12 manifest carries zero intersectionintersectionAn address that names a location by two crossing streets ('5th & Main') rather than a number and street. Mailwoman tags the two streets as intersection_a and intersection_b — a negative-space format that starved the early model.-named shardsshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row., so the weightparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. sampled nothing. Recorded on #487.

The audit (scripts/eval/audit-affix-misses.ts)

Hypothesis: misses are out-of-distribution surface forms the shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. builder never varies. Refuted. Misses spread evenly across the shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row.'s bread-and-butter forms (prefix-abbr 65% missed, prefix-full 38%, suffix-abbr 56%, suffix-full 57% — 1 W Pratt St is a miss). No form featurefeatureAn input signal a model conditions on. Beyond the raw tokens, Mailwoman feeds soft features — gazetteer-membership channels and the postcode anchor — that inform predictions without overriding them. separates hits from misses.

What separates them: every missed affix is absorbed into the street spanspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree. (model street="w pratt st", "se division st", "e 63rd street"). The modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' labelscomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag. the affix confidently — the other way.

The other way is what the corpus teaches

Sampled 1M rows across 5 base shardsshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. (v0.3.0 train):

  • 69.4% of streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.-bearing rows labelcomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag. an affix surface monolithically (30.8% start with a directional, 64.5% end with a common suffix — South County Road 175 West is all B/I-street).
  • Effective gradientgradientThe direction and rate at which the loss would change if each parameter were nudged. Training follows the gradient downhill to reduce error. Huge gradients are tamed by gradient clipping. mass: ~467M contradictory examples vs 90K × 5.0 = 450K shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row.-weighted split examples. At least 1,039:1, flooring base source-weightsparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. at 1.0.
  • The loader amplifies it: augment.py::_expand_token labelcomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag. inheritance plus augment_directional_prob: 0.3 mints fresh monolithic variants every epoch.

One mechanism explains every observation in the ladder: the 64.9 equilibrium is the mixing ratio (architecture-independent by construction); the transient decay is early shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. exposure being reabsorbed; P=100 because the base never splits, so the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' never falsely splits; the frozen headattention headOne of several parallel attention computations in a layer, each free to focus on a different kind of relationship between tokens. Their outputs are concatenated — 'multi-head attention'. Mailwoman uses 4 heads.'s R≈25 because contradictory supervision collapsed the representationhidden stateThe model's internal vector for a token after the encoder has mixed in surrounding context. The contextualized representation the classifier head reads to assign a label. distinction the headattention headOne of several parallel attention computations in a layer, each free to focus on a different kind of relationship between tokens. Their outputs are concatenated — 'multi-head attention'. Mailwoman uses 4 heads. would have needed.

Disposition

Fix is a corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus. lever, filed as #511: a deterministic loader-level relabel pass (leading directional → street_prefix, trailing USPS C1 suffix from @mailwoman/codexstreet_suffix, after augmentation, lineage-stamped). Pre-registered expectations and design points live on the issue. Any retrain needs operator GO per #492's standing rule.

Method note for future audits: the audit script must mirror score-affix's ship-config construction exactly (tokenizertokenizerThe component that converts a raw address string into a sequence of numeric token IDs the model can process. Mailwoman's tokenizer is a SentencePiece unigram model trained specifically on postal addresses. file + anchor lookup + gazetteergazetteerA geographical index that maps place names and postcodes to real-world coordinates. Mailwoman uses a custom-built Who's On First (WOF) SQLite database as its gazetteer — the 'atlas' half of the grammar/atlas architecture. lexicon + suppressGazetteerNearPostcode). Its first run graded the default symlink with zero-filled channels — all-"(nothing)" output is that crash's signature, and rates above 100% mean double-counted rows.