Skip to main content

#727 stage-2 Phase 2 — export verdict: all three gates PASS

Gates (pre-registered in 2026-07-15-727-stage2-phase2-export.md):

  1. ONNXONNX (Open Neural Network Exchange). An open format for machine learning models that enables interoperability between training frameworks and inference runtimes. Mailwoman ships its trained model as an ONNX file so it can run in Node.js and the browser via onnxruntime. span_scores ≈ torch (atol 1e-3 fp32fp32 / fp1632-bit and 16-bit floating-point formats. Mailwoman trains in bf16 (a 16-bit variant) and exports the ONNX model in int8 for size.) on random inputs — parity or no ship.
  2. int8 size delta vs v264's 39.8 MB within +1 MB.
  3. Fetching ONLY logits from the spanspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree.-enabled graph costs ≤5% latency vs the v264 graph.

Results​

gatebarmeasuredverdict
1. ONNXONNX (Open Neural Network Exchange). An open format for machine learning models that enables interoperability between training frameworks and inference runtimes. Mailwoman ships its trained model as an ONNX file so it can run in Node.js and the browser via onnxruntime.↔torch parity≤ 1e-3worst 7.3e-06 over 5 real addresses (fp32fp32 / fp1632-bit and 16-bit floating-point formats. Mailwoman trains in bf16 (a 16-bit variant) and exports the ONNX model in int8 for size.)PASS
2. int8 size≤ +1 MB40,065,060 vs 39,838,216 = +0.22 MB (+0.57%)PASS
3. logitslogitA raw, unnormalized per-label score the model outputs before softmax. Priors and biases are added in logit space, then softmax turns logits into probabilities.-only latency≤ +5%−13.9% (3.51 ms vs 4.08 ms, CPU ORT, S=128, 200 iters)PASS

Honesty note on gate 3: the negative delta is NOT the spanspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree. headattention headOne of several parallel attention computations in a layer, each free to focus on a different kind of relationship between tokens. Their outputs are concatenated — 'multi-head attention'. Mailwoman uses 4 heads. making inferenceinferenceRunning the trained model on new input to get predictions, as opposed to training, which produces the model. In Mailwoman that means a small transformer encoder reads an address string and classifies every token — house number, street, locality, region, postcode, and the rest. A Who's On First gazetteer can feed soft location hints into the pass, but the model makes the final call on every label. Where a generative model writes text token by token, Mailwoman's output is a retrieval-augmented token classification: one label per input piece. faster — the two graphs come from different export runs, and toolchain/graph-optimization drift plausibly explains a 14% gap. The defensible claim is only the gate's own: a runtime that never fetches span_scores pays nothing measurable. ORT prunes the unfetched branch, as designed.

The Phase-3 decode budget: fetching logits+span_scores costs +0.75 ms over logitslogitA raw, unnormalized per-label score the model outputs before softmax. Priors and biases are added in logit space, then softmax turns logits into probabilities.-only (CPU, S=128). That is the input-side cost the JS k-best decoderdecoderIn a transformer encoder-decoder model, the part that produces output sequences. Mailwoman's classifier is encoder-only (no decoder); the 'CRF decoder' is a different thing — a structured-prediction layer that picks the best label sequence from the encoder's outputs. inherits; the decode itself is JS-side on ~100–200 pruned spansspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree. (consult: "decode ~free").

Artifacts​

fp32fp32 / fp1632-bit and 16-bit floating-point formats. Mailwoman trains in bf16 (a 16-bit variant) and exports the ONNX model in int8 for size.output-v301-span-head-lr-s42/model.onnx (volume) — outputs [logits, locale_logits, span_scores], full production input set (anchor/gaz/countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.)
int8models/quantized/model-v301-step-2000-int8.onnx (volume) — outputs verified on the artifact, not assumed from the sync
sidecarsemi-crf-transitions.json — {segment_types[17], max_span: 8, transitions[17][17], start, end}; the axis ships IN the file (PLACETYPE_ORDER class)

What changed in code​

  • export_to_onnx: the five wrappers gain a with_spans toggle mirroring with_locale; output_names built from both flags. SpanspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree.-less export byte-compatible (tested: ["logits"]).
  • package_weights.export_semi_crf_transitions() — the JSON sidecar, None for spanspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree.-less modelsneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.', following the export_crf_transitions precedent (transitions are decode-time data, not graph).
  • sync_src_727_stage2 verify extended to 10 checks (export + sidecar presence) — a stale container-side export would silently produce a graph without span_scores.
  • onnxscript==0.7.0 added to the [train] extra (the dynamo exporter's dep; pinned to the ModalModalA cloud GPU platform (modal.com) where Mailwoman trains its neural models on NVIDIA A100 GPUs. Training runs are launched via scripts/modal/train_remote.py and typically complete in ~1 hour. image's version). Collateral: uv re-resolve bumped trackio 0.27.0→0.30.3.

What Phase 3 consumes​

  1. The int8 graph's span_scores output — shape (B, S, 8, 17), fetched by name.
  2. semi-crf-transitions.json — decode-time transition grammar, axis read from the file.
  3. The stagestageOne of the dataflow stages in the runtime pipeline (normalize, locale gate, kind classify, phrase group, token classify, sequence correct, reconcile, resolve). Distinct from tier (model vocabulary) and phase (plan milestone).-1 boundary headattention headOne of several parallel attention computations in a layer, each free to focus on a different kind of relationship between tokens. Their outputs are concatenated — 'multi-head attention'. Mailwoman uses 4 heads. is NOT exported (trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input.-only, unchanged) — PhasephaseA milestone in the implementation plan (Foundation, Corpus, Training, Integration, and forward-looking phases). Distinct from stage (runtime pipeline) and tier (model vocabulary). 3's spanspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree. pruning uses the spanspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree. scores themselves (top-K per position) rather than the aux headattention headOne of several parallel attention computations in a layer, each free to focus on a different kind of relationship between tokens. Their outputs are concatenated — 'multi-head attention'. Mailwoman uses 4 heads.'s probs; revisit only if the candidate set blows past ~200.

PhasephaseA milestone in the implementation plan (Foundation, Corpus, Training, Integration, and forward-looking phases). Distinct from stage (runtime pipeline) and tier (model vocabulary). 3 (JS/WASM k-best in neural/ + neural-web/) is unblocked. The decoderdecoderIn a transformer encoder-decoder model, the part that produces output sequences. Mailwoman's classifier is encoder-only (no decoder); the 'CRF decoder' is a different thing — a structured-prediction layer that picks the best label sequence from the encoder's outputs.'s reference implementation is mailwoman/eval-harness/oracle-k.ts (same recurrence, learned scores replacing the summed-BIO stand-in) and the Python SemiMarkovCRF.decode (brute-force verified).