Operations

Audience

🧪 Operator documentation. For contributors running corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus. builds, trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input. experiments, and releases. If you want to use Mailwoman, see Getting started.

Working norms

You are an autonomous agent. Optimize for legibility to a human checking in once a day, not for clever one-shot completions.

Pacing

• Work in small, complete unitsunitA subdivision of a building — apartment, suite, floor — that refines a street address. Mailwoman's unit component; a designator plus identifier forms a subpremise.. A "unitunitA subdivision of a building — apartment, suite, floor — that refines a street address. Mailwoman's unit component; a designator plus identifier forms a subpremise." is anything that ends with a green test suite or a working command.
• Commit at every unitunitA subdivision of a building — apartment, suite, floor — that refines a street address. Mailwoman's unit component; a designator plus identifier forms a subpremise. boundary. Better too many commits than too few.
• Do not start Phase N+1 until Phase N's success criteria checklist is fully green and committed.

Decision discipline

When you hit a fork in the road:

1. If the plan answers it: follow the plan.
2. If the plan is silent and the decision is reversible (file naming, test structure, internal variable names): pick the most conventional option and move on.
3. If the plan is silent and the decision is hard to reverse (public API surface, schema change, npm package name, license commitment, data format): write the decision and the alternatives to DECISIONS.md and pick the most conservative option (the one that preserves the most future options).
4. Never block waiting for input. If a decision feels blockingblockingThe first stage of entity resolution: generate candidate record pairs using cheap, high-recall keys (geo cell, canonical address, phone) instead of comparing every record to every other (O(n²)). The matcher only scores pairs that survive blocking., write it to DECISIONS.md under a ## BLOCKED heading and continue with adjacent work.

Commit messages

Format:

<phase>: <terse imperative>

<optional body: rationale, what changed, what didn't>

Refs: <plan file paths if relevant>

Example:

phase-0: wrap legacy classifiers in ClassificationProposal adapter

All existing rule classifiers now route through wrapLegacyClassifier.
Output shape is identical to pre-refactor for SDK consumers.
Solver code untouched.

Refs: reference/INTERFACES.md, phases/PHASE_0_foundation.md §3

Branching

• Work on a branch named neural/phase-N-<slug> per phasephaseA milestone in the implementation plan (Foundation, Corpus, Training, Integration, and forward-looking phases). Distinct from stage (runtime pipeline) and tier (model vocabulary)..
• Squash-merge to main at phasephaseA milestone in the implementation plan (Foundation, Corpus, Training, Integration, and forward-looking phases). Distinct from stage (runtime pipeline) and tier (model vocabulary). completion.
• Tag main at each phasephaseA milestone in the implementation plan (Foundation, Corpus, Training, Integration, and forward-looking phases). Distinct from stage (runtime pipeline) and tier (model vocabulary). completion: neural-phase-N-complete.

Progress reporting

LOG.md

Append-only, one line per meaningful event. Format:

YYYY-MM-DD HH:MM | phase-N | <what was done> | <next>

Examples:

2026-05-16 14:30 | phase-0 | added ComponentTag union, BIO_LABELS derivation, 47 unit tests passing | wrap rule classifiers
2026-05-16 16:05 | phase-0 | wrapped 12 rule classifiers via adapter, zero behavior change | ClassifierPolicy registry
2026-05-17 09:12 | phase-1 | OSM PBF adapter streams 100k rows in 38s, memory steady at 220MB | WOF admin adapter

Keep it terse. The radio-console format.

DECISIONS.md

Decisions that affect future code. One entry per decision. Format:

## YYYY-MM-DD — <decision title>

**Context:** what was being done

**Options considered:**
1. <option> — pros, cons
2. <option> — pros, cons

**Chosen:** <option>

**Rationale:** <one paragraph>

**Reversibility:** <reversible | costly | irreversible>

BLOCKERS.md

Only created if you have a genuine blocker. One entry per blocker.

## YYYY-MM-DD — <blocker title>

**What's blocked:** <task>
**Why:** <reason>
**What you tried:** <attempts>
**What would unblock:** <answer needed / resource needed / decision needed>

Always continue with adjacent work while a blocker is open. Do not idle.

Code quality

Tests

• UnitunitA subdivision of a building — apartment, suite, floor — that refines a street address. Mailwoman's unit component; a designator plus identifier forms a subpremise. tests for every classifier (rule and neural alike).
• Integration tests for every adapter against a small fixture in corpus/fixtures/.
• Golden-set tests for the full pipelinestaged pipelineMailwoman's runtime architecture: a sequence of pure-function stages (normalize → query-shape → locale-gate → kind-classifier → phrase-grouper → classifier → decoder) connected by typed handoffs. Each stage is published as its own npm package.. Regression here blocks merge.
• Use Vitest (already in Mailwoman).

Types

• Strict TypeScript. strict: true in every package.
• No any. Use unknown and narrow.
• Public API exports must have explicit type annotations.

Linting and formatting

• ESLint (already in Mailwoman). Run on every commit.
• Prettier (already in Mailwoman). Run on every commit.
• Add a pre-commit hook via Husky (already present) if not already enforced.

Performance budgets

Path	Budget
Rule-only classify, single address	< 5ms p95
Neural classify, single address, CPU	< 20ms p95
Neural classify, single address, GPU	< 5ms p95
Cold modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' load	< 2s
TokenizertokenizerThe component that converts a raw address string into a sequence of numeric token IDs the model can process. Mailwoman's tokenizer is a SentencePiece unigram model trained specifically on postal addresses. parity check, 10k addresses	< 30s

If a change blows a budget, profile before optimizing. Add a microbenchmark to bench/ so the regression doesn't return.

Dependencies

Adding a dependency requires:

1. Justification in the commit message (why this, why not X, why not a few lines of code).
2. License check — must be MIT, Apache-2.0, BSD, or ISC. AGPL-compatible licenses fine for the package itself (Mailwoman is AGPL) but verify case-by-case.
3. Size check — flag in commit if it adds > 1MB to bundled output.
4. Stewardship check — verify the dep is actively maintained (commit in last 12 months, > 1 maintainer or > 100 stars).

Approved dependencies for this project:

• onnxruntime-node — ONNXONNX (Open Neural Network Exchange). An open format for machine learning models that enables interoperability between training frameworks and inference runtimes. Mailwoman ships its trained model as an ONNX file so it can run in Node.js and the browser via onnxruntime. inferenceinferenceRunning the trained model on new input to get predictions, as opposed to training, which produces the model. In Mailwoman that means a small transformer encoder reads an address string and classifies every token — house number, street, locality, region, postcode, and the rest. A Who's On First gazetteer can feed soft location hints into the pass, but the model makes the final call on every label. Where a generative model writes text token by token, Mailwoman's output is a retrieval-augmented token classification: one label per input piece.
• @bloomberg/sentencepiece-wasm (or equivalent) — SentencePieceSentencePieceA language-independent subword tokenizer that splits text into pieces using a unigram language model. Mailwoman uses a SentencePiece tokenizer with a 48,000-token vocabulary and byte-fallback, trained on address data rather than general text. tokenization in JS. Verify and pick.
• @dsnp/parquetjs — ParquetParquetThe open columnar file format the corpus is written and streamed in. The training pipeline reads shards row-by-row from Parquet. read/write in TS
• osm-pbf-parser-node (or equivalent) — OSMOpenStreetMap (OSM). A community-curated global map database (ODbL-licensed) with addr:* tagged features and place hierarchies. A secondary corpus source and a source of street names. PBF streaming. Verify and pick.
• better-sqlite3 — for WOFWOF (Who's On First). An open-source gazetteer of places maintained by Mapzen/whosonfirst. Mailwoman builds a custom SQLite database from WOF GeoJSON repos, extended with postcode data, importance scores, and coincident-role relations. SQLite reads
• fast-csv — CSV parsingaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates. for government data
• fastest-levenshtein — alignmentalignmentThe step in the corpus pipeline that takes a (raw, components) pair from an adapter and produces a (raw, tokens, BIO labels) row by finding each component's text inside the raw string and labeling the matching tokens. fuzzy matching
• Existing Mailwoman deps — all retained

Data handling

Storage layout in the home lab

/data/
  corpus/
    sources/           # raw downloads, never modified
      osm/
      wof/
      ban/
      openaddresses/
      usgov/
    intermediate/      # adapter outputs before alignment
    aligned/           # post-alignment Parquet shards
    versioned/         # frozen corpus versions, e.g. corpus-v0.1.0/
  models/
    checkpoints/       # PyTorch training checkpoints
    onnx/              # exported ONNX models
    quantized/         # int8 quantized models
  eval/
    golden/            # hand-labeled golden set, locked, versioned
    splits/            # train/val/test split manifests

Versioning

• CorpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus. versions: corpus-v<major>.<minor>.<patch>. Major = schema change. Minor = source added. Patch = synthesis tweak.
• Modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' versions: model-v<major>.<minor>.<patch>-<locale>. Same scheme.
• Both versions appear in ModelCard.trainedOn and LabeledRow.corpus_version so any prediction is traceable to its trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input. data.
• Package / release versions (the published mailwoman npm package + the live demo) are a SEPARATE, independent line — the user-facing product version (currently 4.4.0). It does not share a counter with the corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus. or modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' versions: the package marched 4.0 → 4.4 through the parity campaignparity campaignThe systematic effort to close the gap between Mailwoman's neural parser and the legacy rule-based v0 parser on every address component tag. Tracked through per-tag parity tables and gated by honest-eval probes. while the corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus. stayed on its own corpus-v0.x line. A release pins which corpus-v<…> and model-v<…> it trained on (via ModelCard.trainedOn), but its own number belongs to the product, not the data.
• Don't read across the lines. Seeing corpus-v0.5.0 next to package 4.4.0 and inferring a regression is the common mistake — they track different things (the data the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' learns from vs. the product you install), like a database schema version vs. an app version. The build that produces corpus-v0.5.0 ships, after trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input., as the next package release (4.5.0 / 5.0.0), continuing the v4 line.

What never goes in git

• Raw source data (too large, redistribution risk for some sources)
• Trained modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' checkpointscheckpointA saved snapshot of the model weights and optimizer state during training. Mailwoman saves a checkpoint periodically so training can resume after a GPU hang. over 100MB (use Git LFS or external storage)
• Anything containing PII

What goes in git:

• All code
• CorpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus. manifests (file lists + checksums, not the files)
• EvalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. splits (lists of source IDs in each split)
• The hand-labeled golden setgolden setA hand-labeled evaluation dataset of US, French, and adversarial addresses used as ground truth for evals. (small, human-verified)
• Adapter fixtures (small, hand-crafted, license-clean)

Communication with the human

The human checks in periodically. They will read:

1. LOG.md first
2. DECISIONS.md if LOG.md references a decision
3. BLOCKERS.md if you have one open
4. The actual code only if something looks off in the logs

Write logs assuming the reader hasn't seen the code yet. "Wrapped 12 rule classifiers" is fine. "Refactored the thing" is not.

When the human asks a question, answer in the radio-console format (see user preferences they've set). Lead with →. No greetings.

When you finish a phase

1. Run the full test suite. Green.
2. Run benchmarks. Within budget.
3. Update LOG.md with phasephaseA milestone in the implementation plan (Foundation, Corpus, Training, Integration, and forward-looking phases). Distinct from stage (runtime pipeline) and tier (model vocabulary).-complete entry.
4. Squash-merge the phasephaseA milestone in the implementation plan (Foundation, Corpus, Training, Integration, and forward-looking phases). Distinct from stage (runtime pipeline) and tier (model vocabulary). branch to main.
5. Tag main: git tag neural-phase-N-complete && git push --tags.
6. Open phases/PHASE_<N+1>_*.md and begin.

When you finish all phases

Do not "polish" indefinitely. Write a final entry in LOG.md saying phasephaseA milestone in the implementation plan (Foundation, Corpus, Training, Integration, and forward-looking phases). Distinct from stage (runtime pipeline) and tier (model vocabulary). 6 is complete, run one final test pass, push, and stop. The human will pick it up from there.