Skip to main content

Report a parsing bug

Outcome. You have a report that can be turned into a failing test without a follow-up question, and you have already ruled out the two configurations that produce a wrong-looking result on purpose.

Prerequisites​

  • The exact input string that broke, copied rather than retyped. Whitespace and punctuation are the bug about half the time, and retyping normalizesnormalizeStage 1 of the runtime pipeline: deterministic input preprocessing (Unicode NFC, punctuation normalization, whitespace collapse). Returns a NormalizedInput with an offsetMap that maps normalized positions back to the raw input. them away.

1. Rule out the two false alarms​

Both of these look like parser bugs and are not. Checking takes one command each.

A missing coordinate on a parseaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates.-only install. mailwoman parse never resolves anything; it returns components. If you expected a latitude/longitude, you wanted mailwoman geocode and a gazetteergazetteerA geographical index that maps place names and postcodes to real-world coordinates. Mailwoman uses a custom-built Who's On First (WOF) SQLite database as its gazetteer — the 'atlas' half of the grammar/atlas architecture..

A coordinate in the wrong countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. from an ambiguous input. A bare postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one. matches several countries' formats, and the resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon. picks by rank. mailwoman geocode --default-country US "10001" and mailwoman geocode "10001" disagree, and neither is a parser failure. Handle PO boxes and other edge kinds has the measurements.

2. Capture the pipeline trace​

--debug is the flag. There is no --trace.

mailwoman parse --debug "1275 Pennsylvania Ave NW, Washington, DC 20004"

It emits one JSON object with eight keys: input, normalized, locale, kind, path, queryShape, timing and tree. The whole object is worth attaching, but four fields answer most first questions, so lead with them:

mailwoman parse --debug "$ADDRESS" | jq '{input, path, kind: {kind: .kind.kind, confidence: .kind.confidence}, timing}'
{
  "input": "1275 Pennsylvania Ave NW, Washington, DC 20004",
  "path": "full",
  "kind": {
    "kind": "structured_address",
    "confidence": 0.9
  },
  "timing": {
    "normalize": 0.405547999999726,
    "place-country": 1.315497999999934,
    "query-shape": 0.7962270000002718,
    "locale-gate": 0.16185200000018085,
    "kind-classifier": 0.5810759999999391,
    "phrase-grouper": 1.3690379999998186,
    "token-classify": 230.3101270000002,
    "grouper-audit": 0.27730899999960457
  }
}

path and kind are the two that explain the result before anyone reads the tree. path: "fast-path" means the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' never ran — the input was classified confidently enough to synthesize a node directly, so a wrong answer there is a classifier bug rather than a modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' one. A surprising kind explains a surprising set of components.

The normalized block matters when the input carries unusual characters: it lists the transforms that were applied before the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' saw anything, so it distinguishes "the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' read it wrong" from "the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' never saw what you typed".

3. Capture the environment​

mailwoman doctor --json
{
  "checks": [
    {
      "id": "weights",
      "label": "Model weights (en-us)",
      "core": true,
      "status": "ok",
      "detail": "package:@mailwoman/neural-weights-en-us · model.onnx 39.4 MB, tokenizer.model 1.6 MB"
    },
    {
      "id": "node-version",
      "label": "Node runtime",
      "core": true,
      "status": "ok",
      "detail": "node v26.2.0 (engines: >=24.18.0)"
    },
    {
      "id": "onnxruntime",
      "label": "ONNX runtime",
      "core": true,
      "status": "ok",
      "detail": "onnxruntime-node loadable"
    },
    {
      "id": "data-root",
      "label": "Data root",
      "core": false,
      "status": "ok",
      "detail": "/tmp/mw-fresh-demo ($MAILWOMAN_DATA_ROOT) — exists, writable"
    }
  ]
}

That is most of a reproduction: which modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' version, which Node build, which data layerslayerOne transformer block — attention plus a feed-forward network, with normalization and residual connections — applied to every position. Stacking layers lets the model build up richer representations; Mailwoman's encoder has 6. are present. The weightsparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. line in particular pins the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.', and parsesaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates. are not comparable across modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' versions.

The demo runs the same parser and resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon. in a browser tab against the published modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.', and it reads the address out of the query string. /demo?q=1275+Pennsylvania+Ave+NW,+Washington+DC opens with your input already in the box.

Paste that link into the issue. It costs you nothing and it removes the install step on the other side. If the demo parsesaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates. it correctly and your install does not, that difference is itself the finding — the demo pins a published modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' version, so you have found a regression or a configuration gap.

Verify​

Before you file, confirm the report has all four piecesECE (Expected Calibration Error). A metric that measures how well a model's confidence scores align with its actual accuracy. Lower is better. Mailwoman's held-out ECE drops from 0.067 (raw) to 0.0035 (calibrated).:

{
  echo "## Input";      printf '%s\n' "$ADDRESS"
  echo "## Parse";      mailwoman parse --debug "$ADDRESS"
  echo "## Doctor";     mailwoman doctor --json
} > mailwoman-bug-report.md

Then add the two things no command can produce: what you expected, and what you got.

Where to file​

github.com/sister-software/mailwoman/issues. Support has the full list of what makes a report actionable, plus the separate route for a security issue — which is email, never a public issue.

Send the address that broke rather than a description of the class of address that broke. A concrete string becomes a test case; a description becomes a conversation.

Limits​

  • A wrong coordinate may be a data gap rather than a parser bug, and the two get fixed by different people. If the components are right and the coordinate is wrong, say so in the title — that routes it to the gazetteergazetteerA geographical index that maps place names and postcodes to real-world coordinates. Mailwoman uses a custom-built Who's On First (WOF) SQLite database as its gazetteer — the 'atlas' half of the grammar/atlas architecture. rather than the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.'.
  • --debug prints the input back. If the address is one you cannot paste into a public issue, say so and a private route will be arranged; do not sanitize the string, because the sanitized version tends to parseaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates. fine.
  • LocalelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for. matters and defaults to en-US. A French address parsed with the en-US modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' is expected to be worse. Name the localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for. you passed, or that you passed none.