FST Gazetteer LM

Shipped

PhasesphaseA milestone in the implementation plan (Foundation, Corpus, Training, Integration, and forward-looking phases). Distinct from stage (runtime pipeline) and tier (model vocabulary). 1-2 shipped in v0.5.2 (#170, #173). Wikipedia importanceWikipedia importanceA place-notability score derived from the count and language spread of a place's Wikipedia articles, normalized to [0, 1]. A resolver ranking prior that helps pick the prominent same-named place. integration in #173. Unified SQLite builder in #176. PhasephaseA milestone in the implementation plan (Foundation, Corpus, Training, Integration, and forward-looking phases). Distinct from stage (runtime pipeline) and tier (model vocabulary). 3 (autocomplete) partially shipped. PhasephaseA milestone in the implementation plan (Foundation, Corpus, Training, Integration, and forward-looking phases). Distinct from stage (runtime pipeline) and tier (model vocabulary). 4 (browser) shipped — the /demo page loads a 9 MB FSTFST (finite-state transducer). A compact automaton that reads an input sequence and emits an output sequence. Mailwoman encodes gazetteer names and street affixes as FSTs for fast prefix matching and prior injection without search overhead. binary and passes it as an emission prioremission priorA log-probability bias injected into the model's emission logits from an external signal — gazetteer frequency, an FST, a Wikipedia-importance score — combined with the learned weights at decode time. to the neural classifierneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.'.

Goal: Pre-compute a finite-state transducerFST (finite-state transducer). A compact automaton that reads an input sequence and emits an output sequence. Mailwoman encodes gazetteer names and street affixes as FSTs for fast prefix matching and prior injection without search overhead. from the WOFWOF (Who's On First). An open-source gazetteer of places maintained by Mapzen/whosonfirst. Mailwoman builds a custom SQLite database from WOF GeoJSON repos, extended with postcode data, importance scores, and coincident-role relations. SQLite gazetteergazetteerA geographical index that maps place names and postcodes to real-world coordinates. Mailwoman uses a custom-built Who's On First (WOF) SQLite database as its gazetteer — the 'atlas' half of the grammar/atlas architecture. that maps tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. sequences → (placetype, wof_id, parent_chain, importance) entries. Use it as an emission prior in the neural Viterbi decoder, as a CLI introspection tool, and as the autocomplete backend.

Principle: Pay down the combinatorial cross-product of "all valid place-name paths through the WOFWOF (Who's On First). An open-source gazetteer of places maintained by Mapzen/whosonfirst. Mailwoman builds a custom SQLite database from WOF GeoJSON repos, extended with postcode data, importance scores, and coincident-role relations. hierarchy" at build time. At query time, walking the FSTFST (finite-state transducer). A compact automaton that reads an input sequence and emits an output sequence. Mailwoman encodes gazetteer names and street affixes as FSTs for fast prefix matching and prior injection without search overhead. is O(depth), not O(gazetteer_size).

Shipped metrics (US admin)

Metric	Value
FSTFST (finite-state transducer). A compact automaton that reads an input sequence and emits an output sequence. Mailwoman encodes gazetteer names and street affixes as FSTs for fast prefix matching and prior injection without search overhead. statesregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.	114,214
Name insertions	163,271
Binary size	5.57 MB
Load time	~10 ms
Build time (from unified SQLite)	2.7 s
Build time (unified SQLite from 293K GeoJSON)	43 s
Wikipedia importanceWikipedia importanceA place-notability score derived from the count and language spread of a place's Wikipedia articles, normalized to [0, 1]. A resolver ranking prior that helps pick the prominent same-named place. matches	47,348 places
Population fallback	108,111 places

---

Architecture overview

FST data structure

FSTState {
  id: u32                              // dense index into states[]
  edges: Vec<FSTEdge>                  // 2-5 entries avg
  places: Vec<PlaceEntry>              // accepting states only, 1-5 entries avg
}

FSTEdge {
  token: string                        // interned, lowercase, NFKC-normalized
  target: u32                          // target state id
}

PlaceEntry {
  wof_id: u32                          // WOF numeric ID
  placetype: u8                        // PlacetypeId enum
  parent_chain: Vec<u32>               // [country_id, region_id, county_id?], leaf-to-root
  population: u32
  lat: f32, lon: f32
}

Tokenization is whitespace-split with punctuation stripping. The FSTFST (finite-state transducer). A compact automaton that reads an input sequence and emits an output sequence. Mailwoman encodes gazetteer names and street affixes as FSTs for fast prefix matching and prior injection without search overhead. edge labelcomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag. is the normalized tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words.. Accepting statesregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. carry ALL valid interpretations — "New York" ends at a stateregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. with entries for both NYC (localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy.) and NY stateregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. (regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.). The FSTFST (finite-state transducer). A compact automaton that reads an input sequence and emits an output sequence. Mailwoman encodes gazetteer names and street affixes as FSTs for fast prefix matching and prior injection without search overhead. never picks; the neural modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' + reconciler do.

Build pipeline

WOF SQLite (spr + names tables)
         │
         ▼
  fst-builder.ts  ← resolver-wof-sqlite/fst-builder.ts
         │
         ├─ Normalize names (NFKC, lowercase, strip punctuation)
         ├─ Tokenize each name + alt_names into token sequences
         ├─ Insert into incremental trie (shared prefixes)
         ├─ At each terminal: attach PlaceEntry
         ├─ Determinize + Minimize (Hopcroft)
         │
         ▼
  fst.bin  ← compact binary (~8-12 MB for US)
         │
         ▼
  FSTMatcher class (fast prefix walk)

Integration with the neural pipeline

The FSTFST (finite-state transducer). A compact automaton that reads an input sequence and emits an output sequence. Mailwoman encodes gazetteer names and street affixes as FSTs for fast prefix matching and prior injection without search overhead. provides per-tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. emission biasesadditive biasA logit adjustment added before softmax from a prior or external knowledge source, typically soft-capped so it influences but never overrides the model's prediction. The mechanism behind shallow fusion., composing additively with the existing QueryShape soft priorsoft priorOutside knowledge fed to the model or resolver as overridable evidence (a feature, a score term) rather than a hard filter. A focus-country hint becomes an anchor feature; a focus-point becomes a ranking term.:

let emissions = logits
if (opts?.queryShape) {
  emissions = addEmissionMatrix(emissions, buildEmissionPriors(...))
}
if (opts?.fst) {
  emissions = addEmissionMatrix(emissions, buildFSTEmissionPriors(...))
}
// → Viterbi decode over biased emissions

The FSTFST (finite-state transducer). A compact automaton that reads an input sequence and emits an output sequence. Mailwoman encodes gazetteer names and street affixes as FSTs for fast prefix matching and prior injection without search overhead. prior gets STRONGER as the prefix grows longer. A bare "S" spreads bias across many places; "San Fran" concentrates it on San Francisco.

WFST analogy to speech recognition

Speech recognition	Address parsingaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates.
Acoustic modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' (DNN)	Neural classifierneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' (transformerneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.')
Pronunciation lexicon (L)	FSTFST (finite-state transducer). A compact automaton that reads an input sequence and emits an output sequence. Mailwoman encodes gazetteer names and street affixes as FSTs for fast prefix matching and prior injection without search overhead. (tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. sequences → place entries)
Language modellanguage model (LM). A model that assigns probabilities to sequences of tokens. Used here mostly as a prior — an FST or n-gram model that biases the decoder toward plausible sequences via shallow fusion. (G)	Address grammar (valid component sequences)
Shallow fusionshallow fusionBlending an external knowledge source (a gazetteer, an FST prior) into a model's decision as a signal, rather than retraining the model or overriding its output. Mailwoman applies it at the input layer (the gazetteer anchor as membership features) and at decode time (an FST prior biasing emissions). RAG-shaped, but the retrieval lands as feature vectors, not prompt text. at decode time	Additive emission biasesadditive biasA logit adjustment added before softmax from a prior or external knowledge source, typically soft-capped so it influences but never overrides the model's prediction. The mechanism behind shallow fusion. at ViterbiViterbi decodingA dynamic programming algorithm that finds the most likely sequence of hidden states (labels) given a sequence of observations (token emissions). Mailwoman uses Viterbi over a linear-chain CRF to produce globally coherent BIO label sequences from per-token model scores. time

The neural modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' handles non-gazetteergazetteerA geographical index that maps place names and postcodes to real-world coordinates. Mailwoman uses a custom-built Who's On First (WOF) SQLite database as its gazetteer — the 'atlas' half of the grammar/atlas architecture. components (streetsstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels., venuesvenueA named, non-address place — a business, building, park, or stadium. Mailwoman's free-text point-of-interest component, added as a Tier 2 fine label., house numbershouse numberThe numeric or alphanumeric identifier of a building on a street. Mailwoman's house_number component; its position relative to the street name flips between locales., typos). The FSTFST (finite-state transducer). A compact automaton that reads an input sequence and emits an output sequence. Mailwoman encodes gazetteer names and street affixes as FSTs for fast prefix matching and prior injection without search overhead. handles gazetteergazetteerA geographical index that maps place names and postcodes to real-world coordinates. Mailwoman uses a custom-built Who's On First (WOF) SQLite database as its gazetteer — the 'atlas' half of the grammar/atlas architecture. components (countries, regionsregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality., localities). They compose via shallow fusionshallow fusionBlending an external knowledge source (a gazetteer, an FST prior) into a model's decision as a signal, rather than retraining the model or overriding its output. Mailwoman applies it at the input layer (the gazetteer anchor as membership features) and at decode time (an FST prior biasing emissions). RAG-shaped, but the retrieval lands as feature vectors, not prompt text. — same as modern ASR systemsexpectation-maximizationAn iterative algorithm that estimates model parameters when some variables are unobserved. In Mailwoman's matcher, EM learns the Fellegi-Sunter m and u parameters from unlabeled data — no training labels needed..

---

CLI proof-of-concept

mailwoman fst build

mailwoman fst build --db /path/to/wof-admin-us.db --output fst-en-US.bin --countries US

mailwoman fst query

mailwoman fst query 'New York' --fst fst-en-US.bin --show-continuations

Example output:

"new york" (complete match)
├── Accepts: 2 interpretations
│   ├── locality New York City    pop 8,804,000  wof:85977539
│   │   chain: US ← New York (state) ← New York (city)
│   └── region   New York State   pop 20,200,000 wof:85688543
│       chain: US ← New York (state)
└── Valid continuations:
    ├── "ny"    → region (NY state, disambiguated)
    ├── "10001" → postalcode
    └── [end]   → valid terminal

Negative evidence example — 'Buffalo Health Clinic, Buffalo':

"buffalo" → 14 places (locality Buffalo, NY is top)
"health"  → NOT IN FST (no gazetteer match)
  → Strong signal: this span is NOT an admin component
  → Fall back to neural model for typing (likely venue)

---

Locale strategy

Per-localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for. FSTsFST (finite-state transducer). A compact automaton that reads an input sequence and emits an output sequence. Mailwoman encodes gazetteer names and street affixes as FSTs for fast prefix matching and prior injection without search overhead. (recommended for v1). One FSTFST (finite-state transducer). A compact automaton that reads an input sequence and emits an output sequence. Mailwoman encodes gazetteer names and street affixes as FSTs for fast prefix matching and prior injection without search overhead. per localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for.: fst-en-US.bin, fst-fr-FR.bin. Filter by countrycountryThe top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head. during build. Names in all languages are included (a user typing "Munich" in an en-US context won't match — and that's correct, Munich isn't in the US).

LocalelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for.	Places	FSTFST (finite-state transducer). A compact automaton that reads an input sequence and emits an output sequence. Mailwoman encodes gazetteer names and street affixes as FSTs for fast prefix matching and prior injection without search overhead. size
en-US	~30K	~8 MB
fr-FR	~36K (communes)	~10 MB
ja-JP	~1,800	~5 MB
en-GB	~20K	~6 MB

Why not combined+filter

A single FSTFST (finite-state transducer). A compact automaton that reads an input sequence and emits an output sequence. Mailwoman encodes gazetteer names and street affixes as FSTs for fast prefix matching and prior injection without search overhead. with per-edge localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for. bitsets is space-efficient but query-slower. For v1, per-localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for. FSTsFST (finite-state transducer). A compact automaton that reads an input sequence and emits an output sequence. Mailwoman encodes gazetteer names and street affixes as FSTs for fast prefix matching and prior injection without search overhead. match the existing per-localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for. weightsparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.'. Revisit if multi-localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for. deployment becomes necessary.

---

Size estimates

Admin only (PhasephaseA milestone in the implementation plan (Foundation, Corpus, Training, Integration, and forward-looking phases). Distinct from stage (runtime pipeline) and tier (model vocabulary). 1): ~8-12 MB for US. Smaller than the current 35 MB WOFWOF (Who's On First). An open-source gazetteer of places maintained by Mapzen/whosonfirst. Mailwoman builds a custom SQLite database from WOF GeoJSON repos, extended with postcode data, importance scores, and coincident-role relations. slim DB but encodes more structural information (parent chainsparent chainThe sequence of administrative parents above a place — Springfield → Sangamon County → Illinois → United States. Used to check the geographic coherence of a parse., population, valid continuations).

With streetsstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. (PhasephaseA milestone in the implementation plan (Foundation, Corpus, Training, Integration, and forward-looking phases). Distinct from stage (runtime pipeline) and tier (model vocabulary). 2+): ~200-500 MB for full US streetsstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. (3-5M unique streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. names). Too large for browser — shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. per metro area (~5 MB each) for browser deployment. Server-side loads the full set.

---

Implementation phases

Phase 1: FST builder + CLI — shipped (#170)

• resolver-wof-sqlite/fst-builder.ts, fst-matcher.ts, fst-types.ts
• resolver-wof-sqlite/fst-serialize.ts (binary format, VERSION 2)
• scripts/fst-query.ts (interactive CLI)
• 24 integration tests against WOFWOF (Who's On First). An open-source gazetteer of places maintained by Mapzen/whosonfirst. Mailwoman builds a custom SQLite database from WOF GeoJSON repos, extended with postcode data, importance scores, and coincident-role relations. US admin data

Phase 2: Neural emission prior — shipped (#170, #173)

• neural/fst-prior.ts — buildFSTEmissionPriors() with Wikipedia importanceWikipedia importanceA place-notability score derived from the count and language spread of a place's Wikipedia articles, normalized to [0, 1]. A resolver ranking prior that helps pick the prominent same-named place. weighting
• neural/classifier.ts — FSTFST (finite-state transducer). A compact automaton that reads an input sequence and emits an output sequence. Mailwoman encodes gazetteer names and street affixes as FSTs for fast prefix matching and prior injection without search overhead. threaded through ParseOpts.fst
• core/pipeline/runtime-pipeline.ts — FSTFST (finite-state transducer). A compact automaton that reads an input sequence and emits an output sequence. Mailwoman encodes gazetteer names and street affixes as FSTs for fast prefix matching and prior injection without search overhead. threaded through RuntimePipelineStages
• Wikipedia importanceWikipedia importanceA place-notability score derived from the count and language spread of a place's Wikipedia articles, normalized to [0, 1]. A resolver ranking prior that helps pick the prominent same-named place. ETL: scripts/build-importance.ts
• RegionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.-aware localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. bias guard: neural/query-shape-prior.ts (#174)

Phase 3: Autocomplete prototype — partially shipped (#170)

• resolver-wof-sqlite/fst-autocomplete.ts — prefix walk + BFS expansion
• CLI not yet wired (standalone script only)

Phase 4: Browser deployment — shipped

• resolver-wof-sqlite/fst-deserialize-web.ts — browser-compatible deserializer using DataView + TextDecoder (no Node Buffer dependency)
• /demo page loads fst-en-US.bin (~9 MB) and passes it to classifier.parse() as opts.fst
• FSTFST (finite-state transducer). A compact automaton that reads an input sequence and emits an output sequence. Mailwoman encodes gazetteer names and street affixes as FSTs for fast prefix matching and prior injection without search overhead. module import, binary fetch, and deserialization happen in parallel with ONNXONNX (Open Neural Network Exchange). An open format for machine learning models that enables interoperability between training frameworks and inference runtimes. Mailwoman ships its trained model as an ONNX file so it can run in Node.js and the browser via onnxruntime. modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' load
• Graceful degradation: if the FSTFST (finite-state transducer). A compact automaton that reads an input sequence and emits an output sequence. Mailwoman encodes gazetteer names and street affixes as FSTs for fast prefix matching and prior injection without search overhead. fails to load, the demo runs without it

---

Key design decisions

1. FSTFST (finite-state transducer). A compact automaton that reads an input sequence and emits an output sequence. Mailwoman encodes gazetteer names and street affixes as FSTs for fast prefix matching and prior injection without search overhead. is an emission PRIORemission priorA log-probability bias injected into the model's emission logits from an external signal — gazetteer frequency, an FST, a Wikipedia-importance score — combined with the learned weights at decode time., not a replacement. The neural modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' remains the authority for non-gazetteergazetteerA geographical index that maps place names and postcodes to real-world coordinates. Mailwoman uses a custom-built Who's On First (WOF) SQLite database as its gazetteer — the 'atlas' half of the grammar/atlas architecture. components. The FSTFST (finite-state transducer). A compact automaton that reads an input sequence and emits an output sequence. Mailwoman encodes gazetteer names and street affixes as FSTs for fast prefix matching and prior injection without search overhead. handles what the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' can't: knowing which place namestoponymA proper name for a geographic place. exist and their hierarchies.

2. Wikipedia importanceWikipedia importanceA place-notability score derived from the count and language spread of a place's Wikipedia articles, normalized to [0, 1]. A resolver ranking prior that helps pick the prominent same-named place.-weighted bias when ambiguous. Each place carries a [0,1] importance scoreimportance scoreA precomputed per-place prominence score (blending Wikipedia importance with population) used to rank same-name gazetteer candidates. derived from Wikipedia link count (Nominatim methodology). "New York" biases both localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. (0.95) and regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. (0.85) proportionally. Washington DC localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. (0.815) correctly outranks Washington stateregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. (0.764) despite lower population. Formula: importance × biasScale × maxBias (linear, capped at 3.0 logitslogitA raw, unnormalized per-label score the model outputs before softmax. Priors and biases are added in logit space, then softmax turns logits into probabilities.).

3. Negative suppression on non-place labelscomponent tagOne of the 33 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag.. When the FSTFST (finite-state transducer). A compact automaton that reads an input sequence and emits an output sequence. Mailwoman encodes gazetteer names and street affixes as FSTs for fast prefix matching and prior injection without search overhead. matches a place nametoponymA proper name for a geographic place., B-streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels., I-streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels., B-house_number, I-house_number, and B-venuevenueA named, non-address place — a business, building, park, or stadium. Mailwoman's free-text point-of-interest component, added as a Tier 2 fine label. receive -1.5 logitlogitA raw, unnormalized per-label score the model outputs before softmax. Priors and biases are added in logit space, then softmax turns logits into probabilities. suppression. This narrows the gap between place-tag and non-place-tag logitslogitA raw, unnormalized per-label score the model outputs before softmax. Priors and biases are added in logit space, then softmax turns logits into probabilities. without overriding the modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.'.

4. Negative evidence is free. When a tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. doesn't extend any FSTFST (finite-state transducer). A compact automaton that reads an input sequence and emits an output sequence. Mailwoman encodes gazetteer names and street affixes as FSTs for fast prefix matching and prior injection without search overhead. path, that's a strong signal it's NOT an admin component — the neural modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' handles it alone. This is how "Buffalo Health Clinic" gets correctly NOT-biased toward localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy..

5. The FSTFST (finite-state transducer). A compact automaton that reads an input sequence and emits an output sequence. Mailwoman encodes gazetteer names and street affixes as FSTs for fast prefix matching and prior injection without search overhead. eliminates reconciler beam searchbeam searchA decoding search that keeps the top-k partial candidates at each position. Used in Stage-5 reconciliation to explore the (phrase, tag, resolver) space with controlled pruning. for admin components. Only concordant paths exist in the FSTFST (finite-state transducer). A compact automaton that reads an input sequence and emits an output sequence. Mailwoman encodes gazetteer names and street affixes as FSTs for fast prefix matching and prior injection without search overhead.. Invalid admin combinations were pruned at build time.

6. Autocomplete is a prefix walk, not an index scan. O(depth × branching) vs Elasticsearch's O(matches). The FSTFST (finite-state transducer). A compact automaton that reads an input sequence and emits an output sequence. Mailwoman encodes gazetteer names and street affixes as FSTs for fast prefix matching and prior injection without search overhead. IS the autocomplete index.

---

Relationship to existing architecture

The FSTFST (finite-state transducer). A compact automaton that reads an input sequence and emits an output sequence. Mailwoman encodes gazetteer names and street affixes as FSTs for fast prefix matching and prior injection without search overhead. is the "pay-it-forward" implementation of the mail-carrier philosophy: each carrier's knowledge is pre-compiled into a structure that narrows the search space with each tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. consumed. The deliberative assembly (phrase grouperphrase grouperStage 2.7 of the runtime pipeline: proposes coherent input units (street phrase, locality phrase, postcode, etc.) with structural kind hypotheses. Decouples boundary discovery from type classification so the classifier answers 'what type?' not 'where?' + neural modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' + reconciler) still adjudicates — the FSTFST (finite-state transducer). A compact automaton that reads an input sequence and emits an output sequence. Mailwoman encodes gazetteer names and street affixes as FSTs for fast prefix matching and prior injection without search overhead. just gives them much better priors to work with.

Where the QueryShape soft priorsoft priorOutside knowledge fed to the model or resolver as overridable evidence (a feature, a score term) rather than a hard filter. A focus-country hint becomes an anchor feature; a focus-point becomes a ranking term. says "this 5-digit tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. is probably a postcodepostcodeThe country-specific postal code (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one." (structural pattern), the FSTFST (finite-state transducer). A compact automaton that reads an input sequence and emits an output sequence. Mailwoman encodes gazetteer names and street affixes as FSTs for fast prefix matching and prior injection without search overhead. prior says "this tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. sequence matches 'New York' which is either localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. WOFWOF (Who's On First). An open-source gazetteer of places maintained by Mapzen/whosonfirst. Mailwoman builds a custom SQLite database from WOF GeoJSON repos, extended with postcode data, importance scores, and coincident-role relations.:85977539 or regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. WOFWOF (Who's On First). An open-source gazetteer of places maintained by Mapzen/whosonfirst. Mailwoman builds a custom SQLite database from WOF GeoJSON repos, extended with postcode data, importance scores, and coincident-role relations.:85688543" (factual knowledge from the gazetteergazetteerA geographical index that maps place names and postcodes to real-world coordinates. Mailwoman uses a custom-built Who's On First (WOF) SQLite database as its gazetteer — the 'atlas' half of the grammar/atlas architecture.). Both are additive biasesadditive biasA logit adjustment added before softmax from a prior or external knowledge source, typically soft-capped so it influences but never overrides the model's prediction. The mechanism behind shallow fusion.; neither overrides the neural modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.'.

See also

• QUERY_SHAPE.md — the existing structural-prior system this extends
• DEMO_PRESET_DIAGNOSIS.md — the localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy./regionregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. confusion this addresses
• TRAINING_RECIPE_LEVERS.md — trainingtrainingThe process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input.-side fixes (complementary)
• docs/articles/understanding/exotic-poi/ — venuevenueA named, non-address place — a business, building, park, or stadium. Mailwoman's free-text point-of-interest component, added as a Tier 2 fine label. detection that benefits from FSTFST (finite-state transducer). A compact automaton that reads an input sequence and emits an output sequence. Mailwoman encodes gazetteer names and street affixes as FSTs for fast prefix matching and prior injection without search overhead. negative evidence