Reconcile — the multiplicative-score trap

A short article on the one non-obvious knob inside StagestageOne of the dataflow stages in the runtime pipeline (normalize, locale gate, kind classify, phrase group, token classify, sequence correct, reconcile, resolve). Distinct from tier (model vocabulary) and phase (plan milestone). 5's joint decoderdecoderIn a transformer encoder-decoder model, the part that produces output sequences. Mailwoman's classifier is encoder-only (no decoder); the 'CRF decoder' is a different thing — a structured-prediction layer that picks the best label sequence from the encoder's outputs.. If you skip it the reconciler converges on the empty parseaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates. for every input.

The full walkthrough of what StagestageOne of the dataflow stages in the runtime pipeline (normalize, locale gate, kind classify, phrase group, token classify, sequence correct, reconcile, resolve). Distinct from tier (model vocabulary) and phase (plan milestone). 5 does and why is in Joint decoding — a walkthrough. This article assumes you have read that one and only zooms in on the scoring formula.

Symptom

StagestageOne of the dataflow stages in the runtime pipeline (normalize, locale gate, kind classify, phrase group, token classify, sequence correct, reconcile, resolve). Distinct from tier (model vocabulary) and phase (plan milestone). 5 picks a parse treeparse treeThe hierarchical address structure produced by decoding BIO labels into spans and nesting them per the schema's containment rules (house_number → street → locality → region → country). by beam searchbeam searchA decoding search that keeps the top-k partial candidates at each position. Used in Stage-5 reconciliation to explore the (phrase, tag, resolver) space with controlled pruning. over (span × tag × resolver candidate) combinations. The score for each combination is:

score = phrase_confidence × classifier_confidence × resolver_score × concordance_bonus

All four factors are in [0, 1]. We multiply them.

First implementation, all 14 contract tests passed. Then every kryptonite fixture failed with expected … to be defined: the winning beam was empty. The reconciler returned no parseaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates. at all, not just an incorrect one.

This took about 15 minutes to diagnose because the algorithm reads natural ("multiply the confidences"), and the bug is implicit in the multiplicative framing.

Why empty wins

Each factor in [0, 1] strictly lowers the running product. Log-space makes the trap obvious:

• log(0.9) = -0.105
• log(0.85) = -0.163
• log(0.7) = -0.357

Every factor we multiply is a negative number in log-space. So accepting any slot reduces the running log-score. The empty beam starts at log-score 0 (empty product = 1, log 1 = 0) and stays there. Every other beam goes negative the moment it accepts a slot.

With beamWidth = 1 (greedy), the empty beam dominates immediately and beam pruning keeps only it. With beamWidth > 1, the empty beam survives forever as the highest-scoring path and the search returns it as the winner.

This is not specific to our use case — any joint decoderdecoderIn a transformer encoder-decoder model, the part that produces output sequences. Mailwoman's classifier is encoder-only (no decoder); the 'CRF decoder' is a different thing — a structured-prediction layer that picks the best label sequence from the encoder's outputs. that composes confidences multiplicatively will hit it. It is what the reconciler is supposed to be doing (picking high-evidence interpretations), so the search needs a structural reason to prefer accepting evidence over skipping it.

Fix — inclusion bonus, scaled per word

Add a log-bonus for each word covered by an accepted slot. The bonus shifts the trade-off so accepting a high-confidence slot is netneural networkA model made of layers of simple numeric units whose connection strengths (weights) are learned from data. The transformer encoder at Mailwoman's core is a neural network.-positive in log-space.

In core/pipeline/reconcile.ts:

const INCLUSION_LOG_BONUS = Math.log(2.5) // ≈ +0.916, × the slot's word count

What log(2.5) means: any slot whose per-word geometric mean of factors exceeds 1/2.5 = 0.4 is netneural networkA model made of layers of simple numeric units whose connection strengths (weights) are learned from data. The transformer encoder at Mailwoman's core is a neural network.-positive in log-space and worth including. Slots below that aren't.

Concretely: a one-word slot with phrase=0.85, classifier=0.7, resolver=0.9, concordance=0.95 has product 0.508. With the bonus added, that slot's log-contribution is log(0.508) + log(2.5) = log(1.27) > 0. It is netneural networkA model made of layers of simple numeric units whose connection strengths (weights) are learned from data. The transformer encoder at Mailwoman's core is a neural network.-positive and will be accepted. Drop any factor below ~0.7 across the board and the slot becomes netneural networkA model made of layers of simple numeric units whose connection strengths (weights) are learned from data. The transformer encoder at Mailwoman's core is a neural network.-negative and gets skipped — the search prefers to leave it out, which is also the right answer when evidence is weak.

The 2.5 is empirical. e (≈ 2.718) is the natural-log analogue, equivalent to "a slot with product > 1/e survives". We landed on 2.5 after the kryptonite fixtures rather than 2.718 — slightly more permissive, slightly easier to reason about as "factors averaging above 0.4". Future tuning should be driven by an evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. corpuscorpusThe BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus. large enough to justify a change.

Why per word, not per slot

The first implementation granted the bonus once per accepted slot. That carries two structural biases, both measured on the bare query "New York City" with no resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon. wired (the live demo's configuration, 2026-06-11):

1. Fragments beat coveragecoverageThe fraction of a population or region for which a data source has real, non-placeholder entries — e.g. 47% rooftop coverage on Texas addresses. Distinct from accuracy on the rows that are present.. The per-tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. logitlogitA raw, unnormalized per-label score the model outputs before softmax. Priors and biases are added in logit space, then softmax turns logits into probabilities. aggregation gives interior fragments inflated confidence — City aggregates the I-localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy. mass of a tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. that sits inside a localitylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighbourhood in the hierarchy., scoring locality=0.92, while the full New York City spanspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree. scores 0.60. With a flat per-slot bonus, the beam paid nothing for leaving New York unexplained: the bare locality="City" won at +0.475 over the full spanspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree.'s +0.199, and the grouper-auditgrouper-auditA validation pass checking that phrase-grouper spans are internally consistent — no overlaps, no contradictions with BIO structural rules. Audit errors must be zero on a shipping model. then promoted the orphaned York proposal to a spurious region node. Final output: {region: "York", locality: "City"} — for an input the plain argmax path parsesaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates. correctly.
2. More spansspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree. collect more bonuses. Two interpretations covering the same words were not scored on evidence alone — the one split into more slots banked one bonus per slot, a flat subsidy for fragmentation.

Scaling the bonus by the slot's word count makes coveragecoverageThe fraction of a population or region for which a data source has real, non-placeholder entries — e.g. 47% rooftop coverage on Texas addresses. Distinct from accuracy on the rows that are present. the thing being rewarded. Equal-coveragecoverageThe fraction of a population or region for which a data source has real, non-placeholder entries — e.g. 47% rooftop coverage on Texas addresses. Distinct from accuracy on the rows that are present. interpretations collect equal bonus (the evidence factors alone decide between them), and a covering interpretation out-scores one that silently drops words unless the extra evidence is weak. On the query above, the full spanspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree. now wins at +2.03 (three words' bonus) vs +0.48 for the bare fragment.

Word counting is whitespace-delimited with a floor of one — scripts without spaces (CJK) degrade to the old per-slot behavior rather than misbehaving.

Why this knob is not public

INCLUSION_LOG_BONUS is a module-level constant, not a configurable option. Reasons:

• Exposing it as an opt invites callers to set it to 0, which defeats the purpose of the reconciler — the empty parseaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates. comes back.
• The sensible range is narrow. Below log(2) (≈ 0.693) the empty parseaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates. re-dominates on most inputs; above log(4) (≈ 1.386) the search starts accepting noise spansspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree.. Operators who want to tune this should be reading the reconciler's tests, not the public API.
• A per-localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for. knob may justify itself later (Japanese vs English may have different "what counts as low-confidence evidence" floors). When the evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. data exists to justify per-localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for. priors, we can expose this as a controlled knob. Today it would be premature.

Adjacent pitfall — kSpan as a global cap

The same file had a parallel bug worth knowing about. The first impl applied kSpan = 3 as a global "top-k phrase proposals by confidence" cap. That silently dropped Houston at 0.65 confidence in NY-NY Steakhouse, Houston, TX because three other proposals had higher confidence elsewhere in the input.

The right semantic is per-start-position: top-k overlapping proposals at each anchor, not a global cap across the whole input. Phrase proposals at offset 0 and offset 18 are not competing for the same slot; pruning them together throws away independent information.

This is the same shape of bug — a natural-reading parameterparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. (kSpan = 3) that silently breaks the contract — and the same fix (anchor by structure, not by ranking).

Forward-looking — anywhere this pattern recurs

Any future stagestageOne of the dataflow stages in the runtime pipeline (normalize, locale gate, kind classify, phrase group, token classify, sequence correct, reconcile, resolve). Distinct from tier (model vocabulary) and phase (plan milestone). that composes confidences multiplicatively will need the same treatment:

• The v0.5.1 learned spanspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree. proposer (PHASE_8_E_learned_span_proposer.md) will produce its own confidence distribution. If it feeds into a search with multiplicative scoring, the inclusion bonusinclusion bonusA per-word credit added in log space to accepted spans during reconciliation, so an empty parse can't win on multiplicative scoring just by asserting nothing. needs to be applied again — adding a layerlayerOne transformer block — attention plus a feed-forward network, with normalization and residual connections — applied to every position. Stacking layers lets the model build up richer representations; Mailwoman's encoder has 6. of factors does not change the trap.
• The "optimal-cover" solver mentioned in earlier design notes (a candidate decoderdecoderIn a transformer encoder-decoder model, the part that produces output sequences. Mailwoman's classifier is encoder-only (no decoder); the 'CRF decoder' is a different thing — a structured-prediction layer that picks the best label sequence from the encoder's outputs. that picks a non-overlapping set of spansspanA contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree. to cover the input) is structurally identical. Same trap, same fix.
• Any joint decodingjoint decodingA decode strategy where the neural model's proposals and rule-based solver's output are reconciled into a single parse tree. Formerly the default (Route A Phase II), now retired in favor of argmax after it was found to break the street+house_number geocode precondition. over additional axes (e.g., × alternative-language candidate) needs the bonus scaled — the empty parseaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates. cost goes up as the axes multiply, so the bonus per accepted slot may need to grow with them.

See also

• Joint decoding — a walkthrough — the full picture of what StagestageOne of the dataflow stages in the runtime pipeline (normalize, locale gate, kind classify, phrase group, token classify, sequence correct, reconcile, resolve). Distinct from tier (model vocabulary) and phase (plan milestone). 5 does on a kryptonite input
• reconcile.ts — the actual implementation
• v0.4.0 ablation campaign — the campaign whose mixed results made the missing reconcile rung visible