Eval reports

The measurement record behind every released modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' and resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon. change: gates, benchmarks, and diagnostics. Reports are grouped by what they measure, in roughly the order you'd read them to see what the system does and how it improved.

By topic

• Retrospectives — day- and session-level wrap-ups that narrate an arc: what shipped, what failed, what the next lever is.
• Competitive & parity — headattention headOne of several parallel attention computations in a layer, each free to focus on a different kind of relationship between tokens. Their outputs are concatenated — 'multi-head attention'. Mailwoman uses 4 heads.-to-headattention headOne of several parallel attention computations in a layer, each free to focus on a different kind of relationship between tokens. Their outputs are concatenated — 'multi-head attention'. Mailwoman uses 4 heads. against Nominatim, PeliasPeliasAn open-source geocoder, Mailwoman's spiritual predecessor., and libpostallibpostalAn open-source C address parser used by Pelias. Mailwoman's rule-based v0 and neural classifier supersede it.; the parity scorecardsparity scorecardThe authoritative per-tag table tracking neural-vs-v0/Pelias F1 and resolver accuracy across head-to-head arenas. It answers 'where are we at parity, where do we still bleed?' and governs the parity campaign's priorities. and cross-modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' failure reports.
• Model versions — per-release evalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. matrices, ship-gates, promotions, and error analyses. The version-over-version scoreboard.
• Resolver & geo — coordinate accuracy: the resolverresolverThe component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon. ladder, the coarse placer, situssitusThe physical site address of a property, as opposed to the owner's mailing address. Parcel records often carry both; the divergence is a real-world data-quality challenge. cascade, and the postcode anchoranchor inferenceA technique where structured knowledge (postcode locations, gazetteer place names) is injected into the model as soft input features — not as deterministic overrides. The model still decides the final labels, but the anchor signal biases it toward correct admin tags..
• Matcher & dedup — record matchingrecord matchingThe process of determining whether two database records refer to the same real-world entity. Mailwoman's matcher uses a geocode-first approach (match the resolved place, not the address string) with Fellegi-Sunter probabilistic scoring. and entity resolutionrecord matchingThe process of determining whether two database records refer to the same real-world entity. Mailwoman's matcher uses a geocode-first approach (match the resolved place, not the address string) with Fellegi-Sunter probabilistic scoring.: the Fellegi-SunterFellegi-SunterA probabilistic record linkage model that computes match probability from agreement-level log-likelihood ratios: log₂(m/u) where m is the probability of agreement given a true match and u is the probability of agreement by chance. Mailwoman learns m and u label-free via expectation-maximization. and learned scorers, dedup ceilings, cross-source generalizationgeneralizationA trained model's performance on data unlike its training set — new regions, new input distributions. The property honest eval is designed to measure..
• Calibration — confidence and uncertainty: isotonic + conformal calibrationconformal calibrationA calibration method that adjusts per-prediction uncertainty so a target coverage (e.g. 90%) is guaranteed on held-out data, without retraining the model., per-localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for. reliability, the interpolationinterpolationA geocoding technique that estimates a coordinate along a street segment based on the house number range. Used as the middle tier of Mailwoman's geocode cascade when exact address-point data is unavailable. radius.
• Coverage — geographic breadth: fill rates and multi-localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for. coveragecoverageThe fraction of a population or region for which a data source has real, non-placeholder entries — e.g. 47% rooftop coverage on Texas addresses. Distinct from accuracy on the rows that are present..
• Experiments — one-off probes, stress tests, and feasibility studies that didn't (or didn't yet) become a release lever.
• Night shifts — dated autonomous-session logs. Mostly kept in-repo for the record, not published.

Cross-cutting

• Pre-publish 2D eval gate — the mechanical guard every neural-weightsparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values. release clears.
• Per-tag score trends — generated per-tag scores across releases (regenerated after each ledger row).

Reports are point-in-time. EvalevalRunning the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error. sets, channels, and quantization evolve between eras, so adjacent columns compare cleanly and distant ones are directional — each report statesregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. its own conditions.