Matcher scale — pure-Node block → score → cluster (#K)

Generated by scripts/record-matcher/matcher-scale.ts. Synthetic geo-clustered records (~3 per distinct place), resolved with the geo-first baseline (collapsed spatial + EMexpectation-maximizationAn iterative algorithm that estimates model parameters when some variables are unobserved. In Mailwoman's matcher, EM learns the Fellegi-Sunter m and u parameters from unlabeled data — no training labels needed.) in ONE Node process — no Elasticsearch, no server. GeocodinggeocodingThe process of converting an address into geographic coordinates (latitude and longitude). Mailwoman geocodes in a multi-tier cascade: exact address-point match → street interpolation → locality centroid. Each tier is progressively coarser but more widely available. is out of scope (measured separately); this isolates the matcher's block/score/cluster cost vs N.

records	candidate pairs	entities	wall-clock	resolve rate	RSS (high-water)
10,000	12,321	3,333	0.19 s	52,038/s	141 MB
50,000	106,766	16,665	1.49 s	33,641/s	302 MB
100,000	327,260	33,333	4.40 s	22,726/s	463 MB
250,000	1,673,915	83,333	19.80 s	12,626/s	1085 MB
500,000	6,191,571	166,667	68.53 s	7,296/s	2299 MB

Reading

The matcher resolves 500,000 records in 68.53 s in a single Node process — block → Fellegi-SunterFellegi-SunterA probabilistic record linkage model that computes match probability from agreement-level log-likelihood ratios: log₂(m/u) where m is the probability of agreement given a true match and u is the probability of agreement by chance. Mailwoman learns m and u label-free via expectation-maximization. → union-find clusteringclusteringThe final stage of entity resolution: resolve non-transitive pairwise match decisions (A↔B, B↔C, but not A↔C) into canonical entities via union-find with path compression. Each cluster of records becomes one resolved entity., no external service. Geo-cell + canonical-key blockingblockingThe first stage of entity resolution: generate candidate record pairs using cheap, high-recall keys (geo cell, canonical address, phone) instead of comparing every record to every other (O(n²)). The matcher only scores pairs that survive blocking. keeps candidate generation near-linear (it never enumerates the O(n²) pair space), so the wall-clock scales with the number of blocked pairs, not the square of N. That is the architectural claim made concrete: a calibrated streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.-level geocoder feeding the blockingblockingThe first stage of entity resolution: generate candidate record pairs using cheap, high-recall keys (geo cell, canonical address, phone) instead of comparing every record to every other (O(n²)). The matcher only scores pairs that survive blocking. lets entity resolutionrecord matchingThe process of determining whether two database records refer to the same real-world entity. Mailwoman's matcher uses a geocode-first approach (match the resolved place, not the address string) with Fellegi-Sunter probabilistic scoring. run in-process at hundreds of thousands of records, where a string-similarity matcher would need a search cluster to make the candidate generation tractable.

RSS is the shared-process high-water mark across the sweep (each size reuses the heap), so it is an upper bound, not a per-N working set. The largest size that fits is bounded by the record objects + the link array; beyond that, shardshardA partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row. by geography (each cell is independent) — the same partition the blockingblockingThe first stage of entity resolution: generate candidate record pairs using cheap, high-recall keys (geo cell, canonical address, phone) instead of comparing every record to every other (O(n²)). The matcher only scores pairs that survive blocking. already implies.

One honest caveat on the shape: these are uniform-random US coordinates, so as N rises the geo-cell collision rate climbs and the candidate-pair count grows faster than N (visible in the table). Real address data clusters into a roughly fixed set of places, so its pair/record ratio is flatter — the wall-clock here, which tracks the number of PAIRS, is a conservative upper bound on a realistic workload of the same N.