Wikipedia Importance Scores

Place importance scoresimportance scoreA precomputed per-place prominence score (blending Wikipedia importance with population) used to rank same-name gazetteer candidates. derived from Wikipedia link count, replacing raw population as the FSTFST (finite-state transducer). A compact automaton that reads an input sequence and emits an output sequence. Mailwoman encodes gazetteer names and street affixes as FSTs for fast prefix matching and prior injection without search overhead. emission prioremission priorA log-probability bias injected into the model's emission logits from an external signal — gazetteer frequency, an FST, a Wikipedia-importance score — combined with the learned weights at decode time. weightparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values.. Shipped in #173.

Source

Nominatim's wikimedia-importance.csv.gz — 19M rows mapping Wikidata IDs to importance scoresimportance scoreA precomputed per-place prominence score (blending Wikipedia importance with population) used to rank same-name gazetteer candidates.. The score is log(total_links) / log(max_links) where total_links = internal links + cross-language links to the Wikipedia article. Normalized to [0, 1] where the United States article (5.2M links) = 1.0.

ETL pipeline

wikimedia-importance.csv.gz (19M rows, ~250 MB compressed)
         │
         ▼
  scripts/build-importance.ts
         │
         ├─ Load WOF concordances (other_source='wd:id') → Set of needed Wikidata IDs
         ├─ Stream-decompress TSV, filter to matching IDs only
         ├─ Collapse duplicates by MAX(importance) per Wikidata ID
         ├─ JOIN: wof_id → concordance wikidata_id → importance score
         ├─ Write place_importance(id, importance) table
         └─ Population fallback: min(1.0, log2(1+pop/1000)/14) for places without Wikidata

Coverage (US admin)

Source	Places
Wikipedia importanceWikipedia importanceA place-notability score derived from the count and language spread of a place's Wikipedia articles, normalized to [0, 1]. A resolver ranking prior that helps pick the prominent same-named place. (via Wikidata concordanceconcordanceA joint-decode signal that rewards a parse whose spans form a consistent Who's On First parent-child chain and vetoes contradictory ones — a hard veto for conflicts, a log-space bonus for full agreement.)	47,348
Population fallback	108,111
Total in place_importance	155,459

How scores flow into the FST

1. build-importance.ts writes place_importance table into the WOFWOF (Who's On First). An open-source gazetteer of places maintained by Mapzen/whosonfirst. Mailwoman builds a custom SQLite database from WOF GeoJSON repos, extended with postcode data, importance scores, and coincident-role relations. SQLite
2. fst-builder.ts reads place_importance (falls back to place_population → pseudo-importance)
3. PlaceEntry.importance carries the score through serialization (Float32 in the binary FSTFST (finite-state transducer). A compact automaton that reads an input sequence and emits an output sequence. Mailwoman encodes gazetteer names and street affixes as FSTs for fast prefix matching and prior injection without search overhead.)
4. fst-prior.ts computes bias: importance × biasScale × maxBias (linear, capped at 3.0 logitslogitA raw, unnormalized per-label score the model outputs before softmax. Priors and biases are added in logit space, then softmax turns logits into probabilities.)

Why not population

Signal	Washington DC	Washington stateregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.	Winner
Population	678K	7.6M	StateregionThe first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality. (wrong for bare "Washington")
Wikipedia importanceWikipedia importanceA place-notability score derived from the count and language spread of a place's Wikipedia articles, normalized to [0, 1]. A resolver ranking prior that helps pick the prominent same-named place.	0.815	0.764	DC (correct — more culturally prominent)

Population is an administrative headcount. Wikipedia importance captures cultural prominence — how often the place is referenced, linked to, and discussed across languages.

Regeneration

Run after any WOFWOF (Who's On First). An open-source gazetteer of places maintained by Mapzen/whosonfirst. Mailwoman builds a custom SQLite database from WOF GeoJSON repos, extended with postcode data, importance scores, and coincident-role relations. data refresh:

node scripts/build-importance.js --db /path/to/wof-unified.db [--tsv /path/to/wikimedia-importance.csv.gz]

The TSV is cached at /tmp/wikimedia-importance.csv.gz after first download. Pass --tsv to skip the download.

See also

• FST Gazetteer LM — the FSTFST (finite-state transducer). A compact automaton that reads an input sequence and emits an output sequence. Mailwoman encodes gazetteer names and street affixes as FSTs for fast prefix matching and prior injection without search overhead. architecture this feeds into
• Nominatim importance docs — upstream methodology