Wiki /
Back to index
Wiki › OSM › OSM — OpenStreetMap Platform

OSM — OpenStreetMap Platform

Draft Updated today
Edit

OSM — OpenStreetMap Platform

iNeed Asset Name

Open Street Map Server (ImpOSM)

What it is

OpenStreetMap data platform. ImpOSM is the import/processing component.
Managed by Zeeshan's team (GIS PM). Carlos handles DB/infra support.

Servers (PROD, dpko.un.org, RHEL 7)

  • dfs-imposm-01 — ImpOSM import server
  • dfs-osmpgsql-01, 02, 03 — PostgreSQL DB servers
  • GIS servers: dfs-gisosm-01 (PROD), dfs-gisosm-11 (PRE)
  • GIS DB: dfs-gispgsql-02 (PROD), dfs-gispgsql-12 (PRE)

Active Tasks

  • OSM data migration - pending tasks (overdue Mar 19)
  • Migration to Proxmox - Prepare OSM data server migration (overdue Mar 20)
  • OSM Monthly patches - Consolidate procedure (due Mar 31)
  • OSM Maps Batch 5A (due Mar 31)

Projects

  • New OSM updates procedure - Consolidate and execute (no deadline)
  • Migration to Proxmox - Phase 2 (overdue Mar 6)

FW Rules Issue (Current)

FW rules requested for OSM server access got stuck in security approval with Antonio Candelaria.
Zeeshan spoke to Antonio. Carlos to debrief with Zeeshan (call due this week).

Knowledge Gaps

  • What does the monthly patch procedure involve step by step?
  • What does OSM Batch 5A mean?
  • What is the OSM data migration pending — what data?

Infrastructure Detail — unosm-r-app-01

Compiled 2026-03-31

Server Overview

Property Value
Hostname unosm-r-app-01
OS RHEL (Red Hat Enterprise Linux)
Role OSM data processing

Filesystem Layout

Mount Size Notes
/ 53G Root filesystem
/var 15G Logs and system data
/data 1000G Main data volume
/boot 994M Boot partition
/tmp 63G RAM-backed (tmpfs)

Note: Application data (imposm diffs, service account home dirs) lives under /home on root, NOT on /data.

Service Accounts

Account Purpose
_osmprod Main OSM production — runs imposm
_osmstat OSM statistics processing
_osm2pgsql osm2pgsql bulk imports
adm.pfrizzer Admin user (UN domain)

Application Stack

Imposm

  • Location: /home/_osmprod/imposm/
  • Versions: 0.11.1, 0.14.0, 0.14.2
  • Function: keeps PostGIS DB in sync with OSM planet via daily replication diffs
  • Replication source: https://planet.openstreetmap.org/replication/day/
  • Diff storage: /home/_osmprod/imposm/diffs/osm_planet/ and osm_planet_buildings/
  • State file: /home/_osmprod/imposm/diffs/osm_planet/last.state.txt

Imposm downloader behaviour:

  • Uses parallel workers; temp files named SEQ.osc.gz~ID during download
  • On success: renamed to final filename, temp files deleted automatically
  • On interruption: orphaned temp files left behind — must be cleaned up manually
  • Also contains LevelDB caches: nodes/, ways/, relations/, coords_rel_index/, ways_index/

osm2pgsql

  • Service account: _osm2pgsql
  • Handles bulk imports into PostGIS

osmosis 0.49.2 — present under imposm bin dir

Installed Agents

Tool Purpose
Elastic Agent (/opt/Elastic/Agent) Monitoring + log/metric forwarding
Nessus Agent (/opt/nessus_agent) Vulnerability scanning
filebeat Additional log shipping
GCC Toolsets 12/13/14 Build toolchains (compiled software present)

Security & Identity

  • Domain-joined to global.un.org via SSSD
  • auditd running (audit logging active)
  • filebeat + Elastic Agent for log shipping

UNOSM Architecture (detailed reference)

Mental Model

  • Write brain: unosm-r-db-01
  • Read helpers: unosm-r-db-02, unosm-r-db-03
  • Importer: unosm-r-app-01
  • User entry points: unosm-r-app-02 (Brindisi), unosm-p-app-02 (Valencia)
  • Traffic cop: NetScaler → osmpgsql.dfs.un.org
  • Exporter: unosm-r-data-01

Database Layer (PostgreSQL)

Master

  • unosm-r-db-01 (Brindisi)
  • Handles all writes: INSERT, UPDATE, DELETE
  • Manages transactions locally
  • Generates WAL logs

Slaves

  • unosm-r-db-02 (Brindisi)
  • unosm-r-db-03 (Valencia)
  • Read-only replicas
  • Receive WAL logs asynchronously
  • Can be slightly behind the master

Replication Model

  • Asynchronous replication
  • Master commits without waiting for slaves
  • Faster writes, possible replication lag

Failure Behavior

  • No automatic failover
  • If unosm-r-db-01 fails:
    • Manual promotion of unosm-r-db-02 or unosm-r-db-03 required

Application Layer

Data Ingestion

  • unosm-r-app-01
  • Imports OpenStreetMap data using Imposm
  • Feeds data into unosm-r-db-01

Client Access

  • unosm-r-app-02 (Brindisi)
  • unosm-p-app-02 (Valencia)
  • Receive user requests
  • Distribute read queries to database slaves

Load Balancing & Traffic Flow

Global Load Balancer

  • NetScaler
  • Public endpoint: osmpgsql.dfs.un.org
  • Routes traffic based on:
    • Client location
    • Site load
    • Performance metrics

Current Behavior

  • Active load balancing mainly within Brindisi
  • Valencia is available but less actively used

Export & Supporting Services

Data Export

  • unosm-r-data-01
  • Dedicated to export jobs
  • Prevents heavy exports from impacting core services

TL;DR Flow

Users → NetScaler → App Servers → Read SlavesImports → unosm-r-app-01 → unosm-r-db-01Writes → unosm-r-db-01 → WAL → Slaves

Infrastructure
Context: OSM
PROD
NameHostnameIPStatusRoleOSLocationHostingHypervisorCPUsMemoryDisksStackTags
dfs-imposm-01 dfs-imposm-01.dpko.un.org 10.130.202.37 active RHEL 7 VLC UN Vector Tiles
Notes: UN Vector Tiles app - Related infrastructure
dfs-osmpgsql-01 dfs-osmpgsql-01.dpko.un.org 10.129.75.169 active RH7 VLC UN Vector Tiles
Notes: UN Vector Tiles app - Related infrastructure
dfs-osmpgsql-02 dfs-osmpgsql-02.dpko.un.org 10.129.75.170 active RH7 VLC UN Vector Tiles
Notes: UN Vector Tiles app - Related infrastructure
dfs-osmpgsql-03 dfs-osmpgsql-03.dpko.un.org 10.129.75.171 active RH7 VLC UN Vector Tiles
Notes: UN Vector Tiles app - Related infrastructure
unosm-p-app-02 unosm-p-app-02.global.un.org 10.130.3.22 active User Interface VLC private-cloud Proxmox
unosm-r-app-01 unosm-r-app-01.global.un.org 10.128.128.49 active Data Import RHEL 9.7 Unknown private-cloud VMWare 12 125 80 GB, 500 GB, 500 GB
Notes: sdb and sdc are some kind of cluster as both added create the /data partition: sdb 8:16 0 500G 0 disk └─data-lv_data 253:2 0 1000G 0 lvm /data sdc 8:32 0 500G 0 disk └─data-lv_data 253:2 0 1000G 0 lvm /data sr0 11:0 1 1024M 0 rom
unosm-r-app-02 unosm-r-app-02.global.un.org 10.130.207.88 active User Interface BDS private-cloud Proxmox
unosm-r-data-01 unosm-r-data-01.global.un.org 10.128.136.63 active Data Export RHEL 9.6 Unknown private-cloud VMWare 8 32 100 GB, 500 GB
unosm-r-db-01 unosm-r-db-01.global.un.org 10.130.132.154 active Database RHEL 9.7 Unknown private-cloud Proxmox 16 188 80 GB, 2000 GB PostgreSQL
Notes: Write node
unosm-r-db-02 unosm-r-db-02.global.un.org 10.130.132.155 active Database RHEL 9.7 Unknown private-cloud Proxmox 16 188 80 GB, 1500 GB PostgreSQL
Notes: Read-only node
unosm-r-db-03 unosm-r-db-03.global.un.org 10.130.132.158 active Database RHEL 9.7 Unknown private-cloud Proxmox 16 188 80 GB, 1500 GB PostgreSQL
Notes: Read-only node
RFS Tracking
RFSTeamTypeDescription
No RFS entries yet.
No linked firewall application in this context.
Attachments
Related pages
Created: 2026-01-22 10:22 · Updated: 2026-04-17 14:01