Skip to content

Consolidate the fleet's 19 Railway Postgres instances: census first, then retire, then dedupe, and co-locate only by ruling


Consolidate the fleet’s 19 Railway Postgres instances: census first, then retire, then dedupe, and co-locate only by ruling

Section titled “Consolidate the fleet’s 19 Railway Postgres instances: census first, then retire, then dedupe, and co-locate only by ruling”

Date: 2026-10-21 Status: open Supersedes: none Superseded-by: none — current MR-compliance: MR-4, MR-7, MR-20

The first run of eva cycle monthly-railway-cost (eva-hq docs/eng/fleet/railway-cost-2026-10-21.md, eva-hq/eva#194, cycle 2026-09-15 → 2026-10-15) counted Railway datastores by image, not by service name. It found 19 Postgres, 4 Redis, 1 Mongo and 1 NATS. The 2026-08-31 census (ledger D-5) had counted 8 Postgres by name. The ledger records the correction as RF-09: “count by image”.

What the 19 cost, from the census’s lever table: $15.88 per cycle, 30% of usage, averaging 73 MB each. Usage was $52.21 by API against $53.07 on the dashboard. Only three Postgres are named per instance in the census’s top-12:

projectservice$/cycleavg MB
SKYFLOW·BLUEPostgres3.08283
DEVARNO·CLOUDPostgres1.79173
NULL0·TOLLPostgres1.15104

So the other 16 cost about $9.86 between them, about $0.62 each.

The ledger has already ruled on where the lever is. D-6 (2026-08-31) measured a flat working set (peak/avg median 1.48) and concluded: “the lever is container COUNT, not container size”. Tuning shared_buffers saves nothing on a platform that bills consumed memory. The eva-hq finding docs/findings/20261021-1330-sevalla-eval-railway-cost.md declined a move to Sevalla. It ranks “8 Postgres → 1–2” as Railway lever 3 and routes fleet-binding consolidation here, as a **Status:** open doc for a FLIGHT countersign.

Three constraints shape every option:

  1. Two projects are indistinguishable by address. ASGARD • CODES and DEVARNO • CLOUD both expose postgres.railway.internal:5432, database railway. Rotating the wrong one takes down airlock, ares, chat-service and pebble-mcp. This is a recorded near-miss, not a theory (mary-wiki docs/decisions/2026-08-14-go-asgard-rotation-and-215-deploy.md; skill railway-target-disambiguation). Host, port and database name do not identify a target. Only the project name plus its sibling-service list does. Consolidation multiplies the moments where this trap is live.
  2. Private networking stops at the project boundary. *.railway.internal is free and private only within one project and environment (ledger M1, R7). A Postgres shared by services in different projects has to be reached over the public TCP proxy, *.proxy.rlwy.net. That traffic is billed egress, and the database becomes reachable from the internet. That is the exact shape of the DEVARNO·CLOUD Redis finding (ledger B-6, RF-09, RF-10): a cache on the public proxy that became the fleet’s largest egress line.
  3. The per-instance list does not exist in any readable artefact. The census publishes the count, the total and the top 12. A 19-row table needs a read of Railway’s API, which needs operator approval for this session. This doc therefore proposes dispositions by class, with named instances only where a source names them.

Proposed for FLIGHT. Nothing in this doc is executed by merging it.

1. Census before any write. An operator-approved, read-only census by image produces one row per Postgres instance:

  • project name and sibling-service list;
  • $/cycle and average MB;
  • whether the latest deployment succeeded;
  • which services reference it, matched by variable name and host hash, never by value.

Every later step cites that table. Until it exists, no Postgres is moved, merged or deleted under this decision.

2. Options. FLIGHT picks one row.

#OptionSaving (census basis)Blast radiusReversible?Verdict proposed
AStatus quo, plus memory tuning≈ $0 (D-6: footprints are flat working sets)noneyesRejected. It tunes the one thing D-6 proved is not the lever.
BStaged: retire → dedupe within project → co-locate only by per-instance rulingRetire: whatever Doc 2 retires. Dedupe: about $0.6 per removed duplicate. Co-locate: the rest of the ~$8–11 the census predicts.one project at a timeeach stage stands aloneProposed.
COne shared cluster for everything, reached over the public TCP proxy~$8–11 less, minus new egressevery consumer at once; database internet-reachableexpensive to undoRejected. Constraint 2: it rebuilds the Redis public-proxy finding with a database.
DMove databases off Railway to a fixed-cost VPS Postgresunknown until sizedevery moved consumerexpensiveOut of scope. The Sevalla finding makes a VPS tier its own decision, and only if levers 2–4 leave it worth doing.

3. Dispositions by class (option B).

ClassInstancesDispositionGate
K — keep separateDEVARNO·CLOUD Postgres (shared substrate: airlock, ares, chat-service, pebble-mcp); ASGARD·CODES Postgres (mary-wiki FDR persister); NULL0·TOLL Postgres (money spine); TRACEO·CAT Postgres (live billing, traceo-cat M-22); SKYFLOW·BLUE Postgres ($3.08, live launch-to-revenue, deadline 2026-11-14)Keep. No co-location.none
R — retire with its projectany Postgres in SO1·IO; the .me datastore behind SKYFLOW•ME skyflow-server/reach; any other project Doc 2 rules retiredRemoved by that project’s retirement, never by this decision.so1: devarno-cloud plan R4 (after R1–R3). .me: skyflow-hq M6.3 HARD ORDER D3, only after the M6.6.b export reconciles and is countersigned.
D — dedupe within a projectany project whose census rows show more than one Postgres. 19 by image against 8 by name suggests several do.Fold the duplicate into the project’s primary Postgres. Private networking is unchanged.per project: census row, consumer list, sibling-service list stated in the PR.
C — co-location candidatessmall internal instances with no revenue dependency. Repo evidence names Railway Postgres in KAHN (kahn-hq/backend/kahn/storage/__init__.py:12), STRATT·WORKS (stratt-run/services/orchestrator-http/Dockerfile:22), CHRONICLE (chronicle-cat/docs/superpowers/plans/2026-09-09-b7-postgres-migration.md:57), DEVAQUA (devaqua-blue/README.md:14), CHOCO·GATEWAY, NESTR (nestr-tools/docs/decisions/2026-10-20-phase-12-friction-round.md:59), ARNO·HOST (arno-host/docs/runbooks/database-url-provision-and-rotation.md:69) and others. Unconfirmed against Railway: the census in step 1 confirms or drops each one.Co-locating means moving the services into one project with one Postgres (one database per tenant). It never means pointing them at another project’s proxy. One ruling per instance.FLIGHT, per instance, after step 1.

4. Order. Step 1 (census), then R (retire), then D (dedupe), then C (co-locate). Each write goes through the procedure in railway-target-disambiguation, in this order:

  1. name the project;
  2. show its sibling services;
  3. match them to intent;
  4. write.

Each credential move goes through credential-rotation-propagation, in this order:

  1. enumerate the consumers;
  2. write the new value;
  3. verify every consumer reads it;
  4. revoke the old value.
  • A — status quo plus tuning. Rejected. D-6 measured flat working sets, so a cap above the working set reclaims nothing.
  • C — one public-proxy cluster. Rejected. Constraint 2: billed egress, and a database reachable from the internet.
  • D — a VPS Postgres tier. Deferred to its own decision, per the Sevalla finding’s lever 6.
  • Write the 19-row table from repo evidence alone. Rejected. Constraint 1: a table built from what repos say about addresses is exactly the identification method the disambiguation skill refuses.

For code: none. A co-location under class C changes the moved services’ DATABASE_URL reference. That change goes in each owning repo’s PR, under its own review.

For docs: the step-1 census lands in eva-hq as a dated fleet doc beside docs/eng/fleet/railway-cost-2026-10-21.md. Each class-C ruling is appended here as a scribe note (landing-authority M3) or written as its own decision.

For in-flight phases: none paused. Two known additions to the count:

  • the P3 live TOLL gate brings its own Postgres (null0-toll/toll docs/hardening/p3-production-cutover-checklist.md, null0-toll/toll#159);
  • that instance is class K from birth. The P3 decision forbids sharing it.

For invariants: none. The disambiguation rule is a skill and a recorded near-miss. It is not a numbered invariant, and this doc does not make it one (MR-9).

What this decision does NOT do:

  • It does not read or write Railway.
  • It does not delete a volume.
  • It does not touch DEVARNO·CLOUD Redis. That has its own levers: hubble#90 is deployed and RF-10 has the 2026-10-22 read pending.
  • It does not decide any project’s retirement. That is 2026-10-21-railway-projects-so1-tektree-skyflow.md.
  • eva-hq/eva docs/eng/fleet/railway-cost-2026-10-21.md (the 19 count, $15.88 and 30%, top-12 rows, lever 3)
  • eva-hq/eva docs/findings/20261021-1330-sevalla-eval-railway-cost.md (Sevalla declined; lever ordering; fleet-binding decisions routed to petrova-codes)
  • Ledger ~/Documents/clutter/grug/task/runtime-cost-fanout.md (operator-local): M1 egress and private networking; R7; D-5; D-6 “container COUNT”; B-6; RF-09 “count by image”; RF-10
  • mary-wiki/mary docs/decisions/2026-08-14-go-asgard-rotation-and-215-deploy.md (the near-miss behind the disambiguation rule)
  • skyflow-blue/skyflow-hq MILESTONES.md M6.3 (HARD ORDER D3 on the .me datastore)
  • devarno-cloud/devarno-cloud docs/superpowers/plans/2026-10-08-retire-so1-io.md (R4: mission.so1.io on Railway, “disambiguate the project first”)
  • null0-toll/toll docs/decisions/2026-10-14-p3-production-polar-via-a-separate-gate.md (a second TOLL Postgres, unshared by design)
  • docs/decisions/2026-10-21-railway-projects-so1-tektree-skyflow.md, docs/decisions/2026-10-21-stratt-orchestrator-http-crashed.md (sibling drafts)
  • Subagent: Claude Opus 5.5, 2026-10-21 (drafted from the eva-hq census, the Sevalla finding and the cost ledger; no Railway read or write)
  • Human countersign — picks option A, B, C or D; if B, approves the step-1 read-only census, and confirms the class-K list — flight@arno.host, 2026-10-27 (jzero.run T1)