Consolidate the fleet's 19 Railway Postgres instances: census first, then retire, then dedupe, and co-locate only by ruling
rank: decision outranks: []
Section titled “rank: decision outranks: []”Consolidate the fleet’s 19 Railway Postgres instances: census first, then retire, then dedupe, and co-locate only by ruling
Section titled “Consolidate the fleet’s 19 Railway Postgres instances: census first, then retire, then dedupe, and co-locate only by ruling”Date: 2026-10-21 Status: open Supersedes: none Superseded-by: none — current MR-compliance: MR-4, MR-7, MR-20
Context
Section titled “Context”The first run of eva cycle monthly-railway-cost (eva-hq
docs/eng/fleet/railway-cost-2026-10-21.md, eva-hq/eva#194, cycle
2026-09-15 → 2026-10-15) counted Railway datastores by image, not by
service name. It found 19 Postgres, 4 Redis, 1 Mongo and 1 NATS. The
2026-08-31 census (ledger D-5) had counted 8 Postgres by name. The ledger
records the correction as RF-09: “count by image”.
What the 19 cost, from the census’s lever table: $15.88 per cycle, 30% of usage, averaging 73 MB each. Usage was $52.21 by API against $53.07 on the dashboard. Only three Postgres are named per instance in the census’s top-12:
| project | service | $/cycle | avg MB |
|---|---|---|---|
| SKYFLOW·BLUE | Postgres | 3.08 | 283 |
| DEVARNO·CLOUD | Postgres | 1.79 | 173 |
| NULL0·TOLL | Postgres | 1.15 | 104 |
So the other 16 cost about $9.86 between them, about $0.62 each.
The ledger has already ruled on where the lever is. D-6 (2026-08-31) measured
a flat working set (peak/avg median 1.48) and concluded: “the lever is
container COUNT, not container size”. Tuning shared_buffers saves nothing
on a platform that bills consumed memory. The eva-hq finding
docs/findings/20261021-1330-sevalla-eval-railway-cost.md declined a move to
Sevalla. It ranks “8 Postgres → 1–2” as Railway lever 3 and routes
fleet-binding consolidation here, as a **Status:** open doc for a FLIGHT
countersign.
Three constraints shape every option:
- Two projects are indistinguishable by address. ASGARD • CODES and
DEVARNO • CLOUD both expose
postgres.railway.internal:5432, databaserailway. Rotating the wrong one takes downairlock,ares,chat-serviceandpebble-mcp. This is a recorded near-miss, not a theory (mary-wikidocs/decisions/2026-08-14-go-asgard-rotation-and-215-deploy.md; skillrailway-target-disambiguation). Host, port and database name do not identify a target. Only the project name plus its sibling-service list does. Consolidation multiplies the moments where this trap is live. - Private networking stops at the project boundary.
*.railway.internalis free and private only within one project and environment (ledger M1, R7). A Postgres shared by services in different projects has to be reached over the public TCP proxy,*.proxy.rlwy.net. That traffic is billed egress, and the database becomes reachable from the internet. That is the exact shape of the DEVARNO·CLOUD Redis finding (ledger B-6, RF-09, RF-10): a cache on the public proxy that became the fleet’s largest egress line. - The per-instance list does not exist in any readable artefact. The census publishes the count, the total and the top 12. A 19-row table needs a read of Railway’s API, which needs operator approval for this session. This doc therefore proposes dispositions by class, with named instances only where a source names them.
Decision
Section titled “Decision”Proposed for FLIGHT. Nothing in this doc is executed by merging it.
1. Census before any write. An operator-approved, read-only census by image produces one row per Postgres instance:
- project name and sibling-service list;
$/cycle and average MB;- whether the latest deployment succeeded;
- which services reference it, matched by variable name and host hash, never by value.
Every later step cites that table. Until it exists, no Postgres is moved, merged or deleted under this decision.
2. Options. FLIGHT picks one row.
| # | Option | Saving (census basis) | Blast radius | Reversible? | Verdict proposed |
|---|---|---|---|---|---|
| A | Status quo, plus memory tuning | ≈ $0 (D-6: footprints are flat working sets) | none | yes | Rejected. It tunes the one thing D-6 proved is not the lever. |
| B | Staged: retire → dedupe within project → co-locate only by per-instance ruling | Retire: whatever Doc 2 retires. Dedupe: about $0.6 per removed duplicate. Co-locate: the rest of the ~$8–11 the census predicts. | one project at a time | each stage stands alone | Proposed. |
| C | One shared cluster for everything, reached over the public TCP proxy | ~$8–11 less, minus new egress | every consumer at once; database internet-reachable | expensive to undo | Rejected. Constraint 2: it rebuilds the Redis public-proxy finding with a database. |
| D | Move databases off Railway to a fixed-cost VPS Postgres | unknown until sized | every moved consumer | expensive | Out of scope. The Sevalla finding makes a VPS tier its own decision, and only if levers 2–4 leave it worth doing. |
3. Dispositions by class (option B).
| Class | Instances | Disposition | Gate |
|---|---|---|---|
| K — keep separate | DEVARNO·CLOUD Postgres (shared substrate: airlock, ares, chat-service, pebble-mcp); ASGARD·CODES Postgres (mary-wiki FDR persister); NULL0·TOLL Postgres (money spine); TRACEO·CAT Postgres (live billing, traceo-cat M-22); SKYFLOW·BLUE Postgres ($3.08, live launch-to-revenue, deadline 2026-11-14) | Keep. No co-location. | none |
| R — retire with its project | any Postgres in SO1·IO; the .me datastore behind SKYFLOW•ME skyflow-server/reach; any other project Doc 2 rules retired | Removed by that project’s retirement, never by this decision. | so1: devarno-cloud plan R4 (after R1–R3). .me: skyflow-hq M6.3 HARD ORDER D3, only after the M6.6.b export reconciles and is countersigned. |
| D — dedupe within a project | any project whose census rows show more than one Postgres. 19 by image against 8 by name suggests several do. | Fold the duplicate into the project’s primary Postgres. Private networking is unchanged. | per project: census row, consumer list, sibling-service list stated in the PR. |
| C — co-location candidates | small internal instances with no revenue dependency. Repo evidence names Railway Postgres in KAHN (kahn-hq/backend/kahn/storage/__init__.py:12), STRATT·WORKS (stratt-run/services/orchestrator-http/Dockerfile:22), CHRONICLE (chronicle-cat/docs/superpowers/plans/2026-09-09-b7-postgres-migration.md:57), DEVAQUA (devaqua-blue/README.md:14), CHOCO·GATEWAY, NESTR (nestr-tools/docs/decisions/2026-10-20-phase-12-friction-round.md:59), ARNO·HOST (arno-host/docs/runbooks/database-url-provision-and-rotation.md:69) and others. Unconfirmed against Railway: the census in step 1 confirms or drops each one. | Co-locating means moving the services into one project with one Postgres (one database per tenant). It never means pointing them at another project’s proxy. One ruling per instance. | FLIGHT, per instance, after step 1. |
4. Order. Step 1 (census), then R (retire), then D (dedupe), then C
(co-locate). Each write goes through the procedure in
railway-target-disambiguation, in this order:
- name the project;
- show its sibling services;
- match them to intent;
- write.
Each credential move goes through credential-rotation-propagation, in this
order:
- enumerate the consumers;
- write the new value;
- verify every consumer reads it;
- revoke the old value.
Alternatives considered
Section titled “Alternatives considered”- A — status quo plus tuning. Rejected. D-6 measured flat working sets, so a cap above the working set reclaims nothing.
- C — one public-proxy cluster. Rejected. Constraint 2: billed egress, and a database reachable from the internet.
- D — a VPS Postgres tier. Deferred to its own decision, per the Sevalla finding’s lever 6.
- Write the 19-row table from repo evidence alone. Rejected. Constraint 1: a table built from what repos say about addresses is exactly the identification method the disambiguation skill refuses.
Consequences
Section titled “Consequences”For code: none. A co-location under class C changes the moved services’
DATABASE_URL reference. That change goes in each owning repo’s PR, under
its own review.
For docs: the step-1 census lands in eva-hq as a dated fleet doc beside
docs/eng/fleet/railway-cost-2026-10-21.md. Each class-C ruling is appended
here as a scribe note (landing-authority M3) or written as its own decision.
For in-flight phases: none paused. Two known additions to the count:
- the P3 live TOLL gate brings its own Postgres (null0-toll/toll
docs/hardening/p3-production-cutover-checklist.md, null0-toll/toll#159); - that instance is class K from birth. The P3 decision forbids sharing it.
For invariants: none. The disambiguation rule is a skill and a recorded near-miss. It is not a numbered invariant, and this doc does not make it one (MR-9).
What this decision does NOT do:
- It does not read or write Railway.
- It does not delete a volume.
- It does not touch DEVARNO·CLOUD Redis. That has its own levers: hubble#90 is deployed and RF-10 has the 2026-10-22 read pending.
- It does not decide any project’s retirement. That is
2026-10-21-railway-projects-so1-tektree-skyflow.md.
References
Section titled “References”- eva-hq/eva
docs/eng/fleet/railway-cost-2026-10-21.md(the 19 count, $15.88 and 30%, top-12 rows, lever 3) - eva-hq/eva
docs/findings/20261021-1330-sevalla-eval-railway-cost.md(Sevalla declined; lever ordering; fleet-binding decisions routed to petrova-codes) - Ledger
~/Documents/clutter/grug/task/runtime-cost-fanout.md(operator-local): M1 egress and private networking; R7; D-5; D-6 “container COUNT”; B-6; RF-09 “count by image”; RF-10 - mary-wiki/mary
docs/decisions/2026-08-14-go-asgard-rotation-and-215-deploy.md(the near-miss behind the disambiguation rule) - skyflow-blue/skyflow-hq
MILESTONES.mdM6.3 (HARD ORDER D3 on the.medatastore) - devarno-cloud/devarno-cloud
docs/superpowers/plans/2026-10-08-retire-so1-io.md(R4: mission.so1.io on Railway, “disambiguate the project first”) - null0-toll/toll
docs/decisions/2026-10-14-p3-production-polar-via-a-separate-gate.md(a second TOLL Postgres, unshared by design) docs/decisions/2026-10-21-railway-projects-so1-tektree-skyflow.md,docs/decisions/2026-10-21-stratt-orchestrator-http-crashed.md(sibling drafts)
Sign-off
Section titled “Sign-off”- Subagent: Claude Opus 5.5, 2026-10-21 (drafted from the eva-hq census, the Sevalla finding and the cost ledger; no Railway read or write)
- Human countersign — picks option A, B, C or D; if B, approves the step-1 read-only census, and confirms the class-K list — flight@arno.host, 2026-10-27 (jzero.run T1)