STRATT orchestrator-http is CRASHED and still billed: diagnose read-only, then fix or suspend; delete only through a retirement
rank: decision outranks: []
Section titled “rank: decision outranks: []”STRATT orchestrator-http is CRASHED and still billed: diagnose read-only, then fix or suspend; delete only through a retirement
Section titled “STRATT orchestrator-http is CRASHED and still billed: diagnose read-only, then fix or suspend; delete only through a retirement”Date: 2026-10-21 Status: open Supersedes: none Superseded-by: none — current MR-compliance: MR-4, MR-7, MR-20
Context
Section titled “Context”The eva-hq Railway census for cycle 2026-09-15 → 2026-10-15 (eva-hq
docs/eng/fleet/railway-cost-2026-10-21.md) lists 7 services whose latest
deployment is not SUCCESS. The first is “STRATT orchestrator-http CRASHED
($0.74/cycle while crashed)”. Its lever 5 reads: “Fix or stop STRATT
orchestrator-http (CRASHED, still $0.74/cycle) | $0.74 | stratt owner”. The
STRATT·WORKS project total is $1.47/cycle. The census does not split the
remaining $0.73 by service.
The money is small. What sits behind the service is not. orchestrator-http
is STRATT’s HTTP surface: stratt-hq/stratt-run, services/orchestrator-http,
“the auth surface (Hono, 25 routes)”, per .github/workflows/orchestrator-http.yml:3.
It reads a Railway Postgres (services/orchestrator-http/Dockerfile:22,
DATABASE_URL, “Railway plugin”). Three consumers read it, by variable
name:
| Consumer | Where | Variable |
|---|---|---|
| stratt.works (User-B surface) | stratt-run/apps/stratt-works/src/lib/orchestrator-client.ts:23 | ORCHESTRATOR_URL, default https://api.stratt.works |
| meridian | stratt-run/apps/meridian/src/pages/api/graph/blast-radius.ts:5, .../audit/sign.ts:57 | ORCHESTRATOR_HTTP_URL |
| petrova host: StrattSource, neverland edge E9 | this repo: contracts/petrova.host.consumption.v0.yaml (stratt: GET /councils, /graph, /fingerprints) | PETROVA_HOST_STRATT_API, PETROVA_HOST_STRATT_BEARER |
So the third consumer is this control plane’s own federation. Whether StrattSource is failing in production now is not established here. That would need a production read.
The cause of the crash is not known. The last change to
services/orchestrator-http on main is 508afc3 (2026-09-10, CI moved to the
hermes pool). One known Railway failure shape matches “an app that will not
start” and costs nothing to rule in or out: a database service with no running
deployment, which shows up as connection reset by peer to *.proxy.rlwy.net.
eva-hq prompt railway-dead-service-diagnosis proves it read-only and makes
no write until the operator approves. Whether that is this crash is unknown
until its logs are read.
Decision
Section titled “Decision”Proposed for FLIGHT. Nothing is executed by merging this doc.
1. Options. FLIGHT picks one row.
| # | Option | Saving | What breaks | Reversible? | Verdict proposed |
|---|---|---|---|---|---|
| F | Fix: diagnose read-only, then restore | $0. The service runs and bills as designed. | nothing; the three consumers recover | yes | Proposed, time-boxed (§2) |
| S | Suspend: remove the active deployment and keep the service, its variables and its volume | ≈ $0.74/cycle | nothing that works today breaks, since a crashed service already serves nothing. The consumers stay down. | yes: redeploy | Proposed fallback if F’s time box expires |
| D | Delete: remove the service, and its Postgres with it | ≈ $0.74/cycle, plus the database’s share of the $0.73 rest | three consumers lose their target permanently; the database is gone | no | Rejected as a standalone act. Allowed only as the last step of a retire-a-surface plan that first removes or repoints all three consumers. |
2. The fix, time-boxed.
- Name the project, STRATT·WORKS, and show its sibling-service list. Do this
before anything else (
railway-target-disambiguation). The Postgres behind this service is one of the 19 in2026-10-21-railway-postgres-consolidation.md. - With operator approval, read the crashed deployment’s logs and the
project’s deployment states. Run
railway-dead-service-diagnosisif the symptom matches it. - If the cause is a dead database service, the prompt’s restore route
applies: redeploy it on its existing volume, then move to the private
${{Postgres.DATABASE_URL}}reference. Operator approval comes first. - If the cause is in the code, the stratt owner fixes it in a stratt-run PR.
- Time box: if no root cause is named by 2026-11-04, two weeks out, apply S, and record the reason as a scribe note here.
3. After F or S, verify.
- The latest deployment state reads SUCCESS (F), or no active deployment (S).
stratt.worksand meridian are probed by their owners.- After F, the petrova host StrattSource answers on
/graph.
Alternatives considered
Section titled “Alternatives considered”- D — delete now. Rejected. It cannot be undone. It removes a database whose contents nobody has inventoried. Three named consumers would point at nothing, one of them this control plane.
- Leave it crashed and billing. Rejected. It pays $0.74/cycle for a service that serves nothing, and it hides a consumer outage behind a cost line.
- Suspend first, diagnose later. Not the default. A suspended service has no deployment logs left to read, so suspending first can destroy the evidence for the fix. S is the fallback, not the opening move.
Consequences
Section titled “Consequences”For code: under F, possibly a stratt-run fix PR, owned by stratt. None in this repo.
For docs: the outcome of F or S is appended here as a scribe note (landing-authority M3). Under D, the retirement record supersedes this doc (MR-7).
For in-flight phases: neverland edge E9 (STRATT federation) depends on this service. Its state is reported, not changed, by this doc.
For invariants: none.
What this decision does NOT do:
- It does not read or write Railway.
- It does not change the STRATT·WORKS Postgres. That is
2026-10-21-railway-postgres-consolidation.md. - It does not decide whether stratt.works is a product the estate keeps.
References
Section titled “References”- eva-hq/eva
docs/eng/fleet/railway-cost-2026-10-21.md(census: CRASHED, $0.74/cycle; lever 5; STRATT·WORKS $1.47) - eva-hq/eva
prompts/railway-dead-service-diagnosis/(read-only diagnosis, restore on the existing volume) - stratt-hq/stratt-run
services/orchestrator-http/(Dockerfile:22,railway.json),.github/workflows/orchestrator-http.yml:3 - stratt-hq/stratt-run
apps/stratt-works/src/lib/orchestrator-client.ts:23,apps/meridian/src/pages/api/graph/blast-radius.ts:5 contracts/petrova.host.consumption.v0.yaml(strattsource, edge E9),docs/decisions/2026-07-07-e9-stratt-federation-blocked-on-read-surface.mddocs/decisions/2026-10-21-railway-postgres-consolidation.md(sibling draft)
Sign-off
Section titled “Sign-off”- Subagent: Claude Opus 5.5, 2026-10-21 (drafted from the eva-hq census and the stratt-run and petrova sources; no Railway read or write)
- Human countersign — picks F, S or D; under F, approves the read-only log read and the 2026-11-04 time box — flight@arno.host, 2026-10-27 (jzero.run T1)