petrova CI leaves the HERMES eu-1 fleet: gates move to a hosted nightly, crons drop to once a day

---

rank: decision outranks: []

petrova CI leaves the HERMES eu-1 fleet: gates move to a hosted nightly, crons drop to once a day

Date: 2026-10-24 Status: open Supersedes: docs/decisions/2026-10-01-deep-probe-runs-in-ci.md (partial: only its runner, the self-hosted ci-shared pool, and its 8-hour cadence. The workflow, its commit path and the 12h/48h staleness lines stand.) Superseded-by: none — current MR-compliance: MR-4, MR-7, MR-9

Context

Every job in this repository runs on the HERMES eu-1 fleet host: 35 jobs in 29 workflows, all on [self-hosted, linux, arch, x64, ci-shared]. eu-1 is being retired with no successor (devarno-cloud/hermes docs/decisions/2026-10-24-eu1-retires-with-no-successor.md, plan docs/superpowers/plans/2026-10-24-eu1-retirement-no-successor.md Task 14, lane 7). eu-1 cannot stop until no job in any org routes to it. The retire gate measures this:

$ hermesctl retire-gate --evidence ~/retire-eu1/lanes/petrova-codes-1749 --gate routing --org petrova-codes
GATE RESULT OFFENDERS
routing FAIL 35
G1 petrova-codes/petrova .github/workflows/agents-snapshot.yml snapshot labels=self-hosted,linux,arch,x64,ci-shared
...                                  (all 35 in petrova; templates, wip-cap-fixture and petrova-canary have none)
exit=1

The evidence was collected fresh on 2026-10-24 at 17:49Z, with 0 failed fetches.

The replacement is GitHub-hosted ubuntu-latest. This repository is private on the Free plan, so hosted minutes are metered. The retirement plan caps each org at 1,500 hosted min/mo (gate G4). Today petrova-codes uses 0. The retirement decision’s D-2 rules on scheduled loops. A loop that runs more often than once a day moves to a hosted run once a day. The only exception is a loop the operator names for Mac launchd, and none is named. The cron inventory (devarno-cloud/hermes docs/findings/20261024-1600-ops-crons-on-hermes.md) lists mary-diagnose.yml (hourly) and state-sweep.yml (every 30 min) for an operator call.

Three facts about this repository bound the change:

1. No pre-merge check here has ever been required. The branch-protection API returns 403 on this plan (“Upgrade to GitHub Pro”, re-checked 2026-10-24). CLAUDE.md says so: a red docs-invariants PR can still be merged. The local backstop is scripts/git-hooks/pre-push, which runs petrova validate petrova-codes, the same check as docs-invariants.yml.
2. Per-PR hosted CI does not fit the quota. Over the 30 days to 2026-10-24, pull-request runs of the gate workflows would have cost about 1,140 hosted minutes. integration-spine-ci.yml alone was 84 PR runs × 7 job-minutes. Its push-to-main trigger has no paths filter, so it ran 194 times. Neither fits beside the crons.
3. Two checks are tied to the pull-request event. registry-slug-scope.yml reads github.event.pull_request.base.sha, the title and the body. mary-diagnose-eval.yml diffs the prompt pin against the PR base. The other 11 gate workflows read the whole tree and no PR context. This was checked: their scripts make no origin/main, merge-base, GITHUB_BASE_REF or refs/pull reads. The one exception is doc-ids.mjs, which scans main and every open PR, so a nightly run still covers the whole namespace.

Decision

1. Every job leaves the fleet. All 35 jobs move to runs-on: ubuntu-latest. No job goes to the Mac lab. The heaviest job is the deep probe, which takes about 4 minutes.

2. Gate CI becomes a hosted nightly batch. A new nightly-gates.yml runs at 03:37 UTC. Its decide step is copied from the merged references (stratt-hq/stratt-run#362, traceo-cat/traceo-mcp-server#385). It compares HEAD with the head of the last finished nightly-gates.yml run on main, whether that run was green or red:

• unchanged: skip;
• moved, or no finished run yet, or a base_sha override: run.

Because a standing red is not re-run until main moves, it costs one batch, not one batch every night.

The 11 whole-tree gate workflows drop push and pull_request and become workflow_call + workflow_dispatch. The batch calls them:

• citation-resolution
• doc-id-collision (the call grants pull-requests: read)
• docs-invariants
• ignored-tracked
• integration-spine-ci
• law-fetch-evidence
• markdown-structure
• panto-expressions
• probe-run-evidence
• projection-drift
• workflow-scope

The PR-era concurrency block in each callee is removed. In a called workflow, github.workflow is the caller’s name, so every callee would have shared one group and cancelled the others.

3. The two PR-bound checks keep pull_request, on hosted runners. They are registry-slug-scope.yml and mary-diagnose-eval.yml. Their paths filters are unchanged. Moving them to a nightly would make them vacuous, because they have no base to diff against.

4. Event-driven workflows keep their events.

• sync-docs.yml keeps push to main, with paths. It is a live publish.
• main-red-alert.yml keeps workflow_run. Nightly gates is added to its list, because a failed callee surfaces as a failed Nightly gates run.
• state-sweep.yml keeps repository_dispatch (petrova-state-stale).
• Both sync-* mirrors keep their repository_dispatch types.
• multi-owner-app-auth-e2e.yml and wip-cap-e2e.yml stay dispatch-only.

5. Cron cadences. These follow D-2’s default. Times are UTC and staggered off :00. Each one is ordered before the 08:30 digest, which reads the last 24 hours.

Workflow	Before	After	Note
state-sweep.yml	*/30 * * * *	7 6 * * *	operator call (OC-1); after the deep probe
mary-diagnose.yml	11 * * * *	11 7 * * *	operator call (OC-2); after mary-wiki’s 06:41 snapshot
deep-probe.yml	23 */8 * * *	23 5 * * *	before the sweep, so the sweep’s probe-age check reads it fresh
credential-liveness.yml	41 */6 * * *	41 5 * * *
sync-playbook-from-eva.yml	17 */6 * * *	17 4 * * *	repository_dispatch kept
sync-eva-consumption-schema.yml	37 */6 * * *	37 4 * * *	repository_dispatch kept
fleet-snapshot.yml	0 7 * * *	unchanged	already daily
agents-snapshot.yml	30 7 * * *	unchanged	already daily
deploy-batch.yml	0 8 * * *	unchanged	already daily
scheduled-workflow-digest.yml	30 8 * * *	unchanged	already daily
irina-cycle.yml	0 9 * * 1	unchanged	weekly is already under once a day; see OC-3
irina-history-freshness.yml	30 12 * * 2	unchanged	weekly
nightly-gates.yml (new)	none	37 3 * * *	the gate batch

6. What breaks, flagged and not widened. No threshold is changed by this decision. Each line below names a reader whose premise was the old cadence. Each needs either its own ruling or an operator call.

#	Reader	Premise	Effect at once a day
B-1	host/src/freshness.ts STALE_THRESHOLD_MS = 2h. Ratified by 2026-08-05-sweep-heartbeat-and-stale-threshold.md, whose basis is the */30 cadence. It feeds the console’s ”⚠ stale” card, petrova.sweeps.freshness and the scene line.	sweep every 30 min	Every repo reads stale for about 22 hours a day.
B-2	junction-zero-run src/lib/vendored/stale-threshold.ts, a vendored copy at 2h, and constants.ts EXPECT.sweepRun (30 min + 2h) and repoSweep.	sweep every 30 min	jzero /health and the estate pages read stale for most of the day.
B-3	cli/src/verbs/petrova_act_reconcile_drift.ts refuses with STALE_SWEEP_REPORT when last_full_sweep is more than 60 min old.	a sweep is always under 60 min old	Every reconcile_drift needs a sweep first, through petrova.sweeps.trigger or gh workflow run state-sweep.yml, about 3 hosted min each. This was already true between ticks. Now it is true almost always.
B-4	host/src/watchdog/thresholds.ts (N-01) sweepDeadMs = 4h, PROPOSED and unratified.	jitter population under 4h	Once wired, it would raise sweep_dead every day.
B-5	scripts/check-probe-staleness.mjs, which state-sweep runs with --warn-hours 12 --error-hours 48. The 2026-10-01 decision chose 8h to leave “five missed runs of slack before the 48h error”.	probe every 8h	Ordered 05:23 → 06:07, the sweep sees a probe about 1h old. Slack before the 48h error drops from five missed runs to one.
B-6	jzero EXPECT.probe = 24h, a jzero policy.	probe at least every 24h	The newest probe sits near 24h before each run. GitHub cron jitter (p50 about 1h, thresholds.ts) makes it read stale for part of each day.
B-7	host/src/sources/mary-diagnose.ts reads governance-diagnose-<run_id> for mary-wiki’s newest governance-snapshot run, which runs every 6h on hosted runners.	a tick every hour	3 of 4 snapshot runs get no diagnosis. The mary-wiki governance page shows no GENERATED statement for about 18h a day. Model spend falls by the same factor.
B-8	irina-cycle.yml “Dead-man ping” posts to vars.DEADMAN_PUSH_URL = http://100.121.65.77:8428/…, a tailnet address. The reader is deep-space-network deploy/durable/vmalert/irina.yml (IrinaCycleStale and its absence arm, 14 days).	the runner is on the tailnet	A hosted runner cannot reach it. The step is continue-on-error, so the cycle stays green, but no sample lands. IrinaCycleStale fires about 14 days after the flip. Operator call (OC-3).
B-9	Mirror lag for sync-playbook-from-eva and sync-eva-consumption-schema.	6h	Up to 24h. repository_dispatch would cut that to minutes, but no run in the last 30 days came from a dispatch, so eva-hq is not sending one.
B-10	credential-liveness detection latency.	6h	Up to 24h. Red runs already reach #petrova only through the daily digest, so the page latency barely moves.

7. The governance guarantee that changes. Before this change, every PR touching a gated path ran its gates before merge. Those runs were advisory, because nothing could require them (Context 1), but the PR showed red.

After it, the 11 whole-tree gates run after merge, once a night, against main. A red night pages #petrova through main-red-alert (Nightly gates failed) and appears in the next digest. Pre-merge checking becomes a convention: gh workflow run <gate>.yml --ref <branch>.

Three things still check before merge:

• the pre-push petrova validate hook;
• registry-slug-scope and mary-diagnose-eval on their PRs;
• jzero’s countersign flow, which lists only merged records and so is unaffected.

The two-class landing authority (2026-10-10-two-class-landing-authority.md §2) lets an M lander land a PR when every check on its head is success, skipped or neutral. With no PR checks, that condition holds vacuously. So whoever builds landing-class (§5) must also make the M lander dispatch docs-invariants.yml on the branch, and require it green, before landing. LANDING_MECHANICAL_ENABLED is not 1 today, so nothing lands mechanically yet.

8. Quota. Billing rounds each job up to the whole minute. The costs below come from 8 recent runs per workflow on eu-1. The conservative column uses each job’s slowest run, and the expected column its median. state-sweep commits to main every day, so main moves every night and the batch always runs.

Item	Runs/mo	Conservative min/mo	Expected min/mo
nightly-gates (decide 1 + 11 callees: spine 7, projection 2 or 1, nine others 1 each)	30	570	540
state-sweep (30 cron + about 20 repository_dispatch, 3 min)	50	150	150
deep-probe (5 max / 3 median)	30	150	90
agents-snapshot (2)	30	60	60
fleet-snapshot (2 max / 1 median)	30	60	30
credential-liveness, deploy-batch, scheduled-workflow-digest, mary-diagnose, sync-playbook-from-eva, sync-eva-consumption-schema (1 each)	6 × 30	180	180
irina-cycle (resolve 1 + 7 slugs × 2 + record 1)	4.35	70	70
irina-history-freshness	4.35	5	5
sync-docs (push to main, 30-day rate)	82	82	82
main-red-alert (only red completions run the job)	about 40	40	20
registry-slug-scope (PR, 30-day rate)	33	33	33
mary-diagnose-eval (PR, 30-day rate; pin 1 + two evals)	9	45	36
dispatch-only e2e	about 2	2	2
Total		1,447	1,298

That leaves 53 min/mo of headroom on the conservative figure, against the 1,500 limit. It is thin. If hosted runs prove slower than eu-1, or G4 projects over 1,500, these levers apply in order. None is applied by this decision.

1. Skip the batch when the only commits since the last run are the sweep’s chore(state) commits. This saves up to 540 min/mo.
2. Debounce petrova-state-stale dispatches.
3. Move deep-probe to the Mac lab.

9. Sequencing.

1. This decision merges open, so jzero can list it.
2. FLIGHT countersigns in jzero.run /approvals.
3. The workflow PR merges. It carries [skip ci] on every commit and on the squash subject.

The workflow PR proves its callees by dispatching them on its branch. It dispatches only workflows that write nothing and notify no one. It never dispatches a cron that writes state or posts. nightly-gates.yml cannot be dispatched before it exists on main. Its first scheduled run, at 03:37Z the night after the merge, is its proof. The scribe close (open → closed) follows the countersign, as usual.

Operator calls, with defaults

Call	Question	Default if not ruled
OC-1	Does state-sweep run once a day (B-1 to B-5), or does FLIGHT name it for Mac launchd at */30? No hosted cadence faster than daily fits: every 6h adds about 270 min/mo.	once a day
OC-2	Does mary-diagnose run once a day (B-7), or does FLIGHT name it for Mac launchd?	once a day
OC-3	How does the IRINA dead-man ping reach the DSN durable plane from a hosted runner (B-8)? (a) a public, authenticated ingest on DSN; (b) retire the ping and its alert under a DSN decision; (c) relay from the Mac lab. A tailnet auth key in Actions secrets is not offered, as in grace-hq DEC-0092.	none. The alert fires about 14 days after the flip unless (a), (b) or (c) lands
OC-4	If OC-1 stays daily, do B-1 to B-4 get a successor threshold ruling, or does the console simply read “stale” by design?	thresholds unchanged; the console over-marks

Alternatives considered

• Keep pull_request on every gate, on hosted runners. Rejected on the quota. About 1,140 PR minutes a month, plus about 880 for everything outside the batch, comes to about 2,020. That is over the 1,500 limit, and over the Free plan’s 2,000, before any push-to-main run is counted.
• Move the gates to the Mac lab. Rejected. A lab result is evidence, not a gate (the hermes decision’s Consequences), and the jobs are small. The lab is for suites too heavy for the hosted quota.
• Keep state-sweep at */30 on hosted runners. Rejected. 48 runs a day × 3 min is about 4,320 min/mo, nearly three times the whole limit.
• Widen the staleness thresholds in this change. Rejected. Each one was ratified on a measured premise (the 2026-08-05 decision, N-01). Changing it silently, inside a CI move, is the defect class thresholds.ts warns about. It is OC-4.

Consequences

For code:

• Every workflow under .github/workflows/ changes runs-on. The 11 gates change triggers, and nightly-gates.yml is added. This lands in one workflow PR after this decision’s countersign.
• No cli/, host/ or dashboard/ change.

For docs:

• CLAUDE.md: the “self-hosted ci-shared pool” line, and the “6-hourly” wording for sync-playbook-from-eva.yml, are corrected in the workflow PR.

For in-flight phases:

• hermes plan Task 14, petrova-codes lane. Its G1 goes to 0 once the workflow PR merges.

For invariants:

• No MR change. MR-7 is honoured by partial supersession of the 2026-10-01 deep-probe decision, which is not edited.

What this decision does NOT do

• It does not change any staleness threshold, alert rule or refusal limit (B-1 to B-6, OC-4).
• It does not touch eu-1, runner registrations or repo settings.
• It does not change templates, wip-cap-fixture or petrova-canary. None of them routes to eu-1.
• It does not move anything to the Mac lab or name a launchd loop. Only FLIGHT can name one.

References

• devarno-cloud/hermes docs/decisions/2026-10-24-eu1-retires-with-no-successor.md: D-2 and the retirement condition.
• devarno-cloud/hermes docs/superpowers/plans/2026-10-24-eu1-retirement-no-successor.md: Task 14 and the Global Constraints (G4 = 1,500).
• devarno-cloud/hermes docs/findings/20261024-1545-eu1-retire-gate-baseline.md and 20261024-1600-ops-crons-on-hermes.md: the 35 jobs and the 13 crons.
• eva-hq/eva prompts/gha-governed-org-nightly-flip: the procedure, decision first.
• docs/decisions/2026-08-05-sweep-heartbeat-and-stale-threshold.md: the 2h basis (B-1).
• docs/decisions/2026-10-01-deep-probe-runs-in-ci.md: the 8h and 48h basis (B-5), partially superseded.
• docs/decisions/2026-10-10-two-class-landing-authority.md: the vacuous-checks consequence (§7).
• docs/decisions/2026-08-15-n01-alert-thresholds.md: N-01 (B-4).
• grace-hq/grace#210, stratt-hq/stratt-run#362, traceo-cat/traceo-mcp-server#385 and smo1-io/purr-api#81: the reference nightly batches.

Sign-off

• Subagent: Claude Opus 5.5 (hermes plan Task 14, petrova-codes lane, 2026-10-24) (drafted from fresh retire-gate evidence petrova-codes-1749 and 30 days of run history read on 2026-10-24)
• Human countersign — the hosted destination for all 35 jobs, the nightly batch replacing per-PR gates, the once-a-day cadences (OC-1, OC-2 at their defaults unless ruled), and the flagged breaks B-1 to B-10 accepted as listed — flight@arno.host, 2026-10-27 (jzero.run T1)