N-01's alert budgets are separate from the console's display threshold
Date: 2026-08-15
Status: open
Supersedes: none — proposes a settlement for Q-ALERT-1
Superseded-by: none — current
Terminal state of this record: a measured proposal and a built classifier.
The thresholds are unratified; Q-ALERT-1 assigns the ruling to FLIGHT and
this record is the crew’s proposal, not the ruling.
Context
Section titled “Context”Q-ALERT-1 (docs/findings/20260811-1749-ptv-scf-0001-p0-open-questions.md:232-249)
blocks N-01 on one question — what distinguishes a host outage from a late
sweep — and states the settlement: “the watchdog must read something the sweep
writes and something the host’s own liveness proves, so the two failure modes
separate. A threshold review is also required; 2h is currently below the observed
healthy maximum.”
Both halves are addressed here. The classifier is built; the threshold is proposed with the measurement behind it.
The measurement
Section titled “The measurement”59 consecutive gaps between commits touching state/, read from git history on
2026-08-15:
min 22m | p50 56m | p90 1h37m | p95 1h48m | max 2h59mover 2h: 2 of 59 (3.4%) over 3h: 0 over 4h: 0The declared cron is */30 (.github/workflows/state-sweep.yml). Observed p50
is nearly twice that — GitHub deferring scheduled workflows under load, which is
expected rather than a fault.
This confirms D-10 on the numbers. host/src/freshness.ts:16 sets a 2h
threshold, and its comment claims the window absorbs observed gaps of “~2-3h”,
which cannot be true of a 2h window. Against the sample, 2h is exceeded by 3.4%
of healthy gaps. A watchdog built on it fires on healthy runs — the
automation-candidates document’s own words, “a check that will cry wolf on day
one”, and it would have done so roughly twice in this sample.
Decision (proposed)
Section titled “Decision (proposed)”N-01 gets its own budgets. freshness.ts is left alone.
The reason is not that 2h is wrong. It is that one number is being asked to do two jobs with opposite error preferences:
| Question | Wants | Cost of being wrong | |
|---|---|---|---|
Display staleness (freshness.ts) | is what I am looking at current? | sensitivity | cheap — a reader sees a label and applies judgement |
| Alert staleness (N-01) | should this wake someone? | specificity | expensive — spends the alert budget, trains the operator to ignore it |
A single threshold cannot be tuned for both, and tuning it for display is precisely why it is wrong for alerting. So the console’s 2h label stays defensible as a display choice, and the watchdog does not inherit it.
| Budget | Proposed | Reasoning |
|---|---|---|
sweepDeadMs | 3h30m | Observed healthy max is 2h59m; this clears it by ~17%. Not 3h, which the data touches. Not 6h, which would let a dead sweep sit through most of a working day. Fires zero times against the measured history, so any firing is information rather than noise. |
snapshotLagMs | 6h | This condition is currently true in production and is an estate fault (D-09/D-12), not a transient. Set above a normal deploy interval so it reports a genuinely stuck snapshot rather than the minutes between a sweep commit and the next build. |
wroteNothingMs | 3h | Above healthy max, below sweepDeadMs, so a sweep running without committing is caught in the window before the dead alert would fire. Narrow by construction. |
watchdogSilenceMs | 3h | The watchdog is specified hourly, so three missed runs. Two would fire on a single deferred run — the same wolf-crying mistake one layer up. |
They live in host/src/watchdog/thresholds.ts as exported data, with the
measurement beside them, so a ruling changes one file rather than a search.
Five conditions, not one “stale”
Section titled “Five conditions, not one “stale””The automation document names three states the watchdog must separate; D-11 adds a fourth; the dead-man’s-switch is the fifth. Each has a different fix, so each is a different condition. Collapsing them means re-deriving the distinction every time the alert fires, which — as the operator put it — you will stop doing by the fourth firing.
| Condition | What it means | Why it is not the others |
|---|---|---|
watchdog_silent | the watchdog itself has stopped completing | reported ahead of everything else: if the watcher is dead, nothing else it says is current |
host_unreachable | the host did not answer | not a sweep fault; sweep logs will look fine, and D-11 is exactly this being misread as staleness |
sweep_dead | no write to state/ past budget | the sweep is the fault; check the workflow’s run history |
console_reading_stale_snapshot | sweep wrote, served surface is behind | the sweep is healthy — this is the build-time freeze, and reporting it as a sweep fault sends the operator to the wrong logs |
sweep_ran_wrote_nothing | running on schedule, committing nothing | looks healthiest from outside; every freshness check passes while nothing is measured |
Ordering is deliberate: a condition that explains the ones below it is reported instead of them.
The dead-man’s-switch, and what it does not cover
Section titled “The dead-man’s-switch, and what it does not cover”The watchdog writes a heartbeat on every completed run, and classify() reports
watchdog_silent when that heartbeat ages past budget.
This is partial, and is recorded as partial. A process cannot report its own total silence. What is closed:
- a watchdog that runs but fails before completing — caught on the next run.
- a watchdog that has stopped entirely — caught only if something other than the watchdog reads the heartbeat.
What is open: if the watchdog never runs and nobody reads the heartbeat, nothing fires. Closing that needs a third party which expects a heartbeat and complains at its absence — an external cron monitor, or the console treating a stale heartbeat as a first-class verdict rather than as a field.
Stating this rather than claiming a complete switch is the whole point: a dead-man’s-switch that overstates itself converts an unknown into a false assurance, which is the exact failure it was built to prevent. Raised as F-43.
Alert discipline
Section titled “Alert discipline”Notify once, with the age; then quiet until recovery or a second, longer threshold. Recovery is notified too — without it, “fixed” and “the watchdog also died” are indistinguishable, which re-creates the ambiguity the switch exists to remove. Suppressed alerts still appear in the run record; a suppressed alert must remain observable.
Alternatives considered
Section titled “Alternatives considered”- Reuse
STALE_THRESHOLD_MS(2h). Rejected on the measurement: it fires on 3.4% of healthy gaps. A test pins this — if someone later “simplifies” by reusing it, the test that breaks says why. - Raise
freshness.tsto 3h30m and share one number. Rejected: it makes the console less informative to make the alert quieter, trading a cheap error for an expensive one in the wrong direction. Display and alerting want opposite tunings. - One
sweep_stalealert with detail in the body. Rejected — this is the thing the automation document specifically warns against, and the operator’s own framing is the argument: a single signal makes you re-derive the distinction on every firing, and by the fourth you will not. - Ratify by measurement alone, no human ruling. Rejected:
Q-ALERT-1assigns the threshold to FLIGHT. The crew proposes; measurement makes the proposal defensible, it does not make it ratified.
Consequences
Section titled “Consequences”host/src/watchdog/{classify,thresholds,notify,heartbeat}.ts, 21 tests. Pure functions, no I/O — the hard part is which state are we in, and that is now testable without a network.- Not yet wired. No workflow, no schedule, no notification transport. Wiring a watchdog to fire on unratified thresholds would put the alert budget at risk on numbers nobody ruled. That is the next act, after the ruling.
freshness.tsuntouched, deliberately.- D-10 and D-11 both have a mechanism that addresses them; neither is closed here — closure is a verification round’s act.
References
Section titled “References”docs/findings/20260811-1749-ptv-scf-0001-p0-open-questions.md:232-249— Q-ALERT-1.docs/findings/20260811-1749-ptv-scf-0001-p0-delta-register.md— D-09, D-10, D-11, D-12.docs/PETROVA-AUTOMATION-CANDIDATES.md:81-99— N-01, and the three-state requirement.host/src/freshness.ts:9-16— the display threshold and the inconsistent comment..github/workflows/state-sweep.yml— the declared*/30cadence.
Sign-off
Section titled “Sign-off”- Subagent: PTV-SCF-0001 (session 2026-08-15)
- FLIGHT: ☑ (proxy) ruling — the four budgets, the decision to keep alert and
display thresholds separate, and the partial dead-man’s-switch as described.
-
Ruled by
human:devarnoon 2026-08-15, by explicit directive in session (“Ruling all four, with the measured basis as stated, and the revision schedule in the sign-off rather than a separate doc”). Ticked by the agent as scribe, not as signatory, perdocs/decisions/2026-08-15-proxy-countersign.md. -
The ruled values are the CORRECTED ones, not the four in this record’s body. The body proposed
sweepDeadMs3h30m andwroteNothingMs3h off a 59-gap sample that began at the end of a 27.9-day outage and therefore measured the recovery rather than the history. That defect and its corrections aredocs/findings/20260815-1955-sweep-dead-90-percent-of-window.md(PTV-FND-0027). The body is left as written — a superseded proposal with its error visible is worth more than a silently corrected one (L3,MR-7). Ruled, binding:Budget Ruled Basis sweepDeadMs4h in the clean gap between the jitter population (max 3h32m) and the smallest real outage (just over 4h) snapshotLagMs6h above a normal deploy interval wroteNothingMsnull — disabled not observable from commit timestamps at all; needs the workflow run history (F-44) watchdogSilenceMs3h three missed hourly runs host/src/watchdog/thresholds.tsalready carries exactly these. -
Revision schedule, recorded here rather than in a separate doc at the operator’s direction: these are ratified as binding, not as correct. They rest on one 106.9-day sample whose two populations may not be stable — the estate spent 90.3% of that window with a dead sweep, so the jitter population is measured from a minority of the record. Re-derive both populations and propose revisions when either holds: (a) 90 days elapse from 2026-08-15, or (b) any alert fires whose gap falls inside the jitter population, which is the signal that the boundary between the two has moved. A revision is a new record; these values do not drift silently.
-
N-01 may now be wired. That was the only thing this ruling gated. Wiring is a separate act and does not belong to the session that ruled it.
-