If it pages, it's real. If it's silent, it's healthy.
Unattended automation makes two promises to the human it serves, and the second is the hard one. Anyone can send alerts; the discipline is earning the right to be believed — never paging on a blip, never going quiet on a real failure, and never letting 2am pay the price of either. This is the ops fleet behind the (fictional) fund from Editions 002 & 003. Break it six different ways and watch two feeds diverge: everything the system saw, and the very few things that reach the phone.
Honest-AI note. Most of this page is deterministic engineering, not AI — that's the point. The event streams are computed live in your browser by the same rules described (probe → retry → confirm → dedup → quiet-hours queue); no live model call is made. The one AI touch, labelled where it appears: in production, an LLM drafts the diagnosis line inside each page — cause, evidence, suggested fix, never a bare "it failed". Those drafts are pre-written here.
Scenario: —
01Three outcomes, not two
A failed probe means unknown, not down. Retry first: blip → log and move on; still failing → CONFIRMED, page once, then dedup until the state changes. Alert fatigue isn't a volume problem — it's a classification problem.
02Quiet hours defer — never drop
Nothing pages 22:00–07:00. But a red queued at 02:00 arrives at 07:00, guaranteed — the scoreboard's "dropped: 0" is the invariant the whole design defends. Protecting sleep and protecting the truth are the same feature.
03Watch the delivery, not the job
"The script exited 0" is not "the report went out." Delivery gates hold late output rather than send it as if on time; escalating early-checks make a missed report loud before the deadline, not discovered after. And the pager itself is probed — with an e-mail fallback when the messenger dies.
Plain-language key (probe, confirmed-down, dedup, delivery gate, quiet hours)
- Probe
- A tiny scheduled check — "is this thing alive and doing its job right now?"
- Confirmed-down
- Failed at least twice across a retry window. Only this state may page a human.
- Dedup
- One incident, one page: while the state stays "down", repeat checks stay silent; recovery is a state change, so it notifies.
- Delivery gate
- A deadline after which output is held rather than sent late — readers can trust that what arrives, arrived on time.
- Quiet hours
- A window (here 22:00–07:00) when pages queue instead of firing. Deferred, never dropped.