Product · Sentinels
Turn a completed investigation into a Sentinel.
Every deep investigation your agent completes is expensive to run once and impossible to reuse. A Sentinel is a typed, immutable, executable DAG compiled from that investigation. The exploration happens once; every run after is deterministic and costs a fraction of the original tokens.
Bind a Sentinel to a schedule and it runs as a smart alert that evaluates composite signals no threshold rule can express. Bind it to a pager webhook or a deploy hook and it runs as a triage runbook that gathers the evidence a human would pull and attributes root cause.
A Sentinel, in one line
kind
typed DAG
immutable, sha256-identified spec
source
compiled
from a completed agent investigation
modality
alert · runbook
on a schedule, or bound to a webhook
runtime
deterministic
budgeted, sandboxed, replayable
01 · How teams use them
One primitive. Two modalities.
A Sentinel is a single primitive that serves two operational jobs. Bound to a schedule, it runs as a smart alert: it evaluates composite signals no single-metric threshold can express, runs a causal check before it fires, and stays quiet the rest of the time. Bound to a webhook (a pager, a deploy, a git event), it runs as a triage runbook: it gathers the evidence a human would pull, correlates recent changes, and attributes a candidate root cause to the incident. Same DAG shape, same runtime, same replay bundle.
- · Alerts fire on the actual condition, not on a proxy signal that spiked
- · Runbooks attach evidence and attribution to the incident that opened them
- · Every finding traces back to the exact Sentinel that emitted it
- · Findings dedupe so a stuck condition does not repage every run
Compares baseline vs current error rate and latency, runs a causal check against the deployment, and files a finding only when the change is materially responsible for the degradation.
Walks downstream from the failing SLO, lines up recent config and deploy events against the burn window, and ranks candidate causes by confidence.
Fires only when queue depth, retry rate, and p99 shift line up in the same window. The pattern that no single-signal threshold catches until it is already user-visible.
Watches per-initiative token spend across every agent session. Flags spend outside the +2σ envelope and attributes it to the owning engineer and PR.
Detects concurrent misses on hot cache keys before the origin latency shows up on the p99 chart. Fires early enough to matter.
Pulls open incidents, deploys in the last 24h, and services with elevated error rates. Writes a two-paragraph brief and posts it to the on-call channel.
02 · What they look like
Three examples, wired end to end.
Three of the use cases above, sketched as DAGs. Each node is one of the six kinds shown in the next section; each edge is a hard dependency; each downstream node blocks on its upstream schema. This is the shape a Sentinel takes once /mechanize has compiled it.
- · Colors follow the node-kind palette in the next section
- · Fan-out, linear, and convergent shapes — same primitives, different procedures
- · Illustrative sketches, not artifacts pulled from a live registry
trigger · on deploy webhook
trigger · on SLO burn breach
trigger · every 5 min
03 · The compiler
Type /mechanize when an investigation is worth keeping.
An investigation is a conversation. The agent asks, queries, corrects, retries, and eventually concludes. Most conversations end there. When one is worth reusing, you type /mechanize: the compiler walks the captured session, keeps the tool calls and transforms that produced the answer, drops the false starts, and emits a typed DAG that runs again without the conversation. Every retained step carries an audit record explaining why it was kept and how its literal inputs got parameterized.
- · Immutable: identity is sha256 over the normalized spec
- · Typed: every node has an input and output schema
- · Portable: tools are abstract capability IDs, bound at runtime
- · Auditable: every node records why the compiler kept it
The compiler walks the captured session, drops the false starts, and emits a typed DAG.
04 · The node model
Six node kinds. Every step is exactly one of them.
Each retained action from the source investigation becomes a node of one of six kinds. Three infrastructural (tool, condition, emit) and three analytical (function, llm, ask_human). The choice among the analytical three follows a rule: if the judgment can be a deterministic transform over declared inputs, it is a function; otherwise, if the model can be trusted to decide without human ratification, it is an llm; otherwise it is an ask_human. That rule is what keeps a Sentinel as mechanical as the investigation actually was.
- · tool · mechanically invokable capability, resolved by ID at runtime
- · function · deterministic transform, sandboxed, content-addressed
- · llm · bounded evidence, schema-valid output, inconclusive allowed
- · condition · restricted expressions, no I/O, no eval
- · emit · findings with dedupe keys and evidence chains
- · ask_human · declared question, typed answer, timeout policy
Resolved by capability ID, not by MCP display name. Same Sentinel runs on Cardinal MCP or Prometheus MCP if both satisfy telemetry.query-timeseries.
One JSON in, one JSON out. Content-addressed source. No network, read-only artifacts, wall-clock timeout. Every generated function ships with fixtures and tests.
One task, declared evidence, token limits. Model policy, not vendor. May return inconclusive. No undisclosed tool calls. Cannot access the original conversation at run time.
Boolean ops, numeric comparisons, references to inputs and upstream outputs, and a small set of pure functions. Everything else is a function node.
Local emission to stdout, file, or webhook binding. Findings dedupe on sentinel + variation + dedupe key. Downstream routing is a Binding, not a compiler concern.
One explicit question, typed answer schema, timeout policy. Blocks downstream on that branch until answered. Preserves the human-in-the-loop moments the original investigation had.
05 · From laptop to cluster
Local first. GitOps when it earns it. Cluster when it runs.
v0 lives on the laptop that ran the investigation. .mechanize/ on disk, one CLI, no cluster required. When a Sentinel is worth keeping around, v1 promotes it: publish opens a PR against the sentinels repo, CI validates and replays, merge builds function-node images and materializes Kubernetes CRDs, and a controller runs the schedule you set. The admission webhook enforces immutability, digest match, redaction, and capability binding at apply time, so nothing lands in the cluster unreviewed.
- · Sentinel, SentinelBinding, SentinelSchedule, SentinelRun CRDs
- · Sentinels repo is additive-only; rewrites are new directories with new digests
- · GitHub Actions templates for validate and apply, generated by mechanize scaffold ci
- · Direct provisioning skips CI for local dev, never against shared clusters
on the laptop
Compile, validate, and replay in the same session the investigation ran in. No cluster required, no cloud dependencies.
into git
publish opens a PR against your sentinels repo with the spec, replay bundle, audit log, and reuse metrics attached to the body.
through CI
validate.yaml runs on every PR. apply.yaml runs on merge to main. Admission is enforced by the cluster; nothing in CI can bypass it.
on the cluster
Sentinel, SentinelBinding, SentinelSchedule, and SentinelRun are custom resources. A controller reconciles the schedule and posts findings to status.
Run the investigation once. Every future run is a Sentinel.
Install the Cardinal plugin in your coding agent, do the investigation you'd have done anyway, and type /mechanize. The Sentinel lands in .mechanize/ on your laptop. Promote to your sentinels repo when it's earned it.