> ## Documentation Index
> Fetch the complete documentation index at: https://graph.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.
# CI checks
> Adapt and improve your CI pipelines with ease
This repo dogfoods graph in its own pull-request checks: a **docs-drift gate** (an inferred exit gate as a merge check) and a **specialized reviewer** (structured findings posted as PR comments by the plan itself). Every piece ships in this repo — the plans in `./.graph/plans/`, the repo-local tools in `./.graph/tools/`, the workflow in `.github/workflows/graph-checks.yaml` — and every PR against graph runs them.
To replicate this setup in your own repository, hand the work to your coding agent.
```shellscript theme={null}
npx skills add tylerdavis/graph
```
This installs the [/graph-github-actions-setup](https://github.com/tylerdavis/graph/blob/main/skills/graph-github-actions-setup/SKILL.md) skill (Claude Code, Cursor, Codex, and most other agents). It walks the agent through choosing checks, scaffolding `./.graph/`, generating the workflow, and wiring secrets. The rest of this page explains the pieces the skill assembles.
The finished shape is the headline. Each CI job is:
```yaml theme={null}
jobs:
docs-drift:
runs-on: ubuntu-24.04
container:
image: ghcr.io/tylerdavis/graph:v0.11.0 # graph + git/jq/gh baked in
credentials:
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
steps:
- uses: actions/checkout@v4
with: { fetch-depth: 0 } # plans diff refs locally
- run: graph plan run docs_drift --input base="$BASE" --input head="$HEAD"
```
No install step (the release workflow publishes the image), no shell logic (plans own it), no output parsing (exit codes and annotations are the contract). The jobs run in seconds.
## The pieces
**Ephemeral storage.** `GRAPH_STORAGE=memory` keeps runs stateless — nothing to clean up. Plans don't need the [shape cache](/tools/shape-cache) anyway: you authored the paths.
**Repo-carried setup.** graph layers `./.graph/config.toml` over the global config and loads `./.graph/{plans,tools}/`, so the repo carries everything the checks need — config (secrets as `${ENV}` references that fail loudly when missing), plans, and the rubric tool — all reviewed like any other code.
**The github tool pack.** `[tools] packs = ["github"]` enables the [bundled tools](/tools/builtins) the plans build on: `git_changed_files`, `git_diff`, `git_file`, `git_grep`, `git_log`, `gh_pr_meta`, `gh_pr_comment`, `gh_pr_inline_comments`, `gh_pr_review_threads`, `gh_pr_thread_sync`, `gh_pr_ticket`, `gh_release`.
## `docs_drift` — an inferred gate as a merge check
The repo rule, from its CLAUDE.md: behavior changes in `crates/` must update `docs/` in the same PR. As a plan, the *decision* — "is this diff a behavioral change?" — is one judged yes/no call, and everything around it is deterministic and free:
```yaml theme={null}
steps:
- id: E0
tool_name: builtin__git_changed_files
input: { base: "{{input.base}}", head: "{{input.head}}", prefix: "crates/" }
- id: E1
tool_name: exit
input:
when: { value: "{{E0.count}}", op: eq, to: 0 }
status: success
message: no crate code changed — docs gate not applicable
output: { docs_required: false, reason: no_crate_changes }
- id: E2
tool_name: builtin__git_changed_files
input: { base: "{{input.base}}", head: "{{input.head}}", prefix: "docs/" }
- id: E3
tool_name: exit
input:
when: { value: "{{E2.count}}", op: gt, to: 0 }
status: success
message: docs updated alongside crate changes
output: { docs_required: false, reason: docs_touched }
- id: E4
tool_name: builtin__git_diff
input: { base: "{{input.base}}", head: "{{input.head}}",
paths: "crates/", exclude: "", max_bytes: 150000 }
- id: E5
tool_name: exit
input:
infer: |
Does this diff change user-visible behavior of the graph CLI …
Pure refactors, comment-only changes, test-only changes, and
internal renames do not count. Answer yes only if the behavior a
user or plan author observes would change.
{{E4.text}}
status: error
message: >-
behavioral change without a docs/ update — CLAUDE.md requires docs
land in the same PR as the behavior change
output:
docs_required: false
reason: "{{E5.reason}}"
```
**Lessons:**
* **Short-circuit before you infer.** E1 and E3 are `when` gates over [github pack](/tools/builtins) tool output — a docs-only PR exits success in milliseconds with zero LLM calls. The `infer` gate is the last resort, and it sees only the truncated `crates/` diff. Worst case per PR: one judge call on the cheap `judge` role.
* **The gate is the CI contract.** `status: error` maps to process exit code 4 — distinct from exit 1 (the plan itself broke). The job needs no branching: exit 4 fails the step, and under `GRAPH_EVENTS=github` graph prints the gate's message as a `::error::` annotation itself.
* **Every exit path emits the same shape.** Both success gates and the fall-through `output` produce `{docs_required, reason}` — callers parse one contract. On a non-fired infer gate, `{{E5.reason}}` carries the judge's reasoning into the output for free (the judge explains *why* no docs are needed).
* **Ask the judge a falsifiable question.** The prompt defines what does *not* count (refactors, test-only changes). The judge role answers "yes only when the data clearly supports it", so the framing sets the false-positive rate. Live calibration: it flagged a commit adding an env var and a JSON field ("observable by users and plan authors"), and passed a dead-code removal with its reasoning in `reason`.
## `graph_review_9000` — a composed reviewer
The reviewer is three plans and a worktree-capable tool family, split so the review engine runs anywhere and only the wrapper knows GitHub exists:
* **`graph_review_core`** — the review engine, purely local (git only). Fetches the changed files, diff, and full file contents for a change set; clusters the diff into functional groups; scouts each scoutworthy group's blast radius with bounded [`agent` steps](/plans/agent-step); runs the eight-pass rubric; returns structured JSON (`output:` finish — framing, a severity-ordered pass table, findings). It takes previously known findings as a suppression list, so callers with history get only what's new.
* **`graph_review_thread_audit`** — the thread lifecycle, GitHub-coupled. Given the prior findings read back from PR review threads, it verifies fixes against the file contents at head and applies the transitions (fixed → resolved with a reply, dismissed by a human reply → declined; insufficient evidence keeps a finding open).
* **`graph_review_9000`** — the wrapper CI runs. PR metadata, thread readback, a `decide` gate that calls the audit only when prior findings exist, the core review, and the posting — inline threads for new findings, a marker-keyed summary comment that refreshes in place. Open threads and human replies are never deleted.
The CI step is still `graph plan run graph_review_9000 --input pr="$PR"` — nothing else. The wrapper:
```yaml theme={null}
steps:
- id: E0
tool_name: builtin__gh_pr_meta
input: { pr: "{{input.pr}}" }
- id: E2
tool_name: builtin__git_diff # gate-only: is there anything to review?
input: { base: "{{E0.base_sha}}", head: "{{E0.head_sha}}",
paths: ".", exclude: "Cargo.lock CHANGELOG.md", max_bytes: 1024 }
- id: E4b
tool_name: builtin__gh_pr_review_threads # prior findings + thread states
input: { pr: "{{input.pr}}", marker: "graph_review_9000:finding" }
- id: G1
tool_name: decide # audit only when history exists
input:
if: { value: "{{E4b.has_findings}}", op: eq, to: true }
then:
tool_name: plan__graph_review_thread_audit
input: { pr: "{{input.pr}}", base: "{{E0.base_sha}}", head: "{{E0.head_sha}}",
title: "{{E0.title}}", findings: "{{E4b.findings}}" }
else:
tool_name: builtin__reshape # the audit's output contract, empty
input:
shape: { findings: [], threads: [], still_open_count: 0,
resolved_now_count: 0, declined_now_count: 0,
transitions_count: 0, has_transitions: false }
- id: E3
tool_name: exit # after thread hygiene, so reverts settle
input:
when: { value: "{{E2.text}}", op: empty }
status: success
message: nothing to review
- id: C1
tool_name: plan__graph_review_core # the engine; returns structured JSON
input: { base: "{{E0.base_sha}}", head: "{{E0.head_sha}}",
title: "{{E0.title}}", body: "{{E0.body}}",
known_findings: "{{G1.result.findings}}" }
- id: E6
tool_name: map # findings -> comment bodies + payloads
input:
over: "{{C1.findings}}"
concurrency: 8
do:
tool_name: builtin__reshape
input:
shape:
path: "{{item.file}}"
line: "{{item.line}}"
payload: { label: "{{item.label}}", severity: "{{item.severity}}", ... }
body: |
> [!{{item.alert_type}}]
> **{{item.severity_label}}: {{item.finding}}**
> {{item.explanation}}
- id: E7
tool_name: builtin__gh_pr_thread_sync # creates only; the audit did transitions
input:
pr: "{{input.pr}}"
head_sha: "{{E0.head_sha}}"
marker: "graph_review_9000:finding"
creates: "{{E6.results}}"
threads: []
- id: E8
tool_name: builtin__gh_pr_comment
input:
pr: "{{input.pr}}"
marker: "graph_review_9000:summary"
body: |
> [!{{C1.overall_alert_type}}]
> **graph_review_9000** · {{C1.overall_framing}}
| Pass | Area | Status |
|---|---|---|
{{#C1.passes}}| {{number}} | {{area}} | {{status_emoji}} {{status_label}} |
{{/C1.passes}}
🧵 {{E7.created_count}} new · {{G1.result.still_open_count}} open ·
{{G1.result.resolved_now_count}} resolved · {{G1.result.declined_now_count}} declined
```
Inside `graph_review_core`, the engine keeps the shape its predecessor established — complete file list (`repo_changed_files`), diff (`repo_diff`), full contents mapped 8 at a time (`repo_file`), an empty-diff exit, a grouping inference, a `filter` to scoutworthy groups, one context scout per group (an `agent` loop over `repo_grep`/`repo_file`, up to six in parallel), then the eight-pass rubric with the caller's `known_findings` as a suppression list, finishing with an `output:` map.
### The same engine, locally
The core's evidence tools are `user__repo_*` exec tools in `./.graph/tools/` — copies of the github pack's `git_*` tools with one addition the pack versions don't have: an empty `head`/`ref` means **the working tree**. `repo_changed_files`/`repo_diff` diff staged + unstaged changes against the merge-base of `base` and HEAD; `repo_file`/`repo_grep` read and search the checkout as it sits on disk. So the same plan CI runs over committed shas reviews uncommitted work:
```shellscript theme={null}
# review the working tree against main (stage new files first: git add -N .)
graph plan run graph_review_core --input base=main
# review a committed range
graph plan run graph_review_core --input base=origin/main --input head=HEAD
```
Structured findings print to stdout as JSON — pipeable to `jq`, or actioned directly by an agent calling `plan__graph_review_core`. In this repo the intended rhythm is a finalize hook: an agent finishing a worktree runs the core, addresses what it finds, and only then opens the PR that triggers the wrapper.
Before trusting a plan like this with write access, step through it on a real PR in the [workbench](/workbench/plan-workbench) — a breakpoint on the posting steps shows exactly what would be sent:
**Lessons:**
* **The rubric is the product.** The `infer` instruction enumerates *this repo's* invariants pass by pass and instructs "omit findings that found nothing to flag". A focused reviewer beats a generic one precisely because it refuses to comment on everything else. Its predecessor caught real bugs in this repo's own PRs — including one in the PR that changed the reviewer itself.
* **The rubric can live in the plan.** `builtin__infer` accepts both the instruction and a caller-supplied `output_schema`, so the whole rubric is one step inside `graph_review_core` with no prompt tool to keep in sync. (The retired `pr_review` split them into a `./.graph/tools/` prompt tool — still the right shape when several plans share one rubric.)
* **Sub-plans return data; branches must agree on a contract.** A called plan with an `output:` finish hands its rendered JSON straight to the caller's step — `{{C1.findings}}` is the core's findings array, no unwrapping ([composition](/plans/overview#composition)). The `decide`'s two branches exploit the flip side: everything downstream reads `{{G1.result.*}}`, so the else-branch `reshape` emits the audit plan's exact output keys as empty literals — the rejoined plan can't tell which branch ran, which is the whole point of [branching](/plans/branching).
* **Gate expensive history work on whether history exists.** A first review has no threads to audit, but the audit's prompt carries the full diff and file contents — review-sized money for a guaranteed-empty answer. The `decide` on `{{E4b.has_findings}}` makes the skip structural: a first-run PR pays for the review and nothing else.
* **Presentation is a template, not code.** `builtin__reshape` projects each finding into its comment body and the summary renders with [sections](/plans/template-language#sections) — iteration builds the pass table, inverted sections handle the clean case. Logic stays in steps; the markdown is pure presentation.
* **Ordering is asked for, not computed.** The template dialect can't sort, so the schema asks the model for `blocker_ordinal` and a severity-ordered `passes` array, and the templates iterate in the order they arrive. Logic belongs in the steps or the schema — never the markdown.
* **Declare the output schema — it's enforced.** The schema pins `passes` to exactly eight (`minItems`/`maxItems`) and marks every finding field required. Models occasionally omit required fields even under provider-forced structured output; graph validates the result against the declared schema and runs one `repair`-role fix-up, so `{{E5.overall_framing}}` can never be a missing key at render time.
* **Review the whole diff.** The diff is scoped to nothing (`paths: "."`) minus generated files — an early version reviewed only `crates/` and `docs/`, which blinded it to the highest-leverage changes in an infra PR: workflows, plans, and the exec tools they run.
* **Ground absence claims in the complete file list, never the diff.** The diff is byte-capped and git orders hunks by path, so on a big PR the `docs/` hunks are exactly what falls off the end — an early version reported "docs not updated" false positives on a PR whose docs edits sat past the truncation point. The fix is layered, inside the core: `repo_changed_files` (E1) feeds the reviewer a complete, never-truncated file list, the `filter` (E3b) drops deleted entries whose contents can't exist in the reviewed state — so the `map` over `repo_file` (E4) reads whole files rather than hunks without dying on a deletion, and its results stay index-aligned with `{{E3b.items}}` by construction — `repo_diff` appends a visible `[diff truncated: …]` marker when it cuts, and the rubric judges docs-parity and test-parity only from the complete list. Per-file truncation is the same trap one level down: the first run of the split reviewer asserted two output keys were missing from a 42KB plan file fetched under a 30KB budget — the keys sat past the cut, behind a visible `[file truncated: …]` marker. Hence a corollary in the finding bar (truncated evidence can never support an absence claim — 'unverified' at most, and the audit may not resolve or hold a finding on it either) and a per-file budget sized so the repo's largest reviewed files arrive whole.
* **Ask the model to refute itself.** The instruction closes with a gate: try to disprove each blocker against the diff and demote what doesn't survive, and mark a pass `unverified` rather than `pass` for anything you couldn't check. A false clear costs as much as a false blocker.
* **Findings flow through the pipeline, enriched at each step.** `reshape` (E6) adds `path`/`body`/`payload` to each finding without dropping the original fields; `gh_pr_thread_sync` (E7) passes those extras through and adds `url` where a comment posted. So the blockers list (E8) renders from one array — severity and label from the review, the link from the post — with no join logic, which the [template dialect](/plans/template-language) couldn't express anyway. Findings the model couldn't pin to a line (or whose line the API rejects) simply come back without `url`, and the inverted section renders them as plain text.
* **Findings are threads with a lifecycle, not comments to repost.** Each finding's comment embeds a hidden JSON payload (its label, severity, file, line, headline); `gh_pr_review_threads` (E4b) reads them back next run with thread states — open, resolved, declined (a human closed or dismissed it). Open threads and human replies are never deleted, so a conversation on a finding survives every push.
* **Resolution is judged against the code, not against re-detection.** The audit plan transitions an open finding only when the code at head shows the fix, and treats insufficient evidence as "stays open" — if it instead resolved whatever the reviewer failed to re-find, one flaky review pass would silently close real findings. Declines come only from explicit human replies, never from silence. The core's rubric then sees the audited findings purely as a suppression list (`known_findings`): report only what's new.
* **The file contents at head are the resolution evidence, not the diff.** A fix that lands in a later commit of the same PR leaves no hunk in the cumulative base-to-head diff — the flagged region simply stops appearing — so the audit is told to judge from the finding's location in the full file contents. Live calibration: the first mixed-state run kept a fixed finding open precisely because the instruction let the model wait for a diff hunk that could never exist.
* **Blast radius is scouted, not guessed — one scout per functional change.** The diff and changed-file contents can't show who *calls* what changed. Inside the core, an inference (E4e) clusters the diff into functional groups (newline-joined `file_list`/`symbol_list` strings keep the per-group templates logic-less), then a `map` (E4f) runs an [`agent` step](/plans/agent-step) per group — a bounded loop over read-only `repo_grep`/`repo_file` chasing that group's callers, implementations, and tests in the reviewed state — up to six in parallel, each returning a dossier that echoes its group name. The chain sits after the empty-diff gate (it only earns its cost when there's something to review), and E5 renders each dossier behind a `{{#final}}` guard, so a scout that exhausts its budget costs its group's evidence rather than the run. Focused scouts dig deeper than one generalist — deep enough that the round budget, not curiosity, has to be the limit.
* **Scouts are dispatched only where code changed.** A prose-only group — prompts, docs, configuration — has nothing a repository search can find, and a scout pointed at one will grind its whole round budget grepping fragments of English (live calibration: a prompt-only PR burned 60 rounds finding nothing before this guard existed). So the grouping call marks each group `scoutworthy`, a `filter` (E4e2) drops the rest before the map spends a round, and E5 sees which groups went deliberately unscouted via the filter's `dropped` half — selection that narrows what runs next without losing what is known. The scout prompt keeps a bail-immediately instruction as the backstop for grouping misjudgments.
* **Thread hygiene runs before the nothing-to-review gate.** The readback (E4b) and the decide-called audit plan — which applies its transitions internally — sit ahead of the wrapper's empty-diff exit, and the sync's two halves are split across it (the audit transitions, E7 creates). A fully reverted PR has an empty diff — exactly the case where every open finding just got fixed — so gating the lifecycle behind the exit would strand resolved-in-practice threads open forever, blocking merges on repos that require conversation resolution.
* **The summary comment still refreshes in place.** `gh_pr_comment` (E8) takes its own `marker`, embeds it as a hidden tag, and find-and-edits the comment carrying it. Prefer a marker even for a lone plan: without one it falls back to `--edit-last`, which keys on the token identity, so the day a second plan comments on the same PR as the same bot the two summaries start overwriting each other. One marker per plan per comment surface.
* **Inline comments stay advisory — but resolution now means something.** They post as plain review comments, not a `REQUEST_CHANGES` review, so they only gate merging on repos with *Require conversation resolution before merging* enabled — and on those repos the lifecycle pulls its weight: fixed findings resolve themselves out of the gate's way, while a finding the author disagrees with is one human "resolve conversation" click (read back as `declined`, never re-raised).
## The workflow
The full file is `.github/workflows/graph-checks.yaml`; the notable choices:
* **The release image is the runtime.** `container: ghcr.io/tylerdavis/graph:vX.Y.Z` carries graph plus `git`, `jq`, and `gh` (pinned from official releases — distro packages lag the JSON fields and flags the pack tools use) and the `safe.directory` git config that Actions' foreign-owned workspace mounts require. The image tag is the version pin; `packages: read` plus `GITHUB_TOKEN` credentials pull it while private.
* **`GRAPH_EVENTS=github`** makes a failing plan annotate the run itself — the one sanctioned case of graph writing non-deliverable output to stdout, failure-paths only ([details](/reference/scripting-contract#streams)).
* **Guards are cheap and layered**: drafts skipped, fork PRs skipped (they have no `ANTHROPIC_API_KEY`; if one slipped through, the unset `${VAR}` makes the provider unusable and the first model call fails naming the variable — the backstop), one concurrency group per PR with cancel-in-progress.
* **Cost per PR**: worst case one `judge`-role call (docs-drift) plus two review-sized calls — the review itself, and the transition audit only when prior findings exist (each carries full file contents alongside the diff; the audit needs the same grounding to verify a fix). A first review pays for the review alone — the `decide` skips the audit outright; drafts pay nothing.
* **One job per plan, no `needs` between them.** The two jobs are independent and run in parallel; only docs-drift can fail the check (exit 4). The reviewer always exits 0 — its comments are the deliverable, not a gate.
Want the reviewer to comment under its own name and avatar instead of **github-actions\[bot]**? That's a token swap — see [A custom bot identity](/cookbook/bot-identity).
## Related topics
- [Built-ins](/tools/builtins.md)
- [Automation recipes](/cookbook/automation.md)
- [Introduction](/getting-started/introduction.md)
- [Scripting contract](/reference/scripting-contract.md)
- [Quickstart](/getting-started/quickstart.md)