Independent LLM PR review¶

The one fixed rule: the model that wrote the code never reviews it. Either Claude or DeepSeek can plan or code, chosen per task, and the reviewer is always the other one. Independent review is only a mandatory, blocking gate for high-risk paths — see CLAUDE.md, "Review scope", at the repo root, for the current list (tenant scoping, auth, secrets, migrations, billing, new models, the review loop's own workflows). Everything else ships on self-review + pytest + docs-drift alone, with a weekly DeepSeek sweep as a backstop rather than a per-PR gate.

2026-09 change of default. Reviewing every PR was the original design and it was correct — but too slow to sustain for solo pre-launch development, largely for reasons this page's own "Gotchas" section documents (async comment-triggered CI, DeepSeek's coding pace, a 20–35 minute round trip per review). The mechanism below is unchanged; what changed is when it fires.

Everything here is still a comment on an issue or a pull request when it does fire — no tools to install, same from a laptop or the GitHub mobile app.

The commands¶

Two you'll use constantly, two you'll rarely need.

Most PRs need neither review command. A PR that touches no high-risk path gets llm-review set to success straight away. On a high-risk PR: if DeepSeek wrote it, Claude reviews automatically once pytest succeeds (/review forces a re-run); if Claude wrote it — now the common case — comment /review-deepseek.

Rules that apply to all four:

• It must be the first thing in the comment. "Can you /oc fix this?" does nothing. That's deliberate: it means quoting an earlier comment doesn't set an agent off again.
• Everything after the phrase is the instruction, in plain English.
• Only someone with write access to the repo can trigger them.

Examples worth copying¶

On an issue, once the plan is agreed:

/oc implement the plan in this issue

/oc implement this, but only the model and migration — leave the
template work for a follow-up

On a PR, after a review:

/oc the review flagged an unscoped query at apps/documents/views.py:142 —
fix it and add the cross-household test

/oc this is failing pytest on test_topic_filter — work out why and fix it

/oc merge main into this branch and resolve the conflict in services.py

You can also comment on a specific line in the PR's Files tab, and opencode picks up the file and line automatically:

/oc this needs the require_owner() guard

Forcing a review (rarely needed — Claude's runs automatically):

/review

/review-deepseek

Neither takes an instruction; the brief is already written (.github/prompts/review.md).

From a terminal¶

Identical, just posted a different way:

gh issue comment 239 --repo ollyoxford/maud --body "/oc implement the plan in this issue"
gh pr comment 240 --repo ollyoxford/maud --body "/review"

From the GitHub mobile app¶

1. Open the repo → Issues (or Pull requests)
2. Tap the issue or PR
3. Scroll to the bottom, past any existing comments
4. Type into the comment box and send

It's the ordinary comment box — the same one you'd use to write a note to yourself. There is no special agent UI, no button, no menu. If a suggestion menu pops up when you type /, that's GitHub's own slash commands: ignore it and keep typing.

A worked example, end to end¶

1. Plan. Open a Claude session, agree what you're building, and have it write the issue: the plan plus acceptance criteria. Say it's #251.
2. Code. Claude writes it on a claude/<slug> branch, opens a PR (say #252) and turns auto-merge on for it.
3. Wait for CI. pytest, docs-drift and the review gate run on it.
4. Low-risk? llm-review goes green on its own and GitHub merges the PR. Nothing to do. The weekly sweep is the only independent look it gets.
5. High-risk? llm-review sits pending with "comment /review-deepseek". Comment it on #252.
6. DeepSeek posts a findings table, adds review:pass or review:changes-requested, and publishes the status.
7. Fix, if needed — back in the Claude session. The new push clears the verdict; comment /review-deepseek again.
8. Merge happens by itself once the label says review:pass and CI is green.

What's still yours: tapping Update branch when a PR falls behind main, resolving conflicts, commenting /review-deepseek on high-risk PRs, and merging anything that touches .github/workflows/ by hand. Auto-merge stays off for those PRs because a PR that rewrites the gate is judged by its own rewrite. The rule itself is in CLAUDE.md, "Auto-merge".

In the alternate mode (DeepSeek codes via /oc implement the plan in this issue), step 5 is automatic instead: Claude reviews the moment pytest goes green, and fixes go back via /oc.

Why it exists¶

Claude plans most PRs here. A model reviewing its own diff shares the blind spot that produced the bug, so the reviewer is deliberately a different model family with no shared context.

pytest and docs-drift already catch regressions and drift. What they can't catch is judgement: a misread plan, a query that skipped the scoped manager, a memory rule quietly broken. That's the gap this fills — and every merge to main auto-deploys within ~2 minutes, so there's no staging between a merged PR and real users' documents.

How it actually works¶

Three ideas, if you haven't worked with GitHub Actions before.

A workflow is a recipe file in .github/workflows/. GitHub reads it, and when something happens in the repo it rents a fresh computer for a few minutes and runs the recipe there. Nothing runs on your machine.

An event is the "something happens" — a comment posted, a commit pushed, a label added. Each recipe declares which events it cares about. Ours listen for comments — and, for the review, for another workflow finishing.

A token is a password for robots. A recipe usually needs to act back on GitHub — post a comment, add a label, push code — so GitHub mints it a temporary one, GITHUB_TOKEN, created fresh per run and destroyed after.

So /oc isn't a command in any special sense. It's an ordinary comment that a recipe happens to be watching for. That's why it works from a phone with nothing installed: you aren't running anything, you're leaving a note, and something on GitHub's servers is watching for notes shaped like that. It's also why the comment stays in the thread forever — a permanent record of exactly what you asked for.

When the review runs¶

Only when the PR is high-risk. review-gate.yml reads the PR's changed files against scripts/review-required-paths.txt (read from main, so a PR can't edit itself off the list), and also scans the diff for any added class … line in a models.py. No match → llm-review goes straight to success and nothing else happens. A match → the rest of this section applies.

On a high-risk PR, the review runs automatically when the Tests workflow succeeds — unless it's a claude/* branch, which needs /review-deepseek (see "The two modes").

Reviewing is the one stage in the loop with no decision in it — you always want the review — so a manual trigger there was ceremony. Worse than ceremony: a step you always say yes to is one you eventually forget, and the failure mode is merging unreviewed while believing you have a review process. A gate that depends on remembering isn't a gate.

Hanging it off Tests-succeeded also gets the ordering right for free. The reviewer only ever sees code the suite has already passed, so its attention goes on judgement rather than on what pytest already proves.

The stages that do hold a decision stay yours: whether the plan is right before code gets written, which findings to accept, and whether to merge. Three touch points instead of four, and the one that went is the one you'd never have said no at.

• Draft PRs are skipped — that's how you say "not yet, still pushing", without needing a command to withhold.
• Every green test run re-reviews. The verdict label is cleared on each push anyway, so anything else would leave a stale pass standing. The concurrency group cancels a review a newer push has superseded.

The two modes¶

The rule underneath everything: the model that wrote it never reviews it.

Neither mode is the default; pick per task. If Claude plans and DeepSeek codes, Claude ends up both planning and reviewing. That's fine — reviewing someone else's implementation of your own spec is not self-review, and the spec's author is well placed to spot a misreading. The gap it leaves is an ambiguous plan: Claude won't flag its own under-specification, so if the code is odd in a way that traces back to the brief, that's a plan bug.

The reviewer is chosen from the branch name rather than from your memory of who wrote what. A claude/* branch never auto-reviews, and /review refuses on it by hand too, pointing you at /review-deepseek. Those PRs sit at llm-review pending with "comment /review-deepseek" written into the status, so the one case that still needs you can't go unnoticed. The guard is only as honest as the branch name, so any branch Claude commits to is named claude/<slug> — Claude Code on the web does that automatically; from the CLI you name it yourself, which is where it's easy to get wrong.

Both reviewers read the same brief, deliberately. If they read different briefs you couldn't tell a model difference from a prompt difference.

Repeat reviews¶

A PR gets reviewed again on every green test run, so a long-lived PR accumulates reviews. The brief tells each reviewer to read the ones before it and to report only what is new or genuinely still open — a finding an independent reviewer has explicitly discharged is closed, and re-raising it is noise rather than a second opinion.

This was learned the hard way on #247, which collected four reviews carrying about one and a half reviews' worth of distinct content. Each pass re-raised findings the previous one had resolved, which is the same "manufactured findings train the reader to skim" failure the brief warns about, reached from the other direction.

The brief also tells a reviewer that finds its own model family in the commit trailers to say so and stop rather than post a findings table anyway. A thorough self-review is still a self-review, and a page of authoritative-looking analysis buries the one line that matters: this needs the other reviewer.

Trigger phrases must not overlap¶

/review-deepseek also starts with /review, so the Claude workflow carries an explicit negation. Without it, one comment fires both reviewers: double cost, and they race each other for the verdict label. Remember that before adding a third phrase.

How it's wired¶

The reviewer workflows concatenate AGENTS.md and the brief into one prompt. The rulebook is inlined rather than auto-discovered — discovery is convention, not contract, and a reviewer that silently doesn't know the house rules still produces confident-looking output. That's also why the brief doesn't restate the invariants: CLAUDE.md stays the single place they live.

Design decisions worth not undoing¶

• Reviewers are read-only. contents: read, so they comment and label but physically cannot push. If a reviewer disagrees with you, it has to argue.
• Every workflow is gated on author_association. opencode has no built-in permission check.
• Reviewer checkouts land on main, not the PR head. An issue_comment event carries no PR ref, and this is what we want: a PR can't edit the criteria it's judged by.
• startsWith, not contains. Quoting a previous review comment would otherwise re-trigger the job.

The coding agent¶

/oc <instruction> on an issue opens a PR; on a PR it commits to that branch.

It authenticates through the opencode GitHub App rather than GITHUB_TOKEN, and that is not incidental — see the first gotcha below.

A push onto a PR that has come into conflict with main creates no pull_request runs (GitHub can't build refs/pull/N/merge), which would leave a green-looking PR over code the tests never saw. So the agent records the PR head before it runs and, if the head moved, asserts that a workflow run exists for the new head within ~2 minutes — failing the job and commenting on the PR if not.

A fix round (/oc address … on a PR with review findings) ends with one line per finding — Addressed (file, test) or Not addressed (why) — so an incomplete round is visible before a re-review is spent on it.

The merge gate¶

review-gate.yml publishes an llm-review commit status. A PR that touches no high-risk path gets success immediately. On a high-risk PR: success on review:pass, failure on review:changes-requested, pending otherwise. On a new push it clears both verdict labels and resets to pending — that half is the point, because otherwise a stale pass keeps the status green over commits the reviewer never saw.

review-deepseek.yml publishes the status itself after reviewing, because its labels are written with GITHUB_TOKEN and so never trigger the gate's labeled event (gotcha 1 again).

llm-review has been a required check in branch protection, alongside pytest and docs-drift, since 2026-09-24. Before the auto-success path existed that would have blocked every PR; now it only blocks the ones that need a review. It's also what makes auto-merge safe. If llm-review weren't required, auto-merge would put high-risk PRs straight through without their review.

Rate each review afterwards¶

On merge, add one of review:caught-real-bug or review:noise. It takes a second and it's the only way to find out whether the review is earning its place — a review nobody trusts is worse than no review, because it launders the diff.

Gotchas that will cost you an afternoon¶

Two of these (1, 2) are the same underlying fact; three more (3, 4, 6) are all "don't let strangers spend your money or read your keys"; the rest is housekeeping.

1. Anything done with GITHUB_TOKEN doesn't count as an event.

GitHub's rule, to stop infinite loops — otherwise a workflow that pushes a commit would start a workflow that pushes a commit, forever.

The trap: if the coding agent pushed using that token, GitHub would treat the push as a non-event, so pytest and docs-drift would never run on it. And the PR wouldn't look broken — it would look clean, because no checks are pending at all. Absence of a red X reads as "passed". Hence the opencode GitHub App: a separate identity, so its pushes are real events and CI fires. Don't "simplify" that back to use_github_token: true.

2. That same rule is why triggers are comments, not labels.

Tapping a label is a nicer phone interaction, so the original design was label → a workflow posts /review → the review workflow sees it. But the middle step would post using GITHUB_TOKEN, so the reviewer would never wake up. Comments trigger things; labels record state.

3. author_association is the thing keeping strangers out.

With every comment GitHub reports the commenter's relationship to the repo — OWNER, MEMBER, COLLABORATOR, CONTRIBUTOR or NONE. Our workflows require one of the first three. Without that check, anyone who can comment could run an agent with write access, on our money.

4. Never use pull_request_target.

There are two ways to trigger on a PR. The safe one withholds secrets when the PR comes from a stranger's fork. pull_request_target hands over full secrets — and people reach for it precisely because they want the secrets, then check out the stranger's code and run it with access to their keys. It's the classic way repos get their secrets stolen. The comment trigger gives us secrets safely instead.

5. share: false matters.

opencode can publish a transcript of the agent's session to a public URL, and it defaults to on for public repos. That transcript contains chunks of the codebase.

6. Fork PRs get no secrets.

If someone forks the repo and opens a PR, GitHub deliberately hides our secrets from workflows running on it — so none of this fires on outside contributions. Fine while this is a solo repo; a real design question the day it isn't.

7. Model IDs move.

Providers retire names. The deepseek-chat and deepseek-reasoner aliases retired 2026-07-24. A workflow failing with "unknown model" is almost always this; check models.dev for the current ID.

8. claude-code-action needs id-token: write even though it never pushes.

It authenticates as the Claude GitHub App by exchanging a GitHub OIDC token, and minting that token needs the scope. Read-only-on-code and OIDC are separate axes, so "this job only comments, it needs no write permissions" is the wrong inference — and it fails at the very last step, after checkout and prompt-building have all succeeded, with Could not fetch an OIDC token repeated three times.

9. The spend risk is loops, not tokens.

A review is roughly 4p. What actually costs money is two agents triggering each other, or one hanging for hours. timeout-minutes kills a job after 20–30 minutes, and the per-PR concurrency group means a second run cancels the first rather than stacking.

10. The reviewer needs checkout credentials; the coding agent must not have them.

opencode fetches the PR with a plain git fetch origin <branch>. With persist-credentials: false that dies with could not read Username for 'https://github.com', and every /review-deepseek run here failed that way until 2026-09. The reviewer is contents: read, so keeping credentials is harmless. The coding agent is the opposite case. Its pushes have to go through the opencode App, because a push made with GITHUB_TOKEN never triggers CI (gotcha 1). A fix that's right for one of these workflows is wrong for the other, so don't copy it across.

11. claude-code-action refuses bot-pushed branches unless told otherwise.

By default it blocks non-human triggering actors. Every /oc-coded PR is pushed by the opencode GitHub App, so without allowed_bots: "opencode-agent,opencode-agent[bot]" the auto path would refuse exactly the PRs it exists for. The manual /review path is typed by a human, so it's only the auto-review-on-green path that trips over this.

Configuration¶

• DEEPSEEK_API_KEY and CLAUDE_CODE_OAUTH_TOKEN — repository secrets (Settings → Secrets and variables → Actions). Not the local .env: a workflow runs in a fresh container that never sees it.
• The opencode GitHub App must be installed on the repository.
• Both reviewer models are pinned. Leave them pinned — if the model can drift, a change in findings can't be attributed to the model or to the brief.

If nothing happens¶

In rough order of likelihood:

1. The phrase wasn't the first thing in the comment.
2. The workflow isn't merged to main yet. Comment-triggered workflows always run the version on the default branch — so a new one added in a PR does nothing until that PR merges, and testing it on its own PR is impossible. (pull_request-triggered workflows are the opposite: they run the PR branch's version, so review-gate.yml does take effect immediately. Easy to trip over, because two workflows in the same folder behave differently.)
3. The opencode GitHub App isn't installed.
4. A secret is missing or the DeepSeek balance is zero (a key with no credit looks valid and fails every call).
5. The model ID retired — see gotcha 7.

The Actions tab shows every run and its logs, and is the first place to look.

Background¶

The full design and its reasoning: docs/multi-llm-pr-review-plan.md (maud-specific) and docs/two-model-review-playbook.md (repo-agnostic).