AI code review · issue tracking · merge gate
Nobody reads the diff.Now they have to.
Code review
risk 74/100Adds team invites and a role column. The invite acceptance route trusts the role field from the request body, so an invited member can make themselves an owner.
The role is read straight from the POST body and written to the membership row. Anyone with an invite link can POST role: "OWNER" and take over the workspace. Take the role from the invite record instead.
Before this can merge, someone other than @dani has to answer 6 questions about what this diff does.
Take the quiz →Invitee chooses their own role
N+1 on the members page
Invite tokens are sequential
Webhook retries double-charge
Unhandled rejection in the mailer
After this change, what decides the role a new member gets?
Merging is blocked
Required status check quiz sign-off has not passed.
All checks have passed
Signed off by @sam — scored 100% on the review quiz.
Code review
risk 74/100Adds team invites and a role column. The invite acceptance route trusts the role field from the request body, so an invited member can make themselves an owner.
The role is read straight from the POST body and written to the membership row. Anyone with an invite link can POST role: "OWNER" and take over the workspace. Take the role from the invite record instead.
Before this can merge, someone other than @dani has to answer 6 questions about what this diff does.
Take the quiz →An AI reviewer reads every pull request and files what it finds on a board you actually work. Then it writes a quiz about the real diff — and the merge stays blocked until someone other than the author passes it.
The uncomfortable truth
LGTM
Approvals arrive in ninety seconds on diffs that take twenty minutes to read. Everyone knows it. Nobody says it, because saying it means being the person who slows the team down.
The author is gone
The AI wrote it, the author skimmed it, the reviewer rubber-stamped it. When it breaks at 2am there is genuinely nobody who understands the code — and the git blame points at a prompt.
Bots you learn to ignore
A review bot that comments on everything gets muted like everything else. A finding nobody tracks and nobody answers for is just noise with syntax highlighting.
01 — Review
Every pull request, read properly.
The bot reads the diff and the current state of every file the diff touches, so it can see what a change breaks somewhere else — not just what it says locally.
Every finding has to name a concrete failure: what input or sequence produces what wrong outcome. If it can't describe that, it doesn't get reported. A clean diff comes back clean and says so.
- Authorization holes on new endpoints and actions
- Edge cases the change doesn't survive
- Callers the diff forgot to update
- Migrations that break rows you already have
Code review
risk 74/100Adds team invites and a role column. The invite acceptance route trusts the role field from the request body, so an invited member can make themselves an owner.
The role is read straight from the POST body and written to the membership row. Anyone with an invite link can POST role: "OWNER" and take over the workspace. Take the role from the invite record instead.
Before this can merge, someone other than @dani has to answer 6 questions about what this diff does.
Take the quiz →Invitee chooses their own role
N+1 on the members page
Invite tokens are sequential
Webhook retries double-charge
Unhandled rejection in the mailer
02 — Track
Findings become work, not comments.
A review comment scrolls away the moment the PR merges. Everything the bot finds lands on a board instead — with a severity, an owner, a status and the code in context — so it survives the merge and someone has to actually close it.
- Drag through to do, in progress, blocked, done
- Opens with the offending lines, at the reviewed commit
- Assign it, discuss it, keep the history
- Push any issue to GitHub Issues — closing it there closes it here
03 — Sign off
The part no other review bot does.
Clicking approve costs nothing, so it means nothing. Every review generates five to seven questions about this diff — what the changed function returns now, which flows it affects, what happens on the retry.
You can only answer them by reading the change. That's the entire mechanism: about four minutes of real cost attached to the moment someone says “this is fine to ship”, and a permanent record of who said it.
- Never the author — they can't vouch for their own code
- Write access required, checked against GitHub
- Graded server-side; answers never reach the browser
- Bound to the commit — a new push clears it
After this change, what decides the role a new member gets?
Merging is blocked
Required status check quiz sign-off has not passed.
All checks have passed
Signed off by @sam — scored 100% on the review quiz.
04 — The gate
It shows up where the decision happens.
The quiz posts to GitHub as a status check called quiz sign-off. Make it required in branch protection and the merge button stays off until someone passes.
Nothing is forced on you: until you tick that box it's a check like any other. The bot never merges, never pushes, and never writes to your code — only its own check and its own comment.
If the bot itself breaks, the check goes neutral, not red — a broken review tool should never be able to permanently block your team's merges.
05 — What it looks for
The bugs that survive the tests.
Authorization holes
A new endpoint that trusts the id it was handed. Can user A reach user B's data by changing a number? The single most common defect in AI-written code.
Correctness under edge cases
Empty collections, first runs, concurrent callers, retries, promises never awaited, inverted conditions that read correctly at a glance.
Regressions you didn't see
Callers of a changed function the diff forgot to update, return shapes that quietly changed, guards removed on the way past.
Data and migrations
Migrations that break rows that already exist, defaults that silently change meaning, queries added inside loops.
Performance cliffs
Unbounded fetches, new query paths with no index behind them, blocking work on a hot path.
Nothing at all
A clean diff gets an empty findings list and a low risk score. A reviewer that always finds something is a reviewer you stop believing.
06 — Setup
Two minutes, then it runs itself.
- 1
Install the GitHub App
Pick the repos it can see. Read-only on your code; it writes nothing but its own check and its own comment.
- 2
Open a pull request
The bot reviews the diff, posts its findings, files them on your board, and drops the sign-off link on the PR.
- 3
Make "quiz sign-off" required
One checkbox in branch protection. That's the step that turns a suggestion into a gate — until you tick it, the check shows red but nothing is blocked.
07 — Straight answers
The objections, up front.
Won't a quiz on every PR slow my team down?
It costs the reviewer a few minutes — roughly what reading the diff properly costs anyway, which is the point. You choose which branches it gates, so you can start with the ones where a bad merge actually hurts.
What stops someone brute-forcing the answers?
Questions come one at a time with no going back, the correct answers never leave the server, a failed attempt doesn't tell you which ones you missed, and there's a cooldown before you can retry.
Can the author just sign off on their own PR?
No. We compare the signer's GitHub login against the PR author and refuse, and we check they have write access on the repo. That's why sign-off needs GitHub even if you signed up with email.
What happens when the bot is wrong?
Findings are issues on your board, not blocking gates — close one as won't-fix and move on. Only the sign-off quiz gates the merge, and that's testing whether a human read the diff, not whether the bot was right.
Do you store or train on my code?
No. Code is read to produce the review and not retained beyond it, and it's never used to train anything. The app is read-only on your repo contents.
What if I push more commits?
Sign-off is bound to a commit, not a pull request. A new push clears the sign-off and generates a fresh quiz on the new diff — nobody vouches for code they haven't seen.
Make someone answer for the merge.
Install it on one repo, open a pull request, and see what comes back. If the findings aren't worth reading you've lost two minutes — and nothing is gated until you say so.