Every AI-security demo includes the same scene: an LLM looks at a vulnerability, opens a PR with the fix, the reviewer clicks merge. The pitch lands. Buyers nod. Procurement signs.
We shipped exactly that feature in January. Three months later we ripped most of it out. Here's what happened, and what we shipped instead.
In the Dockier stack · Fix drafts live in Supabase Postgres as a 1-to-N relation to findings (findings.id → fix_drafts.finding_id). They're generated lazily by thecode-analysisFastify service the first time a finding is opened. When a draft is promoted to a PR, thegit-integrationservice uses the user's PAT to open it via the GitHub / GitLab / Bitbucket API.
What we expected
The hypothesis was simple: scanners produce findings, findings need fixes, fixes are PRs. Cutting the manual work from "review finding → write fix → open PR" to "review PR" should be net positive for every team.
What actually happened
- PRs piled up unread. Across our 12 design partners, the median time-to-review for an AI-opened PR was 8 days. The median for a human-opened PR in the same repos was 19 hours.
- Context evaporated. By the time someone looked at the PR, the finding it was fixing had been triaged, deprioritised, deduplicated, or superseded. The PR's commit message couldn't possibly know.
- Trust eroded. Half of our partners had at least one PR that fixed a finding by deleting the call site. Technically correct. Brand-eroding. One bad PR like that sets the default for the next ten reviews.
The fundamental issue: a PR is the wrong unit of work for a low-confidence decision. PRs imply commitment. They imply someone read the change, agreed with it, and is asking for it to ship. AI fixes are an opening offer, not a commitment.
What we shipped instead
The new primitive is what we call a fix draft. It sits inside the finding view, not in the repo. It's a diff. You can read it, edit it inline, ask for an alternative, then click "Open as PR" when you actually want it to be a PR.
Three things changed when we moved from PRs to fix drafts:
- Remediation rate tripled. Across the same 12 design partners, findings with an attached fix draft were resolved 3.1× more often than findings with an auto-opened PR.
- Trust improved. Reviewers no longer associated "Dockier opened a PR" with "low-quality change in my repo." When they did open one, it was deliberate.
- Cost dropped. We weren't generating fix code for findings nobody would ever look at. Drafts are generated lazily on first click.
The deeper lesson
AI features want to be commitments because commitments are demo-able. But the unit of work in a developer's day is rarely a commitment — it's an option. Auto-complete is the canonical example: it suggests, you accept or ignore, the suggestion costs nothing if you skip it. AI fix PRs violated that contract.
Our internal heuristic now: if a generated artefact would survive being deleted without your approval, it shouldn't be created proactively. PRs survive. Comments survive. Issues survive. Drafts don't — they sit attached to a finding and get cleaned up automatically.
We kept one auto-PR flow: when the finding is a known-deterministic upgrade (e.g. "this CVE is fixed by bumping requests to 2.32.1") and the change passes the repo's existing CI in a sandbox, we'll open the PR automatically. Those have a 94% merge rate. Everything else is a draft.