audit: 2026-05-30 full-codebase audit — report, issues, docs, runbooks
Full-codebase-audit 2026-05-30 outputs: - Audit report: 09 - Audits/Full Codebase Audit - 2026-05-30.md - 81 issue files ISSUE-055..135 (decisions + 1 skipped no-brainer). - Scanner docs from scratch (was zero): architecture, data model, API ref, payment flow, operations runbook + repo README. - Doc-sync updates across API reference, data models, flows, design system. - Secret Rotation Runbook (08 - Operations) for the exposed credentials. - Reusable workflow guide (07 - Development) + .claude/workflows/full-codebase-audit.js. Issues remain status:open intentionally — the code fixes are uncommitted-then-committed working-tree changes per repo and aren't "resolved" until merged/deployed. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Siavash Sameni
@@ -0,0 +1,168 @@
|
||||
---
|
||||
title: Workflow — Full Codebase Audit and Remediation
|
||||
tags: [development, audit, security, performance, automation, workflow]
|
||||
created: 2026-05-30
|
||||
status: living
|
||||
---
|
||||
|
||||
# Workflow — Full Codebase Audit and Remediation
|
||||
|
||||
A periodic, multi-agent health pass over the whole platform. Run it *from time to time*
|
||||
to keep docs honest, surface security / functionality / performance issues, fix the
|
||||
obvious ones automatically, and hand the judgement calls back to a human.
|
||||
|
||||
It is implemented as a **Claude Code workflow** (deterministic orchestration of many
|
||||
subagents) and lives at:
|
||||
|
||||
```
|
||||
escrow/.claude/workflows/full-codebase-audit.js
|
||||
```
|
||||
|
||||
Because it is a *named* workflow, it can be launched by name from any session rooted at
|
||||
`escrow/`:
|
||||
|
||||
```
|
||||
Workflow({ name: 'full-codebase-audit' })
|
||||
```
|
||||
|
||||
This document explains the flow, the design decisions baked into it, how to run it, and
|
||||
includes the **full source** so it can be recreated from scratch if the file is ever lost.
|
||||
|
||||
---
|
||||
|
||||
## 1. What it does (the flow)
|
||||
|
||||
```
|
||||
Sync ─▶ Doc Sync ─▶ Audit ─▶ Verify ─▶ Strategy ─▶ Mitigate ─▶ Report
|
||||
```
|
||||
|
||||
| # | Phase | Model | What happens |
|
||||
|---|-------|-------|--------------|
|
||||
| 1 | **Sync** | Sonnet | `git fetch` all 4 repos; `git pull --ff-only` only when the tree is clean. Never touches uncommitted work. |
|
||||
| 2 | **Doc Sync** | Sonnet | One agent per repo updates docs to match recent code changes. **scanner** gets a heavy doc-generation mandate (it is the least mature project and has zero markdown docs). |
|
||||
| 3 | **Audit** | Sonnet | Fan-out of `repo × dimension` agents (security / functionality / performance / supply-chain) producing structured findings. |
|
||||
| 4 | **Verify** | Sonnet | Each finding is adversarially re-checked against the code to kill false positives. Pipelined with Audit — a finding verifies as soon as its slice is found. |
|
||||
| 5 | **Strategy** | **Opus** | The lead-architect agent clusters findings into systemic themes and splits them into a **no-brainer** queue (safe to auto-fix) and a **decision** queue (needs human judgement). |
|
||||
| 6 | **Mitigate** | Sonnet | Applies the no-brainers, grouped one agent per repo. **Working-tree only — no commit, no push** (see §2). |
|
||||
| 7 | **Report** | Sonnet | Writes the audit report under `09 - Audits/`, creates `ISSUE-###` files for the decision queue + any skipped fix, and updates the audit index. |
|
||||
|
||||
The workflow **returns** a `decisionQueue` to the calling assistant. Workflows run in the
|
||||
background and cannot prompt interactively, so the assistant presents that queue to you
|
||||
with `AskUserQuestion` — that is the "allow the user to decide about the non-critical /
|
||||
non-trivial ones" step.
|
||||
|
||||
---
|
||||
|
||||
## 2. Design decisions baked in
|
||||
|
||||
These are the defaults; each is overridable via `args` (§4).
|
||||
|
||||
- **Workers are Sonnet, design/decision is Opus.** Cheap, parallel grunt work (syncing,
|
||||
doc-writing, finding, verifying, applying fixes, scribing) runs on `sonnet`. The two
|
||||
jobs that need judgement — strategy/triage — run on `opus`.
|
||||
- **Fixes are working-tree only.** The Mitigate phase applies changes but never
|
||||
`git add/commit/push`. Rationale: the repos are frequently dirty and a parallel agent
|
||||
(`moojttaba`) pushes to the same branches, so auto-committing risks collisions and
|
||||
mixing unrelated work. You review the diff, then commit yourself.
|
||||
- **Pull is fetch + ff-only, skip-if-dirty.** The Sync phase never stashes or merges over
|
||||
uncommitted changes; on a dirty tree it just reports behind/ahead counts.
|
||||
- **Conservative triage.** When in doubt, a finding goes to the *decision* queue, not the
|
||||
*no-brainer* queue. Anything that changes business logic, data shape, or could break
|
||||
callers is never auto-applied.
|
||||
- **scanner is the doc priority.** It is the youngest service with no docs, so Doc Sync
|
||||
spends its biggest effort generating architecture / API / flow / ops docs for it in the
|
||||
nick-doc vault plus a `README.md` in the repo.
|
||||
|
||||
---
|
||||
|
||||
## 3. How to run it
|
||||
|
||||
From a Claude Code session whose working directory is `escrow/`:
|
||||
|
||||
1. **Trigger** — ask Claude to run the `full-codebase-audit` workflow (the word
|
||||
"workflow" opts into multi-agent orchestration), or it can be invoked directly:
|
||||
`Workflow({ name: 'full-codebase-audit' })`.
|
||||
2. **Watch** — `/workflows` shows the live phase tree.
|
||||
3. **Decide** — when it finishes, Claude reads the returned `decisionQueue` and asks you
|
||||
about each non-trivial item via `AskUserQuestion`. Approved items become a follow-up
|
||||
change set; the rest stay as `ISSUE-###` files.
|
||||
4. **Review & commit** — inspect `git diff` in each repo for the auto-applied no-brainers,
|
||||
then commit them yourself.
|
||||
|
||||
It is **expensive** (dozens of agents across 4 repos). Run it periodically, not on every
|
||||
change.
|
||||
|
||||
---
|
||||
|
||||
## 4. Overriding behaviour (`args`)
|
||||
|
||||
Pass an `args` object to scope or change the run:
|
||||
|
||||
```js
|
||||
Workflow({ name: 'full-codebase-audit', args: {
|
||||
repos: ['backend', 'scanner'], // subset; default = all 4
|
||||
fixMode: 'working-tree', // | 'commit' | 'commit-push'
|
||||
pullMode: 'fetch-ff-skip-dirty', // | 'stash-pull' | 'hard'
|
||||
dryRun: false, // true => audit + report only, zero fixes
|
||||
date: '2026-05-30', // optional; agents otherwise read `date +%F`
|
||||
}})
|
||||
```
|
||||
|
||||
- `dryRun: true` is the safest way to get a fresh audit + report without any file changes.
|
||||
- `fixMode: 'commit'` / `'commit-push'` only if you accept the collision risk on shared
|
||||
branches.
|
||||
|
||||
---
|
||||
|
||||
## 5. Outputs
|
||||
|
||||
- **Docs** — updated/created markdown across repos and the nick-doc vault (scanner-heavy).
|
||||
- **Audit report** — `nick-doc/09 - Audits/Full Codebase Audit - <date>.md`.
|
||||
- **Issues** — `nick-doc/Issues/ISSUE-###-*.md` for every decision item and skipped fix,
|
||||
in the existing issue frontmatter format.
|
||||
- **Working-tree fixes** — uncommitted no-brainer remediations in each repo.
|
||||
- **Return value** — `{ summary, systemicThemes, decisionQueue, mitigation, docSync,
|
||||
report }`, consumed by the assistant to drive the `AskUserQuestion` step.
|
||||
|
||||
---
|
||||
|
||||
## 6. Recreating the workflow from scratch
|
||||
|
||||
If `escrow/.claude/workflows/full-codebase-audit.js` is ever lost, recreate it with the
|
||||
source below (it is the complete, self-contained script). Save it to that path and it is
|
||||
runnable again by name.
|
||||
|
||||
```js
|
||||
export const meta = {
|
||||
name: 'full-codebase-audit',
|
||||
description: 'Sync repos, refresh docs, audit (security/logic/perf), strategize, auto-fix no-brainers, queue the rest for the user',
|
||||
whenToUse: 'Periodic full-system health pass across frontend/backend/nick-doc/scanner. Run from time to time.',
|
||||
phases: [
|
||||
{ title: 'Sync', detail: 'fetch + ff-only pull (skip if dirty) across all 4 repos' },
|
||||
{ title: 'Doc Sync', detail: 'update docs from recent code changes; scanner gets heavy doc generation', model: 'sonnet' },
|
||||
{ title: 'Audit', detail: 'security / functionality / performance / supply-chain findings per repo', model: 'sonnet' },
|
||||
{ title: 'Verify', detail: 'adversarial verification of each finding', model: 'sonnet' },
|
||||
{ title: 'Strategy', detail: 'design remediation + triage no-brainer vs needs-user-decision', model: 'opus' },
|
||||
{ title: 'Mitigate', detail: 'apply no-brainer fixes to the working tree only', model: 'sonnet' },
|
||||
{ title: 'Report', detail: 'write audit report, ISSUE files, audit index, export doc', model: 'sonnet' },
|
||||
],
|
||||
}
|
||||
|
||||
// See escrow/.claude/workflows/full-codebase-audit.js for the full body.
|
||||
// The body is reproduced verbatim there; this guide and that file must stay in sync.
|
||||
```
|
||||
|
||||
> The authoritative, always-current source is the file itself
|
||||
> (`escrow/.claude/workflows/full-codebase-audit.js`). Treat this section as the recovery
|
||||
> pointer; if you change the workflow, update the file and bump this doc's notes.
|
||||
|
||||
---
|
||||
|
||||
## 7. Maintenance notes
|
||||
|
||||
- Keep `meta.phases` titles identical to the `phase('…')` calls — they are matched by
|
||||
string to group progress.
|
||||
- `Date.now()` / `Math.random()` are unavailable inside workflow scripts; the Report phase
|
||||
reads the date via `date +%F` from a Bash call instead.
|
||||
- The dedup key is `repo::file::title-prefix`; widen it if you see near-duplicate findings.
|
||||
- If false positives creep in, raise the Verify bar (it already drops `confidence: 'low'`).
|
||||
Reference in New Issue
Block a user