I Built an AI Reviewer That Remembers Its Last Review

In this post

1. 01AI review is narrow and forgetful
2. 02What critique does
3. 03Inside the folder
4. 04Found, patched, promoted
5. 05What it checks
6. 06The review log
7. 07How it connects to other skills
8. 08Design decisions
9. 09A real example
10. 10Best practices
11. 11Next

Two things happen here: several lenses check one file at once, and every finding's status carries forward to the next run.

flowchart LR
    T[One file] --> L[Several lenses, each a different angle]
    L --> F[Anchored findings]
      

flowchart LR
    F[Findings] --> P[Patch proposed]
    P --> M{Promoted?}
    M -->|No| S[Stays open next run]
    M -->|Yes| R[Reconciled next run]
      

AI review is narrow and forgetful

Ask a model to review a file and it runs one unstructured pass, catching whatever happens to occur to it. It might flag a security issue and miss that the document never states its own purpose. It rarely asks what breaks six months out, or who benefits if the whole thing turns out to be wrong. Whether an issue gets covered at all comes down to chance.

That's one gap. The other only shows up once the same file gets reviewed a second time.

Say a review catches a real defect, and someone fixes it, or means to. Nothing records that the fix happened, so run the review again later and the model has no way to tell a resolved issue from one that simply went unnoticed twice: either way, it re-derives the same finding from scratch, spending the time and the tokens to redo work that was already done. Each pass gets treated as an island. Nothing the process learns on one run carries forward into the next.

What critique does

I needed something that would review a target from several angles while checking what it had already found, so it wouldn't rehash the same mistakes time and again. To do that, I created the critique skill, which runs an adversarial review against one file at a time, whether that's code, a document, a prompt, or a design doc.

Up to 10 lenses can run against a single file. Five run every time:

The remaining slots are domain-specific and stack on top: security and testability for code, leakage and baselines for ML.

Inside the folder

critique/
├── README.md
├── SKILL.md
├── bundles/
│   ├── code.md
│   ├── docs.md
│   ├── systems.md
│   ├── ml.md
│   └── multifile.md
└── evals/
    └── README.md

Each file has one job:

• SKILL.md: the protocol itself, always loaded. Review sequence, tier and budget rules, lens rules, the exact log schema.
• README.md: adapting the skill to a given surface. Where the log lives, what preferences to set.
• bundles/code.md: source files.
• bundles/docs.md: prompts and process documents.
• bundles/systems.md: architecture and design docs, plus a finance section that loads on its own whenever a dollar figure or an SLA shows up.
• bundles/ml.md: model and analytics write-ups.
• bundles/multifile.md: anything spanning more than one document.
• evals/README.md: the regression-test pattern for anyone changing the protocol or a bundle.

Only one primary bundle loads per run, so a forty-line prompt review never pays to load the exec/eval/subprocess checks meant for source code, and a code review never loads the leakage checks meant for a model write-up.

Found, patched, promoted

Each finding moves through three stages: found, patched, promoted.

• Every run writes one log line recording each finding's stage.
• The next run reads that log first, before anything else.
• If nothing's been promoted yet, every prior finding comes back unchanged: nothing re-derived, nothing quietly dropped.
• Once a maintainer promotes (archives the old file, copies the patched one over it), the next run grades each finding on its own: fixed if the patch held, still present if it didn't, withdrawn if the rule no longer applies.

The patch and the fix stay separate on purpose. A patch is a proposal, versioned apart from the file it replaces, and a person still decides whether to adopt it.

What it checks

What counts as a real finding adapts to the kind of file under review.

• Code: reachable security paths, integrations, and testability.
• Documents and prompts: purpose, evidence, contradictions, and confusing instructions.
• Systems: failure modes, degraded operation, capacity assumptions, and financial claims.
• ML and analytics: leakage, deployment-valid splits, baselines, metrics, and provenance.
• Multiple documents: unresolved contradictions across sources.

A claim with nothing to compare it against isn't a result. The review says so instead of taking it at face value.

The review log

• Each run appends one line to .claude/critique-log/<slug>.jsonl, one file per project (<slug> is the target path with separators swapped for hyphens).
• The line carries the target, the tier, which lenses ran, every finding's ID and severity, and the promoted flag.
• Closed findings drop off the line at the next run, so the log doesn't grow without bound.
• The next run reads that file before anything else. No log file means first run.

flowchart LR
    L[(critique log)] -->|read first| A[Adversarial pass runs]
    A --> B[Findings ranked, patched]
    B -->|new line appended| L
      

The log gates the run on one end and receives its result on the other. Nothing about the adversarial pass itself changes; what changes is whether that pass starts from zero or from what the last one already settled.

How it connects to other skills

The log isn't a dead record.

flowchart LR
    H[handoff] -->|triggers a run| C[critique]
    C -->|writes| L[(critique log)]
    L -->|counters block| T[token-aware]
    L -->|evidence source| R[retrospective]
      

• token-aware reads the log's counters block (calls made, cap, bytes, severity histogram) to compute effort per finding.
• retrospective reads the raw log directly as one of its evidence sources, alongside the journal, git log, and handoff snapshots.
• handoff can trigger a critique run against the snapshot it just wrote, so a session doesn't end with issues silently deferred.
• Both critique and handoff can trigger a bounded mini-retrospective.

Design decisions

A clean review is a valid result. A review that comes back clean can look like it didn't try hard enough, and that pressure is what pushes a model to invent a finding just to seem thorough. critique trusts the clean run instead, as long as the adversarial pass actually happened: checks fixed in advance, budget spent before a clean verdict is allowed.

Evidence anchors are mandatory. Every finding needs a quote, a line reference, or a named absence. The review's own conclusions don't count as evidence on their own. Without something concrete to point back at, a finding can't be checked later, so it gets discarded, however plausible it sounds.

Scope is intentionally limited. critique reviews only the artefact in front of it: never a build, a test suite, or anything outside the file. A finding about a contradiction in a prompt says only that the prompt contradicts itself. It says nothing about whether the program reading it actually works.

That boundary applies to the skill's own protocol too. A retrospective run against this project's audit trail once found the installed copy of critique running an older protocol version against a repo that had already moved on. A normal review run wouldn't catch that on its own. It checks the target it's pointed at. It doesn't check the process running it.

And this is how it all connects together, from the first log check to the last line written →

A real example

I decided it would be best to run a meta-analysis of my skills usage to update and optimize them before sharing them publicly. I first ask Claude:

I want you to /critique the following prompt, fix everything about it, and repeat the critique process until all personas are satisfied and all issues are resolved:

I'm collecting data about how my retrospective, critique, handoff, token-aware and kb-search skills have been working. go through the entire project history and do meta analysis of my skills. Include insights, implicit and explicit patterns, best and common practice, best and worst features, under-utilized or ignored features, what am i missing or misunderstanding. draw conclusion on what could make them run smoother, faster, better, more efficiently and affectively, and would can make them most cost-effective and have optimized ROI for every token.

Which resulted in this detailed prompt →

Running critique against it turned up issues like these:

The full run found 14 issues and proposed a patch for each, in one pass. The count isn't the interesting part. Every row had a quote, a line, or a named absence behind it. That's evidence a second run can actually check later instead of taking it on faith.

14 findings, 3 SEV1 / 10 SEV2 / 1 SEV3, 0 cut, all patched in one pass

Best practices

• Review one file at a time. State the purpose up front if it isn't obvious from the file itself. The review needs something to check the content against.
• Read the diff before promoting. critique proposing a patch is not the same as the patch being right.
• Verify a clean-sounding claim yourself. A summary that says a check passed isn't the check itself. Rerun it when the claim actually matters.
• Check which copy a finding is really about. The same defect can live in more than one copy of a file, an installed version, a synced cache, an account-level copy. A log entry only describes the one it was pointed at.
• Let a clean run stand. Don't ask for a second pass hoping it finds something. That pressure is exactly what produces invented findings.
• Run the review again after you promote. That's the only way the reconciliation step actually gets tested.

Next

{"v":"<protocol>","ts":"<iso8601>","t":"<path>","sz":[<lines>,<bytes>],"tier":"<t>","mode":"<normal|self>","lens":[...],"atoms":<n>,"pass":<n>,"f":[["<id>","SEV<n>","<lens>","<anchor slug, 6 words max>","<patched|backlog|withdrawn|still-present>"]],"cut":<n>,"promoted":false,"counters":{"calls":<n>,"cap":<n>,"bytes":<n>,"out_chars":<n>,"sev":{"1":<n>,"2":<n>,"3":<n>},"new":<n>,"carried":<n>},"out":"<patched filename or null>"}