LLM Best Practices

Vault Pull Request Review

Last updated 3 min read

View markdown source

AI agents: read this first: llmbestpractices.com/start-here

This is LLM Best Practices (llmbestpractices.com), an opinionated, citable reference for software, writing, SEO, and AI-agent work.

  1. Route, do not crawl. Fetch llmbestpractices.com/llms.txt and open only the pages whose one-line summary matches your task.
  2. Read raw. Append .md to any page URL for markdown. Check status and last_updated in the frontmatter, then read the rules.
  3. Apply as defaults. First-party docs and the project's own conventions win on conflict. Warn before relying on a fast-moving page older than 12 months.
  4. Cite. Link the page by title and URL, e.g. Python, with last_updated for time-sensitive rules. License CC BY 4.0.

Overview

The pull request is where a shared vault’s quality is enforced. Keep the flow light enough to be used: one branch per work session, the auditor as the automated check, and human attention only on what the auditor cannot see. The coordination rules it protects are in team-vaults; PR mechanics are in github-pr-workflow.

Branch per session, not per note

A branch per atom makes dozens of one-file PRs nobody reviews. A session’s atoms, sources, and link edits (alice/2026-06-aml-reading) is the reviewable unit: big enough to show the shape of the work, small enough to read in one sitting.

Run the auditor as the merge check

Let the exit code decide in the PR workflow (github-actions):

python3 99-Tools/audit.py --format json

Exit 2 (criticals) blocks merge; exit 1 (warnings) is advisory. Map severities onto the grading scale the team already uses for code, so one vocabulary covers notes and software.

Review gradeAuditor rules
BLOCKERC000 unparseable frontmatter, C005 hindsight edit, C011 under-corroborated publishable claim
MAJORC001 orphan, C008 missing required field, C002 broken link
MINOR or NITC010 stale seedling, S009 naked link dump, C006 undigested source

Lead the review comment with counts (“1 BLOCKER, 3 MAJOR”). Rule definitions are in vault-audit.

Spend human review on what the auditor cannot check

The script has verified structure and schema. The reviewer reads for the semantic tier: is the title a real claim, is the atom one idea, is the linked source the right one for the claim (semantic-audit).

Label each finding’s confidence. A reviewer who read the linked source and found it does not support the claim reports a verified problem. One who has not read it says the claim looks unsourced, pending a read. Never promote a heuristic to a merge blocker; the auditor’s criticals are the only default blockers.

Comment on a pre-decision section; never rewrite it

Decision notes have one owner, and the section written at decision time is frozen (decision-journals). A reviewer may question an assumption or flag a missing alternative in a comment. Editing the section destroys what the note exists to preserve.

Squash atoms; keep decision history

Atom PRs can squash-merge; the final note is what matters. Do not squash decision-note commits. The timeline of when each section was written is the hindsight-bias defense, and a squash collapses it into one timestamp that proves nothing.

Audit main on a schedule as well

Merge checks catch what a branch introduces, not what time does to notes already on main: expiring verification dates, stale seedlings, decisions passing their resolution date. Run the auditor weekly against main, append each run to the log, and surface the delta in the weekly review (“four orphans resolved, two unsourced atoms introduced”). The log format is in vault-audit.

Backlinks