Overview
A vault without an auditor degrades invisibly: orphans accumulate, required fields go missing, and verification dates go stale without announcing it. The auditor is a script in the vault (99-Tools/audit.py) that reads the vault spec and reports structural and schema violations deterministically. This page covers running it, the rule catalog, triage, autofix, and the re-run. Judgment-level defects are the separate pass in semantic-audit.
Split checks into three buckets and automate two
- Bucket 1, structural. Orphans, broken links, missing frontmatter, filename, folder placement. Framework-agnostic.
- Bucket 2, schema. Framework-specific: an ADR with no alternatives, a theme atom with too few data points.
- Bucket 3, semantic. Is the title a claim? Does the atom hold one idea? Was the pre-decision section rewritten? This needs reading.
Buckets 1 and 2 are code and are never done by hand or by a model. Bucket 3 is where judgment belongs.
Configure from the spec, and treat a missing spec as a failed run
The auditor reads vault-spec.yaml for note types, required fields, folder mapping, thresholds, and severity_overrides. Adding a type or making a field required is a spec edit with no code change (vault-frontmatter-schema).
If the spec is missing or unparseable, the auditor falls back to built-in defaults, prints a loud warning, emits rule M000, and exits non-zero. A clean count next to M000 says nothing about the vault’s real rules. Check spec_status in the JSON summary (loaded, missing, or invalid) before trusting a zero.
Tunable thresholds and their defaults:
Key in rules: | Default | Used by |
|---|---|---|
atomicity_byte_threshold | 5000 | C009 |
stale_seedling_days | 30 | C010 |
stale_verification_days | 180 | C012 |
undigested_source_days | 30 | C006 |
decision_modified_buffer_days | 7 | C005 |
corroboration_min | 2 | C011 |
contradiction_aging_days | 90 | C013 |
Run it from the vault root and read the JSON
cd <vault-root> && python3 99-Tools/audit.py --format bothOne run writes 99-Tools/audits/AUDIT.md, audit.json, and a one-line entry in log.md, and archives the previous reports. Reason over the JSON: each violation has a rule id, severity, file, message, and fix hint. Useful flags are --severity LEVEL (that level and above), --folder <dir>, and --quiet. Exit codes serve CI: 0 clean or info only, 1 warnings, 2 criticals. Setup is in bootstrap-a-research-vault.
Read the structural rules (C series)
C-rules fire on every vault, whatever the framework.
| Rule | Finding | Severity |
|---|---|---|
| C000 | Frontmatter unparseable | critical |
| C001 | Orphan atom, no inbound or outbound links | warning |
| C002 | Broken wikilink | warning |
| C003 | Missing frontmatter | warning |
| C004 | Filename does not match the type’s pattern | warning |
| C005 | Decision modified after its decision date | critical |
| C006 | Undigested source: zero atoms past the age threshold | info |
| C007 | Decision past its expected resolution date | warning |
| C008 | Required field missing or empty | warning |
| C009 | Atom past the atomicity byte threshold | warning |
| C010 | Stale seedling | info |
| C011 | Publishable claim failing the corroboration or legal gate | critical |
| C012 | Stale verification on a decision-relevant claim | warning |
| C013 | Unresolved contradiction past its aging threshold | info |
| C014 | Note filed outside its type’s folder | warning |
| C015 | Content note with absent or unmappable type | warning |
The three criticals protect provenance. C000 hides a note from every other check. C005 is a hindsight rewrite (decision-journals). C011 breaks the publishing gate (source-verification). C015 closes a loophole: without it, stripping type would be the easiest way to pass every per-type check.
Read the schema rules (S series) and the meta rule
S-rules encode a rigor framework and fire only on notes that opt in by type or filename prefix (rigor-frameworks).
| Rule | Finding | Severity |
|---|---|---|
| S003 | Theme atom with fewer than three linked data points | warning |
| S004 | Result atom not linked to a hypothesis | warning |
| S006 | Objective atom missing concept, example, or assessment links | warning |
| S007 | ADR missing an alternatives section | warning |
| S008 | Decision missing key_atoms | warning |
| S009 | Naked link dump in an atom | info |
| M000 | Audit ran on built-in defaults | warning |
S001, S002, and S005 are reserved. Custom rules a vault declares use a U prefix (U001, U002) so they never collide with a reserved id. Override severities with severity_overrides in the spec, keyed by rule id; the script’s defaults stay untouched.
Triage by root cause, not rule id
A flat list of 30 violations is noise; the same 30 grouped by cause is a plan.
- Eight orphans on one theme: a missing map, not eight missing links.
- Five unsourced claims from one book: one source-backfill session.
- Three overdue decisions in one class: one resolution-review session.
- Many atomicity hits in one folder: a domain that grew too fast; one refactor session.
- One hindsight rewrite: handle it alone, immediately.
Present criticals first and individually, warnings grouped by cluster with a proposed batch action, and info as a list for the next review.
Autofix only the mechanical tier
--fix repairs what can be derived with certainty, and nothing else.
- C003: adds a frontmatter block, with
typefrom the folder when it maps to one type. - C008: fills
idfrom the filename,createdandmodifiedfrom file mtime,statuswith the seedling default, and structural tags. Semantic fields (title,sources,confidence,decision_date) stay open for a human. - C015: sets
typefrom an unambiguous folder. - C002: repairs a link only when exactly one filename matches.
An autofix that invents a title, source, or confidence value fabricates provenance. Dry run is the default, --apply writes, and an un-versioned vault is refused without --force, since there is no undo. Never rename files or edit prose automatically.
Pause before bulk changes, then re-run
Bulk renames, moves, and schema backfills touching dozens of notes get a scope freeze: the exact path list, what undo looks like, and explicit approval. The report proposes; the owner accepts, doubly so when an agent runs the audit (vault-orchestration). A field that becomes newly required across many notes is a migration, in vault-evolution.
After fixes, run again and report resolved findings by rule id, new findings introduced (a new link can create a broken one), and the net change per severity. Append every run to the log so numbers become a trend, and let the same run gate merges in a shared vault (vault-pull-request-review). The review that reads the trend is in vault-maintenance.