LLM Best Practices

RAG: Citations

4 min read

View markdown source

Overview

Citations turn a RAG answer from a claim into a checkable claim. The rules below force inline citations on every fact, give the user a verifiable link back to the source, and audit the model’s citation behavior so hallucinated sources stop slipping through. For the retrieval that produces these chunks, see rag-retrieval. For the eval harness, see rag-eval.

Cite every fact

If you cannot cite it, you cannot claim it. Treat uncited claims as model failures, not stylistic choices.

  • Every factual sentence ends with at least one citation marker: [1], [2], [3].
  • Conjunctions get their own citations: "Postgres 17 added foo [1] and bar [2]."
  • Opinions and synthesis sentences cite the chunks that ground them.

The exception is meta-text (“Here is what I found”). The exception is not “common knowledge”; in a domain agent, common knowledge is still a retrieval target.

Pass chunk IDs into the prompt

Citations cannot exist without a stable handle. Give every retrieved chunk an ID and require the model to reference it.

Context:
[1] (source: postgres-17-release-notes.md#jit) ...
[2] (source: backend/postgres.md#tuning) ...
[3] (source: ops/cloudflare.md#cache) ...
 
Question: How do I enable JIT in Postgres 17?
 
Rules:
- Answer with inline citations like [1] or [2] for each factual claim.
- If the context does not answer the question, say so. Do not guess.

The numeric ID maps back to the chunk’s metadata (source_url, heading_path, chunk_index). The mapping is what makes the citation auditable.

A citation that says [1] is half a citation. The user needs to land on the exact passage.

  • Construct a deep link per chunk: source_url#heading-anchor or source_url?start_line=42&end_line=68.
  • Markdown sources use the heading slug; code uses line offsets; PDFs use page numbers and bounding boxes.

The chunk’s metadata carries the link. See rag-chunking for the metadata keys.

Defend against hallucinated citations

A model can invent a [4] that does not exist, or cite [1] for a claim [1] does not support. Both are common; both are catchable.

  • Validate citation tokens after generation. Strip any [N] where N is not in the retrieved set.
  • Score faithfulness: did the cited chunk contain the claim? Use a small judge model or a string-overlap heuristic. See rag-eval.
  • Reject and retry when faithfulness falls below a threshold; two retries, then return a fallback.

The validator is non-optional. A pipeline that trusts the model to cite correctly will ship wrong answers with confident sources attached.

Force structured output for the citation channel

Free-form citations drift. Tool use with a JSON Schema locks the format.

{
  "answer": "Postgres 17 enables JIT via the `jit` GUC.",
  "citations": [{ "marker": "[1]", "chunk_id": "abc123", "supports": "the JIT GUC is the toggle" }]
}

The chunk_id references the retrieved set; the validator checks it. The supports field gives the judge model a target for faithfulness scoring. See structured-output for the strict-mode tool-use pattern.

Surface sources in the UI as first-class objects

The citation is the user’s trust anchor. Treat it like one.

  • Render each citation as a clickable chip under the answer: title - heading - last_updated.
  • On hover, show the chunk text in a popover. The user sees what the model saw.
  • On click, open the deep link in a new tab.

Hiding sources behind an i icon defeats the point. The citation should be as visible as the answer.

When the answer cannot be cited, say so

The model must refuse rather than guess. The prompt sets the policy; the UI honors it.

  • System instruction: “If the retrieved context does not contain the answer, say 'I do not have a source for that.' Do not guess or extrapolate.”
  • UI: render the refusal as a styled empty-state. Offer a “show what was retrieved” link so the user can audit the miss.
  • Log every refusal. A spike means the retrieval changed, not the model.

A refusal is a signal. A wrong answer with a fake citation is the failure.

Score citation behavior as part of the eval

Citation quality is a metric, not a vibe.

  • Citation coverage: percentage of factual sentences with at least one citation.
  • Citation validity: percentage of [N] markers that map to a real retrieved chunk.
  • Citation faithfulness: percentage of citations where the cited chunk supports the claim.

Track all three on the regression dashboard in rag-eval. A model that lifts answer relevance while citation faithfulness drops is a regression dressed as a win.

Graph View

Backlinks