LLM Best Practices

Ollama: Modelfile

Last updated 3 min read

View markdown source

AI agents: read this first: llmbestpractices.com/start-here

This is LLM Best Practices (llmbestpractices.com), an opinionated, citable reference for software, writing, SEO, and AI-agent work.

  1. Route, do not crawl. Fetch llmbestpractices.com/llms.txt and open only the pages whose one-line summary matches your task.
  2. Read raw. Append .md to any page URL for markdown. Check status and last_updated in the frontmatter, then read the rules.
  3. Apply as defaults. First-party docs and the project's own conventions win on conflict. Warn before relying on a fast-moving page older than 12 months.
  4. Cite. Link the page by title and URL, e.g. Python, with last_updated for time-sensitive rules. License CC BY 4.0.

Overview

A Modelfile is the Ollama equivalent of a Dockerfile: it turns a base model into a named local model with a fixed system prompt, parameters, and template. Check it into the repo beside the agent code so the model, prompt, and parameters change together.

Pin the base model with an explicit FROM tag

FROM is the only required instruction. Use a full tag, including the size and quantization, so a registry update cannot change the base underneath you.

FROM qwen3-coder:30b-a3b-q4_K_M
  • Pull the base first with ollama pull; ollama create does not re-download layers you already have.
  • FROM also accepts a local GGUF file or a safetensors directory, which is how you ship models to air-gapped hosts.
  • Ollama does not quantize GGUF files on import. Quantize with llama.cpp’s llama-quantize first, then point FROM at the result. For quantization levels see ollama.

Keep SYSTEM short and stable

SYSTEM sets the system message for every conversation. State role, output format, and hard constraints; put per-request context in the user message. See system-prompts and structured-output.

FROM qwen3-coder:30b-a3b-q4_K_M
SYSTEM """
You are a code reviewer for a TypeScript monorepo.
Return one JSON object: {"summary": string, "issues": [{"file": string, "line": number, "severity": string, "message": string}], "approved": boolean}.
Output nothing outside the JSON object.
"""
PARAMETER temperature 0.2
PARAMETER num_ctx 16384
PARAMETER num_predict 2048
PARAMETER stop "</review>"

Set only the parameters the task needs

PARAMETER overrides model defaults. Documented names include temperature (default 0.8), top_k (40), top_p (0.9), min_p, seed, repeat_penalty, repeat_last_n, num_ctx, num_predict, and stop (repeatable).

  • temperature: low for extraction and classification, higher for creative work.
  • num_ctx: pins the context window for this model. It is the supported way to set context for OpenAI-compatible clients, which cannot send it per request.
  • num_predict: cap it so batch jobs cannot run away.
  • stop: end generation at a delimiter when you parse structured text.

Override TEMPLATE only when the default is wrong

Registry models ship the right chat template inside the model. Override TEMPLATE (Go template syntax) only for a model that lacks one or imports with a wrong one. A wrong template is the usual cause of “the model ignores instructions”: compare the output with a bare ollama run of the base model and with the model card’s template.

Other instructions

  • MESSAGE: seed few-shot turns (MESSAGE user ..., MESSAGE assistant ...).
  • LICENSE: attach the license text when you redistribute.
  • REQUIRES: declare the minimum Ollama version.
  • CAPABILITY: declare an extra capability on an imported model.

Build, version, and clean up

ollama create code-reviewer-v2 -f Modelfile
ollama run code-reviewer-v2 "Review: function add(a, b) { return a + b }"
ollama show code-reviewer-v2 --modelfile
ollama rm code-reviewer-v1
  • Name models by role and version. Create the new version and switch traffic; do not overwrite a model that production is serving.
  • The Modelfile is portable; the built model is local to the host.
  • Models built from one base share its layers on disk, so removing code-reviewer-v1 does not delete the base.

Backlinks