---
title: "Ollama: Modelfile"
slug: "ollama-modelfile"
category: "ai-agents"
tags: ["ai-agents", "ollama", "modelfile", "local-llm", "system-prompts", "configuration", "gguf"]
status: "stable"
last_updated: 2026-08-14
summary: "How to write an Ollama Modelfile to pin a base model, system prompt, and generation parameters into a versioned local image."
related: ["[[ai-agents/ollama]]", "[[ai-agents/ollama-model-selection]]", "[[ai-agents/ollama-quantization]]", "[[ai-agents/ollama-serving]]", "[[ai-agents/system-prompts]]", "[[prompt-engineering/prompt-design]]", "[[ai-agents/structured-output]]"]
---

## Overview

A Modelfile is a declarative configuration file that turns a base model into a named local image with a pinned system prompt, generation parameters, and template. It is the Ollama equivalent of a Dockerfile: check it in alongside your agent code so the model and the prompt move together. Every production agent that runs on Ollama should be built from a Modelfile, not from a bare `ollama run` call.

## Start every Modelfile with a pinned `FROM` tag

`FROM` specifies the base model. Use the full model tag including quantization. A tag without a quantization suffix can resolve to a different file when the registry adds a new variant.

```
FROM qwen3-coder:30b-q4_K_M
```

- Always include the quantization suffix (`q4_K_M`, `q8_0`, etc.). See [[ai-agents/ollama-quantization]] for the quantization levels.
- You can also `FROM` a local GGUF file path: `FROM /models/qwen3.6-27b-q4.gguf`. Use this when distributing models in air-gapped environments.
- The base model must be present on the host before `ollama create` runs. Pull it first with `ollama pull`.

## Write the `SYSTEM` instruction as a single stable block

`SYSTEM` sets the system prompt that every conversation starts with. Keep it focused on role, output format, and hard constraints.

```
FROM qwen3-coder:30b-q4_K_M

SYSTEM """
You are a code reviewer for a TypeScript monorepo.
Return your review as a JSON object with keys:
  - "summary": one sentence
  - "issues": list of {file, line, severity, message}
  - "approved": boolean

Do not include any text outside the JSON object.
"""
```

See [[ai-agents/system-prompts]] for the rules on system prompt structure and [[ai-agents/structured-output]] for format constraints. Avoid dynamic content in the system prompt; dynamic context belongs in the user message.

## Set `PARAMETER` values to match the task

`PARAMETER` lines override the model's default generation settings. Set only the parameters your task requires.

```
PARAMETER temperature 0.2
PARAMETER num_ctx 16384
PARAMETER num_predict 2048
PARAMETER top_p 0.9
PARAMETER stop "</review>"
```

Key parameters and their effects:

- `temperature`: 0.0 to 0.2 for extraction and classification; 0.7 to 1.0 for creative tasks.
- `num_ctx`: the context window in tokens. Set to the largest input you actually send, not the model's maximum.
- `num_predict`: maximum output tokens. Always set this in batch pipelines to prevent runaway generation.
- `stop`: one or more stop sequences. Useful for structured output templates where the model should stop after a closing delimiter.

## Use `TEMPLATE` only when the base model requires a custom chat format

Most models in the Ollama registry ship with the correct chat template embedded in the GGUF file. Override `TEMPLATE` only when the default is wrong or absent.

```
TEMPLATE """{{ if .System }}<|system|>
{{ .System }}<|end|>
{{ end }}{{ if .Prompt }}<|user|>
{{ .Prompt }}<|end|>
<|assistant|>
{{ end }}{{ .Response }}<|end|>"""
```

Incorrect templates are the most common source of "the model ignores instructions" bugs. When output quality is lower than expected from a bare `ollama run`, compare your template to the one in the original model card.

## Build and test the image with `ollama create` and `ollama run`

```bash
ollama create code-reviewer -f Modelfile
ollama run code-reviewer "Review this function: function add(a, b) { return a + b }"
```

The `ollama create` step compiles the Modelfile and registers the image locally. It does not re-download the base model if it is already present.

- Name images by role and version: `code-reviewer-v2`, `support-triage-v1`.
- Check the Modelfile into version control. The image itself is local to the host; the Modelfile is portable.
- When updating a Modelfile, increment the version in the name. Do not overwrite a running image in production; create the new image and switch traffic.

## List, inspect, and clean up images

```bash
ollama list                # All local images and their sizes
ollama show code-reviewer  # Modelfile and parameters for a specific image
ollama rm code-reviewer-v1 # Remove an old image to reclaim disk space
```

Images from Modelfiles share the base model layers on disk. Removing `code-reviewer-v1` does not remove the underlying `qwen3-coder:30b-q4_K_M` layers; those are shared with any other image built from the same base.

## Related

- [[ai-agents/ollama]]
- [[ai-agents/ollama-model-selection]]
- [[ai-agents/ollama-quantization]]
- [[ai-agents/ollama-serving]]
- [[ai-agents/system-prompts]]
- [[prompt-engineering/prompt-design]]
- [[ai-agents/structured-output]]