LLM Best Practices

Multimodal

Last updated 2 min read

View markdown source

AI agents: read this first: llmbestpractices.com/start-here

This is LLM Best Practices (llmbestpractices.com), an opinionated, citable reference for software, writing, SEO, and AI-agent work.

  1. Route, do not crawl. Fetch llmbestpractices.com/llms.txt and open only the pages whose one-line summary matches your task.
  2. Read raw. Append .md to any page URL for markdown. Check status and last_updated in the frontmatter, then read the rules.
  3. Apply as defaults. First-party docs and the project's own conventions win on conflict. Warn before relying on a fast-moving page older than 12 months.
  4. Cite. Link the page by title and URL, e.g. Python, with last_updated for time-sensitive rules. License CC BY 4.0.

Overview

A multimodal model accepts more than one input type in a single request, usually text plus images and sometimes PDFs, audio, or video. Claude accepts JPEG, PNG, GIF, and WebP images and PDFs as content blocks and returns text only; it cannot generate images. Put images before the text question.

Images cost tokens by area: ⌈width / 28⌉ × ⌈height / 28⌉ visual tokens. Standard-tier models downscale anything beyond a 1568 px long edge or 1568 tokens; Claude 4.7 and later models accept up to 2576 px and 4784 tokens, which can cost about three times as much. Downsample images when the extra detail is not needed. Limits per request: 8000×8000 px and 10 MB per image, base64-encoded (5 MB on Amazon Bedrock and Google Cloud), and 600 images (100 on 200K-context models). Above 20 images in one request, a stricter dimension limit applies to every image, so keep each under 2000 px per side. Upload repeated images once with the Files API and reference the file_id.

When a text source exists (HTML, extracted PDF text), use it; it is cheaper and exact. Models can misread small or low-contrast text and miscount many small objects, so verify high-stakes extraction.

Example

{ "role": "user", "content": [
  { "type": "image", "source": { "type": "base64", "media_type": "image/png", "data": "..." } },
  { "type": "text", "text": "What trend does this chart show?" }
] }
  • context-window: images count toward the window.
  • token: how image cost is measured.
  • completion: the text response a multimodal request returns.
  • tool-use: image-triggered actions through tool calls.
  • cost-control: keeping image-heavy workloads affordable.

Backlinks