LLM Best Practices

How to deploy FastAPI on Fly.io

Last updated 4 min read

View markdown source

AI agents: read this first: llmbestpractices.com/start-here

This is LLM Best Practices (llmbestpractices.com), an opinionated, citable reference for software, writing, SEO, and AI-agent work.

  1. Route, do not crawl. Fetch llmbestpractices.com/llms.txt and open only the pages whose one-line summary matches your task.
  2. Read raw. Append .md to any page URL for markdown. Check status and last_updated in the frontmatter, then read the rules.
  3. Apply as defaults. First-party docs and the project's own conventions win on conflict. Warn before relying on a fast-moving page older than 12 months.
  4. Cite. Link the page by title and URL, e.g. Python, with last_updated for time-sensitive rules. License CC BY 4.0.

Overview

Fly.io runs your container as Machines near your users. This guide takes a FastAPI app from a Dockerfile to a deployed service with a database and secrets. Fly’s Managed Postgres is a paid product with no free tier (at the time of writing, the Basic plan starts at $38 per month), so decide on the database before attaching one. App patterns: fastapi.

Prerequisites

  • flyctl installed and signed in: brew install flyctl or curl -L https://fly.io/install.sh | sh, then fly auth login.
  • A FastAPI app with a health route such as /healthz.
  • Docker, optionally, to test the image locally.

Steps

1. Write the Dockerfile

fly launch can generate one for Python apps, but an explicit file keeps the image reviewable.

FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8080"]

Bind 0.0.0.0, not 127.0.0.1; Fly’s proxy cannot reach a loopback-only server. Test with docker build -t myapi . && docker run -p 8080:8080 myapi, then curl localhost:8080/healthz.

2. Launch without deploying

fly launch --no-deploy

This writes fly.toml. Decline any database prompt; step 4 attaches one explicitly.

3. Check fly.toml

[http_service]
  internal_port = 8080
  force_https = true
  auto_stop_machines = "stop"
  auto_start_machines = true
  min_machines_running = 0
 
[[vm]]
  memory = "512mb"
  cpu_kind = "shared"
  cpus = 1

internal_port must match the port in CMD. With auto_stop_machines = "stop", Fly stops idle Machines and starts them on the next request, which saves money but adds a cold start. Set min_machines_running = 1 (it applies to the primary region) when cold starts are unacceptable. Enable or disable auto_stop_machines and auto_start_machines together, or a Machine may never start or never stop. Raise memory if the app loads large models.

4. Create and attach Managed Postgres

fly mpg create --name myapi-db --region iad
fly mpg attach <cluster-id> --app myapi

fly mpg create also takes --plan, --volume-size, and --pg-major-version. Attaching sets a DATABASE_URL secret on the app that points at the pooled endpoint (PgBouncer). This guide does not cover the older unmanaged fly postgres clusters.

import os
DATABASE_URL = os.environ["DATABASE_URL"]

5. Set secrets and deploy

fly secrets set SECRET_KEY="$(openssl rand -hex 32)" --app myapi
fly deploy --app myapi

Secrets are encrypted and exposed as environment variables; never put them in fly.toml or the Dockerfile. Setting a secret restarts the app’s Machines.

6. Scale if needed

fly scale count 2 --app myapi        # two Machines

Verify it worked

fly status --app myapi
curl -sI https://myapi.fly.dev/healthz | head -1           # HTTP/2 200
fly secrets list --app myapi                               # lists DATABASE_URL and SECRET_KEY by name

Common errors

  • The container exits immediately, or the proxy reports no healthy upstream. The server binds loopback, or internal_port differs from the listening port.
  • DATABASE_URL is missing. The attach step was skipped or used the wrong cluster ID; fly mpg list shows IDs.
  • “No Dockerfile found”. Run the command from the directory that holds it.
  • High latency on first request after idle. Autostop woke a stopped Machine; set min_machines_running = 1.
  • Out-of-memory kills. Raise memory under [[vm]]; fly logs shows OOM messages.

FAQ

Does Fly.io have a free Postgres tier?

No. Fly’s Managed Postgres is a paid product with no free tier; at the time of writing, the Basic plan starts at $38 per month.

Why does Fly report no healthy upstream for my FastAPI app?

The server binds 127.0.0.1 or internal_port differs from the listening port. Bind 0.0.0.0 and make internal_port match the port in CMD.

How do I avoid cold starts on Fly.io?

Set min_machines_running = 1, which applies to the primary region; with auto_stop_machines = "stop", idle Machines stop and the next request pays a cold start.

Where should I put secrets for a Fly.io app?

Use fly secrets set, which stores them encrypted and exposes them as environment variables. Never put secrets in fly.toml or the Dockerfile, and expect a restart when you set one.

Backlinks