LLM Best Practices

How to deploy FastAPI on Fly.io

4 min read

View markdown source

Overview

Fly.io runs containers close to users, provisions free Postgres clusters, and charges only for what runs. This guide takes a FastAPI app from zero to deployed: Dockerfile, fly.toml, secrets, Postgres add-on, scale configuration, and a live health-check URL. The FastAPI patterns the app should follow are in fastapi.

Prerequisites

  • flyctl installed and authenticated. brew install flyctl or curl -L https://fly.io/install.sh | sh, then fly auth login.
  • A FastAPI application with a requirements.txt or pyproject.toml. The app must bind on 0.0.0.0:8080 (Fly’s default internal port).
  • Docker installed locally for optional local testing.
  • A free Fly.io account. The Postgres add-on is also free at the hobby tier.

Steps

1. Write the Dockerfile

FROM python:3.12-slim
 
WORKDIR /app
 
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
 
COPY . .
 
EXPOSE 8080
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8080"]

Pin the Python version. The slim variant keeps the image under 200 MB. If you use pyproject.toml, replace the COPY requirements.txt block with RUN pip install ..

Build and test locally:

docker build -t myapi .
docker run -p 8080:8080 myapi
curl http://localhost:8080/healthz

2. Initialize the Fly app

fly launch --no-deploy

Accept the detected app name or pick your own. The command writes fly.toml. Decline the Postgres prompt for now; you will add it manually in step 4.

3. Configure fly.toml

Open the generated fly.toml and confirm these settings:

[build]
 
[http_service]
  internal_port = 8080
  force_https = true
  auto_stop_machines = "stop"
  auto_start_machines = true
  min_machines_running = 0
 
[[vm]]
  memory = "512mb"
  cpu_kind = "shared"
  cpus = 1

auto_stop_machines = "stop" scales to zero when idle, which keeps costs low for low-traffic APIs. Set min_machines_running = 1 for production apps that cannot tolerate cold starts. Set memory to "1gb" if the app loads ML models or large dependency trees.

4. Add a Postgres cluster

fly postgres create --name myapi-db --region iad

This provisions a single-node Postgres cluster. Attach it to your app to inject DATABASE_URL automatically:

fly postgres attach myapi-db --app myapi

Fly sets DATABASE_URL as an app secret. The FastAPI app reads it as an environment variable:

import os
DATABASE_URL = os.getenv("DATABASE_URL")

See postgres for the connection pool settings appropriate for a containerized environment.

5. Set application secrets

Secrets are stored encrypted and injected as environment variables at runtime. Never put them in fly.toml or the Dockerfile.

fly secrets set SECRET_KEY="$(openssl rand -hex 32)" --app myapi
fly secrets set ALLOWED_ORIGINS="https://yourdomain.com" --app myapi
 
# Verify.
fly secrets list --app myapi

6. Deploy

fly deploy --app myapi

Fly builds the image from the local Dockerfile, pushes it to the Fly registry, and creates one machine. The deploy log shows the build stages. Total time is 60 to 90 seconds on a warm builder.

7. Scale

Set minimum replicas for production:

# Scale to 2 machines in the same region.
fly scale count 2 --app myapi
 
# Or set auto-scale bounds.
fly autoscale set min=1 max=3 --app myapi

For multi-region, add regions:

fly regions add lhr --app myapi

Verify it worked

# 1. The app is running.
fly status --app myapi
 
# 2. The health check endpoint returns 200.
curl -sI https://myapi.fly.dev/healthz | head -1
# expected: HTTP/2 200
 
# 3. Database connection is alive.
fly ssh console --app myapi -C "python -c 'import os; import psycopg2; conn = psycopg2.connect(os.environ[\"DATABASE_URL\"]); print(conn.status)'"
 
# 4. Secrets are set.
fly secrets list --app myapi

Common errors

  • Container exits immediately with code 1. The CMD binding uses 127.0.0.1 instead of 0.0.0.0. Fly routes external traffic to the internal network; localhost-only binds fail.
  • DATABASE_URL is undefined. The Postgres attach step was skipped, or fly postgres attach targeted a different app name. Re-run fly postgres attach myapi-db --app myapi.
  • Deploy fails with “no Dockerfile found.” Run fly deploy from the directory containing the Dockerfile.
  • Cold start latency is high. Set min_machines_running = 1 in fly.toml and redeploy.
  • Out-of-memory crash. Increase memory in fly.toml under [[vm]]. Check fly logs --app myapi for OOM messages.

Graph View

Backlinks