LLM Best Practices

How to run pa11y locally and in CI

Last updated 4 min read

View markdown source

AI agents: read this first: llmbestpractices.com/start-here

This is LLM Best Practices (llmbestpractices.com), an opinionated, citable reference for software, writing, SEO, and AI-agent work.

  1. Route, do not crawl. Fetch llmbestpractices.com/llms.txt and open only the pages whose one-line summary matches your task.
  2. Read raw. Append .md to any page URL for markdown. Check status and last_updated in the frontmatter, then read the rules.
  3. Apply as defaults. First-party docs and the project's own conventions win on conflict. Warn before relying on a fast-moving page older than 12 months.
  4. Cite. Link the page by title and URL, e.g. Python, with last_updated for time-sensitive rules. License CC BY 4.0.

Overview

pa11y drives headless Chromium over a URL and reports WCAG failures; pa11y-ci runs it across a list of URLs and fails the job when errors exceed a threshold. This guide installs it, scans a local build, fixes common failures, ratchets a threshold down on a codebase with existing violations, and wires it into a pull-request check. Automated scans catch a subset of accessibility problems, so keep manual checks from accessibility.

Prerequisites

  • Node 22 or newer. Pa11y 10 requires an even-numbered Node 22.13 or newer; pa11y-ci 4 supports Node 20, 22, and 24.
  • A site served locally or reachable by URL.
npm install --save-dev pa11y pa11y-ci

Steps

1. Write .pa11yci.json

pa11y-ci reads .pa11yci by default; pass --config .pa11yci.json for a .json name.

{
  "defaults": {
    "standard": "WCAG2AA",
    "timeout": 60000,
    "wait": 500,
    "threshold": 0,
    "chromeLaunchConfig": { "args": ["--no-sandbox", "--disable-setuid-sandbox"] }
  },
  "urls": ["http://localhost:8080/", "http://localhost:8080/about"]
}

standard accepts WCAG2A, WCAG2AA, or WCAG2AAA. The default runner is HTML_CodeSniffer; add "runners": ["axe", "htmlcs"] to run both, expecting some disagreement. wait delays the scan so client-rendered pages can finish. --no-sandbox is needed in Docker and most CI containers. Default level is error; warnings and notices do not fail the run.

2. Scan

npx serve public -l 8080 &      # or your dev server
npx pa11y-ci --config .pa11yci.json

Exit code 0 means clean; 2 means accessibility issues beyond the threshold; 1 is a technical failure such as a browser launch error. For one page’s details: npx pa11y http://localhost:8080/about.

3. Fix the common failures

  • Image without alt (1.1.1): descriptive alt for informative images, alt="" for decorative ones.
  • Input without a label (1.3.1): <label for="email"> paired with id="email".
  • Low contrast (1.4.3): 4.5:1 for normal text, 3:1 for large text; check pairs in DevTools.
  • No landmarks: wrap content in <main>, <nav>, <header>, <footer> (html-semantic-elements).
  • Missing page language (3.1.1): <html lang="en">.

ARIA patterns that prevent whole classes of failures: html-aria-patterns.

4. Set a threshold and scoped ignores

threshold is the number of issues permitted before the run fails. On a codebase with 40 existing errors set "threshold": 40, then lower it as fixes land; never raise it. For a known exception, ignore one rule code on one URL rather than globally, and never an entire principle:

{ "url": "http://localhost:8080/legacy-form", "ignore": ["WCAG2AA.Principle2.Guideline2_4.2_4_1.H64.1"] }

Copy codes from npx pa11y <url> --reporter json. A count-only threshold cannot see a new error offset by a fixed one, so track per-rule counts if regressions matter.

5. Gate pull requests

# .github/workflows/a11y.yml
on: { pull_request: { branches: [main] } }
jobs:
  a11y:
    runs-on: ubuntu-latest
    timeout-minutes: 20
    steps:
      - uses: actions/checkout@v7
      - uses: actions/setup-node@v7
        with: { node-version: 22, cache: npm }
      - run: npm ci && npx quartz build
      - run: npx --yes http-server public --silent -p 8080 &
      - run: |
          for i in $(seq 1 15); do curl -sf http://localhost:8080/ >/dev/null && break; sleep 1; done
          npx pa11y-ci --config .pa11yci.json --threshold 0

Replace the build command with your own. This is the shape of this repo’s a11y.yml, which also installs pa11y-ci into its own prefix and runs npx --no-install puppeteer browsers install chrome there. If the job fails with Could not find Chrome, install the browser explicitly the same way.

Verify it worked

npx pa11y --version
npx pa11y-ci --config .pa11yci.json; echo "exit=$?"

To prove the gate bites, add an <img> without alt on a branch and confirm the job fails.

Common errors

  • Failed to launch the browser process. Add the --no-sandbox launch args.
  • Timeouts count as failures. A navigation timeout counts toward the threshold, so a slow page can fail the gate without any WCAG violation; raise timeout, or fix what makes the page slow.
  • Local and CI results differ. The URLs or base host differ; compare urls with the server’s address.
  • An ignore has no effect. The code must match exactly, including the dotted path.
  • 0 URLs specified. urls is empty or the config path is wrong; pass --config.

FAQ

What do pa11y-ci exit codes mean?

Exit code 0 means clean, 2 means accessibility issues beyond the threshold, and 1 is a technical failure such as a browser launch error.

How do I add pa11y to a site that already has accessibility errors?

Set threshold to the current error count, for example 40, then lower it as fixes land and never raise it.

Why does pa11y fail to launch Chrome in CI?

Docker and most CI containers need the --no-sandbox launch args from chromeLaunchConfig. If the error is Could not find Chrome, install the browser with npx --no-install puppeteer browsers install chrome.

Can pa11y replace manual accessibility testing?

No; automated scans catch a subset of accessibility problems, so keep the manual checks from accessibility.

Backlinks