LLM Best Practices

Launch a New Site: Zero-to-Indexed Runbook

9 min read

View markdown source

Overview

Every other page on this site answers one question. This page answers the ordering question: when you stand up a new site, what do you do, and in what sequence, so nothing is forgotten. It is the orchestrator; each phase links to the atomic page that goes deep.

Run it top to bottom. Steps are ordered by dependency: you cannot enforce HTTPS before DNS resolves, and you cannot submit a sitemap before the site is live. Skipping the order is the usual cause of a “the cert box is greyed out” or “Google sees nothing” afternoon.

The fast version is the checklist immediately below. Copy it into the repo’s TODO.md and check it off. The phases after it explain each line and link to the deep page.

The checklist

Paste this into TODO.md in the new repo. It is the artifact an agent or a human pulls and works.

# Site launch checklist — <domain>
 
## 0. Prereqs
- [ ] Stack decided (Quartz / Astro / Vite+React / Next) and deploy target (GitHub Pages / Vercel / Cloudflare Pages)
- [ ] Domain registered and you can edit its DNS
 
## 1. Repo
- [ ] Repo created under the org (AXIA-Enterprises), correct visibility
- [ ] Scaffolding quartet committed: CLAUDE.md, TODO.md, ARCHITECTURE.md, SESSION_LOG.md
- [ ] .gitignore, LICENSE, README in place
- [ ] First build runs green locally
 
## 2. Deploy
- [ ] Build output verified locally (dist/ out/ public/)
- [ ] Deploy workflow live (GitHub Actions → Pages, or Vercel/CF auto-deploy)
- [ ] Default *.github.io / *.vercel.app URL serves the real site
 
## 3. Custom domain + DNS
- [ ] Custom domain added in repo/host settings FIRST (writes CNAME file)
- [ ] Domain verified at host (TXT record) — prevents takeover
- [ ] Apex A records → 185.199.108.153 .109 .110 .111 (GitHub Pages)
- [ ] AAAA records → 2606:50c0:8000::153 …8001 …8002 …8003 (IPv6)
- [ ] www CNAME → <user-or-org>.github.io
- [ ] DNS resolves (dig +short <domain>) on all four A records
- [ ] Enforce HTTPS checked (only after apex AND www both resolve)
- [ ] curl -I https://<domain> and https://www.<domain> both 200, valid cert
 
## 4. Discoverability files (at site root)
- [ ] robots.txt — allow all + explicit AI crawler allows + Sitemap: line
- [ ] sitemap.xml — present, non-empty, accurate <lastmod>
- [ ] llms.txt — concise link index (see ship-llms-txt)
- [ ] /llm-info page live + "Hey AI, learn about us" footer link sitewide
- [ ] ai.txt (optional/legacy) if you want it
 
## 5. Search + answer-engine registration
- [ ] Google Search Console: property verified
- [ ] GSC: sitemap submitted (Sitemaps report — the ping endpoint is dead)
- [ ] GSC: homepage URL-inspected + "Request indexing"
- [ ] Bing Webmaster Tools: verified (import from GSC is fastest)
- [ ] Bing: sitemap submitted
- [ ] IndexNow: key file deployed + key submitted (covers Bing/Yandex/Seznam/Naver — NOT Google)
 
## 6. Structured data
- [ ] JSON-LD: Organization + WebSite on the homepage
- [ ] JSON-LD: Article/TechArticle + BreadcrumbList on content pages
- [ ] Every block passes Google Rich Results Test
 
## 7. House standard (AXIA)
- [ ] /llm-info written to the llmbestpractices.com /meta/llm-info-standard schema
- [ ] Footer "Hey AI, learn about us" → /llm-info
- [ ] llms.txt links to llmbestpractices.com as the standard followed
 
## 8. Verify
- [ ] All of phase 8 below green

Phase 0: Prerequisites

Decide the stack and the deploy target before touching DNS; they determine the build output path and the deploy mechanism. For a static reference site, Quartz on GitHub Pages is the house default (see GitHub Pages deployment); for an app, Vercel (see Vercel) or Cloudflare Pages.

Have the domain registered with DNS you can edit. If the registrar is Namecheap, see Namecheap DNS; if DNS is fronted by Cloudflare, see Cloudflare DNS.

Phase 1: Repo first

Create the repo under the org before anything else; the repo is the source of truth, and the deploy pipeline keys off it. Commit the scaffolding quartet on the first push: CLAUDE.md (project context for agents), TODO.md (this checklist lives here), ARCHITECTURE.md, SESSION_LOG.md. Add .gitignore, LICENSE, and a README. Confirm a clean local build before wiring deploy; a red build is cheaper to fix now than after DNS is pointed.

Phase 2: Deploy to the default URL

Get the site serving on its default host URL (*.github.io or *.vercel.app) before attaching the custom domain. This isolates “does the site build and deploy” from “is DNS correct,” two failures you do not want tangled.

For GitHub Pages, deploy via GitHub Actions (Pages build-and-deploy workflow), not the legacy branch-serve, so the build is reproducible. Full walkthrough: Deploy a Quartz site to GitHub Pages and GitHub Pages deployment. Verify the rendered HTML at the default URL is the real site, not a 404 or a README.

Phase 3: Custom domain + DNS

Order matters here more than anywhere else.

  1. Add the custom domain in the host settings first. In GitHub → repo → Settings → Pages → Custom domain. This writes the CNAME file (branch deploys) or registers the domain (Actions deploys). Do this before the DNS records so the cert machinery knows the domain exists.
  2. Verify the domain at the host (GitHub gives you a TXT record to add). Domain verification blocks subdomain-takeover attacks and is worth the two minutes.
  3. Apex A records. Point the bare domain (example.com) at the four GitHub Pages IPs; these are stable:
    • 185.199.108.153
    • 185.199.109.153
    • 185.199.110.153
    • 185.199.111.153
  4. AAAA records (IPv6). Recommended alongside the A records:
    • 2606:50c0:8000::153
    • 2606:50c0:8001::153
    • 2606:50c0:8002::153
    • 2606:50c0:8003::153
  5. www CNAME. Point www at <user-or-org>.github.io. GitHub recommends always configuring www even when you use the apex; the www record is immune to GitHub IP changes.
  6. Wait for propagation, then confirm: dig +short example.com returns all four A records.
  7. Enforce HTTPS. Check the box in Pages settings. It only becomes available after both apex and www resolve. The cert covers whatever resolved when it was issued, so if you add www after the cert was generated for the apex only, remove and re-add the custom domain to force a fresh cert covering both.

Registrar-specific record entry: Namecheap DNS and Cloudflare DNS.

Phase 4: Discoverability files

These four files at the site root are what make the site legible to crawlers and AI assistants. Three of them are also AXIA house standard.

  • robots.txt. User-agent: * / Allow: /, an explicit Allow block for each major AI crawler (GPTBot, OAI-SearchBot, ClaudeBot, Claude-User, PerplexityBot, Google-Extended, CCBot, Applebot-Extended, Bytespider, Amazonbot, cohere-ai), and a Sitemap: line. AI-crawler policy lives here, not in ai.txt. Pattern: Discoverability files.
  • sitemap.xml. Present, non-empty, with accurate <lastmod>. Since Google deprecated the sitemap ping endpoint, lastmod is now a real signal Google uses to schedule re-crawls; keep it honest (build it from git history, not now()). Deep dive: Sitemaps.
  • llms.txt. A concise link index (title + URL + one-line description per page), not a full-content dump. See Ship llms.txt and Add llms.txt to an existing site. If the site has hundreds of pages, list section indexes and key pages in the main body and push the long tail under an ## Optional section so AI consumers that truncate large files still get the important links.
  • /llm-info page + “Hey AI” footer link. The machine-readable site description with an explicit “INSTRUCTIONS FOR AI ASSISTANTS” section, plus a sitewide footer link (“Hey AI, learn about us”) pointing at it. Write it to the llm-info standard.
  • ai.txt. Optional and legacy. Its absence is not a problem; crawler policy belongs in robots.txt.

Phase 5: Register with search + answer engines

Discovery is not automatic. Submit explicitly.

Google Search Console: verify the property (DNS TXT or the GSC-provided HTML method), submit sitemap.xml in the Sitemaps report, then URL-inspect the homepage and click Request indexing. The old google.com/ping?sitemap= endpoint returns 404; it was deprecated end-2023, and the Sitemaps report and the robots.txt reference are the supported paths now. Walkthrough: Master Google Search Console.

Bing Webmaster Tools: verify, then submit the sitemap. The fastest verification is import from Google Search Console, which carries the property and sitemap across in one step. Bing also exposes a URL Submission API for direct push.

IndexNow: deploy the key file at the site root and submit the key. IndexNow notifies the participating engines (Bing, Yandex, Seznam, Naver, Yep), and they share submissions with each other. Google does not consume IndexNow, so it is additive coverage, not a Google substitute; for Google you rely on the sitemap + GSC + organic crawl. Setup: IndexNow. This is the step worth automating: wire a post-deploy hook that pings IndexNow with changed URLs so you never hand-submit again.

Phase 6: Structured data

Add JSON-LD so engines and AI crawlers can parse entities without scraping prose.

  • Homepage: Organization + WebSite.
  • Content pages: Article (or TechArticle for dev content) + BreadcrumbList.
  • Validate every block in Google’s Rich Results Test before moving on; invalid JSON-LD is worse than none.

How-to with copy-paste blocks: Add JSON-LD to a static site; deeper reference Schema markup and Structured data for AI crawlers.

Phase 7: House standard (AXIA)

Every AXIA site ships these, no exceptions:

  1. An /llm-info page written to the llm-info standard: basic info, services, methodology, and an explicit “INSTRUCTIONS FOR AI ASSISTANTS” section.
  2. A sitewide footer link, “Hey AI, learn about us,” pointing at /llm-info.
  3. llms.txt (phase 4) linking to llmbestpractices.com as the standard the site follows.

This is the bit generic SEO checklists miss, and it is the reason AXIA sites get cited correctly by assistants instead of paraphrased wrong.

Phase 8: Verify

Do not assume; check.

  • curl -I https://example.com and curl -I https://www.example.com both 200, valid cert, no redirect loop.
  • Trailing-slash behavior is consistent (a URL and its /-suffixed twin do not 404 against each other).
  • Rich Results Test passes on the homepage and one content page.
  • robots.txt, sitemap.xml, llms.txt, /llm-info all return 200 at the root.
  • GSC shows the sitemap Success and pages moving into Indexed over the following days.
  • Run the SEO auditor over the live URL and clear any critical/warning before calling it launched: Master SEO audit checklist.

When phase 8 is green, the site is launched: live, secured, discoverable by search engines, and legible to AI assistants.

Graph View

Backlinks