LLM Best Practices

How to run an SEO-safe site migration

5 min read

View markdown source

Overview

A site migration is a domain change, an HTTPS upgrade, a URL restructure, or a platform swap. Each one rewrites the URL set Google has indexed. Done well, the migration costs a week of fluctuation and then settles. Done badly, it costs months of recovery or never recovers. This playbook lands the change with the redirect map as the single source of truth.

Prerequisites

  • Crawl access to the current site (Screaming Frog, Sitebulb, or wget --spider).
  • A staging host that can serve the new site with X-Robots-Tag: noindex on every response.
  • Owner access to Google Search Console and Bing Webmaster Tools for both the old and new properties.
  • DNS control with TTL lowered to 300 seconds at least 48 hours before cutover.
  • An analytics baseline: 30 days of clicks, impressions, and average position per top URL.

Steps

1. Crawl the current site for a full URL inventory

Run a full-site crawl and export every URL that returns 200. Include canonical, status code, title, H1, indexability, and inbound link count. This is the source-of-truth list for the redirect map.

# Screaming Frog headless example
screamingfrogseospider \
  --crawl https://oldsite.example.com \
  --headless \
  --save-crawl \
  --export-tabs "Internal:HTML" \
  --output-folder ./crawl/

Also pull the top 1,000 URLs from Google Search Console (Performance > Pages, 16-month window) and the top 500 referring URLs from Ahrefs or your backlink tool. Union all three sets. URLs in GSC or backlink data that did not appear in the crawl are orphan pages or off-site assets; redirect them too.

2. Build the 1:1 redirect map old to new

For every old URL, write the new URL. One row per source. No chains, no loops, no many-to-one collapses unless the merge is deliberate.

old_url,new_url,reason
https://oldsite.example.com/blog/post-1,https://newsite.example.com/articles/post-1,renamed-folder
https://oldsite.example.com/images/hero.png,https://newsite.example.com/static/hero.png,asset-move
https://oldsite.example.com/llms.txt,https://newsite.example.com/llms.txt,passthrough

Include robots.txt, sitemap.xml, llms.txt, ai.txt, and all OG image URLs. Missing those is the single most common migration mistake; see discoverability-files for the full list.

3. Stage on a noindex preview

Deploy the new site to a staging host with these response headers on every URL:

X-Robots-Tag: noindex, nofollow

Confirm with curl -I https://staging.example.com/ before serving real content. Do not rely on <meta name="robots"> alone; staging crawlers honor the header faster.

Wire the redirect map into the new platform’s redirect engine (Cloudflare Bulk Redirects, Vercel _redirects, Cloudflare Pages _redirects, or Nginx return 301). See set-up-quartz-with-cloudflare-pages for the Cloudflare Pages syntax.

4. Verify with a parallel crawl on staging

Crawl staging with the same tool. Compare three columns to the production crawl: URL count, indexable count, title match. Resolve every diff before cutover.

diff <(sort prod-urls.txt) <(sort staging-urls.txt)

Run the redirect map through a checker. Every old URL must return 301 (not 302, not 200, not 404) and resolve to the mapped new URL in one hop. Chains break.

5. Flip DNS at the low-traffic window

Pick the hour with the lowest GSC click volume across the past 30 days. Change the A or CNAME record to point at the new host. Because TTL is 300 seconds, propagation completes in under 10 minutes for most resolvers.

Remove the X-Robots-Tag: noindex header on the new host the moment DNS flips. Confirm with:

curl -I https://example.com/ | grep -i x-robots-tag
# (no output is correct)

6. Submit the new sitemap to GSC and Bing

In GSC for the new property: Sitemaps > Add a new sitemap > https://example.com/sitemap.xml. In Bing Webmaster Tools: Sitemaps > Submit. In GSC for the old property: use the Change of Address tool only if the domain itself changed.

Ping indexnow with the new sitemap URL to accelerate Bing and Yandex discovery.

7. Monitor coverage and 404s daily for four weeks

Watch four signals every day for four weeks, then weekly thereafter:

  • GSC Coverage report: Errors and Excluded counts must trend down.
  • GSC Performance: total clicks and impressions stabilize within 14 days.
  • Server logs: any URL returning 404 with referrer traffic is a missing redirect; add it.
  • 301 chain detector: re-run weekly to catch new chains added by content changes.

See master-google-search-console for the GSC report layout.

Verify it worked

# 1. Top 50 URLs all return 301 -> 200 in one hop.
while read url; do
  curl -s -o /dev/null -w "%{http_code} %{redirect_url}\n" "$url"
done < top-50-old-urls.txt
 
# 2. Sitemap is fetched (GSC Sitemaps report shows "Success").
 
# 3. Aggregate clicks recover to within 90% of baseline by week 4.

Common errors

  • Redirect chains. /old 301 to /intermediate 301 to /new halves the equity Google passes. Flatten every chain to one hop.
  • Missing redirects for image URLs. Pinterest pins, embedded blog images, and OG card URLs all die silently. Include every asset in the map.
  • Forgetting to redirect llms.txt, robots.txt, ai.txt, humans.txt, and sitemap.xml. Agents and crawlers hit these first; a 404 here is a trust signal.
  • Cutting over with the staging noindex header still set. The new site is invisible to Google for as long as the header lives. Always grep for it post-cutover.
  • Using 302 instead of 301. Temporary redirects do not pass full equity; Google may keep the old URL indexed indefinitely. Use 301; see redirects for the status-code rules.
  • Changing more than one variable at once. A simultaneous domain change, redesign, and CMS swap makes regression analysis impossible. Sequence the changes.

Graph View

Backlinks