Free survey
Starting surveyrobots.txt, then XML sitemaps, then a live page sample. Typical scans finish in 8–40 seconds.
Critical
The live page returned 4xx or 5xx.
Advertising a broken URL in a sitemap asks crawlers to fetch a failure.
Product URL in the sitemap now 404s after a catalog prune.
Deep-checked URLs record the terminal HTTP status.
curl -I the URL. Check CDN/WAF blocks that look like 403/429.
Remove dead URLs from the sitemap, or restore the page. Prefer 301 to a replacement over a soft 404.
Sitemapper