Free survey

Starting survey

robots.txt, then XML sitemaps, then a live page sample. Typical scans finish in 8–40 seconds.

0%0s elapsedestimating…

  1. robots.txt
  2. XML sitemaps
  3. Live page sample
  4. Score & snapshot
Sitemapper

Guide

Robots ↔ sitemap conflicts

URLs in the sitemap should not be Disallow’d in robots.txt.

What Sitemapper flags

Sitemapper flags ROBOTS_DISALLOWED_IN_SITEMAP and can put that into an agent task pack.

Scan a site