Free survey
Starting surveyrobots.txt, then XML sitemaps, then a live page sample. Typical scans finish in 8–40 seconds.
Warning
The sitemap lists a URL that robots.txt disallows.
Google may still index a blocked URL if it is linked elsewhere, but it cannot fetch it. Advertising a blocked URL in a sitemap is a conflict.
Disallow: /admin/ while https://example.com/admin/dashboard appears in sitemap.xml.
Sitemapper parses robots.txt with Google-style longest-match rules and tests each sitemap path.
Use Google’s robots.txt tester or curl the path with a Googlebot user-agent after checking Disallow/Allow.
Sitemapper