Guide

Robots ↔ sitemap conflicts

URLs in the sitemap should not be Disallow’d in robots.txt. Sitemapper flags ROBOTS_DISALLOWED_IN_SITEMAP and can put that into an agent task pack.

Scan a site