Free SEO tool
Free Robots.txt and Sitemap Checker
Two small files decide a lot about crawling. This tool reads your robots.txt, finds your XML sitemap, counts its URLs and spot-checks that the pages it lists really work.
Log in to use this tool
All of our SEO tools are free with an account. Sign up in under a minute, no card needed.
What it checks
What this tool looks at
Crawl rules
Is the file reachable, is the site blocked for Googlebot or for all crawlers, and does it point to a sitemap?
Sitemap discovery
We look at the addresses in robots.txt and at the usual locations, and read sitemap indexes too.
Spot check
Ten random sitemap URLs are requested to catch redirects, errors and http:// leftovers.
How to use it
Three steps
Enter your website
Use your main domain.
We fetch both files
The crawl rules first, then the sitemap they point to.
Read the results
Fix failures first, then the warnings.
Good to know
- It does not test every URL in a large sitemap, only a sample of ten.
- It checks whether the whole site is blocked, not every individual path rule.
- It cannot tell whether Google has indexed the pages. Use Google Search Console for that.
FAQ
Questions about this tool
Do I need a robots.txt file?
No, but it is good practice. Without one, crawlers assume everything is allowed. Having one also lets you point to your sitemap.
What should a sitemap contain?
Only pages you want indexed that return a 200 status: no redirects, no error pages and no noindex pages.
Does disallowing a page in robots.txt remove it from Google?
No. Disallow stops crawling, not indexing. To keep a page out of results, use a noindex tag on a page that crawlers are allowed to read.
More free tools
Keep going
AI Crawler Checker
See which AI crawlers your site allows or blocks, and get the lines to fix it.
Meta Tag Checker & Google Preview
Check a page’s title, description and tags, and preview how Google may show it.
Broken Link Checker
Find dead and redirecting links on any page.
llms.txt Generator
Draft a guide to your site for AI assistants, built from your sitemap.
Schema Markup Generator
Create JSON-LD for organizations, businesses, articles, FAQs, products and breadcrumbs.
SEO Intent Analyzer
Type a keyword to see the search intent behind it, with title ideas and a page outline.
Want the full picture?
This tool checks one thing. The free ZIT Solution SEO audit checks your homepage and up to 50 pages: technical SEO, on-page, content, schema, internal links, speed and AI search readiness, with a prioritized fix list.