Sitemap & Robots Checker
Check whether your public sitemap and robots.txt files are reachable and structurally recognisable.
Website report
Scan result
Your scan results will appear here.
Check results
Detected details
How to use this tool
- Enter your website's root URL, such as https://example.com.
- Click Run free check to inspect the standard /robots.txt and /sitemap.xml locations.
- Review whether the files respond, whether a sitemap structure and URL entries are detected, and which sitemap directives appear in robots.txt.
- Open the files on your own domain and check that important public pages are included and not unintentionally blocked.
- If you update either file, submit the sitemap URL in Google Search Console and monitor its processing status.
Example
For a site at https://example.com, the standard files checked are https://example.com/robots.txt and https://example.com/sitemap.xml. If your sitemap uses a different location, this basic check may not find it automatically.
Frequently Asked Questions
What does robots.txt do?
It gives compliant crawlers instructions about which URL paths they may or may not crawl. It is not a security mechanism and does not guarantee that a URL is hidden from search results.
Does robots.txt guarantee that a page will not be indexed?
No. A blocked URL can sometimes still be indexed if discovered through other links. Use appropriate noindex controls when you need to prevent indexing, while ensuring crawlers can access the directive.
What is an XML sitemap?
It is a file that lists important URLs to help search engines discover pages. Submitting a sitemap does not guarantee indexing.
Why does the tool show zero URLs?
The sitemap may be unavailable, malformed, use a format not detected by this check, or be hosted at a different path.
How often should I update my sitemap?
Update it when important indexable pages are added, removed or their canonical URLs change. Keep it consistent with the pages you want search engines to discover.
ToolMitra reports are automated technical indicators based on publicly accessible responses. Results can be incomplete or affected by website configuration, network conditions or bot restrictions. Review important findings manually.
How to check sitemap.xml and robots.txt
- Enter your website's root URL.
- Run the checker to look for publicly accessible sitemap and robots files.
- Review the sitemap URL entries and any broad crawl directives reported.
- Correct the files on your website and run the check again.
Sitemap versus robots.txt
A sitemap helps search engines discover canonical URLs that you want considered for crawling. A robots.txt file gives crawler access instructions. Robots.txt is not a reliable way to keep private information out of search results; use authentication or appropriate access controls for private content. Submitting a sitemap does not guarantee that every URL will be indexed.
Frequently asked questions
Does listing a URL in a sitemap guarantee indexing?
No. Search engines decide whether and when to crawl and index pages.
Can robots.txt remove a page from Google?
Not reliably. Blocking crawling can prevent a crawler from seeing page-level noindex instructions. Use the appropriate removal and access-control methods for your situation.
Where should sitemap.xml be located?
It is commonly placed at the website root and referenced from robots.txt, although sitemaps can also be hosted elsewhere when correctly declared.