Security maturity
robots.txt check
robots.txt guides search engine crawlers. It does not hide anything, and the paths it lists are public.
Check this on your own domain now. It is free with a DomainGuard account, reads only public records, and gives you a report you can send to whoever runs your website.
What we look at
We read /robots.txt and list the paths it asks crawlers to skip.
Why it matters to your business
People sometimes list private areas in robots.txt hoping to hide them, which publishes a map of exactly where to look. It is worth checking that nothing listed there is meant to be secret.
How to fix it
- Protect private areas with a login, not with robots.txt.
- Remove entries that point at sensitive paths.
If you would rather not touch DNS or server settings yourself, forward this page and your report to whoever manages your website, or ask us to do it.