Matan · Security · Exposure
Sensitive paths in robots.txt (Security)
Disallowing /admin in robots.txt advertises that /admin exists.
Pass condition in our engine: robots.txt does not list paths that look like admin, backups, .git or .env.
This Security check is part of SEO-GEO scoring 2.2.1, last updated 2026-09-18. About 90 percent of the evaluation runs on the HTML we crawl, so you can verify the same pass condition with a free report.
1. Problem description
Attackers read robots.txt. Listing /backup, /.git, /phpmyadmin is a map. Disallow does not hide the path from a direct request. Use auth and network controls, not robots.txt, for secrets.
2. Most common causes
- A CMS plugin writes Disallow: /wp-admin/.
- Someone “hid” /.env in robots.txt.
3. How to fix it
- Remove sensitive paths from robots.txt.
- Protect them with authentication and web-server deny rules.
- Allow-list public paths instead of advertising private ones.
Test this check
We fetch only this URL (plus robots.txt / llms.txt at the domain root when the check needs them). We do not crawl the rest of the site like a full report.
Scoring rules for the full site crawl are documented in methodology. Want every category at once? Generate a free report.
Get your free website audit
Enter your URL and receive a full audit with scoring and recommendations. No sign-up required.