Matan · Security · Exposure

Sensitive paths in robots.txt

Disallowing /admin in robots.txt advertises that /admin exists.

Pass condition in our engine: robots.txt does not list paths that look like admin, backups, .git or .env.

1. Problem description

Attackers read robots.txt. Listing /backup, /.git, /phpmyadmin is a map. Disallow does not hide the path from a direct request. Use auth and network controls, not robots.txt, for secrets.

2. Most common causes

  • A CMS plugin writes Disallow: /wp-admin/.
  • Someone “hid” /.env in robots.txt.

3. How to fix it

  1. Remove sensitive paths from robots.txt.
  2. Protect them with authentication and web-server deny rules.
  3. Allow-list public paths instead of advertising private ones.

Test this check

We fetch only this URL (plus robots.txt / llms.txt at the domain root when the check needs them). We do not crawl the rest of the site like a full report.

Scoring rules for the full site crawl are documented in methodology. Want every category at once? Generate a free report.

Get your free SEO & GEO report

Enter your URL and receive a full audit with scoring and recommendations. No sign-up required.

Generate a free reportView sample report