About Web Scanner’s requests
If you found this user agent in your server’s logs:
WebScanner/1.0 (+https://webscanner.pflabs.in/bot)
those requests came from Web Scanner, a tool that shows what is publicly discoverable about a website: its subdomains, their pages, and where each was found. It is built for site owners and their security teams to review their own public footprint.
Who started it
A person starts every scan; there are no automatic or scheduled scans. Before each scan they confirm that they own the domain or have explicit permission from its owner to inspect it, and agree to the Terms of Use. We record that confirmation with each scan.
What it requests
- Passive scans send nothing to your servers. They read only public sources: certificate transparency logs, public subdomain databases, a public web archive, and DNS through public resolvers.
- Standard scans also send a few requests to each of your subdomains that answers: its front page, its
robots.txt, and the sitemaps listed there. - Full crawls also follow the links on those pages, within your domain.
Every request is an ordinary GET, rate-limited per host and capped per scan. The scanner never logs in, submits forms, sends data, guesses addresses or hostnames, or tests for vulnerabilities. It does not try to reach private or internal addresses.
It reads robots.txt only for the sitemaps it lists, and does not treat its other rules as an opt-out. To stop scans of your domain, contact us as described below.
Requests from other places
Web Scanner is open source, and its command-line tool sends the same user agent. Requests from it come from the computer of whoever runs it, not from us; we do not see or store them.
Excluding your domain
To have your domain and its subdomains excluded from future scans on this site, or to have a stored result removed sooner, tell us through our contact page. Say which domain, and that you are its owner or act for them. Stored results are deleted automatically after 7 days in any case.