The Outercite verifier
OuterciteBot/1.0 (change-verification; +https://outercite.com/crawler)
What it does
When a customer tells us they changed a page, we fetch that one page — once — to confirm the change is live. No crawling, no link following, no site discovery. One URL per verification run.
What it will not do
- Fetch a domain its owner has not verified. A typed URL is not permission; verification requires proving control of the domain (Cloudflare zone or DNS record).
- Ignore robots.txt. A
DisallowforOuterciteBotis honoured. A robots.txt we cannot read means we do not fetch — permission that cannot be checked is not assumed. - Follow a redirect off the page’s own origin. Cross-origin redirects end the run.
- Store your pages. We keep a content hash, a byte count, an HTTP status, and at most 200 characters of context per failing check. Never the page.
- Fetch more than 1.5 MB — larger responses are abandoned mid-stream.
Compatibility probes
Separately, our free site check can test whether a site blocks a named AI crawler. Those probes send that crawler’s User-Agent token with our own identity appended — (compatibility probe by Outercite; +https://outercite.com) — so your access log always shows who was actually asking.
Opting out
Add User-agent: OuterciteBot / Disallow: / to your robots.txt, or email team@outercite.com to opt a domain out entirely.
Machine-readable version: /.well-known/outercite-crawler.json