AureoBot

Put this in your robots.txt. It takes effect on the next request.

User-agent: AureoBot
Disallow: /

There is one case where it will not stop us, and it is fairer to say so than to let you find out. If the site has been added to a paying Aureo workspace, the person who runs it asked for the check, and an owner instruction about their own site outranks a file written for uninvited crawlers. Everywhere else, including the free checker anybody can type a domain into, the two lines above are the whole answer.

Blocking us costs you nothing we can see and nothing you can see. There is no ranking here to fall out of.

• /robots.txt
  First and always. Everything after it depends on what this file says.
• /sitemap.xml
  Or whichever sitemap robots.txt names, or /sitemap_index.xml. If it is an index, at most five child sitemaps, one level down.
• /llms.txt
  Presence only. We record whether it is there and never mark anyone down for it.
• /
  The homepage. Read once, capped at a megabyte.

That is the list. We read your sitemap, we do not follow it: no page is fetched because it appeared there. Nothing is submitted, no form is touched, no script is run, and no part of the site behind a login is reachable, because we sign in to nothing.

Among those requests, exactly one goes out under a different name, and it is the thing on this page most worth explaining.

Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.2; +https://openai.com/gptbot (checked by AureoBot, +https://www.aureoapp.ai/bot)

robots.txt is a request and a CDN is an enforcement, and the two disagree more often than site owners expect. A site can welcome GPTBot in the file and have the edge turn it away, and the owner has no way of knowing. So we knock the way GPTBot knocks, once, at the homepage, and read the status code. Our own address is attached to that string on purpose, so this page is where it leads.

If robots.txt refuses AureoBot on the free checker, this request is not sent either. Asking again under somebody else's name after being told no is not something a page like this could survive.

Two reasons, and no others.

• Somebody added the site to Aureo
  Once a week, when their scan runs. One visit, alongside the rest of the cycle, rather than a second one a minute later.
• Somebody ran the free checker at www.aureoapp.ai/check
  That needs no account, so it may not have been anybody you know. It is capped at six checks per domain per ten minutes, which is there so nobody can point it at your site and leave.

• It identifies itself
  AureoBot/1.0 (+https://www.aureoapp.ai/bot)
• https only
  Never plain http, and never a redirect that downgrades to it partway.
• Five seconds
  Per request, then it gives up. At most three redirects.
• Bodies are capped
  512 KB for robots.txt and llms.txt, 1 MB for a page, 5 MB for a sitemap. It stops reading at the limit rather than trusting a content-length header.
• It stores no personal data
  Status codes, whether files exist, the page title and the head of the HTML. Nothing about your visitors, because it never meets them.

Write to hello@aureoapp.ai with the date and the requesting address and we will tell you what it was. If it is traffic you want stopped today, the two lines at the top are faster than we are.