Our crawler
Upranke Index fetches pages only on websites whose owners have connected them through Google Search Console and asked us to check them. If you see our crawler in your logs, someone with verified access to your site asked for it.
User-agents
| User-agent contains | What it does |
|---|---|
UprankeIndexBot | Pre-checks a submitted page: fetches the page, robots.txt, sitemaps, the homepage and the parent page once per check. |
UprankeIndex-AccessCheck | Crawler access check, run when the site owner clicks “Check access”: fetches one page once per listed search or AI crawler token, to see whether your server or CDN treats that crawler differently. These requests say who is really asking. |
What it does not do
- It never submits forms, logs in or follows links beyond the pages listed above.
- It never fetches private network addresses and stops at 3 MB per response.
- It never pretends to be a search engine’s real crawler: access-check requests carry the crawler’s token so firewall rules can match, plus our own name.
Stopping it
Our pre-check reads your robots.txt like search engines do. To stop checks on your site entirely, disconnect the site from Upranke Index in the app, or write to info@upranke.com from an address on your domain and we will block it for every workspace.