SERankingPCFBot fetches publicly available web pages to build search and answer-engine visibility reports for SE Ranking customers. PCF stands for Page Content Fetch.
The crawler is identified by the following user-agent:
Mozilla/5.0 (compatible; SERankingPCFBot/1.0; +https://seranking.com/pcf-crawler)
Verification
The user-agent above can be spoofed. To confirm that a request is genuinely from SERankingPCFBot, use a reverse DNS lookup.
A reverse DNS lookup on the requesting IP returns a subdomain of sr-srv.net. Run a forward lookup on that hostname and confirm it resolves back to the original IP address.
Crawling behaviour
SERankingPCFBot always respects the rules specified in your robots.txt file. There is no setting that allows this crawler to ignore robots.txt.
SERankingPCFBot can take intervals of up to 10 seconds between requests to a site. Higher values will be cut down to this 10-second limit. If no crawl delay is specified, SERankingPCFBot will adjust the frequency of requests to your site according to the current server load.
The crawler collects only publicly available information that any visitor can access, and does not use the collected content to train machine learning models.
If your server returns 4xx or 5xx responses, the crawler slows down automatically.
Managing SERankingPCFBot
Please ensure your robots.txt returns an HTTP 200 status code. If it does not, the crawler will treat the file as non-existent.
Slow the crawler down:
User-agent: SERankingPCFBot Crawl-delay: 10
Block part of your site:
User-agent: SERankingPCFBot Disallow: /private/
Block your entire site:
User-agent: SERankingPCFBot Disallow: /
After you change your robots.txt, please allow up to one day for the crawler to fully stop. Some pages may already be queued for processing.
Contact us
If you notice any problem with SERankingPCFBot, write to help@seranking.com and we will investigate immediately.