Website Audit FAQ

This guide is especially important if you actively use SE Ranking’s Website Audit and On-Page SEO Checker.

The Cloudflare Firewall provides protection from vulnerabilities such as malicious incoming traffic. Sometimes, its automatic rules block our crawlers and prevents SE Ranking’s bots from crawling your website.

But you can easily remove this restriction because Cloudflare provides users with great flexibility. Just follow these steps: 

1. Log in to the Cloudflare dashboard, and select your account and domain.
2. Go to the Security section > click WAF  > choose the Tools subsection 
   
3.  Under IP Access Rules, first enter one of SE Ranking’s IPs. The rule must be set for each IP separately.

144.76.237.123
162.55.90.109
162.55.94.175
195.201.82.119
142.132.195.214
168.119.139.232
148.251.139.87
46.4.23.25
138.201.137.152
116.202.175.208

Select an Allow action. Then for Zone, select whether the rule applies to the current website only or to all websites in the account. Then push an Add button.

Finally, do the same steps with other IPs.

That’s it! If you want to know more, go to the official Cloudflare guide.

If, after whitelisting the IPs, there is still no access, then it is highly likely due to Cloudflare’s Bot Fight Mode. We recommend you deactivate it and then try again. More information about Bot Fight Mode, including how to disable it, can be found on the official page here.

The website audit starts automatically when you create a new project. To restart the audit, click the appropriate button in the Website Audit module. Also, you can set up a schedule for automatic rechecks – once per week or month in the Settings.

Since your website is checked against dozens of parameters, we can’t predict exactly how long it will take the system to complete the audit. It depends on the number of pages and images on the website, whether there is a crawl delay in the robots.txt file, as well as the internal queue in the system. You will be able to monitor the crawling process in the Website Audit module.

You will get a notification via email once the audit report has been generated.

The Website Audit scans all pages of a website and provides an analysis along with suggestions on how to fix identified problems. An example of the report can be found here.

The On-Page SEO Checker audits a certain URL and evaluates how well it is optimized for a certain query. It detects mistakes that negatively affect the given page’s search engine ranking. An example of the report can be found here.

What user agent does your crawler use?

The user agent of our crawler is "SEBot-WA". Our crawler always follows the instructions specified in the robots.txt file.