• Help Center
enEnglish
  • enEnglish
  • jpJapan
  • esEspañol
  • frFrançais
  • Why SE Ranking
    • column
      • Scalable solution
        • for Enterprises
        • for Agencies
        • for SMBs
        • for Enterpreneurs
    • column
      • Trusted by users
        • Case Studies
        • Testimonials
        • Our Data
    • All-in-one SEO platform
  • Features
    • column
      • Featured tools
        • Keyword Rank Tracker
        • On-Page SEO Checker
        • Website Audit
        • Competitor Analysis Tool
        • Backlink Checker
    • column
      • Keyword research
        • Keyword Suggestion Tool
        • Keyword Grouper
      • Content Marketing
        • Content Marketing Tool
        • Content Editor
    • column
      • Other seo tools
        • Backlink Tracking Tool
        • SERP Checker
      • Tools for agencies
        • SEO Reporting Tools
        • White Label SEO Tools
        • Lead Generator
        • API
    • All features
  • Plans & pricing
  • Resources
    • column
      • Blog

        Read our comprehensive guides, case studies, interviews with experts and more

      • Webinars

        Discover a collection of videos on burning SEO topics featuring industry experts

    • column
      • What’s New?

        New features, improvements and updates all in one place so you can make the most out of SE Ranking

      • Academy

        Broaden your SEO expertise by joining online courses created by seasoned SEO specialists

    • column
      • Backlink Monitoring guide

        Boost your link building strategy by putting this guide's best practices to work Get your free ebook

Sign inStart free trial
Log out Projects

Website Audit settings

  • How to use Website Audit
  • Crawl Comparison
  • Website Audit settings
  • Generating a sitemap
Home / Knowledge Base / Projects / Website Audit / Website Audit settings

Website Audit settings

Schedule
Source of pages for website audit
Rules for scanning pages
Parser settings
Limits and restrictions
Report setup

With the help of flexible settings, you can fully control the entire process as well as the audit results. Settings can be accessed from any section of the tool.

settings

Schedule

Under this settings block, you can decide how often you want to audit your website.

The following options are available:

1. Manually. If you select this option, you will have to click on the Restart audit button in order to start the audit. For convenience, this button can be found in any section of the tool.

manually

2. Weekly. The audit will run on a weekly basis on the day of the week and time of your choice.

weekly

3. Monthly. The audit will run on a monthly basis on the day of the week and time of your choice.

monthly

alert The time can be set only in the GMT time zone.

However, you can always start the audit manually even if you have set up automatic checks.

Source of pages for website audit

Under this settings block, you can let the tool know which pages you want to audit.

source

Here are the possible options:

  1. Site pages. The tool will crawl every page of the site, starting from the homepage and following every internal link, not including subdomains.
  2. Subdomains. The tool will crawl every page of the site, including subdomains and their pages.
  3. XML sitemap. The tool will crawl every page specified in the XML sitemap.

After the first audit, the tool will automatically find the XML sitemap given that it’s located at domain.com/sitemap.xml. You can also add a link to the XML sitemap manually by clicking on the Add sitemap button. Once the link has been added, be sure to save your changes.

  1. My list of pages. You can add your own list of web pages to be audited. Upload a file in a TXT or CSV format. Every URL address must be written on a new line.

Rules for scanning pages

Under this settings block, you can set rules for crawling site pages.

For example:

1. Take robots.txt directives into account. When this option is enabled, the tool will crawl the site according to the list of valid instructions in the robots.txt file. Otherwise, the tool will ignore robots.txt file instructions.

rules-for-scanning

2. Ignore Noindex. When this option is enabled, the tool will crawl pages ignoring the Noindex directive (in the format of a meta tag or an HTTP response header).

ignore-noindex3. Ignore Nofollow. By enabling this option, you instruct the SE Ranking bot to follow the links on the page ignoring the Nofollow directive (in the format of a meta tag, an HTTP response header, or a link attribute).

ignore-nofollow4. Additional rules. Using this option, you can additionally allow or block the SE Ranking bot from crawling specific directories of the site. To do this, click on the Add rule button.

add-rule

5. Ignore URL parameters. Here you can specify which UTM tags should be ignored in page URLs during the audit. You can either exclude every parameter or manually select the ones to ignore.

ignore-url

Parser settings

The tool allows you to select a bot that will do the crawling, plus it provides access to pages that are closed off to search robots.

parser-settings

  1. Select a User Agent.

The SE Ranking bot is selected by default. If the analyzed site can’t be reached by our bot, you can choose another option from the list. No matter which User Agent is selected, all pages oа your site will be crawled (even if you select Googlebot-Image).

  1. Authorization on restricted pages.

You can crawl closed off website pages. To do this, give our bot access by specifying your login credentials in the appropriate fields.

Limits and restrictions

This section allows you to configure crawling limits and restrictions. You can set separate limits and restrictions for each site under your account.

limits

Here are the available settings:

  1. The maximum number of pages to scan. Choose the number of pages according to your subscription plan.
  2. Maximum scanning depth. Select the crawl depth.
  3. Maximum number of requests. Depending on the capabilities of the server, you can increase the number of requests the server makes per second to speed up report generation. Alternatively, you can decrease it to reduce the server load. By default, the optimal number is set to 5 requests per second.

Report setup

When analyzing website parameters, SE Ranking looks to current search engine recommendations. Under the Report setup section, you can change the parameters that are taken into account by the tool when crawling sites and putting together reports.

report-setup

    Projects
  • Transfer projects from other services
    • Import keyword ranking history to SE Ranking from Rank Ranger
    • Import keyword ranking history to SE Ranking from AgencyAnalytics
    • Import keyword ranking history to SE Ranking from Moz
    • Import keyword ranking history to SE Ranking from SEOprofiler
    • Import keyword ranking history to SE Ranking from Semrush
  • Rankings
    • General project settings
    • Setting search engines and regions
    • Summary tab overview
    • Grouping and sorting keywords
    • Rankings tab overview
    • Adding a target URL
  • Analytics & Traffic
    • SEO Potential
    • Google Search Console integration
    • Tracking snippets
    • Google Analytics integration
  • Competitors
    • Monitoring competitor rankings
    • SERP Сompetitors
    • Finding all competitors
  • Marketing Plan
    • To-Do List
    • Business Listings
  • Website Audit
    • Analyzing pages, resources and links
    • How to use Website Audit
    • Crawl Comparison
    • Website Audit settings
    • Generating a sitemap
  • Page Changes Monitoring
    • How to analyze page changes
    • How to track page changes
  • Backlink Monitoring
    • How to add backlinks to Disavow list
    • How to analyze backlinks
    • How to add backlinks for monitoring
  • Social Media Management
    • Social media metrics
    • How to create a posting schedule
    • How to connect social media accounts
SE Ranking logo
  • English
    • Русский (Russian)
    • Japan (Japanese)
    • Español (Spanish)
    • Français (French)
Mobile Apps:
Awards:
Capterra Best Value 2022 Leader Winter 2023
© 2023 SE Ranking Limited. All rights reserved
SE RANKING
  • About
  • What’s new
  • Contact
  • Help
  • Ask a question
  • Webinars
  • Academy
  • Schedule live demo
  • Competitors
  • Blog
  • Affiliates
  • Partners’ bonuses
  • Terms of use
  • Privacy policy
FEATURES
  • Keyword Rank Tracker
  • Website Audit
  • Backlink Monitor
  • On-Page Checker
  • SERP Checker
  • Marketing plan
  • Keyword Grouper
  • Keyword Research & Suggestions Tool
  • Competitive Research
  • Backlink checker
  • B2B features
  • Lead Generator
  • SEO Reporting Tools
  • Page Changes Monitoring
  • White Label
  • API
  • Social media tool
  • Free SEO tools
GET IN TOUCH

228 Hamilton Avenue, Palo Alto, California, 94301

+1 415 704 4387

15 Ingestre Place,
London, UK W1F 0DU

+44 20 3868 1173
© 2019 SE Ranking Limited. All rights reserved