Knowledge Base

Website Crawl Settings - Microsoft

Did you find this article helpful?
Copy Link

* Your feedback is too short

By default, Microsoft sets its crawl rates and times based on the load sensed from your server. If you are experiencing bandwidth-related server load issues, such as too many requests too quickly, you may want to reduce how often Microsoft and other search engines crawl your site. Too many requests on your site in a very short period can cause your site to load slowly and even cause load issues on the server. This is especially true of very busy websites or poorly or inefficiently coded websites.

If you are not experiencing any bandwidth-related load issues on your server, it is recommended that you allow Microsoft to determine the optimal crawl rate for your website.

Changing Microsoft's Crawl Settings

Microsoft allows you to set the crawl rate for your site and even allows you to specify the crawl rate by an hour of the day (a crawl pattern). This will enable you to specify Microsoft to crawl your site more aggressively during non-peak hours and less aggressively while a majority of your visitors are online.

Note: Before specifying a crawl pattern, you may want to view your web statistics in cPanel to see when most of your visitors are online.

To change Microsoft's crawl settings:

  1. Log into your Microsoft Bing | Webmaster Tools.
  2. If you have not already done so, enter your site into the Microsoft Bing | Webmaster Tools, then click Add.

    Microsoft Bing - Add Website

  3. You will need to verify your website. You can choose from the different options provided.

    Microsoft Bing - Choose Verification Method

    If you opted for the XML File method, please follow these steps.

    1. Download the .XML file and upload it to your site's document root.
      You can upload the .XML file directly via the File Manager. Please refer to the article How to Upload a File via File Manager.
    2. Once uploaded, click the Verify button.
  4. A "Site addition successful" page will be displayed if verification is successful.

    Website Successfully Added

  5. Click Done.
  6. You will be redirected to the Microsoft Bing homepage.
  7. Click the hamburger menu icon on the top left.


    Crawl Control
  8. Click on Configuration, then select Crawl Control.
  9. On the Crawl Control page, you can use the presents under the Time when your site receives most of the traffic to indicate your site's busiest time.

    Note: If you wish to modify the crawl rate hours based on your local time zone, you can choose the relevant time zone from the Time zone drop-down menu and set up your preferred crawl pattern.

    Crawl Settings

  10. Alternatively, you can also choose to create your custom hourly crawl pattern by moving over the cubes to the desired number of quadrants for the timeline on the x-axis. (More cubes mean a faster crawl (more quadrants), and fewer cubes mean a slower crawl.)

    Crawl Rate

  11. Click the Save Changes button to save your settings.

For more information, you may refer to Microsoft Bing's documentation.


Did you find this article helpful?
Copy Link

* Your feedback is too short