Knowledge Base
Categories: General Information
Telling Google How Often To Crawl Your Website
Google uses sophisticated algorithms for determining how often to crawl your site. Their goal is to crawl as many pages as possible from your site on each visit without overwhelming your server's bandwidth.
Too many requests on your site in a very short period can cause your site to load slowly and even cause load issues on the server. This is especially true of very busy websites or websites that are poorly or inefficiently coded. If you are experiencing bandwidth-related server load issues (i.e. too many requests too quickly), you may want to reduce Google's crawl rate (the time used by Googlebot to crawl your website for an entire domain or subdomain).
If you are not experiencing any bandwidth-related load issues on your server, it is recommended that you allow Google to determine the optimal crawl rate for your website. Please check this article from Google Search Central Reduce the Googlebot crawl rate for the instructions.
* Your feedback is too short