fbpx

Google’s Mueller on Crawl Rate for Big and Small Sites – Search Engine Journal

Google’s Mueller on Crawl Rate for Big and Small Sites

Google’s Mueller answers if big sites have a crawl rate advantage over smaller sites in context of need for URL Submission Tool

It’s possible that popular and more frequently linked websites are crawled more often. Links are a part of the crawling process because Googlebot is crawling from link to link.

“Crawling is independent of website size. Some sites have a gazillion (useless) URLs and luckily we don’t crawl much from them. If you have an example from your site where you’re seeing issues, feel free to add it to the form.”

I think what was overlooked in the above exchange is what the publisher meant when they said a “big site” was crawled more often. John Mueller answered the question literally in terms of how many pages a site contained.

Translate »