Technical SEO
What is crawl budget.
In short
The number of URLs Googlebot can and wants to crawl on a site in a given period. It mostly matters for large sites, such as stores with tens of thousands of products.
What it consists of
Google does not crawl every page on the web all the time. Crawl budget combines two things: how many requests the server can handle without slowing down (crawl capacity) and how much Google wants to visit the site because it is popular and changes often (crawl demand).
Who it affects
Google says it mainly concerns large sites: roughly a million or more unique pages changing weekly, or 10,000 or more pages changing daily. Sites with many URLs marked 'Discovered - currently not indexed' in Search Console are also affected. A small business site usually does not need to worry about it.
What wastes crawl budget
- Filter and sort combinations in a store that create thousands of unnecessary URLs.
- URL parameters such as session IDs or campaign codes.
- Redirect chains and 404 errors in internal links.
- Duplicate pages without canonicals.
- A slow server that makes Google throttle crawling.
How to improve it
Block URLs that should not be crawled in robots.txt, such as internal search results or some filters. Remove unnecessary links to those pages, fix redirects and speed up the server. Keep only URLs meant for the index in your sitemap.