#crawlbudget — Public Fediverse posts
Live and recent posts from across the Fediverse tagged #crawlbudget, aggregated by home.social.
-
Jede Filterkombi in deinem Shop ist eine eigene URL. Bei 8 Filtern schnell 40.000 Seiten für 200 Produkte. 🧮
Crawl-Budget-Selbstmord. Googlebot verbringt mehr Zeit mit "T-Shirt rot XL Nike unter 30€" als mit deiner Hauptkategorie. Relevanzsignale verdünnen sich auf tausende Thin Pages.
Regel: Nur Filterkombis mit echtem Suchvolumen bekommen indexierbare URLs mit eigenem Title und Content. Der Rest: Parameter blocken, noindex, oder nofollow auf Facetten.
-
301 → 301 → 301 → 200. 🔗Jeder Hop kostet Crawl-Budget und verwässert Link Equity. Google folgt maximal rund 10 Weiterleitungen, dann gibt der Bot auf.
In gewachsenen Shops sehe ich regelmäßig Ketten aus 4-5 Redirects, Überreste von Relaunches, Kategorie-Umbauten und HTTPS-Migrationen, die nie aufgeräumt wurden.
Screaming Frog → Redirect Chains Report → alles direkt auf das finale Ziel umbiegen. Einmalige Arbeit, dauerhafter Gewinn.
-
Der einzige Weg zu sehen, was Googlebot wirklich macht: Log Files. 📂Search Console zeigt aggregierte Daten mit Verzögerung. Logs zeigen jede einzelne Anfrage, in Echtzeit, mit Statuscode und User-Agent.
Was ich dort regelmäßig finde: Googlebot crawlt zu 40 % Seiten, die gar nicht ranken sollen. Alte Parameter-URLs, tote Filter, Staging-Reste.
Ohne Log-Analyse optimiert man blind am Crawl-Budget vorbei.
-
Weißt du, wie viele deiner Shop-Seiten Google tatsächlich crawlt? Bei den meisten E-Commerce-Projekten: unter 60 %. 🕷️
Google hat ein begrenztes Crawl-Budget pro Domain. Jede Filterseite, jede Session-URL, jeder Parameter frisst davon. Große Shops mit 500.000+ URLs haben oft das Problem, dass wichtige Produktseiten wochenlang nicht gecrawlt werden.
Logfile-Analyse zeigt sofort wo das Budget verschwendet wird. Nicht raten – messen.
-
Een budget wat elke SEO’er het liefste onbeperkt heeft: het crawl budget. Heb je een laag crawl budget of wil je dat testen? In dit artikel lees je hoe je dit test én kunt optimaliseren. 👉
https://frankwatching.com/archive/2023/08/29/crawl-budget-optimaliseren/ -
Een budget wat elke SEO’er het liefste onbeperkt heeft: het crawl budget. Heb je een laag crawl budget of wil je dat testen? In dit artikel lees je hoe je dit test én kunt optimaliseren. 👉
https://frankwatching.com/archive/2023/08/29/crawl-budget-optimaliseren/ -
2.5x faster responses for Google Crawler!
I rolled out a replica of my application on the West Coast; I noted a 150ms performance improvement in GSC.
#seo #crawlbudget @linode -
2.5x faster responses for Google Crawler!
I rolled out a replica of my application on the West Coast; I noted a 150ms performance improvement in GSC.
#seo #crawlbudget @linode -
Praise da lord, I can finally get the #LinkJuice I’ve been praying for - & don't forget bout muh #CrawlBudget! More optimised than ever 4 da future of my site with #SEO what else could a boi wish for?!
-
Praise da lord, I can finally get the #LinkJuice I’ve been praying for - & don't forget bout muh #CrawlBudget! More optimised than ever 4 da future of my site with #SEO what else could a boi wish for?!
-
2023 is gonna be a great year for #SEO - better crawls, more link juice & infinite crawl budget! (HAHAHAHA THAT'S A GOOD ONE!) #crawlbudget #linkjuice
-
2023 is gonna be a great year for #SEO - better crawls, more link juice & infinite crawl budget! (HAHAHAHA THAT'S A GOOD ONE!) #crawlbudget #linkjuice
-
OMG, sending my page to the Google Gods & praying they give me sum good #linkjuice & a BIGGER #crawlbudget?! Is this what #SEO is all about?? Sheeesh smh
-
OMG, sending my page to the Google Gods & praying they give me sum good #linkjuice & a BIGGER #crawlbudget?! Is this what #SEO is all about?? Sheeesh smh
-
#SEO Tip: for large-scale sites, using a noindex meta tag isn’t a good way to control crawl budget.
It can however indirectly free up crawl budget in the long-run. This is done by Googlebot focusing on other URLs on a site, which can have the long-run impact.
The best practices for improving crawl efficiency (according to Google) include:
- Consolidating duplicate content
- Block crawling of URLs using robots.txt
- Return a 404 or 410 for permanently removed pages
- Keep your sitemaps up-to-date
- Make your pages efficient to loadAs a reminder, crawl budget is an area which should only be of concern to large sites. Google categorises this as sites that either have:
1. 1M+ unique pages with content that changes moderately often (once a week)
2. 10K+ unique pages with rapidly changing content (daily)
3. Sites with a large portion of their total URLs classified in GSC as ‘Discovered - currently not indexed’
This is important information to keep in mind next time you’re asked about how to influence crawl budget, and whether it should be of concern.
-
#SEO Tip: for large-scale sites, using a noindex meta tag isn’t a good way to control crawl budget.
It can however indirectly free up crawl budget in the long-run. This is done by Googlebot focusing on other URLs on a site, which can have the long-run impact.
The best practices for improving crawl efficiency (according to Google) include:
- Consolidating duplicate content
- Block crawling of URLs using robots.txt
- Return a 404 or 410 for permanently removed pages
- Keep your sitemaps up-to-date
- Make your pages efficient to loadAs a reminder, crawl budget is an area which should only be of concern to large sites. Google categorises this as sites that either have:
1. 1M+ unique pages with content that changes moderately often (once a week)
2. 10K+ unique pages with rapidly changing content (daily)
3. Sites with a large portion of their total URLs classified in GSC as ‘Discovered - currently not indexed’
This is important information to keep in mind next time you’re asked about how to influence crawl budget, and whether it should be of concern.
-
If you are a #Marketplace with a lot of products, have you checked if they are all indexed in Google? If your web performance is poor, chances are they are not and you are not benefiting from the #LongTail effect, because of the #CrawlBudget