#crawlers — Public Fediverse posts
Live and recent posts from across the Fediverse tagged #crawlers, aggregated by home.social.
-
September 15, 2026, #Cloudflare will set updated defaults for new domains: #bots classified as #Training or #Agent will be #blocked on pages that display ads, and Search will remain allowed. #Crawlers that combine Search&Training will also be blocked. #AI
https://developers.cloudflare.com/bots/additional-configurations/block-ai-bots/
-
RE: https://social.edu.nl/@wlaatje/116970826745820091
"The web is full of independent archives, hobby databases, local news sites, forums, reference works. Decades of accumulated human effort, running on old code, maintained by small teams or single individuals, quietly holding up far more of our shared knowledge than anyone acknowledges."
And proponents of so-called 'AI' are speedrunning their destruction.
-
Win one of the TEN copies of Vampire Crawlers game by Poncle in #js13k 2026!
https://js13kgames.com/2026/prizes#vampirecrawlers
#gamedev #gamedevjs #Poncle #Vampire #Crawlers #VampireCrawlers #Poncle #Games
-
ICYMI: Microsoft Clarity now flags robots.txt violations inside Bot Analytics: Microsoft Clarity now surfaces robots.txt violations in Bot Analytics, showing publishers which AI crawlers break access rules and what content they target. https://ppc.land/microsoft-clarity-now-flags-robots-txt-violations-inside-bot-analytics/ #MicrosoftClarity #BotAnalytics #SEO #WebAnalytics #Crawlers
-
#Business #Reports
Bots have overtaken humans on the internet · “The open web is now being swamped by AI agents.” https://ilo.im/16dlpj_____
#Bots #Crawlers #WebTraffic #WebAnalytics #AI #Websites #Development #WebDev #Frontend #Backend -
🚀 My new #DDoS book "DDoS: Understanding Real-Life Attacks and Mitigation Strategies" is now also available as an eBook! 🎉
Check it out here: https://ddos-book.com/
I’ve packed in everything I’ve learned from defending major German government sites against groups like Anonymous, Killnet, and NoName057(16).
It covers mitigations against #AI #crawlers and many other defenses for all network layers.
If you find it useful, I’d love it if you could boost and share to help more people defend themselves. ❤️
Thank you! 🙏
#DDoSProtection #NetworkSecurity #DDoS #RealWorldDefense #InfoSec #CyberSecurity #eBook #book
