#web-harvesting — Public Fediverse posts
Live and recent posts from across the Fediverse tagged #web-harvesting, aggregated by home.social.
-
"There’s an accelerating cat-and-mouse game between web publishers and AI crawlers, and we all stand to lose. [...] Beyond the world of commerce, crawlers are essential for monitoring web security, enabling accessibility tools, and preserving historical archives. Academics, journalists, and civil societies also rely on them to conduct crucial investigative research." https://www.technologyreview.com/2025/02/11/1111518/ai-crawler-wars-closed-web/ #WebHarvesting #WebArchiving #DigitalPreservation