home.social

#aiscrapers — Public Fediverse posts

Live and recent posts from across the Fediverse tagged #aiscrapers, aggregated by home.social.

fetched live
  1. @gutenberg_org seems to be down at the moment.

    Let me guess: They are currently under a massive attack by #AIScrapers which are attempting to download all of their ebooks for the 1000th time this month, in the hopes of finding some new scraps of data that they haven't fed into their models?

  2. In other news, #AIScrapers are hammering my poor wiki again - last night, there were more than 36000 (!) "RecentChanges" request within the span of an hour, leading to an extended outage. 😡

    The #AIBubble cannot collapse soon enough.

  3. @Tim_Eagon

    Damnit, are the #AiScrapers causing outages again?

    Right now, the site seems to work.

  4. “If you don't want #AIScrapers training themselves on your website, there's a new way to stop them that doesn't involve server-side blocking or praying they respect your instructions in robots.txt. A team of creatives have teamed up with a typography company to create a new type of font that’ll trick #LLM scrapers into ingesting poisoned gibberish.

    Dubbed #ShieldFont, the #OpenSource project almost seems like magic if you're not familiar with the ins and outs of computer fonts. Look at a #web page written using a ShieldFont font and it’ll appear exactly as one would expect: All the content words (the nouns, verbs, adjectives and adverbs that give a sentence meaning) are the same as the writer originally wrote”

    I’m going to try this ☺️

    #AI / #ArtificialIntelligence / #scrapers / #AIPoisoning / #GSUB <theregister.com/ai-and-ml/2026>

    <github.com/isaqueseneda/shield> (source)

    <shieldfont.org/encoder> (demo)

  5. 'Yang told Fortune that AI agents can’t fully participate in the online economy yet because payments and liability aren’t solved.

    “If we don’t have the infrastructure to do proper payments for agents, then agents aren’t buying and selling, and if agents aren’t buying and selling, then they aren’t generating economic activity,” Yang said.'

    As usual, the Tech Industry sees the "Horrible Future Dystopia" as their next "Business Plan".

    #AIBubble #AIScrapers #TormentNexus
    fortune.com/2026/07/23/dead-in

  6. @pertho

    I have used your #OpenBSD script to keep pf in tune with a list of ip addresses and converted it to FreeBSD, since ftp on FreeBSD is not the same as on OpenBSD.

    Any other (dynamic) blocklist that could be added?

    @grahamperrin @jo @BrideOfLinux

    #noAI #AIscrapers

  7. RE: xoxo.zone/@waxy/11689332743866

    but… how much money did Cloudflare make from #aiscrapers gobbling up the billions of cached web pages they got?

  8. My PineNote blog posts were linked to from the main body of a discussion thread on Hacker News but due to the consistent and prolific AI scraping it's barely a blip on my traffic. Remember when humans used to hug sites to death? In the old days this would absolutely stick out.

    The good news is a human emailed me and asked me for my experience one year in and that was nice (which is also how I found out about the HN post). I'll share my response below until I get around to fleshing out a blog post from it:

    I should absolutely do a write-up a year in. I'll give you the highlights:

    • I lost my original PineNote and after a few months of trying to get it back, I ended up getting a replacement, which should say something all in itself.
    • I wish the device was a little bit smaller and lighter, but I have gotten used to it.
    • I use it as a single function device at a time, mostly writing or reading, very occasional computing.
    • However, I on a month-long trip and I only brought along my work laptop and the PineNote instead of bringing a personal laptop. I can do all the personal computing that I need on the PineNote and my phone. See my setup here: https://shom.dev/posts/20250406_a-pinenote-only-5-day-weekend/
    • I wish Pine64 would update their images with the kernel optimization created by the community member hrdl, they do link to it from their official docs.
    • There is also an interesting project called QuillOS, which provides a nice interface to the whole system, but it's not ready for primetime on PineNote.

    #PineNote #HackerNews #AIScrapers #AIBots

  9. AI scrapers are driving up your hosting costs while real users are left waiting in the digital lobby 🤖

    It is time to take the pressure off your infrastructure by using robots.txt and cache normalization to manage those thirsty bots 💡

    We are sharing how to set sane application limits so your site stays fast for humans and does not turn into a villain story for your budget/.

    👉 developer.upsun.com/posts/insi

    #WebPerformance #DevOps #AIScrapers #TechInsights

  10. RE: mastodon.social/@gamingonlinux

    #RPCS3, open-source PlayStation 3 emulator and debugger, says on Twitter:

    > PSA: #Tencent is aggressively scraping the Internet to build yet another AI slop chatbot, DDoSing many websites in the process.
    >
    > We've found that, as of last week, their scraping bots can now solve Cloudflare challenges and behave like real users while ignoring robots.txt. In the last 24 hours alone, our website received more than 3 million successful requests from Tencent bot IP addresses, plus another 1 million that were blocked by Cloudflare challenges.
    >
    > These recurring DDoS attacks from Tencent have been going on for over a year, and we have been constantly adjusting our firewall rules to filter them while trying not to impact Tencent's real users. Because that is no longer possible, we're now fully blocking Tencent IP addresses, starting with ASN 132203. We recommend other sysadmins do the same.

    x.com/rpcs3/status/20619460007

    #DDoS #AI #AIChatBot #AIScraping #AIScraper #AIScrapers #PlayStation #PlayStation3 #PS3

  11. Why is #twitter not properly identifying itself as a bot when trying to scrape my website? (69.12.56.0/21 is AS63179 is Twitter)

    Could it be cause they're a malicious party training an #aibot?

    (This is extremely low-intensity, but based on the combination of this specific UA and the pages they're trying to reach, I've seen them before, coming in from residential proxies.)

    The funny thing is that bots identifying as bots and observing robots.txt would actually be allowed to reach those particular pages.

    #aiscrapers #scrapers

  12. Traffic sources to my #SelfHosted #Gitea instance. You can clearly see where the real visits are and where the AI scrapers are. Last time I checked, they weren’t triggering any analytic events. They are definitely improving.

    #aiscrapers #ai #llm #LLMs #aislop #homelab #selfhost #selfhosting

  13. @grumpybozo

    30% of web search traffic goes through #AI now.

    The same folk who pontificate about lost web traffic will gleefully tell you they are blocking "#Aiscrapers"

  14. #AIBots may lead to the end of the internet as we know it

    In recent weeks, #OpenDemocracy’s website has been repeatedly brought down by an army of bots. We’re not the only ones

    Matthew Linares
    20 February 2026

    Excerpt: "Slater explained that 'the traffic often arrives through anonymous residential IPs', referring to residential proxy networks that route internet traffic through intermediary servers using IP addresses assigned by internet service providers to real homeowners. This, he said, makes it 'hard to distinguish ‘normal users’ from automated collection'. [That's not right and needs to be changed!!!]

    " 'We're being forced into permanent defence mode. #ResidentialProxyNetworks let #AIScrapers hide in plain sight, rotate identities, and extract data at scale. That shifts real costs onto projects that exist to serve people, not feed training pipelines."

    Read more:
    opendemocracy.net/en/ai-chatbo

    #AISucks #AI #DataMining #Internet #Websites #TechNews #AI #ArtificialIntelligence #BigTech #TechBros

  15. Happy to see some updates on AI.ROBOTS.TXT : « A list of AI agents and robots to block » 🤖 🚫

    github.com/ai-robots-txt/ai.ro

    #NoAI #DNS #AIScrapers

  16. Webspace Invaders - Matthias Ott

    (…) In their hunger for data to train their large language models, companies from all over the world are systematically harvesting every word I’ve ever published, feeding it into their language models to keep them fresh – and the side effect, the collateral damage, is that Kevin in Montreal now can’t read my articles because my hosting provider decided the solution was to block Canada and half the rest of the world.
    I sat there staring at those logs for a while. The irony wasn’t lost on me. This is my little corner of the web. My writing. With my weird little style mixer up there in the top right. And now it is simultaneously being strip-mined by AI companies and effectively made inaccessible to actual humans around the world who might want to read it.
    This is where we are in 2026. (…) Yes, the AI companies need to do better. They actually should throttle their scraping to reasonable levels. They actually should respect the limited resources of small sites. They actually should develop industry standards that don’t externalize costs onto individuals who are just trying to share their work. (…) matthiasott.com

    I can't help but getting really really angry about all this and what it does to the web I used to love.

    #ai #aiScrapers #collateraldamage #exploitation #otemporaomores #Web

    https://webrocker.de/?p=29765

  17. Looks like a new player leads the #ipv4games leaderboard. Who else, but a #proxy provider, just followed by another one of it's kind. Not sure if this would be the best advertisement for a company you'd like to appear as "legit".

    #IPv4 #threatintel #DDoS #AIScrapers

  18. Hmm chatgpt ser ud til at holde fobindelsen til min hjemmeside i live meeeeget længe. Til sammenligning er besøg fra copilot og perplexity normal <1sek.
    #webstats #AIscrapers #weird #danskertrut

  19. Wow, I just read a doozy of a post that had some understandable takes...Also, a very unreasonable and unhinged opinion. They called Anubis, a form of protection against AI Scrapers, malware!

    It was the hottest take possible (one that seemed fueled by consuming 80 tons of Scotch Bonnet chili peppers). I was flabbergasted by it, the creator sarcastically posted this: pony.social/@cadey/11479740961

    This is the blog post the FSF in my opinion, foolishly calling Anubis malware: fsf.org/blogs/sysadmin/our-sma

    My people in Linux Salvation...This is wild, as usual, I just found out about this after watching a Brodie video about it. ROFL

    #Anubis #AIScrapers #FreeSoftware

  20. #AI haters build #tarpits to trap and trick #AIScrapers that ignore robots.txt

    Attackers explain how an anti-spam defense became an AI weapon.

    Ashley Belanger – Jan 28, 2025 4:16 P

    arstechnica.com/tech-policy/20
    #NoAI #NoNukesForAI