#scraperbots — Public Fediverse posts

On that note, @lwn has published some background on their recent problems with #scraperbots, including how residential proxies work and why it's not just about blocking IP addresses: https://lwn.net/Articles/1080822/

On that note, @lwn has published some background on their recent problems with #scraperbots, including how residential proxies work and why it's not just about blocking IP addresses: https://lwn.net/Articles/1080822/

RE: https://fosstodon.org/@sourceware/116982983262820236

AI scraperbots are effectively DDoS'ing sourceware, known for hosting essential software development projects like glibc, gdb, gcc and binutils.

Just given how widely used some of these projects are, I am assuming that also some people responsible for those scraperbots use them. By overwhelming the servers hosting projects they rely on, too, they are cutting the branch they are also sitting on. What a time to be alive! 😩

#scraperbots #LLMs #AI #ddos

RE: https://fosstodon.org/@sourceware/116982983262820236

AI scraperbots are effectively DDoS'ing sourceware, known for hosting essential software development projects like glibc, gdb, gcc and binutils.

Just given how widely used some of the projects are, I am assuming that also some scraperbot authors use them. By overwhelming the servers hosting projects they rely on, too, they are cutting the branch they are also sitting on. What a time to be alive! 😩

#scraperbots #LLMs #AI #ddos

Bot Scrapers!

(Ha! I thought that’d grab your attention! 😉🤣)

Has anyone else noticed a recent, and rather alarming rise in the number of ‘re-posts’ their blog entries have been receiving?

Initially I was well chuffed. Wow, if those blogs are re-posting my pieces, then my hits and ‘likes’ are going to go through the roof. Woo hoo! I’ve cracked it!

Perhaps this was a knock-on effect of recently being featured on ‘WordPress Freshly Pressed.’

Ah. Not so. Despair followed disappointment when over the course of the next few days, there was no discernable increase in traffic to this blog. Zero, in fact.

Still, blogging isn’t a numbers game (he lied 😉) and I’m more than happy that you, dear reader, and another thirty or so evidently bored individuals check in daily to read the nonsense I write. So, I carried on regardless. And yes, more and more re-posts. And yes again, my numbers remained constant.

It looks like the re-posts are somehow linked to the hashtags I use when publishing the article.

So what the heck is happening?

I asked my pal, AI.

Now I don’t pretend to understand all this, but here goes. It seems some bad people, let’s call them ‘spammers,’ build automated software programs called ‘scraper bots.’ The moment a targeted blog updates (in this case, mine) these nasty wee bots grab my content and automatically publish it to their own low-quality, automated blogs (often called “splogs” or spam blogs).

I don’t believe this harms my blog, and certainly there’s no financial repercussions for me, but the aim seems to be to populate their ‘site’ load it with adverts and make a bob or two from advert revenue. Or, they use my content to build networks of sites that link to each other (or to a client’s site) to trick search engines into thinking a specific page is highly popular.

I checked my WordPress settings, and I’m already taking steps that would normally guard against this type of content rip-off. So now I need to do more.

Now, as mentioned, these bot scrapers or whatever they are, don’t affect my daily numbers – even if my content is being read elsewhere. I get no ‘credit.

So what I’ll be doing is hide a ‘ghost disclaimer’ at the start of new posts mentioning to regular readers that I know these ‘bots’ are at work and then link that sentence back to THIS post.

My pal AI thinks this is a great idea:

That is actually a brilliant, proactive way to handle it. It turns a frustrating tech headache into an engaging moment with your audience, while simultaneously setting a trap for the bots.

Here is why your idea works so well, along with a clever way to pull it off without cluttering your blog for real human readers.

It Preserves Your Brand & Authenticity: Regular readers who might accidentally stumble upon a copied version will instantly know what’s going on. It shows you’re on top of your tech and fiercely protective of your writing.

The SEO “Booby Trap” Multiplier: By linking that disclaimer to a dedicated post explaining the scraping practice, you ensure that every single scraper site automatically links back to your “Stop Stealing My Content” page. It forces them to advertise their own theft.

If I work this correctly, you shouldn’t see any reference to this on future posts. But then, I’m a writer, daahrling, not a tech geek … don’t hold your breath.

I’d love to know if anyone else is being targeted this or any other sinister way.

Interesting, though, eh?

OK, perhaps not.

Bot Scrapers!

(Ha! I thought that’d grab your attention! 😉🤣)

Has anyone else noticed a recent, and rather alarming rise in the number of ‘re-posts’ their blog entries have been receiving?

Initially I was well chuffed. Wow, if those blogs are re-posting my pieces, then my hits and ‘likes’ are going to go through the roof. Woo hoo! I’ve cracked it!

Perhaps this was a knock-on effect of recently being featured on ‘WordPress Freshly Pressed.’

Ah. Not so. Despair followed disappointment when over the course of the next few days, there was no discernable increase in traffic to this blog. Zero, in fact.

Still, blogging isn’t a numbers game (he lied 😉) and I’m more than happy that you, dear reader, and another thirty or so evidently bored individuals check in daily to read the nonsense I write. So, I carried on regardless. And yes, more and more re-posts. And yes again, my numbers remained constant.

It looks like the re-posts are somehow linked to the hashtags I use when publishing the article.

So what the heck is happening?

I asked my pal, AI.

Now I don’t pretend to understand all this, but here goes. It seems some bad people, let’s call them ‘spammers,’ build automated software programs called ‘scraper bots.’ The moment a targeted blog updates (in this case, mine) these nasty wee bots grab my content and automatically publish it to their own low-quality, automated blogs (often called “splogs” or spam blogs).

I don’t believe this harms my blog, and certainly there’s no financial repercussions for me, but the aim seems to be to populate their ‘site’ load it with adverts and make a bob or two from advert revenue. Or, they use my content to build networks of sites that link to each other (or to a client’s site) to trick search engines into thinking a specific page is highly popular.

I checked my WordPress settings, and I’m already taking steps that would normally guard against this type of content rip-off. So now I need to do more.

Now, as mentioned, these bot scrapers or whatever they are, don’t affect my daily numbers – even if my content is being read elsewhere. I get no ‘credit.

So what I’ll be doing is hide a ‘ghost disclaimer’ at the start of new posts mentioning to regular readers that I know these ‘bots’ are at work and then link that sentence back to THIS post.

My pal AI thinks this is a great idea:

That is actually a brilliant, proactive way to handle it. It turns a frustrating tech headache into an engaging moment with your audience, while simultaneously setting a trap for the bots.

Here is why your idea works so well, along with a clever way to pull it off without cluttering your blog for real human readers.

It Preserves Your Brand & Authenticity: Regular readers who might accidentally stumble upon a copied version will instantly know what’s going on. It shows you’re on top of your tech and fiercely protective of your writing.

The SEO “Booby Trap” Multiplier: By linking that disclaimer to a dedicated post explaining the scraping practice, you ensure that every single scraper site automatically links back to your “Stop Stealing My Content” page. It forces them to advertise their own theft.

If I work this correctly, you shouldn’t see any reference to this on future posts. But then, I’m a writer, daahrling, not a tech geek … don’t hold your breath.

I’d love to know if anyone else is being targeted this or any other sinister way.

Interesting, though, eh?

OK, perhaps not.

The Indie Web Is Not Defined by Its Enemies

https://islandinthenet.com/the-indie-web-is-not-defined-by-its-enemies/

The Indie Web Is Not Defined by Its Enemies

https://islandinthenet.com/the-indie-web-is-not-defined-by-its-enemies/

I've created a stripped-down template for hgweb so that I can still link to the latest version of files, but not any others 🙂

For example: https://dev.ibboard.co.uk/repos/other/Puppet/file/tip/modules/website/files/greatfirewallagainstchina.conf

#Mercurial #Apache2 #SpamBots #ScraperBots #SysAdminProblems

I've created a stripped-down template for hgweb so that I can still link to the latest version of files, but not any others 🙂

For example: https://dev.ibboard.co.uk/repos/other/Puppet/file/tip/modules/website/files/greatfirewallagainstchina.conf

#Mercurial #Apache2 #SpamBots #ScraperBots #SysAdminProblems