#scrapers — Public Fediverse posts
Live and recent posts from across the Fediverse tagged #scrapers, aggregated by home.social.
-
75 000+ IP au tapis ! ⛔️🛡️
Et encore, ce chiffre ne montre que ce qui réussit à passer la première ligne de défense.En amont, mon fichier de blocage #Nginx personnalisé fait déjà un gros tri sélectif :
🛑 #Scrapers #IA, faux Chrome et outils automatisés reçoivent un #403 immédiat.
🟢 Les instances du #Fediverse ( #Mastodon, #PeerTube... ), elles, entrent sans problème.Nginx filtre les indésirables, #Fail2Ban verrouille le reste. Résultat : un serveur qui respire et des ressources préservées ! 😎🧹
-
75 000+ IP au tapis ! ⛔️🛡️
Et encore, ce chiffre ne montre que ce qui réussit à passer la première ligne de défense.En amont, mon fichier de blocage #Nginx personnalisé fait déjà un gros tri sélectif :
🛑 #Scrapers #IA, faux Chrome et outils automatisés reçoivent un #403 immédiat.
🟢 Les instances du #Fediverse ( #Mastodon, #PeerTube... ), elles, entrent sans problème.Nginx filtre les indésirables, #Fail2Ban verrouille le reste. Résultat : un serveur qui respire et des ressources préservées ! 😎🧹
-
75 000+ IP au tapis ! ⛔️🛡️
Et encore, ce chiffre ne montre que ce qui réussit à passer la première ligne de défense.En amont, mon fichier de blocage #Nginx personnalisé fait déjà un gros tri sélectif :
🛑 #Scrapers #IA, faux Chrome et outils automatisés reçoivent un #403 immédiat.
🟢 Les instances du #Fediverse ( #Mastodon, #PeerTube... ), elles, entrent sans problème.Nginx filtre les indésirables, #Fail2Ban verrouille le reste. Résultat : un serveur qui respire et des ressources préservées ! 😎🧹
-
75 000+ IP au tapis ! ⛔️🛡️
Et encore, ce chiffre ne montre que ce qui réussit à passer la première ligne de défense.En amont, mon fichier de blocage #Nginx personnalisé fait déjà un gros tri sélectif :
🛑 #Scrapers #IA, faux Chrome et outils automatisés reçoivent un #403 immédiat.
🟢 Les instances du #Fediverse ( #Mastodon, #PeerTube... ), elles, entrent sans problème.Nginx filtre les indésirables, #Fail2Ban verrouille le reste. Résultat : un serveur qui respire et des ressources préservées ! 😎🧹
-
75 000+ IP au tapis ! ⛔️🛡️
Et encore, ce chiffre ne montre que ce qui réussit à passer la première ligne de défense.En amont, mon fichier de blocage #Nginx personnalisé fait déjà un gros tri sélectif :
🛑 #Scrapers #IA, faux Chrome et outils automatisés reçoivent un #403 immédiat.
🟢 Les instances du #Fediverse ( #Mastodon, #PeerTube... ), elles, entrent sans problème.Nginx filtre les indésirables, #Fail2Ban verrouille le reste. Résultat : un serveur qui respire et des ressources préservées ! 😎🧹
-
CW: Adult content
Reminder & FYI to ppl new to Bluesky & my page👇 reposting “leaked” (stolen & posted without personal) porn is really harmful to sw. It means you support that practice. I can’t and won’t allow that in my space. You shouldn’t either. #scrapers #sw #sexworkers #porn #swsolidarity #spreadtheword
-
Strava declara la lucha a los scrapers ayer de su salida a bolsa #antes #BOLSA #declara #guerra #IPO #los #raspado_de_datos #salida #scrapers #Strava #ButterWord #Spanish_News Comenta tu opinión 👇
https://butterword.com/strava-declara-la-lucha-a-los-scrapers-ayer-de-su-salida-a-bolsa/?feed_id=83426&_unique_id=6a1d89e384e07 -
Why is #twitter not properly identifying itself as a bot when trying to scrape my website? (69.12.56.0/21 is AS63179 is Twitter)
Could it be cause they're a malicious party training an #aibot?
(This is extremely low-intensity, but based on the combination of this specific UA and the pages they're trying to reach, I've seen them before, coming in from residential proxies.)
The funny thing is that bots identifying as bots and observing robots.txt would actually be allowed to reach those particular pages.
-
Why is #twitter not properly identifying itself as a bot when trying to scrape my website? (69.12.56.0/21 is AS63179 is Twitter)
Could it be cause they're a malicious party training an #aibot?
(This is extremely low-intensity, but based on the combination of this specific UA and the pages they're trying to reach, I've seen them before, coming in from residential proxies.)
The funny thing is that bots identifying as bots and observing robots.txt would actually be allowed to reach those particular pages.
-
Miasma: A tool to trap AI web scrapers in an endless poison pit
https://github.com/austin-weeks/miasma
#HackerNews #Miasma #AI #web #scrapers #Endless #pit #Tech #innovation #Open #source
-
#fediauthors si je pars du postulat que les LLM scrappent le web et copient/aspirent le contenu que je crée sur mon blog. Comment faire pour m'en prémunir, continuer à partager mes idées et protéger mon contenu de ce vol et de l'utilisation de mes récits pour alimenter ces I.A. ?
Quelles solutions, quels outils ?
Dois-je simplement cesser de créer ? Les capsules #Gemini sont elles scrappées elles aussi ? Si non, combien de temps avant qu'elles ne le soient ?
Merci
#scrapers #LLM #voleDeDonnees #commentFaire #blog -
#fediauthors si je pars du postulat que les LLM scrappent le web et copient/aspirent le contenu que je crée sur mon blog. Comment faire pour m'en prémunir, continuer à partager mes idées et protéger mon contenu de ce vol et de l'utilisation de mes récits pour alimenter ces I.A. ?
Quelles solutions, quels outils ?
Dois-je simplement cesser de créer ? Les capsules #Gemini sont elles scrappées elles aussi ? Si non, combien de temps avant qu'elles ne le soient ?
Merci
#scrapers #LLM #voleDeDonnees #commentFaire #blog -
🤖🔒 A fox-led #crusade to bamboozle #AI #scrapers from a "Git forge" that sounds as mythical as it does unnecessary. 29 minutes of your life wasted on a convoluted game of hide-and-seek with bots, because apparently, that's the hill we're choosing to die on. 🦊💻
https://vulpinecitrus.info/blog/guarding-git-forge-ai-scrapers/ #Fox #GitForge #HideAndSeek #TechHumor #HackerNews #ngated -
🤖🔒 A fox-led #crusade to bamboozle #AI #scrapers from a "Git forge" that sounds as mythical as it does unnecessary. 29 minutes of your life wasted on a convoluted game of hide-and-seek with bots, because apparently, that's the hill we're choosing to die on. 🦊💻
https://vulpinecitrus.info/blog/guarding-git-forge-ai-scrapers/ #Fox #GitForge #HideAndSeek #TechHumor #HackerNews #ngated -
🤖🔒 A fox-led #crusade to bamboozle #AI #scrapers from a "Git forge" that sounds as mythical as it does unnecessary. 29 minutes of your life wasted on a convoluted game of hide-and-seek with bots, because apparently, that's the hill we're choosing to die on. 🦊💻
https://vulpinecitrus.info/blog/guarding-git-forge-ai-scrapers/ #Fox #GitForge #HideAndSeek #TechHumor #HackerNews #ngated -
🤖🔒 A fox-led #crusade to bamboozle #AI #scrapers from a "Git forge" that sounds as mythical as it does unnecessary. 29 minutes of your life wasted on a convoluted game of hide-and-seek with bots, because apparently, that's the hill we're choosing to die on. 🦊💻
https://vulpinecitrus.info/blog/guarding-git-forge-ai-scrapers/ #Fox #GitForge #HideAndSeek #TechHumor #HackerNews #ngated -
#Trump Media is partnering with #Perplexity to bring #AIsearch to #TruthSocial. Perplexity, which has a history of using #scrapers to evade websites and #plagiarising content. Despite Trump Media’s mission to end #BigTech’s influence, the partnership with Perplexity, whose investors include #JeffBezos, highlights the #complexities of Big Tech’s influence. https://www.404media.co/trump-is-launching-an-ai-search-engine-powered-by-perplexity/?eicker.news #tech #media #news
-
#Trump Media is partnering with #Perplexity to bring #AIsearch to #TruthSocial. Perplexity, which has a history of using #scrapers to evade websites and #plagiarising content. Despite Trump Media’s mission to end #BigTech’s influence, the partnership with Perplexity, whose investors include #JeffBezos, highlights the #complexities of Big Tech’s influence. https://www.404media.co/trump-is-launching-an-ai-search-engine-powered-by-perplexity/?eicker.news #tech #media #news
-
#Trump Media is partnering with #Perplexity to bring #AIsearch to #TruthSocial. Perplexity, which has a history of using #scrapers to evade websites and #plagiarising content. Despite Trump Media’s mission to end #BigTech’s influence, the partnership with Perplexity, whose investors include #JeffBezos, highlights the #complexities of Big Tech’s influence. https://www.404media.co/trump-is-launching-an-ai-search-engine-powered-by-perplexity/?eicker.news #tech #media #news
-
#Trump Media is partnering with #Perplexity to bring #AIsearch to #TruthSocial. Perplexity, which has a history of using #scrapers to evade websites and #plagiarising content. Despite Trump Media’s mission to end #BigTech’s influence, the partnership with Perplexity, whose investors include #JeffBezos, highlights the #complexities of Big Tech’s influence. https://www.404media.co/trump-is-launching-an-ai-search-engine-powered-by-perplexity/?eicker.news #tech #media #news
-
#Trump Media is partnering with #Perplexity to bring #AIsearch to #TruthSocial. Perplexity, which has a history of using #scrapers to evade websites and #plagiarising content. Despite Trump Media’s mission to end #BigTech’s influence, the partnership with Perplexity, whose investors include #JeffBezos, highlights the #complexities of Big Tech’s influence. https://www.404media.co/trump-is-launching-an-ai-search-engine-powered-by-perplexity/?eicker.news #tech #media #news
-
#Anubis, an #opensource programme designed to #protect #websites from #AIbot #scrapers, has been downloaded nearly 200,000 times since its launch in January. It is used by organisations like GNOME, FFmpeg, and UNESCO. https://www.404media.co/the-open-source-software-saving-the-internet-from-ai-bot-scrapers/?eicker.news #tech #media #news
-
I specialize in software performance and scalability.
To date I'm aware of about 150 techniques and patterns for achieving that. They are all in my own toolbox. (I hope?)
One of them... is request throttling.
#Twitter
#performance
#scalability
#scaling
#scale
#scraping
#scrapers
#spiders
#bots
#programming
#software
#DDoS
#SRE
#Internet
#DistributedSystems
#HPC
#web
#throttling
#RateLimits -
I specialize in software performance and scalability.
To date I'm aware of about 150 techniques and patterns for achieving that. They are all in my own toolbox. (I hope?)
One of them... is request throttling.
#Twitter
#performance
#scalability
#scaling
#scale
#scraping
#scrapers
#spiders
#bots
#programming
#software
#DDoS
#SRE
#Internet
#DistributedSystems
#HPC
#web
#throttling
#RateLimits
