#scraping — Public Fediverse posts
Live and recent posts from across the Fediverse tagged #scraping, aggregated by home.social.
-
Beyond robots.txt: Implementing ai.txt and llms.txt for Purpose-Based Scraping Control, by (not on Mastodon or Bluesky):
-
Beyond robots.txt: Implementing ai.txt and llms.txt for Purpose-Based Scraping Control, by (not on Mastodon or Bluesky):
-
Beyond robots.txt: Implementing ai.txt and llms.txt for Purpose-Based Scraping Control, by (not on Mastodon or Bluesky):
-
Beyond robots.txt: Implementing ai.txt and llms.txt for Purpose-Based Scraping Control, by (not on Mastodon or Bluesky):
-
Beyond robots.txt: Implementing ai.txt and llms.txt for Purpose-Based Scraping Control, by (not on Mastodon or Bluesky):
-
Scrapers vs Wikis: Person who runs a bunch of custom Wiki websites writes about abuse from scrapers
https://weirdgloop.org/blog/clankers
#via:lobsters #robotstxt #scraping #scaling #wiki #web #ai #+ -
Scrapers vs Wikis: Person who runs a bunch of custom Wiki websites writes about abuse from scrapers
https://weirdgloop.org/blog/clankers
#via:lobsters #robotstxt #scraping #scaling #wiki #web #ai #+ -
Scrapers vs Wikis: Person who runs a bunch of custom Wiki websites writes about abuse from scrapers
https://weirdgloop.org/blog/clankers
#via:lobsters #robotstxt #scraping #scaling #wiki #web #ai #+ -
Scrapers vs Wikis: Person who runs a bunch of custom Wiki websites writes about abuse from scrapers
https://weirdgloop.org/blog/clankers
#via:lobsters #robotstxt #scraping #scaling #wiki #web #ai #+ -
#Development #Demos
WebMCP Demo · How AI agents interact with web pages today and tomorrow https://ilo.im/16d3mq_____
#Comparisons #AI #AiAgents #Content #Website #AgenticWeb #WebMCP #Scraping #WebDev #Frontend -
#Development #Demos
WebMCP Demo · How AI agents interact with web pages today and tomorrow https://ilo.im/16d3mq_____
#Comparisons #AI #AiAgents #Content #Website #AgenticWeb #WebMCP #Scraping #WebDev #Frontend -
#Development #Demos
WebMCP Demo · How AI agents interact with web pages today and tomorrow https://ilo.im/16d3mq_____
#Comparisons #AI #AiAgents #Content #Website #AgenticWeb #WebMCP #Scraping #WebDev #Frontend -
#Development #Demos
WebMCP Demo · How AI agents interact with web pages today and tomorrow https://ilo.im/16d3mq_____
#Comparisons #AI #AiAgents #Content #Website #AgenticWeb #WebMCP #Scraping #WebDev #Frontend -
@RainbowFrog moi j'utilise l'extension webscraper pour ce genre de truc https://webscraper.io/ (dans Firefox ou Chrome)
#webscraping #scraping
@belett @Ash_Crow -
@RainbowFrog moi j'utilise l'extension webscraper pour ce genre de truc https://webscraper.io/ (dans Firefox ou Chrome)
#webscraping #scraping
@belett @Ash_Crow -
@RainbowFrog moi j'utilise l'extension webscraper pour ce genre de truc https://webscraper.io/ (dans Firefox ou Chrome)
#webscraping #scraping
@belett @Ash_Crow -
@RainbowFrog moi j'utilise l'extension webscraper pour ce genre de truc https://webscraper.io/ (dans Firefox ou Chrome)
#webscraping #scraping
@belett @Ash_Crow -
@RainbowFrog moi j'utilise l'extension webscraper pour ce genre de truc https://webscraper.io/ (dans Firefox ou Chrome)
#webscraping #scraping
@belett @Ash_Crow -
Meta seems to be doing some truly hostile anti-scraping webdev
all HTML classes have random IDs and i would not be surprised to see that they also change frequently (i will find out soon enough) (example: https://privacycenter.instagram.com/policy )
is there some sort of "fingerprinting" technique for DOM elements for a situation like this? let's say I have the entire page (downloaded with wget) and i want to answer the question "has the content of this page changed?"
-
Meta seems to be doing some truly hostile anti-scraping webdev
all HTML classes have random IDs and i would not be surprised to see that they also change frequently (i will find out soon enough) (example: https://privacycenter.instagram.com/policy )
is there some sort of "fingerprinting" technique for DOM elements for a situation like this? let's say I have the entire page (downloaded with wget) and i want to answer the question "has the content of this page changed?"
-
Meta seems to be doing some truly hostile anti-scraping webdev
all HTML classes have random IDs and i would not be surprised to see that they also change frequently (i will find out soon enough) (example: https://privacycenter.instagram.com/policy )
is there some sort of "fingerprinting" technique for DOM elements for a situation like this? let's say I have the entire page (downloaded with wget) and i want to answer the question "has the content of this page changed?"
-
Meta seems to be doing some truly hostile anti-scraping webdev
all HTML classes have random IDs and i would not be surprised to see that they also change frequently (i will find out soon enough) (example: https://privacycenter.instagram.com/policy )
is there some sort of "fingerprinting" technique for DOM elements for a situation like this? let's say I have the entire page (downloaded with wget) and i want to answer the question "has the content of this page changed?"
-
Meta seems to be doing some truly hostile anti-scraping webdev
all HTML classes have random IDs and i would not be surprised to see that they also change frequently (i will find out soon enough) (example: https://privacycenter.instagram.com/policy )
is there some sort of "fingerprinting" technique for DOM elements for a situation like this? let's say I have the entire page (downloaded with wget) and i want to answer the question "has the content of this page changed?"
-
🎉 Look, another web scraper! 🎉 Because we *definitely* needed one more tool to fetch JSON from #Wikipedia faster than a cheetah on Red Bull. 🐆💨 No doubt, this will revolutionize the already groundbreaking field of #scraping celebrity birthdates. 🙄✨
https://scrapewithruno.com/ #webscraping #JSONtools #celebritybirthdates #technology #HackerNews #ngated -
🎉 Look, another web scraper! 🎉 Because we *definitely* needed one more tool to fetch JSON from #Wikipedia faster than a cheetah on Red Bull. 🐆💨 No doubt, this will revolutionize the already groundbreaking field of #scraping celebrity birthdates. 🙄✨
https://scrapewithruno.com/ #webscraping #JSONtools #celebritybirthdates #technology #HackerNews #ngated -
🎉 Look, another web scraper! 🎉 Because we *definitely* needed one more tool to fetch JSON from #Wikipedia faster than a cheetah on Red Bull. 🐆💨 No doubt, this will revolutionize the already groundbreaking field of #scraping celebrity birthdates. 🙄✨
https://scrapewithruno.com/ #webscraping #JSONtools #celebritybirthdates #technology #HackerNews #ngated -
🎉 Look, another web scraper! 🎉 Because we *definitely* needed one more tool to fetch JSON from #Wikipedia faster than a cheetah on Red Bull. 🐆💨 No doubt, this will revolutionize the already groundbreaking field of #scraping celebrity birthdates. 🙄✨
https://scrapewithruno.com/ #webscraping #JSONtools #celebritybirthdates #technology #HackerNews #ngated -
🎉 Look, another web scraper! 🎉 Because we *definitely* needed one more tool to fetch JSON from #Wikipedia faster than a cheetah on Red Bull. 🐆💨 No doubt, this will revolutionize the already groundbreaking field of #scraping celebrity birthdates. 🙄✨
https://scrapewithruno.com/ #webscraping #JSONtools #celebritybirthdates #technology #HackerNews #ngated -
NO SOUP FOR YOU!
Playwright
+ Ollama
==TRANSLITERATE==
BEAUTIFUL DATABuild a self-auditing data pipeline that keeps my MariaDB in perfect sync.
Full workflow: https://dufospy.com/artificial-intelligence/data-mining-web-scraping-playwright-ollama
-
NO SOUP FOR YOU!
Playwright
+ Ollama
==TRANSLITERATE==
BEAUTIFUL DATABuild a self-auditing data pipeline that keeps my MariaDB in perfect sync.
Full workflow: https://dufospy.com/artificial-intelligence/data-mining-web-scraping-playwright-ollama
-
NO SOUP FOR YOU!
Playwright
+ Ollama
==TRANSLITERATE==
BEAUTIFUL DATABuild a self-auditing data pipeline that keeps my MariaDB in perfect sync.
Full workflow: https://dufospy.com/artificial-intelligence/data-mining-web-scraping-playwright-ollama
-
NO SOUP FOR YOU!
Playwright
+ Ollama
==TRANSLITERATE==
BEAUTIFUL DATABuild a self-auditing data pipeline that keeps my MariaDB in perfect sync.
Full workflow: https://dufospy.com/artificial-intelligence/data-mining-web-scraping-playwright-ollama
-
NO SOUP FOR YOU!
Playwright
+ Ollama
==TRANSLITERATE==
BEAUTIFUL DATABuild a self-auditing data pipeline that keeps my MariaDB in perfect sync.
Full workflow: https://dufospy.com/artificial-intelligence/data-mining-web-scraping-playwright-ollama
-
OpenAI violated Canadian privacy laws, federal and provincial watchdogs say
Commissioners from four of Canada’s privacy watchdogs have found that OpenAI violated Canadian privacy laws while developing and training its early models of ChatGPT.
Philippe Dufresne, Canada’s privacy commissioner, was joined by his provincial counterparts from British Columbia, Alberta, and Québec to announce the findings of a joint investigation into the tech giant. The investigation examined how OpenAI sourced training data for its early, GPT-3.5 and GPT-4 models, which included scraped content from publicly accessible internet sources like social media and blog posts, licensed third party sources like media outlets and stock image vendors, and user interactions with ChatGPT.
Dufresne noted that all four regulators found OpenAI had violated various federal and provincial privacy laws, including the federal Personal Information Protection and Electronic Documents Act (PIPEDA), and its provincial counterparts in Alberta, BC, and Québec.
Read more at BetaKit
#Alberta #BritishColumbia #consent #OPC #scraping -
Los actuales modelos comerciales de IA generativa han sido desarrollados vulnerando el Reglamento General de Protección de Datos (RGPD) y la Ley de Propiedad Intelectual (LPI). Jamás hubo un pedido de consentimiento por parte de las empresas tecnológicas. Por eso hablamos de ROBO DE DATOS.
#AI #genAI #generativeAI #data #datos #robo #robodedatos #theft #stolen #illegal #technology #bigtech #author #scraping
-
Los actuales modelos comerciales de IA generativa han sido desarrollados vulnerando el Reglamento General de Protección de Datos (RGPD) y la Ley de Propiedad Intelectual (LPI). Jamás hubo un pedido de consentimiento por parte de las empresas tecnológicas. Por eso hablamos de ROBO DE DATOS.
#AI #genAI #generativeAI #data #datos #robo #robodedatos #theft #stolen #illegal #technology #bigtech #author #scraping
-
Los actuales modelos comerciales de IA generativa han sido desarrollados vulnerando el Reglamento General de Protección de Datos (RGPD) y la Ley de Propiedad Intelectual (LPI). Jamás hubo un pedido de consentimiento por parte de las empresas tecnológicas. Por eso hablamos de ROBO DE DATOS.
#AI #genAI #generativeAI #data #datos #robo #robodedatos #theft #stolen #illegal #technology #bigtech #author #scraping
-
Los actuales modelos comerciales de IA generativa han sido desarrollados vulnerando el Reglamento General de Protección de Datos (RGPD) y la Ley de Propiedad Intelectual (LPI). Jamás hubo un pedido de consentimiento por parte de las empresas tecnológicas. Por eso hablamos de ROBO DE DATOS.
#AI #genAI #generativeAI #data #datos #robo #robodedatos #theft #stolen #illegal #technology #bigtech #author #scraping
-
Los actuales modelos comerciales de IA generativa han sido desarrollados vulnerando el Reglamento General de Protección de Datos (RGPD) y la Ley de Propiedad Intelectual (LPI). Jamás hubo un pedido de consentimiento por parte de las empresas tecnológicas. Por eso hablamos de ROBO DE DATOS.
#AI #genAI #generativeAI #data #datos #robo #robodedatos #theft #stolen #illegal #technology #bigtech #author #scraping
-
Reescribiendo Nuestro Scraper co…
Procesos técnicos involucrados La arquitectura de un scraper asíncrono se basa en la gestión eficiente de las conexiones.
https://norvik.tech/news/analisis-asyncio-scraper-norvik-tech
#Technology #Asyncio #Scraping #DesarrolloWeb #Python #NorvikTech #DesarrolloSoftware #TechInnovation
-
Reescribiendo Nuestro Scraper co…
Procesos técnicos involucrados La arquitectura de un scraper asíncrono se basa en la gestión eficiente de las conexiones.
https://norvik.tech/news/analisis-asyncio-scraper-norvik-tech
#Technology #Asyncio #Scraping #DesarrolloWeb #Python #NorvikTech #DesarrolloSoftware #TechInnovation
-
Reescribiendo Nuestro Scraper co…
Procesos técnicos involucrados La arquitectura de un scraper asíncrono se basa en la gestión eficiente de las conexiones.
https://norvik.tech/news/analisis-asyncio-scraper-norvik-tech
#Technology #Asyncio #Scraping #DesarrolloWeb #Python #NorvikTech #DesarrolloSoftware #TechInnovation
-
Reescribiendo Nuestro Scraper co…
Procesos técnicos involucrados La arquitectura de un scraper asíncrono se basa en la gestión eficiente de las conexiones.
https://norvik.tech/news/analisis-asyncio-scraper-norvik-tech
#Technology #Asyncio #Scraping #DesarrolloWeb #Python #NorvikTech #DesarrolloSoftware #TechInnovation
-
J'ai open-sourcé mon pipeline de prospection B2B 🧲
Lead Scraper Pro v2.1
→ 6 sources scrappées en parallèle
→ Déduplication + enrichissement email auto
→ 1 CSV propre pour Mailchimp / Lemlist / CRM
Node.js + Playwright · Licence MIT · Gratuit
⭐ github.com/molokoloco/lead-scraper-pro
#B2B #OpenSource #Scraping #LeadGen -
J'ai open-sourcé mon pipeline de prospection B2B 🧲
Lead Scraper Pro v2.1
→ 6 sources scrappées en parallèle
→ Déduplication + enrichissement email auto
→ 1 CSV propre pour Mailchimp / Lemlist / CRM
Node.js + Playwright · Licence MIT · Gratuit
⭐ github.com/molokoloco/lead-scraper-pro
#B2B #OpenSource #Scraping #LeadGen
