#webscraper — Public Fediverse posts

I built a Web-Scraper API that is 6-7x more efficient than current ones

https://scrapewithruno.com/

#HackerNews #webscraper #API #efficiency #techinnovation #programming #tools

I built a Web-Scraper API that is 6-7x more efficient than current ones

https://scrapewithruno.com/

#HackerNews #webscraper #API #efficiency #techinnovation #programming #tools

I built a Web-Scraper API that is 6-7x more efficient than current ones

https://scrapewithruno.com/

#HackerNews #webscraper #API #efficiency #techinnovation #programming #tools

I built a Web-Scraper API that is 6-7x more efficient than current ones

https://scrapewithruno.com/

#HackerNews #webscraper #API #efficiency #techinnovation #programming #tools

I built a Web-Scraper API that is 6-7x more efficient than current ones

https://scrapewithruno.com/

#HackerNews #webscraper #API #efficiency #techinnovation #programming #tools

🔍 / #software / #automation / #scraping

#WebScraper - The #1 web scraping extension

The most popular web scraping extension. Start scraping in minutes. Automate your tasks with our Cloud Scraper. No software to download, no coding needed.

🐱🔗 https://laravista.altervista.org/CatLink/links/454

#catlink #softwareautomation

Tìm kiếm web không cần OpenWebUI? Thử MCP server với Jan để scraper ddgs hoặc quantized models với llama.cpp #TìmKiếmWeb #WebSearch #OpenWebUI #LLaMA #AI #MCP #Jan #QuantizedModels #LlamaCpp #TrợGiúp #HỗTrợ #AItools #WebScraper #TìmKiếmTrựcTuyến #CôngCụTìmKiếm #TrangWeb #TìmKiếmNhanh

https://www.reddit.com/r/LocalLLaMA/comments/1os0xwn/how_to_get_web_search_without_openwebui/

🤖 Ottenere l'elenco di tutte le immagini di una pagina HTML con PHP
Sviluppare un web scraper con PHP per ottenere l'elenco completo di tutte le immagini di un url...

👉 https://www.selectallfromdual.com/blog/1639

#html #php #webscraper

8 Web Scraping & Crawling Tools mit n8n-Anbindung (Workflow-Vorlage zum kostenlosen Download)

Wir schauen uns heute an, wie ihr Web Scraping und Crawling betreiben könnt. Dazu schauen wir uns 8 verschiedene Tools an und verbinden diese auch direkt mit n8n, damit ihr die extrahierten Daten in einem Workflow weiter verarbeiten könnt.

https://www.youtube.com/watch?v=LP571gnIg7A

#n8n #ki #automatisierung #webscraping #webcrawler #webscraper

3/

For more on scraping (as in web-scraping) see here:
https://mastodon.social/@reiver/114353728684249608

CC: @404mediaco

#Scraper #Scraping #WebScraper #WebScraping

2/

Scraping (as in Web Scraping) is the act of extracting data from HTML web-pages where the data is NOT machine-legible.

If the data, even in an HTML web-page, is in a machine-legible format, then it is NOT scraping.

...

And, getting data in JSON (key-value pairs) is definitely NOT scraping — as JSON's purpose is to communicate data in a machine-legible manner.

CC: @404mediaco

#Scraper #Scraping #WebScraper #WebScraping

1/

If these researchers used a typical HTTP-based API that returns JSON, then —

What these researchers did is NOT scraping.

CC: @404mediaco

RE: https://www.404media.co/researchers-scrape-2-billion-discord-messages-and-publish-them-online/

#Scraper #Scraping #WebScraper #WebScraping

Oh joy, another "game-changing" #webscraper named #Scraperr 🤖—because apparently, the internet was just crying out for one more script-kiddie #tool to scrape and bloat their hard drives with HTML they’ll never use. Congrats, #GitHub user, your contribution to the overload of useless data is truly groundbreaking. 🚀🎉
https://github.com/jaypyles/Scraperr #dataoverload #scriptkiddie #HackerNews #ngated

Oh joy, another "game-changing" #webscraper named #Scraperr 🤖—because apparently, the internet was just crying out for one more script-kiddie #tool to scrape and bloat their hard drives with HTML they’ll never use. Congrats, #GitHub user, your contribution to the overload of useless data is truly groundbreaking. 🚀🎉
https://github.com/jaypyles/Scraperr #dataoverload #scriptkiddie #HackerNews #ngated

Oh joy, another "game-changing" #webscraper named #Scraperr 🤖—because apparently, the internet was just crying out for one more script-kiddie #tool to scrape and bloat their hard drives with HTML they’ll never use. Congrats, #GitHub user, your contribution to the overload of useless data is truly groundbreaking. 🚀🎉
https://github.com/jaypyles/Scraperr #dataoverload #scriptkiddie #HackerNews #ngated

Oh joy, another "game-changing" #webscraper named #Scraperr 🤖—because apparently, the internet was just crying out for one more script-kiddie #tool to scrape and bloat their hard drives with HTML they’ll never use. Congrats, #GitHub user, your contribution to the overload of useless data is truly groundbreaking. 🚀🎉
https://github.com/jaypyles/Scraperr #dataoverload #scriptkiddie #HackerNews #ngated

Scraperr – A Self Hosted Webscraper

https://github.com/jaypyles/Scraperr

#HackerNews #Scraperr #Webscraper #SelfHosted #TechTools #OpenSource

#Development #Analyses
The text file that runs the internet · Is a basic social contract of the web falling apart? https://ilo.im/15xzdk

_____
#AI #AiModel #GenerativeAI #WebBot #WebCrawler #WebScraper #SearchEngine #Website #Blog #RobotsTxt

I am so happy with the first own web application 🎉 I have developed: Tris, a simple and free web crawler 🕸️ 🕷️ !

You can try it for free online: https://tris.fly.dev, limited to 3 parallel crawls and 100 links of path depth of 3.

Next thing I will add will be a text input to set a target domain hhh, now I am making it hard! 🙈

#node #nodejs #web #webcrawler #crawler #seo #datatools #webscraper #scraping #seotools #seotool #tris #triswebcrawler #webapp #indie #indiedev

I am so happy to get recommendations on fly.io here. I managed to finally deploy my NodeJS web scraper app. World meet Tris: https://tris.fly.dev

#webscraper #scraping #nodejs #seotools #seo

#Development #Collections
Dark Visitors · A list of known AI agents on the internet https://ilo.im/15xjhu

_____
#WebDev #WebScraper #AI #GenerativeAI #Chatbot #Backend #UserAgent #RobotsTxt

Went through a series of side-quests leading to rabbit holes containing kettles of fish to finally get some data from a #webscraper into a persistent database #arangodb. Learned a few things on the way and documented them in a blag post: https://blag.nullteilerfrei.de/2023/10/13/install-nsq-on-debian-with-init-d-and-nginx/

#cryptography #EllipticCurves #nginx #nsq #debian #pyArango #brainpool

Just shipped V2 of last week's little coding project https://scrubr.app

Tldr: webscraping and parsing tool written in #RubyOnRails to make websites more friendly to the eyes and screen readers.

V2 changes below:

#BuildInPublic #BuildingInPublic #rails #Webscraper

I did it again!

So I created #MastoBot, a generic #Python Mastodon bot that allows anyone to create a bot.

I created a few versions, and I use it for @3dprinting. But naturally, knowing how to implement it and developing functions, I need a use case.

So after a discussion this morning. I spent the entire day writing @Python. Yes, I did it again.

However, this one now has a built-in #webscraper to cross-post new posts fromhttps://discuss.python.org/, because why not.

This @Python required a few things, and updates were made to #MastoBot. I had to make it even more generic, implement an overkil datastore with #Redis, and extend the config system.

@Python will behave exactly like @3dprinting with the added feature of crossposts. These posts will, however, be "follower only" posts, to not polute #Python and just flood everything initially.

The bot will #boost parent posts, allowing for threads and discussions to be created.

The source code will be out tomorrow, just cleaning up.

Exploring some datasets online for a little side project and needed to scrap a dynamic table that was a bit stubborn to the basic approaches.
I wrote a script to help with that.

https://github.com/marcel-gaida/five_sugar/tree/main/Web%20Scraper

Feedback appreciated.

#python #notebook #coding #code #selenium #webscraper #data #exploration #datascience

I have found you. Turns out if you access Beautiful Soup elements and don't call decompose on it, it will cause memory leaks.

#python #beautifulsoup #Webscraper

#OpenRefine, est un logiciel libre de nettoyage et de mise en forme de données, il peut être aussi utilisé comme couteau-suisse de Wikimedia, pour :
- importer des données dans Wikidata
-importer des données dans Commons (nouveauté 2021 🎉)
-importer des fichiers dans Commons (nouveauté 2022 🎉)
- récupérer des pages de n'importe quel projet Wikimédia
- récupérer n'importe quelle page sur Internet (#webScraper).
Merci @belett !
Source : https://programme.wikiconvention.fr/#_session-openrefine
#WikiConvention

oo forgot to mention so secondary #Introduction: I'm also a #Pirate #Piracy because intellectual property is fake, #WebScraper, amateur #Archivist, #Coop erator online and off, a #Solidarity-oriented #Tacticalist of an ill-defined #AffinityGroup strain, and a #LaborUnion #Union ist - ask me about flooding yr workplace with fake applications to block scab hiring during a strike❤️

oo forgot to mention so secondary #Introduction: I'm also a #Pirate #Piracy because intellectual property is fake, #WebScraper, amateur #Archivist, #Coop erator online and off, a #Solidarity-oriented #Tacticalist of an ill-defined #AffinityGroup strain, and a #LaborUnion #Union ist - ask me about flooding yr workplace with fake applications to block scab hiring during a strike❤️

oo forgot to mention so secondary #Introduction: I'm also a #Pirate #Piracy because intellectual property is fake, #WebScraper, amateur #Archivist, #Coop erator online and off, a #Solidarity-oriented #Tacticalist of an ill-defined #AffinityGroup strain, and a #LaborUnion #Union ist - ask me about flooding yr workplace with fake applications to block scab hiring during a strike❤️

oo forgot to mention so secondary #Introduction: I'm also a #Pirate #Piracy because intellectual property is fake, #WebScraper, amateur #Archivist, #Coop erator online and off, a #Solidarity-oriented #Tacticalist of an ill-defined #AffinityGroup strain, and a #LaborUnion #Union ist - ask me about flooding yr workplace with fake applications to block scab hiring during a strike❤️

oo forgot to mention so secondary #Introduction: I'm also a #Pirate #Piracy because intellectual property is fake, #WebScraper, amateur #Archivist, #Coop erator online and off, a #Solidarity-oriented #Tacticalist of an ill-defined #AffinityGroup strain, and a #LaborUnion #Union ist - ask me about flooding yr workplace with fake applications to block scab hiring during a strike❤️