home.social

#etl — Public Fediverse posts

Live and recent posts from across the Fediverse tagged #etl, aggregated by home.social.

  1. CROWler 2.0 has landed 🚀

    Highlights:
    • Redesigned Information Seeds for discovering and onboarding relevant sources
    • Rebuilt email/newsletter discovery that turns useful signals into structured data
    • More powerful AI and non-AI Agents 2.0
    • OpenAPI support for custom API plugins
    • More scalable events and analysis tools
    • Better performance, lower memory use and greater scalability

    github.com/pzaino/thecrowler

    What discovery pipeline will you build next?

    #TheCROWler #DataDiscovery #ETL #OSINT

  2. The Hadoop ecosystem comprises various tools and frameworks designed to handle large-scale data processing and analytics. Let's discuss the core components, namely Hadoop, HBase, and Hive, along with other significant tools such as Pig, Sqoop, Flume, Oozie, and Zookeeper.

    linuxexpert.org/so-you-wanna-d

    #Hadoop #HBase #Hive #BigData #HadoopEcosystem #HDFS #MapReduce #YARN #Pig #Sqoop #Flume #Oozie #Zookeeper #DataProcessing #DataAnalytics #DataWarehousing #ETL #DataIngestion #Security