#datapipeline — Public Fediverse posts

Romans invented data pipelines. We checked. The engineering holds up.

#Rubycon2026 #Ruby #DevHumor #DataPipeline #OpenSource

Every data professional should understand these seven core concepts.

From data warehouses and lakes to pipelines, meshes, and governance, these form the foundation of modern analytics infrastructure.
Mastering them bridges the gap between raw data and actionable business insights.

📕 https://ebokify.com/ai-data-science

#DataEngineering #DataScience #DataAnalytics #ETL #DataWarehouse #BigData #BusinessIntelligence #DataPipeline #DataGovernance

🔥 Hot take alert! 🔥 "I would love there to be a SaaS apocalypse" in observability – that's according to Ari Zilka, CEO at #datapipeline engine startup MyDecisive AI, and a former exec at New Relic and #
Hortonworks. MyDecisive's #opensourcesoftware targets data management costs from #observability vendors such as Datadog by reducing data at the source; it can also proactively alert on and take action on #OpenTelemetry data as it comes in from enterprise applications.

If Zilka has his way, #o11y will return to enterprises as an in-house discipline, but without requiring an army of engineers to set it up.

https://youtu.be/emk_NMicaI8?si=6KeETcQJIoJrJC9h

RE: https://mastodon.social/@TanjaStrange/116186351909288077

This #story isn’t really about a typo.

It’s about:
- digital identity accuracy
- metadata pipelines in media distribution
- algorithmic visibility of artists
- and a broken chain of responsibility

#namesmatter #entertainment #news #metadata #technews #ICYMI #respecttheartist #QAFail #datapipeline #retailers #actorsunions #moviestudios #amazon

There are only three types of tables

https://katharinabrunner.de/2025/11/there-are-only-three-types-of-tables/

NASA's #data collection has undergone massive shifts in lifecycle management, #FAIR_data, technological trends, and policy. Evolving from 1980s magnetic tapes to a network of over 30 online repositories, the tech trends are easiest to identify. Adoption of #DataPipeline and #DataStandards an ongoing focus. #DataGovernance and #DataRescue are emerging. NASA #DataStewardship navigates rapid technology and long-term science.

Bugbee, K., & Ramachandran, R. (2025) https://doi.org/10.1029/2025EA004413 #SciLit

NASA's #data collection has undergone massive shifts in lifecycle management, #FAIR_data, technological trends, and policy. Evolving from 1980s magnetic tapes to a network of over 30 online repositories, the tech trends are easiest to identify. Adoption of #DataPipeline and #DataStandards an ongoing focus. #DataGovernance and #DataRescue are emerging. NASA #DataStewardship navigates rapid technology and long-term science.

Bugbee, K., & Ramachandran, R. (2025) https://doi.org/10.1029/2025EA004413 #SciLit

NASA's #data collection has undergone massive shifts in lifecycle management, #FAIR_data, technological trends, and policy. Evolving from 1980s magnetic tapes to a network of over 30 online repositories, the tech trends are easiest to identify. Adoption of #DataPipeline and #DataStandards an ongoing focus. #DataGovernance and #DataRescue are emerging. NASA #DataStewardship navigates rapid technology and long-term science.

Bugbee, K., & Ramachandran, R. (2025) https://doi.org/10.1029/2025EA004413 #SciLit

NASA's #data collection has undergone massive shifts in lifecycle management, #FAIR_data, technological trends, and policy. Evolving from 1980s magnetic tapes to a network of over 30 online repositories, the tech trends are easiest to identify. Adoption of #DataPipeline and #DataStandards an ongoing focus. #DataGovernance and #DataRescue are emerging. NASA #DataStewardship navigates rapid technology and long-term science.

Bugbee, K., & Ramachandran, R. (2025) https://doi.org/10.1029/2025EA004413 #SciLit

NASA's #data collection has undergone massive shifts in lifecycle management, #FAIR_data, technological trends, and policy. Evolving from 1980s magnetic tapes to a network of over 30 online repositories, the tech trends are easiest to identify. Adoption of #DataPipeline and #DataStandards an ongoing focus. #DataGovernance and #DataRescue are emerging. NASA #DataStewardship navigates rapid technology and long-term science.

Bugbee, K., & Ramachandran, R. (2025) https://doi.org/10.1029/2025EA004413 #SciLit

Have you ever needed to extract text from images embedded in a #PDF? I can highly recommend the open source #CLI tool #OCRmyPDF which is easy to automate in for example a #DataPipeline.

It uses #Tesseract #OCR under the hood and has many options to experiment with to get the best possible accuracy for your language and PDF content.

You can get started with just a few commands:

https://samuelplumppu.se/blog/automated-text-extraction-from-pdf-images-with-ocrmypdf

One Open-source Project Daily

Change data capture for a variety of databases.

https://github.com/debezium/debezium

#1ospd #opensource #apachekafka #cdc #changedatacapture #datapipeline #database #debezium #eventstreaming #kafka #kafkaconnect #kafkaproducer

Test your skills with our Data Engineering Quiz!

Tag a data nerd friend who’d ace it!

#DataEngineering #ETL #TechQuiz #DataNerd #EngineerLife #LearnWithFun #TechCareers #BigData #DataPipeline
#DataEngineerLife #SQL #ApacheSpark
#CareerInTech #LearnDataEngineering
#EngineeringQuiz #DataNerd #UpSkill
#TechCareers #DataDriven

Test your skills with our Data Engineering Quiz!

Tag a data nerd friend who’d ace it!

#DataEngineering #ETL #TechQuiz #DataNerd #EngineerLife #LearnWithFun #TechCareers #BigData #DataPipeline
#DataEngineerLife #SQL #ApacheSpark
#CareerInTech #LearnDataEngineering
#EngineeringQuiz #DataNerd #UpSkill
#TechCareers #DataDriven

#Coroot is excited to partner with #Glassflow to share how you can optimize data streaming, storage, and observability using a fully-#FOSS stack (including tools like GlassFlow, #Kafka, #Clickhouse and Coroot!): https://t.ly/oVAOL

#SRE #DevOps #tech #datapipeline #Python #Apache #linux #observability

Data patterns for community survey data integrated into a data harmonization workflow description, O'Brien etal 2021 tackles the issue of diverse data harmonization for ecological community surveys. https://www.sciencedirect.com/science/article/pii/S1574954121001655 #DataScience #DataPipeline #DataCuration #SciLit

Update: Since this post is getting attention again, I should probably let everyone know that I got #FediHired by a nice little AI company about two months ago 👩🏻‍🎤 thank you all for helping make that possible, and I'm sorry for blowing up your timelines with this update 🤪

Well, the company I work(ed) for just folded. Poof! 💩⛈️.
No notice, no severance, no nothing. 🤬

So I'm asking for boosts and leads. If anyone needs someone who knows data, please reach out.

I'm located in Seattle, WA. Remote preferred.

Last few titles include: CDO, Head of Data, and Sr. Operations Analyst.

I've worked in gaming, fintech, and B2C/C2C marketplaces most recently.

I'm proficient in Python, SQL, statistics, team management, (almost) all things data-related, and a host of other stuff.

I'm opinionated and anti-capitalist, but I also routinely bring in multiple times my department's cost in profits for the companies I work for.

I'm also hella nice (despite my RBF) and easy to get along with.

God, pitching myself is so awkward.

That's the toot. Thanks.

💜💜💜
Edit: I wanted to say thank you to everyone who is sharing this and a huge thank you to everyone who has reached out with encouragement and leads. This community is amazing.

Edit 2: I'm going through all the replies I got, and will reach out to a lot of you soon. I've just finished updating my resume and LinkedIn.

I realize that I'm going to have to give up some of my anonymity when I respond to people, so I'd really appreciate it if anyone I send my resume or LinkedIn profile to could keep my work and personal life separate. Thank you.
💜💜💜

---

**Gif is of parent company explaining our off-boarding process**

#PleaseBoost #Boost #FediHire #LWF #Layoffs #LookingForJob #LookingForWork #Data #DataScience #Python #SQL #Snowflake #ELT #DataPipeline #Analytics #BusinessIntelligence #CDO #HeadOfData #PleaseHelp #Gaming #Fintech #ChiefDataOfficer #HireAlice #ThankYou

Update: Since this post is getting attention again, I should probably let everyone know that I got #FediHired by a nice little AI company about two months ago 👩🏻‍🎤 thank you all for helping make that possible, and I'm sorry for blowing up your timelines with this update 🤪

Well, the company I work(ed) for just folded. Poof! 💩⛈️.
No notice, no severance, no nothing. 🤬

So I'm asking for boosts and leads. If anyone needs someone who knows data, please reach out.

I'm located in Seattle, WA. Remote preferred.

Last few titles include: CDO, Head of Data, and Sr. Operations Analyst.

I've worked in gaming, fintech, and B2C/C2C marketplaces most recently.

I'm proficient in Python, SQL, statistics, team management, (almost) all things data-related, and a host of other stuff.

I'm opinionated and anti-capitalist, but I also routinely bring in multiple times my department's cost in profits for the companies I work for.

I'm also hella nice (despite my RBF) and easy to get along with.

God, pitching myself is so awkward.

That's the toot. Thanks.

💜💜💜
Edit: I wanted to say thank you to everyone who is sharing this and a huge thank you to everyone who has reached out with encouragement and leads. This community is amazing.

Edit 2: I'm going through all the replies I got, and will reach out to a lot of you soon. I've just finished updating my resume and LinkedIn.

I realize that I'm going to have to give up some of my anonymity when I respond to people, so I'd really appreciate it if anyone I send my resume or LinkedIn profile to could keep my work and personal life separate. Thank you.
💜💜💜

---

**Gif is of parent company explaining our off-boarding process**

#PleaseBoost #Boost #FediHire #LWF #Layoffs #LookingForJob #LookingForWork #Data #DataScience #Python #SQL #Snowflake #ELT #DataPipeline #Analytics #BusinessIntelligence #CDO #HeadOfData #PleaseHelp #Gaming #Fintech #ChiefDataOfficer #HireAlice #ThankYou

Update: Since this post is getting attention again, I should probably let everyone know that I got #FediHired by a nice little AI company about two months ago 👩🏻‍🎤 thank you all for helping make that possible, and I'm sorry for blowing up your timelines with this update 🤪

Well, the company I work(ed) for just folded. Poof! 💩⛈️.
No notice, no severance, no nothing. 🤬

So I'm asking for boosts and leads. If anyone needs someone who knows data, please reach out.

I'm located in Seattle, WA. Remote preferred.

Last few titles include: CDO, Head of Data, and Sr. Operations Analyst.

I've worked in gaming, fintech, and B2C/C2C marketplaces most recently.

I'm proficient in Python, SQL, statistics, team management, (almost) all things data-related, and a host of other stuff.

I'm opinionated and anti-capitalist, but I also routinely bring in multiple times my department's cost in profits for the companies I work for.

I'm also hella nice (despite my RBF) and easy to get along with.

God, pitching myself is so awkward.

That's the toot. Thanks.

💜💜💜
Edit: I wanted to say thank you to everyone who is sharing this and a huge thank you to everyone who has reached out with encouragement and leads. This community is amazing.

Edit 2: I'm going through all the replies I got, and will reach out to a lot of you soon. I've just finished updating my resume and LinkedIn.

I realize that I'm going to have to give up some of my anonymity when I respond to people, so I'd really appreciate it if anyone I send my resume or LinkedIn profile to could keep my work and personal life separate. Thank you.
💜💜💜

---

**Gif is of parent company explaining our off-boarding process**

#PleaseBoost #Boost #FediHire #LWF #Layoffs #LookingForJob #LookingForWork #Data #DataScience #Python #SQL #Snowflake #ELT #DataPipeline #Analytics #BusinessIntelligence #CDO #HeadOfData #PleaseHelp #Gaming #Fintech #ChiefDataOfficer #HireAlice #ThankYou

Update: Since this post is getting attention again, I should probably let everyone know that I got #FediHired by a nice little AI company about two months ago 👩🏻‍🎤 thank you all for helping make that possible, and I'm sorry for blowing up your timelines with this update 🤪

Well, the company I work(ed) for just folded. Poof! 💩⛈️.
No notice, no severance, no nothing. 🤬

So I'm asking for boosts and leads. If anyone needs someone who knows data, please reach out.

I'm located in Seattle, WA. Remote preferred.

Last few titles include: CDO, Head of Data, and Sr. Operations Analyst.

I've worked in gaming, fintech, and B2C/C2C marketplaces most recently.

I'm proficient in Python, SQL, statistics, team management, (almost) all things data-related, and a host of other stuff.

I'm opinionated and anti-capitalist, but I also routinely bring in multiple times my department's cost in profits for the companies I work for.

I'm also hella nice (despite my RBF) and easy to get along with.

God, pitching myself is so awkward.

That's the toot. Thanks.

💜💜💜
Edit: I wanted to say thank you to everyone who is sharing this and a huge thank you to everyone who has reached out with encouragement and leads. This community is amazing.

Edit 2: I'm going through all the replies I got, and will reach out to a lot of you soon. I've just finished updating my resume and LinkedIn.

I realize that I'm going to have to give up some of my anonymity when I respond to people, so I'd really appreciate it if anyone I send my resume or LinkedIn profile to could keep my work and personal life separate. Thank you.
💜💜💜

---

**Gif is of parent company explaining our off-boarding process**

#PleaseBoost #Boost #FediHire #LWF #Layoffs #LookingForJob #LookingForWork #Data #DataScience #Python #SQL #Snowflake #ELT #DataPipeline #Analytics #BusinessIntelligence #CDO #HeadOfData #PleaseHelp #Gaming #Fintech #ChiefDataOfficer #HireAlice #ThankYou

Hevo draws in $8 million Series A for its no-code data pipeline service - Hevo founders Manish Jethani and Sourabh Agarwal
According to data pipeline startup Hevo, many small... more: http://feedproxy.google.com/~r/Techcrunch/~3/oyLs5YxILQY/ #nocodedatapipeline #fundings&exits #dataanalytics #datapipeline #enterprise #startups #hevodata #nocode #india #asia #hevo #tc