#deltalake — Public Fediverse posts

Azure costs rising? It may not be compute. Learn how fixing small files, snapshots, and streaming writes cut data costs by 40%. https://hackernoon.com/how-i-cut-our-cloud-bill-by-40percent-without-touching-a-single-data-model #deltalake

Azure costs rising? It may not be compute. Learn how fixing small files, snapshots, and streaming writes cut data costs by 40%. https://hackernoon.com/how-i-cut-our-cloud-bill-by-40percent-without-touching-a-single-data-model #deltalake

The next #PyData #Helsinki #meetup is on 19 March 2026 at Elisa, Ratavartijankatu 5.

On the menu: optimisation problems in electricity markets, a deep dive into #DeltaLake read performance without a cluster, two lightning talks, and an intro to Data & AI at our sponsor Elisa.

And of course our already traditional quiz, where the main challenge is understanding how answers are scored.

https://www.meetup.com/pydatahelsinki/events/313128987/?isFirstPublish=true

#DataScience #DataEngineering #Python

The next #PyData #Helsinki #meetup is on 19 March 2026 at Elisa, Ratavartijankatu 5.

On the menu: optimisation problems in electricity markets, a deep dive into #DeltaLake read performance without a cluster, two lightning talks, and an intro to Data & AI at our sponsor Elisa.

And of course our already traditional quiz, where the main challenge is understanding how answers are scored.

https://www.meetup.com/pydatahelsinki/events/313128987/?isFirstPublish=true

#DataScience #DataEngineering #Python

My most frequently asked question in 2025 was: how do you design for
high-throughput ingestion with #deltalake

Over the last week I took the time to write down exactly how I design for throughput in this blog post: https://www.buoyantdata.com/blog/2026-01-02-design-for-throughput.html

My most frequently asked question in 2025 was: how do you design for
high-throughput ingestion with #deltalake

Over the last week I took the time to write down exactly how I design for throughput in this blog post: https://www.buoyantdata.com/blog/2026-01-02-design-for-throughput.html

Spark SQL for Data Engineering 1 : I am going to start spark sql sessions as series. #sparksql

Spark SQL Part 1 : I am going to start spark sql sessions as series. #sparksql #deltalake #pyspark ' Databricks Notebooks code for ... source

https://quadexcel.com/wp/spark-sql-for-data-engineering-1-i-am-going-to-start-spark-sql-sessions-as-series-sparksql/

Spark SQL for Data Engineering 1 : I am going to start spark sql sessions as series. #sparksql

Spark SQL Part 1 : I am going to start spark sql sessions as series. #sparksql #deltalake #pyspark ' Databricks Notebooks code for ... source

https://quadexcel.com/wp/spark-sql-for-data-engineering-1-i-am-going-to-start-spark-sql-sessions-as-series-sparksql/

Quarkusで作る小売向け統計分析API
https://qiita.com/hiraishi_satoshi/items/416d876a9213303e68cb?utm_campaign=popular_items&utm_medium=feed&utm_source=popular_items

#qiita #Kotlin #Quarkus #deltalake

^ Less then a day to sign up and get our take on #SAP Business Data Cloud and the Delta Sharing ecosystem

#databricks #businessdatacloud #deltalake #deltasharing #dremio #webinar

^ Less then a day to sign up and get our take on #SAP Business Data Cloud and the Delta Sharing ecosystem

#databricks #businessdatacloud #deltalake #deltasharing #dremio #webinar

kafka-delta-ingest was the project that spawned the development of #deltalake for #rustlang, also known as delta-rs.

Last week I decommissioned the last of those processes. I have since made our
ingestion even cheaper but kafka-delta-ingest will always hold a spot in our history

https://brokenco.de/2025/10/30/kafka-delta-ingest-was-fun.html

Tomorrow at 7am PT (14:00 UTC) I'll be doing some #deltalake hacking with some other 🦀 folks on #deltalive!

https://www.twitch.tv/agentdero/schedule

📣I wrote a blog 🚀📰:

The Data Surrender Trap: How Enterprises Are Losing Control in the AI Gold Rush—and the Simple Fix 👇

https://www.softinio.com/post/the-data-surrender-trap/

#AI #Deltalake #DataEngineering #DataGovernance #ArtificialIntelligence

@Schneems I don't know about best practices, but once upon a while ago I wrote why we re-export some symbols in #deltalake https://brokenco.de/2023/07/26/rust-re-export.html

Did you know you can purchase #deltalake the definitive guide from reputable non-monopolistic book sellers?

https://bookshop.org/p/books/delta-lake-the-definitive-guide-modern-data-lakehouse-architectures-with-data-lakes-denny-lee/21429337

You can also pester some of the other authors and I next week in Mountain View, CA: https://lu.ma/okxq0bt1

#ITByte: #DeltaLake is an open-source #Storage framework that enables building a #Lakehouse #Architecture with different compute engines.

Delta Lake preserves the integrity of the original data without sacrificing the performance and agility required for real-time analytics, artificial intelligence (AI), and machine learning (ML) applications.

https://knowledgezone.co.in/trends/explorer?topic=Delta-Lake

#ITByte: #DeltaLake is an open-source #Storage framework that enables building a #Lakehouse #Architecture with different compute engines.

Delta Lake preserves the integrity of the original data without sacrificing the performance and agility required for real-time analytics, artificial intelligence (AI), and machine learning (ML) applications.

https://knowledgezone.co.in/trends/explorer?topic=Delta-Lake

Just caught up with the recent Delta Lake webinar,

> Revolutionizing Delta Lake workflows on AWS Lambda with Polars, DuckDB, Daft & Rust

Some interesting hints there regarding lightweight processing of big-ish data. Easy to relate to any other framework instead of Lambda, e.g. #ApacheAirflow tasks

https://youtu.be/BR9oFD0QMAs

#dataengineering #datascience #duckdb #daft #polars #pandas #python #spark #deltalake #databricks #airflow #bigdata #smalldata

Just caught up with the recent Delta Lake webinar,

> Revolutionizing Delta Lake workflows on AWS Lambda with Polars, DuckDB, Daft & Rust

Some interesting hints there regarding lightweight processing of big-ish data. Easy to relate to any other framework instead of Lambda, e.g. #ApacheAirflow tasks

https://youtu.be/BR9oFD0QMAs

#dataengineering #datascience #duckdb #daft #polars #pandas #python #spark #deltalake #databricks #airflow #bigdata #smalldata

Revolutionizing Data Management: Mooncake-Labs Unveils pg_mooncake v0.1.0

In a significant leap for data processing, Mooncake-Labs has launched pg_mooncake v0.1.0, introducing robust features that enhance PostgreSQL's capabilities. This release empowers users to seamlessly ...

https://news.lavx.hu/article/revolutionizing-data-management-mooncake-labs-unveils-pg-mooncake-v0-1-0

#news #tech #PostgreSQL #DataManagement #DeltaLake

I hope you'll be able to join the #deltalake AMA today at 9am PST with the authors of Delta Lake The Definitive Guide 🤗

https://www.linkedin.com/events/asktheauthors7257759127987261441/

I am so so excited to have a physical copy of #deltalake "The Definitive Guide"

I had some talks earlier this year about #deltalake and #rustlang

I also did so with one hand...because I broke my wrist at the beginning of the conference 🤦

https://brokenco.de/2024/10/17/data-ai-summit-videos.html

If you use #deltalake from Python or Rust, I sure would appreciate it if your company sponsored my work!

https://github.com/sponsors/rtyler

If you use #deltalake for #python, you can now sponsor one of it's main contributors Ion!

https://github.com/ion-elgreco

The #deltalake Python and Rust bindings have change data capture (CDC) read and write support in the works right now.

*Very* exciting stuff happening between now and Data and AI Summit in June!

Finally hitting a groove with this #deltalake code at ... almost 23:00 local time 🤢

#ITByte: #DeltaLake is an open-source #Storage framework that enables building a #Lakehouse #Architecture with different compute engines.

Delta Lake preserves the integrity of the original data without sacrificing the performance and agility required for real-time analytics, artificial intelligence (AI), and machine learning (ML) applications.

https://knowledgezone.co.in/trends/explorer?topic=Delta-Lake

#ITByte: #DeltaLake is an open-source #Storage framework that enables building a #Lakehouse #Architecture with different compute engines.

Delta Lake preserves the integrity of the original data without sacrificing the performance and agility required for real-time analytics, artificial intelligence (AI), and machine learning (ML) applications.

https://knowledgezone.co.in/trends/explorer?topic=Delta-Lake

I am thrilled that I am able to hire another full time Senior Data Engineer in North America for my organization at Scribd.

At a high level my team:

* created the #deltalake Rust bindings, kafka-delta-ingest, and a number of other Rust based data tools.
* has presented multiple times at Data and AI Summit
* works at the leading edge of the Databricks platform
* is remote-first, spanning 3-5 continents depending on the time of year 😆 (we write a lot of stuff down!)

https://jobs.lever.co/scribd/80fcabda-99a9-4b79-b125-e1687baee19d