#apacheiceberg — Public Fediverse posts
Live and recent posts from across the Fediverse tagged #apacheiceberg, aggregated by home.social.
-
https://www.europesays.com/ie/498845/ Google Cloud Introduces Cross-Engine Iceberg Support in BigQuery #AI #ApacheIceberg #Architecture&Design #Cloud #DataCatalog #DataLake #DataPortability #Éire #GoogleBigQuery #GoogleCloud #GoogleCrossEngineIceberg #IE #Ireland #ML&DataEngineering #Technology
-
DuckDB Labs released #DuckLake 1.0 - a data lake format that stores table metadata in a SQL database, rather than spreading it across object storage files.
Key features:
• catalog-stored small updates
• improved sorting and partitioning
• compatibility with Iceberg-style data featuresLearn more ⇨ https://bit.ly/48PsPIS
-
DuckDB Labs released #DuckLake 1.0 - a data lake format that stores table metadata in a SQL database, rather than spreading it across object storage files.
Key features:
• catalog-stored small updates
• improved sorting and partitioning
• compatibility with Iceberg-style data featuresLearn more ⇨ https://bit.ly/48PsPIS
-
DuckDB Labs released #DuckLake 1.0 - a data lake format that stores table metadata in a SQL database, rather than spreading it across object storage files.
Key features:
• catalog-stored small updates
• improved sorting and partitioning
• compatibility with Iceberg-style data featuresLearn more ⇨ https://bit.ly/48PsPIS
-
DuckDB Labs released #DuckLake 1.0 - a data lake format that stores table metadata in a SQL database, rather than spreading it across object storage files.
Key features:
• catalog-stored small updates
• improved sorting and partitioning
• compatibility with Iceberg-style data featuresLearn more ⇨ https://bit.ly/48PsPIS
-
DuckDB Labs released #DuckLake 1.0 - a data lake format that stores table metadata in a SQL database, rather than spreading it across object storage files.
Key features:
• catalog-stored small updates
• improved sorting and partitioning
• compatibility with Iceberg-style data featuresLearn more ⇨ https://bit.ly/48PsPIS
-
Lakehouse architectures allow multiple engines to run on shared data through open table formats like #ApacheIceberg.
But #SQL identifier resolution and catalog naming rules differ across engines - creating hidden interoperability failures.
In this #InfoQ article, Maninder Parmar explains why enforcing consistent naming conventions and cross-engine validation is critical.
📰 Read now: https://bit.ly/4902zeH
-
Lakehouse architectures allow multiple engines to run on shared data through open table formats like #ApacheIceberg.
But #SQL identifier resolution and catalog naming rules differ across engines - creating hidden interoperability failures.
In this #InfoQ article, Maninder Parmar explains why enforcing consistent naming conventions and cross-engine validation is critical.
📰 Read now: https://bit.ly/4902zeH
-
Lakehouse architectures allow multiple engines to run on shared data through open table formats like #ApacheIceberg.
But #SQL identifier resolution and catalog naming rules differ across engines - creating hidden interoperability failures.
In this #InfoQ article, Maninder Parmar explains why enforcing consistent naming conventions and cross-engine validation is critical.
📰 Read now: https://bit.ly/4902zeH
-
Lakehouse architectures allow multiple engines to run on shared data through open table formats like #ApacheIceberg.
But #SQL identifier resolution and catalog naming rules differ across engines - creating hidden interoperability failures.
In this #InfoQ article, Maninder Parmar explains why enforcing consistent naming conventions and cross-engine validation is critical.
📰 Read now: https://bit.ly/4902zeH
-
Lakehouse architectures allow multiple engines to run on shared data through open table formats like #ApacheIceberg.
But #SQL identifier resolution and catalog naming rules differ across engines - creating hidden interoperability failures.
In this #InfoQ article, Maninder Parmar explains why enforcing consistent naming conventions and cross-engine validation is critical.
📰 Read now: https://bit.ly/4902zeH
-
The Data Lakehouse Explained: Why Apache Iceberg Is Quietly Running the Show
https://techlife.blog/posts/data-lakehouse-iceberg
#ApacheIceberg #DataLakehouse #DataWarehouse #DataLake #Snowflake #ApacheSpark #DataEngineering
-
The Data Lakehouse Explained: Why Apache Iceberg Is Quietly Running the Show
https://techlife.blog/posts/data-lakehouse-iceberg
#ApacheIceberg #DataLakehouse #DataWarehouse #DataLake #Snowflake #ApacheSpark #DataEngineering
-
#Pinterest launched a next-gen CDC-based ingestion framework.
Using #ApacheKafka, #ApacheFlink, #ApacheSpark & #ApacheIceberg, they achieved:
• Latency cut from 24+ hours to 15 minutes
• Processing of only changed records
• Support for incremental updates & deletions
• Petabyte-scale data across 1,000+ pipelinesWin: optimized cost & efficiency!
Read the architectural deep dive on InfoQ 👉 https://bit.ly/4rMJB2H
-
#Pinterest launched a next-gen CDC-based ingestion framework.
Using #ApacheKafka, #ApacheFlink, #ApacheSpark & #ApacheIceberg, they achieved:
• Latency cut from 24+ hours to 15 minutes
• Processing of only changed records
• Support for incremental updates & deletions
• Petabyte-scale data across 1,000+ pipelinesWin: optimized cost & efficiency!
Read the architectural deep dive on InfoQ 👉 https://bit.ly/4rMJB2H
-
#Pinterest launched a next-gen CDC-based ingestion framework.
Using #ApacheKafka, #ApacheFlink, #ApacheSpark & #ApacheIceberg, they achieved:
• Latency cut from 24+ hours to 15 minutes
• Processing of only changed records
• Support for incremental updates & deletions
• Petabyte-scale data across 1,000+ pipelinesWin: optimized cost & efficiency!
Read the architectural deep dive on InfoQ 👉 https://bit.ly/4rMJB2H
-
#Pinterest launched a next-gen CDC-based ingestion framework.
Using #ApacheKafka, #ApacheFlink, #ApacheSpark & #ApacheIceberg, they achieved:
• Latency cut from 24+ hours to 15 minutes
• Processing of only changed records
• Support for incremental updates & deletions
• Petabyte-scale data across 1,000+ pipelinesWin: optimized cost & efficiency!
Read the architectural deep dive on InfoQ 👉 https://bit.ly/4rMJB2H
-
#Pinterest launched a next-gen CDC-based ingestion framework.
Using #ApacheKafka, #ApacheFlink, #ApacheSpark & #ApacheIceberg, they achieved:
• Latency cut from 24+ hours to 15 minutes
• Processing of only changed records
• Support for incremental updates & deletions
• Petabyte-scale data across 1,000+ pipelinesWin: optimized cost & efficiency!
Read the architectural deep dive on InfoQ 👉 https://bit.ly/4rMJB2H
-
#AWS announced 2 new capabilities for #S3Tables!
🔹 Intelligent-Tiering storage class that automatically optimizes costs based on access patterns
🔹 Replication support that keeps Apache Iceberg table replicas consistent across AWS regions and accounts - no manual syncing requiredFind out more: https://bit.ly/4qgRn3Y
-
#AWS announced 2 new capabilities for #S3Tables!
🔹 Intelligent-Tiering storage class that automatically optimizes costs based on access patterns
🔹 Replication support that keeps Apache Iceberg table replicas consistent across AWS regions and accounts - no manual syncing requiredFind out more: https://bit.ly/4qgRn3Y
-
#AWS announced 2 new capabilities for #S3Tables!
🔹 Intelligent-Tiering storage class that automatically optimizes costs based on access patterns
🔹 Replication support that keeps Apache Iceberg table replicas consistent across AWS regions and accounts - no manual syncing requiredFind out more: https://bit.ly/4qgRn3Y
-
#AWS announced 2 new capabilities for #S3Tables!
🔹 Intelligent-Tiering storage class that automatically optimizes costs based on access patterns
🔹 Replication support that keeps Apache Iceberg table replicas consistent across AWS regions and accounts - no manual syncing requiredFind out more: https://bit.ly/4qgRn3Y
-
#AWS announced 2 new capabilities for #S3Tables!
🔹 Intelligent-Tiering storage class that automatically optimizes costs based on access patterns
🔹 Replication support that keeps Apache Iceberg table replicas consistent across AWS regions and accounts - no manual syncing requiredFind out more: https://bit.ly/4qgRn3Y
-
#DuckDB now supports end-to-end interaction with Iceberg REST Catalogs directly in the browser - no infrastructure setup required.
With DuckDB-Wasm, users can query, read, and write Iceberg tables seamlessly.
Learn more: https://bit.ly/4qCTYoF
-
Cloudflare has just launched the open beta of its Cloudflare Data Platform - a managed service for ingesting, storing & querying analytical data tables using open standards like Apache Iceberg.
🔍 Dive into the key insights on #InfoQ ⇨ https://bit.ly/49y1tIa
#CloudComputing #DataLake #DataAnalytics #ApacheIceberg #Cloudflare
-
Cloudflare has just launched the open beta of its Cloudflare Data Platform - a managed service for ingesting, storing & querying analytical data tables using open standards like Apache Iceberg.
🔍 Dive into the key insights on #InfoQ ⇨ https://bit.ly/49y1tIa
#CloudComputing #DataLake #DataAnalytics #ApacheIceberg #Cloudflare
-
Cloudflare has just launched the open beta of its Cloudflare Data Platform - a managed service for ingesting, storing & querying analytical data tables using open standards like Apache Iceberg.
🔍 Dive into the key insights on #InfoQ ⇨ https://bit.ly/49y1tIa
#CloudComputing #DataLake #DataAnalytics #ApacheIceberg #Cloudflare
-
scrapy-contrib-bigexporter 0.6.1 released: https://codeberg.org/ZuInnoTe/scrapy-contrib-bigexporters
Added: You can customize Iceberg table location
#scrapy #webscraping #bigdata #iceberg #apacheiceberg #opensource #python
-
scrapy-contrib-bigexporter 0.6.0 released: https://codeberg.org/ZuInnoTe/scrapy-contrib-bigexporters
New: Export your webscraped items in Scrapy to Apache Iceberg tables with simple configuration
#scrapy #webscraping #bigdata #iceberg #apacheiceberg #opensource #python
-
#Netflix scaled 𝐌𝐮𝐬𝐞 to handle 𝐭𝐫𝐢𝐥𝐥𝐢𝐨𝐧-𝐫𝐨𝐰 𝐝𝐚𝐭𝐚𝐬𝐞𝐭𝐬!
➡️ Muse helps teams see which artwork & videos resonate with audiences.
➡️ To keep up with demand, Netflix 𝐫𝐞𝐝𝐞𝐬𝐢𝐠𝐧𝐞𝐝 𝐭𝐡𝐞 𝐝𝐚𝐭𝐚 𝐥𝐚𝐲𝐞𝐫, cutting query latencies by ~50% while keeping results accurate and responsive.🔗 Learn more: https://bit.ly/4gG3HGU
-
Watching the re-indexing of an archival catalog backup of AtoM, I realized:
Indices populated with 18751 documents in 164.84 seconds.
19k Objects?
Thats /nothing/ for a regular #bigDATA tech-tool. This is peanuts.400.000 Objects?
Millions?! - According to documentation of #ApacheIceberg #ObjectStore #Redis #KeyDB, etc: **easy**#DLTP & #GLAM: Storing and using those "objects" in key/value annotated filesystems with bigDATA tools:
**FUN!!**
-
Watching the re-indexing of an archival catalog backup of AtoM, I realized:
Indices populated with 18751 documents in 164.84 seconds.
19k Objects?
Thats /nothing/ for a regular #bigDATA tech-tool. This is peanuts.400.000 Objects?
Millions?! - According to documentation of #ApacheIceberg #ObjectStore #Redis #KeyDB, etc: **easy**#DLTP & #GLAM: Storing and using those "objects" in key/value annotated filesystems with bigDATA tools:
**FUN!!**
-
Watching the re-indexing of an archival catalog backup of AtoM, I realized:
Indices populated with 18751 documents in 164.84 seconds.
19k Objects?
Thats /nothing/ for a regular #bigDATA tech-tool. This is peanuts.400.000 Objects?
Millions?! - According to documentation of #ApacheIceberg #ObjectStore #Redis #KeyDB, etc: **easy**#DLTP & #GLAM: Storing and using those "objects" in key/value annotated filesystems with bigDATA tools:
**FUN!!**
-
Amazon #S3 now supports sort and z-order compaction for #ApacheIceberg tables, promising reduced scan times & lower engine costs.
Available for both S3 Tables and traditional S3 buckets via AWS Glue Data Catalog optimization.
Dive into the details: https://bit.ly/3GyjxWQ
-
📢 Behold, the earth-shattering breakthrough of Nimtable: a web UI to *click* on Apache Iceberg tables! 🙄 Presumably because using command line tools is an insurmountable task for mere mortals. Or maybe it’s just a clever way to make clicking around a web interface the new rocket science. 🚀
https://github.com/nimtable/nimtable #Nimtable #ApacheIceberg #WebUI #Innovation #TechNews #ClickAndGo #HackerNews #ngated -
Nimtable: Open-source web UI to browse and manage Apache Iceberg tables
https://github.com/nimtable/nimtable
#HackerNews #Nimtable #OpenSource #ApacheIceberg #WebUI #DataManagement #DatabaseTools
-
Paris: Apache Iceberg Paris Community Meetup #1, Le jeudi 19 juin 2025 de 18h00 à 21h30. https://www.agendadulibre.org/events/32653 #data #dataLakehouse #dataEngineer #dataScience #dataPlatform #dataWarehouse #apacheIceberg
-
"Centralize Your Data Lake: Apache Polaris Supports Apache Iceberg and Now Delta Lake"
BTW 'Polaris' used to be the name of the UK nuclear deterrent pre 1996. 😬
https://snowflake.com/en/engineering-blog/apache-polaris-supports-iceberg-delta-lake/
-
"Centralize Your Data Lake: Apache Polaris Supports Apache Iceberg and Now Delta Lake"
BTW 'Polaris' used to be the name of the UK nuclear deterrent pre 1996. 😬
https://snowflake.com/en/engineering-blog/apache-polaris-supports-iceberg-delta-lake/
-
"Centralize Your Data Lake: Apache Polaris Supports Apache Iceberg and Now Delta Lake"
BTW 'Polaris' used to be the name of the UK nuclear deterrent pre 1996. 😬
https://snowflake.com/en/engineering-blog/apache-polaris-supports-iceberg-delta-lake/
-
"Centralize Your Data Lake: Apache Polaris Supports Apache Iceberg and Now Delta Lake"
BTW 'Polaris' used to be the name of the UK nuclear deterrent pre 1996. 😬
https://snowflake.com/en/engineering-blog/apache-polaris-supports-iceberg-delta-lake/
-
"Centralize Your Data Lake: Apache Polaris Supports Apache Iceberg and Now Delta Lake"
BTW 'Polaris' used to be the name of the UK nuclear deterrent pre 1996. 😬
https://snowflake.com/en/engineering-blog/apache-polaris-supports-iceberg-delta-lake/
-
What happens when you marry #ClickHouse database with #ApacheIceberg? you could query huge datasets fast and with 10x cheaper storage. Sounds promising, right?
Join me tomorrow on the live stream to find out!
May 20th, 11am PT / 20:00 CET:
https://www.youtube.com/watch?v=VeyTL2JlWp0 -
#ApacheIceberg: What It Is and Why Everyone’s Talking About It
-
Benefits of Apache Iceberg for geospatial data analysis
https://wherobots.com/blog/benefits-of-apache-iceberg-for-geospatial-data-analysis/
#HackerNews #ApacheIceberg #GeospatialData #DataAnalysis #BigData #Analytics
-
R2 Data Catalog: Managed Apache Iceberg tables with zero egress fees - Cloudflare
The Iceberg wars are hotting up. AWS has some competition.
-
Streamlining access to tabular datasets stored in Amazon S3 Tables with DuckDB | AWS Storage Blog
-
Streamlining access to tabular datasets stored in Amazon S3 Tables with DuckDB | AWS Storage Blog
-
Streamlining access to tabular datasets stored in Amazon S3 Tables with DuckDB | AWS Storage Blog
-
Streamlining access to tabular datasets stored in Amazon S3 Tables with DuckDB | AWS Storage Blog