#apacheiceberg — Public Fediverse posts
Live and recent posts from across the Fediverse tagged #apacheiceberg, aggregated by home.social.
-
Next Wednesday: the engineer who built ColdFront live on the architecture.
Most #Postgres databases pay SSD prices for data nobody queries. ColdFront gives you #PostgreSQL to #ApacheIceberg using the same #SQL and the same table names with writable cold tier.
On August 19, 8 AM PST, catch @vyruss on the engineering + TLA+-verified distributed writes. Paul Rothrock will give a live demo.
Q&A at the end.
📅 https://us02web.zoom.us/webinar/register/WN_3oRFbue0QL2UI7j9DmCbFA
-
Next Wednesday: the engineer who built ColdFront live on the architecture.
Most #Postgres databases pay SSD prices for data nobody queries. ColdFront gives you #PostgreSQL to #ApacheIceberg using the same #SQL and the same table names with writable cold tier.
On August 19, 8 AM PST, catch @vyruss on the engineering + TLA+-verified distributed writes. Paul Rothrock will give a live demo.
Q&A at the end.
📅 https://us02web.zoom.us/webinar/register/WN_3oRFbue0QL2UI7j9DmCbFA
-
Next Wednesday: the engineer who built ColdFront live on the architecture.
Most #Postgres databases pay SSD prices for data nobody queries. ColdFront gives you #PostgreSQL to #ApacheIceberg using the same #SQL and the same table names with writable cold tier.
On August 19, 8 AM PST, catch @vyruss on the engineering + TLA+-verified distributed writes. Paul Rothrock will give a live demo.
Q&A at the end.
📅 https://us02web.zoom.us/webinar/register/WN_3oRFbue0QL2UI7j9DmCbFA
-
Next Wednesday: the engineer who built ColdFront live on the architecture.
Most #Postgres databases pay SSD prices for data nobody queries. ColdFront gives you #PostgreSQL to #ApacheIceberg using the same #SQL and the same table names with writable cold tier.
On August 19, 8 AM PST, catch @vyruss on the engineering + TLA+-verified distributed writes. Paul Rothrock will give a live demo.
Q&A at the end.
📅 https://us02web.zoom.us/webinar/register/WN_3oRFbue0QL2UI7j9DmCbFA
-
Next Wednesday: the engineer who built ColdFront live on the architecture.
Most #Postgres databases pay SSD prices for data nobody queries. ColdFront gives you #PostgreSQL to #ApacheIceberg using the same #SQL and the same table names with writable cold tier.
On August 19, 8 AM PST, catch @vyruss on the engineering + TLA+-verified distributed writes. Paul Rothrock will give a live demo.
Q&A at the end.
📅 https://us02web.zoom.us/webinar/register/WN_3oRFbue0QL2UI7j9DmCbFA
-
Amazon MSK Express brokers delivers Apache Kafka data directly to S3 general purpose buckets https://aws.amazon.com/about-aws/whats-new/2026/07/aws-msk-express-brokers-delivers-to-amazon-s3/ and Apache Iceberg S3 Tables https://aws.amazon.com/about-aws/whats-new/2026/07/aws-msk-streaming-tables-for-apache-iceberg/
#AWS #MSK #Kafka #S3 #ApacheIceberg -
Open source transparent #Postgres tiering to Apache Iceberg. Same table names, same SQL, OLTP + analytics + AI in one database. Cold data is still writable - one DELETE handles a GDPR request, no restore-to-hot cycle.
In-process #DuckDB (no sidecar, no daemon). Stock upstream #PostgreSQL. Up to 90% cheaper storage. @vyruss (who built it) + Paul Rothrock live demo - August 19, 8 AM PST.
🔗 https://github.com/pgEdge/ColdFront
📅 https://us02web.zoom.us/webinar/register/WN_3oRFbue0QL2UI7j9DmCbFA -
Open source transparent #Postgres tiering to Apache Iceberg. Same table names, same SQL, OLTP + analytics + AI in one database. Cold data is still writable - one DELETE handles a GDPR request, no restore-to-hot cycle.
In-process #DuckDB (no sidecar, no daemon). Stock upstream #PostgreSQL. Up to 90% cheaper storage. @vyruss (who built it) + Paul Rothrock live demo - August 19, 8 AM PST.
🔗 https://github.com/pgEdge/ColdFront
📅 https://us02web.zoom.us/webinar/register/WN_3oRFbue0QL2UI7j9DmCbFA -
Open source transparent #Postgres tiering to Apache Iceberg. Same table names, same SQL, OLTP + analytics + AI in one database. Cold data is still writable - one DELETE handles a GDPR request, no restore-to-hot cycle.
In-process #DuckDB (no sidecar, no daemon). Stock upstream #PostgreSQL. Up to 90% cheaper storage. @vyruss (who built it) + Paul Rothrock live demo - August 19, 8 AM PST.
🔗 https://github.com/pgEdge/ColdFront
📅 https://us02web.zoom.us/webinar/register/WN_3oRFbue0QL2UI7j9DmCbFA -
Open source transparent #Postgres tiering to Apache Iceberg. Same table names, same SQL, OLTP + analytics + AI in one database. Cold data is still writable - one DELETE handles a GDPR request, no restore-to-hot cycle.
In-process #DuckDB (no sidecar, no daemon). Stock upstream #PostgreSQL. Up to 90% cheaper storage. @vyruss (who built it) + Paul Rothrock live demo - August 19, 8 AM PST.
🔗 https://github.com/pgEdge/ColdFront
📅 https://us02web.zoom.us/webinar/register/WN_3oRFbue0QL2UI7j9DmCbFA -
Open source transparent #Postgres tiering to Apache Iceberg. Same table names, same SQL, OLTP + analytics + AI in one database. Cold data is still writable - one DELETE handles a GDPR request, no restore-to-hot cycle.
In-process #DuckDB (no sidecar, no daemon). Stock upstream #PostgreSQL. Up to 90% cheaper storage. @vyruss (who built it) + Paul Rothrock live demo - August 19, 8 AM PST.
🔗 https://github.com/pgEdge/ColdFront
📅 https://us02web.zoom.us/webinar/register/WN_3oRFbue0QL2UI7j9DmCbFA -
Most Postgres databases store 80-90% of their rows on expensive SSD that nobody's queried in months.
ColdFront is pgEdge's open source answer: moves cold data to Apache Iceberg on object storage (up to 90% cheaper) while keeping the same table names, SQL, and connection string. No application changes. No vendor lock-in.
Jimmy Angelakos (who built it) & Paul Rothrock demo it live - 8/19, 8 AM PST. Register: 📅 https://hubs.la/Q04rdpHQ0
-
Most Postgres databases store 80-90% of their rows on expensive SSD that nobody's queried in months.
ColdFront is pgEdge's open source answer: moves cold data to Apache Iceberg on object storage (up to 90% cheaper) while keeping the same table names, SQL, and connection string. No application changes. No vendor lock-in.
Jimmy Angelakos (who built it) & Paul Rothrock demo it live - 8/19, 8 AM PST. Register: 📅 https://hubs.la/Q04rdpHQ0
-
Most Postgres databases store 80-90% of their rows on expensive SSD that nobody's queried in months.
ColdFront is pgEdge's open source answer: moves cold data to Apache Iceberg on object storage (up to 90% cheaper) while keeping the same table names, SQL, and connection string. No application changes. No vendor lock-in.
Jimmy Angelakos (who built it) & Paul Rothrock demo it live - 8/19, 8 AM PST. Register: 📅 https://hubs.la/Q04rdpHQ0
-
Most Postgres databases store 80-90% of their rows on expensive SSD that nobody's queried in months.
ColdFront is pgEdge's open source answer: moves cold data to Apache Iceberg on object storage (up to 90% cheaper) while keeping the same table names, SQL, and connection string. No application changes. No vendor lock-in.
Jimmy Angelakos (who built it) & Paul Rothrock demo it live - 8/19, 8 AM PST. Register: 📅 https://hubs.la/Q04rdpHQ0
-
Learn about Cloudflare's Town Lake, its unified internal data platform, where billing workloads account for 53% of all platform queries.
Built alongside Skipper, an AI-powered analytics agent, Town Lake unifies access to operational, billing, security, and business data that was previously spread across fragmented systems.
Details here ➤ https://bit.ly/4bvWOGR
#Cloudflare #ApacheIceberg #Postgres #DataAnalytics #DataPlatform #DataGovernance #PlatformEngineering #InfoQ
-
Learn about Cloudflare's Town Lake, its unified internal data platform, where billing workloads account for 53% of all platform queries.
Built alongside Skipper, an AI-powered analytics agent, Town Lake unifies access to operational, billing, security, and business data that was previously spread across fragmented systems.
Details here ➤ https://bit.ly/4bvWOGR
#Cloudflare #ApacheIceberg #Postgres #DataAnalytics #DataPlatform #DataGovernance #PlatformEngineering #InfoQ
-
Learn about Cloudflare's Town Lake, its unified internal data platform, where billing workloads account for 53% of all platform queries.
Built alongside Skipper, an AI-powered analytics agent, Town Lake unifies access to operational, billing, security, and business data that was previously spread across fragmented systems.
Details here ➤ https://bit.ly/4bvWOGR
#Cloudflare #ApacheIceberg #Postgres #DataAnalytics #DataPlatform #DataGovernance #PlatformEngineering #InfoQ
-
Learn about Cloudflare's Town Lake, its unified internal data platform, where billing workloads account for 53% of all platform queries.
Built alongside Skipper, an AI-powered analytics agent, Town Lake unifies access to operational, billing, security, and business data that was previously spread across fragmented systems.
Details here ➤ https://bit.ly/4bvWOGR
#Cloudflare #ApacheIceberg #Postgres #DataAnalytics #DataPlatform #DataGovernance #PlatformEngineering #InfoQ
-
Learn about Cloudflare's Town Lake, its unified internal data platform, where billing workloads account for 53% of all platform queries.
Built alongside Skipper, an AI-powered analytics agent, Town Lake unifies access to operational, billing, security, and business data that was previously spread across fragmented systems.
Details here ➤ https://bit.ly/4bvWOGR
#Cloudflare #ApacheIceberg #Postgres #DataAnalytics #DataPlatform #DataGovernance #PlatformEngineering #InfoQ
-
In most tiering systems, cold data is read-only. A GDPR deletion on archived rows means restore-delete-rearchive: a half-day job.
ColdFront: UPDATE or DELETE archived rows with one SQL statement.
HFS Research analyst Ashish Chaturvedi, in Anirban Ghoshal's InfoWorld piece today. ColdFront appears alongside Databricks, Snowflake & EDB on the OLTP/OLAP divide. The only 100% #OpenSource option, with #Postgres as the interface. -
In most tiering systems, cold data is read-only. A GDPR deletion on archived rows means restore-delete-rearchive: a half-day job.
ColdFront: UPDATE or DELETE archived rows with one SQL statement.
HFS Research analyst Ashish Chaturvedi, in Anirban Ghoshal's InfoWorld piece today. ColdFront appears alongside Databricks, Snowflake & EDB on the OLTP/OLAP divide. The only 100% #OpenSource option, with #Postgres as the interface. -
In most tiering systems, cold data is read-only. A GDPR deletion on archived rows means restore-delete-rearchive: a half-day job.
ColdFront: UPDATE or DELETE archived rows with one SQL statement.
HFS Research analyst Ashish Chaturvedi, in Anirban Ghoshal's InfoWorld piece today. ColdFront appears alongside Databricks, Snowflake & EDB on the OLTP/OLAP divide. The only 100% #OpenSource option, with #Postgres as the interface. -
In most tiering systems, cold data is read-only. A GDPR deletion on archived rows means restore-delete-rearchive: a half-day job.
ColdFront: UPDATE or DELETE archived rows with one SQL statement.
HFS Research analyst Ashish Chaturvedi, in Anirban Ghoshal's InfoWorld piece today. ColdFront appears alongside Databricks, Snowflake & EDB on the OLTP/OLAP divide. The only 100% #OpenSource option, with #Postgres as the interface. -
Cold data is read-only - that's the assumption baked into basically every tiering solution on the market. ColdFront breaks it.
UPDATE & DELETE on archived rows work through standard SQL. A GDPR deletion on five-year-old events is a single DELETE statement. No restore cycle.
DuckDB in-process, Apache Iceberg on any S3. Stock unpatched #PostgreSQL 16/17/18. Beta now, PostgreSQL License. Led by Jimmy Angelakos.
-
Cold data is read-only - that's the assumption baked into basically every tiering solution on the market. ColdFront breaks it.
UPDATE & DELETE on archived rows work through standard SQL. A GDPR deletion on five-year-old events is a single DELETE statement. No restore cycle.
DuckDB in-process, Apache Iceberg on any S3. Stock unpatched #PostgreSQL 16/17/18. Beta now, PostgreSQL License. Led by Jimmy Angelakos.
-
Cold data is read-only - that's the assumption baked into basically every tiering solution on the market. ColdFront breaks it.
UPDATE & DELETE on archived rows work through standard SQL. A GDPR deletion on five-year-old events is a single DELETE statement. No restore cycle.
DuckDB in-process, Apache Iceberg on any S3. Stock unpatched #PostgreSQL 16/17/18. Beta now, PostgreSQL License. Led by Jimmy Angelakos.
-
Cold data is read-only - that's the assumption baked into basically every tiering solution on the market. ColdFront breaks it.
UPDATE & DELETE on archived rows work through standard SQL. A GDPR deletion on five-year-old events is a single DELETE statement. No restore cycle.
DuckDB in-process, Apache Iceberg on any S3. Stock unpatched #PostgreSQL 16/17/18. Beta now, PostgreSQL License. Led by Jimmy Angelakos.
-
pgEdge ColdFront: #PostgreSQL data tiering. Hot data in the heap, cold to Apache Iceberg on S3 - up to 90% lower storage cost.
The cold tier is writable. UPDATE & DELETE on cold rows work in standard SQL. No restore cycle, no rehydration. No app changes.
DuckDB runs in-process. No daemon, no sidecar. PostgreSQL License, beta now. Led by @vyruss.
Press release: 👉 https://www.pgedge.com/press-releases/pgedge-announces-coldfront-for-postgresql
GitHub: 🔗 https://github.com/pgEdge/coldfront
-
pgEdge ColdFront: #PostgreSQL data tiering. Hot data in the heap, cold to Apache Iceberg on S3 - up to 90% lower storage cost.
The cold tier is writable. UPDATE & DELETE on cold rows work in standard SQL. No restore cycle, no rehydration. No app changes.
DuckDB runs in-process. No daemon, no sidecar. PostgreSQL License, beta now. Led by @vyruss.
Press release: 👉 https://www.pgedge.com/press-releases/pgedge-announces-coldfront-for-postgresql
GitHub: 🔗 https://github.com/pgEdge/coldfront
-
pgEdge ColdFront: #PostgreSQL data tiering. Hot data in the heap, cold to Apache Iceberg on S3 - up to 90% lower storage cost.
The cold tier is writable. UPDATE & DELETE on cold rows work in standard SQL. No restore cycle, no rehydration. No app changes.
DuckDB runs in-process. No daemon, no sidecar. PostgreSQL License, beta now. Led by @vyruss.
Press release: 👉 https://www.pgedge.com/press-releases/pgedge-announces-coldfront-for-postgresql
GitHub: 🔗 https://github.com/pgEdge/coldfront
-
pgEdge ColdFront: #PostgreSQL data tiering. Hot data in the heap, cold to Apache Iceberg on S3 - up to 90% lower storage cost.
The cold tier is writable. UPDATE & DELETE on cold rows work in standard SQL. No restore cycle, no rehydration. No app changes.
DuckDB runs in-process. No daemon, no sidecar. PostgreSQL License, beta now. Led by @vyruss.
Press release: 👉 https://www.pgedge.com/press-releases/pgedge-announces-coldfront-for-postgresql
GitHub: 🔗 https://github.com/pgEdge/coldfront
-
pgEdge ColdFront: #PostgreSQL data tiering. Hot data in the heap, cold to Apache Iceberg on S3 - up to 90% lower storage cost.
The cold tier is writable. UPDATE & DELETE on cold rows work in standard SQL. No restore cycle, no rehydration. No app changes.
DuckDB runs in-process. No daemon, no sidecar. PostgreSQL License, beta now. Led by @vyruss.
Press release: 👉 https://www.pgedge.com/press-releases/pgedge-announces-coldfront-for-postgresql
GitHub: 🔗 https://github.com/pgEdge/coldfront
-
I'lll spare you the "I did a thing" cliché and just say I'm really proud to share what I've been building lately: #pgEdge ColdFront is live.
Figuring out how to move #PostgreSQL data to #S3 while keeping it queryable and updatable as if it were still local, without requiring any application code changes, is exactly the kind of hard problem I enjoy.
Check out the link below to see how it all works under the hood.
-
I'lll spare you the "I did a thing" cliché and just say I'm really proud to share what I've been building lately: #pgEdge ColdFront is live.
Figuring out how to move #PostgreSQL data to #S3 while keeping it queryable and updatable as if it were still local, without requiring any application code changes, is exactly the kind of hard problem I enjoy.
Check out the link below to see how it all works under the hood.
-
I'lll spare you the "I did a thing" cliché and just say I'm really proud to share what I've been building lately: #pgEdge ColdFront is live.
Figuring out how to move #PostgreSQL data to #S3 while keeping it queryable and updatable as if it were still local, without requiring any application code changes, is exactly the kind of hard problem I enjoy.
Check out the link below to see how it all works under the hood.
-
I'lll spare you the "I did a thing" cliché and just say I'm really proud to share what I've been building lately: #pgEdge ColdFront is live.
Figuring out how to move #PostgreSQL data to #S3 while keeping it queryable and updatable as if it were still local, without requiring any application code changes, is exactly the kind of hard problem I enjoy.
Check out the link below to see how it all works under the hood.
-
I'lll spare you the "I did a thing" cliché and just say I'm really proud to share what I've been building lately: #pgEdge ColdFront is live.
Figuring out how to move #PostgreSQL data to #S3 while keeping it queryable and updatable as if it were still local, without requiring any application code changes, is exactly the kind of hard problem I enjoy.
Check out the link below to see how it all works under the hood.
-
🧊 pgEdge ColdFront beta is out - transparent data tiering for #PostgreSQL. Fully writable cold tier.
Hot data stays in the heap. Cold data moves to Iceberg on S3 at up to 90% lower cost. UPDATE & DELETE on archived rows, same SQL. No rehydration. No code changes.
DuckDB runs in-process - no daemon, no RPC. C extension routes DML to the correct tier transparently.
Development by Jimmy Angelakos. Blog by Antony Pegg: 📖 https://www.pgedge.com/blog/introducing-coldfront-seamlessly-uniting-oltp-analytics-and-ai-workloads-on-postgresql
-
🧊 pgEdge ColdFront beta is out - transparent data tiering for #PostgreSQL. Fully writable cold tier.
Hot data stays in the heap. Cold data moves to Iceberg on S3 at up to 90% lower cost. UPDATE & DELETE on archived rows, same SQL. No rehydration. No code changes.
DuckDB runs in-process - no daemon, no RPC. C extension routes DML to the correct tier transparently.
Development by Jimmy Angelakos. Blog by Antony Pegg: 📖 https://www.pgedge.com/blog/introducing-coldfront-seamlessly-uniting-oltp-analytics-and-ai-workloads-on-postgresql
-
🧊 pgEdge ColdFront beta is out - transparent data tiering for #PostgreSQL. Fully writable cold tier.
Hot data stays in the heap. Cold data moves to Iceberg on S3 at up to 90% lower cost. UPDATE & DELETE on archived rows, same SQL. No rehydration. No code changes.
DuckDB runs in-process - no daemon, no RPC. C extension routes DML to the correct tier transparently.
Development by Jimmy Angelakos. Blog by Antony Pegg: 📖 https://www.pgedge.com/blog/introducing-coldfront-seamlessly-uniting-oltp-analytics-and-ai-workloads-on-postgresql
-
🧊 pgEdge ColdFront beta is out - transparent data tiering for #PostgreSQL. Fully writable cold tier.
Hot data stays in the heap. Cold data moves to Iceberg on S3 at up to 90% lower cost. UPDATE & DELETE on archived rows, same SQL. No rehydration. No code changes.
DuckDB runs in-process - no daemon, no RPC. C extension routes DML to the correct tier transparently.
Development by Jimmy Angelakos. Blog by Antony Pegg: 📖 https://www.pgedge.com/blog/introducing-coldfront-seamlessly-uniting-oltp-analytics-and-ai-workloads-on-postgresql
-
https://www.europesays.com/ie/498845/ Google Cloud Introduces Cross-Engine Iceberg Support in BigQuery #AI #ApacheIceberg #Architecture&Design #Cloud #DataCatalog #DataLake #DataPortability #Éire #GoogleBigQuery #GoogleCloud #GoogleCrossEngineIceberg #IE #Ireland #ML&DataEngineering #Technology
-
DuckDB Labs released #DuckLake 1.0 - a data lake format that stores table metadata in a SQL database, rather than spreading it across object storage files.
Key features:
• catalog-stored small updates
• improved sorting and partitioning
• compatibility with Iceberg-style data featuresLearn more ⇨ https://bit.ly/48PsPIS
-
DuckDB Labs released #DuckLake 1.0 - a data lake format that stores table metadata in a SQL database, rather than spreading it across object storage files.
Key features:
• catalog-stored small updates
• improved sorting and partitioning
• compatibility with Iceberg-style data featuresLearn more ⇨ https://bit.ly/48PsPIS
-
DuckDB Labs released #DuckLake 1.0 - a data lake format that stores table metadata in a SQL database, rather than spreading it across object storage files.
Key features:
• catalog-stored small updates
• improved sorting and partitioning
• compatibility with Iceberg-style data featuresLearn more ⇨ https://bit.ly/48PsPIS
-
DuckDB Labs released #DuckLake 1.0 - a data lake format that stores table metadata in a SQL database, rather than spreading it across object storage files.
Key features:
• catalog-stored small updates
• improved sorting and partitioning
• compatibility with Iceberg-style data featuresLearn more ⇨ https://bit.ly/48PsPIS
-
DuckDB Labs released #DuckLake 1.0 - a data lake format that stores table metadata in a SQL database, rather than spreading it across object storage files.
Key features:
• catalog-stored small updates
• improved sorting and partitioning
• compatibility with Iceberg-style data featuresLearn more ⇨ https://bit.ly/48PsPIS
-
Lakehouse architectures allow multiple engines to run on shared data through open table formats like #ApacheIceberg.
But #SQL identifier resolution and catalog naming rules differ across engines - creating hidden interoperability failures.
In this #InfoQ article, Maninder Parmar explains why enforcing consistent naming conventions and cross-engine validation is critical.
📰 Read now: https://bit.ly/4902zeH
-
Lakehouse architectures allow multiple engines to run on shared data through open table formats like #ApacheIceberg.
But #SQL identifier resolution and catalog naming rules differ across engines - creating hidden interoperability failures.
In this #InfoQ article, Maninder Parmar explains why enforcing consistent naming conventions and cross-engine validation is critical.
📰 Read now: https://bit.ly/4902zeH
-
Lakehouse architectures allow multiple engines to run on shared data through open table formats like #ApacheIceberg.
But #SQL identifier resolution and catalog naming rules differ across engines - creating hidden interoperability failures.
In this #InfoQ article, Maninder Parmar explains why enforcing consistent naming conventions and cross-engine validation is critical.
📰 Read now: https://bit.ly/4902zeH
-
Lakehouse architectures allow multiple engines to run on shared data through open table formats like #ApacheIceberg.
But #SQL identifier resolution and catalog naming rules differ across engines - creating hidden interoperability failures.
In this #InfoQ article, Maninder Parmar explains why enforcing consistent naming conventions and cross-engine validation is critical.
📰 Read now: https://bit.ly/4902zeH
-
Lakehouse architectures allow multiple engines to run on shared data through open table formats like #ApacheIceberg.
But #SQL identifier resolution and catalog naming rules differ across engines - creating hidden interoperability failures.
In this #InfoQ article, Maninder Parmar explains why enforcing consistent naming conventions and cross-engine validation is critical.
📰 Read now: https://bit.ly/4902zeH
-
The Data Lakehouse Explained: Why Apache Iceberg Is Quietly Running the Show
https://techlife.blog/posts/data-lakehouse-iceberg
#ApacheIceberg #DataLakehouse #DataWarehouse #DataLake #Snowflake #ApacheSpark #DataEngineering
-
The Data Lakehouse Explained: Why Apache Iceberg Is Quietly Running the Show
https://techlife.blog/posts/data-lakehouse-iceberg
#ApacheIceberg #DataLakehouse #DataWarehouse #DataLake #Snowflake #ApacheSpark #DataEngineering
-
#Pinterest launched a next-gen CDC-based ingestion framework.
Using #ApacheKafka, #ApacheFlink, #ApacheSpark & #ApacheIceberg, they achieved:
• Latency cut from 24+ hours to 15 minutes
• Processing of only changed records
• Support for incremental updates & deletions
• Petabyte-scale data across 1,000+ pipelinesWin: optimized cost & efficiency!
Read the architectural deep dive on InfoQ 👉 https://bit.ly/4rMJB2H
-
#Pinterest launched a next-gen CDC-based ingestion framework.
Using #ApacheKafka, #ApacheFlink, #ApacheSpark & #ApacheIceberg, they achieved:
• Latency cut from 24+ hours to 15 minutes
• Processing of only changed records
• Support for incremental updates & deletions
• Petabyte-scale data across 1,000+ pipelinesWin: optimized cost & efficiency!
Read the architectural deep dive on InfoQ 👉 https://bit.ly/4rMJB2H
-
#Pinterest launched a next-gen CDC-based ingestion framework.
Using #ApacheKafka, #ApacheFlink, #ApacheSpark & #ApacheIceberg, they achieved:
• Latency cut from 24+ hours to 15 minutes
• Processing of only changed records
• Support for incremental updates & deletions
• Petabyte-scale data across 1,000+ pipelinesWin: optimized cost & efficiency!
Read the architectural deep dive on InfoQ 👉 https://bit.ly/4rMJB2H
-
#Pinterest launched a next-gen CDC-based ingestion framework.
Using #ApacheKafka, #ApacheFlink, #ApacheSpark & #ApacheIceberg, they achieved:
• Latency cut from 24+ hours to 15 minutes
• Processing of only changed records
• Support for incremental updates & deletions
• Petabyte-scale data across 1,000+ pipelinesWin: optimized cost & efficiency!
Read the architectural deep dive on InfoQ 👉 https://bit.ly/4rMJB2H
-
#Pinterest launched a next-gen CDC-based ingestion framework.
Using #ApacheKafka, #ApacheFlink, #ApacheSpark & #ApacheIceberg, they achieved:
• Latency cut from 24+ hours to 15 minutes
• Processing of only changed records
• Support for incremental updates & deletions
• Petabyte-scale data across 1,000+ pipelinesWin: optimized cost & efficiency!
Read the architectural deep dive on InfoQ 👉 https://bit.ly/4rMJB2H