#apachearrow — Public Fediverse posts

The main thing AI coding is giving me, is the bandwidth to actually vendor libraries and to shape them how to use I want, or to simply rebuild them at will.

#ApacheArrow is great, but in C++ it's 250gb + to build from on windows and that's just obscene...

The main thing AI coding is giving me, is the bandwidth to actually vendor libraries and to shape them how to use I want, or to simply rebuild them at will.

#ApacheArrow is great, but in C++ it's 250gb + to build from on windows and that's just obscene...

RE: https://mastodon.social/@sovtechfund/116731663840822816

Really honoured and excited to get support from @sovtechfund the coming year to work on @geopandas and its stack (#Shapely, @pandas_dev, pyogrio, #ApacheArrow, etc)! Looking forward to have some more time for those projects. And to getting to know the other fellows!

RE: https://mastodon.social/@sovtechfund/116731663840822816

Really honoured and excited to get support from @sovtechfund the coming year to work on @geopandas and its stack (#Shapely, @pandas_dev, pyogrio, #ApacheArrow, etc)! Looking forward to have some more time for those projects. And to getting to know the other fellows!

mssql-python 1.6: Unblocking Your Threads
#python #bulkcopy #ApacheArrow
https://techcommunity.microsoft.com/blog/sqlserver/mssql-python-1-5-apache-arrow-sql-variant-and-native-uuids/4510363?wt.mc_id=DP-MVP-4015656

#TIL about a #graph #knowledge engine that supports versioning #git style: #Omnigraph

https://github.com/ModernRelay/omnigraph

#OpenSource #Rustlang #ApacheArrow

#TIL about a #graph #knowledge engine that supports versioning #git style: #Omnigraph

https://github.com/ModernRelay/omnigraph

#OpenSource #Rustlang #ApacheArrow

R Consortium webinar: Scale up data analysis in R with Arrow—fast, memory-efficient analytics without a DB or cluster.

With Dr Nic Crane (Arrow R maintainer, Apache Arrow PMC).

Register: https://r-consortium.org/webinars/scaling-up-data-analysis-in-r-with-arrow.html

#rstats #apachearrow

R Consortium webinar: Scale up data analysis in R with Arrow—fast, memory-efficient analytics without a DB or cluster.

With Dr Nic Crane (Arrow R maintainer, Apache Arrow PMC).

Register: https://r-consortium.org/webinars/scaling-up-data-analysis-in-r-with-arrow.html

#rstats #apachearrow

I may have fallen down the rabbit hole here but...
#apacheArrow had my curiosity but now had my attention...
And I'm building wal2arrow for #postgres and #mysql...
It's kind of awesome not to have json /avro around in the hot paths.

I may have fallen down the rabbit hole here but...
#apacheArrow had my curiosity but now had my attention...
And I'm building wal2arrow for #postgres and #mysql...
It's kind of awesome not to have json /avro around in the hot paths.

I'm aiming for a weekly Apache Sedona DB drop...this week's edition: wrangling some uncooperative input data into clean GeoParquet output using SedonaDB's excellent ApacheArrow interop. Bonus: learn about the postage stamps of farm field dotting the prairies are organized! #GISChat #ApacheArrow

https://dewey.dunnington.ca/post/2025/geodata-cleaning-with-arrow-and-sedonadb/

I'm aiming for a weekly Apache Sedona DB drop...this week's edition: wrangling some uncooperative input data into clean GeoParquet output using SedonaDB's excellent ApacheArrow interop. Bonus: learn about the postage stamps of farm field dotting the prairies are organized! #GISChat #ApacheArrow

https://dewey.dunnington.ca/post/2025/geodata-cleaning-with-arrow-and-sedonadb/

𝐒𝐨𝐦𝐞𝐭𝐢𝐦𝐞𝐬, 𝐭𝐡𝐞 𝐛𝐞𝐬𝐭 𝐰𝐚𝐲 𝐭𝐨 𝐦𝐨𝐯𝐞 𝐟𝐨𝐫𝐰𝐚𝐫𝐝 𝐢𝐬 𝐭𝐨 𝐬𝐭𝐚𝐫𝐭 𝐨𝐯𝐞𝐫.

In this #InfoQ article, Paul Dix shares the story of #𝐈𝐧𝐟𝐥𝐮𝐱𝐃𝐁’𝐬 𝐜𝐨𝐦𝐩𝐥𝐞𝐭𝐞 𝐫𝐞𝐰𝐫𝐢𝐭𝐞, moving to #RustLang and a new stack -Apache Flight, Data Fusion, Apache Arrow & Parquet (FDAP).

He explores:
✅ The mechanics of the rewrite
✅ The benefits of the new architecture
✅ The different product versions

🔗 Read now: https://bit.ly/46eSvMV

#SoftwareArchitecture #ApacheArrow #Databases

𝐒𝐨𝐦𝐞𝐭𝐢𝐦𝐞𝐬, 𝐭𝐡𝐞 𝐛𝐞𝐬𝐭 𝐰𝐚𝐲 𝐭𝐨 𝐦𝐨𝐯𝐞 𝐟𝐨𝐫𝐰𝐚𝐫𝐝 𝐢𝐬 𝐭𝐨 𝐬𝐭𝐚𝐫𝐭 𝐨𝐯𝐞𝐫.

In this #InfoQ article, Paul Dix shares the story of #𝐈𝐧𝐟𝐥𝐮𝐱𝐃𝐁’𝐬 𝐜𝐨𝐦𝐩𝐥𝐞𝐭𝐞 𝐫𝐞𝐰𝐫𝐢𝐭𝐞, moving to #RustLang and a new stack -Apache Flight, Data Fusion, Apache Arrow & Parquet (FDAP).

He explores:
✅ The mechanics of the rewrite
✅ The benefits of the new architecture
✅ The different product versions

🔗 Read now: https://bit.ly/46eSvMV

#SoftwareArchitecture #ApacheArrow #Databases

OpenTelemetry Protocol with Apache Arrow

https://opentelemetry.io/blog/2025/otel-arrow-phase-2/

#HackerNews #OpenTelemetry #ApacheArrow #DataInstrumentation #TechTrends

OpenTelemetry Protocol with Apache Arrow

https://opentelemetry.io/blog/2025/otel-arrow-phase-2/

#HackerNews #OpenTelemetry #ApacheArrow #DataInstrumentation #TechTrends

My browser WASM’t prepared for this. Using DuckDB, Apache Arrow and Web Workers in real life | by Motif Analytics | Feb, 2025 | Medium

https://motifanalytics.medium.com/my-browser-wasmt-prepared-for-this-using-duckdb-apache-arrow-and-web-workers-in-real-life-e3dd4695623d

#DuckDB #ApacheArrow #DataAnalysis

My browser WASM’t prepared for this. Using DuckDB, Apache Arrow and Web Workers in real life | by Motif Analytics | Feb, 2025 | Medium

https://motifanalytics.medium.com/my-browser-wasmt-prepared-for-this-using-duckdb-apache-arrow-and-web-workers-in-real-life-e3dd4695623d

#DuckDB #ApacheArrow #DataAnalysis

My Browser WASM't Prepared for This. Using DuckDB, Apache Arrow and Web Workers

https://motifanalytics.medium.com/my-browser-wasmt-prepared-for-this-using-duckdb-apache-arrow-and-web-workers-in-real-life-e3dd4695623d

#HackerNews #MyBrowserWASM #PreparedForThis #DuckDB #ApacheArrow #WebWorkers

My Browser WASM't Prepared for This. Using DuckDB, Apache Arrow and Web Workers

https://motifanalytics.medium.com/my-browser-wasmt-prepared-for-this-using-duckdb-apache-arrow-and-web-workers-in-real-life-e3dd4695623d

#HackerNews #MyBrowserWASM #PreparedForThis #DuckDB #ApacheArrow #WebWorkers

#apachearrow and #gdal both rely on aws-c-cmmon and related packages

Having no trust in anything #Amazon related (though there are certainly good people at AWS Labs, too), a question:

Isn't there a way to make arrow and gdal depend on some other packages?

@jorisvandenbossche @gdal

#apachearrow and #gdal both rely on aws-c-cmmon and related packages

Having no trust in anything #Amazon related (though there are certainly good people at AWS Labs, too), a question:

Isn't there a way to make arrow and gdal depend on some other packages?

@jorisvandenbossche @gdal

For all the #data nerds out there, check this Humble Bundle ebook offer: https://www.humblebundle.com/books/tools-for-data-engineers-packt-books

It has books on #DataEngineering #ApacheArrow #DataGovernance #duckdb and much more, and you can allocate your donation to the #CoralGuardin charity

For all the #data nerds out there, check this Humble Bundle ebook offer: https://www.humblebundle.com/books/tools-for-data-engineers-packt-books

It has books on #DataEngineering #ApacheArrow #DataGovernance #duckdb and much more, and you can allocate your donation to the #CoralGuardin charity

I recently taught a tutorial on #DuckDB and #ApacheArrow with #Rstats at @latinr_conf The slides and content are available on GitHub https://github.com/fmichonneau/2024-latinr-duckdb-arrow/
The tutorial was recorded and I'll add the link to this thread when it becomes available.

I recently taught a tutorial on #DuckDB and #ApacheArrow with #Rstats at @latinr_conf The slides and content are available on GitHub https://github.com/fmichonneau/2024-latinr-duckdb-arrow/
The tutorial was recorded and I'll add the link to this thread when it becomes available.

The Apache Arrow ecosystem is an amazing enabler for a whole new range of database systems. Tonbo is a K/V store that uses Arrow+Parquet on storage systems ranging from OPFS (browser filesystem) to S3. And of course you get SQL for free with Apache DataFusion! #ApacheArrow #DataFusion #RustLang
https://tonbo.io/blog/introducing-tonbo

The Apache Arrow ecosystem is an amazing enabler for a whole new range of database systems. Tonbo is a K/V store that uses Arrow+Parquet on storage systems ranging from OPFS (browser filesystem) to S3. And of course you get SQL for free with Apache DataFusion! #ApacheArrow #DataFusion #RustLang
https://tonbo.io/blog/introducing-tonbo

New on our blog: #OpenTelemetry Protocol with #ApacheArrow in Production https://buff.ly/3XZE0IP