#blosc2 β Public Fediverse posts
Live and recent posts from across the Fediverse tagged #blosc2, aggregated by home.social.
-
RE: https://fosstodon.org/@Blosc2/116766180354235478
I am both humbled and amazed of how much you can build with modern LLMs, #Textual and #Blosc2
Seriously, if there is something that a LLM can boost creativity, this is TUI making.
-
π Our #PyDataGlobal 2025 tutorial recording on modern #Blosc2 & #Caterva2 features is out!
We show how compression is more than just a space saver, boosting performance for large in-memory & out-of-memory arrays via auto-chunking & parallelism.We also cover: π Serving Blosc2/#HDF5 data online with Caterva2 βοΈ Computing directly in the cloud (no downloads needed!)
Watch here: π https://www.youtube.com/watch?v=tUvSI3EpTBQ&list=PLGVZCDnMOq0qmerwB1eITnr5AfYRGm0DF&index=80
-
Interact with your vasts remote datasets right in your phone! π±
I've built a demo Jupyter notebook that connects to a Cat2Cloud server from an Android phone and slices into an 8 TB dataset, downloading a 1 MB chunk in under 100 milliseconds. β‘
The 8 TB dataset is from the Gaia DR3 catalogue. As it turns out, there are ~1000 stars in a cube of 100 light-years in our vicinity; the space is mostly empty.π π
Try this out by visiting: https://cat2.cloud/demo/roots/@public/large/slice-gaia-3d.ipynb
-
πIronPill 2π
In the second of our series of short videos ("ironPills") showcasing ironArray's work, we see how Blosc2 can be used to power heavy-duty linear algebra (100GB!) workflows
β‘1.5-2x faster than PyTorch + h5py!
𧱠automated chunking optimised for your machine's cache hierarchy
π simple one-line syntax ππππππΈ.ππππππ(π°, π±, πππππππ='πππ.ππΈππ')See blog here: https://ironarray.io/blog/la-blosc
-
π IronPill 1π
In the first of a series of short videos ("ironPills") showcasing ironArray's work, we see how Blosc2 can be used to calculate Fourier approximations:
β‘5x faster than NumPy
π£ fraction of the memory footprint
π pythonic one-line syntax πππ(πβ*βπππ(π)β+βπβ*βπππ(π),βππ‘ππ=π·)See full notebook here: https://github.com/Blosc/python-blosc2/blob/main/examples/ndarray/ironpill1.ipynb
(inspired by this blog post: https://towardsdatascience.com/numexpr-the-faster-than-numpy-library-that-no-ones-heard-of/)
-
π£οΈ Announcing Python-Blosc2 3.8.0 π
A step closer to compliance with the array-api standard: data-apis.org/array-api!
This is an effort across all array-based libraries so that your code works (e.g. for both blosc2 and NumPy) by simply changing the import statement below!Highlights:
β C-Blosc2 updated to latest 2.21.2
β Incorporate isnan, isfinite, isinf
β Better indexing coverage
β linspace and arange functions more numerically stable
β Improved array-api compliance -
#Blosc2 now runs directly in your browser! Leveraging the power of #WASM, #Pyodide, and #JupyterLite, you can harness efficient, adaptable compression through the web's universal interface. Experience the future of large-scale data processing without leaving your browser window.
Compress Better, Compute Bigger, Share Faster
-
π’ We are pleased to announce the integration of a new stack feature in #Blosc2 π, which allows for stacking large arrays along a new axis.
Performance benchmarks show that while aligned chunks yield the best results, #Blosc2 with unaligned chunks can still outperform #NumPyβa welcome discovery! π
Many thanks to Luke Shaw for his excellent work on this new functionality. π
We've updated our recent blog post:
Check it out! π https://www.blosc.org/posts/blosc2-new-concatenate/#stacking-arrays -
π Excited to share more about Caterva2, your ultimate gateway to Blosc2/HDF5 repositories! π
Caterva2 is designed to redefine how you interact with large datasets.
Want to see it in action? π€ We've just released a new introductory video showcasing Caterva2's main functionalities! π¬
π https://ironarray.io/caterva2
#Caterva2 #Blosc2 #HDF5 #BigData #DataManagement #FreeSoftware #Python #DataScience #Tech
-
Now it's @FrancescAlted to introduce the #Blosc2 #compression algorithm to reduce #HDF5 file size.