home.social

#blake3 — Public Fediverse posts

Live and recent posts from across the Fediverse tagged #blake3, aggregated by home.social.

  1. The performance of blake3 multithreaded is very impressive:

    hmac-sha256 1.00 GB 0.327s <-- with hw accelerated sha256

    blake2b-256 1.00 GB 0.952s <-- pure sw, used in borg 1.x

    blake3 1.00 GB 0.442s <-- pure sw, single-threaded

    blake3-mt 1.00 GB 0.078s <-- pure sw, with MT.

    🚀 🚀 🚀

  2. Have pushed 0.9.5-dev branch to codeberg of foxing ( codeberg.org/aenertia/foxing/s ) in preparation for release tagging. A LOT of features and a couple of bug-fixes now the packet/file processing engine has stabilized ; including Semantic Routing to Parsers for Metadata Extraction and in-path Binary analysis using local ORT/BERT models ; letting you get semantic search powers for free when you copy something with foxingd/fxcp #linux #filesystem #bert #vectordb #postgres #xfs #stratis #blake3 #localllm

  3. Have pushed 0.9.5-dev branch to codeberg of foxing ( codeberg.org/aenertia/foxing/s ) in preparation for release tagging. A LOT of features and a couple of bug-fixes now the packet/file processing engine has stabilized ; including Semantic Routing to Parsers for Metadata Extraction and in-path Binary analysis using local ORT/BERT models ; letting you get semantic search powers for free when you copy something with foxingd/fxcp #linux #filesystem #bert #vectordb #postgres #xfs #stratis #blake3 #localllm

  4. Have pushed 0.9.5-dev branch to codeberg of foxing ( codeberg.org/aenertia/foxing/s ) in preparation for release tagging. A LOT of features and a couple of bug-fixes now the packet/file processing engine has stabilized ; including Semantic Routing to Parsers for Metadata Extraction and in-path Binary analysis using local ORT/BERT models ; letting you get semantic search powers for free when you copy something with foxingd/fxcp #linux #filesystem #bert #vectordb #postgres #xfs #stratis #blake3 #localllm

  5. Have pushed 0.9.5-dev branch to codeberg of foxing ( codeberg.org/aenertia/foxing/s ) in preparation for release tagging. A LOT of features and a couple of bug-fixes now the packet/file processing engine has stabilized ; including Semantic Routing to Parsers for Metadata Extraction and in-path Binary analysis using local ORT/BERT models ; letting you get semantic search powers for free when you copy something with foxingd/fxcp #linux #filesystem #bert #vectordb #postgres #xfs #stratis #blake3 #localllm

  6. Have pushed 0.9.5-dev branch to codeberg of foxing ( codeberg.org/aenertia/foxing/s ) in preparation for release tagging. A LOT of features and a couple of bug-fixes now the packet/file processing engine has stabilized ; including Semantic Routing to Parsers for Metadata Extraction and in-path Binary analysis using local ORT/BERT models ; letting you get semantic search powers for free when you copy something with foxingd/fxcp #linux #filesystem #bert #vectordb #postgres #xfs #stratis #blake3 #localllm

  7. 📦 Efficient file syncing for massive datasets? Forget classic rsync!

    #BLAKE3 hashes instead of timestamps – hash 70GB in just ~20 seconds thanks to multithreading across all CPU cores

    #DevOps #Linux #SysAdmin #OpenSource #FileSync

    📊 Algorithm comparison for 70GB:
    - BLAKE3: ~20 seconds
    - MD5: ~1.5 minutes
    - SHA256: ~4.5 minutes

    🔍 The trick: Create a hash baseline, compare with `comm -13`, then use `rsync --files-from` to transfer only actually changed files

    🧵 👇

  8. @glepage

    ...what I really cannot comprehend is why there is no #BEP to migrate #BitTorrent to #CDC and #BLAKE3, and I guess I can't (won't) ask because last time I checked signing up to their mailing list required a #Google account

  9. @glepage

    ...what I really cannot comprehend is why there is no #BEP to migrate #BitTorrent to #CDC and #BLAKE3, and I guess I can't (won't) ask because last time I checked signing up to their mailing list required a #Google account

  10. @glepage

    ...what I really cannot comprehend is why there is no #BEP to migrate #BitTorrent to #CDC and #BLAKE3, and I guess I can't (won't) ask because last time I checked signing up to their mailing list required a #Google account

  11. Today I got to know that #nix 2.27 added #blake3 support and proper submodules and lfs support to #flakes! 🎉

  12. Other than supporting legacy systems or implementing cryptographic password functions that *should* be slow, why aren't your teams using GNU or BSD tar with #zstd compression and #xxHash or #Blake3 checksums yet? If time is money, please stop wasting either with legacy algorithms when there are faster, more secure, and more trusted options currently available.

  13. Other than supporting legacy systems or implementing cryptographic password functions that *should* be slow, why aren't your teams using GNU or BSD tar with #zstd compression and #xxHash or #Blake3 checksums yet? If time is money, please stop wasting either with legacy algorithms when there are faster, more secure, and more trusted options currently available.

  14. Other than supporting legacy systems or implementing cryptographic password functions that *should* be slow, why aren't your teams using GNU or BSD tar with #zstd compression and #xxHash or #Blake3 checksums yet? If time is money, please stop wasting either with legacy algorithms when there are faster, more secure, and more trusted options currently available.

  15. the syncing means that it uses (I think) a different distribution protocol than classic IPFS, but it's still content-addressable. (For instance it uses #blake3 hashing so you can incrementally check the integrity of big files) github.com/n0-computer/iroh

  16. Content addressable storage is simple and elegant right up to the point when you need to support multiple hash functions and/or digest sizes. After that, it becomes a much more difficult beast to tame.

    But tame that fucking beast I will, goddamnit!

    I'm really trying to learn from Git's difficulty in migrating to something other than sha1:
    git-scm.com/docs/hash-function

    One thing I'm doing differently is supporting multiple hash functions and digest sizes from the Git-go (hehe).

  17. Maybe one day we will get Blake3 in hashlib #Python or not

    bugs.python.org/issue39298

    The thread is full of insightful comments from power consumption, compilers, cpu, social interactions in the development of an open source language, complexity... it's anthropology from a computer science and security perspective.

    You can learn a lot by just reading at some issues... and forgot about your initial search why there is no Blake3 in Python.

    #cryptography #blake3