#ahalodeck — Public Fediverse posts

I'm not the only, and definitely not the first one to (re)consider the #filesystem for #database use.

The #BeFS documentation (from #HaikuOS website) clearly states - and does that - already (2004):

> "[key/value metadata attributes] add considerable power to a file system and endow the file system with many of the features normally associated with a database."

https://www.haiku-os.org/legacy-docs/practical-file-system-design.pdf

#AHAlodeck

I'm not the only, and definitely not the first one to (re)consider the #filesystem for #database use.

The #BeFS documentation (from #HaikuOS website) clearly states - and does that - already (2004):

> "[key/value metadata attributes] add considerable power to a file system and endow the file system with many of the features normally associated with a database."

https://www.haiku-os.org/legacy-docs/practical-file-system-design.pdf

#AHAlodeck

A truly magic and beautiful generic object identifier (ID) syntax:

⭐️$TIMESTAMP*$CONTEXT*$RANDOM❤️

They auto-cluster related datas in an inavoidable, nor requiring any "data-janitor" actions to work.

It transforms any digital data collection into a #DataSoil, which can easily be shaped into a #DataGarden.

I call them "CollisionFriendly IDs" (or #CFIDs for short)

It also provides auto dup-detection

+ #fuzzy #mnemonic search

+ Stays shorter than default path+filename S3 IDs

#ahalodeck

I have manually created over several hundred-thousand digital filesystem objects on my storage(s).

Even way more blocks, the filesystem must keep assigned and working properly.

The whole world trust the filesystem to keep those bits in order and accessible.

Why not trust the filesystem to do the same with more objects - and split everything into xattr-annotated objects?

It'd be doing the same thing: Simply for /more/ objects.

#AHAlodeck

I have manually created over several hundred-thousand digital filesystem objects on my storage(s).

Even way more blocks, the filesystem must keep assigned and working properly.

The whole world trust the filesystem to keep those bits in order and accessible.

Why not trust the filesystem to do the same with more objects - and split everything into xattr-annotated objects?

It'd be doing the same thing: Simply for /more/ objects.

#AHAlodeck

https://cloud.arkthis.com/index.php/s/AHAlodeck2026?dir=/videos

A 50 minute show-and-tell, about #AHAlodeck, and how #xattrs on #ZFS can be used as #DataCentric catalog/environment.

Enjoy!

Perfect for #longterm preservation #dltp - and any other digital data collections.

https://cloud.arkthis.com/index.php/s/AHAlodeck2026?dir=/videos

A 50 minute show-and-tell, about #AHAlodeck, and how #xattrs on #ZFS can be used as #DataCentric catalog/environment.

Enjoy!

Perfect for #longterm preservation #dltp - and any other digital data collections.

#AHAlodeck R&D news:

- Testdataset: 386.5 GB
- 120.000 filesystem objects (files and folders)
- 7616839 key/value informations gathered/extracted as #xattrs.

Result: A 55 MB .tar.bz2 #holotar.

Fully search-and-filter-able by any of the 7 Mio xattrs.

Extraction step required only once (6h)
Afterwards:
Indexing done in 60 SECONDS! 🤩 🥇

#AHAlodeck R&D news:

- Testdataset: 386.5 GB
- 120.000 filesystem objects (files and folders)
- 7616839 key/value informations gathered/extracted as #xattrs.

Result: A 55 MB .tar.bz2 #holotar.

Fully search-and-filter-able by any of the 7 Mio xattrs.

Extraction step required only once (6h)
Afterwards:
Indexing done in 60 SECONDS! 🤩 🥇

Same data (key/value pairs) viewed in different application:

`apt install eiciel`

That's a #DataCentric paradigm in action! You can literally shoot any mixed (annotated) data at that.

It really puts fun back into using data daily.

#AHAlodeck

Same data (key/value pairs) viewed in different application:

`apt install eiciel`

That's a #DataCentric paradigm in action! You can literally shoot any mixed (annotated) data at that.

It really puts fun back into using data daily.

#AHAlodeck

Handed in my presentation proposal for #nttw10 #notimetowait

Presenting "#DataGardens" 🌻️🌻️🌻️
Powered by #AHAlodeck to host any arbitrary data, offload key/value to filesystem - and transform datas into annotated related object graphs.

Pure "#ScienceFaction"!
#WeAllHaveBigDataNow

in #ZFS on #Linux, #Proxmox, #Debian, #Ubuntu #FOSS #FinF #ObjectStorage

Handed in my presentation proposal for #nttw10 #notimetowait

Presenting "#DataGardens" 🌻️🌻️🌻️
Powered by #AHAlodeck to host any arbitrary data, offload key/value to filesystem - and transform datas into annotated related object graphs.

Pure "#ScienceFaction"!
#WeAllHaveBigDataNow

in #ZFS on #Linux, #Proxmox, #Debian, #Ubuntu #FOSS #FinF #ObjectStorage

Download #wikidata dump for testing #ahalodeck and #xattrs on #zfs long-term storage... 😎

Found this while doing so:

https://www.wikidata.org/wiki/Wikidata:Lists/lexemes/en

It's a statistical breakdown of the #English #language. #Interesting.

Download #wikidata dump for testing #ahalodeck and #xattrs on #zfs long-term storage... 😎

Found this while doing so:

https://www.wikidata.org/wiki/Wikidata:Lists/lexemes/en

It's a statistical breakdown of the #English #language. #Interesting.

#AHAlodeck R&D news:

Created holotar-copy of a real-world mixed audio collection (recording production and digitized tracks archive): ~120.000 filesystem objects (files/folders)

(donated by recording engineers)

filesystem metadata-only = 175 MB (.tar)
+exiftool metadata as xattrs = 252 MB (.tar)

...compressed (.tar.bz2): **6,3 MB** ( 🤯 😎 🤓 💾 ❗)

**This is AMAZING!**

I can browse, order, catalog and manifest any object in this collection using standard #GNU #Linux tools.
@beet_keeper

#AHAlodeck R&D news:

Created holotar-copy of a real-world mixed audio collection (recording production and digitized tracks archive): ~120.000 filesystem objects (files/folders)

(donated by recording engineers)

filesystem metadata-only = 175 MB (.tar)
+exiftool metadata as xattrs = 252 MB (.tar)

...compressed (.tar.bz2): **6,3 MB** ( 🤯 😎 🤓 💾 ❗)

**This is AMAZING!**

I can browse, order, catalog and manifest any object in this collection using standard #GNU #Linux tools.
@beet_keeper

#AHAlodeck: My best friend wrote a mini-basic fulltext indexer for #xattrs in the shell.

Took ~1.5 seconds (!) to index ~14.000 files with "de-embedded" music tags.

🥳 🎉
I love xattrs.
Imagine doing this with embedded metadata tags? 😉

#AHAlodeck: My best friend wrote a mini-basic fulltext indexer for #xattrs in the shell.

Took ~1.5 seconds (!) to index ~14.000 files with "de-embedded" music tags.

🥳 🎉
I love xattrs.
Imagine doing this with embedded metadata tags? 😉

https://git-annex.branchable.com/git-annex-metadata/

git seems to have embedded key-value #metadata annotation capabilities... 😻 #AHAlodeck ahoi!

https://github.com/ArkThis/AHAlodeck/blob/main/res/0byte-db_entry.png

#xattr #linux #ext4 #zfs #samba #smb

Right-click-edit-metadata happily in-use over 1 year now: Amazing!

#AHAlodeck #Recoll #SENlabs

so **each** key/value-pair of #metadata can be about 64kB.

This is sufficient to hold most annotations stored as #EmbeddedMetadata currently present in most file formats.
(eg #mp3, #mp4, #iptc, #tiff, #jpg, etc)

But from then on, the file-format becomes irrelevant for this task!

#ahalodeck #digipres #audiovisual #cataloging

Very helpful insights on "how much actual bytes" can be stored in #xattrs on `btrfs`:

https://github.com/kdave/btrfs-progs/issues/917

Current limit = about the nodesize (it was formatted with).

nodesize - sizeof(struct btrfs_header) - sizeof(struct btrfs_item) - sizeof(btrfs_dir_item)

Current default = 16K - 101 - 25 - 30 , which is 16228 bytes

Maximum (for now) is ~64kB, if formatted like this:

`$ mkfs.btrfs -n 64K /dev/sdX`

#ahalodeck #metadata #digipres

https://vectorvfs.readthedocs.io/en/latest/index.html

#VectorFS: "transforms your Linux filesystem into a vector database by leveraging the native VFS (Virtual File System) extended attributes."

This proofs my design to offload /many/ #database functionality onto the filesystems is a working way to go.

*nice* 😌 🛋️ #AHAlodeck

https://sen-labs.org/

"Link local files, reference data seamlessly, and manage your personal knowledge with just your file browser – all open-source and integrated into your filesystem."

Sounds familiar?

No this is not my #AHAlodeck, but I'm very happy someone has the same ideas - and even happier that we're now in contact with each other! 🥳

"Personal Knowledge Graphs: why, what, and where to?"
(Jul 4, 2021)

Ivo Velitchkov

https://youtu.be/7ik0JNnoXBc

Very interesting. It confirms my observations and research on #AHAlodeck:

It's time we allow ourselves to do-digital-data differently. #GLAM #DLTP #Archiving

Oh, I'm loving this!

> "The Data-Centric revolution puts data at the center of the enterprise. Applications are optional visitors to the data."

https://datacentricmanifesto.org/

#AHAlodeck #RDF #GraphQL #xattrs

https://personalknowledgegraphs.com/#/page/extending%20the%20desktop%20into%20a%20personal%20knowledge%20graph%20with%20sen

I'm not alone! 🥳

Yes! It's time to offload meta+data issues to the filesystem. A common, standard ground - and scale with hardware - not #EternalMigration neverending human-hours...

#Haiku #BeOS #AHAlodeck

Me engaging on #Smbd ("Samba") mailing lists about support for "larger" (eg 64kB) key/value attributes on filesystem level (#xattrs):

https://lists.samba.org/archive/samba-technical/2025-February/139325.html

Good news! 🥳

> "in 2021 we added smbd max xattr size with a default of 64KB."

#AHAlodeck #DLTP

Reading a "History of Graph Databases" article on #redis:

https://redis.io/glossary/graph-database/

Having worked with mostly SQL and table data, I feel like entering a new era of data handling!

(changing data paradigms, changes everything - 😎 🌈 🦄 ) #AHAlodeck

I truly believe it's time to take "thinking data differently" more serious and read up on Wikidata-related data-modeling stuff:

https://www.mediawiki.org/wiki/Wikibase/DataModel/Primer

#DigiPres #Metadata #Cataloging #AHAlodeck 🌈 🤯 💾