#rle — Public Fediverse posts

#ReleaseSaturday — This week I've been working on extracting, refactoring & generalizing the minimal column store database I've been using for my personal knowledge/media management toolset, and I'm happy to share it with the world now:

https://thi.ng/column-store

This is an in-memory column store database with:

- Customizable column storage types with configurable min/max cardinality, support for optional and/or tuple-values, default values
- Support for custom column type implementations
- Optional dictionary encoding of column values (memory & filesize saving)
- Powerful extensible multi-term query engine with built-in OR/AND/NOR/NAND operators and predicate-based matchers (column, row, partial row). Queries can be pre-built and then executed as standard JS iterables
- Optional bitfield indexing for dramatic query acceleration (esp. for complex multi-term queries)
- Dynamic adding/removing of columns
- JSON serialization with optional RLE compression (in my PKM dataset with ~20k items, the RLE compressed version is only 29% of the normal JSON serialization)

I hope the readme and code examples give a decent overview for now... I've been using the overall system for a couple of years now, but this new packaged version is still marked as _alpha_. Everything's still being worked on.

Also, for those wondering what's the point of this all and why not using SQLite etc. — I find there're many use cases for a which a pure JSON-based approach is more than sufficient (without requiring extra tools and interfacing layers). The structure/storage model and the bitfield optimizations enable very fast query performance (compared to other JSON db's I've tried in the past)...

(Including all dependencies [only some other thi.ng packages], the entire DB package is ~6KB brotli'd, 19KB uncompressed...)

#ThingUmbrella #TypeScript #JavaScript #JSON #Database #QueryEngine #RLE #SmallWeb

Awesome!

“The Burrows-Wheeler Transform”, Robert Aboukhalil (https://sandbox.bio/concepts/bwt).

Via HN: https://news.ycombinator.com/item?id=45532352

On Lobsters: https://lobste.rs/s/qar3sx/burrows_wheeler_transform_bwt

#BWT #BurrowsWheelerTransform #Encoding #Compression #RLE #Programming #Algorithms

Как я создал архиватор из задачки с техсобеса: сжатие файлов с помощью RLE

Привет, меня зовут Рома. Я работаю в KTS на позиции Python backend-разработчика. Однажды мне взбрело в голову написать собственную имплементацию алгоритма сжатия RLE . В этой статье рассказываю подробнее про RLE: что это за зверь такой, где используется, чем плох, чем хорош, и какие неожиданные сложности могут возникнуть при попытке имплементации.

https://habr.com/ru/companies/kts/articles/831440/

#RLE #сжатие_данных #bzip2 #задача #алгоритмы

So last Tuesday, after my wife's last RLE appointment in Tukwila (an eye thing, this was the "lockdown"), we had breakfast at the Huckleberry Square in Burien. Then we did a walk in Lincoln Park looking for the troll. This was followed by a stroll along Alki. Ended with beers at the Alki Tap Shack. The black IPA was good. #PNW #Seattle #Alki #WestSeattle #walking #RLE #photography #GooglePixel

Delta-Rle-Huffman (DRH) Texture Format

Всем привет! В этой статье я опишу алгоритм работы формата сжатия изображений без потерь. Сжатие использует известные методики, которые и дали ему название. Проект начинался с простых экспериментов, которые вышли из под контроля. Не смотря на то, что формат чаще сжимает лучше чем png, никакого практического применения этот формат не имеет, оставаясь чисто академическим. Внимание! В статье много картинок. Кому интересно, добро пожаловать под кат!

https://habr.com/ru/articles/815781/

#Сжатие_изображений #Хаффман #дельта_кодирование #rle

@jonthegeek @coolbutuseless @teunbrand

Here is an approach to do #2D #RLE applied to the processing of #ECG images: https://link.springer.com/chapter/10.1007/978-981-13-5802-9_86, and also there is a 3D RLE approach for volumetric medical image data: https://www.tandfonline.com/doi/abs/10.1080/13682199.2019.1565695
In my previous #Bioinformatics life, RLE was used, there is rle2() for the #Bio3D package which uses indices (http://thegrantlab.org/bio3d/reference/rle2.html).
Also, there is an improvement of RLE with a bit of Huffman encoding thrown in: https://arxiv.org/abs/2101.05329

According to my calendar, today in 2010 I joined FB as Janet. That was a pretty major milestone in my social transition. I'd already been in therapy about 6 months by then. Back then, the so-called Real Life Experience (RLE) was a standard requirement if you wanted to be able to access hormones. My RLE began this month. I don't remember the exact date I first went out as myself other than to therapy.
#trans #transgender #RLE #milestones