#fileformats — Public Fediverse posts
Live and recent posts from across the Fediverse tagged #fileformats, aggregated by home.social.
-
Declarative all the way down: Building PRONOM signatures with JSONID
by @beet_keeperPRONOM signatures are a form of declarative language, you describe the anticipated behavior in PRONOM’s regular expression syntax and tools like DROID, FIDO, and Siegfried will interpret those instructions and attempt to match them against different files to return a file format identification.
Normally, you will write PRONOM signatures by hand but doing so for file formats based on other file format building blocks can lead to inconsistencies. Bertrand Caron previously also recognized this in the XML formats that are described with PRONOM signatures on Wikidata.
XML can use single quotes ‘ (hex: 0x27) and double quotes ” (hex: 0x22) for attribute data, and so, do we make a PRONOM signature with multiple sequences anticipating the use of either?
The answer is more often than not likely to be yes, because the appearance of these values are often helpful for identifying boundaries for strings that we know must exist.
But the more file formats that we need to add to PRONOM that are based on foundational formats like XML, or JSON, or similar, the more inconsistencies will creep in, such as sequences that are looking specifically for one byte sequence over another.
The issue extends further if file formats allow data to appear at the beginning of file, or we need to account for a variable amount of white-space, or we want to start thinking about multi-byte character encoding.
We can, and in the XML issue described by myself and Caron, I think the recommendation is very much to create editorial standards for signatures for file formats based on other baseline, structured data formats like XML, JSON, YAML, and so on.
And standards are well and good, but what if tooling could help us?
For JSON this is exactly what I have tried to do in JSONID.
What does this feature look like? And what does it get us? Let’s take a look.
#declarativeProgramming #digipres #DigitalPreservation #DROID #FIDO #FileFormatIdentification #FileFormats #JSON #jsonid #JSONL #NTTW #NTTW9 #PRONOM #RDM #ResearchData #siegfried #StructuredData #structuredText #TOML #YAML -
Declarative all the way down: Building PRONOM signatures with JSONID
by @beet_keeperPRONOM signatures are a form of declarative language, you describe the anticipated behavior in PRONOM’s regular expression syntax and tools like DROID, FIDO, and Siegfried will interpret those instructions and attempt to match them against different files to return a file format identification.
Normally, you will write PRONOM signatures by hand but doing so for file formats based on other file format building blocks can lead to inconsistencies. Bertrand Caron previously also recognized this in the XML formats that are described with PRONOM signatures on Wikidata.
XML can use single quotes ‘ (hex: 0x27) and double quotes ” (hex: 0x22) for attribute data, and so, do we make a PRONOM signature with multiple sequences anticipating the use of either?
The answer is more often than not likely to be yes, because the appearance of these values are often helpful for identifying boundaries for strings that we know must exist.
But the more file formats that we need to add to PRONOM that are based on foundational formats like XML, or JSON, or similar, the more inconsistencies will creep in, such as sequences that are looking specifically for one byte sequence over another.
The issue extends further if file formats allow data to appear at the beginning of file, or we need to account for a variable amount of white-space, or we want to start thinking about multi-byte character encoding.
We can, and in the XML issue described by myself and Caron, I think the recommendation is very much to create editorial standards for signatures for file formats based on other baseline, structured data formats like XML, JSON, YAML, and so on.
And standards are well and good, but what if tooling could help us?
For JSON this is exactly what I have tried to do in JSONID.
What does this feature look like? And what does it get us? Let’s take a look.
#declarativeProgramming #digipres #DigitalPreservation #DROID #FIDO #FileFormatIdentification #FileFormats #JSON #jsonid #JSONL #NTTW #NTTW9 #PRONOM #RDM #ResearchData #siegfried #StructuredData #structuredText #TOML #YAML -
I almost cancelled my Adobe subscription last year - then realised I couldn't.
Software like Adobe's isn't just selling tools, it's locking you in. The real cost isn't the subscription. It's that your work - effects, libraries, templates built over years - lives in proprietary formats. Cancel, and you lose access to your own assets.
It's not about price. It's about owning the way you work.
Ever been locked out of an old project?
#todormotion #DreamDesignDeliver #fileformats #Adobe -
I almost cancelled my Adobe subscription last year - then realised I couldn't.
Software like Adobe's isn't just selling tools, it's locking you in. The real cost isn't the subscription. It's that your work - effects, libraries, templates built over years - lives in proprietary formats. Cancel, and you lose access to your own assets.
It's not about price. It's about owning the way you work.
Ever been locked out of an old project?
#todormotion #DreamDesignDeliver #fileformats #Adobe -
I almost cancelled my Adobe subscription last year - then realised I couldn't.
Software like Adobe's isn't just selling tools, it's locking you in. The real cost isn't the subscription. It's that your work - effects, libraries, templates built over years - lives in proprietary formats. Cancel, and you lose access to your own assets.
It's not about price. It's about owning the way you work.
Ever been locked out of an old project?
#todormotion #DreamDesignDeliver #fileformats #Adobe -
I almost cancelled my Adobe subscription last year - then realised I couldn't.
Software like Adobe's isn't just selling tools, it's locking you in. The real cost isn't the subscription. It's that your work - effects, libraries, templates built over years - lives in proprietary formats. Cancel, and you lose access to your own assets.
It's not about price. It's about owning the way you work.
Ever been locked out of an old project?
#todormotion #DreamDesignDeliver #fileformats #Adobe -
I almost cancelled my Adobe subscription last year - then realised I couldn't.
Software like Adobe's isn't just selling tools, it's locking you in. The real cost isn't the subscription. It's that your work - effects, libraries, templates built over years - lives in proprietary formats. Cancel, and you lose access to your own assets.
It's not about price. It's about owning the way you work.
Ever been locked out of an old project?
#todormotion #DreamDesignDeliver #fileformats #Adobe -
Maintenance begins at creation, so why are we not creating better?
by @beet_keeperThe beats are the same. You work for government, or academia (lets face it, that’s probably where 90% of the work is) you have a deliverable; you save it; you print to PDF; you store it on an institutional repository with some metadata (or Zenodo, OSF or equivalent) and its done.
There’s a small chance that it’s FAIR (Findable, Accessible, Interoperable, Reusable) right? It has metadata that can be discovered by an audience looking for it and can be indexed by search engines. The data is potentially accessible if published correctly. They’re not particularly interoperable or easily converted, and PDFs aren’t really designed for reuse, even if tools like Apache Tika help ease the burden of extracting artifacts. It’s just a PDF, why are we even talking about FAIR? There begins a story…
The beats are the same, yet, we work in digital preservation, our backgrounds are in GLAM or software, why do we want to shoot ourselves in the foot? Why are we not using our skills to create better?
#Archives #BetterPoster #ContinuumModel #createToMaintain #digipres #DigitalArchiving #DigitalContunuity #digitalLiteracy #DigitalPreservation #FAIR #FileFormats #GLAM #informationRecordsMangagement #NationalDigitalStewardshipAlliance #NDSA #OpenAccess #OpenData #PDF #RDM #ResearchDataLifecycle #RIM -
Maintenance begins at creation, so why are we not creating better?
by @beet_keeperThe beats are the same. You work for government, or academia (lets face it, that’s probably where 90% of the work is) you have a deliverable; you save it; you print to PDF; you store it on an institutional repository with some metadata (or Zenodo, OSF or equivalent) and its done.
There’s a small chance that it’s FAIR (Findable, Accessible, Interoperable, Reusable) right? It has metadata that can be discovered by an audience looking for it and can be indexed by search engines. The data is potentially accessible if published correctly. They’re not particularly interoperable or easily converted, and PDFs aren’t really designed for reuse, even if tools like Apache Tika help ease the burden of extracting artifacts. It’s just a PDF, why are we even talking about FAIR? There begins a story…
The beats are the same, yet, we work in digital preservation, our backgrounds are in GLAM or software, why do we want to shoot ourselves in the foot? Why are we not using our skills to create better?
#Archives #BetterPoster #ContinuumModel #createToMaintain #digipres #DigitalArchiving #DigitalContunuity #digitalLiteracy #DigitalPreservation #FAIR #FileFormats #GLAM #informationRecordsMangagement #NationalDigitalStewardshipAlliance #NDSA #OpenAccess #OpenData #PDF #RDM #ResearchDataLifecycle #RIM -
Hot take on digital sovereignty of #fileformats #digipres "A license tells you who owns the software, while the format tells you who owns the data", "Digital sovereignty is not achieved by changing who hosts the software, but by changing the format in which data is encoded." https://blog.documentfoundation.org/blog/2026/04/02/document-formats-a-mystery-to-many/
-
Hot take on digital sovereignty of #fileformats #digipres "A license tells you who owns the software, while the format tells you who owns the data", "Digital sovereignty is not achieved by changing who hosts the software, but by changing the format in which data is encoded." https://blog.documentfoundation.org/blog/2026/04/02/document-formats-a-mystery-to-many/
-
All in a day's work for #archival #superheroes :
> What the data is that data?
> If only I had #METADATA ! 😁 🍰 -
All in a day's work for #archival #superheroes :
> What the data is that data?
> If only I had #METADATA ! 😁 🍰 -
Do you think custom text file formats are a good idea? or should we just always use JSON/YAML/TOML?
-
Do you think custom text file formats are a good idea? or should we just always use JSON/YAML/TOML?
-
✨Bringing light to #FileFormats ✨
Tough on the outside with food for thought inside!
File formats reveal a lot about the different meanings of an object.
CPP-008 and CPP-010 help you uncover them through #FileFormatIdentification and #FileFormatValidation:
http://tiny.cc/cpp-desc -
✨Bringing light to #FileFormats ✨
Tough on the outside with food for thought inside!
File formats reveal a lot about the different meanings of an object.
CPP-008 and CPP-010 help you uncover them through #FileFormatIdentification and #FileFormatValidation:
http://tiny.cc/cpp-desc -
Peeking Inside Gigantic Zips with Only Kilobytes
https://ritiksahni.com/blog/peeking-inside-gigantic-zips-with-only-kilobytes/
#HackerNews #Peeking #Inside #Gigantic #Zips #with #Only #Kilobytes #datacompression #fileformats #technology #programming #HackerNews
-
Peeking Inside Gigantic Zips with Only Kilobytes
https://ritiksahni.com/blog/peeking-inside-gigantic-zips-with-only-kilobytes/
#HackerNews #Peeking #Inside #Gigantic #Zips #with #Only #Kilobytes #datacompression #fileformats #technology #programming #HackerNews
-
Peeking Inside Gigantic Zips with Only Kilobytes
https://ritiksahni.com/blog/peeking-inside-gigantic-zips-with-only-kilobytes/
#HackerNews #Peeking #Inside #Gigantic #Zips #with #Only #Kilobytes #datacompression #fileformats #technology #programming #HackerNews
-
Peeking Inside Gigantic Zips with Only Kilobytes
https://ritiksahni.com/blog/peeking-inside-gigantic-zips-with-only-kilobytes/
#HackerNews #Peeking #Inside #Gigantic #Zips #with #Only #Kilobytes #datacompression #fileformats #technology #programming #HackerNews
-
Peeking Inside Gigantic Zips with Only Kilobytes
https://ritiksahni.com/blog/peeking-inside-gigantic-zips-with-only-kilobytes/
#HackerNews #Peeking #Inside #Gigantic #Zips #with #Only #Kilobytes #datacompression #fileformats #technology #programming #HackerNews
-
Sure, we've all seen Mark YAML and even JSON Statham, but have y'all met TOML Holland yet?
-
Sure, we've all seen Mark YAML and even JSON Statham, but have y'all met TOML Holland yet?
-
Ah, the "Binary Formats Gallery"—because what the world really needed was a #museum for file formats, complete with GraphViz diagrams and hex dump visualizers! 🤓🖥️ I mean, who doesn't want to spend their Sunday afternoon compiling Kaitai Struct libraries just to admire the metadata of a Quake 2 model? 🌟🔍
https://formats.kaitai.io/ #BinaryFormatsGallery #FileFormats #TechArt #KaitaiStruct #Quake2 #HackerNews #ngated -
Ah, the "Binary Formats Gallery"—because what the world really needed was a #museum for file formats, complete with GraphViz diagrams and hex dump visualizers! 🤓🖥️ I mean, who doesn't want to spend their Sunday afternoon compiling Kaitai Struct libraries just to admire the metadata of a Quake 2 model? 🌟🔍
https://formats.kaitai.io/ #BinaryFormatsGallery #FileFormats #TechArt #KaitaiStruct #Quake2 #HackerNews #ngated -
Ah, the "Binary Formats Gallery"—because what the world really needed was a #museum for file formats, complete with GraphViz diagrams and hex dump visualizers! 🤓🖥️ I mean, who doesn't want to spend their Sunday afternoon compiling Kaitai Struct libraries just to admire the metadata of a Quake 2 model? 🌟🔍
https://formats.kaitai.io/ #BinaryFormatsGallery #FileFormats #TechArt #KaitaiStruct #Quake2 #HackerNews #ngated -
Ah, the "Binary Formats Gallery"—because what the world really needed was a #museum for file formats, complete with GraphViz diagrams and hex dump visualizers! 🤓🖥️ I mean, who doesn't want to spend their Sunday afternoon compiling Kaitai Struct libraries just to admire the metadata of a Quake 2 model? 🌟🔍
https://formats.kaitai.io/ #BinaryFormatsGallery #FileFormats #TechArt #KaitaiStruct #Quake2 #HackerNews #ngated -
Revisiting bsdiff as a tool for digital preservation
by @beet_keeperI introduced bsdiff in a blog in 2014. bsdiff compares the differences between two files, e.g. broken_file_a and corrected_file_b and creates a patch that can be applied to broken_file_a to generate a byte-for-byte match for corrected_file_b.
On the face of it, in an archive, we probably only care about corrected_file_2 and so why would we care about a technology that patches a broken file?
In all of the use-cases we can imagine the primary reasons are cost savings and removing redundancy in file storage or transmission of digital information. In one very special case we can record the difference between broken_file_a and corrected_file_b and give users a totally objective method of recreating corrected_file_b from broken_file_a providing 100% verifiable proof of the migration pathway taken between the two files.
#ac3 #Archives #audio #audiovisual #Audit #authenticity #av #Bash #bsdiff #checksums #Code4Lib #corruption #corruptionIndex #digipres #DigitalArchiving #DigitalForensics #digitalLiteracy #DigitalPreservation #DigitalStorage #diplomatics #FileFormats #flac #glitch #glitchAudio #GlitchArt #integrity #mp3 #PreservationAnalysis #PreservationMetadata #provenance #sensitivityIndex #Storage #wav
-
Revisiting bsdiff as a tool for digital preservation
by @beet_keeperI introduced bsdiff in a blog in 2014. bsdiff compares the differences between two files, e.g.
broken_file_aandcorrected_file_band creates apatchthat can be applied tobroken_file_ato generate a byte-for-byte match forcorrected_file_b.On the face of it, in an archive, we probably only care about
corrected_file_2and so why would we care about a technology that patches a broken file?In all of the use-cases we can imagine the primary reasons are cost savings and removing redundancy in file storage or transmission of digital information. In one very special case we can record the difference between
Continue reading “Revisiting bsdiff as a tool for digital preservation”…broken_file_aandcorrected_file_band give users a totally objective method of recreatingcorrected_file_bfrombroken_file_aproviding 100% verifiable proof of the migration pathway taken between the two files.#ac3 #archives #audio #audiovisual #audit #authenticity #av #bash #bsdiff #checksums #code4lib #corruption #corruptionIndex #digipres #digitalArchiving #digitalForensics #digitalLiteracy #digitalPreservation #digitalStorage #diplomatics #fileFormats #glitch #glitchAudio #glitchart #integrity #preservationAnalysis #preservationMetadata #provenance #sensitivityIndex #storage
-
Revisiting bsdiff as a tool for digital preservation
by @beet_keeperI introduced bsdiff in a blog in 2014. bsdiff compares the differences between two files, e.g. broken_file_a and corrected_file_b and creates a patch that can be applied to broken_file_a to generate a byte-for-byte match for corrected_file_b.
On the face of it, in an archive, we probably only care about corrected_file_2 and so why would we care about a technology that patches a broken file?
In all of the use-cases we can imagine the primary reasons are cost savings and removing redundancy in file storage or transmission of digital information. In one very special case we can record the difference between broken_file_a and corrected_file_b and give users a totally objective method of recreating corrected_file_b from broken_file_a providing 100% verifiable proof of the migration pathway taken between the two files.
#ac3 #Archives #audio #audiovisual #Audit #authenticity #av #Bash #bsdiff #checksums #Code4Lib #corruption #corruptionIndex #digipres #DigitalArchiving #DigitalForensics #digitalLiteracy #DigitalPreservation #DigitalStorage #diplomatics #FileFormats #flac #glitch #glitchAudio #GlitchArt #integrity #mp3 #PreservationAnalysis #PreservationMetadata #provenance #sensitivityIndex #Storage #wav
-
Revisiting bsdiff as a tool for digital preservation
by @beet_keeperI introduced bsdiff in a blog in 2014. bsdiff compares the differences between two files, e.g. broken_file_a and corrected_file_b and creates a patch that can be applied to broken_file_a to generate a byte-for-byte match for corrected_file_b.
On the face of it, in an archive, we probably only care about corrected_file_2 and so why would we care about a technology that patches a broken file?
In all of the use-cases we can imagine the primary reasons are cost savings and removing redundancy in file storage or transmission of digital information. In one very special case we can record the difference between broken_file_a and corrected_file_b and give users a totally objective method of recreating corrected_file_b from broken_file_a providing 100% verifiable proof of the migration pathway taken between the two files.
#ac3 #Archives #audio #audiovisual #Audit #authenticity #av #Bash #bsdiff #checksums #Code4Lib #corruption #corruptionIndex #digipres #DigitalArchiving #DigitalForensics #digitalLiteracy #DigitalPreservation #DigitalStorage #diplomatics #FileFormats #flac #glitch #glitchAudio #GlitchArt #integrity #mp3 #PreservationAnalysis #PreservationMetadata #provenance #sensitivityIndex #Storage #wav
-
They really fucked up with the new #fileformats. Why even call it #webp. Just put the #png extension on it and no one would have to know if you made it backward compatible, your moronic apes. Just put #jpeg at the end of #jxl files.
Dumbass engineers they should hire me. I could have told them this.
-
They really fucked up with the new #fileformats. Why even call it #webp. Just put the #png extension on it and no one would have to know if you made it backward compatible, your moronic apes. Just put #jpeg at the end of #jxl files.
Dumbass engineers they should hire me. I could have told them this.
