#fileformatidentification — Public Fediverse posts
Live and recent posts from across the Fediverse tagged #fileformatidentification, aggregated by home.social.
-
Declarative all the way down: Building PRONOM signatures with JSONID
by @beet_keeperPRONOM signatures are a form of declarative language, you describe the anticipated behavior in PRONOM’s regular expression syntax and tools like DROID, FIDO, and Siegfried will interpret those instructions and attempt to match them against different files to return a file format identification.
Normally, you will write PRONOM signatures by hand but doing so for file formats based on other file format building blocks can lead to inconsistencies. Bertrand Caron previously also recognized this in the XML formats that are described with PRONOM signatures on Wikidata.
XML can use single quotes ‘ (hex: 0x27) and double quotes ” (hex: 0x22) for attribute data, and so, do we make a PRONOM signature with multiple sequences anticipating the use of either?
The answer is more often than not likely to be yes, because the appearance of these values are often helpful for identifying boundaries for strings that we know must exist.
But the more file formats that we need to add to PRONOM that are based on foundational formats like XML, or JSON, or similar, the more inconsistencies will creep in, such as sequences that are looking specifically for one byte sequence over another.
The issue extends further if file formats allow data to appear at the beginning of file, or we need to account for a variable amount of white-space, or we want to start thinking about multi-byte character encoding.
We can, and in the XML issue described by myself and Caron, I think the recommendation is very much to create editorial standards for signatures for file formats based on other baseline, structured data formats like XML, JSON, YAML, and so on.
And standards are well and good, but what if tooling could help us?
For JSON this is exactly what I have tried to do in JSONID.
What does this feature look like? And what does it get us? Let’s take a look.
#declarativeProgramming #digipres #DigitalPreservation #DROID #FIDO #FileFormatIdentification #FileFormats #JSON #jsonid #JSONL #NTTW #NTTW9 #PRONOM #RDM #ResearchData #siegfried #StructuredData #structuredText #TOML #YAML -
Declarative all the way down: Building PRONOM signatures with JSONID
by @beet_keeperPRONOM signatures are a form of declarative language, you describe the anticipated behavior in PRONOM’s regular expression syntax and tools like DROID, FIDO, and Siegfried will interpret those instructions and attempt to match them against different files to return a file format identification.
Normally, you will write PRONOM signatures by hand but doing so for file formats based on other file format building blocks can lead to inconsistencies. Bertrand Caron previously also recognized this in the XML formats that are described with PRONOM signatures on Wikidata.
XML can use single quotes ‘ (hex: 0x27) and double quotes ” (hex: 0x22) for attribute data, and so, do we make a PRONOM signature with multiple sequences anticipating the use of either?
The answer is more often than not likely to be yes, because the appearance of these values are often helpful for identifying boundaries for strings that we know must exist.
But the more file formats that we need to add to PRONOM that are based on foundational formats like XML, or JSON, or similar, the more inconsistencies will creep in, such as sequences that are looking specifically for one byte sequence over another.
The issue extends further if file formats allow data to appear at the beginning of file, or we need to account for a variable amount of white-space, or we want to start thinking about multi-byte character encoding.
We can, and in the XML issue described by myself and Caron, I think the recommendation is very much to create editorial standards for signatures for file formats based on other baseline, structured data formats like XML, JSON, YAML, and so on.
And standards are well and good, but what if tooling could help us?
For JSON this is exactly what I have tried to do in JSONID.
What does this feature look like? And what does it get us? Let’s take a look.
#declarativeProgramming #digipres #DigitalPreservation #DROID #FIDO #FileFormatIdentification #FileFormats #JSON #jsonid #JSONL #NTTW #NTTW9 #PRONOM #RDM #ResearchData #siegfried #StructuredData #structuredText #TOML #YAML -
✨Bringing light to #FileFormats ✨
Tough on the outside with food for thought inside!
File formats reveal a lot about the different meanings of an object.
CPP-008 and CPP-010 help you uncover them through #FileFormatIdentification and #FileFormatValidation:
http://tiny.cc/cpp-desc -
✨Bringing light to #FileFormats ✨
Tough on the outside with food for thought inside!
File formats reveal a lot about the different meanings of an object.
CPP-008 and CPP-010 help you uncover them through #FileFormatIdentification and #FileFormatValidation:
http://tiny.cc/cpp-desc -
File formats as Emoji: 0xffae
by @beet_keepertldr: https://emoji.exponentialdecay.co.uk
File Formats As Emoji (0xFFAE or 0xffae) might be my most random file format hack yet. Indeed, it is a random page generator! But it generates random pages of file formats represented as Emoji.
The idea came in 2016 with radare releasing a new version that supported an emoji hexdump. I wondered whether I could do something fun combining file
#0xffae #Code #Coding #digipres #digitalLiteracy #DigitalPreservation #emoji #FileFormat #FileFormatIdentification #FileFormats #learning #PRONOM #pyscript #Python #SkeletonTestCorpus #teaching