Parsing XML EXIF from .avif files (plus a rant)
27 points by AndrewStephens
27 points by AndrewStephens
JPEG XL is the new hotness to replace JPEG + PNGs (with much lower encoding times). Heck, browser are finally (re)picking it up as if it wasn’t a grand Google conspiracy despite most artists & art-leaning tools encouraging it. I’m excited to have progressive decoding to come back instead of all these WordPress hacks that use JavaScript + extra images for these style of video container formats that don’t have a low res display. /my-own-rant
It’s extremely silly how there’s compression for pixels in various formats but then Adobe’s software puts a whole bunch of uncompressed XML in there. And not only that, they write padding around the XML so that the XML could be edited in place without affecting other offsets! It’s amazingly hostile to small file sizes.
JPEG XL is unique in supporting Brotli-compressed XMP (and EXIF). Unfortunately, consumers of ISOBMFF-based formats that existed before JPEG XL don’t know about these box types, so the MP4, AVIF, and HEIF don’t get these box types “for free”.
And even with JPEG XL, compressing metadata is optional and Adobe’s software stores XMP uncompressed in JPEG XL, too, so you need to post-process Adobe-produced JPEG XL. (Exiftool can compress XMP without changing the pixel data, but Adobe also sets the pixel compression effort so low that I just take JPEG XL out of Lightroom Classic at max quality and recompress both pixels and metadata afterwards.)
But this is really pushing it - for a modern format they had the opportunity to produce something future-proof and sane. I am not sure if HEIF is the silliest binary format I have encountered but it ranks.
I believe HEIF is a rebadged version of the MP4 container format, which was a rebadged version of the QuickTime container format designed by Apple in the 1990s, when video was something postage-stamp-sized you had to open in a special app to view. I'm not surprised that something that started as a simple, future-proof IFF-based format got more complex over time, as people tacked on features like "interleaving streams of data" and "supporting files over 2GB" rather than throw the whole thing out and start from scratch every time. XML
EXIF is similar, I think - there's a binary encoding for the basic EXIF information, but (unlike IFF) it's not self-describing, so you already have to know what format the data is in before you parse it, and there's multiple deliberately-incompatible undocumented, proprietary variants. Adobe said "this is nuts, we should store this in a format that's simple and future-proof" and it just so happened that XML was the flavour of the day. It's verbose and a pain to deal with, but at least you can add new fields to it (with proper namespaces!) and existing readers can quietly ignore it.
Bjarne Stroustrup once responded to criticisms of C++ saying something like "there's two kinds of languages: the kind people hate and the kind nobody uses". I think there's something similar with file-formats; either a format is useful enough that people want to bolt their thing on the side rather than start a new ecosystem, or nobody uses it. I agree that these kinds of format quirks are a pain to deal with, but I've come to see them as signs of success rather than failure. They're the result of future-proofing, not the lack of it.
OP here. One of the problems I have with complex binary formats like this is that they end up getting implemented in web browsers, where untrusted content is basically the default. Historically, many of the security issues with browsers have been the insecure parsing of malformed binary formats such as fonts and image files. I am all for flexibility but these formats are too convoluted. Starting again with a sane and extensible self describing block format for the container might have been a better choice than trying to shoehorn a new format into a decades old container.
The very first thing in Firefox that was rewritten in Rust was the MP4 parser (that also got used for AVIF when AVIF was added).