An interesting quick dive into how some of the artifacts of historical implementations can carry forward decades later. I'm saving this one. While everyone on HN is probably familiar with a pile of examples of this in our own work particularly if you do anything lower level, this is a nice example because it's really easy to grasp even for total non-programmers and gives a feel for the layers upon layers the digital world is built on. Details on the x86-64 bootstrap sequence are interesting to tech people but this is the sort of thing I might show in a school class.
That said at first I thought the author was being a bit tongue in cheek on the suggestion to fix it with:
>A downside to making any changes is that the badly formatted GIFs won't render any more. I reckon just be bold, like the release of Netscape 2.0...If Netscape was fine with breaking poorly-made GIFs, we should be too!
But we kind of have a bit more content now then when Netscape 2 was released, and it seems they're serious?
Support more than 256 colours in a single frame of GIF animation
Support fast GIFs (10ms delay)
No confusing behaviour with small delay = slow GIF
Better compression for GIFs with multiple small areas updated per frame
But frankly at this point why? We've got good universally adapted image formats that do more than 256 colors. We've got webm and so on (even Apple caved on that one in the end) for baked animation which is much smaller and more powerful than gif, and plenty of CPU and libraries for generation on the fly. We've got powerful ways to do animation purely with HTML/CSS/SVG/JS. GIF is a historical artifact with a lot of its value at this point purely about old things. This wouldn't even retroactively bring newer capabilities to older systems because it would require updated code which by definition means the ability to run new browsers and thus modern capabilities.
I get they really enjoy hacking on gif but come on. Only way I can see it working is if all the players agreed on some metadata that could be added to gifs which wanted 'standards' vs 'historical' playback with the default remaining historical, and potentially also make it a browser setting. That'd still leave the ad issue, but maybe at this point that's moot given the advancements in blockers (and it'd be a very easy thing to look for as an indication of an ad, and thus be selected against).
The spiritual successor to animated GIFs — by that I mean nonlossy pixel-perfect animations — would be APNG, which is supported by virtually all current modern browsers: https://caniuse.com/apng
That sounds way less awesome, when you consider it only had 256 colors, so allmost all content did indeed loose a lot, by converting to gif.
And this is actually the first time I have heard of apng and interesting that it is even now widely avaiable. But I do not really see a broad use case, compared to webm.
Because file size matters usually much more, than pixel perfect animation.
I wanted to convert a short screencast to a gif, result: hundreds of megabyte big monster, with crappy quality.
> That sounds way less awesome, when you consider it only had 256 colors, so allmost all content did indeed loose a lot, by converting to gif.
Nonlossy can be used for pixel perfect animations, lossy formats cannot.
> Because file size matters usually much more, than pixel perfect animation.
It depends. For video, that is true. For user interface, any glitch is distracting. Using JPEG for user interface images like buttons is in most cases a bad idea. Each format is useful in different situations.
> I wanted to convert a short screencast to a gif, result: hundreds of megabyte big monster, with crappy quality.
You are right, that is not a use case for GIF or equivalent formats.
I suspect you could use JPEG as a lossless format, you'd just have to write your own custom encoder.
In some sense, JPEG is just a really weird programming language. So a JPEG file is just a really weird program that gets interpreted by a JPEG decoder. Most normal JPEG encoders try to find a short 'program' in the 'JPEG programming language' that gets interpreted by the decoder to recreate a particular image reasonably well.
But there's no reason you couldn't ask for a perfect reproduction.
As far as I know, JPEG decoders are lossless and deterministic.
Of course, you point still stands that JPEG is not a good file format for storing UI elements. Even if in theory you could torture the JPEG standard enough to make this barely work.
"You are right, that is not a use case for GIF or equivalent formats. "
It is not, but this was still the main use case of gif - sharing short video sequences.
With crappy quality, but it played everywhere.
Apng is not the successor to that. Webm is, or will be.
So there might be niche cases in UI or games, where apng might make sense, but in most cases, I doubt people would see a difference between apng and webm - but they will notice faster or shorter loading times.
The use-case isn’t video, it’s small icon-style color animations or animated pixel art (like the example in the article). It’s the same difference as between JPEG and PNG.
Well, yes - and for this use case I consider apng.
But my point was, that the main use case of gif, were short videos and animations, people shared. And for this, I am glad that webm seems now broadly avaiable for that role.
Well… that use case only started after the use of animated GIFs for small pixel-art animations already existed for many years. I always considered the video use, after it started, as a misuse/aberration. I certainly agree that webm is the right tool for the video use case. But I care about the original use cases of animated GIFs. If you want to get past the limitations of the format for those use cases, APNG should be the answer.
Per-frame palettes allow you to hack this a bit if you don't mind a few fractions of a second draw time for a static image¹² or only parts of the frame need to update each time for an animation.
Sadly the support for APNG isn't great. For example on Discord GIFs works, but not APNGs. I also know of another service where, due to internal caching (IP hiding), APNGs get broken and don't animate.
I heard that APNGs used to work, but then people just uploaded their APNGs instead of using emotes, and using emotes from different servers is the main selling point of Dicord's freemium "nitro" AFAIK...
Another reason could be that Discord caches images on its servers to hide viewer's IP from the original server hosting the image. The caching involves compression that sometimes breaks the image.
Discord's media display stack seems to be broken (for lack of a better word) in some very naïve ways. For example, embedding files with the ".gifv" extension (used on several popular websites to denote gif-like animations which are encoded in a modern video codec) doesn't work, even though they're valid video files expressing a webm or mp4 mime type. It instead displays a static image, which means they're reading in the video, then generating a thumbnail and ignoring the fact it's a video.
GIF is still the only format that supports animated frames, and is supported in most applications that support images. I sometimes create some animated WEBP to post somewhere, but the outcome is always the same, I have to go back and convert it to GIF or it won’t display.
That's kind of the point. What makes you think that people still using suboptimal GIFs are going to want to go update those GIFs to comply with breaking changes in how the spec is interpreted?
people won't change but if the gif library were updated then maybe browsers could support a modern rev with unambiguously defined behavior and compression improvements, maintain compatibility for older gif87a and gif89a files, and then a path would exist for gif authoring tools to eventually output gif222 by default at some point in the future (instead of throwing our hands up and letting this crusty old tech define our internet forever)
I don't believe the source code is still available officially (although it used to be through Microsoft's Shared Source Initiative and Government Security Program), but the Windows 2000 source code was leaked back in 2004.
GitHub is owned by Microsoft, does it even make sense for Microsoft to DMCA itself? It can just remove the content, or leave it, it owns the copyright and the platform, it can do whatever it wants (probably not, but IANAL).
// Okay, bug 30784 time. It's possible that we only got 1
// measly byte in the last data block. Rare, but it does happen.
// In that case, the additional byte may still not supply us with
// enough bits for the next code, so, as Mars Needs Women, IE
// Needs Data.
if ( end >= _gifinfo.lastbit && !_gifinfo.get_done )
Rate limiting or clamping "0" delays upwards can also help for polling applications where you want to be as fast as possible, but you're never going to be actually zero. For example if you have a thread that's pulling images from a camera (publishing at 30 fps) in a loop, you can delay for 1ms between polls and nobody is going to know, but that can have an enormous impact on CPU usage; especially on embedded platforms. Instead of running at 100% thread usage (calling camera.read() and waiting for it to return true), the spin/loop cost is basically free and you only need to worry about the cost of acquiring/decoding the frame. In theory branch prediction should help because you'll take the "no image yet" branch 99% of the time, but in practice that just makes the loop iterate faster. I learned this the hard way writing a custom camera publisher in ROS, but I've seen it a lot in tutorial code and I think a lot of beginners make that mistake.
Author here - loving all the discussion! I was a bit intimidated to submit this to hn - I'm humbled that someone else thought it was worth posting. Of course, happy to answer any questions anyone may have!
> You'll need to enable Javascript to see this section sorry! There's a slider that lets you see how changing the delay in a GIF file will affect the end results. It's pretty cool.
Thank you: this is an excellent way of handling the situation.
It’s not very common for people to handle these sorts of situations elegantly. I disable JavaScript by default (mostly for performance, frankly), and I encounter such elegance only once or twice a year.
(For myself, I take care on my site in the few places I ever use JavaScript, and also even vary my content in feeds where it depends on JavaScript or site styles, which I’ve never noticed anyone else do. make-and-git-diff-test-harness and dark-theme-implementation are slugs of two sample articles that both vary by JavaScript and feed, I won’t drop links.)
Great writing and research. You added great examples to make it clear, both in code and gifs. It was both fun to read and interesting. I was surprised to see Netscape running, I might try as well to play with the first JavaScript.
Thank you! I used VirtualBox running Windows 95, but there were a lot of roadblocks - finding a Netscape 2.0 installer, mounting it etc.
Unfortunately I didn't document the process at all and have forgotten everything, so I can't give much useful info, apologies. Just know that there is pain down that path.
Prime example of trade-off between cleanliness of implementation and backwards compatibility. As much as I like a pristine, pure spec, the market winners are clear. It pays to keep product debt around.
My favorite piece of GIF trivia is that in the original spec, GIF animations would play once and only once. Netscape used the Application Extension block to add number of loops or loop indefinitely. An Application Extension requires an application identifier. So basically every animated GIF ever made has the string `NETSCAPE2.0` in it. https://en.wikipedia.org/wiki/GIF#Animated_GIF
I'm not aware of any tools; I converted this document by hand in a few hours. GIF is a popular standard with cultural value, which is why I would do this; I wouldn't do it for any random obscure document. The IETF RFC website has links to HTML and PDF versions, but...
IMO a better, though more complex, solution would be to keep the frame delays as specified in the GIF, but to disable GIF looping if the sum of all frame delays was less than 10 ms. That way you get the one good use case (>256 colour gifs) but malicious or broken GIFs with 0 frame delays don't hang your machine. This is complicated by the fact that browsers (used to?) display the first frame of a GIF before the complete animation was loaded. But you could handle that easily enough by making this decision at the time of first loop rather than at initial GIF load.
(Fortunately, this is all largely irrelevant nowadays.)
> We follow Firefox's behavior and use a duration of 100 ms for any frames that specify a duration of <= 10 ms.
That is interesting & funny. I wonder why there wasn’t just a minimum value.
This article doesn’t talk about what happens after the delay is triggered, or that a value of 10ms won’t work for the average browser, video hardware & monitor. That’d be a refresh rate of 100 frames per second, and a lot of monitors are 60Hz refresh. Don’t browsers default to 60Hz refresh as well? (MDN docs mention requestAnimationFrame callbacks are usually 60/sec, or 16.7ms apart, for example.) This means with a timeout of 20ms, you’ll probably get a janky animation where one frame gets doubled every 6 frames. I guess with GIF it’s not possible to solve that since the spec doesn’t allow you to match typical refresh rates.
Yeah, as a gamedev I was surprised not to see this in the article. There's often a default assumption in renderers that "60 fps" is the baseline (and most typical devices will be limited to that). Obviously gaming devices and newer smartphones push above that to 120, 144 or 240 fps. But in gamedev you're always faced with this problem of update frequency of anything that will be displayed to the player; if you update too fast (or too slow), you're gonna get "stutters" and skipped (or stuck) frames as the state updates faster than the renderer can render it (when too fast), or at a rate that's out of sync with the actual frames being pushed to the monitor (too slow, at a rate that's not evenly divisible by the frame rate).
I don't know why the animated gif seems as popular as ever, I can't think of any positives about it... bad colours, large file sizes and are encoders more readily available than proper video encoders? Also, supporting it fully now requires either threading or an event loop. The value isn't worth the complexity.
Partly because of how they're presented to the user. You send a video, there's usually a play button, a bit of a delay to load, and sound. Gifs play automatically, first frame loads with barely any pause, and there's no sound ever. So, they present two different use cases, which, at the end of the day, have nothing to do with the format itself. But it can show that to humans, often the ends matter, not the means.
It's also a UI issue. On most OSs I can copy/paste an image like it was text. And a gif counts as an image so I can copy/paste images without having to save them. And since many modern UIs are hostile to saving videos, its the difference between holding and copying vs having to take a screen recording or finding a video download website to rip it.
They usually work where jpg and png images work too, most importantly in most places where you render Markdown. No matter how more efficient a real video file would be, often a short gif right there in the README.md is the perfect way to convey a piece of information.
It's still a reasonable format for its original use - images with large areas of solid color where only a small percentage of pixels change across a small number of frames.
If your animation has <256 colors then it's even lossless, something not possible with any of the video codecs.
Lossless h264 requires 4:4:4 chroma, which is allowed by the spec but not supported by many consumer playback methods. (QuickTime Player, for example, only supports the yuv420p pixel format.)
It is part of the High 4:4:4 Predictive profile, but you can use it with 4:2:0 subsampling if you want. It's more accurate that the all profiles H.264 defined kinda sucked except for Main, and no one really thought 10bit, 422 or especially 444 were useful and completely ignored those profiles until it was too late to be worth implementing.
Actually, lossless video is also rather marginally useful, especially given the decode cost of the arithmetic coding H.264/HEVC/VP9/AV1 have.
Ironically for your example, lossless, 4:4:4, and 10bit H.264 actually will play back on the new M1 Macs in QuickTime Player, though with a warning about it.
Given that the reason for the forced delay is historical abuse by ads, I'm wondering if back in the days this didn't lead to an arms race between browsers and ads (I wouldn't know because I always use an ad-blocker). For example, after canvas was introduced ads could use those instead.
If you think that's crazy, there are other things like this that browser vendors do that are not standardized, and when you start writing a renderer to figure out why your output doesn't match, you find all of these silent agreements everywhere.
That said at first I thought the author was being a bit tongue in cheek on the suggestion to fix it with:
>A downside to making any changes is that the badly formatted GIFs won't render any more. I reckon just be bold, like the release of Netscape 2.0...If Netscape was fine with breaking poorly-made GIFs, we should be too!
But we kind of have a bit more content now then when Netscape 2 was released, and it seems they're serious?
But frankly at this point why? We've got good universally adapted image formats that do more than 256 colors. We've got webm and so on (even Apple caved on that one in the end) for baked animation which is much smaller and more powerful than gif, and plenty of CPU and libraries for generation on the fly. We've got powerful ways to do animation purely with HTML/CSS/SVG/JS. GIF is a historical artifact with a lot of its value at this point purely about old things. This wouldn't even retroactively bring newer capabilities to older systems because it would require updated code which by definition means the ability to run new browsers and thus modern capabilities.I get they really enjoy hacking on gif but come on. Only way I can see it working is if all the players agreed on some metadata that could be added to gifs which wanted 'standards' vs 'historical' playback with the default remaining historical, and potentially also make it a browser setting. That'd still leave the ad issue, but maybe at this point that's moot given the advancements in blockers (and it'd be a very easy thing to look for as an indication of an ad, and thus be selected against).